2024-05-08T22:20:08.713 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T22:20:08.713 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117 --owner scheduled_pdonnell@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/orig.config.yaml 2024-05-08T22:20:08.766 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:20:08.876 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7698117' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T22:20:08.876 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T22:20:08.877 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T22:20:08.878 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T22:20:08.880 INFO:teuthology.task.internal:Checking packages... 2024-05-08T22:20:08.912 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T22:20:08.913 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T22:20:08.913 INFO:teuthology.packaging:ref: None 2024-05-08T22:20:08.913 INFO:teuthology.packaging:tag: None 2024-05-08T22:20:08.913 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:20:08.913 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:20:08.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:20:09.143 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:20:09.145 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T22:20:09.151 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T22:20:09.152 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T22:20:09.175 INFO:teuthology.task.internal:Saving configuration 2024-05-08T22:20:09.190 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T22:20:09.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T22:20:09.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:14:10.778574', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-05-08T22:20:09.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:14:10.780364', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-05-08T22:20:09.248 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T22:20:09.258 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T22:20:09.259 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T22:20:09.259 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T22:20:09.340 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faa9e7b84c0>, signals=[15]) 2024-05-08T22:20:09.341 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T22:20:09.351 INFO:teuthology.task.internal:Opening connections... 2024-05-08T22:20:09.352 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:20:09.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:20:09.456 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:20:09.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:20:09.557 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T22:20:09.564 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-05-08T22:20:09.571 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-05-08T22:20:09.572 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-05-08T22:20:09.621 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Ubuntu" 2024-05-08T22:20:09.621 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-05-08T22:20:09.621 INFO:teuthology.orchestra.run.smithi179.stdout:ID=ubuntu 2024-05-08T22:20:09.621 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE=debian 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="20.04" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_CODENAME=focal 2024-05-08T22:20:09.622 INFO:teuthology.orchestra.run.smithi179.stdout:UBUNTU_CODENAME=focal 2024-05-08T22:20:09.623 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-05-08T22:20:09.642 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-05-08T22:20:09.648 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-05-08T22:20:09.649 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-05-08T22:20:09.697 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="20.04" 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=focal 2024-05-08T22:20:09.698 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=focal 2024-05-08T22:20:09.698 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-05-08T22:20:09.716 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T22:20:09.730 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T22:20:09.743 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T22:20:09.743 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:20:09.746 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:20:09.752 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T22:20:09.759 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T22:20:09.759 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:20:09.762 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:20:09.776 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T22:20:09.834 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T22:20:09.850 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T22:20:09.850 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T22:20:09.851 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:20:09.851 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:20:09.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:20:09.857 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-08T22:20:09.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:20:09.858 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-08T22:20:09.864 INFO:teuthology.orchestra.run.smithi179.stdout:5.4.0-124-generic 2024-05-08T22:20:09.864 INFO:teuthology.task.kernel:Running kernel on smithi179: 5.4.0-124-generic 2024-05-08T22:20:09.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get clean 2024-05-08T22:20:09.865 INFO:teuthology.orchestra.run.smithi195.stdout:5.4.0-124-generic 2024-05-08T22:20:09.865 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.4.0-124-generic 2024-05-08T22:20:09.865 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-05-08T22:20:09.968 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-05-08T22:20:10.155 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get update 2024-05-08T22:20:10.160 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:20:10.166 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-05-08T22:20:10.240 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-05-08T22:20:10.255 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-05-08T22:20:10.353 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:20:10.354 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-05-08T22:20:10.354 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-05-08T22:20:10.416 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-05-08T22:20:10.543 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,296 kB] 2024-05-08T22:20:10.694 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [972 kB] 2024-05-08T22:20:10.709 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [520 kB] 2024-05-08T22:20:10.715 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-05-08T22:20:10.715 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.7 kB] 2024-05-08T22:20:10.715 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,916 kB] 2024-05-08T22:20:10.738 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [407 kB] 2024-05-08T22:20:10.741 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T22:20:10.741 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [783 kB] 2024-05-08T22:20:10.747 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,185 kB] 2024-05-08T22:20:10.755 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [748 kB] 2024-05-08T22:20:10.755 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [284 kB] 2024-05-08T22:20:10.758 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-05-08T22:20:10.759 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.2 kB] 2024-05-08T22:20:10.759 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-05-08T22:20:10.759 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,880 B] 2024-05-08T22:20:10.779 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-05-08T22:20:10.779 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-05-08T22:20:10.780 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-05-08T22:20:10.780 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-05-08T22:20:10.781 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-05-08T22:20:10.800 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-05-08T22:20:10.800 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-05-08T22:20:10.856 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,920 kB] 2024-05-08T22:20:10.869 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,920 kB] 2024-05-08T22:20:10.890 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [972 kB] 2024-05-08T22:20:10.910 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [438 kB] 2024-05-08T22:20:10.915 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-05-08T22:20:10.915 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,799 kB] 2024-05-08T22:20:10.945 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.4 kB] 2024-05-08T22:20:10.946 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [391 kB] 2024-05-08T22:20:10.950 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T22:20:10.951 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [959 kB] 2024-05-08T22:20:10.962 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [657 kB] 2024-05-08T22:20:10.970 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [202 kB] 2024-05-08T22:20:10.972 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-05-08T22:20:10.972 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.0 kB] 2024-05-08T22:20:10.972 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-05-08T22:20:10.987 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,904 B] 2024-05-08T22:20:10.987 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-05-08T22:20:10.990 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,296 kB] 2024-05-08T22:20:11.042 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [520 kB] 2024-05-08T22:20:11.046 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-05-08T22:20:11.046 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,916 kB] 2024-05-08T22:20:11.072 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.7 kB] 2024-05-08T22:20:11.072 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [407 kB] 2024-05-08T22:20:11.076 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T22:20:11.076 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,185 kB] 2024-05-08T22:20:11.086 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [783 kB] 2024-05-08T22:20:11.093 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [284 kB] 2024-05-08T22:20:11.096 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-05-08T22:20:11.096 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-05-08T22:20:11.096 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.2 kB] 2024-05-08T22:20:11.114 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,880 B] 2024-05-08T22:20:11.114 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-05-08T22:20:11.115 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-05-08T22:20:11.115 INFO:teuthology.orchestra.run.smithi179.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-05-08T22:20:11.115 INFO:teuthology.orchestra.run.smithi179.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-05-08T22:20:11.116 INFO:teuthology.orchestra.run.smithi179.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-05-08T22:20:11.136 INFO:teuthology.orchestra.run.smithi179.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-05-08T22:20:11.137 INFO:teuthology.orchestra.run.smithi179.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-05-08T22:20:11.363 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [748 kB] 2024-05-08T22:20:11.377 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [438 kB] 2024-05-08T22:20:11.386 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-05-08T22:20:11.386 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,799 kB] 2024-05-08T22:20:11.470 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.4 kB] 2024-05-08T22:20:11.471 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [391 kB] 2024-05-08T22:20:11.478 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T22:20:11.478 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [959 kB] 2024-05-08T22:20:11.496 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [657 kB] 2024-05-08T22:20:11.671 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [202 kB] 2024-05-08T22:20:12.098 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-05-08T22:20:12.103 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.0 kB] 2024-05-08T22:20:12.110 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-05-08T22:20:12.112 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,904 B] 2024-05-08T22:20:12.114 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-05-08T22:20:14.949 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 20.2 MB in 3s (6,990 kB/s) 2024-05-08T22:20:15.215 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 20.2 MB in 3s (7,027 kB/s) 2024-05-08T22:20:16.369 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:20:16.395 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T22:20:16.465 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:20:16.653 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-05-08T22:20:16.654 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-05-08T22:20:16.680 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:20:16.705 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T22:20:16.785 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:20:16.839 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-05-08T22:20:16.840 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T22:20:16.840 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.4.0-181-generic 2024-05-08T22:20:16.840 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.4.0-181-generic linux-modules-extra-5.4.0-181-generic 2024-05-08T22:20:16.841 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-05-08T22:20:16.841 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-05-08T22:20:16.841 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-05-08T22:20:16.841 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2024-05-08T22:20:16.899 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-05-08T22:20:16.899 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T22:20:16.899 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-181-generic linux-modules-5.4.0-181-generic 2024-05-08T22:20:16.900 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.4.0-181-generic 2024-05-08T22:20:16.901 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-05-08T22:20:16.902 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-08T22:20:16.996 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-08T22:20:16.997 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-08T22:20:17.085 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T22:20:17.085 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 77.1 MB of archives. 2024-05-08T22:20:17.085 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 380 MB of additional disk space will be used. 2024-05-08T22:20:17.085 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-181-generic amd64 5.4.0-181.201 [15.0 MB] 2024-05-08T22:20:17.191 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2024-05-08T22:20:17.191 INFO:teuthology.orchestra.run.smithi179.stdout: linux-generic linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T22:20:17.191 INFO:teuthology.orchestra.run.smithi179.stdout: linux-headers-generic linux-image-5.4.0-181-generic 2024-05-08T22:20:17.191 INFO:teuthology.orchestra.run.smithi179.stdout: linux-modules-5.4.0-181-generic linux-modules-extra-5.4.0-181-generic 2024-05-08T22:20:17.192 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2024-05-08T22:20:17.193 INFO:teuthology.orchestra.run.smithi179.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-05-08T22:20:17.193 INFO:teuthology.orchestra.run.smithi179.stdout:Recommended packages: 2024-05-08T22:20:17.193 INFO:teuthology.orchestra.run.smithi179.stdout: thermald 2024-05-08T22:20:17.247 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2024-05-08T22:20:17.247 INFO:teuthology.orchestra.run.smithi179.stdout: linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T22:20:17.247 INFO:teuthology.orchestra.run.smithi179.stdout: linux-image-5.4.0-181-generic linux-modules-5.4.0-181-generic 2024-05-08T22:20:17.248 INFO:teuthology.orchestra.run.smithi179.stdout: linux-modules-extra-5.4.0-181-generic 2024-05-08T22:20:17.249 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2024-05-08T22:20:17.249 INFO:teuthology.orchestra.run.smithi179.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-08T22:20:17.316 INFO:teuthology.orchestra.run.smithi179.stdout:3 upgraded, 5 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T22:20:17.316 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 77.1 MB of archives. 2024-05-08T22:20:17.316 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 380 MB of additional disk space will be used. 2024-05-08T22:20:17.316 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-181-generic amd64 5.4.0-181.201 [15.0 MB] 2024-05-08T22:20:17.914 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-181-generic amd64 5.4.0-181.201 [10.5 MB] 2024-05-08T22:20:18.135 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-181-generic amd64 5.4.0-181.201 [39.2 MB] 2024-05-08T22:20:18.263 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-181-generic amd64 5.4.0-181.201 [10.5 MB] 2024-05-08T22:20:18.535 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-181-generic amd64 5.4.0-181.201 [39.2 MB] 2024-05-08T22:20:18.876 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.181.179 [1,896 B] 2024-05-08T22:20:18.876 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.181.179 [2,596 B] 2024-05-08T22:20:18.876 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181 all 5.4.0-181.201 [11.0 MB] 2024-05-08T22:20:19.115 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181-generic amd64 5.4.0-181.201 [1,383 kB] 2024-05-08T22:20:19.154 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.181.179 [2,464 B] 2024-05-08T22:20:19.785 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.181.179 [1,896 B] 2024-05-08T22:20:19.785 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.181.179 [2,596 B] 2024-05-08T22:20:19.786 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181 all 5.4.0-181.201 [11.0 MB] 2024-05-08T22:20:19.825 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 77.1 MB in 2s (40.6 MB/s) 2024-05-08T22:20:20.022 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-modules-5.4.0-181-generic. 2024-05-08T22:20:20.065 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181-generic amd64 5.4.0-181.201 [1,383 kB] 2024-05-08T22:20:20.100 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.181.179 [2,464 B] 2024-05-08T22:20:20.810 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 77.1 MB in 3s (24.2 MB/s) 2024-05-08T22:20:20.999 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.4.0-181-generic. 2024-05-08T22:20:21.772 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-05-08T22:20:21.777 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../0-linux-modules-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:21.869 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:22.710 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-05-08T22:20:22.715 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:22.812 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:23.794 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-image-5.4.0-181-generic. 2024-05-08T22:20:23.811 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../1-linux-image-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:23.903 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:24.755 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.4.0-181-generic. 2024-05-08T22:20:24.768 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:24.874 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:24.997 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-181-generic. 2024-05-08T22:20:25.009 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:25.047 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:25.991 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-181-generic. 2024-05-08T22:20:26.007 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:26.049 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:29.745 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../3-linux-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:29.877 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:30.203 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:30.331 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-image-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:30.676 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-headers-5.4.0-181. 2024-05-08T22:20:30.689 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../5-linux-headers-5.4.0-181_5.4.0-181.201_all.deb ... 2024-05-08T22:20:30.733 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T22:20:30.886 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:31.013 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:31.382 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:31.508 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:31.844 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.4.0-181. 2024-05-08T22:20:31.858 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.4.0-181_5.4.0-181.201_all.deb ... 2024-05-08T22:20:31.903 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T22:20:36.090 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-headers-5.4.0-181-generic. 2024-05-08T22:20:36.111 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../6-linux-headers-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:36.148 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:37.290 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.4.0-181-generic. 2024-05-08T22:20:37.313 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T22:20:37.356 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:38.872 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:38.999 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-headers-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:39.326 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:39.985 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T22:20:40.137 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T22:20:40.466 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:40.565 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:41.685 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:41.701 INFO:teuthology.orchestra.run.smithi179.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:20:41.701 INFO:teuthology.orchestra.run.smithi179.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-181-generic 2024-05-08T22:20:41.842 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T22:20:41.969 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:42.095 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:42.825 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:20:42.825 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-181-generic 2024-05-08T22:20:42.953 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T22:20:43.054 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:43.172 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:43.814 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-headers-generic (5.4.0.181.179) ... 2024-05-08T22:20:43.931 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-image-generic (5.4.0.181.179) ... 2024-05-08T22:20:44.058 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-generic (5.4.0.181.179) ... 2024-05-08T22:20:44.184 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:44.256 INFO:teuthology.orchestra.run.smithi179.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-08T22:20:44.256 INFO:teuthology.orchestra.run.smithi179.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:20:44.357 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.4.0.181.179) ... 2024-05-08T22:20:44.483 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.4.0.181.179) ... 2024-05-08T22:20:44.609 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.4.0.181.179) ... 2024-05-08T22:20:44.757 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T22:20:44.824 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-08T22:20:44.825 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:21:06.122 INFO:teuthology.orchestra.run.smithi179.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-08T22:21:06.122 INFO:teuthology.orchestra.run.smithi179.stdout:Sourcing file `/etc/default/grub' 2024-05-08T22:21:06.146 INFO:teuthology.orchestra.run.smithi179.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T22:21:06.176 INFO:teuthology.orchestra.run.smithi179.stdout:Generating grub configuration file ... 2024-05-08T22:21:06.465 INFO:teuthology.orchestra.run.smithi179.stdout:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:21:06.490 INFO:teuthology.orchestra.run.smithi179.stdout:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:21:06.746 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-08T22:21:06.746 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2024-05-08T22:21:06.751 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T22:21:06.770 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2024-05-08T22:21:06.885 INFO:teuthology.orchestra.run.smithi179.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T22:21:06.896 INFO:teuthology.orchestra.run.smithi179.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T22:21:07.082 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:21:07.107 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:21:07.519 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T22:21:07.531 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T22:21:08.952 INFO:teuthology.orchestra.run.smithi179.stdout:done 2024-05-08T22:21:09.368 DEBUG:teuthology.orchestra.run.smithi179:> dpkg -s linux-image-generic 2024-05-08T22:21:09.423 INFO:teuthology.orchestra.run.smithi179.stdout:Package: linux-image-generic 2024-05-08T22:21:09.423 INFO:teuthology.orchestra.run.smithi179.stdout:Status: install ok installed 2024-05-08T22:21:09.423 INFO:teuthology.orchestra.run.smithi179.stdout:Priority: optional 2024-05-08T22:21:09.423 INFO:teuthology.orchestra.run.smithi179.stdout:Section: kernel 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Installed-Size: 20 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: amd64 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Source: linux-meta 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Version: 5.4.0.181.179 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Recommends: thermald 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout:Description: Generic Linux kernel image 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout: This package will always depend on the latest generic kernel image 2024-05-08T22:21:09.424 INFO:teuthology.orchestra.run.smithi179.stdout: available. 2024-05-08T22:21:09.425 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-181-generic 2024-05-08T22:21:09.425 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T22:21:09.425 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T22:21:09.425 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T22:21:09.425 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T22:21:09.425 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get clean 2024-05-08T22:21:09.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get update 2024-05-08T22:21:09.544 INFO:teuthology.orchestra.run.smithi195.stdout:done 2024-05-08T22:21:09.634 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:21:09.645 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T22:21:09.666 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T22:21:09.744 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T22:21:09.952 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 20 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.4.0.181.179 2024-05-08T22:21:10.015 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T22:21:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T22:21:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-05-08T22:21:10.016 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-05-08T22:21:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-05-08T22:21:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-05-08T22:21:10.016 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-181-generic 2024-05-08T22:21:10.016 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T22:21:10.016 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T22:21:10.016 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T22:21:10.016 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T22:21:10.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-05-08T22:21:10.110 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-05-08T22:21:10.247 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:21:10.258 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T22:21:10.278 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T22:21:10.359 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T22:21:11.442 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:21:11.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T22:21:11.535 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:21:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-08T22:21:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-08T22:21:11.974 INFO:teuthology.orchestra.run.smithi179.stdout:linux-image-generic is already the newest version (5.4.0.181.179). 2024-05-08T22:21:11.974 INFO:teuthology.orchestra.run.smithi179.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T22:21:11.976 DEBUG:teuthology.orchestra.run.smithi179:> dpkg -s linux-image-generic 2024-05-08T22:21:11.997 INFO:teuthology.orchestra.run.smithi179.stdout:Package: linux-image-generic 2024-05-08T22:21:11.997 INFO:teuthology.orchestra.run.smithi179.stdout:Status: install ok installed 2024-05-08T22:21:11.997 INFO:teuthology.orchestra.run.smithi179.stdout:Priority: optional 2024-05-08T22:21:11.997 INFO:teuthology.orchestra.run.smithi179.stdout:Section: kernel 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Installed-Size: 20 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: amd64 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Source: linux-meta 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Version: 5.4.0.181.179 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Recommends: thermald 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout:Description: Generic Linux kernel image 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout: This package will always depend on the latest generic kernel image 2024-05-08T22:21:11.998 INFO:teuthology.orchestra.run.smithi179.stdout: available. 2024-05-08T22:21:11.999 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-05-08T22:21:12.005 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.nQVU2I9fCH 2024-05-08T22:21:12.005 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nQVU2I9fCH 2024-05-08T22:21:12.018 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.nQVU2I9fCH 2024-05-08T22:21:12.052 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:21:12.057 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.nQVU2I9fCH is 9KB 2024-05-08T22:21:12.071 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.nQVU2I9fCH 2024-05-08T22:21:12.075 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T22:21:12.077 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.135 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:21:12.135 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.135 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.141 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:21:12.182 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-181-generic 2024-05-08T22:21:12.182 DEBUG:teuthology.orchestra.run.smithi179:> sudo update-grub 2024-05-08T22:21:12.329 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-05-08T22:21:12.330 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-05-08T22:21:12.551 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.4.0.181.179). 2024-05-08T22:21:12.552 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T22:21:12.553 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 20 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.4.0.181.179 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T22:21:12.576 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T22:21:12.577 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-05-08T22:21:12.577 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-05-08T22:21:12.577 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-05-08T22:21:12.577 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-05-08T22:21:12.577 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-05-08T22:21:12.582 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.eGtFV46iOO 2024-05-08T22:21:12.582 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.eGtFV46iOO 2024-05-08T22:21:12.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.eGtFV46iOO 2024-05-08T22:21:12.682 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.eGtFV46iOO is 9KB 2024-05-08T22:21:12.696 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.eGtFV46iOO 2024-05-08T22:21:12.702 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.759 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:21:12.759 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.759 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-08T22:21:12.801 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-181-generic 2024-05-08T22:21:12.801 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2024-05-08T22:21:13.490 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub' 2024-05-08T22:21:13.495 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T22:21:13.508 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-05-08T22:21:13.752 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:21:13.764 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:21:14.106 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T22:21:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T22:21:14.132 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2024-05-08T22:21:14.143 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T22:21:14.157 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-05-08T22:21:14.420 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T22:21:14.431 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T22:21:14.819 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T22:21:14.830 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T22:21:15.112 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-05-08T22:21:15.114 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-05-08T22:21:15.836 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-05-08T22:21:15.837 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-05-08T22:21:45.142 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T22:21:45.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-08T22:21:45.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:21:45.841 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T22:21:45.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:21:45.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:22:45.207 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T22:22:45.845 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T22:22:54.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-08T22:22:54.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:22:54.473 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-08T22:22:54.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:22:54.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:22:55.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-08T22:22:55.689 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-181-generic"... 2024-05-08T22:22:55.690 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-08T22:22:55.744 INFO:teuthology.orchestra.run.smithi179.stdout:5.4.0-181-generic 2024-05-08T22:22:55.744 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-181-generic vs 5.4.0-181-generic 2024-05-08T22:22:55.744 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T22:22:55.744 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T22:22:56.745 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T22:22:56.745 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T22:22:56.835 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-05-08T22:22:56.859 DEBUG:teuthology.parallel:result is None 2024-05-08T22:22:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-05-08T22:23:09.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:23:09.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:23:10.161 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-08T22:23:10.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:23:10.720 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-181-generic"... 2024-05-08T22:23:10.720 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-08T22:23:10.726 INFO:teuthology.orchestra.run.smithi195.stdout:5.4.0-181-generic 2024-05-08T22:23:10.727 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-181-generic vs 5.4.0-181-generic 2024-05-08T22:23:10.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T22:23:10.727 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T22:23:11.728 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T22:23:11.728 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T22:23:11.824 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-05-08T22:23:11.849 DEBUG:teuthology.parallel:result is None 2024-05-08T22:23:11.850 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T22:23:11.857 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T22:23:11.857 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:23:11.860 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:23:11.867 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T22:23:11.876 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T22:23:11.883 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T22:23:11.884 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:23:11.910 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:23:11.936 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T22:23:11.945 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T22:23:11.945 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:23:11.948 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:23:11.966 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:23:11.977 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:23:11.991 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:23:12.002 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:23:12.004 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T22:23:12.011 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T22:23:12.011 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:23:12.022 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:23:12.038 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T22:23:12.046 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T22:23:12.047 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:23:12.049 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:23:12.055 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:23:12.062 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:23:12.069 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:23:12.069 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:23:12.139 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:23:12.146 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:23:12.153 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:23:12.153 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:23:12.173 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-05-08T22:23:12.175 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-05-08T22:23:12.266 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T22:23:12.276 INFO:teuthology.task.internal:Starting timer... 2024-05-08T22:23:12.276 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T22:23:12.399 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T22:23:12.424 DEBUG:teuthology.task.selinux:Excluding smithi179: OS 'ubuntu' does not support SELinux 2024-05-08T22:23:12.424 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2024-05-08T22:23:12.424 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T22:23:12.424 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-08T22:23:12.424 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T22:23:12.424 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T22:23:12.443 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T22:23:12.460 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T22:23:12.461 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi179.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T22:28:56.291 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi179.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-05-08T22:28:56.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-08T22:28:56.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:28:56.388 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-08T22:28:56.516 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-08T22:28:56.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:28:56.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:28:56.611 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-08T22:28:56.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-08T22:28:56.737 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T22:28:56.749 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T22:28:56.749 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:28:56.749 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:28:56.752 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:28:56.752 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:28:56.786 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-05-08T22:28:56.786 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Command line: ntpd -gq 2024-05-08T22:28:56.787 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: proto: precision = 0.088 usec (-23) 2024-05-08T22:28:56.787 INFO:teuthology.orchestra.run.smithi179.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T22:28:56.788 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T22:28:56.788 INFO:teuthology.orchestra.run.smithi179.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen normally on 3 enp3s0f1 172.21.15.179:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen normally on 4 lo [::1]:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:725b%5]:123 2024-05-08T22:28:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:28:56 ntpd[13053]: Listening on routing socket on fd #22 for interface updates 2024-05-08T22:28:56.790 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-05-08T22:28:56.790 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Command line: ntpd -gq 2024-05-08T22:28:56.790 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: proto: precision = 0.078 usec (-24) 2024-05-08T22:28:56.790 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T22:28:56.791 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T22:28:56.791 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen normally on 4 lo [::1]:123 2024-05-08T22:28:56.792 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2024-05-08T22:28:56.793 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:28:56 ntpd[13042]: Listening on routing socket on fd #22 for interface updates 2024-05-08T22:28:57.788 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:57 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:28:57.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:57 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:28:58.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:28:58 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:28:59.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:28:59.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:28:59.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:28:59.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:28:59.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:28:59.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:28:59 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:00.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:00.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:00.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:00 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:00.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:00 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:01.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:01.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:01 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:01.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:01.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:01 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:02.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:02 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:02.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:02 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:03.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:03.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:03 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:03.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:03 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:04.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:04 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stdout: 8 May 22:29:04 ntpd[13042]: ntpd: time slew +0.000808 s 2024-05-08T22:29:04.789 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew +0.000808s 2024-05-08T22:29:04.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T22:29:04.790 INFO:teuthology.orchestra.run.smithi195.stderr: 8 May 22:29:04 ntpd[13042]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:04.874 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:05.787 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T22:29:05.788 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T22:29:05.789 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T22:29:05.789 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T22:29:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: 8 May 22:29:05 ntpd[13053]: ntpd: time slew +0.000045 s 2024-05-08T22:29:05.790 INFO:teuthology.orchestra.run.smithi179.stdout:ntpd: time slew +0.000045s 2024-05-08T22:29:05.790 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T22:29:05.790 INFO:teuthology.orchestra.run.smithi179.stderr: 8 May 22:29:05 ntpd[13053]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T22:29:05.859 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T22:29:05.859 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2024-05-08T22:29:05.859 INFO:teuthology.orchestra.run.smithi179.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:05.859 INFO:teuthology.orchestra.run.smithi179.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:05.859 INFO:teuthology.orchestra.run.smithi179.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:05.860 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T22:29:05.860 INFO:teuthology.run_tasks:Running task install... 2024-05-08T22:29:05.872 DEBUG:teuthology.task.install:project ceph 2024-05-08T22:29:05.872 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T22:29:05.872 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T22:29:05.872 INFO:teuthology.task.install:Using flavor: default 2024-05-08T22:29:05.878 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T22:29:05.879 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T22:29:05.879 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-key list | grep Ceph 2024-05-08T22:29:05.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2024-05-08T22:29:05.982 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-05-08T22:29:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-05-08T22:29:06.040 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T22:29:06.040 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T22:29:06.041 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T22:29:06.041 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T22:29:06.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:29:06.054 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T22:29:06.054 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T22:29:06.055 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T22:29:06.055 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T22:29:06.055 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:29:06.196 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default/ 2024-05-08T22:29:06.196 INFO:teuthology.task.install.deb:Package version is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:29:06.266 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default/ 2024-05-08T22:29:06.266 INFO:teuthology.task.install.deb:Package version is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:29:06.605 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:29:06.605 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T22:29:06.613 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:29:06.613 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T22:29:06.622 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-05-08T22:29:06.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get update 2024-05-08T22:29:06.818 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:29:06.818 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T22:29:06.828 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T22:29:06.850 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T22:29:06.926 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T22:29:06.926 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T22:29:06.926 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal InRelease 2024-05-08T22:29:06.978 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-05-08T22:29:06.994 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T22:29:07.033 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release.gpg 2024-05-08T22:29:07.052 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal InRelease 2024-05-08T22:29:07.076 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T22:29:07.086 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-05-08T22:29:07.123 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-05-08T22:29:07.144 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main i386 Packages [3,414 B] 2024-05-08T22:29:07.194 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release.gpg 2024-05-08T22:29:07.263 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-05-08T22:29:07.410 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main i386 Packages [3,414 B] 2024-05-08T22:29:07.506 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 28.7 kB in 1s (42.5 kB/s) 2024-05-08T22:29:07.594 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 28.7 kB in 1s (35.9 kB/s) 2024-05-08T22:29:08.610 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:29:08.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1322-gd21e4f96-1focal cephadm=18.2.2-1322-gd21e4f96-1focal ceph-mds=18.2.2-1322-gd21e4f96-1focal ceph-mgr=18.2.2-1322-gd21e4f96-1focal ceph-common=18.2.2-1322-gd21e4f96-1focal ceph-fuse=18.2.2-1322-gd21e4f96-1focal ceph-test=18.2.2-1322-gd21e4f96-1focal ceph-volume=18.2.2-1322-gd21e4f96-1focal radosgw=18.2.2-1322-gd21e4f96-1focal python3-rados=18.2.2-1322-gd21e4f96-1focal python3-rgw=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal python3-rbd=18.2.2-1322-gd21e4f96-1focal libcephfs2=18.2.2-1322-gd21e4f96-1focal libcephfs-dev=18.2.2-1322-gd21e4f96-1focal librados2=18.2.2-1322-gd21e4f96-1focal librbd1=18.2.2-1322-gd21e4f96-1focal rbd-fuse=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal cephfs-shell=18.2.2-1322-gd21e4f96-1focal cephfs-top=18.2.2-1322-gd21e4f96-1focal cephfs-mirror=18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:29:08.702 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:29:08.704 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:29:08.728 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1322-gd21e4f96-1focal cephadm=18.2.2-1322-gd21e4f96-1focal ceph-mds=18.2.2-1322-gd21e4f96-1focal ceph-mgr=18.2.2-1322-gd21e4f96-1focal ceph-common=18.2.2-1322-gd21e4f96-1focal ceph-fuse=18.2.2-1322-gd21e4f96-1focal ceph-test=18.2.2-1322-gd21e4f96-1focal ceph-volume=18.2.2-1322-gd21e4f96-1focal radosgw=18.2.2-1322-gd21e4f96-1focal python3-rados=18.2.2-1322-gd21e4f96-1focal python3-rgw=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal python3-rbd=18.2.2-1322-gd21e4f96-1focal libcephfs2=18.2.2-1322-gd21e4f96-1focal libcephfs-dev=18.2.2-1322-gd21e4f96-1focal librados2=18.2.2-1322-gd21e4f96-1focal librbd1=18.2.2-1322-gd21e4f96-1focal rbd-fuse=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal cephfs-shell=18.2.2-1322-gd21e4f96-1focal cephfs-top=18.2.2-1322-gd21e4f96-1focal cephfs-mirror=18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:29:08.796 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:29:08.913 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-05-08T22:29:08.914 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-05-08T22:29:08.983 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-08T22:29:08.983 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-08T22:29:09.102 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-05-08T22:29:09.104 INFO:teuthology.orchestra.run.smithi195.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-05-08T22:29:09.105 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-08T22:29:09.106 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-05-08T22:29:09.107 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2024-05-08T22:29:09.166 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-05-08T22:29:09.168 INFO:teuthology.orchestra.run.smithi179.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-05-08T22:29:09.169 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-05-08T22:29:09.169 INFO:teuthology.orchestra.run.smithi179.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T22:29:09.169 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T22:29:09.169 INFO:teuthology.orchestra.run.smithi179.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T22:29:09.169 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T22:29:09.170 INFO:teuthology.orchestra.run.smithi179.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-05-08T22:29:09.171 INFO:teuthology.orchestra.run.smithi179.stdout:Recommended packages: 2024-05-08T22:29:09.172 INFO:teuthology.orchestra.run.smithi179.stdout: btrfs-tools 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-05-08T22:29:09.232 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-05-08T22:29:09.234 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-05-08T22:29:09.235 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 librbd1 2024-05-08T22:29:09.291 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2024-05-08T22:29:09.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T22:29:09.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T22:29:09.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T22:29:09.292 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-05-08T22:29:09.292 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-05-08T22:29:09.292 INFO:teuthology.orchestra.run.smithi179.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-05-08T22:29:09.292 INFO:teuthology.orchestra.run.smithi179.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-05-08T22:29:09.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-05-08T22:29:09.294 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2024-05-08T22:29:09.295 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 librbd1 2024-05-08T22:29:09.306 INFO:teuthology.orchestra.run.smithi195.stdout:2 upgraded, 116 newly installed, 0 to remove and 278 not upgraded. 2024-05-08T22:29:09.306 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 199 MB of archives. 2024-05-08T22:29:09.306 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 977 MB of additional disk space will be used. 2024-05-08T22:29:09.306 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-05-08T22:29:09.332 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-05-08T22:29:09.374 INFO:teuthology.orchestra.run.smithi179.stdout:2 upgraded, 116 newly installed, 0 to remove and 278 not upgraded. 2024-05-08T22:29:09.374 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 199 MB of archives. 2024-05-08T22:29:09.374 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 977 MB of additional disk space will be used. 2024-05-08T22:29:09.374 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-05-08T22:29:09.374 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-05-08T22:29:09.396 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-05-08T22:29:09.406 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-05-08T22:29:09.415 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-05-08T22:29:09.422 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-05-08T22:29:09.437 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-05-08T22:29:09.439 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-05-08T22:29:09.444 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.2-1322-gd21e4f96-1focal [3,446 kB] 2024-05-08T22:29:09.473 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-05-08T22:29:09.478 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.2-1322-gd21e4f96-1focal [3,446 kB] 2024-05-08T22:29:09.480 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-05-08T22:29:09.521 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-05-08T22:29:09.525 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-05-08T22:29:09.529 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-05-08T22:29:09.537 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-05-08T22:29:09.539 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-05-08T22:29:09.539 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-05-08T22:29:09.540 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-05-08T22:29:09.541 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-05-08T22:29:09.546 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-05-08T22:29:09.546 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-05-08T22:29:09.548 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-05-08T22:29:09.569 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-05-08T22:29:09.569 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-05-08T22:29:09.569 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-05-08T22:29:09.571 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T22:29:09.571 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-05-08T22:29:09.572 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-05-08T22:29:09.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-05-08T22:29:09.574 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-05-08T22:29:09.588 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-05-08T22:29:09.590 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-05-08T22:29:09.592 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-05-08T22:29:09.592 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-05-08T22:29:09.599 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-05-08T22:29:09.602 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-05-08T22:29:09.604 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-05-08T22:29:09.605 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-05-08T22:29:09.606 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-05-08T22:29:09.609 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-05-08T22:29:09.610 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-05-08T22:29:09.611 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-05-08T22:29:09.612 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-05-08T22:29:09.623 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-05-08T22:29:09.624 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-05-08T22:29:09.625 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-05-08T22:29:09.626 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-05-08T22:29:09.628 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-05-08T22:29:09.630 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-05-08T22:29:09.630 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-05-08T22:29:09.639 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-05-08T22:29:09.640 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-05-08T22:29:09.640 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-05-08T22:29:09.641 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-05-08T22:29:09.642 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-05-08T22:29:09.643 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T22:29:09.644 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-05-08T22:29:09.645 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-05-08T22:29:09.647 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-05-08T22:29:09.648 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-05-08T22:29:09.662 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-05-08T22:29:09.663 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-05-08T22:29:09.664 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-05-08T22:29:09.665 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-05-08T22:29:09.665 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-05-08T22:29:09.666 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-05-08T22:29:09.667 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-05-08T22:29:09.669 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-05-08T22:29:09.669 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-05-08T22:29:09.670 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-05-08T22:29:09.672 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.2-1322-gd21e4f96-1focal [4,035 kB] 2024-05-08T22:29:09.683 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-05-08T22:29:09.684 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T22:29:09.685 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-05-08T22:29:09.692 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-05-08T22:29:09.692 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-05-08T22:29:09.693 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-05-08T22:29:09.694 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-05-08T22:29:09.696 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-05-08T22:29:09.699 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-05-08T22:29:09.699 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-05-08T22:29:09.704 INFO:teuthology.orchestra.run.smithi179.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-05-08T22:29:09.704 INFO:teuthology.orchestra.run.smithi179.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-05-08T22:29:09.706 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-05-08T22:29:09.708 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-05-08T22:29:09.709 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-05-08T22:29:09.710 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-05-08T22:29:09.711 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-05-08T22:29:09.750 INFO:teuthology.orchestra.run.smithi179.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-05-08T22:29:09.753 INFO:teuthology.orchestra.run.smithi179.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-05-08T22:29:09.754 INFO:teuthology.orchestra.run.smithi179.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-05-08T22:29:09.755 INFO:teuthology.orchestra.run.smithi179.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-05-08T22:29:09.756 INFO:teuthology.orchestra.run.smithi179.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-05-08T22:29:09.758 INFO:teuthology.orchestra.run.smithi179.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-05-08T22:29:09.759 INFO:teuthology.orchestra.run.smithi179.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-05-08T22:29:09.760 INFO:teuthology.orchestra.run.smithi179.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-05-08T22:29:09.793 INFO:teuthology.orchestra.run.smithi179.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T22:29:09.794 INFO:teuthology.orchestra.run.smithi179.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-05-08T22:29:09.824 INFO:teuthology.orchestra.run.smithi179.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.2-1322-gd21e4f96-1focal [744 kB] 2024-05-08T22:29:09.835 INFO:teuthology.orchestra.run.smithi179.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-05-08T22:29:09.837 INFO:teuthology.orchestra.run.smithi179.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-05-08T22:29:09.839 INFO:teuthology.orchestra.run.smithi179.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-05-08T22:29:09.840 INFO:teuthology.orchestra.run.smithi179.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-05-08T22:29:09.841 INFO:teuthology.orchestra.run.smithi179.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-05-08T22:29:09.850 INFO:teuthology.orchestra.run.smithi179.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.2-1322-gd21e4f96-1focal [387 kB] 2024-05-08T22:29:09.869 INFO:teuthology.orchestra.run.smithi179.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.2-1322-gd21e4f96-1focal [31.9 kB] 2024-05-08T22:29:09.870 INFO:teuthology.orchestra.run.smithi179.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.2-1322-gd21e4f96-1focal [201 kB] 2024-05-08T22:29:09.878 INFO:teuthology.orchestra.run.smithi179.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.2-1322-gd21e4f96-1focal [62.5 kB] 2024-05-08T22:29:09.881 INFO:teuthology.orchestra.run.smithi179.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.2-1322-gd21e4f96-1focal [365 kB] 2024-05-08T22:29:09.893 INFO:teuthology.orchestra.run.smithi179.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.2-1322-gd21e4f96-1focal [6,591 kB] 2024-05-08T22:29:09.964 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-05-08T22:29:09.997 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-05-08T22:29:09.998 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-05-08T22:29:10.033 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-05-08T22:29:10.033 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-05-08T22:29:10.034 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-05-08T22:29:10.035 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-05-08T22:29:10.035 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-05-08T22:29:10.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-05-08T22:29:10.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-05-08T22:29:10.069 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-05-08T22:29:10.069 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-05-08T22:29:10.102 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-05-08T22:29:10.108 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-05-08T22:29:10.111 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-05-08T22:29:10.111 INFO:teuthology.orchestra.run.smithi179.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-05-08T22:29:10.114 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-05-08T22:29:10.115 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-05-08T22:29:10.118 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-05-08T22:29:10.122 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-05-08T22:29:10.122 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-05-08T22:29:10.164 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-05-08T22:29:10.168 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-05-08T22:29:10.199 INFO:teuthology.orchestra.run.smithi179.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.2-1322-gd21e4f96-1focal [113 kB] 2024-05-08T22:29:10.199 INFO:teuthology.orchestra.run.smithi179.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-05-08T22:29:10.200 INFO:teuthology.orchestra.run.smithi179.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.2-1322-gd21e4f96-1focal [476 kB] 2024-05-08T22:29:10.202 INFO:teuthology.orchestra.run.smithi179.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-05-08T22:29:10.203 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-05-08T22:29:10.208 INFO:teuthology.orchestra.run.smithi179.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.2-1322-gd21e4f96-1focal [25.1 MB] 2024-05-08T22:29:10.255 INFO:teuthology.orchestra.run.smithi179.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-05-08T22:29:10.255 INFO:teuthology.orchestra.run.smithi179.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-05-08T22:29:10.256 INFO:teuthology.orchestra.run.smithi179.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-05-08T22:29:10.256 INFO:teuthology.orchestra.run.smithi179.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-05-08T22:29:10.256 INFO:teuthology.orchestra.run.smithi179.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-05-08T22:29:10.256 INFO:teuthology.orchestra.run.smithi179.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-05-08T22:29:10.257 INFO:teuthology.orchestra.run.smithi179.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-05-08T22:29:10.258 INFO:teuthology.orchestra.run.smithi179.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-05-08T22:29:10.280 INFO:teuthology.orchestra.run.smithi179.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-05-08T22:29:10.287 INFO:teuthology.orchestra.run.smithi179.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-05-08T22:29:10.293 INFO:teuthology.orchestra.run.smithi179.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-05-08T22:29:10.296 INFO:teuthology.orchestra.run.smithi179.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-05-08T22:29:10.298 INFO:teuthology.orchestra.run.smithi179.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-05-08T22:29:10.299 INFO:teuthology.orchestra.run.smithi179.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-05-08T22:29:10.302 INFO:teuthology.orchestra.run.smithi179.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-05-08T22:29:10.302 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-05-08T22:29:10.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-05-08T22:29:10.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-05-08T22:29:10.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-05-08T22:29:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-05-08T22:29:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-05-08T22:29:10.330 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T22:29:10.330 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-05-08T22:29:10.331 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-05-08T22:29:10.332 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-05-08T22:29:10.332 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-05-08T22:29:10.351 INFO:teuthology.orchestra.run.smithi179.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-05-08T22:29:10.367 INFO:teuthology.orchestra.run.smithi179.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-05-08T22:29:10.371 INFO:teuthology.orchestra.run.smithi179.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-05-08T22:29:10.448 INFO:teuthology.orchestra.run.smithi179.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-05-08T22:29:10.449 INFO:teuthology.orchestra.run.smithi179.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-05-08T22:29:10.449 INFO:teuthology.orchestra.run.smithi179.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-05-08T22:29:10.452 INFO:teuthology.orchestra.run.smithi179.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-05-08T22:29:10.472 INFO:teuthology.orchestra.run.smithi179.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T22:29:10.473 INFO:teuthology.orchestra.run.smithi179.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-05-08T22:29:10.476 INFO:teuthology.orchestra.run.smithi179.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-05-08T22:29:10.476 INFO:teuthology.orchestra.run.smithi179.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-05-08T22:29:10.477 INFO:teuthology.orchestra.run.smithi179.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-05-08T22:29:10.482 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.2-1322-gd21e4f96-1focal [4,035 kB] 2024-05-08T22:29:11.182 INFO:teuthology.orchestra.run.smithi179.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.2-1322-gd21e4f96-1focal [9,604 kB] 2024-05-08T22:29:11.449 INFO:teuthology.orchestra.run.smithi179.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.2-1322-gd21e4f96-1focal [238 kB] 2024-05-08T22:29:11.458 INFO:teuthology.orchestra.run.smithi179.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1322-gd21e4f96-1focal [148 kB] 2024-05-08T22:29:11.462 INFO:teuthology.orchestra.run.smithi179.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.2-1322-gd21e4f96-1focal [1,573 kB] 2024-05-08T22:29:11.502 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.2-1322-gd21e4f96-1focal [744 kB] 2024-05-08T22:29:11.521 INFO:teuthology.orchestra.run.smithi179.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.2-1322-gd21e4f96-1focal [6,637 kB] 2024-05-08T22:29:11.678 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.2-1322-gd21e4f96-1focal [387 kB] 2024-05-08T22:29:11.772 INFO:teuthology.orchestra.run.smithi179.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.2-1322-gd21e4f96-1focal [24.5 MB] 2024-05-08T22:29:11.789 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.2-1322-gd21e4f96-1focal [31.9 kB] 2024-05-08T22:29:11.793 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.2-1322-gd21e4f96-1focal [201 kB] 2024-05-08T22:29:11.848 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.2-1322-gd21e4f96-1focal [62.5 kB] 2024-05-08T22:29:11.869 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.2-1322-gd21e4f96-1focal [365 kB] 2024-05-08T22:29:11.955 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.2-1322-gd21e4f96-1focal [6,591 kB] 2024-05-08T22:29:12.763 INFO:teuthology.orchestra.run.smithi179.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.2-1322-gd21e4f96-1focal [13.7 kB] 2024-05-08T22:29:12.764 INFO:teuthology.orchestra.run.smithi179.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.2-1322-gd21e4f96-1focal [854 kB] 2024-05-08T22:29:12.778 INFO:teuthology.orchestra.run.smithi179.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.2-1322-gd21e4f96-1focal [2,291 kB] 2024-05-08T22:29:12.808 INFO:teuthology.orchestra.run.smithi179.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.2-1322-gd21e4f96-1focal [237 kB] 2024-05-08T22:29:12.819 INFO:teuthology.orchestra.run.smithi179.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.2-1322-gd21e4f96-1focal [143 kB] 2024-05-08T22:29:12.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.2-1322-gd21e4f96-1focal [7,012 kB] 2024-05-08T22:29:13.086 INFO:teuthology.orchestra.run.smithi179.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1322-gd21e4f96-1focal [7,876 kB] 2024-05-08T22:29:13.381 INFO:teuthology.orchestra.run.smithi179.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.2-1322-gd21e4f96-1focal [13.9 kB] 2024-05-08T22:29:13.381 INFO:teuthology.orchestra.run.smithi179.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.2-1322-gd21e4f96-1focal [46.7 MB] 2024-05-08T22:29:13.701 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.2-1322-gd21e4f96-1focal [113 kB] 2024-05-08T22:29:13.728 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.2-1322-gd21e4f96-1focal [476 kB] 2024-05-08T22:29:13.861 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.2-1322-gd21e4f96-1focal [25.1 MB] 2024-05-08T22:29:15.590 INFO:teuthology.orchestra.run.smithi179.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.2-1322-gd21e4f96-1focal [119 kB] 2024-05-08T22:29:15.592 INFO:teuthology.orchestra.run.smithi179.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.2-1322-gd21e4f96-1focal [268 kB] 2024-05-08T22:29:15.597 INFO:teuthology.orchestra.run.smithi179.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.2-1322-gd21e4f96-1focal [33.0 kB] 2024-05-08T22:29:15.597 INFO:teuthology.orchestra.run.smithi179.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.2-1322-gd21e4f96-1focal [22.7 kB] 2024-05-08T22:29:15.599 INFO:teuthology.orchestra.run.smithi179.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.2-1322-gd21e4f96-1focal [37.4 kB] 2024-05-08T22:29:15.599 INFO:teuthology.orchestra.run.smithi179.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.2-1322-gd21e4f96-1focal [13.4 MB] 2024-05-08T22:29:15.974 INFO:teuthology.orchestra.run.smithi179.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.2-1322-gd21e4f96-1focal [107 kB] 2024-05-08T22:29:16.610 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 199 MB in 7s (29.8 MB/s) 2024-05-08T22:29:16.760 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T22:29:16.819 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-05-08T22:29:16.823 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-05-08T22:29:16.875 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-05-08T22:29:17.195 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T22:29:17.212 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-05-08T22:29:17.278 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T22:29:17.589 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T22:29:17.605 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-05-08T22:29:17.639 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-05-08T22:29:17.992 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-05-08T22:29:18.011 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-05-08T22:29:18.092 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T22:29:18.488 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T22:29:18.505 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-05-08T22:29:18.546 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T22:29:18.907 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T22:29:18.924 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-05-08T22:29:19.007 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T22:29:19.037 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.2-1322-gd21e4f96-1focal [9,604 kB] 2024-05-08T22:29:19.494 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T22:29:19.512 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:19.564 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:20.214 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T22:29:20.230 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:20.272 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:20.668 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T22:29:20.685 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:20.751 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:21.116 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.2-1322-gd21e4f96-1focal [238 kB] 2024-05-08T22:29:21.145 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-05-08T22:29:21.161 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-05-08T22:29:21.195 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1322-gd21e4f96-1focal [148 kB] 2024-05-08T22:29:21.202 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.2-1322-gd21e4f96-1focal [1,573 kB] 2024-05-08T22:29:21.212 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T22:29:21.572 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.2-1322-gd21e4f96-1focal [6,637 kB] 2024-05-08T22:29:21.631 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../010-librbd1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:21.765 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librbd1 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T22:29:22.536 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../011-librados2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:22.652 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librados2 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T22:29:22.967 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.2-1322-gd21e4f96-1focal [24.5 MB] 2024-05-08T22:29:23.440 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T22:29:23.456 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../012-libcephfs2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:23.498 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:23.859 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rados. 2024-05-08T22:29:23.875 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../013-python3-rados_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:23.942 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:24.312 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T22:29:24.328 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:24.370 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:24.639 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T22:29:24.658 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../015-python3-cephfs_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:24.697 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:25.342 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T22:29:25.357 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:25.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:25.787 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T22:29:25.806 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-05-08T22:29:25.845 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-05-08T22:29:25.906 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.2-1322-gd21e4f96-1focal [13.7 kB] 2024-05-08T22:29:25.906 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.2-1322-gd21e4f96-1focal [854 kB] 2024-05-08T22:29:25.915 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.2-1322-gd21e4f96-1focal [2,291 kB] 2024-05-08T22:29:26.127 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.2-1322-gd21e4f96-1focal [237 kB] 2024-05-08T22:29:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T22:29:26.131 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.2-1322-gd21e4f96-1focal [143 kB] 2024-05-08T22:29:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../018-python3-rbd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:26.148 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.2-1322-gd21e4f96-1focal [7,012 kB] 2024-05-08T22:29:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:26.592 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-05-08T22:29:26.608 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-05-08T22:29:26.650 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T22:29:26.717 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1322-gd21e4f96-1focal [7,876 kB] 2024-05-08T22:29:26.986 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-05-08T22:29:27.006 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-05-08T22:29:27.036 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T22:29:27.278 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.2-1322-gd21e4f96-1focal [13.9 kB] 2024-05-08T22:29:27.279 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.2-1322-gd21e4f96-1focal [46.7 MB] 2024-05-08T22:29:27.439 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T22:29:27.456 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-05-08T22:29:27.497 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T22:29:27.826 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librgw2. 2024-05-08T22:29:27.841 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../022-librgw2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:27.875 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:28.753 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T22:29:28.769 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../023-python3-rgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:28.812 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:29.197 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liboath0. 2024-05-08T22:29:29.212 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-05-08T22:29:29.255 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-05-08T22:29:29.583 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T22:29:29.602 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../025-libradosstriper1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:29.650 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:30.095 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-common. 2024-05-08T22:29:30.115 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../026-ceph-common_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:30.115 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.2-1322-gd21e4f96-1focal [119 kB] 2024-05-08T22:29:30.117 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.2-1322-gd21e4f96-1focal [268 kB] 2024-05-08T22:29:30.123 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.2-1322-gd21e4f96-1focal [33.0 kB] 2024-05-08T22:29:30.124 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.2-1322-gd21e4f96-1focal [22.7 kB] 2024-05-08T22:29:30.124 INFO:teuthology.orchestra.run.smithi195.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.2-1322-gd21e4f96-1focal [37.4 kB] 2024-05-08T22:29:30.125 INFO:teuthology.orchestra.run.smithi195.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.2-1322-gd21e4f96-1focal [13.4 MB] 2024-05-08T22:29:30.144 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:30.604 INFO:teuthology.orchestra.run.smithi195.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.2-1322-gd21e4f96-1focal [107 kB] 2024-05-08T22:29:31.251 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 199 MB in 21s (9,311 kB/s) 2024-05-08T22:29:31.406 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T22:29:31.463 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-05-08T22:29:31.467 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-05-08T22:29:31.513 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-05-08T22:29:31.874 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T22:29:31.893 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-05-08T22:29:31.932 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T22:29:32.269 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T22:29:32.288 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-05-08T22:29:32.327 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-05-08T22:29:32.647 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-05-08T22:29:32.650 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-base. 2024-05-08T22:29:32.665 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-05-08T22:29:32.670 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../027-ceph-base_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:32.744 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:32.755 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T22:29:33.134 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T22:29:33.150 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-05-08T22:29:33.192 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T22:29:33.604 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T22:29:33.622 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-05-08T22:29:33.662 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T22:29:34.013 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T22:29:34.031 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-05-08T22:29:34.062 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-05-08T22:29:34.164 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T22:29:34.181 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:34.242 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:34.466 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T22:29:34.484 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-05-08T22:29:34.532 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-05-08T22:29:34.777 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T22:29:34.796 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:34.827 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:34.869 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T22:29:34.887 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-05-08T22:29:34.927 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-05-08T22:29:35.221 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T22:29:35.238 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T22:29:35.255 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T22:29:35.272 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-05-08T22:29:35.279 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:29:35.338 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-05-08T22:29:35.649 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-mako. 2024-05-08T22:29:35.665 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-05-08T22:29:35.699 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-05-08T22:29:35.708 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T22:29:35.718 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-05-08T22:29:35.757 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T22:29:36.069 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T22:29:36.086 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T22:29:36.127 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T22:29:36.160 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-librbd1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:36.295 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T22:29:36.489 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T22:29:36.507 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-05-08T22:29:36.547 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T22:29:36.883 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webob. 2024-05-08T22:29:36.900 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2024-05-08T22:29:36.941 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-05-08T22:29:36.991 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-librados2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:37.140 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T22:29:37.336 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-soupsieve. 2024-05-08T22:29:37.355 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-05-08T22:29:37.394 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T22:29:37.730 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-bs4. 2024-05-08T22:29:37.748 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-05-08T22:29:37.788 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-05-08T22:29:37.878 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T22:29:37.897 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-libcephfs2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:37.961 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T22:29:38.166 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-05-08T22:29:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T22:29:38.330 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2024-05-08T22:29:38.349 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-python3-rados_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:38.388 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:38.587 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T22:29:38.604 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-05-08T22:29:38.645 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-05-08T22:29:38.799 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T22:29:38.817 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:38.866 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:38.981 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-paste. 2024-05-08T22:29:38.997 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-05-08T22:29:39.072 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T22:29:39.135 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T22:29:39.154 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-cephfs_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:39.194 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:39.418 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T22:29:39.437 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-05-08T22:29:39.476 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T22:29:39.546 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T22:29:39.562 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:39.604 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:39.795 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T22:29:39.815 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-05-08T22:29:39.853 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-05-08T22:29:39.983 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T22:29:40.001 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-05-08T22:29:40.041 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-05-08T22:29:40.148 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T22:29:40.165 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-05-08T22:29:40.231 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-05-08T22:29:41.113 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T22:29:41.130 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-python3-rbd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:41.171 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:41.343 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T22:29:41.360 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-05-08T22:29:41.401 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T22:29:41.557 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-05-08T22:29:41.574 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-05-08T22:29:41.640 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T22:29:41.789 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T22:29:41.805 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-05-08T22:29:41.865 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T22:29:41.993 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-05-08T22:29:42.010 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-05-08T22:29:42.051 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T22:29:42.201 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T22:29:42.216 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:42.259 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:42.404 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T22:29:42.420 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-05-08T22:29:42.487 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T22:29:42.692 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T22:29:42.711 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:42.755 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:42.857 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2024-05-08T22:29:42.874 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-librgw2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:42.915 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:43.108 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T22:29:43.125 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-05-08T22:29:43.166 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T22:29:43.494 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cryptography. 2024-05-08T22:29:43.513 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-05-08T22:29:43.577 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T22:29:43.818 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T22:29:43.834 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-python3-rgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:43.884 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:43.981 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T22:29:43.997 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-05-08T22:29:44.031 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-05-08T22:29:44.206 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0. 2024-05-08T22:29:44.225 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-05-08T22:29:44.262 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-05-08T22:29:44.359 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T22:29:44.375 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../051-ceph-mgr_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:44.417 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:44.598 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T22:29:44.614 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-libradosstriper1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:44.656 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:44.954 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T22:29:44.974 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../052-ceph-mon_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:45.035 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:45.168 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2024-05-08T22:29:45.184 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-ceph-common_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:45.218 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:46.024 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T22:29:46.040 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../053-ceph-osd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:46.082 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:47.757 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2024-05-08T22:29:47.774 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-ceph-base_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:47.859 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:48.302 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph. 2024-05-08T22:29:48.320 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../054-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:48.351 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:48.696 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T22:29:48.712 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../055-ceph-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:48.776 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:49.136 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T22:29:49.155 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-05-08T22:29:49.190 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T22:29:49.194 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-05-08T22:29:49.208 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../056-ceph-mds_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:49.249 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:49.598 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T22:29:49.617 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-05-08T22:29:49.656 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-05-08T22:29:49.811 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephadm. 2024-05-08T22:29:49.826 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../057-cephadm_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:49.868 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:50.009 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T22:29:50.027 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-05-08T22:29:50.067 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-05-08T22:29:50.279 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T22:29:50.296 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-05-08T22:29:50.337 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-05-08T22:29:50.395 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T22:29:50.411 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-05-08T22:29:50.485 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-05-08T22:29:50.740 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T22:29:50.757 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-05-08T22:29:50.944 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T22:29:50.945 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2024-05-08T22:29:50.945 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-05-08T22:29:50.945 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T22:29:51.152 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T22:29:51.171 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:51.210 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:51.259 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T22:29:51.275 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T22:29:51.338 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T22:29:51.655 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jwt. 2024-05-08T22:29:51.672 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-05-08T22:29:51.678 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T22:29:51.695 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-05-08T22:29:51.730 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T22:29:51.745 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T22:29:52.073 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2024-05-08T22:29:52.080 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-attr. 2024-05-08T22:29:52.090 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2024-05-08T22:29:52.097 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-05-08T22:29:52.123 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-05-08T22:29:52.138 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-05-08T22:29:52.500 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T22:29:52.517 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-05-08T22:29:52.519 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-soupsieve. 2024-05-08T22:29:52.534 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-05-08T22:29:52.576 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T22:29:52.592 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-05-08T22:29:52.903 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-py. 2024-05-08T22:29:52.903 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bs4. 2024-05-08T22:29:52.919 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-05-08T22:29:52.922 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-05-08T22:29:52.953 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T22:29:52.970 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-05-08T22:29:53.283 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-atomicwrites. 2024-05-08T22:29:53.297 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-05-08T22:29:53.339 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T22:29:53.382 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T22:29:53.398 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-05-08T22:29:53.443 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T22:29:53.675 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-packaging. 2024-05-08T22:29:53.692 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-05-08T22:29:53.741 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-packaging (20.3-1) ... 2024-05-08T22:29:53.768 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T22:29:53.784 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-05-08T22:29:53.826 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-05-08T22:29:54.078 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T22:29:54.098 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-05-08T22:29:54.136 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-05-08T22:29:54.138 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2024-05-08T22:29:54.157 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-05-08T22:29:54.216 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T22:29:54.474 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T22:29:54.493 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-05-08T22:29:54.506 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T22:29:54.524 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T22:29:54.541 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T22:29:54.598 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T22:29:54.875 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjs-underscore. 2024-05-08T22:29:54.876 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T22:29:54.893 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-05-08T22:29:54.895 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-05-08T22:29:54.934 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T22:29:54.935 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-05-08T22:29:55.236 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-05-08T22:29:55.252 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-05-08T22:29:55.271 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T22:29:55.288 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-05-08T22:29:55.294 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T22:29:55.330 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-05-08T22:29:55.642 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-routes. 2024-05-08T22:29:55.660 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-05-08T22:29:55.691 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T22:29:55.710 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-05-08T22:29:55.721 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-05-08T22:29:55.749 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T22:29:56.104 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T22:29:56.124 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:56.146 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T22:29:56.162 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:56.164 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-05-08T22:29:56.238 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T22:29:56.916 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T22:29:56.932 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:29:56.967 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:57.354 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T22:29:57.373 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:57.412 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:57.775 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T22:29:57.789 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-05-08T22:29:57.832 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T22:29:57.885 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-decorator. 2024-05-08T22:29:57.901 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-05-08T22:29:57.944 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T22:29:58.151 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cryptography. 2024-05-08T22:29:58.167 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-05-08T22:29:58.209 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T22:29:58.271 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-05-08T22:29:58.288 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-05-08T22:29:58.329 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T22:29:58.607 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T22:29:58.626 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-05-08T22:29:58.679 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-05-08T22:29:58.724 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-scipy. 2024-05-08T22:29:58.743 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-05-08T22:29:58.781 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-05-08T22:29:59.041 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T22:29:59.061 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-ceph-mgr_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:59.098 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:29:59.610 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T22:29:59.626 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-ceph-mon_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:29:59.668 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:00.200 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-05-08T22:30:00.217 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-05-08T22:30:00.257 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:00.705 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T22:30:00.723 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-ceph-osd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:00.755 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:00.845 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T22:30:00.861 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-05-08T22:30:00.902 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-05-08T22:30:01.323 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T22:30:01.340 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-05-08T22:30:01.381 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:02.114 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T22:30:02.135 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:02.197 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:02.933 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2024-05-08T22:30:02.951 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:02.991 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:03.367 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T22:30:03.384 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-05-08T22:30:03.385 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T22:30:03.402 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-ceph-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:03.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-05-08T22:30:03.443 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:03.813 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T22:30:03.832 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-05-08T22:30:03.896 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T22:30:03.930 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T22:30:03.949 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-ceph-mds_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:03.988 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:04.242 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T22:30:04.262 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-05-08T22:30:04.300 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T22:30:04.549 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2024-05-08T22:30:04.567 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-cephadm_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:04.607 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:04.639 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T22:30:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-05-08T22:30:04.697 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rsa (4.0-3) ... 2024-05-08T22:30:04.952 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T22:30:04.973 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-05-08T22:30:05.010 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-05-08T22:30:05.059 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T22:30:05.075 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-05-08T22:30:05.116 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-05-08T22:30:05.430 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T22:30:05.447 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-05-08T22:30:05.470 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-blinker. 2024-05-08T22:30:05.488 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-05-08T22:30:05.488 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T22:30:05.528 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T22:30:05.841 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T22:30:05.858 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:05.892 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-oauthlib. 2024-05-08T22:30:05.899 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:05.911 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-05-08T22:30:05.956 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T22:30:06.293 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T22:30:06.312 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-05-08T22:30:06.319 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jwt. 2024-05-08T22:30:06.335 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-05-08T22:30:06.351 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T22:30:06.377 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T22:30:06.687 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T22:30:06.707 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-05-08T22:30:06.711 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-attr. 2024-05-08T22:30:06.728 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-05-08T22:30:06.770 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-05-08T22:30:06.770 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T22:30:07.116 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T22:30:07.123 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T22:30:07.133 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-05-08T22:30:07.141 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-05-08T22:30:07.165 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T22:30:07.181 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-05-08T22:30:07.509 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2024-05-08T22:30:07.526 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-05-08T22:30:07.567 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T22:30:07.737 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T22:30:07.755 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:07.795 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:07.928 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-atomicwrites. 2024-05-08T22:30:07.944 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-05-08T22:30:08.011 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T22:30:08.148 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T22:30:08.165 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-05-08T22:30:08.206 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-05-08T22:30:08.314 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-packaging. 2024-05-08T22:30:08.334 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-05-08T22:30:08.373 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-packaging (20.3-1) ... 2024-05-08T22:30:08.634 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T22:30:08.651 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:30:08.692 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:08.709 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T22:30:08.724 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-05-08T22:30:08.767 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-05-08T22:30:08.987 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package jq. 2024-05-08T22:30:09.004 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:30:09.045 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:09.154 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T22:30:09.173 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T22:30:09.212 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T22:30:09.323 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package socat. 2024-05-08T22:30:09.340 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-05-08T22:30:09.381 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking socat (1.7.3.3-2) ... 2024-05-08T22:30:09.507 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjs-underscore. 2024-05-08T22:30:09.523 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-05-08T22:30:09.565 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T22:30:09.726 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T22:30:09.746 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-05-08T22:30:09.784 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-05-08T22:30:09.867 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-05-08T22:30:09.883 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-05-08T22:30:09.927 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T22:30:10.165 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-test. 2024-05-08T22:30:10.181 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:10.222 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:10.324 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2024-05-08T22:30:10.340 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-05-08T22:30:10.382 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-05-08T22:30:10.693 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T22:30:10.713 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:10.751 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:12.197 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-decorator. 2024-05-08T22:30:12.217 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-05-08T22:30:12.264 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T22:30:12.620 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-05-08T22:30:12.638 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-05-08T22:30:12.683 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T22:30:13.027 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-scipy. 2024-05-08T22:30:13.043 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-05-08T22:30:13.085 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-05-08T22:30:14.462 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-05-08T22:30:14.468 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T22:30:14.479 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-05-08T22:30:14.485 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:14.520 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:14.527 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:14.898 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T22:30:14.914 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:14.947 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:15.099 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T22:30:15.115 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-05-08T22:30:15.156 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-05-08T22:30:15.342 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T22:30:15.358 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-05-08T22:30:15.400 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-05-08T22:30:15.585 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T22:30:15.605 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-05-08T22:30:15.643 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:15.695 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T22:30:15.712 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../100-cephfs-shell_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:15.753 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:16.039 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T22:30:16.056 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../101-cephfs-top_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:16.097 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:16.351 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T22:30:16.368 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:16.395 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package fonts-mathjax. 2024-05-08T22:30:16.415 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T22:30:16.434 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:16.453 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:17.007 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T22:30:17.025 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:17.090 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T22:30:17.517 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-05-08T22:30:17.577 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T22:30:17.579 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T22:30:17.598 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-05-08T22:30:17.637 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-05-08T22:30:17.956 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T22:30:17.975 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-05-08T22:30:18.014 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-05-08T22:30:18.051 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T22:30:18.068 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-05-08T22:30:18.134 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T22:30:18.409 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjs-mathjax. 2024-05-08T22:30:18.428 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T22:30:18.467 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:18.488 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T22:30:18.509 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-05-08T22:30:18.542 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T22:30:18.887 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T22:30:18.904 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-05-08T22:30:18.945 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.0-3) ... 2024-05-08T22:30:19.307 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T22:30:19.324 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-05-08T22:30:19.365 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-05-08T22:30:19.718 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-blinker. 2024-05-08T22:30:19.738 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-05-08T22:30:19.768 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T22:30:20.063 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-oauthlib. 2024-05-08T22:30:20.082 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-05-08T22:30:20.112 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T22:30:20.218 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-natsort-doc. 2024-05-08T22:30:20.236 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-05-08T22:30:20.276 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-05-08T22:30:20.458 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T22:30:20.478 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-05-08T22:30:20.508 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T22:30:20.654 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webencodings. 2024-05-08T22:30:20.671 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-05-08T22:30:20.712 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T22:30:20.819 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T22:30:20.836 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-05-08T22:30:20.877 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T22:30:21.040 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-html5lib. 2024-05-08T22:30:21.057 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-05-08T22:30:21.098 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-05-08T22:30:21.214 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T22:30:21.233 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-05-08T22:30:21.298 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T22:30:21.435 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-05-08T22:30:21.455 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-05-08T22:30:21.518 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T22:30:21.910 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T22:30:21.927 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:21.968 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:22.013 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T22:30:22.030 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T22:30:22.087 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T22:30:22.329 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T22:30:22.347 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-05-08T22:30:22.388 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-05-08T22:30:22.425 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T22:30:22.444 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-05-08T22:30:22.482 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T22:30:22.707 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T22:30:22.724 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:30:22.765 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:22.810 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T22:30:22.831 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-05-08T22:30:22.869 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T22:30:23.060 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2024-05-08T22:30:23.078 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:30:23.124 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:23.246 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T22:30:23.269 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-05-08T22:30:23.305 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T22:30:23.414 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2024-05-08T22:30:23.433 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-05-08T22:30:23.470 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.3.3-2) ... 2024-05-08T22:30:23.598 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T22:30:23.617 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-05-08T22:30:23.666 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T22:30:23.808 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T22:30:23.824 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-05-08T22:30:23.885 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-05-08T22:30:24.237 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2024-05-08T22:30:24.255 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:24.298 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:24.436 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package radosgw. 2024-05-08T22:30:24.455 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../116-radosgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:24.494 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:25.897 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T22:30:25.915 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../117-rbd-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:25.955 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:26.290 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-attr (19.3.0-2) ... 2024-05-08T22:30:26.593 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T22:30:26.744 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T22:30:26.870 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T22:30:27.088 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:27.825 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user cephadm....done 2024-05-08T22:30:28.005 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:28.123 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T22:30:28.408 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T22:30:28.426 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:28.466 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:28.482 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T22:30:28.834 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:28.904 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T22:30:28.925 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:28.960 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-05-08T22:30:28.962 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T22:30:29.349 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T22:30:29.365 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-05-08T22:30:29.417 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-05-08T22:30:29.454 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-05-08T22:30:29.676 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T22:30:29.694 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-cephfs-shell_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:29.743 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:29.873 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T22:30:30.029 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T22:30:30.045 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-cephfs-top_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T22:30:30.087 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:30.199 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-05-08T22:30:30.352 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package fonts-mathjax. 2024-05-08T22:30:30.370 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T22:30:30.409 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:30.576 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-05-08T22:30:30.893 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T22:30:30.913 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T22:30:30.931 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:30.963 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:31.178 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T22:30:31.391 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T22:30:31.408 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-05-08T22:30:31.471 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:31.501 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T22:30:31.763 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:31.846 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T22:30:31.863 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-05-08T22:30:31.881 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-05-08T22:30:31.904 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-05-08T22:30:32.199 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-05-08T22:30:32.275 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjs-mathjax. 2024-05-08T22:30:32.293 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T22:30:32.374 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:32.491 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T22:30:33.035 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T22:30:33.163 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T22:30:33.523 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-05-08T22:30:33.899 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T22:30:34.093 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-natsort-doc. 2024-05-08T22:30:34.112 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-05-08T22:30:34.150 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-05-08T22:30:34.208 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T22:30:34.545 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webencodings. 2024-05-08T22:30:34.563 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-05-08T22:30:34.585 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T22:30:34.606 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T22:30:34.874 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-html5lib. 2024-05-08T22:30:34.877 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T22:30:34.890 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-05-08T22:30:34.931 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-05-08T22:30:35.212 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-05-08T22:30:35.302 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-05-08T22:30:35.317 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-05-08T22:30:35.392 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T22:30:35.513 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-packaging (20.3-1) ... 2024-05-08T22:30:35.814 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T22:30:35.888 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T22:30:35.904 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T22:30:35.965 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T22:30:35.987 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T22:30:36.074 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T22:30:36.341 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T22:30:36.357 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-05-08T22:30:36.358 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T22:30:36.399 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T22:30:36.760 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T22:30:36.779 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-05-08T22:30:36.819 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-05-08T22:30:36.851 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T22:30:37.139 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T22:30:37.188 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T22:30:37.204 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-05-08T22:30:37.246 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T22:30:37.532 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T22:30:37.565 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T22:30:37.583 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-05-08T22:30:37.623 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T22:30:37.892 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up socat (1.7.3.3-2) ... 2024-05-08T22:30:38.051 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:38.335 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2024-05-08T22:30:38.352 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../116-radosgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:38.386 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:30:38.393 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:38.545 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-05-08T22:30:38.913 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T22:30:39.039 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-05-08T22:30:39.198 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-05-08T22:30:39.493 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T22:30:39.802 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:30:39.830 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T22:30:39.848 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../117-rbd-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T22:30:39.888 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:39.962 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T22:30:40.223 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-attr (19.3.0-2) ... 2024-05-08T22:30:40.423 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T22:30:40.593 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T22:30:40.728 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T22:30:40.732 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-05-08T22:30:40.887 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T22:30:41.058 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T22:30:41.080 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:41.218 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T22:30:41.519 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-05-08T22:30:41.721 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2024-05-08T22:30:41.811 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T22:30:41.878 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:42.029 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T22:30:42.121 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T22:30:42.381 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T22:30:42.499 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-05-08T22:30:42.674 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T22:30:42.790 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-05-08T22:30:42.800 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-05-08T22:30:42.951 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T22:30:42.958 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T22:30:43.284 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-05-08T22:30:43.286 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-05-08T22:30:43.644 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T22:30:43.688 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T22:30:43.806 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T22:30:43.929 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-05-08T22:30:44.031 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-05-08T22:30:44.305 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T22:30:44.340 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-05-08T22:30:44.633 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T22:30:44.640 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-05-08T22:30:44.926 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T22:30:45.236 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:45.529 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:45.655 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-05-08T22:30:45.964 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-05-08T22:30:46.112 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:46.238 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-05-08T22:30:46.266 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T22:30:46.577 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T22:30:46.648 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-05-08T22:30:46.730 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T22:30:46.957 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-05-08T22:30:47.090 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-05-08T22:30:47.241 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:30:47.351 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-05-08T22:30:47.374 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T22:30:47.692 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T22:30:47.761 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T22:30:48.090 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-05-08T22:30:48.093 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T22:30:48.395 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T22:30:48.472 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T22:30:48.729 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-05-08T22:30:48.799 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T22:30:49.038 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-packaging (20.3-1) ... 2024-05-08T22:30:49.348 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T22:30:49.515 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T22:30:49.633 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T22:30:49.862 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T22:30:49.917 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T22:30:49.988 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-05-08T22:30:50.139 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:50.265 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rsa (4.0-3) ... 2024-05-08T22:30:50.322 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-05-08T22:30:50.567 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T22:30:50.682 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T22:30:50.710 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-05-08T22:30:51.011 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T22:30:51.041 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T22:30:51.304 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T22:30:51.393 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.3.3-2) ... 2024-05-08T22:30:51.544 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:51.622 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T22:30:51.878 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:30:51.974 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librados2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:52.029 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-05-08T22:30:52.211 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-05-08T22:30:52.378 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:52.414 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T22:30:52.504 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:52.541 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-05-08T22:30:52.664 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:52.693 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-05-08T22:30:52.791 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-routes (2.4.1-2) ... 2024-05-08T22:30:52.985 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T22:30:53.099 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:53.267 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-05-08T22:30:53.303 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:30:53.430 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T22:30:53.551 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librbd1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:53.589 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T22:30:53.661 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:53.812 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:53.899 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T22:30:54.035 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T22:30:54.035 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T22:30:54.183 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-05-08T22:30:54.476 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T22:30:54.505 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:54.602 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T22:30:54.656 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:54.782 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:54.912 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-05-08T22:30:54.941 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T22:30:55.212 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T22:30:55.555 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T22:30:55.870 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:55.917 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-05-08T22:30:55.996 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:56.155 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:56.191 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-05-08T22:30:56.281 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:56.317 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T22:30:56.432 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:56.558 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:56.690 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-05-08T22:30:57.061 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T22:30:57.111 INFO:teuthology.orchestra.run.smithi179.stdout:Adding group ceph....done 2024-05-08T22:30:57.187 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T22:30:57.355 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-05-08T22:30:57.519 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user ceph....done 2024-05-08T22:30:57.681 INFO:teuthology.orchestra.run.smithi179.stdout:Setting system user ceph properties....done 2024-05-08T22:30:57.698 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T22:30:57.707 INFO:teuthology.orchestra.run.smithi179.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T22:30:57.862 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T22:30:57.982 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-05-08T22:30:58.199 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T22:30:58.686 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:58.878 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T22:30:58.879 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T22:30:59.295 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:59.421 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:30:59.455 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:30:59.597 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-05-08T22:30:59.919 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T22:30:59.919 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T22:30:59.957 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-05-08T22:31:00.250 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-05-08T22:31:00.397 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:00.543 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T22:31:00.668 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-05-08T22:31:00.746 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T22:31:01.053 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T22:31:01.150 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:01.389 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-05-08T22:31:01.426 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T22:31:01.426 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T22:31:01.765 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T22:31:01.877 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:02.075 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T22:31:02.106 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T22:31:02.106 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T22:31:02.545 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:03.060 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T22:31:03.061 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T22:31:03.432 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T22:31:03.689 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-05-08T22:31:03.807 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T22:31:03.922 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:03.946 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.0-3) ... 2024-05-08T22:31:04.056 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:04.232 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:04.276 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T22:31:04.401 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-05-08T22:31:04.451 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T22:31:04.451 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T22:31:04.703 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T22:31:04.892 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:04.995 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T22:31:05.052 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:05.169 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:05.330 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T22:31:05.412 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:05.757 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:05.765 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T22:31:05.942 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T22:31:05.969 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-05-08T22:31:06.120 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:06.235 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T22:31:06.254 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:06.397 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:06.540 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.4.1-2) ... 2024-05-08T22:31:06.891 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:07.017 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-05-08T22:31:07.310 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:07.461 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:07.587 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:07.824 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T22:31:07.824 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T22:31:08.314 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:08.440 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:08.608 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:08.734 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T22:31:09.238 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-05-08T22:31:09.638 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:09.789 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:09.879 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T22:31:09.882 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T22:31:09.915 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:09.956 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-08T22:31:10.066 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:10.170 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-08T22:31:10.171 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-08T22:31:10.192 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:10.351 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:flex is already the newest version (2.6.4-6.2). 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-05-08T22:31:10.384 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T22:31:10.385 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T22:31:10.385 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T22:31:10.385 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2024-05-08T22:31:10.386 INFO:teuthology.orchestra.run.smithi179.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-05-08T22:31:10.386 INFO:teuthology.orchestra.run.smithi179.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-05-08T22:31:10.386 INFO:teuthology.orchestra.run.smithi179.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-05-08T22:31:10.386 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-05-08T22:31:10.386 INFO:teuthology.orchestra.run.smithi179.stdout: usb-modeswitch-data wpasupplicant 2024-05-08T22:31:10.389 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2024-05-08T22:31:10.389 INFO:teuthology.orchestra.run.smithi179.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-05-08T22:31:10.389 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-05-08T22:31:10.390 INFO:teuthology.orchestra.run.smithi179.stdout: libengine-pkcs11-openssl 2024-05-08T22:31:10.448 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2024-05-08T22:31:10.449 INFO:teuthology.orchestra.run.smithi179.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-05-08T22:31:10.449 INFO:teuthology.orchestra.run.smithi179.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-05-08T22:31:10.450 INFO:teuthology.orchestra.run.smithi179.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-05-08T22:31:10.450 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-05-08T22:31:10.450 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-05-08T22:31:10.450 INFO:teuthology.orchestra.run.smithi179.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T22:31:10.451 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2024-05-08T22:31:10.452 INFO:teuthology.orchestra.run.smithi179.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-05-08T22:31:10.658 INFO:teuthology.orchestra.run.smithi179.stdout:6 upgraded, 31 newly installed, 0 to remove and 272 not upgraded. 2024-05-08T22:31:10.658 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 25.3 MB of archives. 2024-05-08T22:31:10.658 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 79.9 MB of additional disk space will be used. 2024-05-08T22:31:10.658 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-05-08T22:31:10.962 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2024-05-08T22:31:11.281 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-05-08T22:31:11.282 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-05-08T22:31:11.284 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-05-08T22:31:11.297 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-05-08T22:31:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.22 [1,323 kB] 2024-05-08T22:31:11.364 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2024-05-08T22:31:11.397 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-05-08T22:31:11.399 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-05-08T22:31:11.406 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-05-08T22:31:11.413 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-05-08T22:31:11.414 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.7 [60.7 kB] 2024-05-08T22:31:11.523 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2024-05-08T22:31:11.551 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T22:31:11.579 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-05-08T22:31:11.695 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T22:31:11.859 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-05-08T22:31:11.989 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-05-08T22:31:11.994 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-05-08T22:31:12.012 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T22:31:12.089 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2024-05-08T22:31:12.092 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-05-08T22:31:12.175 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.18-0ubuntu0.20.04.1 [116 kB] 2024-05-08T22:31:12.192 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-05-08T22:31:12.259 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-05-08T22:31:12.259 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.22 [1,586 kB] 2024-05-08T22:31:12.343 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-05-08T22:31:12.469 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:12.508 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-05-08T22:31:12.697 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T22:31:12.697 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T22:31:13.115 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2024-05-08T22:31:13.154 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:13.215 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-05-08T22:31:13.303 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-05-08T22:31:13.313 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-05-08T22:31:13.313 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:13.314 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-05-08T22:31:13.315 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-05-08T22:31:13.316 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.18-0ubuntu0.20.04.1 [1,054 kB] 2024-05-08T22:31:13.348 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-05-08T22:31:13.353 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.18-0ubuntu0.20.04.1 [13.5 MB] 2024-05-08T22:31:13.728 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-05-08T22:31:13.783 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T22:31:13.783 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T22:31:13.892 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-05-08T22:31:13.974 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-05-08T22:31:13.974 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-05-08T22:31:14.135 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-05-08T22:31:14.233 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:14.598 INFO:teuthology.orchestra.run.smithi179.stdout:Preconfiguring packages ... 2024-05-08T22:31:14.648 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T22:31:14.788 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 25.3 MB in 4s (6,782 kB/s) 2024-05-08T22:31:15.063 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:15.068 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:15.084 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:15.232 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:15.306 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T22:31:15.307 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T22:31:15.761 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:16.013 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T22:31:16.014 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T22:31:16.405 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:16.454 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:16.950 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T22:31:16.950 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T22:31:17.397 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:17.548 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:17.692 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:17.897 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:17.900 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:17.916 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T22:31:17.916 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T22:31:18.029 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:18.401 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:18.461 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:18.543 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:18.661 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:18.694 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:19.012 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:19.055 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T22:31:19.432 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T22:31:19.609 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T22:31:19.894 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T22:31:20.178 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:20.183 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:20.952 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T22:31:21.504 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:21.635 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T22:31:22.174 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T22:31:22.332 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.22) over (1.1.1f-1ubuntu2.16) ... 2024-05-08T22:31:22.780 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-05-08T22:31:22.846 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T22:31:22.864 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-05-08T22:31:24.423 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T22:31:24.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package dump. 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking dump (0.4b46-6) ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package indent. 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-05-08T22:31:24.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T22:31:24.504 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-05-08T22:31:24.540 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T22:31:24.558 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.7_amd64.deb ... 2024-05-08T22:31:24.623 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T22:31:24.719 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-05-08T22:31:24.720 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-05-08T22:31:24.920 INFO:teuthology.orchestra.run.smithi195.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-05-08T22:31:24.921 INFO:teuthology.orchestra.run.smithi195.stdout:flex is already the newest version (2.6.4-6.2). 2024-05-08T22:31:24.921 INFO:teuthology.orchestra.run.smithi195.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-05-08T22:31:24.921 INFO:teuthology.orchestra.run.smithi195.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-05-08T22:31:24.921 INFO:teuthology.orchestra.run.smithi195.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-05-08T22:31:24.921 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T22:31:24.922 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T22:31:24.922 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T22:31:24.922 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-05-08T22:31:24.922 INFO:teuthology.orchestra.run.smithi195.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-05-08T22:31:24.923 INFO:teuthology.orchestra.run.smithi195.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-05-08T22:31:24.923 INFO:teuthology.orchestra.run.smithi195.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-05-08T22:31:24.923 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-05-08T22:31:24.923 INFO:teuthology.orchestra.run.smithi195.stdout: usb-modeswitch-data wpasupplicant 2024-05-08T22:31:24.924 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-05-08T22:31:24.924 INFO:teuthology.orchestra.run.smithi195.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-05-08T22:31:24.924 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-05-08T22:31:24.924 INFO:teuthology.orchestra.run.smithi195.stdout: libengine-pkcs11-openssl 2024-05-08T22:31:24.937 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T22:31:24.954 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:24.984 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-05-08T22:31:24.984 INFO:teuthology.orchestra.run.smithi195.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-05-08T22:31:24.984 INFO:teuthology.orchestra.run.smithi195.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-05-08T22:31:24.985 INFO:teuthology.orchestra.run.smithi195.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-05-08T22:31:24.985 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-05-08T22:31:24.985 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-05-08T22:31:24.985 INFO:teuthology.orchestra.run.smithi195.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T22:31:24.986 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-05-08T22:31:24.987 INFO:teuthology.orchestra.run.smithi195.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-05-08T22:31:24.994 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:31:25.051 INFO:teuthology.orchestra.run.smithi195.stdout:6 upgraded, 31 newly installed, 0 to remove and 272 not upgraded. 2024-05-08T22:31:25.051 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 25.3 MB of archives. 2024-05-08T22:31:25.051 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 79.9 MB of additional disk space will be used. 2024-05-08T22:31:25.052 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-05-08T22:31:25.213 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-05-08T22:31:25.214 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-05-08T22:31:25.215 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-05-08T22:31:25.220 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-05-08T22:31:25.221 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.22 [1,323 kB] 2024-05-08T22:31:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-05-08T22:31:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-05-08T22:31:25.247 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-05-08T22:31:25.248 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-05-08T22:31:25.249 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.7 [60.7 kB] 2024-05-08T22:31:25.255 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-05-08T22:31:25.256 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-05-08T22:31:25.258 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-05-08T22:31:25.258 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-05-08T22:31:25.276 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2024-05-08T22:31:25.276 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-05-08T22:31:25.282 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.18-0ubuntu0.20.04.1 [116 kB] 2024-05-08T22:31:25.284 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-05-08T22:31:25.294 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-05-08T22:31:25.294 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.22 [1,586 kB] 2024-05-08T22:31:25.309 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-05-08T22:31:25.309 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-05-08T22:31:25.319 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2024-05-08T22:31:25.335 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-05-08T22:31:25.361 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-05-08T22:31:25.366 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-05-08T22:31:25.366 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-05-08T22:31:25.367 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-05-08T22:31:25.372 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.18-0ubuntu0.20.04.1 [1,054 kB] 2024-05-08T22:31:25.382 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T22:31:25.387 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-05-08T22:31:25.389 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.18-0ubuntu0.20.04.1 [13.5 MB] 2024-05-08T22:31:25.400 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:25.440 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:25.614 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-05-08T22:31:25.614 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-05-08T22:31:25.689 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-05-08T22:31:25.690 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-05-08T22:31:25.690 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-05-08T22:31:25.734 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T22:31:25.755 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:25.817 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:26.092 INFO:teuthology.orchestra.run.smithi195.stdout:Preconfiguring packages ... 2024-05-08T22:31:26.313 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 25.3 MB in 1s (36.3 MB/s) 2024-05-08T22:31:26.365 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T22:31:26.384 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:26.631 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:26.635 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:26.681 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:26.815 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:27.610 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T22:31:27.627 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2024-05-08T22:31:27.702 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T22:31:28.047 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:28.064 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T22:31:28.081 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T22:31:28.122 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:28.542 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T22:31:28.558 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../14-libpq5_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:28.600 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:28.987 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T22:31:29.005 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:29.045 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:29.348 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T22:31:29.366 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:29.423 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:29.701 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T22:31:29.719 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T22:31:29.759 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:29.985 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:29.989 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:30.123 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:30.313 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T22:31:30.332 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-05-08T22:31:30.371 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-05-08T22:31:30.562 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T22:31:30.746 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T22:31:30.758 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package modemmanager. 2024-05-08T22:31:30.779 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:30.860 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:31.099 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:31.549 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T22:31:31.566 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2024-05-08T22:31:31.600 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T22:31:31.605 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:31.606 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T22:31:31.770 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T22:31:32.130 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager. 2024-05-08T22:31:32.146 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T22:31:32.204 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:32.238 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:32.392 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T22:31:32.918 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T22:31:33.055 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.22) over (1.1.1f-1ubuntu2.16) ... 2024-05-08T22:31:33.321 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ppp. 2024-05-08T22:31:33.340 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-05-08T22:31:33.601 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T22:31:33.603 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T22:31:33.618 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-05-08T22:31:33.659 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-05-08T22:31:34.019 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T22:31:34.035 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-05-08T22:31:34.071 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package dump. 2024-05-08T22:31:34.077 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-05-08T22:31:34.088 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-05-08T22:31:34.137 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking dump (0.4b46-6) ... 2024-05-08T22:31:34.457 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package indent. 2024-05-08T22:31:34.473 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T22:31:34.475 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:34.489 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-05-08T22:31:34.516 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T22:31:34.548 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-05-08T22:31:34.876 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T22:31:34.896 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-05-08T22:31:34.908 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T22:31:34.929 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-05-08T22:31:34.934 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-05-08T22:31:35.179 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T22:31:35.248 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client-12. 2024-05-08T22:31:35.268 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../26-postgresql-client-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:35.305 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:35.961 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T22:31:35.980 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-05-08T22:31:36.068 INFO:teuthology.orchestra.run.smithi179.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T22:31:36.088 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-05-08T22:31:36.610 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-12. 2024-05-08T22:31:36.631 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../28-postgresql-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:36.672 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:36.841 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T22:31:36.862 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.7_amd64.deb ... 2024-05-08T22:31:36.898 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T22:31:37.160 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T22:31:37.179 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T22:31:37.209 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:31:37.571 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T22:31:37.591 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:37.629 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:37.915 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T22:31:37.932 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:37.998 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:38.301 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T22:31:38.322 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:38.351 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:38.738 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T22:31:38.758 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2024-05-08T22:31:38.795 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T22:31:39.065 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql. 2024-05-08T22:31:39.085 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:39.123 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-05-08T22:31:39.190 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T22:31:39.208 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T22:31:39.248 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:39.393 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T22:31:39.411 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:39.451 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-05-08T22:31:39.601 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T22:31:39.620 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../14-libpq5_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:39.651 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:39.729 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T22:31:39.746 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:39.787 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T22:31:40.029 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T22:31:40.048 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:40.081 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T22:31:40.087 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:40.098 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-05-08T22:31:40.139 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-05-08T22:31:40.448 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T22:31:40.468 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:40.507 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:40.585 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T22:31:40.602 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-05-08T22:31:40.654 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T22:31:40.767 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T22:31:40.785 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T22:31:40.856 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:40.997 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-05-08T22:31:41.242 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:41.354 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T22:31:41.374 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-05-08T22:31:41.454 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-05-08T22:31:41.577 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:41.736 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-05-08T22:31:41.863 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T22:31:42.022 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T22:31:42.301 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package modemmanager. 2024-05-08T22:31:42.321 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T22:31:42.437 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T22:31:42.437 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T22:31:42.444 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:42.917 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:43.143 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T22:31:43.160 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2024-05-08T22:31:43.200 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T22:31:43.764 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager. 2024-05-08T22:31:43.781 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T22:31:43.786 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T22:31:43.880 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:44.307 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:44.433 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-05-08T22:31:44.582 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-05-08T22:31:44.654 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ppp. 2024-05-08T22:31:44.672 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-05-08T22:31:44.962 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T22:31:45.318 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T22:31:45.338 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-05-08T22:31:45.376 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-05-08T22:31:45.658 INFO:teuthology.orchestra.run.smithi179.stdout:Adding user postgres to group ssl-cert 2024-05-08T22:31:45.763 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T22:31:45.780 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-05-08T22:31:45.872 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-05-08T22:31:45.960 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:45.960 INFO:teuthology.orchestra.run.smithi179.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T22:31:46.158 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T22:31:46.175 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-05-08T22:31:46.196 INFO:teuthology.orchestra.run.smithi179.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T22:31:46.196 INFO:teuthology.orchestra.run.smithi179.stdout:Removing obsolete dictionary files: 2024-05-08T22:31:46.207 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-05-08T22:31:46.514 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T22:31:46.537 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-12. 2024-05-08T22:31:46.553 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../26-postgresql-client-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:46.595 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:47.250 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T22:31:47.268 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-05-08T22:31:47.305 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:47.339 INFO:teuthology.orchestra.run.smithi195.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T22:31:47.370 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-05-08T22:31:47.431 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-05-08T22:31:47.583 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:47.684 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T22:31:48.031 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-12. 2024-05-08T22:31:48.048 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../28-postgresql-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T22:31:48.101 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:48.697 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-05-08T22:31:48.941 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up dump (0.4b46-6) ... 2024-05-08T22:31:49.037 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T22:31:49.101 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:49.269 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T22:31:49.395 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T22:31:49.596 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T22:31:49.773 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T22:31:49.801 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql. 2024-05-08T22:31:49.820 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:49.859 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-05-08T22:31:49.924 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:50.112 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T22:31:50.130 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:50.161 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-05-08T22:31:50.300 INFO:teuthology.orchestra.run.smithi179.stdout:Creating new PostgreSQL cluster 12/main ... 2024-05-08T22:31:50.314 INFO:teuthology.orchestra.run.smithi179.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-05-08T22:31:50.369 INFO:teuthology.orchestra.run.smithi179.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:This user must also own the server process. 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:The default text search configuration will be set to "english". 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:Data page checksums are disabled. 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:50.370 INFO:teuthology.orchestra.run.smithi179.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-05-08T22:31:50.371 INFO:teuthology.orchestra.run.smithi179.stdout:creating subdirectories ... ok 2024-05-08T22:31:50.371 INFO:teuthology.orchestra.run.smithi179.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T22:31:50.391 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default max_connections ... 100 2024-05-08T22:31:50.418 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default shared_buffers ... 128MB 2024-05-08T22:31:50.422 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T22:31:50.425 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default time zone ... Etc/UTC 2024-05-08T22:31:50.425 INFO:teuthology.orchestra.run.smithi179.stdout:creating configuration files ... ok 2024-05-08T22:31:50.440 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-05-08T22:31:50.480 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T22:31:50.530 INFO:teuthology.orchestra.run.smithi179.stdout:running bootstrap script ... ok 2024-05-08T22:31:50.766 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T22:31:50.784 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-05-08T22:31:50.841 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-05-08T22:31:51.074 INFO:teuthology.orchestra.run.smithi179.stdout:performing post-bootstrap initialization ... ok 2024-05-08T22:31:51.246 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T22:31:51.264 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-05-08T22:31:51.313 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T22:31:51.623 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-05-08T22:31:51.819 INFO:teuthology.orchestra.run.smithi179.stdout:syncing data to disk ... ok 2024-05-08T22:31:51.819 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:51.819 INFO:teuthology.orchestra.run.smithi179.stdout:Success. You can now start the database server using: 2024-05-08T22:31:51.819 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:51.819 INFO:teuthology.orchestra.run.smithi179.stdout: pg_ctlcluster 12 main start 2024-05-08T22:31:51.820 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:51.875 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:52.138 INFO:teuthology.orchestra.run.smithi179.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-05-08T22:31:52.138 INFO:teuthology.orchestra.run.smithi179.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-05-08T22:31:52.210 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:52.227 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T22:31:52.361 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-05-08T22:31:52.487 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T22:31:52.713 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T22:31:53.142 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T22:31:53.142 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T22:31:53.617 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:31:54.557 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T22:31:54.647 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T22:31:54.773 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-05-08T22:31:54.924 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-05-08T22:31:55.207 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:31:55.334 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:55.484 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-05-08T22:31:55.685 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:31:55.812 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-05-08T22:31:55.971 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:56.033 INFO:teuthology.orchestra.run.smithi195.stdout:Adding user postgres to group ssl-cert 2024-05-08T22:31:56.282 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:31:56.282 INFO:teuthology.orchestra.run.smithi195.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T22:31:56.533 INFO:teuthology.orchestra.run.smithi195.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T22:31:56.533 INFO:teuthology.orchestra.run.smithi195.stdout:Removing obsolete dictionary files: 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout:which means they are currently configured by ifupdown: 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout:- enp3s0f1 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T22:31:56.723 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:31:56.880 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T22:31:56.890 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T22:31:57.208 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T22:31:57.740 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T22:31:58.160 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T22:31:58.275 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:31:58.310 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-05-08T22:31:58.401 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-05-08T22:31:58.436 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T22:31:58.561 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:31:58.579 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T22:31:58.695 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T22:31:58.855 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-05-08T22:31:58.981 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:59.141 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:31:59.267 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:31:59.563 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T22:31:59.563 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T22:31:59.597 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-05-08T22:31:59.843 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up dump (0.4b46-6) ... 2024-05-08T22:31:59.974 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T22:32:00.036 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:32:00.120 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-05-08T22:32:00.162 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T22:32:00.229 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T22:32:00.313 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T22:32:00.381 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T22:32:00.439 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T22:32:00.554 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T22:32:00.657 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T22:32:00.817 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T22:32:00.876 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T22:32:01.180 INFO:teuthology.orchestra.run.smithi195.stdout:Creating new PostgreSQL cluster 12/main ... 2024-05-08T22:32:01.200 INFO:teuthology.orchestra.run.smithi195.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:This user must also own the server process. 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:The default text search configuration will be set to "english". 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout:Data page checksums are disabled. 2024-05-08T22:32:01.253 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:01.254 INFO:teuthology.orchestra.run.smithi195.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-05-08T22:32:01.254 INFO:teuthology.orchestra.run.smithi195.stdout:creating subdirectories ... ok 2024-05-08T22:32:01.254 INFO:teuthology.orchestra.run.smithi195.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T22:32:01.270 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default max_connections ... 100 2024-05-08T22:32:01.288 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default shared_buffers ... 128MB 2024-05-08T22:32:01.303 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default time zone ... Etc/UTC 2024-05-08T22:32:01.304 INFO:teuthology.orchestra.run.smithi195.stdout:creating configuration files ... ok 2024-05-08T22:32:01.412 INFO:teuthology.orchestra.run.smithi195.stdout:running bootstrap script ... ok 2024-05-08T22:32:01.930 INFO:teuthology.orchestra.run.smithi195.stdout:performing post-bootstrap initialization ... ok 2024-05-08T22:32:02.811 INFO:teuthology.orchestra.run.smithi195.stdout:syncing data to disk ... ok 2024-05-08T22:32:02.812 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:02.812 INFO:teuthology.orchestra.run.smithi195.stdout:Success. You can now start the database server using: 2024-05-08T22:32:02.812 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:02.812 INFO:teuthology.orchestra.run.smithi195.stdout: pg_ctlcluster 12 main start 2024-05-08T22:32:02.812 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:03.142 INFO:teuthology.orchestra.run.smithi195.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-05-08T22:32:03.142 INFO:teuthology.orchestra.run.smithi195.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-05-08T22:32:03.220 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T22:32:06.207 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:32:06.333 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:32:06.492 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-05-08T22:32:06.693 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:32:06.819 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-05-08T22:32:06.970 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T22:32:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:07.710 INFO:teuthology.orchestra.run.smithi195.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T22:32:07.710 INFO:teuthology.orchestra.run.smithi195.stdout:which means they are currently configured by ifupdown: 2024-05-08T22:32:07.710 INFO:teuthology.orchestra.run.smithi195.stdout:- enp3s0f1 2024-05-08T22:32:07.711 INFO:teuthology.orchestra.run.smithi195.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T22:32:07.711 INFO:teuthology.orchestra.run.smithi195.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T22:32:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:07.872 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T22:32:07.997 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-05-08T22:32:08.148 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-05-08T22:32:08.198 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T22:32:08.719 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T22:32:09.018 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T22:32:09.021 DEBUG:teuthology.parallel:result is None 2024-05-08T22:32:09.232 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T22:32:09.391 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-05-08T22:32:09.534 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T22:32:09.685 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T22:32:09.803 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-05-08T22:32:09.970 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:32:10.096 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T22:32:10.281 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T22:32:10.554 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T22:32:10.554 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T22:32:11.089 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-05-08T22:32:11.193 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T22:32:11.327 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T22:32:11.504 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T22:32:11.797 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T22:32:19.193 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-05-08T22:32:19.318 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-05-08T22:32:20.158 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T22:32:20.162 DEBUG:teuthology.parallel:result is None 2024-05-08T22:32:20.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:32:20.327 DEBUG:teuthology.orchestra.run.smithi179:> dpkg-query -W -f '${Version}' ceph 2024-05-08T22:32:20.351 INFO:teuthology.orchestra.run.smithi179.stdout:18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:32:20.352 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:32:20.352 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322-gd21e4f96-1focal is installed. 2024-05-08T22:32:20.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:32:20.580 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2024-05-08T22:32:20.605 INFO:teuthology.orchestra.run.smithi195.stdout:18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:32:20.605 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T22:32:20.605 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322-gd21e4f96-1focal is installed. 2024-05-08T22:32:20.607 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T22:32:20.607 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:20.607 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:32:20.624 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:20.624 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:32:20.641 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T22:32:20.641 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:20.641 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:32:20.678 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:32:20.694 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:20.694 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:32:20.710 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:32:20.771 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T22:32:20.771 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:20.771 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:32:20.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:32:20.844 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:20.845 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:32:20.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:32:20.869 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T22:32:20.870 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:20.870 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:32:20.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:32:20.943 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:20.944 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:32:20.959 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:32:20.968 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T22:32:21.054 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T22:32:21.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /var/log/ceph 2024-05-08T22:32:21.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-05-08T22:32:21.072 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T22:32:21.072 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T22:32:21.074 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T22:32:21.131 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T22:32:21.132 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T22:32:21.134 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T22:32:21.152 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T22:32:21.152 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T22:32:21.152 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T22:32:21.153 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:21.153 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T22:32:21.158 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T22:32:21.239 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:21.239 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T22:32:21.246 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T22:32:21.332 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T22:32:21.332 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T22:32:21.332 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117', 'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7698117', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240508.150423-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2738, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'sleep_before_teardown': 0, 'subset': '17/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240508.150423-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi179.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_19:22:21', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T22:32:21.333 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T22:32:21.337 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T22:32:21.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T22:32:21.387 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T22:32:21.402 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:21.402 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:32:21.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:32:21.407 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-05-08T22:32:21.412 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-08 22:27:17.510505361 +0000 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-08 22:27:17.306481915 +0000 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-08 22:27:17.306481915 +0000 2024-05-08T22:32:21.413 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-05-08T22:32:21.413 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:32:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-08T22:32:21.425 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-08T22:32:21.425 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.00043956 s, 1.2 MB/s 2024-05-08T22:32:21.426 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:32:21.481 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-08 22:27:17.918552254 +0000 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-08 22:27:17.702527428 +0000 2024-05-08T22:32:21.492 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-08 22:27:17.702527428 +0000 2024-05-08T22:32:21.493 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-05-08T22:32:21.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:32:21.507 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-08T22:32:21.507 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-08T22:32:21.507 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000370854 s, 1.4 MB/s 2024-05-08T22:32:21.508 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:32:21.516 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-05-08T22:32:21.564 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 370 Links: 1 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-08 22:27:18.362601489 +0000 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-08 22:27:18.138576865 +0000 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-08 22:27:18.138576865 +0000 2024-05-08T22:32:21.565 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-05-08T22:32:21.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:32:21.576 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-08T22:32:21.576 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-08T22:32:21.576 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000407308 s, 1.3 MB/s 2024-05-08T22:32:21.577 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:32:21.586 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-08 22:27:18.794648981 +0000 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-08 22:27:18.578625235 +0000 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-08 22:27:18.578625235 +0000 2024-05-08T22:32:21.594 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-05-08T22:32:21.594 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:32:21.651 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-08T22:32:21.651 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-08T22:32:21.651 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000376212 s, 1.4 MB/s 2024-05-08T22:32:21.652 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:32:21.659 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:21.659 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:21.659 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:32:21.667 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:32:21.668 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-05-08T22:32:21.676 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:32:21.676 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-08 22:27:26.113569760 +0000 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-08 22:27:25.893574833 +0000 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-08 22:27:25.893574833 +0000 2024-05-08T22:32:21.677 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-05-08T22:32:21.678 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:32:21.734 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-08T22:32:21.734 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-08T22:32:21.734 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000301038 s, 1.7 MB/s 2024-05-08T22:32:21.736 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:32:21.744 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-05-08T22:32:21.749 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-08 22:27:26.549559704 +0000 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-08 22:27:26.321564962 +0000 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-08 22:27:26.321564962 +0000 2024-05-08T22:32:21.750 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-05-08T22:32:21.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:32:21.806 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-08T22:32:21.806 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-08T22:32:21.806 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.00030716 s, 1.7 MB/s 2024-05-08T22:32:21.808 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:32:21.815 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-05-08T22:32:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:32:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-08 22:27:26.761554816 +0000 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-08 22:27:26.757554907 +0000 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-08 22:27:26.757554907 +0000 2024-05-08T22:32:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-05-08T22:32:21.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:32:21.878 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-08T22:32:21.878 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-08T22:32:21.878 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000294774 s, 1.7 MB/s 2024-05-08T22:32:21.879 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:32:21.887 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 6h/6d Inode: 376 Links: 1 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-08 22:27:27.157545684 +0000 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-08 22:27:27.153545775 +0000 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-08 22:27:27.153545775 +0000 2024-05-08T22:32:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-05-08T22:32:21.892 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:32:21.950 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-08T22:32:21.950 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-08T22:32:21.950 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.00029113 s, 1.8 MB/s 2024-05-08T22:32:21.952 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:32:21.959 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:21.959 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T22:32:21.959 INFO:tasks.ceph:Generating config... 2024-05-08T22:32:21.960 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T22:32:21.961 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T22:32:21.961 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T22:32:21.961 INFO:tasks.ceph:[client] pid file = 2024-05-08T22:32:21.961 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T22:32:21.961 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T22:32:21.961 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T22:32:21.962 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T22:32:21.962 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T22:32:21.962 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T22:32:21.963 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T22:32:21.963 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T22:32:21.964 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T22:32:22.004 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T22:32:22.015 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T22:32:22.101 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T22:32:22.115 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.179'), ('mon.b', '172.21.15.195'), ('mon.c', '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]')] 2024-05-08T22:32:22.115 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T22:32:22.116 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:22.116 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T22:32:22.121 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.179 --add b 172.21.15.195 --addv c '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:monmaptool: generated fsid 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:setting min_mon_release = pacific 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:epoch 0 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:fsid 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:last_changed 2024-05-08T22:32:22.214536+0000 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:created 2024-05-08T22:32:22.214536+0000 2024-05-08T22:32:22.212 INFO:teuthology.orchestra.run.smithi179.stdout:min_mon_release 16 (pacific) 2024-05-08T22:32:22.213 INFO:teuthology.orchestra.run.smithi179.stdout:election_strategy: 1 2024-05-08T22:32:22.213 INFO:teuthology.orchestra.run.smithi179.stdout:0: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.a 2024-05-08T22:32:22.213 INFO:teuthology.orchestra.run.smithi179.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-05-08T22:32:22.213 INFO:teuthology.orchestra.run.smithi179.stdout:2: [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] mon.c 2024-05-08T22:32:22.213 INFO:teuthology.orchestra.run.smithi179.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T22:32:22.214 DEBUG:teuthology.orchestra.run.smithi179:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T22:32:22.221 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 88a906d6-7c22-496e-b9c7-b34c51a47fa8... 2024-05-08T22:32:22.222 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:32:22.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:32:22.252 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-08T22:32:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-08T22:32:22.254 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-08T22:32:22.255 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-08T22:32:22.256 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-08T22:32:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-08T22:32:22.258 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-08T22:32:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-08T22:32:22.260 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:32:22.261 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:32:22.262 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-08T22:32:22.263 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:32:22.264 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = True 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = True 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:32:22.265 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:32:22.266 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-05-08T22:32:22.266 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-05-08T22:32:22.266 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:32:22.266 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-05-08T22:32:22.267 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:32:22.268 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:32:22.269 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-05-08T22:32:22.270 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.271 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.272 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-05-08T22:32:22.273 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.274 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-05-08T22:32:22.275 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-05-08T22:32:22.276 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:32:22.277 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:32:22.278 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:32:22.279 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:32:22.281 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T22:32:22.281 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T22:32:22.377 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T22:32:22.377 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:22.377 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T22:32:22.383 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:22.383 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T22:32:22.390 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:32:22.390 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:22.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T22:32:22.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T22:32:22.411 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:22.411 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:22.459 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:32:22.459 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:22.460 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T22:32:22.460 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T22:32:22.487 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:22.487 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:22.492 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T22:32:22.492 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T22:32:22.492 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T22:32:22.543 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T22:32:22.546 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T22:32:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T22:32:22.605 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T22:32:22.606 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T22:32:22.657 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T22:32:22.660 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T22:32:22.672 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T22:32:22.716 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T22:32:22.719 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T22:32:22.734 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T22:32:22.781 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T22:32:22.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T22:32:22.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T22:32:22.843 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T22:32:22.846 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T22:32:22.858 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T22:32:22.858 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T22:32:22.900 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T22:32:22.912 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T22:32:22.951 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T22:32:22.964 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T22:32:23.005 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T22:32:23.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T22:32:23.100 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T22:32:23.113 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T22:32:23.113 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T22:32:23.114 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T22:32:23.130 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:23.131 INFO:tasks.ceph:role: osd.0 2024-05-08T22:32:23.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:32:23.131 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T22:32:23.711 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:23.711 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:23.712 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:23.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-08T22:32:23.714 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T22:32:23.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T22:32:23.818 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:23.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:23.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T22:32:23.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:23.877 INFO:tasks.ceph:role: osd.1 2024-05-08T22:32:23.877 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:32:23.877 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T22:32:24.374 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:24.374 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:24.374 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:24.375 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:24.377 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-08T22:32:24.377 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T22:32:24.410 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T22:32:24.464 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:24.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:24.466 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T22:32:24.475 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:24.475 INFO:tasks.ceph:role: osd.2 2024-05-08T22:32:24.475 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:32:24.475 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:24.935 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:24.936 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:24.936 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:24.936 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:24.937 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-08T22:32:24.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T22:32:24.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T22:32:25.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:25.008 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:25.008 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T22:32:25.020 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:25.020 INFO:tasks.ceph:role: osd.3 2024-05-08T22:32:25.020 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:32:25.020 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:25.523 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:25.524 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:25.524 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:25.524 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:25.524 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:25.525 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-08T22:32:25.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T22:32:25.586 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T22:32:25.599 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:25.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:25.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:25.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.669+0000 7fd7dd61e880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T22:32:25.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.673+0000 7fd7dd61e880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T22:32:25.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.673+0000 7fd7dd61e880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T22:32:25.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.673+0000 7fd7dd61e880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T22:32:25.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.673+0000 7fd7dd61e880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T22:32:25.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:25.677+0000 7fd7dd61e880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T22:32:28.705 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T22:32:28.722 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:28.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.809+0000 7fdc8f9ef880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T22:32:28.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.809+0000 7fdc8f9ef880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T22:32:28.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.809+0000 7fdc8f9ef880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T22:32:28.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.809+0000 7fdc8f9ef880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T22:32:28.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.809+0000 7fdc8f9ef880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T22:32:28.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:28.813+0000 7fdc8f9ef880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T22:32:31.800 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T22:32:31.817 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:31.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T22:32:31.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T22:32:31.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T22:32:31.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T22:32:31.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T22:32:31.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:31.857+0000 7f057e819880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T22:32:34.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T22:32:34.938 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T22:32:34.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T22:32:34.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T22:32:34.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T22:32:34.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T22:32:34.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:34.981+0000 7f0d0b868880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T22:32:37.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T22:32:38.014 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T22:32:38.030 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:38.031 INFO:tasks.ceph:role: osd.4 2024-05-08T22:32:38.031 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:32:38.031 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:38.072 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:38.074 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-05-08T22:32:38.074 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T22:32:38.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T22:32:38.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:38.145 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:38.145 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T22:32:38.155 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:38.156 INFO:tasks.ceph:role: osd.5 2024-05-08T22:32:38.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:32:38.156 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T22:32:38.243 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:38.243 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:38.243 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:38.243 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:38.244 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:38.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-05-08T22:32:38.245 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T22:32:38.260 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T22:32:38.269 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:38.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:38.270 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T22:32:38.281 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:38.281 INFO:tasks.ceph:role: osd.6 2024-05-08T22:32:38.281 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:32:38.281 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:38.367 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:38.369 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-05-08T22:32:38.369 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T22:32:38.389 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T22:32:38.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:38.399 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:38.400 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T22:32:38.410 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T22:32:38.410 INFO:tasks.ceph:role: osd.7 2024-05-08T22:32:38.410 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:32:38.411 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T22:32:38.453 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T22:32:38.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-05-08T22:32:38.454 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T22:32:38.517 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T22:32:38.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:32:38.529 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T22:32:38.529 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T22:32:38.607 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:38.603+0000 7f17bbe5f880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T22:32:41.600 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T22:32:41.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:41.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T22:32:41.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T22:32:41.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T22:32:41.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T22:32:41.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T22:32:41.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:41.703+0000 7fbafb859880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T22:32:44.699 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T22:32:44.717 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T22:32:44.763 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:44.758+0000 7f72feff5880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T22:32:47.775 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T22:32:47.788 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T22:32:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T22:32:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T22:32:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T22:32:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T22:32:47.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:47.874+0000 7f52b0823880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T22:32:50.855 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T22:32:50.871 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T22:32:50.872 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:50.872 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T22:32:50.888 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:50.888 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T22:32:50.945 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:50.945 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T22:32:50.954 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:50.954 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T22:32:50.968 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:50.968 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T22:32:51.022 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:51.022 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T22:32:51.036 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:51.036 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T22:32:51.050 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.050 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T22:32:51.067 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.067 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T22:32:51.124 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.124 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T22:32:51.135 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T22:32:51.191 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.192 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T22:32:51.201 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.201 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T22:32:51.217 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.217 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T22:32:51.279 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:51.279 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T22:32:51.287 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:51.287 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T22:32:51.294 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.294 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T22:32:51.301 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:51.301 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T22:32:51.307 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T22:32:51.308 DEBUG:teuthology.orchestra.run.smithi179:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T22:32:51.310 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.y] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.0] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.1] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.2] 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww== 2024-05-08T22:32:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.3] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.x] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.4] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.5] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.6] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q== 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.7] 2024-05-08T22:32:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ== 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout:[client.0] 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA== 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout:[client.2] 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA== 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q== 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout:[client.3] 2024-05-08T22:32:51.322 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw== 2024-05-08T22:32:51.323 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-05-08T22:32:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw== 2024-05-08T22:32:51.323 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:32:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA== 2024-05-08T22:32:51.324 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[client.2] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA== 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2024-05-08T22:32:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q== 2024-05-08T22:32:51.326 INFO:teuthology.orchestra.run.smithi195.stdout:[client.3] 2024-05-08T22:32:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw== 2024-05-08T22:32:51.326 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T22:32:51.334 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:32:51.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T22:32:51.368 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:32:51.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.420 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.469 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.471 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.567 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.569 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.618 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.669 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.671 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.719 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.767 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T22:32:51.769 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T22:32:51.816 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.819 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.871 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.873 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T22:32:51.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.927 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:51.977 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.010 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.063 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.114 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.116 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T22:32:52.165 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.220 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.273 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.276 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.324 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.326 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T22:32:52.378 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T22:32:52.378 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T22:32:52.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T22:32:52.531 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T22:32:52.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T22:32:52.564 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T22:32:52.718 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T22:32:52.735 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T22:32:52.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T22:32:52.854 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T22:32:52.869 DEBUG:teuthology.orchestra.run.smithi179:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:52.871 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T22:32:52.876 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T22:32:52.876 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T22:32:52.876 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T22:32:52.878 INFO:tasks.ceph.mon.a:Started 2024-05-08T22:32:52.878 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T22:32:52.878 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T22:32:52.880 INFO:tasks.ceph.mon.b:Started 2024-05-08T22:32:52.880 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T22:32:52.881 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T22:32:52.883 INFO:tasks.ceph.mon.c:Started 2024-05-08T22:32:52.883 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T22:32:52.883 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T22:32:52.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T22:32:52.885 INFO:tasks.ceph.mgr.y:Started 2024-05-08T22:32:52.885 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T22:32:52.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T22:32:52.888 INFO:tasks.ceph.mgr.x:Started 2024-05-08T22:32:52.888 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T22:32:52.888 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T22:32:53.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 Processor -- start 2024-05-08T22:32:53.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 -- start start 2024-05-08T22:32:53.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf840cbd30 con 0x7fcf84143b60 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf84071de0 con 0x7fcf841441c0 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf890a3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8406c570 con 0x7fcf8406bbc0 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56788/0 (socket says 172.21.15.179:56788) 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 learned_addr learned my addr 172.21.15.179:0/1956849904 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf83fff700 1 --1- 172.21.15.179:0/1956849904 >> v1:172.21.15.179:6789/0 conn(0x7fcf84143b60 0x7fcf840cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58058/0 (socket says 172.21.15.179:58058) 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:53.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.012+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T22:32:53.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.016+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.016+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.016+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:53.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.216+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.216+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:53.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.216+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.216+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.216+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:32:53.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.220+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.220+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:53.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.220+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.220+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.220+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:32:53.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.620+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.620+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:53.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.620+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.620+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.620+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:32:53.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.624+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:53.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.624+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:53.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.624+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:53.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.624+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:53.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:53.624+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:32:54.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.424+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:54.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.424+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:54.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.424+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:54.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.424+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:54.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.424+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:32:54.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.428+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:54.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.428+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:54.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.428+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:54.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.428+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:54.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:54.428+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:32:56.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 msgr2=0x7fcf8406c030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:56.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf8406bbc0 0x7fcf8406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:56.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 msgr2=0x7fcf840718a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:56.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.179:3300/0 conn(0x7fcf841441c0 0x7fcf840718a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:56.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 -- 172.21.15.179:0/1956849904 >> v1:172.21.15.179:6789/0 conn(0x7fcf84143b60 legacy=0x7fcf840cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 -- 172.21.15.179:0/1956849904 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf84143f40 con 0x7fcf700061f0 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.012+0000 7fcf81ffb700 1 -- 172.21.15.179:0/1956849904 --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf84071de0 con 0x7fcf70003720 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.016+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.016+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.016+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.016+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:56.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.016+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:56.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.220+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:56.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.220+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T22:32:56.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.220+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:56.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.220+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:56.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.220+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:32:56.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.624+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:56.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.624+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T22:32:56.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.624+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:56.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.624+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:56.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:56.624+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:32:57.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:57.428+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:57.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:57.428+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T22:32:57.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:57.428+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:57.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:57.428+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:57.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:57.428+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:32:58.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773459064 0 0) 0x7fcf84143f40 con 0x7fcf700061f0 2024-05-08T22:32:58.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf6c0035d0 con 0x7fcf700061f0 2024-05-08T22:32:58.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.1 v1:172.21.15.195:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021402124 0 0) 0x7fcf6c0035d0 con 0x7fcf700061f0 2024-05-08T22:32:58.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 msgr2=0x7fcf70005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:58.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 --2- 172.21.15.179:0/1956849904 >> v2:172.21.15.195:3300/0 conn(0x7fcf70003720 0x7fcf70005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 --> v1:172.21.15.195:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7000af80 con 0x7fcf700061f0 2024-05-08T22:32:58.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.1 v1:172.21.15.195:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fcf78009510 con 0x7fcf700061f0 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 >> v1:172.21.15.195:6789/0 conn(0x7fcf700061f0 legacy=0x7fcf70008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf888a2700 1 -- 172.21.15.179:0/1956849904 reap_dead start 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 0x7fcf6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf6c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84071de0 con 0x7fcf6c003a20 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.076+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 0x7fcf6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c00c2a0 con 0x7fcf6c005350 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c00c460 con 0x7fcf6c004870 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf6c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 msgr2=0x7fcf6c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:58.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf837fe700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 msgr2=0x7fcf6c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 msgr2=0x7fcf6c004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 msgr2=0x7fcf6c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf6c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 -- 172.21.15.179:0/1956849904 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0035d0 con 0x7fcf6c003a20 2024-05-08T22:32:58.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.080+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 0x7fcf6c003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf74007d10 tx=0x7fcf6c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680797397495d613 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017460 con 0x7fcf6c003a20 2024-05-08T22:32:58.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740175e0 con 0x7fcf6c003a20 2024-05-08T22:32:58.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf82ffd700 1 -- 172.21.15.179:0/1956849904 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017990 con 0x7fcf6c003a20 2024-05-08T22:32:58.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 -- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 msgr2=0x7fcf6c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 0x7fcf6c003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf74007d10 tx=0x7fcf6c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 -- 172.21.15.179:0/1956849904 shutdown_connections 2024-05-08T22:32:58.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf6c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c004870 0x7fcf6c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1956849904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c003a20 0x7fcf6c003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 -- 172.21.15.179:0/1956849904 >> 172.21.15.179:0/1956849904 conn(0x7fcf84070d80 msgr2=0x7fcf84071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 -- 172.21.15.179:0/1956849904 shutdown_connections 2024-05-08T22:32:58.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.084+0000 7fcf890a3700 1 -- 172.21.15.179:0/1956849904 wait complete. 2024-05-08T22:32:58.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.088+0000 7fcf890a3700 1 Processor -- start 2024-05-08T22:32:58.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.088+0000 7fcf890a3700 1 -- start start 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf8406a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8406cde0 con 0x7fcf6c004870 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8415e1f0 con 0x7fcf6c005350 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8406d0a0 con 0x7fcf6c003a20 2024-05-08T22:32:58.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:49250/0 (socket says 172.21.15.179:49250) 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53022/0 (socket says 172.21.15.179:53022) 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 -- 172.21.15.179:0/1477531452 learned_addr learned my addr 172.21.15.179:0/1477531452 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf888a2700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf8406a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 -- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 msgr2=0x7fcf8413a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 -- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 msgr2=0x7fcf8413a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf83fff700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 -- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 msgr2=0x7fcf8413a3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:58.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 -- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 msgr2=0x7fcf8406a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf8406a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 -- 172.21.15.179:0/1477531452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8406d380 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf837fe700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf78015c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78004230 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7800c040 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8406d660 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78009540 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8415d480 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcf78013a40 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcf78000e40 con 0x7fcf6c004870 2024-05-08T22:32:58.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.092+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8404e700 con 0x7fcf6c004870 2024-05-08T22:32:58.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.096+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fcf78024020 con 0x7fcf6c004870 2024-05-08T22:32:58.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.168+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcf84144630 con 0x7fcf6c004870 2024-05-08T22:32:58.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.172+0000 7fcf80ff9700 1 -- 172.21.15.179:0/1477531452 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcf7801b070 con 0x7fcf6c004870 2024-05-08T22:32:58.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 msgr2=0x7fcf8406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf78015c60 comp rx=0 tx=0).stop 2024-05-08T22:32:58.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 shutdown_connections 2024-05-08T22:32:58.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c005350 0x7fcf8406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf6c004870 0x7fcf8406a320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 --2- 172.21.15.179:0/1477531452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c003a20 0x7fcf8413a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 >> 172.21.15.179:0/1477531452 conn(0x7fcf84070d80 msgr2=0x7fcf840ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 shutdown_connections 2024-05-08T22:32:58.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:32:58.176+0000 7fcf890a3700 1 -- 172.21.15.179:0/1477531452 wait complete. 2024-05-08T22:32:58.180 INFO:teuthology.orchestra.run.smithi179.stderr:adjusted tunables profile to default 2024-05-08T22:32:58.194 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T22:32:58.194 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T22:32:58.194 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:58.195 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T22:32:58.204 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:58.204 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T22:32:58.213 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:58.213 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T22:32:58.222 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:32:58.222 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T22:32:58.235 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:58.236 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T22:32:58.251 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:58.251 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T22:32:58.282 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:58.278+0000 7f7ac116f2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:32:58.288 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:58.284+0000 7f60792c52c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:32:58.295 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:58.295 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T22:32:58.349 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:32:58.349 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T22:32:58.358 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 93a60be3-29ef-4198-a764-e318fbff88fc 0 2024-05-08T22:32:58.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 Processor -- start 2024-05-08T22:32:58.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 -- start start 2024-05-08T22:32:58.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc50c0cf860 0x7fc50c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c0b6580 con 0x7fc50c15ea60 2024-05-08T22:32:58.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c0b4e30 con 0x7fc50c0cf860 2024-05-08T22:32:58.472 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc5125bc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0b6860 con 0x7fc50c15eef0 2024-05-08T22:32:58.473 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.466+0000 7fc511dbb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc50c0cf860 0x7fc50c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44282/0 (socket says 172.21.15.195:44282) 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2769206610 >> v2:172.21.15.179:3300/0 conn(0x7fc50c0cf860 0x7fc50c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.195:56594/0 (socket says 172.21.15.195:56594) 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 -- 172.21.15.195:0/2769206610 learned_addr learned my addr 172.21.15.195:0/2769206610 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 -- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 -- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc511dbb700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc503fff700 1 -- 172.21.15.195:0/2769206610 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756328042 0 0) 0x7fc50c0b6580 con 0x7fc50c15ea60 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc503fff700 1 -- 172.21.15.195:0/2769206610 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f80035d0 con 0x7fc50c15ea60 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 -- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c161180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:58.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 -- 172.21.15.195:0/2769206610 >> v1:172.21.15.179:6789/0 conn(0x7fc50c15ea60 legacy=0x7fc50c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 -- 172.21.15.195:0/2769206610 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0b6b40 con 0x7fc50c0cf860 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0cf860 0x7fc50c0b45e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc504009d90 tx=0x7fc50400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ca53795f350fb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc503fff700 1 -- 172.21.15.195:0/2769206610 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400e070 con 0x7fc50c0cf860 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc503fff700 1 -- 172.21.15.195:0/2769206610 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50400bb20 con 0x7fc50c0cf860 2024-05-08T22:32:58.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc503fff700 1 -- 172.21.15.195:0/2769206610 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504012430 con 0x7fc50c0cf860 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- 172.21.15.195:0/2769206610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0cf860 msgr2=0x7fc50c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0cf860 0x7fc50c0b45e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc504009d90 tx=0x7fc50400b330 comp rx=0 tx=0).stop 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- 172.21.15.195:0/2769206610 shutdown_connections 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2769206610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0cf860 0x7fc50c0b45e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- 172.21.15.195:0/2769206610 >> 172.21.15.195:0/2769206610 conn(0x7fc50c0b2770 msgr2=0x7fc50c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- 172.21.15.195:0/2769206610 shutdown_connections 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- 172.21.15.195:0/2769206610 wait complete. 2024-05-08T22:32:58.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 Processor -- start 2024-05-08T22:32:58.477 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.470+0000 7fc5125bc700 1 -- start start 2024-05-08T22:32:58.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 0x7fc50c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15ea60 0x7fc50c14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0c5d00 con 0x7fc50c15ea60 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c14fc00 con 0x7fc50c0cf860 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0c47b0 con 0x7fc50c15eef0 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44296/0 (socket says 172.21.15.195:44296) 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 0x7fc50c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 -- 172.21.15.195:0/2505355090 learned_addr learned my addr 172.21.15.195:0/2505355090 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc510db9700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15ea60 0x7fc50c14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 -- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 -- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc511dbb700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 -- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 msgr2=0x7fc50c0b16f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.480 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 -- 172.21.15.195:0/2505355090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15ea60 msgr2=0x7fc50c14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15ea60 0x7fc50c14f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 -- 172.21.15.195:0/2505355090 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc504009a70 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5115ba700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 0x7fc50c0c57c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc504016000 tx=0x7fc504004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504012430 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504020070 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c0c4a00 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504004280 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c0c4fc0 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc5040046e0 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc504022070 con 0x7fc50c0cf860 2024-05-08T22:32:58.481 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.474+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c070af0 con 0x7fc50c0cf860 2024-05-08T22:32:58.482 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.478+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc504027480 con 0x7fc50c0cf860 2024-05-08T22:32:58.557 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.554+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "93a60be3-29ef-4198-a764-e318fbff88fc", "id": 0} v 0) v1 -- 0x7fc50c04e700 con 0x7fc50c0cf860 2024-05-08T22:32:58.563 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.558+0000 7fc501ffb700 1 -- 172.21.15.195:0/2505355090 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93a60be3-29ef-4198-a764-e318fbff88fc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc5040278e0 con 0x7fc50c0cf860 2024-05-08T22:32:58.563 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-05-08T22:32:58.564 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 msgr2=0x7fc50c0c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.564 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 0x7fc50c0c57c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc504016000 tx=0x7fc504004af0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.564 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 shutdown_connections 2024-05-08T22:32:58.564 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15eef0 0x7fc50c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.565 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15ea60 0x7fc50c14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.565 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 --2- 172.21.15.195:0/2505355090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c0cf860 0x7fc50c0c57c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.565 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 >> 172.21.15.195:0/2505355090 conn(0x7fc50c0b2770 msgr2=0x7fc50c0cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.565 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 shutdown_connections 2024-05-08T22:32:58.565 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.562+0000 7fc5125bc700 1 -- 172.21.15.195:0/2505355090 wait complete. 2024-05-08T22:32:58.578 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f20a6991-00e2-4a3f-9d0f-15db51c8e53d 1 2024-05-08T22:32:58.693 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 Processor -- start 2024-05-08T22:32:58.693 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- start start 2024-05-08T22:32:58.694 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17a815b0c0 0x7f17a80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17a806ef90 0x7f17a80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a80cfc00 con 0x7f17a80b6240 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80cfee0 con 0x7f17a806ef90 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80d01c0 con 0x7f17a815b0c0 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17a815b0c0 0x7f17a80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17a815b0c0 0x7f17a80cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33030/0 (socket says 172.21.15.195:33030) 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 -- 172.21.15.195:0/1541507947 learned_addr learned my addr 172.21.15.195:0/1541507947 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17ad757700 1 --2- 172.21.15.195:0/1541507947 >> v2:172.21.15.179:3300/0 conn(0x7f17a806ef90 0x7f17a80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 -- 172.21.15.195:0/1541507947 >> v2:172.21.15.179:3300/0 conn(0x7f17a806ef90 msgr2=0x7f17a80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 --2- 172.21.15.195:0/1541507947 >> v2:172.21.15.179:3300/0 conn(0x7f17a806ef90 0x7f17a80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 -- 172.21.15.195:0/1541507947 >> v1:172.21.15.179:6789/0 conn(0x7f17a80b6240 legacy=0x7f17a80b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.695 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 -- 172.21.15.195:0/1541507947 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80d04a0 con 0x7f17a815b0c0 2024-05-08T22:32:58.696 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a7fff700 1 --2- 172.21.15.195:0/1541507947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80cf5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17a80aff30 tx=0x7f179c009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee47ede7af93b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.696 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a77fe700 1 -- 172.21.15.195:0/1541507947 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c00b8a0 con 0x7f17a815b0c0 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a77fe700 1 -- 172.21.15.195:0/1541507947 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c00ba00 con 0x7f17a815b0c0 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17a77fe700 1 -- 172.21.15.195:0/1541507947 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c00bd90 con 0x7f17a815b0c0 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- 172.21.15.195:0/1541507947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 msgr2=0x7f17a80cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 --2- 172.21.15.195:0/1541507947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80cf5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17a80aff30 tx=0x7f179c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- 172.21.15.195:0/1541507947 shutdown_connections 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 --2- 172.21.15.195:0/1541507947 >> v2:172.21.15.179:3300/0 conn(0x7f17a806ef90 0x7f17a80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 --2- 172.21.15.195:0/1541507947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80cf5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.690+0000 7f17adf58700 1 -- 172.21.15.195:0/1541507947 >> 172.21.15.195:0/1541507947 conn(0x7f17a80d0ca0 msgr2=0x7f17a80d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- 172.21.15.195:0/1541507947 shutdown_connections 2024-05-08T22:32:58.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- 172.21.15.195:0/1541507947 wait complete. 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 Processor -- start 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- start start 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 0x7f17a815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80a7820 con 0x7f17a806ef90 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8148060 con 0x7f17a815b0c0 2024-05-08T22:32:58.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81479e0 con 0x7f17a80b6240 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44300/0 (socket says 172.21.15.195:44300) 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 -- 172.21.15.195:0/2964954854 learned_addr learned my addr 172.21.15.195:0/2964954854 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17acf56700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 0x7f17a815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 -- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 msgr2=0x7f17a8147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 -- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 msgr2=0x7f17a8147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a7fff700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 -- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 msgr2=0x7f17a8147090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 -- 172.21.15.195:0/2964954854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 msgr2=0x7f17a815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 0x7f17a815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 -- 172.21.15.195:0/2964954854 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c009570 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17acf56700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 0x7f17a815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17ad757700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80c8dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f17a0007d90 tx=0x7f17a000e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0018070 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a000ebe0 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a80a7a00 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0003a40 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81475d0 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f17a0014720 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.694+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f17a001d870 con 0x7f17a815b0c0 2024-05-08T22:32:58.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.698+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a81481a0 con 0x7f17a815b0c0 2024-05-08T22:32:58.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.698+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f17a001d410 con 0x7f17a815b0c0 2024-05-08T22:32:58.779 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.774+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f20a6991-00e2-4a3f-9d0f-15db51c8e53d", "id": 1} v 0) v1 -- 0x7f17a804e700 con 0x7f17a815b0c0 2024-05-08T22:32:58.786 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17a57fa700 1 -- 172.21.15.195:0/2964954854 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f20a6991-00e2-4a3f-9d0f-15db51c8e53d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f17a0014b60 con 0x7f17a815b0c0 2024-05-08T22:32:58.786 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 msgr2=0x7f17a80c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80c8dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f17a0007d90 tx=0x7f17a000e3c0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 shutdown_connections 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17a815b0c0 0x7f17a80c8dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17a80b6240 0x7f17a8147090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 --2- 172.21.15.195:0/2964954854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17a806ef90 0x7f17a815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 >> 172.21.15.195:0/2964954854 conn(0x7f17a80d0ca0 msgr2=0x7f17a806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 shutdown_connections 2024-05-08T22:32:58.788 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.782+0000 7f17adf58700 1 -- 172.21.15.195:0/2964954854 wait complete. 2024-05-08T22:32:58.800 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new a137b814-8dec-4873-b5f0-eb2a7ed72087 2 2024-05-08T22:32:58.915 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.910+0000 7faaf6798700 1 Processor -- start 2024-05-08T22:32:58.915 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.910+0000 7faaf6798700 1 -- start start 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 0x7faaf00d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faaf00d1190 0x7faaf00b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf00d16c0 con 0x7faaf00cdb20 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00d2ca0 con 0x7faaf01605e0 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00d19a0 con 0x7faaf00d1190 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faaf00d1190 0x7faaf00b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faaf00d1190 0x7faaf00b87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33060/0 (socket says 172.21.15.195:33060) 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 -- 172.21.15.195:0/1106766049 learned_addr learned my addr 172.21.15.195:0/1106766049 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf5796700 1 --2- 172.21.15.195:0/1106766049 >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 0x7faaf00d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 -- 172.21.15.195:0/1106766049 >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 msgr2=0x7faaf00d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 --2- 172.21.15.195:0/1106766049 >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 0x7faaf00d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.918 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 -- 172.21.15.195:0/1106766049 >> v1:172.21.15.179:6789/0 conn(0x7faaf00cdb20 legacy=0x7faaf00b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 -- 172.21.15.195:0/1106766049 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00d1c80 con 0x7faaf00d1190 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf5796700 1 --2- 172.21.15.195:0/1106766049 >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 0x7faaf00d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf4f95700 1 --2- 172.21.15.195:0/1106766049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00d1190 0x7faaf00b87b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faaf008b2d0 tx=0x7faaec009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b6828af3919874 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faae7fff700 1 -- 172.21.15.195:0/1106766049 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00b8a0 con 0x7faaf00d1190 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faae7fff700 1 -- 172.21.15.195:0/1106766049 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec00ba00 con 0x7faaf00d1190 2024-05-08T22:32:58.919 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faae7fff700 1 -- 172.21.15.195:0/1106766049 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00bd90 con 0x7faaf00d1190 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- 172.21.15.195:0/1106766049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00d1190 msgr2=0x7faaf00b87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 --2- 172.21.15.195:0/1106766049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00d1190 0x7faaf00b87b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faaf008b2d0 tx=0x7faaec009bf0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- 172.21.15.195:0/1106766049 shutdown_connections 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 --2- 172.21.15.195:0/1106766049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00d1190 0x7faaf00b87b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 --2- 172.21.15.195:0/1106766049 >> v2:172.21.15.179:3300/0 conn(0x7faaf01605e0 0x7faaf00d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- 172.21.15.195:0/1106766049 >> 172.21.15.195:0/1106766049 conn(0x7faaf00b4940 msgr2=0x7faaf00b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- 172.21.15.195:0/1106766049 shutdown_connections 2024-05-08T22:32:58.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.914+0000 7faaf6798700 1 -- 172.21.15.195:0/1106766049 wait complete. 2024-05-08T22:32:58.921 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 Processor -- start 2024-05-08T22:32:58.921 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 -- start start 2024-05-08T22:32:58.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 0x7faaf0154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaf01605e0 0x7faaf015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00ccfd0 con 0x7faaf01605e0 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00cccf0 con 0x7faaf00cdb20 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf6798700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf015b820 con 0x7faaf00d1190 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 0x7faaf0154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44302/0 (socket says 172.21.15.195:44302) 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 -- 172.21.15.195:0/4141572958 learned_addr learned my addr 172.21.15.195:0/4141572958 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 -- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 msgr2=0x7faaf0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 -- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 msgr2=0x7faaf0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:58.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5f97700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaf01605e0 0x7faaf015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf4f95700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 -- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 msgr2=0x7faaf0159040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 -- 172.21.15.195:0/4141572958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaf01605e0 msgr2=0x7faaf015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaf01605e0 0x7faaf015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 -- 172.21.15.195:0/4141572958 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009570 con 0x7faaf00cdb20 2024-05-08T22:32:58.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.918+0000 7faaf5796700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 0x7faaf0154830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae000e810 tx=0x7faae000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:58.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0004230 con 0x7faaf00cdb20 2024-05-08T22:32:58.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae0004390 con 0x7faaf00cdb20 2024-05-08T22:32:58.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf015bb00 con 0x7faaf00cdb20 2024-05-08T22:32:58.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae00036c0 con 0x7faaf00cdb20 2024-05-08T22:32:58.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf00cc9c0 con 0x7faaf00cdb20 2024-05-08T22:32:58.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7faae0003b40 con 0x7faaf00cdb20 2024-05-08T22:32:58.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7faae0003df0 con 0x7faaf00cdb20 2024-05-08T22:32:58.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.922+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf004e700 con 0x7faaf00cdb20 2024-05-08T22:32:58.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.926+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7faae0020060 con 0x7faaf00cdb20 2024-05-08T22:32:58.929 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:58.928+0000 7f60792c52c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:32:58.961 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:58.958+0000 7f7ac116f2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:32:58.988 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:58.984+0000 7f60792c52c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:32:59.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:58.998+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a137b814-8dec-4873-b5f0-eb2a7ed72087", "id": 2} v 0) v1 -- 0x7faaf00b4400 con 0x7faaf00cdb20 2024-05-08T22:32:59.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.002+0000 7faae5ffb700 1 -- 172.21.15.195:0/4141572958 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a137b814-8dec-4873-b5f0-eb2a7ed72087", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7faae000f040 con 0x7faaf00cdb20 2024-05-08T22:32:59.007 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 msgr2=0x7faaf0154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 0x7faaf0154830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae000e810 tx=0x7faae000eb20 comp rx=0 tx=0).stop 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 shutdown_connections 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaf01605e0 0x7faaf015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaf00d1190 0x7faaf0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 --2- 172.21.15.195:0/4141572958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaf00cdb20 0x7faaf0154830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 >> 172.21.15.195:0/4141572958 conn(0x7faaf00b4940 msgr2=0x7faaf00b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 shutdown_connections 2024-05-08T22:32:59.010 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.006+0000 7faaf6798700 1 -- 172.21.15.195:0/4141572958 wait complete. 2024-05-08T22:32:59.020 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.018+0000 7f7ac116f2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:32:59.022 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 1ff8732a-d46d-4a7b-8c83-72a3604c7d15 3 2024-05-08T22:32:59.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.130+0000 7fa68c36a700 1 Processor -- start 2024-05-08T22:32:59.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.130+0000 7fa68c36a700 1 -- start start 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa6840b51b0 0x7fa6840cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6840cd1e0 con 0x7fa6840b4880 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6840cd4c0 con 0x7fa68415d510 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840cde60 con 0x7fa6840b51b0 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a106700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa68415d510 0x7fa6840b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:58744/0 (socket says 172.21.15.195:58744) 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a106700 1 -- 172.21.15.195:0/383479277 learned_addr learned my addr 172.21.15.195:0/383479277 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a907700 1 --2- 172.21.15.195:0/383479277 >> v2:172.21.15.195:3300/0 conn(0x7fa6840b51b0 0x7fa6840cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa689104700 1 -- 172.21.15.195:0/383479277 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522407252 0 0) 0x7fa6840cd4c0 con 0x7fa68415d510 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa689104700 1 -- 172.21.15.195:0/383479277 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6700035d0 con 0x7fa68415d510 2024-05-08T22:32:59.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a907700 1 -- 172.21.15.195:0/383479277 >> v1:172.21.15.195:6789/0 conn(0x7fa68415d510 legacy=0x7fa6840b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a907700 1 -- 172.21.15.195:0/383479277 >> v1:172.21.15.179:6789/0 conn(0x7fa6840b4880 legacy=0x7fa68406acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a907700 1 -- 172.21.15.195:0/383479277 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6840cd7a0 con 0x7fa6840b51b0 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68a907700 1 --2- 172.21.15.195:0/383479277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6840b51b0 0x7fa6840cb4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009dc0 tx=0x7fa67c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb6cf7dcd07d25b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa689104700 1 -- 172.21.15.195:0/383479277 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00e070 con 0x7fa6840b51b0 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa689104700 1 -- 172.21.15.195:0/383479277 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c00bae0 con 0x7fa6840b51b0 2024-05-08T22:32:59.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa689104700 1 -- 172.21.15.195:0/383479277 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00be90 con 0x7fa6840b51b0 2024-05-08T22:32:59.141 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- 172.21.15.195:0/383479277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6840b51b0 msgr2=0x7fa6840cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.141 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 --2- 172.21.15.195:0/383479277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6840b51b0 0x7fa6840cb4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009dc0 tx=0x7fa67c00b330 comp rx=0 tx=0).stop 2024-05-08T22:32:59.141 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- 172.21.15.195:0/383479277 shutdown_connections 2024-05-08T22:32:59.141 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 --2- 172.21.15.195:0/383479277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6840b51b0 0x7fa6840cb4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.141 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- 172.21.15.195:0/383479277 >> 172.21.15.195:0/383479277 conn(0x7fa684098a70 msgr2=0x7fa6840b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- 172.21.15.195:0/383479277 shutdown_connections 2024-05-08T22:32:59.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.134+0000 7fa68c36a700 1 -- 172.21.15.195:0/383479277 wait complete. 2024-05-08T22:32:59.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 Processor -- start 2024-05-08T22:32:59.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- start start 2024-05-08T22:32:59.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 0x7fa6841538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 0x7fa6840cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684153e00 con 0x7fa6840b4880 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68409d4a0 con 0x7fa68415d510 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68409c890 con 0x7fa6840b51b0 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a907700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 0x7fa6840cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44304/0 (socket says 172.21.15.195:44304) 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 -- 172.21.15.195:0/1782055390 learned_addr learned my addr 172.21.15.195:0/1782055390 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 0x7fa6841538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 -- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 msgr2=0x7fa684099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 -- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 msgr2=0x7fa684099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:59.145 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa689905700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 -- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 msgr2=0x7fa684099d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 -- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 msgr2=0x7fa6840cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 0x7fa6840cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 -- 172.21.15.195:0/1782055390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009a70 con 0x7fa6840b4880 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a907700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 0x7fa6840cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68a106700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 0x7fa6841538c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c410 tx=0x7fa68000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680005d90 con 0x7fa6840b4880 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6800070d0 con 0x7fa6840b4880 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68409cb70 con 0x7fa6840b4880 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680007440 con 0x7fa6840b4880 2024-05-08T22:32:59.146 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68409d130 con 0x7fa6840b4880 2024-05-08T22:32:59.147 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa68001c440 con 0x7fa6840b4880 2024-05-08T22:32:59.147 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.138+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa68001cb80 con 0x7fa6840b4880 2024-05-08T22:32:59.147 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.142+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6840990b0 con 0x7fa6840b4880 2024-05-08T22:32:59.150 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.146+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa680017070 con 0x7fa6840b4880 2024-05-08T22:32:59.162 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.160+0000 7f60792c52c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:32:59.198 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.194+0000 7f7ac116f2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:32:59.232 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.230+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1ff8732a-d46d-4a7b-8c83-72a3604c7d15", "id": 3} v 0) v1 -- 0x7fa684073290 con 0x7fa6840b4880 2024-05-08T22:32:59.238 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa67affd700 1 -- 172.21.15.195:0/1782055390 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ff8732a-d46d-4a7b-8c83-72a3604c7d15", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa68001c6f0 con 0x7fa6840b4880 2024-05-08T22:32:59.238 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-05-08T22:32:59.239 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 msgr2=0x7fa6841538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 0x7fa6841538c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c410 tx=0x7fa68000c720 comp rx=0 tx=0).stop 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 shutdown_connections 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa68415d510 0x7fa6840cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6840b51b0 0x7fa684099d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 --2- 172.21.15.195:0/1782055390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6840b4880 0x7fa6841538c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 >> 172.21.15.195:0/1782055390 conn(0x7fa684098a70 msgr2=0x7fa68406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 shutdown_connections 2024-05-08T22:32:59.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.234+0000 7fa68c36a700 1 -- 172.21.15.195:0/1782055390 wait complete. 2024-05-08T22:32:59.250 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new b3e3bccc-a4dc-45c5-bed7-704ec27b247b 4 2024-05-08T22:32:59.356 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 Processor -- start 2024-05-08T22:32:59.357 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- start start 2024-05-08T22:32:59.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4e90143830 0x7f4e900b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e900b4630 con 0x7f4e900b39a0 2024-05-08T22:32:59.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e900b4bf0 con 0x7f4e90143830 2024-05-08T22:32:59.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900b4910 con 0x7f4e900ce810 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4e90143830 0x7f4e900b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e957ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4e90143830 0x7f4e900b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33098/0 (socket says 172.21.15.195:33098) 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 -- 172.21.15.195:0/1578399552 learned_addr learned my addr 172.21.15.195:0/1578399552 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e957ca700 1 -- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e900b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e957ca700 1 -- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e900b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e957ca700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e957ca700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 -- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e900b3460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 -- 172.21.15.195:0/1578399552 >> v1:172.21.15.179:6789/0 conn(0x7f4e900b39a0 legacy=0x7f4e9006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 -- 172.21.15.195:0/1578399552 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900b2c60 con 0x7f4e90143830 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e95fcb700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e90143830 0x7f4e900b3f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009910 tx=0x7f4e8c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82c2c043f4b06fae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e94fc9700 1 -- 172.21.15.195:0/1578399552 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004230 con 0x7f4e90143830 2024-05-08T22:32:59.361 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e94fc9700 1 -- 172.21.15.195:0/1578399552 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c004390 con 0x7f4e90143830 2024-05-08T22:32:59.361 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e94fc9700 1 -- 172.21.15.195:0/1578399552 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c009e60 con 0x7f4e90143830 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e90143830 msgr2=0x7f4e900b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e90143830 0x7f4e900b3f50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009910 tx=0x7f4e8c009c20 comp rx=0 tx=0).stop 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- 172.21.15.195:0/1578399552 shutdown_connections 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e900b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/1578399552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e90143830 0x7f4e900b3f50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- 172.21.15.195:0/1578399552 >> 172.21.15.195:0/1578399552 conn(0x7f4e900cfbb0 msgr2=0x7f4e900cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- 172.21.15.195:0/1578399552 shutdown_connections 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.354+0000 7f4e96fcd700 1 -- 172.21.15.195:0/1578399552 wait complete. 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 Processor -- start 2024-05-08T22:32:59.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 -- start start 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 0x7f4e901568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e90143830 0x7f4e901586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9013e8d0 con 0x7f4e90143830 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015b9c0 con 0x7f4e900b39a0 2024-05-08T22:32:59.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e96fcd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e90156e10 con 0x7f4e900ce810 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 0x7f4e901568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44326/0 (socket says 172.21.15.195:44326) 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 -- 172.21.15.195:0/3666562628 learned_addr learned my addr 172.21.15.195:0/3666562628 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 -- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e9013c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 -- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e9013c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e957ca700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e967cc700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e90143830 0x7f4e901586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 -- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 msgr2=0x7f4e9013c370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 -- 172.21.15.195:0/3666562628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e90143830 msgr2=0x7f4e901586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e90143830 0x7f4e901586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.358+0000 7f4e95fcb700 1 -- 172.21.15.195:0/3666562628 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c009570 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e95fcb700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 0x7f4e901568d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c015000 tx=0x7f4e8c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00bd00 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c01f070 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c003900 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9013cdf0 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9013d3b0 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4e8c021030 con 0x7f4e900b39a0 2024-05-08T22:32:59.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.362+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f4e8c00be60 con 0x7f4e900b39a0 2024-05-08T22:32:59.368 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.366+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9004e700 con 0x7f4e900b39a0 2024-05-08T22:32:59.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.366+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4e8c01b440 con 0x7f4e900b39a0 2024-05-08T22:32:59.395 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.392+0000 7f60792c52c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:32:59.418 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.414+0000 7f7ac116f2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:32:59.448 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.442+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3e3bccc-a4dc-45c5-bed7-704ec27b247b", "id": 4} v 0) v1 -- 0x7f4e9004e700 con 0x7f4e900b39a0 2024-05-08T22:32:59.450 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.448+0000 7f60792c52c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:32:59.454 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e867fc700 1 -- 172.21.15.195:0/3666562628 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3e3bccc-a4dc-45c5-bed7-704ec27b247b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e8c003d60 con 0x7f4e900b39a0 2024-05-08T22:32:59.454 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 msgr2=0x7f4e901568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 0x7f4e901568d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c015000 tx=0x7f4e8c004770 comp rx=0 tx=0).stop 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 shutdown_connections 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e90143830 0x7f4e901586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e900ce810 0x7f4e9013c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 --2- 172.21.15.195:0/3666562628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e900b39a0 0x7f4e901568d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 >> 172.21.15.195:0/3666562628 conn(0x7f4e900cfbb0 msgr2=0x7f4e900b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 shutdown_connections 2024-05-08T22:32:59.456 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.450+0000 7f4e96fcd700 1 -- 172.21.15.195:0/3666562628 wait complete. 2024-05-08T22:32:59.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 5dcd29a4-3561-422e-852c-c59df7967e5a 5 2024-05-08T22:32:59.470 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.466+0000 7f7ac116f2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:32:59.578 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 Processor -- start 2024-05-08T22:32:59.579 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 -- start start 2024-05-08T22:32:59.581 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f22b413dcd0 0x7f22b406f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22b415b4e0 0x7f22b415b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b4140620 con 0x7f22b413dcd0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b415c260 con 0x7f22b415b4e0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b988a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4140830 con 0x7f22b413e0b0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22b415b4e0 0x7f22b415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22b415b4e0 0x7f22b415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33112/0 (socket says 172.21.15.195:33112) 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 -- 172.21.15.195:0/3023193786 learned_addr learned my addr 172.21.15.195:0/3023193786 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b3fff700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b8888700 1 --2- 172.21.15.195:0/3023193786 >> v2:172.21.15.179:3300/0 conn(0x7f22b413dcd0 0x7f22b406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b3fff700 1 -- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 msgr2=0x7f22b406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:32:59.582 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b3fff700 1 -- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 msgr2=0x7f22b406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b3fff700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 -- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 msgr2=0x7f22b406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b3fff700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 -- 172.21.15.195:0/3023193786 >> v2:172.21.15.179:3300/0 conn(0x7f22b413dcd0 msgr2=0x7f22b406f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 --2- 172.21.15.195:0/3023193786 >> v2:172.21.15.179:3300/0 conn(0x7f22b413dcd0 0x7f22b406f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 -- 172.21.15.195:0/3023193786 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4140b10 con 0x7f22b415b4e0 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b9089700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b415b4e0 0x7f22b415b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f22b406e4d0 tx=0x7f22ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a954c37753e459 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.574+0000 7f22b37fe700 1 -- 172.21.15.195:0/3023193786 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00e070 con 0x7f22b415b4e0 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b37fe700 1 -- 172.21.15.195:0/3023193786 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac00b9c0 con 0x7f22b415b4e0 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b37fe700 1 -- 172.21.15.195:0/3023193786 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00bd70 con 0x7f22b415b4e0 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b415b4e0 msgr2=0x7f22b415b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.583 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b415b4e0 0x7f22b415b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f22b406e4d0 tx=0x7f22ac00b330 comp rx=0 tx=0).stop 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- 172.21.15.195:0/3023193786 shutdown_connections 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b415b4e0 0x7f22b415b930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- 172.21.15.195:0/3023193786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b413e0b0 0x7f22b406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- 172.21.15.195:0/3023193786 >> v2:172.21.15.179:3300/0 conn(0x7f22b413dcd0 0x7f22b406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- 172.21.15.195:0/3023193786 >> 172.21.15.195:0/3023193786 conn(0x7f22b40cfbb0 msgr2=0x7f22b40cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- 172.21.15.195:0/3023193786 shutdown_connections 2024-05-08T22:32:59.584 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- 172.21.15.195:0/3023193786 wait complete. 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 Processor -- start 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- start start 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b413e0b0 0x7f22b415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b406d1c0 con 0x7f22b413e0b0 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.578+0000 7f22b988a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c92d0 con 0x7f22b413dcd0 2024-05-08T22:32:59.586 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b988a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c95b0 con 0x7f22b415b4e0 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33116/0 (socket says 172.21.15.195:33116) 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 -- 172.21.15.195:0/991571301 learned_addr learned my addr 172.21.15.195:0/991571301 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b9089700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b3fff700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b413e0b0 0x7f22b415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b9089700 1 -- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 msgr2=0x7f22b4154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b9089700 1 -- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 msgr2=0x7f22b4154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b9089700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b9089700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 -- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 msgr2=0x7f22b4154ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 -- 172.21.15.195:0/991571301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b413e0b0 msgr2=0x7f22b415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b413e0b0 0x7f22b415d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.587 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 -- 172.21.15.195:0/991571301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac009b30 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b8888700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f22a800d800 tx=0x7f22a800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8014620 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8014780 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b40c9890 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8014b10 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b40c9e50 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f22a801d400 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f22a8012a00 con 0x7f22b413dcd0 2024-05-08T22:32:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.582+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b404e700 con 0x7f22b413dcd0 2024-05-08T22:32:59.589 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.586+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f22a803bdc0 con 0x7f22b413dcd0 2024-05-08T22:32:59.642 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.640+0000 7f60792c52c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:32:59.673 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.670+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5dcd29a4-3561-422e-852c-c59df7967e5a", "id": 5} v 0) v1 -- 0x7f22b4070560 con 0x7f22b413dcd0 2024-05-08T22:32:59.679 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.674+0000 7f7ac116f2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:32:59.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b17fa700 1 -- 172.21.15.195:0/991571301 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5dcd29a4-3561-422e-852c-c59df7967e5a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f22a802a070 con 0x7f22b413dcd0 2024-05-08T22:32:59.683 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 msgr2=0x7f22b413eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f22a800d800 tx=0x7f22a800db10 comp rx=0 tx=0).stop 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 shutdown_connections 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b415b4e0 0x7f22b4154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b413e0b0 0x7f22b415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 --2- 172.21.15.195:0/991571301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b413dcd0 0x7f22b413eea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.678+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 >> 172.21.15.195:0/991571301 conn(0x7f22b40cfbb0 msgr2=0x7f22b40cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.682+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 shutdown_connections 2024-05-08T22:32:59.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.682+0000 7f22b988a700 1 -- 172.21.15.195:0/991571301 wait complete. 2024-05-08T22:32:59.701 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 87009f83-8317-498e-bc33-0cf930ae5b71 6 2024-05-08T22:32:59.708 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.704+0000 7f60792c52c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:32:59.747 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.742+0000 7f7ac116f2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:32:59.772 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:32:59.768+0000 7f60792c52c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:32:59.803 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.798+0000 7fbfdcb4d700 1 Processor -- start 2024-05-08T22:32:59.804 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.798+0000 7fbfdcb4d700 1 -- start start 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbfd80ae720 0x7fbfd80aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbfd80b4170 0x7fbfd80b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd80b5340 con 0x7fbfd80afb00 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd80b4730 con 0x7fbfd80ae720 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd80b4a10 con 0x7fbfd80b4170 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbfd80b4170 0x7fbfd80b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbfd80b4170 0x7fbfd80b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33122/0 (socket says 172.21.15.195:33122) 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 -- 172.21.15.195:0/1465099852 learned_addr learned my addr 172.21.15.195:0/1465099852 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.805 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd659c700 1 --2- 172.21.15.195:0/1465099852 >> v2:172.21.15.179:3300/0 conn(0x7fbfd80ae720 0x7fbfd80aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 -- 172.21.15.195:0/1465099852 >> v2:172.21.15.179:3300/0 conn(0x7fbfd80ae720 msgr2=0x7fbfd80aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 --2- 172.21.15.195:0/1465099852 >> v2:172.21.15.179:3300/0 conn(0x7fbfd80ae720 0x7fbfd80aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 -- 172.21.15.195:0/1465099852 >> v1:172.21.15.179:6789/0 conn(0x7fbfd80afb00 legacy=0x7fbfd80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 -- 172.21.15.195:0/1465099852 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80b4cf0 con 0x7fbfd80b4170 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd5d9b700 1 --2- 172.21.15.195:0/1465099852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80b4170 0x7fbfd80b1b30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80711a0 tx=0x7fbfcc009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81309139328f7da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd559a700 1 -- 172.21.15.195:0/1465099852 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011830 con 0x7fbfd80b4170 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd559a700 1 -- 172.21.15.195:0/1465099852 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc011990 con 0x7fbfd80b4170 2024-05-08T22:32:59.807 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfd559a700 1 -- 172.21.15.195:0/1465099852 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011d20 con 0x7fbfd80b4170 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/1465099852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80b4170 msgr2=0x7fbfd80b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/1465099852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80b4170 0x7fbfd80b1b30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80711a0 tx=0x7fbfcc009bf0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/1465099852 shutdown_connections 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/1465099852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80b4170 0x7fbfd80b1b30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/1465099852 >> v2:172.21.15.179:3300/0 conn(0x7fbfd80ae720 0x7fbfd80aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/1465099852 >> 172.21.15.195:0/1465099852 conn(0x7fbfd8098a70 msgr2=0x7fbfd80b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/1465099852 shutdown_connections 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/1465099852 wait complete. 2024-05-08T22:32:59.808 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 Processor -- start 2024-05-08T22:32:59.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.802+0000 7fbfdcb4d700 1 -- start start 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 0x7fbfd80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfd80afb00 0x7fbfd80c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80c6d30 con 0x7fbfd80afb00 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd8159b30 con 0x7fbfd80ae720 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80c7ec0 con 0x7fbfd80b4170 2024-05-08T22:32:59.811 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 0x7fbfd80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:44346/0 (socket says 172.21.15.195:44346) 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 -- 172.21.15.195:0/4232631706 learned_addr learned my addr 172.21.15.195:0/4232631706 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd5d9b700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfd80afb00 0x7fbfd80c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 -- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 msgr2=0x7fbfd81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 -- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 msgr2=0x7fbfd81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd6d9d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 -- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 msgr2=0x7fbfd81595f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 -- 172.21.15.195:0/4232631706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfd80afb00 msgr2=0x7fbfd80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfd80afb00 0x7fbfd80c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 -- 172.21.15.195:0/4232631706 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009570 con 0x7fbfd80ae720 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfd659c700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 0x7fbfd80c7290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800e870 tx=0x7fbfc800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:59.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8013680 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc80137e0 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd80c7900 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8013b70 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd80c48a0 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbfc8005a40 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbfc80067f0 con 0x7fbfd80ae720 2024-05-08T22:32:59.813 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.806+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd80c4700 con 0x7fbfd80ae720 2024-05-08T22:32:59.815 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.810+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fbfd80c4700 con 0x7fbfd80ae720 2024-05-08T22:32:59.816 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:32:59.814+0000 7f7ac116f2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:32:59.885 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.882+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "87009f83-8317-498e-bc33-0cf930ae5b71", "id": 6} v 0) v1 -- 0x7fbfd8073290 con 0x7fbfd80ae720 2024-05-08T22:32:59.898 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfc6ffd700 1 -- 172.21.15.195:0/4232631706 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87009f83-8317-498e-bc33-0cf930ae5b71", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbfc8029020 con 0x7fbfd80ae720 2024-05-08T22:32:59.898 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-05-08T22:32:59.899 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 msgr2=0x7fbfd80c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 0x7fbfd80c7290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800e870 tx=0x7fbfc800eb80 comp rx=0 tx=0).stop 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 shutdown_connections 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbfd80b4170 0x7fbfd81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfd80afb00 0x7fbfd80c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 --2- 172.21.15.195:0/4232631706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfd80ae720 0x7fbfd80c7290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.890+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 >> 172.21.15.195:0/4232631706 conn(0x7fbfd8098a70 msgr2=0x7fbfd80ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.894+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 shutdown_connections 2024-05-08T22:32:59.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:32:59.894+0000 7fbfdcb4d700 1 -- 172.21.15.195:0/4232631706 wait complete. 2024-05-08T22:32:59.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 35ef7868-ddcf-4c3b-98e8-e018bd51d06c 7 2024-05-08T22:33:00.026 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 Processor -- start 2024-05-08T22:33:00.027 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 -- start start 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb698158b00 0x7fb6980ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69806b4e0 con 0x7fb6980ca890 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6980cad30 con 0x7fb6980cce20 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69d227700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6980cb010 con 0x7fb698158b00 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69759e700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb6980cce20 0x7fb69806bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:58754/0 (socket says 172.21.15.195:58754) 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.022+0000 7fb69759e700 1 -- 172.21.15.195:0/2603767552 learned_addr learned my addr 172.21.15.195:0/2603767552 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 --2- 172.21.15.195:0/2603767552 >> v2:172.21.15.179:3300/0 conn(0x7fb698158b00 0x7fb6980ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268352988 0 0) 0x7fb6980cad30 con 0x7fb6980cce20 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6840035d0 con 0x7fb6980cce20 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.1 v1:172.21.15.195:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1630120331 0 0) 0x7fb6840035d0 con 0x7fb6980cce20 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 >> v2:172.21.15.179:3300/0 conn(0x7fb698158b00 msgr2=0x7fb6980ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:00.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 --2- 172.21.15.195:0/2603767552 >> v2:172.21.15.179:3300/0 conn(0x7fb698158b00 0x7fb6980ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 >> v1:172.21.15.179:6789/0 conn(0x7fb6980ca890 legacy=0x7fb6980cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 --> v1:172.21.15.195:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69806b750 con 0x7fb6980cce20 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.1 v1:172.21.15.195:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fb6900019f0 con 0x7fb6980cce20 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 >> v1:172.21.15.195:6789/0 conn(0x7fb6980cce20 legacy=0x7fb69806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 0x7fb684003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb684004870 0x7fb684004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684005230 0x7fb68400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980cb010 con 0x7fb684004870 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68400c0d0 con 0x7fb684005230 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68400c290 con 0x7fb684003a20 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 0x7fb684003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69759e700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684005230 0x7fb68400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69659c700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb684004870 0x7fb684004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 -- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684005230 msgr2=0x7fb68400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684005230 0x7fb68400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 -- 172.21.15.195:0/2603767552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb684004870 msgr2=0x7fb684004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb684004870 0x7fb684004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 -- 172.21.15.195:0/2603767552 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840035d0 con 0x7fb684003a20 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69759e700 1 -- 172.21.15.195:0/2603767552 reap_dead start 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb696d9d700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 0x7fb684003e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb688009b70 tx=0x7fb6840058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f182129d5f275b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e6c0 con 0x7fb684003a20 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800e820 con 0x7fb684003a20 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb695d9b700 1 -- 172.21.15.195:0/2603767552 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800ebb0 con 0x7fb684003a20 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 -- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 msgr2=0x7fb684003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 0x7fb684003e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb688009b70 tx=0x7fb6840058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 -- 172.21.15.195:0/2603767552 shutdown_connections 2024-05-08T22:33:00.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 --2- 172.21.15.195:0/2603767552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb684003a20 0x7fb684003e40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 -- 172.21.15.195:0/2603767552 >> 172.21.15.195:0/2603767552 conn(0x7fb6980b18d0 msgr2=0x7fb6980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:00.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 -- 172.21.15.195:0/2603767552 shutdown_connections 2024-05-08T22:33:00.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.026+0000 7fb69d227700 1 -- 172.21.15.195:0/2603767552 wait complete. 2024-05-08T22:33:00.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 Processor -- start 2024-05-08T22:33:00.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- start start 2024-05-08T22:33:00.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb698152540 0x7fb6980c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980c36d0 con 0x7fb698152540 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980c6560 con 0x7fb684003a20 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6980c44c0 con 0x7fb6980c5b70 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33166/0 (socket says 172.21.15.195:33166) 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 -- 172.21.15.195:0/4209600053 learned_addr learned my addr 172.21.15.195:0/4209600053 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69759e700 1 -- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 msgr2=0x7fb6980c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69759e700 1 -- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 msgr2=0x7fb6980c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69759e700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69659c700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb698152540 0x7fb6980c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69759e700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:33:00.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 -- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 msgr2=0x7fb6980c6020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 -- 172.21.15.195:0/4209600053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb698152540 msgr2=0x7fb6980c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb698152540 0x7fb6980c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 -- 172.21.15.195:0/4209600053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6980c38b0 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb696d9d700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6880090b0 tx=0x7fb6880047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e6c0 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688013e30 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688004270 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6980c3b90 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6980c4150 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb6880036c0 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.030+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb68801cba0 con 0x7fb684003a20 2024-05-08T22:33:00.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.034+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb698073290 con 0x7fb684003a20 2024-05-08T22:33:00.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.034+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb688017070 con 0x7fb684003a20 2024-05-08T22:33:00.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.106+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35ef7868-ddcf-4c3b-98e8-e018bd51d06c", "id": 7} v 0) v1 -- 0x7fb6980b2600 con 0x7fb684003a20 2024-05-08T22:33:00.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb68f7fe700 1 -- 172.21.15.195:0/4209600053 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35ef7868-ddcf-4c3b-98e8-e018bd51d06c", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb688009740 con 0x7fb684003a20 2024-05-08T22:33:00.121 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 msgr2=0x7fb6980c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6880090b0 tx=0x7fb6880047e0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 shutdown_connections 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6980c5b70 0x7fb6980c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb698152540 0x7fb6980c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 --2- 172.21.15.195:0/4209600053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb684003a20 0x7fb6980c2d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 >> 172.21.15.195:0/4209600053 conn(0x7fb6980b18d0 msgr2=0x7fb6980b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 shutdown_connections 2024-05-08T22:33:00.123 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-08T22:33:00.118+0000 7fb69d227700 1 -- 172.21.15.195:0/4209600053 wait complete. 2024-05-08T22:33:00.136 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T22:33:00.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T22:33:00.138 INFO:tasks.ceph.osd.0:Started 2024-05-08T22:33:00.138 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T22:33:00.139 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T22:33:00.141 INFO:tasks.ceph.osd.1:Started 2024-05-08T22:33:00.141 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T22:33:00.141 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T22:33:00.143 INFO:tasks.ceph.osd.2:Started 2024-05-08T22:33:00.143 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T22:33:00.143 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T22:33:00.146 INFO:tasks.ceph.osd.3:Started 2024-05-08T22:33:00.146 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T22:33:00.146 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T22:33:00.148 INFO:tasks.ceph.osd.4:Started 2024-05-08T22:33:00.148 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T22:33:00.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T22:33:00.150 INFO:tasks.ceph.osd.5:Started 2024-05-08T22:33:00.150 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T22:33:00.151 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T22:33:00.153 INFO:tasks.ceph.osd.6:Started 2024-05-08T22:33:00.153 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T22:33:00.153 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T22:33:00.155 INFO:tasks.ceph.osd.7:Started 2024-05-08T22:33:00.155 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:00.174 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:00.172+0000 7f60792c52c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:33:00.236 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:00.234+0000 7f7ac116f2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:33:00.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.256+0000 7f5d91a53700 1 Processor -- start 2024-05-08T22:33:00.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.256+0000 7f5d91a53700 1 -- start start 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d8c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c15b7e0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0ccb00 con 0x7f5d8c06bbc0 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0ce870 con 0x7f5d8c0cbe40 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5d8c06bfa0 0x7f5d8c0b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58846/0 (socket says 172.21.15.179:58846) 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 -- 172.21.15.179:0/2095935512 learned_addr learned my addr 172.21.15.179:0/2095935512 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d8c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 msgr2=0x7f5d8c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 msgr2=0x7f5d8c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d8c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d8c0ce330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922242599 0 0) 0x7f5d8c15b7e0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c0035d0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328137290 0 0) 0x7f5d8c0ccb00 con 0x7f5d8c06bbc0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d8c15b7e0 con 0x7f5d8c06bbc0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 484884469 0 0) 0x7f5d7c0035d0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 msgr2=0x7f5d8c0ce330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d8c0ce330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 >> v1:172.21.15.195:6789/0 conn(0x7f5d8c06bbc0 legacy=0x7f5d8c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0cc2c0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5d840019f0 con 0x7f5d8c06bfa0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 >> v1:172.21.15.179:6789/0 conn(0x7f5d8c06bfa0 legacy=0x7f5d8c0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d7c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004140 0x7f5d7c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d7c004db0 0x7f5d7c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0ce870 con 0x7f5d7c004db0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c00bbc0 con 0x7f5d7c004140 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c00bd80 con 0x7f5d8c0cbe40 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d90a51700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d7c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d7c004db0 0x7f5d7c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004140 0x7f5d7c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 msgr2=0x7f5d7c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c0cbe40 0x7f5d7c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004140 msgr2=0x7f5d7c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004140 0x7f5d7c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 -- 172.21.15.179:0/2095935512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0035d0 con 0x7f5d7c004db0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d7c004db0 0x7f5d7c005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d84002230 tx=0x7f5d7c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a64d8f87c2fa71f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840072d0 con 0x7f5d7c004db0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84007430 con 0x7f5d7c004db0 2024-05-08T22:33:00.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d8b7fe700 1 -- 172.21.15.179:0/2095935512 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840077a0 con 0x7f5d7c004db0 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- 172.21.15.179:0/2095935512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d7c004db0 msgr2=0x7f5d7c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 --2- 172.21.15.179:0/2095935512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d7c004db0 0x7f5d7c005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d84002230 tx=0x7f5d7c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91252700 1 -- 172.21.15.179:0/2095935512 reap_dead start 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- 172.21.15.179:0/2095935512 shutdown_connections 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- 172.21.15.179:0/2095935512 >> 172.21.15.179:0/2095935512 conn(0x7f5d8c070c70 msgr2=0x7f5d8c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:00.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- 172.21.15.179:0/2095935512 shutdown_connections 2024-05-08T22:33:00.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.260+0000 7f5d91a53700 1 -- 172.21.15.179:0/2095935512 wait complete. 2024-05-08T22:33:00.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 Processor -- start 2024-05-08T22:33:00.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 -- start start 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004db0 0x7f5d8c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c142f00 con 0x7f5d8c142a50 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15ac20 con 0x7f5d7c004db0 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91a53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c13c4f0 con 0x7f5d8c1f1100 2024-05-08T22:33:00.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91252700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91252700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53072/0 (socket says 172.21.15.179:53072) 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d91252700 1 -- 172.21.15.179:0/1144852281 learned_addr learned my addr 172.21.15.179:0/1144852281 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.264+0000 7f5d90a51700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004db0 0x7f5d8c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d8bfff700 1 -- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 msgr2=0x7f5d8c142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d8bfff700 1 -- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 msgr2=0x7f5d8c142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d8bfff700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 -- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 msgr2=0x7f5d8c142510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 -- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004db0 msgr2=0x7f5d8c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004db0 0x7f5d8c15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 -- 172.21.15.179:0/1144852281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c13a350 con 0x7f5d8c142a50 2024-05-08T22:33:00.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91252700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d84001c30 tx=0x7f5d84001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84025030 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1397d0 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840019d0 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84012640 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c139d90 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5d84028070 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5d84026070 con 0x7f5d8c142a50 2024-05-08T22:33:00.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.268+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c0caa60 con 0x7f5d8c142a50 2024-05-08T22:33:00.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.272+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5d84018070 con 0x7f5d8c142a50 2024-05-08T22:33:00.315 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:00.312+0000 7f60792c52c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:33:00.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.336+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d8c067740 con 0x7f5d8c142a50 2024-05-08T22:33:00.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d897fa700 1 -- 172.21.15.179:0/1144852281 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5d8400a3f0 con 0x7f5d8c142a50 2024-05-08T22:33:00.341 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:00.342 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":10,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:00.115678+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:00.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 msgr2=0x7f5d8c137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d84001c30 tx=0x7f5d84001d00 comp rx=0 tx=0).stop 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 shutdown_connections 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d8c142a50 0x7f5d8c137530 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d8c1f1100 0x7f5d8c142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 --2- 172.21.15.179:0/1144852281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d7c004db0 0x7f5d8c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 >> 172.21.15.179:0/1144852281 conn(0x7f5d8c070c70 msgr2=0x7f5d8c15d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 shutdown_connections 2024-05-08T22:33:00.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:00.340+0000 7f5d91a53700 1 -- 172.21.15.179:0/1144852281 wait complete. 2024-05-08T22:33:00.353 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T22:33:00.353 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T22:33:00.359 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:00.354+0000 7f7ac116f2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:33:00.515 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-05-08T22:33:00.510+0000 7ff04f5af880 -1 Falling back to public interface 2024-05-08T22:33:00.515 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-05-08T22:33:00.510+0000 7f64ce165880 -1 Falling back to public interface 2024-05-08T22:33:00.523 INFO:tasks.ceph.osd.0.smithi179.stderr:2024-05-08T22:33:00.520+0000 7f7aebdb1880 -1 Falling back to public interface 2024-05-08T22:33:00.526 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-05-08T22:33:00.522+0000 7f62c431b880 -1 Falling back to public interface 2024-05-08T22:33:00.677 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:00.676+0000 7f60792c52c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:33:00.724 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:00.718+0000 7f7ac116f2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:33:00.773 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-05-08T22:33:00.770+0000 7f4746251880 -1 Falling back to public interface 2024-05-08T22:33:00.782 INFO:tasks.ceph.osd.1.smithi179.stderr:2024-05-08T22:33:00.780+0000 7f52b968a880 -1 Falling back to public interface 2024-05-08T22:33:00.798 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-05-08T22:33:00.796+0000 7f2b46729880 -1 Falling back to public interface 2024-05-08T22:33:00.810 INFO:tasks.ceph.osd.2.smithi179.stderr:2024-05-08T22:33:00.808+0000 7fd5972a3880 -1 Falling back to public interface 2024-05-08T22:33:00.870 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:00.868+0000 7f60792c52c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:33:00.922 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:00.914+0000 7f7ac116f2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:33:00.933 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:00.932+0000 7f60792c52c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:33:00.986 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:00.982+0000 7f7ac116f2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:33:01.097 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:01.092+0000 7f60792c52c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:33:01.149 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:01.146+0000 7f7ac116f2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:33:01.157 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:01.259 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 Processor -- start 2024-05-08T22:33:01.259 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 -- start start 2024-05-08T22:33:01.260 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f690c0b3460 0x7f690c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:01.260 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:01.260 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0b4200 con 0x7f690c0718f0 2024-05-08T22:33:01.260 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0cec80 con 0x7f690c0b3460 2024-05-08T22:33:01.261 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6912fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0b2350 con 0x7f690c0b27e0 2024-05-08T22:33:01.262 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f691159c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:01.262 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f690bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f690c0b3460 0x7f690c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:01.262 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f691159c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:49334/0 (socket says 172.21.15.179:49334) 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f6910d9b700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f690c0718f0 0x7f690c071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:39300/0 (socket says 172.21.15.179:39300) 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.256+0000 7f691159c700 1 -- 172.21.15.179:0/562228808 learned_addr learned my addr 172.21.15.179:0/562228808 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f691159c700 1 -- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 msgr2=0x7f690c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f691159c700 1 -- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 msgr2=0x7f690c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f691159c700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f691159c700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690bfff700 1 -- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 msgr2=0x7f690c159c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690bfff700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690b7fe700 1 -- 172.21.15.179:0/562228808 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701661747 0 0) 0x7f690c0b4200 con 0x7f690c0718f0 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690bfff700 1 -- 172.21.15.179:0/562228808 >> v1:172.21.15.195:6789/0 conn(0x7f690c0718f0 legacy=0x7f690c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690bfff700 1 -- 172.21.15.179:0/562228808 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0b3c40 con 0x7f690c0b3460 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690bfff700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b3460 0x7f690c06f120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009850 tx=0x7f68f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f6f0fbd4d63ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690b7fe700 1 -- 172.21.15.179:0/562228808 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011830 con 0x7f690c0b3460 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690b7fe700 1 -- 172.21.15.179:0/562228808 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4011990 con 0x7f690c0b3460 2024-05-08T22:33:01.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f690b7fe700 1 -- 172.21.15.179:0/562228808 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011d20 con 0x7f690c0b3460 2024-05-08T22:33:01.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 -- 172.21.15.179:0/562228808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b3460 msgr2=0x7f690c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:01.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b3460 0x7f690c06f120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009850 tx=0x7f68f4009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:01.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 -- 172.21.15.179:0/562228808 shutdown_connections 2024-05-08T22:33:01.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0b27e0 0x7f690c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 --2- 172.21.15.179:0/562228808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b3460 0x7f690c06f120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 -- 172.21.15.179:0/562228808 >> 172.21.15.179:0/562228808 conn(0x7f690c0cc750 msgr2=0x7f690c0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 -- 172.21.15.179:0/562228808 shutdown_connections 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 -- 172.21.15.179:0/562228808 wait complete. 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.260+0000 7f6912fff700 1 Processor -- start 2024-05-08T22:33:01.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- start start 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0718f0 0x7f690c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c0b3460 0x7f690c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1f3eb0 con 0x7f690c0b27e0 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c147020 con 0x7f690c0b3460 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1f3bd0 con 0x7f690c0718f0 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53080/0 (socket says 172.21.15.179:53080) 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 -- 172.21.15.179:0/1082384444 learned_addr learned my addr 172.21.15.179:0/1082384444 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f691159c700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c0b3460 0x7f690c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:01.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6910d9b700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0718f0 0x7f690c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 -- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0718f0 msgr2=0x7f690c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0718f0 0x7f690c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 -- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c0b3460 msgr2=0x7f690c158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c0b3460 0x7f690c158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 -- 172.21.15.179:0/1082384444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40094b0 con 0x7f690c0b27e0 2024-05-08T22:33:01.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f690bfff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f68f4014040 tx=0x7f68f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4004590 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f401e070 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c1f6630 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003900 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c1f6bf0 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f68f402d450 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f68f4003d60 con 0x7f690c0b27e0 2024-05-08T22:33:01.269 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.264+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c067770 con 0x7f690c0b27e0 2024-05-08T22:33:01.272 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.268+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f68f4020070 con 0x7f690c0b27e0 2024-05-08T22:33:01.336 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.332+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f690c1f6de0 con 0x7f690c0b27e0 2024-05-08T22:33:01.337 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.332+0000 7f69097fa700 1 -- 172.21.15.179:0/1082384444 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f68f4015020 con 0x7f690c0b27e0 2024-05-08T22:33:01.337 INFO:teuthology.misc.health.smithi179.stdout: 2024-05-08T22:33:01.337 INFO:teuthology.misc.health.smithi179.stdout:{"epoch":10,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:00.115678+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:01.338 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 msgr2=0x7f690c158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:01.338 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f68f4014040 tx=0x7f68f4004770 comp rx=0 tx=0).stop 2024-05-08T22:33:01.338 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 shutdown_connections 2024-05-08T22:33:01.338 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c0b3460 0x7f690c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.338 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c0b27e0 0x7f690c158770 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.339 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 --2- 172.21.15.179:0/1082384444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f690c0718f0 0x7f690c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:01.339 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 >> 172.21.15.179:0/1082384444 conn(0x7f690c0cc750 msgr2=0x7f690c071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:01.339 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 shutdown_connections 2024-05-08T22:33:01.339 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:01.336+0000 7f6912fff700 1 -- 172.21.15.179:0/1082384444 wait complete. 2024-05-08T22:33:01.348 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T22:33:01.898 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:01.894+0000 7f7ac116f2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:33:01.910 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:01.908+0000 7f60792c52c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:33:01.968 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:01.962+0000 7f7ac116f2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:33:01.983 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:01.980+0000 7f60792c52c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:33:02.131 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:02.126+0000 7f7ac116f2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:33:02.144 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:02.140+0000 7f60792c52c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:33:02.197 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:02.194+0000 7f7ac116f2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:33:02.212 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:02.208+0000 7f60792c52c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:33:02.278 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:02.274+0000 7f7ac116f2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:33:02.288 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:02.284+0000 7f60792c52c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:33:02.571 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:02.566+0000 7f7ac116f2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:33:02.613 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:02.612+0000 7f60792c52c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:33:02.640 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:02.638+0000 7f7ac116f2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:33:02.689 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:02.688+0000 7f60792c52c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:33:03.339 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:03.334+0000 7f7ac116f2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:33:03.440 INFO:tasks.ceph.mgr.y.smithi179.stderr:2024-05-08T22:33:03.436+0000 7f60792c52c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:33:03.923 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-05-08T22:33:03.918+0000 7f62c431b880 -1 osd.7 0 log_to_monitors true 2024-05-08T22:33:03.954 INFO:tasks.ceph.osd.0.smithi179.stderr:2024-05-08T22:33:03.952+0000 7f7aebdb1880 -1 osd.0 0 log_to_monitors true 2024-05-08T22:33:03.999 INFO:tasks.ceph.osd.2.smithi179.stderr:2024-05-08T22:33:03.996+0000 7fd5972a3880 -1 osd.2 0 log_to_monitors true 2024-05-08T22:33:04.213 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-05-08T22:33:04.210+0000 7f64ce165880 -1 osd.5 0 log_to_monitors true 2024-05-08T22:33:04.385 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:04.382+0000 7f7aa7cea700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T22:33:04.386 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-05-08T22:33:04.382+0000 7f7aa7cea700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T22:33:04.386 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-05-08T22:33:04.382+0000 7ff04f5af880 -1 osd.4 0 log_to_monitors true 2024-05-08T22:33:04.469 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-05-08T22:33:04.468+0000 7f2b46729880 -1 osd.3 0 log_to_monitors true 2024-05-08T22:33:04.596 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-05-08T22:33:04.594+0000 7f4746251880 -1 osd.6 0 log_to_monitors true 2024-05-08T22:33:04.623 INFO:tasks.ceph.osd.1.smithi179.stderr:2024-05-08T22:33:04.620+0000 7f52b968a880 -1 osd.1 0 log_to_monitors true 2024-05-08T22:33:06.667 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-05-08T22:33:06.662+0000 7f62c015a700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T22:33:06.670 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-05-08T22:33:06.666+0000 7f64c9fa4700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T22:33:06.674 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-05-08T22:33:06.670+0000 7f64c1a53700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:06.680 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-05-08T22:33:06.678+0000 7ff04b3ee700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T22:33:06.685 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-05-08T22:33:06.682+0000 7ff042e9d700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:06.693 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-05-08T22:33:06.690+0000 7f62b7c09700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:07.092 INFO:tasks.ceph.osd.0.smithi179.stderr:2024-05-08T22:33:07.088+0000 7f7ae7bf0700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T22:33:07.119 INFO:tasks.ceph.osd.0.smithi179.stderr:2024-05-08T22:33:07.116+0000 7f7adf69f700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:07.139 INFO:tasks.ceph.osd.2.smithi179.stderr:2024-05-08T22:33:07.136+0000 7fd5930e2700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T22:33:07.169 INFO:tasks.ceph.osd.2.smithi179.stderr:2024-05-08T22:33:07.168+0000 7fd58ab91700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:07.297 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-05-08T22:33:07.294+0000 7f4742a17700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T22:33:07.303 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-05-08T22:33:07.298+0000 7f4739b3f700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:07.793 INFO:tasks.ceph.osd.1.smithi179.stderr:2024-05-08T22:33:07.792+0000 7f52b5e50700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T22:33:07.801 INFO:tasks.ceph.osd.1.smithi179.stderr:2024-05-08T22:33:07.796+0000 7f52acf78700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:07.828 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-05-08T22:33:07.824+0000 7f2b42568700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T22:33:07.835 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-05-08T22:33:07.832+0000 7f2b3a017700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:08.154 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:08.261 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.256+0000 7f4f866cd700 1 Processor -- start 2024-05-08T22:33:08.261 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- start start 2024-05-08T22:33:08.263 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4f780e1360 0x7f4f780535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f78054c00 0x7f4f78053ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f780e4de0 0x7f4f780e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f78055d10 con 0x7f4f780e4de0 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f78055090 con 0x7f4f780e1360 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78055370 con 0x7f4f78054c00 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f780e4de0 0x7f4f780e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f780e4de0 0x7f4f780e1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54176/0 (socket says 172.21.15.179:54176) 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2941278455 learned_addr learned my addr 172.21.15.179:0/2941278455 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f856cb700 1 --2- 172.21.15.179:0/2941278455 >> v2:172.21.15.195:3300/0 conn(0x7f4f780e1360 0x7f4f780535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.264 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f84eca700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f78054c00 0x7f4f78053ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2941278455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f78054c00 msgr2=0x7f4f78053ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f78054c00 0x7f4f78053ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2941278455 >> v2:172.21.15.195:3300/0 conn(0x7f4f780e1360 msgr2=0x7f4f780535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2941278455 >> v2:172.21.15.195:3300/0 conn(0x7f4f780e1360 0x7f4f780535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2941278455 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78055650 con 0x7f4f780e4de0 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f780e1740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f80068c40 tx=0x7f4f8004f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca796c4ca987d3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f73fff700 1 -- 172.21.15.179:0/2941278455 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80067250 con 0x7f4f780e4de0 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f73fff700 1 -- 172.21.15.179:0/2941278455 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f800673b0 con 0x7f4f780e4de0 2024-05-08T22:33:08.265 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f73fff700 1 -- 172.21.15.179:0/2941278455 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80067960 con 0x7f4f780e4de0 2024-05-08T22:33:08.266 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- 172.21.15.179:0/2941278455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 msgr2=0x7f4f780e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.266 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f780e1740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f80068c40 tx=0x7f4f8004f750 comp rx=0 tx=0).stop 2024-05-08T22:33:08.266 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- 172.21.15.179:0/2941278455 shutdown_connections 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f780e1740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2941278455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f78054c00 0x7f4f78053ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2941278455 >> v2:172.21.15.195:3300/0 conn(0x7f4f780e1360 0x7f4f780535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- 172.21.15.179:0/2941278455 >> 172.21.15.179:0/2941278455 conn(0x7f4f780200f0 msgr2=0x7f4f78056e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- 172.21.15.179:0/2941278455 shutdown_connections 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.260+0000 7f4f866cd700 1 -- 172.21.15.179:0/2941278455 wait complete. 2024-05-08T22:33:08.267 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.264+0000 7f4f866cd700 1 Processor -- start 2024-05-08T22:33:08.268 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.264+0000 7f4f866cd700 1 -- start start 2024-05-08T22:33:08.270 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.270 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.270 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.270 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78052d20 con 0x7f4f780e4de0 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7804ecd0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78052110 con 0x7f4f780e1360 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f84eca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f856cb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.271 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f84eca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54184/0 (socket says 172.21.15.179:54184) 2024-05-08T22:33:08.273 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f856cb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:40548/0 (socket says 172.21.15.179:40548) 2024-05-08T22:33:08.273 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:48098/0 (socket says 172.21.15.179:48098) 2024-05-08T22:33:08.273 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f84eca700 1 -- 172.21.15.179:0/2837909757 learned_addr learned my addr 172.21.15.179:0/2837909757 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 msgr2=0x7f4f78176590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2837909757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 msgr2=0x7f4f7804e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 -- 172.21.15.179:0/2837909757 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8004f5c0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f856cb700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f84eca700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:08.275 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f85ecc700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f80068960 tx=0x7f4f80073960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80099070 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80066c80 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80067250 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f780523f0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.268+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f780529b0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f78052b40 con 0x7f4f7804ccf0 2024-05-08T22:33:08.276 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f4f80066200 con 0x7f4f7804ccf0 2024-05-08T22:33:08.277 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f71ffb700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 0x7f4f64068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.277 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3911+0+0 (secure 0 0 0) 0x7f4f800fded0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.277 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f856cb700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 0x7f4f64068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.277 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.272+0000 7f4f856cb700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 0x7f4f64068b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0112c0 tx=0x7f4f7c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.288 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.284+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4f800cbab0 con 0x7f4f7804ccf0 2024-05-08T22:33:08.461 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.456+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f78052e90 con 0x7f4f7804ccf0 2024-05-08T22:33:08.462 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.460+0000 7f4f71ffb700 1 -- 172.21.15.179:0/2837909757 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10603 (secure 0 0 0) 0x7f4f80067790 con 0x7f4f7804ccf0 2024-05-08T22:33:08.463 INFO:teuthology.misc.health.smithi179.stdout: 2024-05-08T22:33:08.463 INFO:teuthology.misc.health.smithi179.stdout:{"epoch":15,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:08.451509+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.460+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 msgr2=0x7f4f64068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 0x7f4f64068b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0112c0 tx=0x7f4f7c009240 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 msgr2=0x7f4f7804c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f80068960 tx=0x7f4f80073960 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 shutdown_connections 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f640666a0 0x7f4f64068b60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7804ccf0 0x7f4f7804c130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f780e4de0 0x7f4f7804e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 --2- 172.21.15.179:0/2837909757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f780e1360 0x7f4f78176590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 >> 172.21.15.179:0/2837909757 conn(0x7f4f780200f0 msgr2=0x7f4f780e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:08.466 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 shutdown_connections 2024-05-08T22:33:08.467 INFO:teuthology.misc.health.smithi179.stderr:2024-05-08T22:33:08.464+0000 7f4f866cd700 1 -- 172.21.15.179:0/2837909757 wait complete. 2024-05-08T22:33:08.479 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T22:33:08.479 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T22:33:08.479 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:33:08.479 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:33:08.481 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:33:08.481 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:33:08.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:33:08.483 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:33:08.483 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:33:08.483 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:33:08.485 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:33:08.486 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:33:08.486 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:33:08.488 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:33:08.488 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T22:33:08.491 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:08.565 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:33:08.565 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:33:08.567 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:33:08.571 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:33:08.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 Processor -- start 2024-05-08T22:33:08.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 -- start start 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e81616e0 0x7f11e80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11e80c2930 0x7f11e806de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e80b3f60 con 0x7f11e80cd150 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e8161ac0 con 0x7f11e80c2930 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ee742700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e80b16d0 con 0x7f11e81616e0 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11e80c2930 0x7f11e806de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11e80c2930 0x7f11e806de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54228/0 (socket says 172.21.15.179:54228) 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ecf3f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f11e80cd150 0x7f11e80c2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58642/0 (socket says 172.21.15.179:58642) 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 -- 172.21.15.179:0/63860266 learned_addr learned my addr 172.21.15.179:0/63860266 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11ed740700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e81616e0 0x7f11e80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11e7fff700 1 -- 172.21.15.179:0/63860266 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521143689 0 0) 0x7f11e80b3f60 con 0x7f11e80cd150 2024-05-08T22:33:08.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 -- 172.21.15.179:0/63860266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e81616e0 msgr2=0x7f11e80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:08.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e81616e0 0x7f11e80ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 -- 172.21.15.179:0/63860266 >> v1:172.21.15.179:6789/0 conn(0x7f11e80cd150 legacy=0x7f11e80c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 -- 172.21.15.179:0/63860266 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80b41b0 con 0x7f11e80c2930 2024-05-08T22:33:08.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.596+0000 7f11edf41700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e806de60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009910 tx=0x7f11e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722a6566b226807a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11e7fff700 1 -- 172.21.15.179:0/63860266 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0011830 con 0x7f11e80c2930 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11e7fff700 1 -- 172.21.15.179:0/63860266 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e0011990 con 0x7f11e80c2930 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11e7fff700 1 -- 172.21.15.179:0/63860266 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0011d40 con 0x7f11e80c2930 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- 172.21.15.179:0/63860266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 msgr2=0x7f11e806de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e806de60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009910 tx=0x7f11e0009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- 172.21.15.179:0/63860266 shutdown_connections 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e806de60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 --2- 172.21.15.179:0/63860266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e81616e0 0x7f11e80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- 172.21.15.179:0/63860266 >> 172.21.15.179:0/63860266 conn(0x7f11e8098a70 msgr2=0x7f11e80b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:08.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- 172.21.15.179:0/63860266 shutdown_connections 2024-05-08T22:33:08.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- 172.21.15.179:0/63860266 wait complete. 2024-05-08T22:33:08.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 Processor -- start 2024-05-08T22:33:08.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.600+0000 7f11ee742700 1 -- start start 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 0x7f11e8099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 0x7f11e80c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8154640 con 0x7f11e80c2930 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8153750 con 0x7f11e81616e0 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8153a30 con 0x7f11e80cd150 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54232/0 (socket says 172.21.15.179:54232) 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 -- 172.21.15.179:0/666444286 learned_addr learned my addr 172.21.15.179:0/666444286 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11edf41700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 0x7f11e80c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ecf3f700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 0x7f11e8099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 -- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 msgr2=0x7f11e8099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 0x7f11e8099500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 -- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 msgr2=0x7f11e80c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 0x7f11e80c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 -- 172.21.15.179:0/666444286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0009570 con 0x7f11e80c2930 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11edf41700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 0x7f11e80c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ecf3f700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 0x7f11e8099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ed740700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e9d0 tx=0x7f11dc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc009720 con 0x7f11e80c2930 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc00c850 con 0x7f11e80c2930 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e8153d10 con 0x7f11e80c2930 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00cbc0 con 0x7f11e80c2930 2024-05-08T22:33:08.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e81542d0 con 0x7f11e80c2930 2024-05-08T22:33:08.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e804e700 con 0x7f11e80c2930 2024-05-08T22:33:08.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.604+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f11dc00cd20 con 0x7f11e80c2930 2024-05-08T22:33:08.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.608+0000 7f11e5ffb700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 0x7f11d4068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.608+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f11dc083d50 con 0x7f11e80c2930 2024-05-08T22:33:08.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.608+0000 7f11ecf3f700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 0x7f11d4068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.608+0000 7f11ecf3f700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 0x7f11d4068b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11e80d0310 tx=0x7f11d8001a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.616+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11dc0515d0 con 0x7f11e80c2930 2024-05-08T22:33:08.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.788+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11e806e750 con 0x7f11e80c2930 2024-05-08T22:33:08.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11e5ffb700 1 -- 172.21.15.179:0/666444286 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10603 (secure 0 0 0) 0x7f11dc054bf0 con 0x7f11e80c2930 2024-05-08T22:33:08.794 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:08.794 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":15,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:08.451509+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 msgr2=0x7f11d4068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 0x7f11d4068b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11e80d0310 tx=0x7f11d8001a30 comp rx=0 tx=0).stop 2024-05-08T22:33:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 msgr2=0x7f11e8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e9d0 tx=0x7f11dc00ed90 comp rx=0 tx=0).stop 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 shutdown_connections 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11d40666a0 0x7f11d4068b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e81616e0 0x7f11e80c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e80cd150 0x7f11e8099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 --2- 172.21.15.179:0/666444286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e80c2930 0x7f11e8152e40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 >> 172.21.15.179:0/666444286 conn(0x7f11e8098a70 msgr2=0x7f11e80cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 shutdown_connections 2024-05-08T22:33:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.792+0000 7f11ee742700 1 -- 172.21.15.179:0/666444286 wait complete. 2024-05-08T22:33:08.808 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-08T22:33:08.809 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:08.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 Processor -- start 2024-05-08T22:33:08.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 -- start start 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd62406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd62406dc20 0x7fd62406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6240cd200 con 0x7fd6240ce2a0 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6240cd4e0 con 0x7fd62406dc20 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.920+0000 7fd629f8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240cd7c0 con 0x7fd6241434d0 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd62406dc20 0x7fd62406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd62978a700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd6240ce2a0 0x7fd6240b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58646/0 (socket says 172.21.15.179:58646) 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd62406dc20 0x7fd62406e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54246/0 (socket says 172.21.15.179:54246) 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd62978a700 1 -- 172.21.15.179:0/2822375131 learned_addr learned my addr 172.21.15.179:0/2822375131 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd628f89700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd62406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 -- 172.21.15.179:0/2822375131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 msgr2=0x7fd62406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:08.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd62406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 -- 172.21.15.179:0/2822375131 >> v1:172.21.15.179:6789/0 conn(0x7fd6240ce2a0 legacy=0x7fd6240b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 -- 172.21.15.179:0/2822375131 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240cdaa0 con 0x7fd62406dc20 2024-05-08T22:33:08.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd623fff700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd62406dc20 0x7fd62406e640 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd61400baf0 tx=0x7fd61400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d3c65d8dba7dc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd6237fe700 1 -- 172.21.15.179:0/2822375131 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400dc30 con 0x7fd62406dc20 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd6237fe700 1 -- 172.21.15.179:0/2822375131 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400dd90 con 0x7fd62406dc20 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd6237fe700 1 -- 172.21.15.179:0/2822375131 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614004250 con 0x7fd62406dc20 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- 172.21.15.179:0/2822375131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd62406dc20 msgr2=0x7fd62406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd62406dc20 0x7fd62406e640 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd61400baf0 tx=0x7fd61400be00 comp rx=0 tx=0).stop 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- 172.21.15.179:0/2822375131 shutdown_connections 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd62406dc20 0x7fd62406e640 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 --2- 172.21.15.179:0/2822375131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd62406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- 172.21.15.179:0/2822375131 >> 172.21.15.179:0/2822375131 conn(0x7fd6240ce9d0 msgr2=0x7fd6240cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- 172.21.15.179:0/2822375131 shutdown_connections 2024-05-08T22:33:08.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- 172.21.15.179:0/2822375131 wait complete. 2024-05-08T22:33:08.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 Processor -- start 2024-05-08T22:33:08.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.924+0000 7fd629f8b700 1 -- start start 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6240ce2a0 0x7fd6241336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd624070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624130df0 con 0x7fd6240c49a0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240c4f10 con 0x7fd6240ce2a0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240c51f0 con 0x7fd6241434d0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54252/0 (socket says 172.21.15.179:54252) 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 -- 172.21.15.179:0/3609645532 learned_addr learned my addr 172.21.15.179:0/3609645532 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd623fff700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd624070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd628f89700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6240ce2a0 0x7fd6241336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 -- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 msgr2=0x7fd624070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd624070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 -- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6240ce2a0 msgr2=0x7fd6241336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6240ce2a0 0x7fd6241336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 -- 172.21.15.179:0/3609645532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61400b750 con 0x7fd6240c49a0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd62978a700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd61c002000 tx=0x7fd61c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00ad40 con 0x7fd6240c49a0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c0070d0 con 0x7fd6240c49a0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240c54d0 con 0x7fd6240c49a0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00aea0 con 0x7fd6240c49a0 2024-05-08T22:33:08.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.928+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624133c00 con 0x7fd6240c49a0 2024-05-08T22:33:08.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624067740 con 0x7fd6240c49a0 2024-05-08T22:33:08.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fd61c007480 con 0x7fd6240c49a0 2024-05-08T22:33:08.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd6217fa700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 0x7fd604068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:08.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd61c083a20 con 0x7fd6240c49a0 2024-05-08T22:33:08.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd628f89700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 0x7fd604068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:08.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.932+0000 7fd628f89700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 0x7fd604068b60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b9d0 tx=0x7fd60c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:08.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:08.940+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd61c051340 con 0x7fd6240c49a0 2024-05-08T22:33:09.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.148+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62404e700 con 0x7fd6240c49a0 2024-05-08T22:33:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.148+0000 7fd6217fa700 1 -- 172.21.15.179:0/3609645532 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fd61c018090 con 0x7fd6240c49a0 2024-05-08T22:33:09.152 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:09.152 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":54155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 msgr2=0x7fd604068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 0x7fd604068b60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b9d0 tx=0x7fd60c008040 comp rx=0 tx=0).stop 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 msgr2=0x7fd624158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd61c002000 tx=0x7fd61c00a340 comp rx=0 tx=0).stop 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 shutdown_connections 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6040666a0 0x7fd604068b60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6240c49a0 0x7fd624158e40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6241434d0 0x7fd624070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 --2- 172.21.15.179:0/3609645532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6240ce2a0 0x7fd6241336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 >> 172.21.15.179:0/3609645532 conn(0x7fd6240ce9d0 msgr2=0x7fd62415ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 shutdown_connections 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:09.152+0000 7fd629f8b700 1 -- 172.21.15.179:0/3609645532 wait complete. 2024-05-08T22:33:09.156 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 2 2024-05-08T22:33:11.168 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:11.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 Processor -- start 2024-05-08T22:33:11.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 -- start start 2024-05-08T22:33:11.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a9406eaf0 0x7f6a940b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9415d970 0x7f6a9415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9406ce80 con 0x7f6a94143640 2024-05-08T22:33:11.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9415e340 con 0x7f6a9406eaf0 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94143a20 con 0x7f6a9415d970 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a9406eaf0 0x7f6a940b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a9406eaf0 0x7f6a940b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54254/0 (socket says 172.21.15.179:54254) 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 -- 172.21.15.179:0/3351158448 learned_addr learned my addr 172.21.15.179:0/3351158448 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a92d9d700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9415d970 0x7f6a9415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 -- 172.21.15.179:0/3351158448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9415d970 msgr2=0x7f6a9415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9415d970 0x7f6a9415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 -- 172.21.15.179:0/3351158448 >> v1:172.21.15.195:6789/0 conn(0x7f6a94143640 legacy=0x7f6a940cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 -- 172.21.15.179:0/3351158448 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940cf220 con 0x7f6a9406eaf0 2024-05-08T22:33:11.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a91d9b700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a9406eaf0 0x7f6a940b2a50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00bbb0 tx=0x7f6a7c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562ad6bad445220a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9159a700 1 -- 172.21.15.179:0/3351158448 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00dd30 con 0x7f6a9406eaf0 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9159a700 1 -- 172.21.15.179:0/3351158448 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00de90 con 0x7f6a9406eaf0 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.280+0000 7f6a9159a700 1 -- 172.21.15.179:0/3351158448 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c010690 con 0x7f6a9406eaf0 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/3351158448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a9406eaf0 msgr2=0x7f6a940b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a9406eaf0 0x7f6a940b2a50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00bbb0 tx=0x7f6a7c00bec0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/3351158448 shutdown_connections 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9415d970 0x7f6a9415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- 172.21.15.179:0/3351158448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a9406eaf0 0x7f6a940b2a50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/3351158448 >> 172.21.15.179:0/3351158448 conn(0x7f6a940cfae0 msgr2=0x7f6a940cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/3351158448 shutdown_connections 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/3351158448 wait complete. 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 Processor -- start 2024-05-08T22:33:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- start start 2024-05-08T22:33:11.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 0x7f6a94141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 0x7f6a9415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940ab980 con 0x7f6a94143640 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940abc60 con 0x7f6a9415d970 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940abf40 con 0x7f6a9406eaf0 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54264/0 (socket says 172.21.15.179:54264) 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 -- 172.21.15.179:0/583244292 learned_addr learned my addr 172.21.15.179:0/583244292 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9259c700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 0x7f6a94141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a92d9d700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 0x7f6a9415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 -- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 msgr2=0x7f6a94141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 0x7f6a94141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 -- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 msgr2=0x7f6a9415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 0x7f6a9415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 -- 172.21.15.179:0/583244292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c00b810 con 0x7f6a94143640 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a92d9d700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 0x7f6a9415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9259c700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 0x7f6a94141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a91d9b700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6a9406b390 tx=0x7f6a7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003d10 con 0x7f6a94143640 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c003e90 con 0x7f6a94143640 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.284+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a940ac220 con 0x7f6a94143640 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0106b0 con 0x7f6a94143640 2024-05-08T22:33:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94142260 con 0x7f6a94143640 2024-05-08T22:33:11.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9404e700 con 0x7f6a94143640 2024-05-08T22:33:11.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f6a7c02c440 con 0x7f6a94143640 2024-05-08T22:33:11.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a8affd700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 0x7f6a74068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.288+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6a7c0855f0 con 0x7f6a94143640 2024-05-08T22:33:11.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.296+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6a7c052e20 con 0x7f6a94143640 2024-05-08T22:33:11.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.324+0000 7f6a9259c700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 0x7f6a74068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.328+0000 7f6a9259c700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 0x7f6a74068b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a840012e0 tx=0x7f6a84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a94142aa0 con 0x7f6a94143640 2024-05-08T22:33:11.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a8affd700 1 -- 172.21.15.179:0/583244292 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3537 (secure 0 0 0) 0x7f6a7c056440 con 0x7f6a94143640 2024-05-08T22:33:11.511 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:11.511 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:33:11.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 msgr2=0x7f6a74068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 0x7f6a74068b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a840012e0 tx=0x7f6a84009040 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 msgr2=0x7f6a940ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.508+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6a9406b390 tx=0x7f6a7c004770 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 shutdown_connections 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6a740666a0 0x7f6a74068b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9415d970 0x7f6a9415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a94143640 0x7f6a940ad310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 --2- 172.21.15.179:0/583244292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a9406eaf0 0x7f6a94141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 >> 172.21.15.179:0/583244292 conn(0x7f6a940cfae0 msgr2=0x7f6a9406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 shutdown_connections 2024-05-08T22:33:11.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.512+0000 7f6a9359e700 1 -- 172.21.15.179:0/583244292 wait complete. 2024-05-08T22:33:11.515 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 3 2024-05-08T22:33:11.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:11.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 Processor -- start 2024-05-08T22:33:11.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- start start 2024-05-08T22:33:11.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40ce060 0x7f3af406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3af40add40 0x7f3af406b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af4161ac0 con 0x7f3af40cc310 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af40ae550 con 0x7f3af40add40 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af406abb0 con 0x7f3af40ce060 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3af40add40 0x7f3af406b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3af40add40 0x7f3af406b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54276/0 (socket says 172.21.15.179:54276) 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 -- 172.21.15.179:0/3707773786 learned_addr learned my addr 172.21.15.179:0/3707773786 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af917b700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40ce060 0x7f3af406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 -- 172.21.15.179:0/3707773786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40ce060 msgr2=0x7f3af406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40ce060 0x7f3af406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 -- 172.21.15.179:0/3707773786 >> v1:172.21.15.195:6789/0 conn(0x7f3af40cc310 legacy=0x7f3af40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 -- 172.21.15.179:0/3707773786 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af40cdaa0 con 0x7f3af40add40 2024-05-08T22:33:11.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3af997c700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40add40 0x7f3af406b8a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009910 tx=0x7f3af0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71f2a8a41caa919 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3ae7fff700 1 -- 172.21.15.179:0/3707773786 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0011830 con 0x7f3af40add40 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3ae7fff700 1 -- 172.21.15.179:0/3707773786 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0011990 con 0x7f3af40add40 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3ae7fff700 1 -- 172.21.15.179:0/3707773786 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0011d40 con 0x7f3af40add40 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- 172.21.15.179:0/3707773786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40add40 msgr2=0x7f3af406b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40add40 0x7f3af406b8a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009910 tx=0x7f3af0009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- 172.21.15.179:0/3707773786 shutdown_connections 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40add40 0x7f3af406b8a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 --2- 172.21.15.179:0/3707773786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40ce060 0x7f3af406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- 172.21.15.179:0/3707773786 >> 172.21.15.179:0/3707773786 conn(0x7f3af40cb660 msgr2=0x7f3af40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- 172.21.15.179:0/3707773786 shutdown_connections 2024-05-08T22:33:11.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.632+0000 7f3afb3df700 1 -- 172.21.15.179:0/3707773786 wait complete. 2024-05-08T22:33:11.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 Processor -- start 2024-05-08T22:33:11.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 -- start start 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40add40 0x7f3af40aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3af40cc310 0x7f3af40c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40c3770 con 0x7f3af40ce060 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40ac6b0 con 0x7f3af40cc310 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3afb3df700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4160470 con 0x7f3af40add40 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af917b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40add40 0x7f3af40aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54286/0 (socket says 172.21.15.179:54286) 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 -- 172.21.15.179:0/120344774 learned_addr learned my addr 172.21.15.179:0/120344774 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af897a700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3af40cc310 0x7f3af40c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 -- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40add40 msgr2=0x7f3af40aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40add40 0x7f3af40aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 -- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3af40cc310 msgr2=0x7f3af40c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3af40cc310 0x7f3af40c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 -- 172.21.15.179:0/120344774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0009570 con 0x7f3af40ce060 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3af997c700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009150 tx=0x7f3af0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.636+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0004510 con 0x7f3af40ce060 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af001e070 con 0x7f3af40ce060 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af40ac330 con 0x7f3af40ce060 2024-05-08T22:33:11.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0003900 con 0x7f3af40ce060 2024-05-08T22:33:11.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af40c31b0 con 0x7f3af40ce060 2024-05-08T22:33:11.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af4067770 con 0x7f3af40ce060 2024-05-08T22:33:11.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f3af0003a60 con 0x7f3af40ce060 2024-05-08T22:33:11.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3ae5ffb700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 0x7f3ad8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3af917b700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 0x7f3ad8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3af0020070 con 0x7f3af40ce060 2024-05-08T22:33:11.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.640+0000 7f3af917b700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 0x7f3ad8068b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3af40b4590 tx=0x7f3ae8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.648+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3af0052920 con 0x7f3af40ce060 2024-05-08T22:33:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.824+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3af40ac7f0 con 0x7f3af40ce060 2024-05-08T22:33:11.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.824+0000 7f3ae5ffb700 1 -- 172.21.15.179:0/120344774 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f3af40ac7f0 con 0x7f3af40ce060 2024-05-08T22:33:11.828 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:11.828 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":18,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:11.474490+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:11.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 msgr2=0x7f3ad8068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 0x7f3ad8068b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3af40b4590 tx=0x7f3ae8009240 comp rx=0 tx=0).stop 2024-05-08T22:33:11.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 msgr2=0x7f3af415ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009150 tx=0x7f3af0004770 comp rx=0 tx=0).stop 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 shutdown_connections 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ad80666a0 0x7f3ad8068b60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3af40cc310 0x7f3af40c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40ce060 0x7f3af415ff30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 --2- 172.21.15.179:0/120344774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3af40add40 0x7f3af40aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 >> 172.21.15.179:0/120344774 conn(0x7f3af40cb660 msgr2=0x7f3af40cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 shutdown_connections 2024-05-08T22:33:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.828+0000 7f3afb3df700 1 -- 172.21.15.179:0/120344774 wait complete. 2024-05-08T22:33:11.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:33:11.842 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:33:11.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.956+0000 7f8532eb2700 1 Processor -- start 2024-05-08T22:33:11.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.956+0000 7f8532eb2700 1 -- start start 2024-05-08T22:33:11.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f852c0d0500 0x7f852c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 0x7f852c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c06f890 con 0x7f852c15b130 2024-05-08T22:33:11.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0b6cc0 con 0x7f852c0d0500 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0cb470 con 0x7f852c06f420 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8531eb0700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f852c15b130 0x7f852c0b6470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58650/0 (socket says 172.21.15.179:58650) 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8531eb0700 1 -- 172.21.15.179:0/3424077655 learned_addr learned my addr 172.21.15.179:0/3424077655 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 --2- 172.21.15.179:0/3424077655 >> v2:172.21.15.179:3300/0 conn(0x7f852c0d0500 0x7f852c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85326b1700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 0x7f852c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8530eae700 1 -- 172.21.15.179:0/3424077655 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284089438 0 0) 0x7f852c06f890 con 0x7f852c15b130 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8530eae700 1 -- 172.21.15.179:0/3424077655 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85140035d0 con 0x7f852c15b130 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 -- 172.21.15.179:0/3424077655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 msgr2=0x7f852c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 0x7f852c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 -- 172.21.15.179:0/3424077655 >> v1:172.21.15.179:6789/0 conn(0x7f852c15b130 legacy=0x7f852c0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 -- 172.21.15.179:0/3424077655 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0cb750 con 0x7f852c0d0500 2024-05-08T22:33:11.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85326b1700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 0x7f852c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f85316af700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c0d0500 0x7f852c06eee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8520009850 tx=0x7f8520009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6f7cd0231c3b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8530eae700 1 -- 172.21.15.179:0/3424077655 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520011830 con 0x7f852c0d0500 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8530eae700 1 -- 172.21.15.179:0/3424077655 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8520011990 con 0x7f852c0d0500 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8530eae700 1 -- 172.21.15.179:0/3424077655 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520011d40 con 0x7f852c0d0500 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- 172.21.15.179:0/3424077655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c0d0500 msgr2=0x7f852c06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c0d0500 0x7f852c06eee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8520009850 tx=0x7f8520009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- 172.21.15.179:0/3424077655 shutdown_connections 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c06f420 0x7f852c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 --2- 172.21.15.179:0/3424077655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c0d0500 0x7f852c06eee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- 172.21.15.179:0/3424077655 >> 172.21.15.179:0/3424077655 conn(0x7f852c0b2770 msgr2=0x7f852c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- 172.21.15.179:0/3424077655 shutdown_connections 2024-05-08T22:33:11.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.960+0000 7f8532eb2700 1 -- 172.21.15.179:0/3424077655 wait complete. 2024-05-08T22:33:11.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 Processor -- start 2024-05-08T22:33:11.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- start start 2024-05-08T22:33:11.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 0x7f852c1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c0d0500 0x7f852c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 0x7f852c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c06daa0 con 0x7f852c15b130 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c06e530 con 0x7f852c06f420 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c06dc90 con 0x7f852c0d0500 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f85326b1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 0x7f852c1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f85326b1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 0x7f852c1553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54308/0 (socket says 172.21.15.179:54308) 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f85326b1700 1 -- 172.21.15.179:0/1416963324 learned_addr learned my addr 172.21.15.179:0/1416963324 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 0x7f852c1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f85316af700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c0d0500 0x7f852c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 -- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c0d0500 msgr2=0x7f852c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c0d0500 0x7f852c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 -- 172.21.15.179:0/1416963324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 msgr2=0x7f852c1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 0x7f852c1553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:11.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 -- 172.21.15.179:0/1416963324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009a70 con 0x7f852c06f420 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8531eb0700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 0x7f852c1f6a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f852400c440 tx=0x7f852400c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524005d90 con 0x7f852c06f420 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400ce10 con 0x7f852c06f420 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85200094b0 con 0x7f852c06f420 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240072d0 con 0x7f852c06f420 2024-05-08T22:33:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0b3ee0 con 0x7f852c06f420 2024-05-08T22:33:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.964+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c04e700 con 0x7f852c06f420 2024-05-08T22:33:11.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.968+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f8524007430 con 0x7f852c06f420 2024-05-08T22:33:11.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.968+0000 7f852a7fc700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 0x7f8518068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:11.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.968+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85240834a0 con 0x7f852c06f420 2024-05-08T22:33:11.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.968+0000 7f85316af700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 0x7f8518068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:11.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.972+0000 7f85316af700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 0x7f8518068b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85200090b0 tx=0x7f852001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:11.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:11.976+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8524050cb0 con 0x7f852c06f420 2024-05-08T22:33:12.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.148+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f852c0b34b0 con 0x7f852c06f420 2024-05-08T22:33:12.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f852a7fc700 1 -- 172.21.15.179:0/1416963324 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85240170a0 con 0x7f852c06f420 2024-05-08T22:33:12.158 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:33:12.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 msgr2=0x7f8518068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 0x7f8518068b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85200090b0 tx=0x7f852001c040 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 msgr2=0x7f852c1f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 0x7f852c1f6a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f852400c440 tx=0x7f852400c750 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 shutdown_connections 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85180666a0 0x7f8518068b60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c15b130 0x7f852c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f852c0d0500 0x7f852c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 --2- 172.21.15.179:0/1416963324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f852c06f420 0x7f852c1f6a50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 >> 172.21.15.179:0/1416963324 conn(0x7f852c0b2770 msgr2=0x7f852c15a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 shutdown_connections 2024-05-08T22:33:12.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.156+0000 7f8532eb2700 1 -- 172.21.15.179:0/1416963324 wait complete. 2024-05-08T22:33:12.177 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:33:12.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 Processor -- start 2024-05-08T22:33:12.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 -- start start 2024-05-08T22:33:12.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa15c0b19e0 0x7fa15c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 0x7fa15c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c0b3980 con 0x7fa15c1539c0 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c159b60 con 0x7fa15c0b3550 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c159e40 con 0x7fa15c0b19e0 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 0x7fa15c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 0x7fa15c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54324/0 (socket says 172.21.15.179:54324) 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15b7fe700 1 -- 172.21.15.179:0/1777488890 learned_addr learned my addr 172.21.15.179:0/1777488890 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa15affd700 1 --1- 172.21.15.179:0/1777488890 >> v1:172.21.15.179:6789/0 conn(0x7fa15c1539c0 0x7fa15c06b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58666/0 (socket says 172.21.15.179:58666) 2024-05-08T22:33:12.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa153fff700 1 --2- 172.21.15.179:0/1777488890 >> v2:172.21.15.195:3300/0 conn(0x7fa15c0b19e0 0x7fa15c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa153fff700 1 -- 172.21.15.179:0/1777488890 >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 msgr2=0x7fa15c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa153fff700 1 --2- 172.21.15.179:0/1777488890 >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 0x7fa15c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.288+0000 7fa153fff700 1 -- 172.21.15.179:0/1777488890 >> v1:172.21.15.179:6789/0 conn(0x7fa15c1539c0 legacy=0x7fa15c06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa153fff700 1 -- 172.21.15.179:0/1777488890 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c15a120 con 0x7fa15c0b19e0 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa153fff700 1 --2- 172.21.15.179:0/1777488890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b19e0 0x7fa15c0b3010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0708c0 tx=0x7fa144009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ae171a10275f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15a7fc700 1 -- 172.21.15.179:0/1777488890 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f8a0 con 0x7fa15c0b19e0 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15a7fc700 1 -- 172.21.15.179:0/1777488890 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400fa00 con 0x7fa15c0b19e0 2024-05-08T22:33:12.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15a7fc700 1 -- 172.21.15.179:0/1777488890 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400fd90 con 0x7fa15c0b19e0 2024-05-08T22:33:12.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 -- 172.21.15.179:0/1777488890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b19e0 msgr2=0x7fa15c0b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1777488890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b19e0 0x7fa15c0b3010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0708c0 tx=0x7fa144009b30 comp rx=0 tx=0).stop 2024-05-08T22:33:12.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 -- 172.21.15.179:0/1777488890 shutdown_connections 2024-05-08T22:33:12.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1777488890 >> v2:172.21.15.179:3300/0 conn(0x7fa15c0b3550 0x7fa15c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1777488890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b19e0 0x7fa15c0b3010 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 -- 172.21.15.179:0/1777488890 >> 172.21.15.179:0/1777488890 conn(0x7fa15c0ce9d0 msgr2=0x7fa15c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:12.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 -- 172.21.15.179:0/1777488890 shutdown_connections 2024-05-08T22:33:12.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.292+0000 7fa15bfff700 1 -- 172.21.15.179:0/1777488890 wait complete. 2024-05-08T22:33:12.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 Processor -- start 2024-05-08T22:33:12.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 -- start start 2024-05-08T22:33:12.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa15c0b19e0 0x7fa15c14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 0x7fa15c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c142d70 con 0x7fa15c1539c0 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06d160 con 0x7fa15c0b3550 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06d440 con 0x7fa15c0b19e0 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 0x7fa15c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa153fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa153fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:48194/0 (socket says 172.21.15.179:48194) 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa153fff700 1 -- 172.21.15.179:0/1982891712 learned_addr learned my addr 172.21.15.179:0/1982891712 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:12.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.296+0000 7fa15affd700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa15c0b19e0 0x7fa15c14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 -- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa15c0b19e0 msgr2=0x7fa15c14e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa15c0b19e0 0x7fa15c14e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 -- 172.21.15.179:0/1982891712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 msgr2=0x7fa15c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 0x7fa15c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 -- 172.21.15.179:0/1982891712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1440094b0 con 0x7fa15c0b3550 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa15b7fe700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 0x7fa15c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa153fff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1440090b0 tx=0x7fa144004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14401e070 con 0x7fa15c0b3550 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1440036c0 con 0x7fa15c0b3550 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c06d720 con 0x7fa15c0b3550 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144003a70 con 0x7fa15c0b3550 2024-05-08T22:33:12.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c14e8a0 con 0x7fa15c0b3550 2024-05-08T22:33:12.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c0cb250 con 0x7fa15c0b3550 2024-05-08T22:33:12.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa144003bd0 con 0x7fa15c0b3550 2024-05-08T22:33:12.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 0x7fa13c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa1440867d0 con 0x7fa15c0b3550 2024-05-08T22:33:12.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.300+0000 7fa15affd700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 0x7fa13c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.304+0000 7fa15affd700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 0x7fa13c068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa14c000f60 tx=0x7fa14c007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.308+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa144053f20 con 0x7fa15c0b3550 2024-05-08T22:33:12.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.484+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa15c14ea30 con 0x7fa15c0b3550 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.488+0000 7fa1537fe700 1 -- 172.21.15.179:0/1982891712 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa144028090 con 0x7fa15c0b3550 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:33:12.491 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:33:12.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.488+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 msgr2=0x7fa13c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.488+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 0x7fa13c068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa14c000f60 tx=0x7fa14c007eb0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.488+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 msgr2=0x7fa15c1424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1440090b0 tx=0x7fa144004770 comp rx=0 tx=0).stop 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 shutdown_connections 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa13c066730 0x7fa13c068bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa15c0b3550 0x7fa15c1424a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c1539c0 0x7fa15c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 --2- 172.21.15.179:0/1982891712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa15c0b19e0 0x7fa15c14e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 >> 172.21.15.179:0/1982891712 conn(0x7fa15c0ce9d0 msgr2=0x7fa15c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 shutdown_connections 2024-05-08T22:33:12.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.492+0000 7fa15bfff700 1 -- 172.21.15.179:0/1982891712 wait complete. 2024-05-08T22:33:12.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:33:12.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:33:12.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 Processor -- start 2024-05-08T22:33:12.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 -- start start 2024-05-08T22:33:12.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 0x7fc30015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30006ddf0 con 0x7fc300157be0 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30006e7c0 con 0x7fc300157800 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30595c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30006d1e0 con 0x7fc30006cd50 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 0x7fc30015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30495a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30495a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:48196/0 (socket says 172.21.15.179:48196) 2024-05-08T22:33:12.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30495a700 1 -- 172.21.15.179:0/3125397222 learned_addr learned my addr 172.21.15.179:0/3125397222 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 -- 172.21.15.179:0/3125397222 >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 msgr2=0x7fc30006e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 --2- 172.21.15.179:0/3125397222 >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 -- 172.21.15.179:0/3125397222 >> v1:172.21.15.195:6789/0 conn(0x7fc300157be0 legacy=0x7fc3000cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 -- 172.21.15.179:0/3125397222 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30006d4c0 con 0x7fc30006cd50 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30495a700 1 --2- 172.21.15.179:0/3125397222 >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc30515b700 1 --2- 172.21.15.179:0/3125397222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 0x7fc30015da30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009d90 tx=0x7fc2f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc4381ef975755 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc2ff7fe700 1 -- 172.21.15.179:0/3125397222 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800e070 con 0x7fc30006cd50 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.620+0000 7fc2ff7fe700 1 -- 172.21.15.179:0/3125397222 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f800bb20 con 0x7fc30006cd50 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc2ff7fe700 1 -- 172.21.15.179:0/3125397222 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8012430 con 0x7fc30006cd50 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- 172.21.15.179:0/3125397222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 msgr2=0x7fc30015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- 172.21.15.179:0/3125397222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 0x7fc30015da30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009d90 tx=0x7fc2f800b330 comp rx=0 tx=0).stop 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- 172.21.15.179:0/3125397222 shutdown_connections 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- 172.21.15.179:0/3125397222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc30006cd50 0x7fc30015da30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- 172.21.15.179:0/3125397222 >> v2:172.21.15.195:3300/0 conn(0x7fc300157800 0x7fc30006e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- 172.21.15.179:0/3125397222 >> 172.21.15.179:0/3125397222 conn(0x7fc3000ce9a0 msgr2=0x7fc3000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- 172.21.15.179:0/3125397222 shutdown_connections 2024-05-08T22:33:12.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- 172.21.15.179:0/3125397222 wait complete. 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 Processor -- start 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- start start 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc300157800 0x7fc3000aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc300153840 0x7fc3001e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001549a0 con 0x7fc3000ac970 2024-05-08T22:33:12.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000ad340 con 0x7fc300153840 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30595c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc300153d20 con 0x7fc300157800 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54338/0 (socket says 172.21.15.179:54338) 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 -- 172.21.15.179:0/1976046898 learned_addr learned my addr 172.21.15.179:0/1976046898 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30495a700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc300157800 0x7fc3000aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc2fffff700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc300153840 0x7fc3001e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 -- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc300157800 msgr2=0x7fc3000aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc300157800 0x7fc3000aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 -- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc300153840 msgr2=0x7fc3001e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc300153840 0x7fc3001e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 -- 172.21.15.179:0/1976046898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009570 con 0x7fc3000ac970 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.624+0000 7fc30515b700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8014040 tx=0x7fc2f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8012430 con 0x7fc3000ac970 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8020070 con 0x7fc3000ac970 2024-05-08T22:33:12.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f8009a70 con 0x7fc3000ac970 2024-05-08T22:33:12.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004280 con 0x7fc3000ac970 2024-05-08T22:33:12.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3001542e0 con 0x7fc3000ac970 2024-05-08T22:33:12.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300154ae0 con 0x7fc3000ac970 2024-05-08T22:33:12.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2f80043e0 con 0x7fc3000ac970 2024-05-08T22:33:12.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.628+0000 7fc2fd7fa700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 0x7fc2ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:12.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.632+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2f8022070 con 0x7fc3000ac970 2024-05-08T22:33:12.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.632+0000 7fc30495a700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 0x7fc2ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:12.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.632+0000 7fc30495a700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 0x7fc2ec068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006d50 tx=0x7fc2f00092e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:12.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.636+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc2f8053120 con 0x7fc3000ac970 2024-05-08T22:33:12.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:12.808+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc30004e770 con 0x7fc3000ac970 2024-05-08T22:33:13.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.500+0000 7fc2fd7fa700 1 -- 172.21.15.179:0/1976046898 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc2f8056740 con 0x7fc3000ac970 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 msgr2=0x7fc2ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 0x7fc2ec068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006d50 tx=0x7fc2f00092e0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 msgr2=0x7fc3000ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8014040 tx=0x7fc2f8004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 shutdown_connections 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc2ec066730 0x7fc2ec068bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc300153840 0x7fc3001e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3000ac970 0x7fc3000ace00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 --2- 172.21.15.179:0/1976046898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc300157800 0x7fc3000aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 >> 172.21.15.179:0/1976046898 conn(0x7fc3000ce9a0 msgr2=0x7fc3000af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:13.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 shutdown_connections 2024-05-08T22:33:13.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.508+0000 7fc30595c700 1 -- 172.21.15.179:0/1976046898 wait complete. 2024-05-08T22:33:13.513 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:33:13.526 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T22:33:13.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 Processor -- start 2024-05-08T22:33:13.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 -- start start 2024-05-08T22:33:13.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f48501544e0 0x7f485006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 0x7f485015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500b4650 con 0x7f485015e2b0 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48500cd6e0 con 0x7f48501544e0 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485006db50 con 0x7f485015d8f0 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484ed9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f485015e2b0 0x7f485015a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58668/0 (socket says 172.21.15.179:58668) 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484ed9d700 1 -- 172.21.15.179:0/325240555 learned_addr learned my addr 172.21.15.179:0/325240555 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 --2- 172.21.15.179:0/325240555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 0x7f485015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484e59c700 1 --2- 172.21.15.179:0/325240555 >> v2:172.21.15.195:3300/0 conn(0x7f48501544e0 0x7f485006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484d59a700 1 -- 172.21.15.179:0/325240555 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083363655 0 0) 0x7f48500b4650 con 0x7f485015e2b0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484d59a700 1 -- 172.21.15.179:0/325240555 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c0035d0 con 0x7f485015e2b0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 -- 172.21.15.179:0/325240555 >> v2:172.21.15.195:3300/0 conn(0x7f48501544e0 msgr2=0x7f485006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 --2- 172.21.15.179:0/325240555 >> v2:172.21.15.195:3300/0 conn(0x7f48501544e0 0x7f485006ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 -- 172.21.15.179:0/325240555 >> v1:172.21.15.179:6789/0 conn(0x7f485015e2b0 legacy=0x7f485015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 -- 172.21.15.179:0/325240555 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485015e740 con 0x7f485015d8f0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.648+0000 7f484dd9b700 1 --2- 172.21.15.179:0/325240555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 0x7f485015dd70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4838009850 tx=0x7f4838009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefe62bf42140f7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484d59a700 1 -- 172.21.15.179:0/325240555 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838011830 con 0x7f485015d8f0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484d59a700 1 -- 172.21.15.179:0/325240555 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838011990 con 0x7f485015d8f0 2024-05-08T22:33:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484d59a700 1 -- 172.21.15.179:0/325240555 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838011d40 con 0x7f485015d8f0 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- 172.21.15.179:0/325240555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 msgr2=0x7f485015dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- 172.21.15.179:0/325240555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 0x7f485015dd70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4838009850 tx=0x7f4838009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- 172.21.15.179:0/325240555 shutdown_connections 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- 172.21.15.179:0/325240555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015d8f0 0x7f485015dd70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- 172.21.15.179:0/325240555 >> v2:172.21.15.195:3300/0 conn(0x7f48501544e0 0x7f485006ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- 172.21.15.179:0/325240555 >> 172.21.15.179:0/325240555 conn(0x7f48500b18d0 msgr2=0x7f48500b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- 172.21.15.179:0/325240555 shutdown_connections 2024-05-08T22:33:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- 172.21.15.179:0/325240555 wait complete. 2024-05-08T22:33:13.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 Processor -- start 2024-05-08T22:33:13.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- start start 2024-05-08T22:33:13.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48501544e0 0x7f48500b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015e2b0 0x7f485015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500b0e10 con 0x7f485015d8f0 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500b0440 con 0x7f48501544e0 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48500af870 con 0x7f485015e2b0 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484ed9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015e2b0 0x7f485015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54346/0 (socket says 172.21.15.179:54346) 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 -- 172.21.15.179:0/3807900650 learned_addr learned my addr 172.21.15.179:0/3807900650 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484e59c700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48501544e0 0x7f48500b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 -- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015e2b0 msgr2=0x7f485015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015e2b0 0x7f485015ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 -- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48501544e0 msgr2=0x7f48500b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48501544e0 0x7f48500b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.652+0000 7f484dd9b700 1 -- 172.21.15.179:0/3807900650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380094b0 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f484dd9b700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48380090b0 tx=0x7f4838004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838004380 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48500afb10 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483801f070 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380036c0 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48500b00d0 con 0x7f485015d8f0 2024-05-08T22:33:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485004e700 con 0x7f485015d8f0 2024-05-08T22:33:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4838021030 con 0x7f485015d8f0 2024-05-08T22:33:13.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.656+0000 7f4846ffd700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 0x7f4830068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:13.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.660+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4838085210 con 0x7f485015d8f0 2024-05-08T22:33:13.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.660+0000 7f484e59c700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 0x7f4830068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.660+0000 7f484e59c700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 0x7f4830068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4840011340 tx=0x7f484000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:13.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.668+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4838052890 con 0x7f485015d8f0 2024-05-08T22:33:13.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:13.840+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f48500b0580 con 0x7f485015d8f0 2024-05-08T22:33:14.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.504+0000 7f4846ffd700 1 -- 172.21.15.179:0/3807900650 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4838006ce0 con 0x7f485015d8f0 2024-05-08T22:33:14.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 msgr2=0x7f4830068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 0x7f4830068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4840011340 tx=0x7f484000b040 comp rx=0 tx=0).stop 2024-05-08T22:33:14.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 msgr2=0x7f485013e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48380090b0 tx=0x7f4838004770 comp rx=0 tx=0).stop 2024-05-08T22:33:14.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 shutdown_connections 2024-05-08T22:33:14.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4830066730 0x7f4830068bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f485015e2b0 0x7f485015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f485015d8f0 0x7f485013e520 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 --2- 172.21.15.179:0/3807900650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48501544e0 0x7f48500b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.524+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 >> 172.21.15.179:0/3807900650 conn(0x7f48500b18d0 msgr2=0x7f48500b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:14.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.528+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 shutdown_connections 2024-05-08T22:33:14.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.528+0000 7f484f59e700 1 -- 172.21.15.179:0/3807900650 wait complete. 2024-05-08T22:33:14.531 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:33:14.543 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:33:14.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.652+0000 7f8ae759e700 1 Processor -- start 2024-05-08T22:33:14.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.652+0000 7f8ae759e700 1 -- start start 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ae80b2ca0 0x7f8ae806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80b22d0 con 0x7f8ae8142fd0 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80b19e0 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815e640 con 0x7f8ae8142890 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ae80b2ca0 0x7f8ae806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ae80b2ca0 0x7f8ae806cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54352/0 (socket says 172.21.15.179:54352) 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 -- 172.21.15.179:0/1389326940 learned_addr learned my addr 172.21.15.179:0/1389326940 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae659c700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 -- 172.21.15.179:0/1389326940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 msgr2=0x7f8ae80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae80b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 -- 172.21.15.179:0/1389326940 >> v1:172.21.15.195:6789/0 conn(0x7f8ae8142fd0 legacy=0x7f8ae815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 -- 172.21.15.179:0/1389326940 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80b1c80 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae5d9b700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae806cda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e50 tx=0x7f8ad000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19821184ee91c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae559a700 1 -- 172.21.15.179:0/1389326940 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000e070 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae559a700 1 -- 172.21.15.179:0/1389326940 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0003a40 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae559a700 1 -- 172.21.15.179:0/1389326940 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0003df0 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- 172.21.15.179:0/1389326940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 msgr2=0x7f8ae806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae806cda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e50 tx=0x7f8ad000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- 172.21.15.179:0/1389326940 shutdown_connections 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae806cda0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 --2- 172.21.15.179:0/1389326940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- 172.21.15.179:0/1389326940 >> 172.21.15.179:0/1389326940 conn(0x7f8ae80cea30 msgr2=0x7f8ae80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- 172.21.15.179:0/1389326940 shutdown_connections 2024-05-08T22:33:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.656+0000 7f8ae759e700 1 -- 172.21.15.179:0/1389326940 wait complete. 2024-05-08T22:33:14.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 Processor -- start 2024-05-08T22:33:14.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 -- start start 2024-05-08T22:33:14.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae8135860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae8142fd0 0x7f8ae8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8132e90 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8133170 con 0x7f8ae8142fd0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8133450 con 0x7f8ae8142890 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54364/0 (socket says 172.21.15.179:54364) 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae659c700 1 -- 172.21.15.179:0/762169834 learned_addr learned my addr 172.21.15.179:0/762169834 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.660+0000 7f8ae5d9b700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae8135860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae6d9d700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae8142fd0 0x7f8ae8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 -- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 msgr2=0x7f8ae8135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae8135860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 -- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae8142fd0 msgr2=0x7f8ae8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae8142fd0 0x7f8ae8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 -- 172.21.15.179:0/762169834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009b30 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae659c700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e760 tx=0x7f8ad800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8013610 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8013770 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8133730 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8013ae0 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8137390 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae804e700 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.664+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ad8013c40 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.668+0000 7f8adeffd700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 0x7f8ac8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:14.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.668+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8ad8083d20 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.668+0000 7f8ae5d9b700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 0x7f8ac8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:14.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.668+0000 7f8ae5d9b700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 0x7f8ac8068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b410 tx=0x7f8ad0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.676+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ad80510f0 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.884+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8ae813b9a0 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.896+0000 7f8adeffd700 1 -- 172.21.15.179:0/762169834 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8ad8021020 con 0x7f8ae80b2ca0 2024-05-08T22:33:14.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.900+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 msgr2=0x7f8ac8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.900+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 0x7f8ac8068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b410 tx=0x7f8ad0009000 comp rx=0 tx=0).stop 2024-05-08T22:33:14.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.900+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 msgr2=0x7f8ae8138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:14.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.900+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e760 tx=0x7f8ad800ea70 comp rx=0 tx=0).stop 2024-05-08T22:33:14.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 shutdown_connections 2024-05-08T22:33:14.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac8066730 0x7f8ac8068bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae8142fd0 0x7f8ae8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae8142890 0x7f8ae8135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 --2- 172.21.15.179:0/762169834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae80b2ca0 0x7f8ae8138340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:14.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 >> 172.21.15.179:0/762169834 conn(0x7f8ae80cea30 msgr2=0x7f8ae806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:14.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 shutdown_connections 2024-05-08T22:33:14.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:14.908+0000 7f8ae759e700 1 -- 172.21.15.179:0/762169834 wait complete. 2024-05-08T22:33:14.921 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T22:33:14.933 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T22:33:14.933 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T22:33:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 Processor -- start 2024-05-08T22:33:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 -- start start 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 0x7f53880d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880d23e0 con 0x7f5388160f30 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538806ad50 con 0x7f538815d2d0 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538d3f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538806b030 con 0x7f5388161ad0 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538cbf5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5388160f30 0x7f538806b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58670/0 (socket says 172.21.15.179:58670) 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f538cbf5700 1 -- 172.21.15.179:0/1461757003 learned_addr learned my addr 172.21.15.179:0/1461757003 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f53877fe700 1 --2- 172.21.15.179:0/1461757003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 0x7f53880d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82617071 0 0) 0x7f53880d23e0 con 0x7f5388160f30 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53700035d0 con 0x7f5388160f30 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193311631 0 0) 0x7f538806ad50 con 0x7f538815d2d0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53880d23e0 con 0x7f538815d2d0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f53877fe700 1 -- 172.21.15.179:0/1461757003 >> v1:172.21.15.195:6789/0 conn(0x7f538815d2d0 legacy=0x7f53880b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2108214690 0 0) 0x7f53700035d0 con 0x7f5388160f30 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f53877fe700 1 -- 172.21.15.179:0/1461757003 >> v1:172.21.15.179:6789/0 conn(0x7f5388160f30 legacy=0x7f538806b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f53877fe700 1 -- 172.21.15.179:0/1461757003 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538806b310 con 0x7f5388161ad0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f53877fe700 1 --2- 172.21.15.179:0/1461757003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 0x7f53880d1b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f537c009880 tx=0x7f537c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471d32ca7f6f9d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800019f0 con 0x7f5388161ad0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c004030 con 0x7f5388161ad0 2024-05-08T22:33:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.044+0000 7f5386ffd700 1 -- 172.21.15.179:0/1461757003 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0043c0 con 0x7f5388161ad0 2024-05-08T22:33:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- 172.21.15.179:0/1461757003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 msgr2=0x7f53880d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1461757003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 0x7f53880d1b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f537c009880 tx=0x7f537c009b90 comp rx=0 tx=0).stop 2024-05-08T22:33:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- 172.21.15.179:0/1461757003 shutdown_connections 2024-05-08T22:33:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1461757003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388161ad0 0x7f53880d1b00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- 172.21.15.179:0/1461757003 >> 172.21.15.179:0/1461757003 conn(0x7f53880d2d90 msgr2=0x7f53880d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- 172.21.15.179:0/1461757003 shutdown_connections 2024-05-08T22:33:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- 172.21.15.179:0/1461757003 wait complete. 2024-05-08T22:33:15.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 Processor -- start 2024-05-08T22:33:15.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.048+0000 7f538d3f6700 1 -- start start 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f538815d2d0 0x7f5388147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5388160f30 0x7f5388157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538815fa70 con 0x7f5388161ad0 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538815fd50 con 0x7f5388160f30 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5388160030 con 0x7f538815d2d0 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:15.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f53877fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5388160f30 0x7f5388157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:15.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54374/0 (socket says 172.21.15.179:54374) 2024-05-08T22:33:15.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 -- 172.21.15.179:0/1002994757 learned_addr learned my addr 172.21.15.179:0/1002994757 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:15.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f5387fff700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f538815d2d0 0x7f5388147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 -- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f538815d2d0 msgr2=0x7f5388147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f538815d2d0 0x7f5388147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 -- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5388160f30 msgr2=0x7f5388157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5388160f30 0x7f5388157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 -- 172.21.15.179:0/1002994757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0094b0 con 0x7f5388161ad0 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538cbf5700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5380001700 tx=0x7f538000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000ab00 con 0x7f5388161ad0 2024-05-08T22:33:15.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000ac60 con 0x7f5388161ad0 2024-05-08T22:33:15.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5388160310 con 0x7f5388161ad0 2024-05-08T22:33:15.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800070d0 con 0x7f5388161ad0 2024-05-08T22:33:15.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.052+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538815f780 con 0x7f5388161ad0 2024-05-08T22:33:15.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5388074460 con 0x7f5388161ad0 2024-05-08T22:33:15.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5380007250 con 0x7f5388161ad0 2024-05-08T22:33:15.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f5384ff9700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 0x7f5374068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:15.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5380083d50 con 0x7f5388161ad0 2024-05-08T22:33:15.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f5387fff700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 0x7f5374068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:15.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.056+0000 7f5387fff700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 0x7f5374068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53780011e0 tx=0x7f5378009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:15.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.064+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5380050f90 con 0x7f5388161ad0 2024-05-08T22:33:15.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.240+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f53880cf0f0 con 0x7f5388161ad0 2024-05-08T22:33:15.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.900+0000 7f5384ff9700 1 -- 172.21.15.179:0/1002994757 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5380018070 con 0x7f5388161ad0 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 msgr2=0x7f5374068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 0x7f5374068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53780011e0 tx=0x7f5378009040 comp rx=0 tx=0).stop 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 msgr2=0x7f53881578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5380001700 tx=0x7f538000cf70 comp rx=0 tx=0).stop 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 shutdown_connections 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5374066730 0x7f5374068bf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5388160f30 0x7f5388157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5388161ad0 0x7f53881578c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 --2- 172.21.15.179:0/1002994757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f538815d2d0 0x7f5388147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 >> 172.21.15.179:0/1002994757 conn(0x7f53880d2d90 msgr2=0x7f53880caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 shutdown_connections 2024-05-08T22:33:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:15.904+0000 7f538d3f6700 1 -- 172.21.15.179:0/1002994757 wait complete. 2024-05-08T22:33:15.924 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:33:16.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 Processor -- start 2024-05-08T22:33:16.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 -- start start 2024-05-08T22:33:16.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72a4070130 0x7f72a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f72a40b4190 0x7f72a4161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a4161d80 con 0x7f72a4161460 2024-05-08T22:33:16.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a406d720 con 0x7f72a4070130 2024-05-08T22:33:16.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72aa3b0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a406cb10 con 0x7f72a40b4190 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72a8bad700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72a4070130 0x7f72a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72a8bad700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72a4070130 0x7f72a406c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48148/0 (socket says 172.21.15.179:48148) 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72a8bad700 1 -- 172.21.15.179:0/550705323 learned_addr learned my addr 172.21.15.179:0/550705323 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.048+0000 7f72a9baf700 1 --2- 172.21.15.179:0/550705323 >> v2:172.21.15.195:3300/0 conn(0x7f72a40b4190 0x7f72a4161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 -- 172.21.15.179:0/550705323 >> v2:172.21.15.195:3300/0 conn(0x7f72a40b4190 msgr2=0x7f72a4161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 --2- 172.21.15.179:0/550705323 >> v2:172.21.15.195:3300/0 conn(0x7f72a40b4190 0x7f72a4161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 -- 172.21.15.179:0/550705323 >> v1:172.21.15.195:6789/0 conn(0x7f72a4161460 legacy=0x7f72a40b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 -- 172.21.15.179:0/550705323 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a406cdf0 con 0x7f72a4070130 2024-05-08T22:33:16.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 --2- 172.21.15.179:0/550705323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a4070130 0x7f72a406c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72a4071a10 tx=0x7f7294009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7399cef39f5e2bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:16.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f729bfff700 1 -- 172.21.15.179:0/550705323 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400b8a0 con 0x7f72a4070130 2024-05-08T22:33:16.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f729bfff700 1 -- 172.21.15.179:0/550705323 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400ba00 con 0x7f72a4070130 2024-05-08T22:33:16.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f729bfff700 1 -- 172.21.15.179:0/550705323 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400bd90 con 0x7f72a4070130 2024-05-08T22:33:16.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- 172.21.15.179:0/550705323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a4070130 msgr2=0x7f72a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:16.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/550705323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a4070130 0x7f72a406c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72a4071a10 tx=0x7f7294009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- 172.21.15.179:0/550705323 shutdown_connections 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/550705323 >> v2:172.21.15.195:3300/0 conn(0x7f72a40b4190 0x7f72a4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/550705323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a4070130 0x7f72a406c510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- 172.21.15.179:0/550705323 >> 172.21.15.179:0/550705323 conn(0x7f72a406f4f0 msgr2=0x7f72a406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- 172.21.15.179:0/550705323 shutdown_connections 2024-05-08T22:33:16.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- 172.21.15.179:0/550705323 wait complete. 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 Processor -- start 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- start start 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a4161460 0x7f72a41f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a41f7590 0x7f72a41fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41f8020 con 0x7f72a40b4190 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4141f80 con 0x7f72a4161460 2024-05-08T22:33:16.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72aa3b0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41f7a60 con 0x7f72a41f7590 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a93ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a93ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48154/0 (socket says 172.21.15.179:48154) 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a93ae700 1 -- 172.21.15.179:0/714224154 learned_addr learned my addr 172.21.15.179:0/714224154 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a8bad700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a4161460 0x7f72a41f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.052+0000 7f72a9baf700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a41f7590 0x7f72a41fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 -- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a41f7590 msgr2=0x7f72a41fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a41f7590 0x7f72a41fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 -- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a4161460 msgr2=0x7f72a41f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a4161460 0x7f72a41f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 -- 172.21.15.179:0/714224154 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009570 con 0x7f72a40b4190 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a93ae700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007fd0 tx=0x7f72a000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00052d0 con 0x7f72a40b4190 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a000ce90 con 0x7f72a40b4190 2024-05-08T22:33:16.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a41f7d40 con 0x7f72a40b4190 2024-05-08T22:33:16.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00036a0 con 0x7f72a40b4190 2024-05-08T22:33:16.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a4153750 con 0x7f72a40b4190 2024-05-08T22:33:16.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a404e700 con 0x7f72a40b4190 2024-05-08T22:33:16.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72a0003800 con 0x7f72a40b4190 2024-05-08T22:33:16.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 0x7f7288068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:16.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f72a00833e0 con 0x7f72a40b4190 2024-05-08T22:33:16.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.056+0000 7f72a8bad700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 0x7f7288068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:16.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.060+0000 7f72a8bad700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 0x7f7288068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7294000c00 tx=0x7f7294008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:16.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.064+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72a0050670 con 0x7f72a40b4190 2024-05-08T22:33:16.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:16.232+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72a4161d80 con 0x7f72a40b4190 2024-05-08T22:33:17.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.915+0000 7f7299ffb700 1 -- 172.21.15.179:0/714224154 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7f72a4161d80 con 0x7f72a40b4190 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 msgr2=0x7f7288068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 0x7f7288068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7294000c00 tx=0x7f7294008030 comp rx=0 tx=0).stop 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 msgr2=0x7f72a4141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007fd0 tx=0x7f72a000c6e0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 shutdown_connections 2024-05-08T22:33:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7288066730 0x7f7288068bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a4161460 0x7f72a41f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a41f7590 0x7f72a41fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 --2- 172.21.15.179:0/714224154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a40b4190 0x7f72a4141840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.927+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 >> 172.21.15.179:0/714224154 conn(0x7f72a406f4f0 msgr2=0x7f72a415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:17.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.931+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 shutdown_connections 2024-05-08T22:33:17.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:17.931+0000 7f72aa3b0700 1 -- 172.21.15.179:0/714224154 wait complete. 2024-05-08T22:33:17.936 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data_ec' created 2024-05-08T22:33:17.949 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T22:33:18.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 Processor -- start 2024-05-08T22:33:18.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 -- start start 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 0x7f58d806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58d80ce5a0 0x7f58d80d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d815d870 con 0x7f58d806c2e0 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d80d2740 con 0x7f58d80ce5a0 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80b7300 con 0x7f58d815d490 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58d80ce5a0 0x7f58d80d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58d80ce5a0 0x7f58d80d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48162/0 (socket says 172.21.15.179:48162) 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 -- 172.21.15.179:0/1648574785 learned_addr learned my addr 172.21.15.179:0/1648574785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58d7fff700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 0x7f58d806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 -- 172.21.15.179:0/1648574785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 msgr2=0x7f58d806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 0x7f58d806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 -- 172.21.15.179:0/1648574785 >> v1:172.21.15.195:6789/0 conn(0x7f58d806c2e0 legacy=0x7f58d80cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 -- 172.21.15.179:0/1648574785 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d806b240 con 0x7f58d80ce5a0 2024-05-08T22:33:18.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58d7fff700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 0x7f58d806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:18.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dc8a6700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d80ce5a0 0x7f58d80d2200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e50 tx=0x7f58d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806002ee3d42f205 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:18.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58d6ffd700 1 -- 172.21.15.179:0/1648574785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000e070 con 0x7f58d80ce5a0 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58d6ffd700 1 -- 172.21.15.179:0/1648574785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000b9c0 con 0x7f58d80ce5a0 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58d6ffd700 1 -- 172.21.15.179:0/1648574785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000bd70 con 0x7f58d80ce5a0 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.063+0000 7f58dd0a7700 1 -- 172.21.15.179:0/1648574785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d80ce5a0 msgr2=0x7f58d80d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d80ce5a0 0x7f58d80d2200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e50 tx=0x7f58d000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 -- 172.21.15.179:0/1648574785 shutdown_connections 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d80ce5a0 0x7f58d80d2200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/1648574785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d815d490 0x7f58d806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 -- 172.21.15.179:0/1648574785 >> 172.21.15.179:0/1648574785 conn(0x7f58d80b4940 msgr2=0x7f58d80b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 -- 172.21.15.179:0/1648574785 shutdown_connections 2024-05-08T22:33:18.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 -- 172.21.15.179:0/1648574785 wait complete. 2024-05-08T22:33:18.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 Processor -- start 2024-05-08T22:33:18.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.067+0000 7f58dd0a7700 1 -- start start 2024-05-08T22:33:18.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806c2e0 0x7f58d814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d80ce5a0 0x7f58d80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cc720 con 0x7f58d815d490 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cca00 con 0x7f58d806c2e0 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80ccce0 con 0x7f58d80ce5a0 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48164/0 (socket says 172.21.15.179:48164) 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 -- 172.21.15.179:0/2300646452 learned_addr learned my addr 172.21.15.179:0/2300646452 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58d7fff700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806c2e0 0x7f58d814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58d77fe700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d80ce5a0 0x7f58d80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 -- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d80ce5a0 msgr2=0x7f58d80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d80ce5a0 0x7f58d80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 -- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806c2e0 msgr2=0x7f58d814f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806c2e0 0x7f58d814f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 -- 172.21.15.179:0/2300646452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009b30 con 0x7f58d815d490 2024-05-08T22:33:18.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dc8a6700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58d0012000 tx=0x7f58d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:18.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0004800 con 0x7f58d815d490 2024-05-08T22:33:18.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d00099b0 con 0x7f58d815d490 2024-05-08T22:33:18.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d80ccfc0 con 0x7f58d815d490 2024-05-08T22:33:18.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d001c680 con 0x7f58d815d490 2024-05-08T22:33:18.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.071+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d814f780 con 0x7f58d815d490 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58d001c7e0 con 0x7f58d815d490 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d804e770 con 0x7f58d815d490 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58d4ff9700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 0x7f58c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f58d0085c50 con 0x7f58d815d490 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58d7fff700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 0x7f58c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:18.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.075+0000 7f58d7fff700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 0x7f58c4068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58c80099f0 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:18.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.083+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f58d008b020 con 0x7f58d815d490 2024-05-08T22:33:18.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.251+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f58d80d0e50 con 0x7f58d815d490 2024-05-08T22:33:18.582 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:33:18.578+0000 7f1ac9d4f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:33:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.923+0000 7f58d4ff9700 1 -- 172.21.15.179:0/2300646452 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58d0052d80 con 0x7f58d815d490 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 msgr2=0x7f58c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 0x7f58c4068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58c80099f0 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 msgr2=0x7f58d80a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58d0012000 tx=0x7f58d0004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 shutdown_connections 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58c4066730 0x7f58c4068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58d815d490 0x7f58d80a9de0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58d80ce5a0 0x7f58d80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 --2- 172.21.15.179:0/2300646452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58d806c2e0 0x7f58d814f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:18.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.943+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 >> 172.21.15.179:0/2300646452 conn(0x7f58d80b4940 msgr2=0x7f58d80b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:18.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.947+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 shutdown_connections 2024-05-08T22:33:18.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:18.947+0000 7f58dd0a7700 1 -- 172.21.15.179:0/2300646452 wait complete. 2024-05-08T22:33:18.952 INFO:teuthology.orchestra.run.smithi179.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T22:33:18.964 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T22:33:19.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.071+0000 7f141e865700 1 Processor -- start 2024-05-08T22:33:19.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.071+0000 7f141e865700 1 -- start start 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.071+0000 7f141e865700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f141815b490 0x7f14180afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180b5460 con 0x7f14180b0080 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180b04d0 con 0x7f141815b0b0 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b4170 con 0x7f141815b490 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141d062700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f141815b490 0x7f14180afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e064700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f14180b0080 0x7f141806abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38744/0 (socket says 172.21.15.179:38744) 2024-05-08T22:33:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e064700 1 -- 172.21.15.179:0/110771601 learned_addr learned my addr 172.21.15.179:0/110771601 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141d062700 1 -- 172.21.15.179:0/110771601 >> v1:172.21.15.195:6789/0 conn(0x7f141815b0b0 legacy=0x7f14180ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141c861700 1 -- 172.21.15.179:0/110771601 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819246110 0 0) 0x7f1414001110 con 0x7f14180b0080 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141d062700 1 -- 172.21.15.179:0/110771601 >> v1:172.21.15.179:6789/0 conn(0x7f14180b0080 legacy=0x7f141806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141d062700 1 -- 172.21.15.179:0/110771601 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180b43c0 con 0x7f141815b490 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141d062700 1 --2- 172.21.15.179:0/110771601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b490 0x7f14180afb40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1408009850 tx=0x7f1408009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31641ee802c25d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141c861700 1 -- 172.21.15.179:0/110771601 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408011830 con 0x7f141815b490 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141c861700 1 -- 172.21.15.179:0/110771601 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408011990 con 0x7f141815b490 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141c861700 1 -- 172.21.15.179:0/110771601 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408011d20 con 0x7f141815b490 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- 172.21.15.179:0/110771601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b490 msgr2=0x7f14180afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 --2- 172.21.15.179:0/110771601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b490 0x7f14180afb40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1408009850 tx=0x7f1408009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- 172.21.15.179:0/110771601 shutdown_connections 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 --2- 172.21.15.179:0/110771601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b490 0x7f14180afb40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- 172.21.15.179:0/110771601 >> 172.21.15.179:0/110771601 conn(0x7f1418098a70 msgr2=0x7f14180b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- 172.21.15.179:0/110771601 shutdown_connections 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.075+0000 7f141e865700 1 -- 172.21.15.179:0/110771601 wait complete. 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 Processor -- start 2024-05-08T22:33:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- start start 2024-05-08T22:33:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b0080 0x7f141814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f141815b490 0x7f1418158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141809a310 con 0x7f141815b0b0 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141809a5f0 con 0x7f141815b490 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141809a8d0 con 0x7f14180b0080 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48186/0 (socket says 172.21.15.179:48186) 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 -- 172.21.15.179:0/2262562889 learned_addr learned my addr 172.21.15.179:0/2262562889 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e064700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f141815b490 0x7f1418158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d863700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b0080 0x7f141814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 -- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b0080 msgr2=0x7f141814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b0080 0x7f141814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 -- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f141815b490 msgr2=0x7f1418158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f141815b490 0x7f1418158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 -- 172.21.15.179:0/2262562889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080094b0 con 0x7f141815b0b0 2024-05-08T22:33:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141d062700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1408014040 tx=0x7f1408004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800ba70 con 0x7f141815b0b0 2024-05-08T22:33:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140801e070 con 0x7f141815b0b0 2024-05-08T22:33:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141809abb0 con 0x7f141815b0b0 2024-05-08T22:33:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800bdf0 con 0x7f141815b0b0 2024-05-08T22:33:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.079+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14181540e0 con 0x7f141815b0b0 2024-05-08T22:33:19.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141809b0d0 con 0x7f141815b0b0 2024-05-08T22:33:19.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f140802d440 con 0x7f141815b0b0 2024-05-08T22:33:19.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f140e7fc700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 0x7f13fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f14080856f0 con 0x7f141815b0b0 2024-05-08T22:33:19.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f141d863700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 0x7f13fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.083+0000 7f141d863700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 0x7f13fc068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14100012e0 tx=0x7f1410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.091+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f141809b0d0 con 0x7f141815b0b0 2024-05-08T22:33:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.299+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f141809b0d0 con 0x7f141815b0b0 2024-05-08T22:33:19.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.307+0000 7f140e7fc700 1 -- 172.21.15.179:0/2262562889 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f141809b0d0 con 0x7f141815b0b0 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 msgr2=0x7f13fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 0x7f13fc068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14100012e0 tx=0x7f1410009040 comp rx=0 tx=0).stop 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 msgr2=0x7f1418099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1408014040 tx=0x7f1408004770 comp rx=0 tx=0).stop 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 shutdown_connections 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13fc066730 0x7f13fc068bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f141815b490 0x7f1418158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f141815b0b0 0x7f1418099850 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 --2- 172.21.15.179:0/2262562889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b0080 0x7f141814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 >> 172.21.15.179:0/2262562889 conn(0x7f1418098a70 msgr2=0x7f141806b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 shutdown_connections 2024-05-08T22:33:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.311+0000 7f141e865700 1 -- 172.21.15.179:0/2262562889 wait complete. 2024-05-08T22:33:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:added data pool 4 to fsmap 2024-05-08T22:33:19.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:19.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.435+0000 7fdc3ffff700 1 Processor -- start 2024-05-08T22:33:19.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.435+0000 7fdc3ffff700 1 -- start start 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc40143640 0x7fdc4006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdc4015d970 0x7fdc4015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3ffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4015e300 con 0x7fdc400cea40 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3ffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4006ce80 con 0x7fdc40143640 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc40143a20 con 0x7fdc4015d970 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc40143640 0x7fdc4006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc40143640 0x7fdc4006eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48190/0 (socket says 172.21.15.179:48190) 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 -- 172.21.15.179:0/898054776 learned_addr learned my addr 172.21.15.179:0/898054776 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:19.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3f7fe700 1 --2- 172.21.15.179:0/898054776 >> v2:172.21.15.195:3300/0 conn(0x7fdc4015d970 0x7fdc4015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 -- 172.21.15.179:0/898054776 >> v2:172.21.15.195:3300/0 conn(0x7fdc4015d970 msgr2=0x7fdc4015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:19.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 --2- 172.21.15.179:0/898054776 >> v2:172.21.15.195:3300/0 conn(0x7fdc4015d970 0x7fdc4015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 -- 172.21.15.179:0/898054776 >> v1:172.21.15.195:6789/0 conn(0x7fdc400cea40 legacy=0x7fdc400b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 -- 172.21.15.179:0/898054776 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400cf220 con 0x7fdc40143640 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3effd700 1 --2- 172.21.15.179:0/898054776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc4006eaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009910 tx=0x7fdc30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d3aa6cc97c679 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3e7fc700 1 -- 172.21.15.179:0/898054776 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30011830 con 0x7fdc40143640 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3e7fc700 1 -- 172.21.15.179:0/898054776 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc30011990 con 0x7fdc40143640 2024-05-08T22:33:19.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.439+0000 7fdc3e7fc700 1 -- 172.21.15.179:0/898054776 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30011d40 con 0x7fdc40143640 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- 172.21.15.179:0/898054776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 msgr2=0x7fdc4006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/898054776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc4006eaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009910 tx=0x7fdc30009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- 172.21.15.179:0/898054776 shutdown_connections 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/898054776 >> v2:172.21.15.195:3300/0 conn(0x7fdc4015d970 0x7fdc4015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/898054776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc4006eaf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- 172.21.15.179:0/898054776 >> 172.21.15.179:0/898054776 conn(0x7fdc400cfae0 msgr2=0x7fdc400cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- 172.21.15.179:0/898054776 shutdown_connections 2024-05-08T22:33:19.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- 172.21.15.179:0/898054776 wait complete. 2024-05-08T22:33:19.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 Processor -- start 2024-05-08T22:33:19.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.443+0000 7fdc3ffff700 1 -- start start 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc400cea40 0x7fdc401f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc4015d970 0x7fdc4015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400c5e70 con 0x7fdc40143640 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400c6150 con 0x7fdc400cea40 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400c6430 con 0x7fdc4015d970 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48202/0 (socket says 172.21.15.179:48202) 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 -- 172.21.15.179:0/3366868878 learned_addr learned my addr 172.21.15.179:0/3366868878 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3f7fe700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc4015d970 0x7fdc4015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3effd700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc400cea40 0x7fdc401f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 -- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc4015d970 msgr2=0x7fdc4015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc4015d970 0x7fdc4015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 -- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc400cea40 msgr2=0x7fdc401f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc400cea40 0x7fdc401f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 -- 172.21.15.179:0/3366868878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30009570 con 0x7fdc40143640 2024-05-08T22:33:19.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc37fff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c410 tx=0x7fdc2800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280052d0 con 0x7fdc40143640 2024-05-08T22:33:19.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28005430 con 0x7fdc40143640 2024-05-08T22:33:19.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc400c6710 con 0x7fdc40143640 2024-05-08T22:33:19.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280055a0 con 0x7fdc40143640 2024-05-08T22:33:19.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400c69f0 con 0x7fdc40143640 2024-05-08T22:33:19.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.447+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc400c8040 con 0x7fdc40143640 2024-05-08T22:33:19.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.451+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc280036a0 con 0x7fdc40143640 2024-05-08T22:33:19.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.451+0000 7fdc377fe700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 0x7fdc20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.451+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdc280833f0 con 0x7fdc40143640 2024-05-08T22:33:19.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.451+0000 7fdc3effd700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 0x7fdc20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.451+0000 7fdc3effd700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 0x7fdc20068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc30000c00 tx=0x7fdc30008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.459+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc400c8040 con 0x7fdc40143640 2024-05-08T22:33:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc4015e300 con 0x7fdc40143640 2024-05-08T22:33:19.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc377fe700 1 -- 172.21.15.179:0/3366868878 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4745 (secure 0 0 0) 0x7fdc4015e300 con 0x7fdc40143640 2024-05-08T22:33:19.670 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:19.671 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:19.306328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 msgr2=0x7fdc20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 0x7fdc20068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc30000c00 tx=0x7fdc30008030 comp rx=0 tx=0).stop 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 msgr2=0x7fdc400c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c410 tx=0x7fdc2800c720 comp rx=0 tx=0).stop 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 shutdown_connections 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc20066730 0x7fdc20068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc4015d970 0x7fdc4015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc40143640 0x7fdc400c5370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 --2- 172.21.15.179:0/3366868878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc400cea40 0x7fdc401f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 >> 172.21.15.179:0/3366868878 conn(0x7fdc400cfae0 msgr2=0x7fdc400d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 shutdown_connections 2024-05-08T22:33:19.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.667+0000 7fdc3ffff700 1 -- 172.21.15.179:0/3366868878 wait complete. 2024-05-08T22:33:19.675 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 7 2024-05-08T22:33:19.685 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:19.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 Processor -- start 2024-05-08T22:33:19.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 -- start start 2024-05-08T22:33:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff49c0ce580 0x7ff49c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c06afc0 con 0x7ff49c0b2120 2024-05-08T22:33:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c15d990 con 0x7ff49c0ce580 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff4a09eb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0d26f0 con 0x7ff49c15d5b0 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff49c0ce580 0x7ff49c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff49c0ce580 0x7ff49c06b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48214/0 (socket says 172.21.15.179:48214) 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 -- 172.21.15.179:0/1288647296 learned_addr learned my addr 172.21.15.179:0/1288647296 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49b7fe700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 -- 172.21.15.179:0/1288647296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 msgr2=0x7ff49c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 -- 172.21.15.179:0/1288647296 >> v1:172.21.15.195:6789/0 conn(0x7ff49c0b2120 legacy=0x7ff49c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 -- 172.21.15.179:0/1288647296 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0d2130 con 0x7ff49c0ce580 2024-05-08T22:33:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49bfff700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c06b450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff494009910 tx=0x7ff494009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1b76fe79d5f3ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.791+0000 7ff49a7fc700 1 -- 172.21.15.179:0/1288647296 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494011830 con 0x7ff49c0ce580 2024-05-08T22:33:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49a7fc700 1 -- 172.21.15.179:0/1288647296 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494011990 con 0x7ff49c0ce580 2024-05-08T22:33:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49a7fc700 1 -- 172.21.15.179:0/1288647296 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494011d40 con 0x7ff49c0ce580 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1288647296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 msgr2=0x7ff49c06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c06b450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff494009910 tx=0x7ff494009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1288647296 shutdown_connections 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c06b450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1288647296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1288647296 >> 172.21.15.179:0/1288647296 conn(0x7ff49c0b4940 msgr2=0x7ff49c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1288647296 shutdown_connections 2024-05-08T22:33:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1288647296 wait complete. 2024-05-08T22:33:19.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 Processor -- start 2024-05-08T22:33:19.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- start start 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 0x7ff49c15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c1f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1f4580 con 0x7ff49c0ce580 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c160500 con 0x7ff49c0b2120 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff4a09eb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c15ff40 con 0x7ff49c15d5b0 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48224/0 (socket says 172.21.15.179:48224) 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 -- 172.21.15.179:0/1575791572 learned_addr learned my addr 172.21.15.179:0/1575791572 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49b7fe700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 0x7ff49c15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49bfff700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c1f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 -- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 msgr2=0x7ff49c1f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c1f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 -- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 msgr2=0x7ff49c15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 0x7ff49c15f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49affd700 1 -- 172.21.15.179:0/1575791572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009570 con 0x7ff49c0ce580 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.795+0000 7ff49b7fe700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 0x7ff49c15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff49affd700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c500 tx=0x7ff48400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840052d0 con 0x7ff49c0ce580 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484005430 con 0x7ff49c0ce580 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c160220 con 0x7ff49c0ce580 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0cd380 con 0x7ff49c0ce580 2024-05-08T22:33:19.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840055a0 con 0x7ff49c0ce580 2024-05-08T22:33:19.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c1f46c0 con 0x7ff49c0ce580 2024-05-08T22:33:19.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4840038e0 con 0x7ff49c0ce580 2024-05-08T22:33:19.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 0x7ff47c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:19.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff484083830 con 0x7ff49c0ce580 2024-05-08T22:33:19.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.799+0000 7ff49b7fe700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 0x7ff47c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:19.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.803+0000 7ff49b7fe700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 0x7ff47c068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff49c072d00 tx=0x7ff48c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:19.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.807+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff49c1f46c0 con 0x7ff49c0ce580 2024-05-08T22:33:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.975+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff49c1f46c0 con 0x7ff49c0ce580 2024-05-08T22:33:19.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.979+0000 7ff493fff700 1 -- 172.21.15.179:0/1575791572 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7ff49c1f46c0 con 0x7ff49c0ce580 2024-05-08T22:33:19.983 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:19.984 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 msgr2=0x7ff47c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 0x7ff47c068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff49c072d00 tx=0x7ff48c00b040 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 msgr2=0x7ff49c1f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c500 tx=0x7ff48400c810 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 shutdown_connections 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff47c066730 0x7ff47c068bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff49c15d5b0 0x7ff49c1f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff49c0ce580 0x7ff49c1f64a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 --2- 172.21.15.179:0/1575791572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff49c0b2120 0x7ff49c15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 >> 172.21.15.179:0/1575791572 conn(0x7ff49c0b4940 msgr2=0x7ff49c15c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 shutdown_connections 2024-05-08T22:33:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:19.983+0000 7ff4a09eb700 1 -- 172.21.15.179:0/1575791572 wait complete. 2024-05-08T22:33:19.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:20.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 Processor -- start 2024-05-08T22:33:20.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 -- start start 2024-05-08T22:33:20.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff33406b6b0 0x7ff3340ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3340c9230 0x7ff3340cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33406dfa0 con 0x7ff33406abb0 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3340ca730 con 0x7ff33406b6b0 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff339ed2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33406d670 con 0x7ff3340c9230 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff33406b6b0 0x7ff3340ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff332ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3340c9230 0x7ff3340cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff33406b6b0 0x7ff3340ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48232/0 (socket says 172.21.15.179:48232) 2024-05-08T22:33:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 -- 172.21.15.179:0/2827043480 learned_addr learned my addr 172.21.15.179:0/2827043480 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 -- 172.21.15.179:0/2827043480 >> v2:172.21.15.195:3300/0 conn(0x7ff3340c9230 msgr2=0x7ff3340cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 --2- 172.21.15.179:0/2827043480 >> v2:172.21.15.195:3300/0 conn(0x7ff3340c9230 0x7ff3340cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 -- 172.21.15.179:0/2827043480 >> v1:172.21.15.195:6789/0 conn(0x7ff33406abb0 legacy=0x7ff33406ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 -- 172.21.15.179:0/2827043480 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33406d950 con 0x7ff33406b6b0 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3337fe700 1 --2- 172.21.15.179:0/2827043480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340ca120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff334067130 tx=0x7ff328009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103f68eba50528e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3327fc700 1 -- 172.21.15.179:0/2827043480 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328011830 con 0x7ff33406b6b0 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3327fc700 1 -- 172.21.15.179:0/2827043480 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328011990 con 0x7ff33406b6b0 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.107+0000 7ff3327fc700 1 -- 172.21.15.179:0/2827043480 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328011d20 con 0x7ff33406b6b0 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- 172.21.15.179:0/2827043480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 msgr2=0x7ff3340ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2827043480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340ca120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff334067130 tx=0x7ff328009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- 172.21.15.179:0/2827043480 shutdown_connections 2024-05-08T22:33:20.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2827043480 >> v2:172.21.15.195:3300/0 conn(0x7ff3340c9230 0x7ff3340cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2827043480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340ca120 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- 172.21.15.179:0/2827043480 >> 172.21.15.179:0/2827043480 conn(0x7ff3340cb660 msgr2=0x7ff3340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- 172.21.15.179:0/2827043480 shutdown_connections 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- 172.21.15.179:0/2827043480 wait complete. 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 Processor -- start 2024-05-08T22:33:20.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- start start 2024-05-08T22:33:20.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33406abb0 0x7ff3341f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3340c9230 0x7ff3340c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341f35a0 con 0x7ff33406b6b0 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340c6830 con 0x7ff33406abb0 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff339ed2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340c63a0 con 0x7ff3340c9230 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff332ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff332ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48240/0 (socket says 172.21.15.179:48240) 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff332ffd700 1 -- 172.21.15.179:0/2262497137 learned_addr learned my addr 172.21.15.179:0/2262497137 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff3337fe700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33406abb0 0x7ff3341f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.111+0000 7ff333fff700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3340c9230 0x7ff3340c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 -- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3340c9230 msgr2=0x7ff3340c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3340c9230 0x7ff3340c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 -- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33406abb0 msgr2=0x7ff3341f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33406abb0 0x7ff3341f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 -- 172.21.15.179:0/2262497137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328009570 con 0x7ff33406b6b0 2024-05-08T22:33:20.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff332ffd700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff324009d70 tx=0x7ff32400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400cd10 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff334158e80 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32400f040 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240146a0 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3340c5d50 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334067770 con 0x7ff33406b6b0 2024-05-08T22:33:20.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.115+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff324014800 con 0x7ff33406b6b0 2024-05-08T22:33:20.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.119+0000 7ff31bfff700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 0x7ff320068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.119+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff324084ed0 con 0x7ff33406b6b0 2024-05-08T22:33:20.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.119+0000 7ff3337fe700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 0x7ff320068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.119+0000 7ff3337fe700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 0x7ff320068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff32801d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.123+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff324051f60 con 0x7ff33406b6b0 2024-05-08T22:33:20.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.291+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3340c6510 con 0x7ff33406b6b0 2024-05-08T22:33:20.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff31bfff700 1 -- 172.21.15.179:0/2262497137 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7ff324055580 con 0x7ff33406b6b0 2024-05-08T22:33:20.298 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:20.298 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:20.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 msgr2=0x7ff320068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 0x7ff320068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff32801d040 comp rx=0 tx=0).stop 2024-05-08T22:33:20.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 msgr2=0x7ff3340c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff324009d70 tx=0x7ff32400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 shutdown_connections 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff320066730 0x7ff320068bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3340c9230 0x7ff3340c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33406b6b0 0x7ff3340c4130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 --2- 172.21.15.179:0/2262497137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33406abb0 0x7ff3341f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 >> 172.21.15.179:0/2262497137 conn(0x7ff3340cb660 msgr2=0x7ff3341547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 shutdown_connections 2024-05-08T22:33:20.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.295+0000 7ff339ed2700 1 -- 172.21.15.179:0/2262497137 wait complete. 2024-05-08T22:33:20.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 Processor -- start 2024-05-08T22:33:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 -- start start 2024-05-08T22:33:20.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 0x7fbf20057bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf2003e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf200e37e0 con 0x7fbf20053da0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200077f0 con 0x7fbf200e3400 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d875700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200581b0 con 0x7fbf2003c0e0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d074700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbf20053da0 0x7fbf200e6ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38756/0 (socket says 172.21.15.179:38756) 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2d074700 1 -- 172.21.15.179:0/3986801150 learned_addr learned my addr 172.21.15.179:0/3986801150 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf27fff700 1 --2- 172.21.15.179:0/3986801150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf2003e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf2c873700 1 --2- 172.21.15.179:0/3986801150 >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 0x7fbf20057bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246576213 0 0) 0x7fbf200e37e0 con 0x7fbf20053da0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf100035d0 con 0x7fbf20053da0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704500104 0 0) 0x7fbf100035d0 con 0x7fbf20053da0 2024-05-08T22:33:20.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf27fff700 1 -- 172.21.15.179:0/3986801150 >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 msgr2=0x7fbf20057bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:20.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.439+0000 7fbf27fff700 1 --2- 172.21.15.179:0/3986801150 >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 0x7fbf20057bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf27fff700 1 -- 172.21.15.179:0/3986801150 >> v1:172.21.15.179:6789/0 conn(0x7fbf20053da0 legacy=0x7fbf200e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf27fff700 1 -- 172.21.15.179:0/3986801150 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20058490 con 0x7fbf2003c0e0 2024-05-08T22:33:20.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2c873700 1 --2- 172.21.15.179:0/3986801150 >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 0x7fbf20057bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:20.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf27fff700 1 --2- 172.21.15.179:0/3986801150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf2003e9d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009850 tx=0x7fbf18009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c353ecd9e8791c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28073d00 con 0x7fbf2003c0e0 2024-05-08T22:33:20.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18004030 con 0x7fbf2003c0e0 2024-05-08T22:33:20.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf277fe700 1 -- 172.21.15.179:0/3986801150 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180043e0 con 0x7fbf2003c0e0 2024-05-08T22:33:20.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- 172.21.15.179:0/3986801150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 msgr2=0x7fbf2003e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 --2- 172.21.15.179:0/3986801150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf2003e9d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009850 tx=0x7fbf18009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- 172.21.15.179:0/3986801150 shutdown_connections 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 --2- 172.21.15.179:0/3986801150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf2003e9d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 --2- 172.21.15.179:0/3986801150 >> v2:172.21.15.195:3300/0 conn(0x7fbf200e3400 0x7fbf20057bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- 172.21.15.179:0/3986801150 >> 172.21.15.179:0/3986801150 conn(0x7fbf2003aae0 msgr2=0x7fbf2003aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- 172.21.15.179:0/3986801150 shutdown_connections 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- 172.21.15.179:0/3986801150 wait complete. 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 Processor -- start 2024-05-08T22:33:20.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.443+0000 7fbf2d875700 1 -- start start 2024-05-08T22:33:20.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf200e0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf200e3400 0x7fbf2017c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20051b80 con 0x7fbf20053da0 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2017bd80 con 0x7fbf200e3400 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2017b4b0 con 0x7fbf2003c0e0 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48256/0 (socket says 172.21.15.179:48256) 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 -- 172.21.15.179:0/298927932 learned_addr learned my addr 172.21.15.179:0/298927932 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2c873700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf200e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d074700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf200e3400 0x7fbf2017c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 -- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 msgr2=0x7fbf200e0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf200e0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 -- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf200e3400 msgr2=0x7fbf2017c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf200e3400 0x7fbf2017c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 -- 172.21.15.179:0/298927932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0094b0 con 0x7fbf20053da0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf27fff700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf18000c00 tx=0x7fbf18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18006ea0 con 0x7fbf20053da0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180094b0 con 0x7fbf20053da0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1801f070 con 0x7fbf20053da0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180045c0 con 0x7fbf20053da0 2024-05-08T22:33:20.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf200505d0 con 0x7fbf20053da0 2024-05-08T22:33:20.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.447+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2017bec0 con 0x7fbf20053da0 2024-05-08T22:33:20.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.451+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf18021030 con 0x7fbf20053da0 2024-05-08T22:33:20.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.451+0000 7fbf257fa700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 0x7fbf14068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.451+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf180852b0 con 0x7fbf20053da0 2024-05-08T22:33:20.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.451+0000 7fbf2c873700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 0x7fbf14068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.451+0000 7fbf2c873700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 0x7fbf14068c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ce0 tx=0x7fbf1c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.459+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbf2017bec0 con 0x7fbf20053da0 2024-05-08T22:33:20.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.667+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf2017bec0 con 0x7fbf20053da0 2024-05-08T22:33:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.667+0000 7fbf257fa700 1 -- 172.21.15.179:0/298927932 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4745 (secure 0 0 0) 0x7fbf2017bec0 con 0x7fbf20053da0 2024-05-08T22:33:20.673 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:20.673 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:19.306328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T22:33:20.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 msgr2=0x7fbf14068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 0x7fbf14068c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ce0 tx=0x7fbf1c009240 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 msgr2=0x7fbf200d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf18000c00 tx=0x7fbf18004770 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 shutdown_connections 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf14066760 0x7fbf14068c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf200e3400 0x7fbf2017c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf20053da0 0x7fbf200d6680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 --2- 172.21.15.179:0/298927932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf2003c0e0 0x7fbf200e0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 >> 172.21.15.179:0/298927932 conn(0x7fbf2003aae0 msgr2=0x7fbf20039460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 shutdown_connections 2024-05-08T22:33:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.671+0000 7fbf2d875700 1 -- 172.21.15.179:0/298927932 wait complete. 2024-05-08T22:33:20.677 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 7 2024-05-08T22:33:20.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:20.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 Processor -- start 2024-05-08T22:33:20.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 -- start start 2024-05-08T22:33:20.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 0x7f8b7c06af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c166400 con 0x7f8b7c0d0d50 2024-05-08T22:33:20.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c165880 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c165b60 con 0x7f8b7c15f420 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 0x7f8b7c06af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 0x7f8b7c06af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48268/0 (socket says 172.21.15.179:48268) 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45534/0 (socket says 172.21.15.179:45534) 2024-05-08T22:33:20.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 -- 172.21.15.179:0/1725216652 learned_addr learned my addr 172.21.15.179:0/1725216652 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 -- 172.21.15.179:0/1725216652 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 msgr2=0x7f8b7c06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 --2- 172.21.15.179:0/1725216652 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 0x7f8b7c06af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 -- 172.21.15.179:0/1725216652 >> v1:172.21.15.195:6789/0 conn(0x7f8b7c0d0d50 legacy=0x7f8b7c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 -- 172.21.15.179:0/1725216652 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c165e40 con 0x7f8b7c15f420 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b7affd700 1 --2- 172.21.15.179:0/1725216652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009910 tx=0x7f8b64009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845c9919a51ea12f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b79ffb700 1 -- 172.21.15.179:0/1725216652 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64011830 con 0x7f8b7c15f420 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b79ffb700 1 -- 172.21.15.179:0/1725216652 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64011990 con 0x7f8b7c15f420 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.791+0000 7f8b79ffb700 1 -- 172.21.15.179:0/1725216652 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64011d40 con 0x7f8b7c15f420 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1725216652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 msgr2=0x7f8b7c161ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1725216652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009910 tx=0x7f8b64009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:20.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1725216652 shutdown_connections 2024-05-08T22:33:20.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1725216652 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c0cdbb0 0x7f8b7c06af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1725216652 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c15f420 0x7f8b7c161ef0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1725216652 >> 172.21.15.179:0/1725216652 conn(0x7f8b7c0b4940 msgr2=0x7f8b7c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1725216652 shutdown_connections 2024-05-08T22:33:20.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1725216652 wait complete. 2024-05-08T22:33:20.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 Processor -- start 2024-05-08T22:33:20.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- start start 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 0x7f8b7c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 0x7f8b7c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c7e40 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c7a70 con 0x7f8b7c15f420 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.795+0000 7f8b7bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c71d0 con 0x7f8b7c0d0d50 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48282/0 (socket says 172.21.15.179:48282) 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 -- 172.21.15.179:0/1411035448 learned_addr learned my addr 172.21.15.179:0/1411035448 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7b7fe700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 0x7f8b7c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7a7fc700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 0x7f8b7c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 -- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 msgr2=0x7f8b7c0c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 0x7f8b7c0c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 -- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 msgr2=0x7f8b7c15b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 0x7f8b7c15b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 -- 172.21.15.179:0/1411035448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009570 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7b7fe700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 0x7f8b7c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7a7fc700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 0x7f8b7c0c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7affd700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009f70 tx=0x7f8b64004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400b9f0 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6401e070 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c0c74b0 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400bd90 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c0c6ec0 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b6402d450 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.799+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c0c7c10 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.803+0000 7f8b73fff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 0x7f8b5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:20.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.803+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b64085460 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.803+0000 7f8b7a7fc700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 0x7f8b5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:20.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.803+0000 7f8b7a7fc700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 0x7f8b5c068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0c64a0 tx=0x7f8b6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:20.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.811+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b7c0c7c10 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.987+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b7c0c7c10 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.987+0000 7f8b73fff700 1 -- 172.21.15.179:0/1411035448 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f8b7c0c7c10 con 0x7f8b7c0cdbb0 2024-05-08T22:33:20.993 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:20.993 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:20.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 msgr2=0x7f8b5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 0x7f8b5c068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0c64a0 tx=0x7f8b6c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:20.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 msgr2=0x7f8b7c0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009f70 tx=0x7f8b64004770 comp rx=0 tx=0).stop 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 shutdown_connections 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b5c066730 0x7f8b5c068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7c15f420 0x7f8b7c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7c0d0d50 0x7f8b7c0c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1411035448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0cdbb0 0x7f8b7c0cb090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 >> 172.21.15.179:0/1411035448 conn(0x7f8b7c0b4940 msgr2=0x7f8b7c163950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 shutdown_connections 2024-05-08T22:33:20.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:20.991+0000 7f8b7bfff700 1 -- 172.21.15.179:0/1411035448 wait complete. 2024-05-08T22:33:21.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:33:14.889801+0000', 'modified': '2024-05-08T22:33:19.306328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54155}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54155': {'gid': 54155, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/3661147208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 3661147208}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 3661147208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-08T22:33:21.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:33:21.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:33:21.476 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:33:21.479 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:33:21.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 Processor -- start 2024-05-08T22:33:21.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- start start 2024-05-08T22:33:21.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b1c14aca0 0x7f2b1c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c0b3710 con 0x7f2b1c0ced90 2024-05-08T22:33:21.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c0b39f0 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c0ccd20 con 0x7f2b1c14aca0 2024-05-08T22:33:21.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b127fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2b1c0ced90 0x7f2b1c0b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38768/0 (socket says 172.21.15.179:38768) 2024-05-08T22:33:21.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b127fc700 1 -- 172.21.15.179:0/2179365346 learned_addr learned my addr 172.21.15.179:0/2179365346 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:21.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2179365346 >> v2:172.21.15.195:3300/0 conn(0x7f2b1c14aca0 0x7f2b1c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1a7fc700 1 -- 172.21.15.179:0/2179365346 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398051938 0 0) 0x7f2b1c0b3710 con 0x7f2b1c0ced90 2024-05-08T22:33:21.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1a7fc700 1 -- 172.21.15.179:0/2179365346 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b080035d0 con 0x7f2b1c0ced90 2024-05-08T22:33:21.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1affd700 1 -- 172.21.15.179:0/2179365346 >> v1:172.21.15.195:6789/0 conn(0x7f2b1c0b31d0 legacy=0x7f2b1c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1affd700 1 -- 172.21.15.179:0/2179365346 >> v1:172.21.15.179:6789/0 conn(0x7f2b1c0ced90 legacy=0x7f2b1c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1affd700 1 -- 172.21.15.179:0/2179365346 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c0b3cd0 con 0x7f2b1c14aca0 2024-05-08T22:33:21.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2179365346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c0ce850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009850 tx=0x7f2b0c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5008e0efaf06dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1a7fc700 1 -- 172.21.15.179:0/2179365346 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c011830 con 0x7f2b1c14aca0 2024-05-08T22:33:21.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1a7fc700 1 -- 172.21.15.179:0/2179365346 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c011990 con 0x7f2b1c14aca0 2024-05-08T22:33:21.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1a7fc700 1 -- 172.21.15.179:0/2179365346 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c011d00 con 0x7f2b1c14aca0 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2179365346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c14aca0 msgr2=0x7f2b1c0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2179365346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c0ce850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009850 tx=0x7f2b0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2179365346 shutdown_connections 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.535+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2179365346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c0ce850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2179365346 >> 172.21.15.179:0/2179365346 conn(0x7f2b1c0b1ed0 msgr2=0x7f2b1c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2179365346 shutdown_connections 2024-05-08T22:33:21.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2179365346 wait complete. 2024-05-08T22:33:21.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 Processor -- start 2024-05-08T22:33:21.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- start start 2024-05-08T22:33:21.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 0x7f2b1c145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 0x7f2b1c1f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1fa8f0 con 0x7f2b1c14aca0 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0cbc80 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.539+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0caec0 con 0x7f2b1c0ced90 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48294/0 (socket says 172.21.15.179:48294) 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/2005340479 learned_addr learned my addr 172.21.15.179:0/2005340479 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 0x7f2b1c145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 0x7f2b1c1f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 -- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 msgr2=0x7f2b1c1f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 0x7f2b1c1f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 -- 172.21.15.179:0/2005340479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 msgr2=0x7f2b1c1fa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 -- 172.21.15.179:0/2005340479 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c0094b0 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 0x7f2b1c1f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 0x7f2b1c145a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c016000 tx=0x7f2b0c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b13fff700 1 -- 172.21.15.179:0/2005340479 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c004550 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b13fff700 1 -- 172.21.15.179:0/2005340479 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c01f070 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b13fff700 1 -- 172.21.15.179:0/2005340479 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0038c0 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c0cb1a0 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.543+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c0cb760 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b13fff700 1 -- 172.21.15.179:0/2005340479 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b0c003a20 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b13fff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 0x7f2afc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 Processor -- start 2024-05-08T22:33:21.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b13fff700 1 -- 172.21.15.179:0/2005340479 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2b0c021070 con 0x7f2b1c0b31d0 2024-05-08T22:33:21.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 -- start start 2024-05-08T22:33:21.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 0x7f2afc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 0x7f2afc068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c15b830 tx=0x7f2b04007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x2573ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x253b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x249ad80 con 0x2574420 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2498510 con 0x2573b00 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2493e00 con 0x256abe0 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48302/0 (socket says 172.21.15.179:48302) 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1333119532 learned_addr learned my addr 172.21.15.179:0/1333119532 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b127fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x253b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1affd700 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x2573ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 msgr2=0x253b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x253b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 msgr2=0x2573ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x2573ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.547+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1333119532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2306960 con 0x2574420 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d500 tx=0x7f2b14001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d634d83c75f0e333 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b10ff9700 1 -- 172.21.15.179:0/1333119532 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400de20 con 0x2574420 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b10ff9700 1 -- 172.21.15.179:0/1333119532 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14003c30 con 0x2574420 2024-05-08T22:33:21.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b10ff9700 1 -- 172.21.15.179:0/1333119532 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005110 con 0x2574420 2024-05-08T22:33:21.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- 172.21.15.179:0/1333119532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 msgr2=0x253b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d500 tx=0x7f2b14001b00 comp rx=0 tx=0).stop 2024-05-08T22:33:21.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- 172.21.15.179:0/1333119532 shutdown_connections 2024-05-08T22:33:21.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x253b9b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x253b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1333119532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x2573ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- 172.21.15.179:0/1333119532 >> 172.21.15.179:0/1333119532 conn(0x22bb4d0 msgr2=0x2555250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- 172.21.15.179:0/1333119532 shutdown_connections 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- 172.21.15.179:0/1333119532 wait complete. 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 -- start start 2024-05-08T22:33:21.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.551+0000 7f2b24a87740 1 Processor -- start 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x2579bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x257a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2451730 con 0x2574420 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x24575b0 con 0x2573b00 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x233faa0 con 0x256abe0 2024-05-08T22:33:21.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 10 client.0 _ll_register_callbacks cb 0x248d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48308/0 (socket says 172.21.15.179:48308) 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/4273795831 learned_addr learned my addr 172.21.15.179:0/4273795831 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1affd700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x2579bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b127fc700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x257a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 msgr2=0x2579bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x2579bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 msgr2=0x257a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x257a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/4273795831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c0b39f0 con 0x2574420 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400daf0 tx=0x7f2b14001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0c21522491eb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:33:21.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14001840 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2b140057a0 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22c81e0 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b14014450 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 10 client.4422 fetch_fsmap learned FSMap version 7 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x245a220 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22c81e0 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f2b1401c030 con 0x2574420 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 10 client.4422 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 20 client.4422 populate_metadata read hostname 'smithi179' 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x256b4b0 0x2587310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 4 client.4422 mds_command: new command op to 54155 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:33:21.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 --> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2312460 con 0x256b4b0 2024-05-08T22:33:21.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1affd700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x256b4b0 0x2587310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2af9ffb700 10 client.4422 ms_handle_connect on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.555+0000 7f2b1affd700 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x256b4b0 0x2587310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3f44e04a898b7e server_cookie=c3c3fed1620ff4b5 in_seq=0 out_seq=0 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2af9ffb700 1 -- 172.21.15.179:0/4273795831 <== mds.0 v2:172.21.15.195:6836/3661147208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2312460 con 0x256b4b0 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2af9ffb700 10 client.4422 handle_command_reply: tid=0 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 client.4422 shutdown 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 msgr2=0x257a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400daf0 tx=0x7f2b14001250 comp rx=0 tx=0).stop 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 shutdown_connections 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x256b4b0 0x2587310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2573b00 0x257a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2af9ffb700 0 client.4422 ms_handle_reset on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2574420 0x257a630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 --2- 172.21.15.179:0/4273795831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x256abe0 0x2579bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 >> 172.21.15.179:0/4273795831 conn(0x22bb4d0 msgr2=0x2554a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 shutdown_connections 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 1 -- 172.21.15.179:0/4273795831 wait complete. 2024-05-08T22:33:21.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.559+0000 7f2b24a87740 20 client.4422 trim_cache size 0 max 16384 2024-05-08T22:33:21.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 Processor -- start 2024-05-08T22:33:21.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 -- start start 2024-05-08T22:33:21.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x253e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x25557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x248d2f0 con 0x256b4b0 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2455e90 con 0x256abe0 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x243c020 con 0x2573b00 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b127fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b127fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48320/0 (socket says 172.21.15.179:48320) 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b127fc700 1 -- 172.21.15.179:0/1217378922 learned_addr learned my addr 172.21.15.179:0/1217378922 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x25557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1affd700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x253e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 msgr2=0x253e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x253e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1217378922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 msgr2=0x25552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 -- 172.21.15.179:0/1217378922 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2440450 con 0x2573b00 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b127fc700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x25557f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2b14001350 tx=0x7f2b14021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14384f6a28ea8908 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b117fa700 1 -- 172.21.15.179:0/1217378922 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14014450 con 0x2573b00 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b117fa700 1 -- 172.21.15.179:0/1217378922 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14003da0 con 0x2573b00 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.583+0000 7f2b117fa700 1 -- 172.21.15.179:0/1217378922 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005810 con 0x2573b00 2024-05-08T22:33:21.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 msgr2=0x25557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x25557f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2b14001350 tx=0x7f2b14021000 comp rx=0 tx=0).stop 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- 172.21.15.179:0/1217378922 shutdown_connections 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x25557f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- 172.21.15.179:0/1217378922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x253e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- 172.21.15.179:0/1217378922 >> 172.21.15.179:0/1217378922 conn(0x22bb4d0 msgr2=0x258bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- 172.21.15.179:0/1217378922 shutdown_connections 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- 172.21.15.179:0/1217378922 wait complete. 2024-05-08T22:33:21.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 -- start start 2024-05-08T22:33:21.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 Processor -- start 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x25d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.587+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x256fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2571ce0 con 0x256b4b0 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x253ec40 con 0x256abe0 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2590f10 con 0x2573b00 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 10 client.0 _ll_register_callbacks cb 0x7f2b1400d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48322/0 (socket says 172.21.15.179:48322) 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 -- 172.21.15.179:0/2747906517 learned_addr learned my addr 172.21.15.179:0/2747906517 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b1b7fe700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x256fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x25d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 -- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 msgr2=0x256fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x256fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 -- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 msgr2=0x25d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x25d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 -- 172.21.15.179:0/2747906517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b080035d0 con 0x256b4b0 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b127fc700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b04014a00 tx=0x7f2b04000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72214d42882b1a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:33:21.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04005b10 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2b0401c040 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b14003c30 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b04005c70 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 10 client.4428 fetch_fsmap learned FSMap version 7 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x245a220 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14003c30 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f2b04006030 con 0x256b4b0 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:33:21.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 20 client.4428 populate_metadata read hostname 'smithi179' 2024-05-08T22:33:21.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x2574420 0x2570330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 4 client.4428 mds_command: new command op to 54155 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:33:21.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 --> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] -- command(tid 0: {"prefix": "status"}) v1 -- 0x23371e0 con 0x2574420 2024-05-08T22:33:21.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x2574420 0x2570330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2af9ffb700 10 client.4428 ms_handle_connect on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.591+0000 7f2b1affd700 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x2574420 0x2570330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2de626560aed4c server_cookie=4bf48bbc98332e7f in_seq=0 out_seq=0 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2af9ffb700 1 -- 172.21.15.179:0/2747906517 <== mds.0 v2:172.21.15.195:6836/3661147208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x23371e0 con 0x2574420 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2af9ffb700 10 client.4428 handle_command_reply: tid=0 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 client.4428 shutdown 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 msgr2=0x25d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b04014a00 tx=0x7f2b04000ed0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 shutdown_connections 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x2574420 0x2570330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2573b00 0x256fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2af9ffb700 0 client.4428 ms_handle_reset on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x256b4b0 0x25d52b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 --2- 172.21.15.179:0/2747906517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x256abe0 0x25d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 >> 172.21.15.179:0/2747906517 conn(0x22bb4d0 msgr2=0x25552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 shutdown_connections 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 1 -- 172.21.15.179:0/2747906517 wait complete. 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b24a87740 20 client.4428 trim_cache size 0 max 16384 2024-05-08T22:33:21.601 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 54155, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 6.7048983599999996, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 7, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 26, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 25, 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 13.028371105 2024-05-08T22:33:21.602 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 msgr2=0x7f2afc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 0x7f2afc068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c15b830 tx=0x7f2b04007de0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 msgr2=0x7f2b1c145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 0x7f2b1c145a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c016000 tx=0x7f2b0c004770 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 shutdown_connections 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2afc066730 0x7f2afc068bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b1c14aca0 0x7f2b1c1fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b1c0ced90 0x7f2b1c1f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 --2- 172.21.15.179:0/2005340479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b1c0b31d0 0x7f2b1c145a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 >> 172.21.15.179:0/2005340479 conn(0x7f2b1c0b1ed0 msgr2=0x7f2b1c1470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 shutdown_connections 2024-05-08T22:33:21.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.595+0000 7f2b1bfff700 1 -- 172.21.15.179:0/2005340479 wait complete. 2024-05-08T22:33:21.614 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:33:21.614 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:21.615 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:33:21.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.727+0000 7f2955a0b700 1 Processor -- start 2024-05-08T22:33:21.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.727+0000 7f2955a0b700 1 -- start start 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2955a0b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f295015d580 0x7f29500b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2955a0b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f29500d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2955a0b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29500d2640 con 0x7f2950160f30 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2955a0b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295006b3a0 con 0x7f295015d580 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2955a0b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b6750 con 0x7f29500d18a0 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2950160f30 0x7f295006b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38774/0 (socket says 172.21.15.179:38774) 2024-05-08T22:33:21.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 -- 172.21.15.179:0/3831056257 learned_addr learned my addr 172.21.15.179:0/3831056257 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:21.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2954a09700 1 --2- 172.21.15.179:0/3831056257 >> v2:172.21.15.195:3300/0 conn(0x7f295015d580 0x7f29500b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294ffff700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f29500d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381178037 0 0) 0x7f29500d2640 con 0x7f2950160f30 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f293c0035d0 con 0x7f2950160f30 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454910088 0 0) 0x7f293c0035d0 con 0x7f2950160f30 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 msgr2=0x7f29500d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f29500d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 >> v2:172.21.15.195:3300/0 conn(0x7f295015d580 msgr2=0x7f29500b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 --2- 172.21.15.179:0/3831056257 >> v2:172.21.15.195:3300/0 conn(0x7f295015d580 0x7f29500b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500d2360 con 0x7f2950160f30 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f29480019f0 con 0x7f2950160f30 2024-05-08T22:33:21.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 >> v1:172.21.15.179:6789/0 conn(0x7f2950160f30 legacy=0x7f295006b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c003a20 0x7f293c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f293c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293c004db0 0x7f293c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295006b3a0 con 0x7f293c004db0 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b6750 con 0x7f293c003a20 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c00ba30 con 0x7f29500d18a0 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293c004db0 0x7f293c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294ffff700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f293c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2954a09700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c003a20 0x7f293c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 -- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 msgr2=0x7f293c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500d18a0 0x7f293c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 -- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c003a20 msgr2=0x7f293c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:21.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c003a20 0x7f293c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 -- 172.21.15.179:0/3831056257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0035d0 con 0x7f293c004db0 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f2954a09700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c003a20 0x7f293c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f295520a700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293c004db0 0x7f293c005230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f294800a860 tx=0x7f293c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcd39e355f2da98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480072d0 con 0x7f293c004db0 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948007450 con 0x7f293c004db0 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.731+0000 7f294f7fe700 1 -- 172.21.15.179:0/3831056257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948007800 con 0x7f293c004db0 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- 172.21.15.179:0/3831056257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293c004db0 msgr2=0x7f293c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 --2- 172.21.15.179:0/3831056257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293c004db0 0x7f293c005230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f294800a860 tx=0x7f293c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f295520a700 1 -- 172.21.15.179:0/3831056257 reap_dead start 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- 172.21.15.179:0/3831056257 shutdown_connections 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- 172.21.15.179:0/3831056257 >> 172.21.15.179:0/3831056257 conn(0x7f29500d2d90 msgr2=0x7f29500d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- 172.21.15.179:0/3831056257 shutdown_connections 2024-05-08T22:33:21.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- 172.21.15.179:0/3831056257 wait complete. 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 Processor -- start 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- start start 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2950157020 0x7f295015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500ca3a0 0x7f29501f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500ca820 con 0x7f29501594d0 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295015a7e0 con 0x7f2950157020 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f2955a0b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b1740 con 0x7f29500ca3a0 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f294ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f294ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48334/0 (socket says 172.21.15.179:48334) 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.735+0000 7f294ffff700 1 -- 172.21.15.179:0/2900217802 learned_addr learned my addr 172.21.15.179:0/2900217802 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f295520a700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500ca3a0 0x7f29501f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f2954a09700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2950157020 0x7f295015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 -- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500ca3a0 msgr2=0x7f29501f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500ca3a0 0x7f29501f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 -- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2950157020 msgr2=0x7f295015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2950157020 0x7f295015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 -- 172.21.15.179:0/2900217802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950159950 con 0x7f29501594d0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294ffff700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2940011a80 tx=0x7f2940011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000f9a0 con 0x7f29501594d0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000fb00 con 0x7f29501594d0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2950159bd0 con 0x7f29501594d0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000feb0 con 0x7f29501594d0 2024-05-08T22:33:21.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295015a190 con 0x7f29501594d0 2024-05-08T22:33:21.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.739+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295015d960 con 0x7f29501594d0 2024-05-08T22:33:21.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.743+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2940014440 con 0x7f29501594d0 2024-05-08T22:33:21.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.743+0000 7f294d7fa700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 0x7f2930068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.743+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2940084b10 con 0x7f29501594d0 2024-05-08T22:33:21.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.743+0000 7f2954a09700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 0x7f2930068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.743+0000 7f2954a09700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 0x7f2930068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ba0 tx=0x7f2938008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.751+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f295015d960 con 0x7f29501594d0 2024-05-08T22:33:21.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.959+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f295015d960 con 0x7f29501594d0 2024-05-08T22:33:21.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.959+0000 7f294d7fa700 1 -- 172.21.15.179:0/2900217802 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f295015d960 con 0x7f29501594d0 2024-05-08T22:33:21.964 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:33:21.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 msgr2=0x7f2930068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 0x7f2930068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ba0 tx=0x7f2938008040 comp rx=0 tx=0).stop 2024-05-08T22:33:21.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 msgr2=0x7f29500c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2940011a80 tx=0x7f2940011e40 comp rx=0 tx=0).stop 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 shutdown_connections 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2930066760 0x7f2930068c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29500ca3a0 0x7f29501f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29501594d0 0x7f29500c9e60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 --2- 172.21.15.179:0/2900217802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2950157020 0x7f295015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 >> 172.21.15.179:0/2900217802 conn(0x7f29500d2d90 msgr2=0x7f29500cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 shutdown_connections 2024-05-08T22:33:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:21.963+0000 7f2955a0b700 1 -- 172.21.15.179:0/2900217802 wait complete. 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:33:21.978 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:33:21.979 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T22:33:21.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:21.979 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:33:21.983 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:33:21.984 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42569sec preferred_lft 42569sec 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:33:21.985 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:21.986 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:33:21.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr flush dev ceph-brx 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set ceph-brx up 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:33:21.986 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:22.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.073 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:33:22.082 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:33:22.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.084 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:33:22.111 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:33:22.111 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:33:22.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.112 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:22.112 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:22.113 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:33:22.113 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:33:22.113 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:33:22.113 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:22.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.194 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:33:22.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.201 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-05-08T22:33:22.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.208 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:22.208 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:22.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T22:33:22.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T22:33:22.208 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:22.278 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T22:33:22.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.278 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:22.278 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:22.278 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T22:33:22.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:33:22.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T22:33:22.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T22:33:22.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T22:33:22.279 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:22.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.353 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:22.353 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:22.353 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:33:22.354 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:33:22.354 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:22.380 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T22:33:22.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:22.380 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:22.385 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T22:33:22.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:22.385 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:22.435 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:22.450 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:22.450 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:22.466 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:22.467 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:22.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.468 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:22.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:33:22.475 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T22:33:22.477 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:22.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:22.492 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:22.503 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:22.504 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:22.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:22.505 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:22.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi179.stderr:2024-05-08T22:33:22.559+0000 7ff930e5a640 -1 init, newargv = 0x55f9777a5ef0 newargc=15 2024-05-08T22:33:22.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi179.stderr:ceph-fuse[31435]: starting ceph client 2024-05-08T22:33:22.576 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi179.stderr:ceph-fuse[31435]: starting fuse 2024-05-08T22:33:23.662 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:23.678 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:23.679 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:23.693 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:23.693 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:23.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:23.694 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:23.743 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:33:23.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:33:23.745 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:23.746 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.admin")) 2024-05-08T22:33:23.747 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:33:23.828 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.admin.31435.asok 2024-05-08T22:33:24.013 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.31435.asok 2024-05-08T22:33:24.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:24.013 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.31435.asok status 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "admin", 2024-05-08T22:33:24.163 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "31435", 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 4440, 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 4440 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:24.164 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2792553649 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2792553649 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.4440 192.168.0.1:0/2792553649", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/2792553649", 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:33:24.165 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 7, 2024-05-08T22:33:24.166 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 26, 2024-05-08T22:33:24.166 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:33:24.166 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:33:24.166 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:33:24.166 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:33:24.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:24.173 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:33:24.173 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:24.178 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:33:24.179 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:24.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:24.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:24.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:24.194 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T22:33:24.292 INFO:teuthology.orchestra.run.smithi179.stdout:# file: . 2024-05-08T22:33:24.292 INFO:teuthology.orchestra.run.smithi179.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T22:33:24.292 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:24.293 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:33:24.293 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:33:24.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:24.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.415+0000 7f068d7bf700 1 Processor -- start 2024-05-08T22:33:24.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- start start 2024-05-08T22:33:24.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06880ce060 0x7f068806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06880cc310 0x7f06880ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0688161ac0 con 0x7f06880add40 2024-05-08T22:33:24.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06880ae550 con 0x7f06880cc310 2024-05-08T22:33:24.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068806abb0 con 0x7f06880ce060 2024-05-08T22:33:24.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06880cc310 0x7f06880ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06880cc310 0x7f06880ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48338/0 (socket says 172.21.15.179:48338) 2024-05-08T22:33:24.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 -- 172.21.15.179:0/3727163446 learned_addr learned my addr 172.21.15.179:0/3727163446 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:24.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f0686ffd700 1 --2- 172.21.15.179:0/3727163446 >> v2:172.21.15.195:3300/0 conn(0x7f06880ce060 0x7f068806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 -- 172.21.15.179:0/3727163446 >> v2:172.21.15.195:3300/0 conn(0x7f06880ce060 msgr2=0x7f068806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:24.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 --2- 172.21.15.179:0/3727163446 >> v2:172.21.15.195:3300/0 conn(0x7f06880ce060 0x7f068806ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 -- 172.21.15.179:0/3727163446 >> v1:172.21.15.179:6789/0 conn(0x7f06880add40 legacy=0x7f068806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 -- 172.21.15.179:0/3727163446 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880ce980 con 0x7f06880cc310 2024-05-08T22:33:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f0685ffb700 1 -- 172.21.15.179:0/3727163446 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689715941 0 0) 0x7f0688161ac0 con 0x7f06880add40 2024-05-08T22:33:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f06867fc700 1 --2- 172.21.15.179:0/3727163446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880cc310 0x7f06880ce440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0688074410 tx=0x7f0678009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1614bc1effeb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f0685ffb700 1 -- 172.21.15.179:0/3727163446 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800b8a0 con 0x7f06880cc310 2024-05-08T22:33:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f0685ffb700 1 -- 172.21.15.179:0/3727163446 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800ba00 con 0x7f06880cc310 2024-05-08T22:33:24.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f0685ffb700 1 -- 172.21.15.179:0/3727163446 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800bd70 con 0x7f06880cc310 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- 172.21.15.179:0/3727163446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880cc310 msgr2=0x7f06880ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 --2- 172.21.15.179:0/3727163446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880cc310 0x7f06880ce440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0688074410 tx=0x7f0678009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- 172.21.15.179:0/3727163446 shutdown_connections 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 --2- 172.21.15.179:0/3727163446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880cc310 0x7f06880ce440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 --2- 172.21.15.179:0/3727163446 >> v2:172.21.15.195:3300/0 conn(0x7f06880ce060 0x7f068806ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- 172.21.15.179:0/3727163446 >> 172.21.15.179:0/3727163446 conn(0x7f06880cb660 msgr2=0x7f06880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- 172.21.15.179:0/3727163446 shutdown_connections 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.419+0000 7f068d7bf700 1 -- 172.21.15.179:0/3727163446 wait complete. 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 Processor -- start 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- start start 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 0x7f06880b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 0x7f06880c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c3d70 con 0x7f06880ce060 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c3f50 con 0x7f06880cc310 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c4230 con 0x7f06880add40 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48350/0 (socket says 172.21.15.179:48350) 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 -- 172.21.15.179:0/475805850 learned_addr learned my addr 172.21.15.179:0/475805850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f0686ffd700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 0x7f06880b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06867fc700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 0x7f06880c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 -- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 msgr2=0x7f06880b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 0x7f06880b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 -- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 msgr2=0x7f06880c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 0x7f06880c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 -- 172.21.15.179:0/475805850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009570 con 0x7f06880ce060 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f0686ffd700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 0x7f06880b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:24.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06867fc700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 0x7f06880c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f06877fe700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f068000cc10 tx=0x7f068000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680003e50 con 0x7f06880ce060 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068000a7a0 con 0x7f06880ce060 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06880c4510 con 0x7f06880ce060 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068000ab50 con 0x7f06880ce060 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.423+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06880c48a0 con 0x7f06880ce060 2024-05-08T22:33:24.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.427+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0688067770 con 0x7f06880ce060 2024-05-08T22:33:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.427+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f068000acb0 con 0x7f06880ce060 2024-05-08T22:33:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.427+0000 7f066f7fe700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 0x7f0674068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.427+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0680083f50 con 0x7f06880ce060 2024-05-08T22:33:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.427+0000 7f0686ffd700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 0x7f0674068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.431+0000 7f0686ffd700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 0x7f0674068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f067c009a10 tx=0x7f067c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.435+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0680050fd0 con 0x7f06880ce060 2024-05-08T22:33:24.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.607+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06880c4ce0 con 0x7f06880ce060 2024-05-08T22:33:24.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.611+0000 7f066f7fe700 1 -- 172.21.15.179:0/475805850 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f0680017090 con 0x7f06880ce060 2024-05-08T22:33:24.616 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:24.616 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 msgr2=0x7f0674068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 0x7f0674068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f067c009a10 tx=0x7f067c008040 comp rx=0 tx=0).stop 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 msgr2=0x7f06880c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f068000cc10 tx=0x7f068000a060 comp rx=0 tx=0).stop 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 shutdown_connections 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0674066730 0x7f0674068bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06880cc310 0x7f06880c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06880ce060 0x7f06880c2710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 --2- 172.21.15.179:0/475805850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06880add40 0x7f06880b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 >> 172.21.15.179:0/475805850 conn(0x7f06880cb660 msgr2=0x7f06880ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 shutdown_connections 2024-05-08T22:33:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.615+0000 7f068d7bf700 1 -- 172.21.15.179:0/475805850 wait complete. 2024-05-08T22:33:24.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:33:24.632 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:33:24.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 Processor -- start 2024-05-08T22:33:24.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 -- start start 2024-05-08T22:33:24.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17b00e1110 0x7f17b0054e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b003cad0 0x7f17b0051b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00563e0 con 0x7f17b0056870 2024-05-08T22:33:24.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b00e14f0 con 0x7f17b00e1110 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00557d0 con 0x7f17b003cad0 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17b00e1110 0x7f17b0054e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f17b0056870 0x7f17b000dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38790/0 (socket says 172.21.15.179:38790) 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b77fe700 1 -- 172.21.15.179:0/1000150297 learned_addr learned my addr 172.21.15.179:0/1000150297 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b67fc700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b003cad0 0x7f17b0051b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 -- 172.21.15.179:0/1000150297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b003cad0 msgr2=0x7f17b0051b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b003cad0 0x7f17b0051b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b5ffb700 1 -- 172.21.15.179:0/1000150297 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273398575 0 0) 0x7f17b00563e0 con 0x7f17b0056870 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 -- 172.21.15.179:0/1000150297 >> v1:172.21.15.179:6789/0 conn(0x7f17b0056870 legacy=0x7f17b000dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 -- 172.21.15.179:0/1000150297 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0055ab0 con 0x7f17b00e1110 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b0054e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009850 tx=0x7f17a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82fe8e0c20844c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b5ffb700 1 -- 172.21.15.179:0/1000150297 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011830 con 0x7f17b00e1110 2024-05-08T22:33:24.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b5ffb700 1 -- 172.21.15.179:0/1000150297 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0011990 con 0x7f17b00e1110 2024-05-08T22:33:24.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.747+0000 7f17b5ffb700 1 -- 172.21.15.179:0/1000150297 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011d20 con 0x7f17b00e1110 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- 172.21.15.179:0/1000150297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 msgr2=0x7f17b0054e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b0054e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009850 tx=0x7f17a0009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- 172.21.15.179:0/1000150297 shutdown_connections 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b003cad0 0x7f17b0051b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1000150297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b0054e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- 172.21.15.179:0/1000150297 >> 172.21.15.179:0/1000150297 conn(0x7f17b0056d90 msgr2=0x7f17b00571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:24.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- 172.21.15.179:0/1000150297 shutdown_connections 2024-05-08T22:33:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- 172.21.15.179:0/1000150297 wait complete. 2024-05-08T22:33:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 Processor -- start 2024-05-08T22:33:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- start start 2024-05-08T22:33:24.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 0x7f17b00d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 0x7f17b003afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00dbfc0 con 0x7f17b00e1110 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00d7650 con 0x7f17b003cad0 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00d78f0 con 0x7f17b0056870 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48370/0 (socket says 172.21.15.179:48370) 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 -- 172.21.15.179:0/1877998591 learned_addr learned my addr 172.21.15.179:0/1877998591 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 0x7f17b00d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b67fc700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 0x7f17b003afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 -- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 msgr2=0x7f17b003afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 0x7f17b003afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 -- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 msgr2=0x7f17b00d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 0x7f17b00d5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b77fe700 1 -- 172.21.15.179:0/1877998591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00094b0 con 0x7f17b00e1110 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 0x7f17b00d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.751+0000 7f17b67fc700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 0x7f17b003afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17b77fe700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17b802ce60 tx=0x7f17b804e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80669e0 con 0x7f17b00e1110 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8066b80 con 0x7f17b00e1110 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00d7bd0 con 0x7f17b00e1110 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8066f30 con 0x7f17b00e1110 2024-05-08T22:33:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00d8190 con 0x7f17b00e1110 2024-05-08T22:33:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b0007480 con 0x7f17b00e1110 2024-05-08T22:33:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.755+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17b8066150 con 0x7f17b00e1110 2024-05-08T22:33:24.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.759+0000 7f17af7fe700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 0x7f1798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:24.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.759+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f17b80fdc20 con 0x7f17b00e1110 2024-05-08T22:33:24.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.759+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 0x7f1798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:24.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.759+0000 7f17b6ffd700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 0x7f1798068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17a00090b0 tx=0x7f17a0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:24.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.763+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17b80cab80 con 0x7f17b00e1110 2024-05-08T22:33:24.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.935+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17b0005890 con 0x7f17b00e1110 2024-05-08T22:33:24.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.935+0000 7f17af7fe700 1 -- 172.21.15.179:0/1877998591 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17b8098020 con 0x7f17b00e1110 2024-05-08T22:33:24.940 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:33:24.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 msgr2=0x7f1798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 0x7f1798068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17a00090b0 tx=0x7f17a0008030 comp rx=0 tx=0).stop 2024-05-08T22:33:24.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 msgr2=0x7f17b003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17b802ce60 tx=0x7f17b804e930 comp rx=0 tx=0).stop 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 shutdown_connections 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1798066730 0x7f1798068bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b00e1110 0x7f17b003b4e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b0056870 0x7f17b003afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 --2- 172.21.15.179:0/1877998591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b003cad0 0x7f17b00d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 >> 172.21.15.179:0/1877998591 conn(0x7f17b0056d90 msgr2=0x7f17b00df1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 shutdown_connections 2024-05-08T22:33:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:24.939+0000 7f17b7fff700 1 -- 172.21.15.179:0/1877998591 wait complete. 2024-05-08T22:33:24.955 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:33:25.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 Processor -- start 2024-05-08T22:33:25.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 -- start start 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f80340cf380 0x7f80340ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803406ea40 0x7f80340ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cdee0 con 0x7f803415d350 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340caca0 con 0x7f80340cf380 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8033fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340caf10 con 0x7f803406ea40 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8032ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f803415d350 0x7f803406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38796/0 (socket says 172.21.15.179:38796) 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f8032ffd700 1 -- 172.21.15.179:0/3056483995 learned_addr learned my addr 172.21.15.179:0/3056483995 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f80327fc700 1 --2- 172.21.15.179:0/3056483995 >> v2:172.21.15.195:3300/0 conn(0x7f80340cf380 0x7f80340ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.071+0000 7f80337fe700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803406ea40 0x7f80340ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528816167 0 0) 0x7f80340cdee0 con 0x7f803415d350 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80200035d0 con 0x7f803415d350 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f80327fc700 1 -- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803406ea40 msgr2=0x7f80340ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f80327fc700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803406ea40 0x7f80340ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f80327fc700 1 -- 172.21.15.179:0/3056483995 >> v1:172.21.15.179:6789/0 conn(0x7f803415d350 legacy=0x7f803406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085585590 0 0) 0x7f80200035d0 con 0x7f803415d350 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f80327fc700 1 -- 172.21.15.179:0/3056483995 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340cb1f0 con 0x7f80340cf380 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f80327fc700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80340cf380 0x7f80340ca220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f801c009850 tx=0x7f801c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e327475cc2ecf35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00f8a0 con 0x7f80340cf380 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c00fa00 con 0x7f80340cf380 2024-05-08T22:33:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8031ffb700 1 -- 172.21.15.179:0/3056483995 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00fd70 con 0x7f80340cf380 2024-05-08T22:33:25.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 -- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80340cf380 msgr2=0x7f80340ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80340cf380 0x7f80340ca220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f801c009850 tx=0x7f801c009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 -- 172.21.15.179:0/3056483995 shutdown_connections 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803406ea40 0x7f80340ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 --2- 172.21.15.179:0/3056483995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80340cf380 0x7f80340ca220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 -- 172.21.15.179:0/3056483995 >> 172.21.15.179:0/3056483995 conn(0x7f80340cfbd0 msgr2=0x7f80340cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 -- 172.21.15.179:0/3056483995 shutdown_connections 2024-05-08T22:33:25.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.075+0000 7f8033fff700 1 -- 172.21.15.179:0/3056483995 wait complete. 2024-05-08T22:33:25.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 Processor -- start 2024-05-08T22:33:25.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 -- start start 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80340cf380 0x7f80340b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803415d350 0x7f80340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341f6a80 con 0x7f803406ea40 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803415a8d0 con 0x7f803415d350 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.079+0000 7f8033fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80341f5e00 con 0x7f80340cf380 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48376/0 (socket says 172.21.15.179:48376) 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 -- 172.21.15.179:0/2618108736 learned_addr learned my addr 172.21.15.179:0/2618108736 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f80337fe700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803415d350 0x7f80340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f80327fc700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80340cf380 0x7f80340b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 -- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80340cf380 msgr2=0x7f80340b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80340cf380 0x7f80340b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 -- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803415d350 msgr2=0x7f80340b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803415d350 0x7f80340b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 -- 172.21.15.179:0/2618108736 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0094b0 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8032ffd700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f802400cb20 tx=0x7f802400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024007480 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80240075e0 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80341f60e0 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024007970 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80341f66a0 con 0x7f803406ea40 2024-05-08T22:33:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.083+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034067770 con 0x7f803406ea40 2024-05-08T22:33:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.087+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f802401b3b0 con 0x7f803406ea40 2024-05-08T22:33:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.087+0000 7f802bfff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 0x7f8014068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.087+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8024083d90 con 0x7f803406ea40 2024-05-08T22:33:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.087+0000 7f80327fc700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 0x7f8014068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.087+0000 7f80327fc700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 0x7f8014068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f801c0090b0 tx=0x7f801c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.091+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8024050e20 con 0x7f803406ea40 2024-05-08T22:33:25.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f803404e700 con 0x7f803406ea40 2024-05-08T22:33:25.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f802bfff700 1 -- 172.21.15.179:0/2618108736 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80240170a0 con 0x7f803406ea40 2024-05-08T22:33:25.267 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:25.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 msgr2=0x7f8014068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 0x7f8014068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f801c0090b0 tx=0x7f801c008030 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 msgr2=0x7f803415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f802400cb20 tx=0x7f802400cb50 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 shutdown_connections 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8014066730 0x7f8014068bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803415d350 0x7f80340b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80340cf380 0x7f80340b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 --2- 172.21.15.179:0/2618108736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803406ea40 0x7f803415a390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.263+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 >> 172.21.15.179:0/2618108736 conn(0x7f80340cfbd0 msgr2=0x7f80340ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.267+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 shutdown_connections 2024-05-08T22:33:25.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.267+0000 7f8033fff700 1 -- 172.21.15.179:0/2618108736 wait complete. 2024-05-08T22:33:25.282 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:33:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 Processor -- start 2024-05-08T22:33:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 -- start start 2024-05-08T22:33:25.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98a415d0c0 0x7f98a40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40d1d20 0x7f98a40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a415d4a0 con 0x7f98a40ce0e0 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40ce570 con 0x7f98a40d1d20 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a406abb0 con 0x7f98a415d0c0 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a27fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40d1d20 0x7f98a40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a27fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40d1d20 0x7f98a40cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41916/0 (socket says 172.21.15.179:41916) 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a27fc700 1 -- 172.21.15.179:0/757574687 learned_addr learned my addr 172.21.15.179:0/757574687 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.399+0000 7f98a2ffd700 1 --2- 172.21.15.179:0/757574687 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d0c0 0x7f98a40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a27fc700 1 -- 172.21.15.179:0/757574687 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d0c0 msgr2=0x7f98a40b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a27fc700 1 --2- 172.21.15.179:0/757574687 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d0c0 0x7f98a40b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a27fc700 1 -- 172.21.15.179:0/757574687 >> v1:172.21.15.195:6789/0 conn(0x7f98a40ce0e0 legacy=0x7f98a4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a27fc700 1 -- 172.21.15.179:0/757574687 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40d2500 con 0x7f98a40d1d20 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a27fc700 1 --2- 172.21.15.179:0/757574687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40d1d20 0x7f98a40cdba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98a408b1c0 tx=0x7f9894009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57334539713fcf2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a1ffb700 1 -- 172.21.15.179:0/757574687 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011830 con 0x7f98a40d1d20 2024-05-08T22:33:25.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a1ffb700 1 -- 172.21.15.179:0/757574687 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894011990 con 0x7f98a40d1d20 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a1ffb700 1 -- 172.21.15.179:0/757574687 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011d20 con 0x7f98a40d1d20 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- 172.21.15.179:0/757574687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40d1d20 msgr2=0x7f98a40cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 --2- 172.21.15.179:0/757574687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40d1d20 0x7f98a40cdba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98a408b1c0 tx=0x7f9894009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- 172.21.15.179:0/757574687 shutdown_connections 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 --2- 172.21.15.179:0/757574687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40d1d20 0x7f98a40cdba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 --2- 172.21.15.179:0/757574687 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d0c0 0x7f98a40b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- 172.21.15.179:0/757574687 >> 172.21.15.179:0/757574687 conn(0x7f98a40d2ed0 msgr2=0x7f98a40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- 172.21.15.179:0/757574687 shutdown_connections 2024-05-08T22:33:25.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- 172.21.15.179:0/757574687 wait complete. 2024-05-08T22:33:25.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 Processor -- start 2024-05-08T22:33:25.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.403+0000 7f98a3fff700 1 -- start start 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a40d1d20 0x7f98a40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 0x7f98a41550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41f5c20 con 0x7f98a40ce0e0 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c8e00 con 0x7f98a415d0c0 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c74b0 con 0x7f98a40d1d20 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41918/0 (socket says 172.21.15.179:41918) 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 -- 172.21.15.179:0/4131338750 learned_addr learned my addr 172.21.15.179:0/4131338750 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a27fc700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a40d1d20 0x7f98a40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a37fe700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 0x7f98a41550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 -- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a40d1d20 msgr2=0x7f98a40c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a40d1d20 0x7f98a40c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 -- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 msgr2=0x7f98a41550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 0x7f98a41550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 -- 172.21.15.179:0/4131338750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009570 con 0x7f98a40ce0e0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a37fe700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 0x7f98a41550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a2ffd700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f989800e840 tx=0x7f989800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98980095a0 con 0x7f98a40ce0e0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40c7790 con 0x7f98a40ce0e0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898013680 con 0x7f98a40ce0e0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98980139f0 con 0x7f98a40ce0e0 2024-05-08T22:33:25.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.407+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40c7d50 con 0x7f98a40ce0e0 2024-05-08T22:33:25.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a404e700 con 0x7f98a40ce0e0 2024-05-08T22:33:25.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9898013b50 con 0x7f98a40ce0e0 2024-05-08T22:33:25.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f988b7fe700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 0x7f9890068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98980842d0 con 0x7f98a40ce0e0 2024-05-08T22:33:25.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f98a27fc700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 0x7f9890068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.411+0000 7f98a27fc700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 0x7f9890068c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.415+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9898051360 con 0x7f98a40ce0e0 2024-05-08T22:33:25.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.587+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98a40676e0 con 0x7f98a40ce0e0 2024-05-08T22:33:25.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.587+0000 7f988b7fe700 1 -- 172.21.15.179:0/4131338750 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9898054980 con 0x7f98a40ce0e0 2024-05-08T22:33:25.591 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 msgr2=0x7f9890068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 0x7f9890068c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894008030 comp rx=0 tx=0).stop 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 msgr2=0x7f98a41f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f989800e840 tx=0x7f989800eb50 comp rx=0 tx=0).stop 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 shutdown_connections 2024-05-08T22:33:25.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066760 0x7f9890068c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a415d0c0 0x7f98a41550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a40d1d20 0x7f98a40c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 --2- 172.21.15.179:0/4131338750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ce0e0 0x7f98a41f56e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 >> 172.21.15.179:0/4131338750 conn(0x7f98a40d2ed0 msgr2=0x7f98a40d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 shutdown_connections 2024-05-08T22:33:25.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.591+0000 7f98a3fff700 1 -- 172.21.15.179:0/4131338750 wait complete. 2024-05-08T22:33:25.606 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:33:25.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 Processor -- start 2024-05-08T22:33:25.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 -- start start 2024-05-08T22:33:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8500ca8b0 0x7fd8500c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 0x7fd8500c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500cf070 con 0x7fd850142b80 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd850142f60 con 0x7fd8500ca8b0 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd856236700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500ce460 con 0x7fd850143170 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd854a33700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8500ca8b0 0x7fd8500c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd855234700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd850142b80 0x7fd8500ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:34194/0 (socket says 172.21.15.179:34194) 2024-05-08T22:33:25.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd855234700 1 -- 172.21.15.179:0/440320770 learned_addr learned my addr 172.21.15.179:0/440320770 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.719+0000 7fd855a35700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 0x7fd8500c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd854a33700 1 -- 172.21.15.179:0/440320770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 msgr2=0x7fd8500c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd854a33700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 0x7fd8500c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd854a33700 1 -- 172.21.15.179:0/440320770 >> v1:172.21.15.179:6789/0 conn(0x7fd850142b80 legacy=0x7fd8500ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd854a33700 1 -- 172.21.15.179:0/440320770 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500ce740 con 0x7fd8500ca8b0 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd847fff700 1 -- 172.21.15.179:0/440320770 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897427754 0 0) 0x7fd8500cf070 con 0x7fd850142b80 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd855a35700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 0x7fd8500c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:25.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd854a33700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8500ca8b0 0x7fd8500c4c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009850 tx=0x7fd84c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5be63dc3630a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd847fff700 1 -- 172.21.15.179:0/440320770 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c011830 con 0x7fd8500ca8b0 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd847fff700 1 -- 172.21.15.179:0/440320770 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c011990 con 0x7fd8500ca8b0 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd847fff700 1 -- 172.21.15.179:0/440320770 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c011d20 con 0x7fd8500ca8b0 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- 172.21.15.179:0/440320770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8500ca8b0 msgr2=0x7fd8500c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8500ca8b0 0x7fd8500c4c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009850 tx=0x7fd84c009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- 172.21.15.179:0/440320770 shutdown_connections 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850143170 0x7fd8500c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 --2- 172.21.15.179:0/440320770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8500ca8b0 0x7fd8500c4c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- 172.21.15.179:0/440320770 >> 172.21.15.179:0/440320770 conn(0x7fd8500cc5d0 msgr2=0x7fd8500cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- 172.21.15.179:0/440320770 shutdown_connections 2024-05-08T22:33:25.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- 172.21.15.179:0/440320770 wait complete. 2024-05-08T22:33:25.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 Processor -- start 2024-05-08T22:33:25.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.723+0000 7fd856236700 1 -- start start 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8500ca8b0 0x7fd8501394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850142b80 0x7fd850157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015a9e0 con 0x7fd850143170 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850159e60 con 0x7fd8500ca8b0 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015a140 con 0x7fd850142b80 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41932/0 (socket says 172.21.15.179:41932) 2024-05-08T22:33:25.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 -- 172.21.15.179:0/3761066771 learned_addr learned my addr 172.21.15.179:0/3761066771 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd854a33700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850142b80 0x7fd850157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855234700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8500ca8b0 0x7fd8501394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 -- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850142b80 msgr2=0x7fd850157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850142b80 0x7fd850157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 -- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8500ca8b0 msgr2=0x7fd8501394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8500ca8b0 0x7fd8501394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 -- 172.21.15.179:0/3761066771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0094b0 con 0x7fd850143170 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd855a35700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd848007ef0 tx=0x7fd84800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84800de30 con 0x7fd850143170 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84800f040 con 0x7fd850143170 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85015a420 con 0x7fd850143170 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848003c80 con 0x7fd850143170 2024-05-08T22:33:25.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8501399e0 con 0x7fd850143170 2024-05-08T22:33:25.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.727+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8500b29f0 con 0x7fd850143170 2024-05-08T22:33:25.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.731+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd848004030 con 0x7fd850143170 2024-05-08T22:33:25.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.731+0000 7fd845ffb700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:25.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.731+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd848084800 con 0x7fd850143170 2024-05-08T22:33:25.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.731+0000 7fd855234700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:25.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.731+0000 7fd855234700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 0x7fd834068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd840000f60 tx=0x7fd840007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:25.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.735+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8480518f0 con 0x7fd850143170 2024-05-08T22:33:25.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.903+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8500b29f0 con 0x7fd850143170 2024-05-08T22:33:25.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.903+0000 7fd845ffb700 1 -- 172.21.15.179:0/3761066771 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd84801a020 con 0x7fd850143170 2024-05-08T22:33:25.907 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:25.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.903+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 msgr2=0x7fd834068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 0x7fd834068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd840000f60 tx=0x7fd840007c00 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 msgr2=0x7fd8501418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd848007ef0 tx=0x7fd84800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 shutdown_connections 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd834066760 0x7fd834068c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd850143170 0x7fd8501418f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd850142b80 0x7fd850157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 --2- 172.21.15.179:0/3761066771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8500ca8b0 0x7fd8501394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 >> 172.21.15.179:0/3761066771 conn(0x7fd8500cc5d0 msgr2=0x7fd8500c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 shutdown_connections 2024-05-08T22:33:25.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:25.907+0000 7fd856236700 1 -- 172.21.15.179:0/3761066771 wait complete. 2024-05-08T22:33:25.922 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:33:26.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 Processor -- start 2024-05-08T22:33:26.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- start start 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 0x7f75680b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f756815cf70 0x7f756806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756815d950 con 0x7f756815d3f0 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756806b2c0 con 0x7f756815cf70 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756815dc30 con 0x7f7568159270 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f756815cf70 0x7f756806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f756815cf70 0x7f756806b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41942/0 (socket says 172.21.15.179:41942) 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 -- 172.21.15.179:0/879529947 learned_addr learned my addr 172.21.15.179:0/879529947 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f75677fe700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 0x7f75680b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 -- 172.21.15.179:0/879529947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 msgr2=0x7f75680b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 0x7f75680b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 -- 172.21.15.179:0/879529947 >> v1:172.21.15.195:6789/0 conn(0x7f756815d3f0 legacy=0x7f756806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 -- 172.21.15.179:0/879529947 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680b1af0 con 0x7f756815cf70 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f75677fe700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 0x7f75680b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f7566ffd700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f756815cf70 0x7f756806b440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7558009910 tx=0x7f7558009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b089ca228a46d7bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f75667fc700 1 -- 172.21.15.179:0/879529947 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800ba60 con 0x7f756815cf70 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f75667fc700 1 -- 172.21.15.179:0/879529947 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800bbc0 con 0x7f756815cf70 2024-05-08T22:33:26.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f75667fc700 1 -- 172.21.15.179:0/879529947 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558009e60 con 0x7f756815cf70 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- 172.21.15.179:0/879529947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f756815cf70 msgr2=0x7f756806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f756815cf70 0x7f756806b440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7558009910 tx=0x7f7558009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- 172.21.15.179:0/879529947 shutdown_connections 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f756815cf70 0x7f756806b440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 --2- 172.21.15.179:0/879529947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7568159270 0x7f75680b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- 172.21.15.179:0/879529947 >> 172.21.15.179:0/879529947 conn(0x7f75680ce960 msgr2=0x7f75680ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- 172.21.15.179:0/879529947 shutdown_connections 2024-05-08T22:33:26.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.039+0000 7f756cdd0700 1 -- 172.21.15.179:0/879529947 wait complete. 2024-05-08T22:33:26.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 Processor -- start 2024-05-08T22:33:26.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- start start 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f756815cf70 0x7f7568157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 0x7f7568157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568147e20 con 0x7f7568159270 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680cd230 con 0x7f756815cf70 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680cd510 con 0x7f756815d3f0 2024-05-08T22:33:26.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41958/0 (socket says 172.21.15.179:41958) 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 -- 172.21.15.179:0/917109264 learned_addr learned my addr 172.21.15.179:0/917109264 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f7566ffd700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f756815cf70 0x7f7568157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f7567fff700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 0x7f7568157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 -- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 msgr2=0x7f7568157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 0x7f7568157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 -- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f756815cf70 msgr2=0x7f7568157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f756815cf70 0x7f7568157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 -- 172.21.15.179:0/917109264 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7558009570 con 0x7f7568159270 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f7567fff700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 0x7f7568157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f75677fe700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fa0 tx=0x7f756000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560003a40 con 0x7f7568159270 2024-05-08T22:33:26.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756000f040 con 0x7f7568159270 2024-05-08T22:33:26.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75680cd7f0 con 0x7f7568159270 2024-05-08T22:33:26.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560003e00 con 0x7f7568159270 2024-05-08T22:33:26.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.043+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75680cddb0 con 0x7f7568159270 2024-05-08T22:33:26.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756804e700 con 0x7f7568159270 2024-05-08T22:33:26.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7560004030 con 0x7f7568159270 2024-05-08T22:33:26.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f754ffff700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 0x7f7554068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75600845a0 con 0x7f7568159270 2024-05-08T22:33:26.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f7566ffd700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 0x7f7554068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.047+0000 7f7566ffd700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 0x7f7554068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.055+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7560051630 con 0x7f7568159270 2024-05-08T22:33:26.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.235+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75680b03c0 con 0x7f7568159270 2024-05-08T22:33:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.235+0000 7f754ffff700 1 -- 172.21.15.179:0/917109264 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7560054c50 con 0x7f7568159270 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 msgr2=0x7f7554068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 0x7f7554068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558008030 comp rx=0 tx=0).stop 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 msgr2=0x7f75681478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fa0 tx=0x7f756000d460 comp rx=0 tx=0).stop 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 shutdown_connections 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7554066730 0x7f7554068bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f756815cf70 0x7f7568157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f756815d3f0 0x7f7568157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 --2- 172.21.15.179:0/917109264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568159270 0x7f75681478e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 >> 172.21.15.179:0/917109264 conn(0x7f75680ce960 msgr2=0x7f756806cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 shutdown_connections 2024-05-08T22:33:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.239+0000 7f756cdd0700 1 -- 172.21.15.179:0/917109264 wait complete. 2024-05-08T22:33:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:33:26.255 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:33:26.255 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:33:26.263 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:26.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 Processor -- start 2024-05-08T22:33:26.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- start start 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 0x7f74c80b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f74c80b2120 0x7f74c80cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c8161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80ce840 con 0x7f74c80b2120 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c806afc0 con 0x7f74c815d5b0 2024-05-08T22:33:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c815d990 con 0x7f74c80ce3b0 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f74c80b2120 0x7f74c80cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f74c80b2120 0x7f74c80cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41966/0 (socket says 172.21.15.179:41966) 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 -- 172.21.15.179:0/535982424 learned_addr learned my addr 172.21.15.179:0/535982424 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c7fff700 1 --2- 172.21.15.179:0/535982424 >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 0x7f74c80b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cc9f5700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c8161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 -- 172.21.15.179:0/535982424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 msgr2=0x7f74c8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c8161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 -- 172.21.15.179:0/535982424 >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 msgr2=0x7f74c80b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 --2- 172.21.15.179:0/535982424 >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 0x7f74c80b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 -- 172.21.15.179:0/535982424 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80d26f0 con 0x7f74c80b2120 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c7fff700 1 --2- 172.21.15.179:0/535982424 >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 0x7f74c80b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cc9f5700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c8161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c77fe700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c80cde70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74c808a630 tx=0x7f74c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9792b4c3e2eb356 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c6ffd700 1 -- 172.21.15.179:0/535982424 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000e070 con 0x7f74c80b2120 2024-05-08T22:33:26.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c6ffd700 1 -- 172.21.15.179:0/535982424 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000b9c0 con 0x7f74c80b2120 2024-05-08T22:33:26.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74c6ffd700 1 -- 172.21.15.179:0/535982424 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000bd70 con 0x7f74c80b2120 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- 172.21.15.179:0/535982424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 msgr2=0x7f74c80cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c80cde70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74c808a630 tx=0x7f74c000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 -- 172.21.15.179:0/535982424 shutdown_connections 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c8161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.371+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/535982424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c80cde70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/535982424 >> v2:172.21.15.195:3300/0 conn(0x7f74c815d5b0 0x7f74c80b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 -- 172.21.15.179:0/535982424 >> 172.21.15.179:0/535982424 conn(0x7f74c80b4940 msgr2=0x7f74c80b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 -- 172.21.15.179:0/535982424 shutdown_connections 2024-05-08T22:33:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 -- 172.21.15.179:0/535982424 wait complete. 2024-05-08T22:33:26.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 Processor -- start 2024-05-08T22:33:26.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.375+0000 7f74cd1f6700 1 -- start start 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 0x7f74c80b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81e44a0 con 0x7f74c80b2120 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81e3890 con 0x7f74c815d5b0 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81e3b70 con 0x7f74c80ce3b0 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41982/0 (socket says 172.21.15.179:41982) 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 -- 172.21.15.179:0/3578103471 learned_addr learned my addr 172.21.15.179:0/3578103471 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:26.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cc9f5700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 0x7f74c80b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c77fe700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 -- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 msgr2=0x7f74c80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c80b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 -- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 msgr2=0x7f74c80b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 0x7f74c80b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 -- 172.21.15.179:0/3578103471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009b30 con 0x7f74c80b2120 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cc9f5700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 0x7f74c80b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c77fe700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c7fff700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00c770 tx=0x7f74bc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc011800 con 0x7f74c80b2120 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc011960 con 0x7f74c80b2120 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc011d10 con 0x7f74c80b2120 2024-05-08T22:33:26.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c81e3e50 con 0x7f74c80b2120 2024-05-08T22:33:26.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c814a6a0 con 0x7f74c80b2120 2024-05-08T22:33:26.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.379+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c804e700 con 0x7f74c80b2120 2024-05-08T22:33:26.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.383+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74bc018360 con 0x7f74c80b2120 2024-05-08T22:33:26.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.383+0000 7f74c4ff9700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 0x7f74b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.383+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74bc084590 con 0x7f74c80b2120 2024-05-08T22:33:26.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.383+0000 7f74c77fe700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 0x7f74b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.383+0000 7f74c77fe700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 0x7f74b4068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b8960 tx=0x7f74c0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.387+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f74bc051570 con 0x7f74c80b2120 2024-05-08T22:33:26.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.559+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c815c9f0 con 0x7f74c80b2120 2024-05-08T22:33:26.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74c4ff9700 1 -- 172.21.15.179:0/3578103471 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f74bc054b90 con 0x7f74c80b2120 2024-05-08T22:33:26.566 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:26.567 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:26.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 msgr2=0x7f74b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 0x7f74b4068bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b8960 tx=0x7f74c0009000 comp rx=0 tx=0).stop 2024-05-08T22:33:26.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 msgr2=0x7f74c814a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00c770 tx=0x7f74bc00cb30 comp rx=0 tx=0).stop 2024-05-08T22:33:26.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 shutdown_connections 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f74b4066730 0x7f74b4068bf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74c815d5b0 0x7f74c80b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74c80ce3b0 0x7f74c80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 --2- 172.21.15.179:0/3578103471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74c80b2120 0x7f74c814a160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 >> 172.21.15.179:0/3578103471 conn(0x7f74c80b4940 msgr2=0x7f74c80b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 shutdown_connections 2024-05-08T22:33:26.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.563+0000 7f74cd1f6700 1 -- 172.21.15.179:0/3578103471 wait complete. 2024-05-08T22:33:26.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:33:26.590 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:33:26.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 Processor -- start 2024-05-08T22:33:26.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 -- start start 2024-05-08T22:33:26.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57cc0ce4d0 0x7f57cc0d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc0d0e80 con 0x7f57cc15d1c0 2024-05-08T22:33:26.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc0d2620 con 0x7f57cc0b8a80 2024-05-08T22:33:26.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d2aa7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc0d2ec0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d0843700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f57cc15d1c0 0x7f57cc0cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:34210/0 (socket says 172.21.15.179:34210) 2024-05-08T22:33:26.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57d0843700 1 -- 172.21.15.179:0/3193809078 learned_addr learned my addr 172.21.15.179:0/3193809078 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:26.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57cbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57cc0ce4d0 0x7f57cc0d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57cb7fe700 1 -- 172.21.15.179:0/3193809078 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703054480 0 0) 0x7f57cc0d0e80 con 0x7f57cc15d1c0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57cb7fe700 1 -- 172.21.15.179:0/3193809078 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b80035d0 con 0x7f57cc15d1c0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.711+0000 7f57cbfff700 1 -- 172.21.15.179:0/3193809078 >> v1:172.21.15.195:6789/0 conn(0x7f57cc0b8a80 legacy=0x7f57cc06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cbfff700 1 -- 172.21.15.179:0/3193809078 >> v1:172.21.15.179:6789/0 conn(0x7f57cc15d1c0 legacy=0x7f57cc0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cbfff700 1 -- 172.21.15.179:0/3193809078 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0d2900 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cbfff700 1 --2- 172.21.15.179:0/3193809078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0d1f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009840 tx=0x7f57b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e6dc73f78e816e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cb7fe700 1 -- 172.21.15.179:0/3193809078 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005e50 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cb7fe700 1 -- 172.21.15.179:0/3193809078 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4006030 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57cb7fe700 1 -- 172.21.15.179:0/3193809078 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40063c0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- 172.21.15.179:0/3193809078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 msgr2=0x7f57cc0d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/3193809078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0d1f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009840 tx=0x7f57b4009b50 comp rx=0 tx=0).stop 2024-05-08T22:33:26.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- 172.21.15.179:0/3193809078 shutdown_connections 2024-05-08T22:33:26.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/3193809078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0d1f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- 172.21.15.179:0/3193809078 >> 172.21.15.179:0/3193809078 conn(0x7f57cc0b4be0 msgr2=0x7f57cc0b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- 172.21.15.179:0/3193809078 shutdown_connections 2024-05-08T22:33:26.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- 172.21.15.179:0/3193809078 wait complete. 2024-05-08T22:33:26.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 Processor -- start 2024-05-08T22:33:26.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.715+0000 7f57d2aa7700 1 -- start start 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57cc0b8a80 0x7f57cc1e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57cc15d1c0 0x7f57cc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc06ae50 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0aa950 con 0x7f57cc0b8a80 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0aab80 con 0x7f57cc15d1c0 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41986/0 (socket says 172.21.15.179:41986) 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 -- 172.21.15.179:0/1837298515 learned_addr learned my addr 172.21.15.179:0/1837298515 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d0843700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57cc0b8a80 0x7f57cc1e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d1044700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57cc15d1c0 0x7f57cc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 -- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57cc15d1c0 msgr2=0x7f57cc15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57cc15d1c0 0x7f57cc15ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 -- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57cc0b8a80 msgr2=0x7f57cc1e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57cc0b8a80 0x7f57cc1e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 -- 172.21.15.179:0/1837298515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4009450 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57cbfff700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57b4013040 tx=0x7f57b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40038c0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4003a80 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc0aae60 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4003e30 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.719+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc0ab420 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc073290 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57b4022070 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57c97fa700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 0x7f57ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:26.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57b4085bd0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57d0843700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 0x7f57ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:26.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.723+0000 7f57d0843700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 0x7f57ac068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57c4007fd0 tx=0x7f57c4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:26.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.727+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57b4052ce0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.899+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57cc0b8650 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.899+0000 7f57c97fa700 1 -- 172.21.15.179:0/1837298515 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57b402a0a0 con 0x7f57cc0ce4d0 2024-05-08T22:33:26.903 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:33:26.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 msgr2=0x7f57ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 0x7f57ac068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57c4007fd0 tx=0x7f57c4007de0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 msgr2=0x7f57cc0aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:26.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57b4013040 tx=0x7f57b4004770 comp rx=0 tx=0).stop 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 shutdown_connections 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ac066730 0x7f57ac068bf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57cc15d1c0 0x7f57cc15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57cc0ce4d0 0x7f57cc0aa410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 --2- 172.21.15.179:0/1837298515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57cc0b8a80 0x7f57cc1e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 >> 172.21.15.179:0/1837298515 conn(0x7f57cc0b4be0 msgr2=0x7f57cc15f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 shutdown_connections 2024-05-08T22:33:26.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:26.903+0000 7f57d2aa7700 1 -- 172.21.15.179:0/1837298515 wait complete. 2024-05-08T22:33:26.918 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:33:27.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.039+0000 7fb4b9e89700 1 Processor -- start 2024-05-08T22:33:27.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.039+0000 7fb4b9e89700 1 -- start start 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb4b415b270 0x7fb4b40d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb4b40cbee0 0x7fb4b40b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b415b650 con 0x7fb4b40d05e0 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40d0b70 con 0x7fb4b415b270 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40b0050 con 0x7fb4b40cbee0 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb4b415b270 0x7fb4b40d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb4b415b270 0x7fb4b40d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41994/0 (socket says 172.21.15.179:41994) 2024-05-08T22:33:27.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 -- 172.21.15.179:0/173706025 learned_addr learned my addr 172.21.15.179:0/173706025 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:27.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b3fff700 1 --2- 172.21.15.179:0/173706025 >> v2:172.21.15.195:3300/0 conn(0x7fb4b40cbee0 0x7fb4b40b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 -- 172.21.15.179:0/173706025 >> v2:172.21.15.195:3300/0 conn(0x7fb4b40cbee0 msgr2=0x7fb4b40b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 --2- 172.21.15.179:0/173706025 >> v2:172.21.15.195:3300/0 conn(0x7fb4b40cbee0 0x7fb4b40b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 -- 172.21.15.179:0/173706025 >> v1:172.21.15.195:6789/0 conn(0x7fb4b40d05e0 legacy=0x7fb4b40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 -- 172.21.15.179:0/173706025 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40b0330 con 0x7fb4b415b270 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b37fe700 1 --2- 172.21.15.179:0/173706025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40d00a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40b72b0 tx=0x7fb4ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633a766c99816ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b27fc700 1 -- 172.21.15.179:0/173706025 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00e070 con 0x7fb4b415b270 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b27fc700 1 -- 172.21.15.179:0/173706025 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac00b910 con 0x7fb4b415b270 2024-05-08T22:33:27.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b27fc700 1 -- 172.21.15.179:0/173706025 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00bcc0 con 0x7fb4b415b270 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- 172.21.15.179:0/173706025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 msgr2=0x7fb4b40d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/173706025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40d00a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40b72b0 tx=0x7fb4ac00b330 comp rx=0 tx=0).stop 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- 172.21.15.179:0/173706025 shutdown_connections 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/173706025 >> v2:172.21.15.195:3300/0 conn(0x7fb4b40cbee0 0x7fb4b40b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/173706025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40d00a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- 172.21.15.179:0/173706025 >> 172.21.15.179:0/173706025 conn(0x7fb4b40d1180 msgr2=0x7fb4b40d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- 172.21.15.179:0/173706025 shutdown_connections 2024-05-08T22:33:27.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.043+0000 7fb4b9e89700 1 -- 172.21.15.179:0/173706025 wait complete. 2024-05-08T22:33:27.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 Processor -- start 2024-05-08T22:33:27.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 -- start start 2024-05-08T22:33:27.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b40cbee0 0x7fb4b415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b40d05e0 0x7fb4b415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40c7c60 con 0x7fb4b415b270 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b415d0b0 con 0x7fb4b40cbee0 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b9e89700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b415d390 con 0x7fb4b40d05e0 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41998/0 (socket says 172.21.15.179:41998) 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 -- 172.21.15.179:0/3722505780 learned_addr learned my addr 172.21.15.179:0/3722505780 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b37fe700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b40cbee0 0x7fb4b415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b2ffd700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b40d05e0 0x7fb4b415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 -- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b40d05e0 msgr2=0x7fb4b415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b40d05e0 0x7fb4b415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 -- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b40cbee0 msgr2=0x7fb4b415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b40cbee0 0x7fb4b415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.047+0000 7fb4b3fff700 1 -- 172.21.15.179:0/3722505780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac009b30 con 0x7fb4b415b270 2024-05-08T22:33:27.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4b3fff700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400e900 tx=0x7fb4a400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4013680 con 0x7fb4b415b270 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a40137e0 con 0x7fb4b415b270 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b415d670 con 0x7fb4b415b270 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4013b70 con 0x7fb4b415b270 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b415dc30 con 0x7fb4b415b270 2024-05-08T22:33:27.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b4073290 con 0x7fb4b415b270 2024-05-08T22:33:27.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4a4013cd0 con 0x7fb4b415b270 2024-05-08T22:33:27.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.051+0000 7fb4abfff700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 0x7fb498068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.055+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb4a40843d0 con 0x7fb4b415b270 2024-05-08T22:33:27.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.055+0000 7fb4b37fe700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 0x7fb498068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.055+0000 7fb4b37fe700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 0x7fb498068c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f80 tx=0x7fb4ac009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.063+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb4a4051460 con 0x7fb4b415b270 2024-05-08T22:33:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4b415ddf0 con 0x7fb4b415b270 2024-05-08T22:33:27.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4abfff700 1 -- 172.21.15.179:0/3722505780 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4a4054a80 con 0x7fb4b415b270 2024-05-08T22:33:27.234 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:27.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 msgr2=0x7fb498068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 0x7fb498068c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f80 tx=0x7fb4ac009000 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 msgr2=0x7fb4b40c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400e900 tx=0x7fb4a400ec10 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 shutdown_connections 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb498066760 0x7fb498068c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b415b270 0x7fb4b40c7720 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4b40d05e0 0x7fb4b415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 --2- 172.21.15.179:0/3722505780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b40cbee0 0x7fb4b415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.231+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 >> 172.21.15.179:0/3722505780 conn(0x7fb4b40d1180 msgr2=0x7fb4b40b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.235+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 shutdown_connections 2024-05-08T22:33:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.235+0000 7fb4b9e89700 1 -- 172.21.15.179:0/3722505780 wait complete. 2024-05-08T22:33:27.249 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:33:27.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 Processor -- start 2024-05-08T22:33:27.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 -- start start 2024-05-08T22:33:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7eb815d200 0x7f7eb80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7eb815d5e0 0x7f7eb80ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb806c550 0x7f7eb806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb80ce950 con 0x7f7eb815d5e0 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb806df30 con 0x7f7eb815d200 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebfff4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb806cf20 con 0x7f7eb806c550 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7eb815d5e0 0x7f7eb80ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7eb815d5e0 0x7f7eb80ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42014/0 (socket says 172.21.15.179:42014) 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 -- 172.21.15.179:0/2211290519 learned_addr learned my addr 172.21.15.179:0/2211290519 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebdd90700 1 --2- 172.21.15.179:0/2211290519 >> v2:172.21.15.195:3300/0 conn(0x7f7eb815d200 0x7f7eb80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebe591700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb806c550 0x7f7eb806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 -- 172.21.15.179:0/2211290519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb806c550 msgr2=0x7f7eb806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb806c550 0x7f7eb806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 -- 172.21.15.179:0/2211290519 >> v2:172.21.15.195:3300/0 conn(0x7f7eb815d200 msgr2=0x7f7eb80ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 --2- 172.21.15.179:0/2211290519 >> v2:172.21.15.195:3300/0 conn(0x7f7eb815d200 0x7f7eb80ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.363+0000 7f7ebd58f700 1 -- 172.21.15.179:0/2211290519 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb80cfa40 con 0x7f7eb815d5e0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebd58f700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb80ceca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80cfc60 tx=0x7f7ea8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9ba0a23e3318a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebcd8e700 1 -- 172.21.15.179:0/2211290519 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8011830 con 0x7f7eb815d5e0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebcd8e700 1 -- 172.21.15.179:0/2211290519 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea8011990 con 0x7f7eb815d5e0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebcd8e700 1 -- 172.21.15.179:0/2211290519 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8011d20 con 0x7f7eb815d5e0 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- 172.21.15.179:0/2211290519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 msgr2=0x7f7eb80ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb80ceca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80cfc60 tx=0x7f7ea8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- 172.21.15.179:0/2211290519 shutdown_connections 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb806c550 0x7f7eb806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/2211290519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb80ceca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/2211290519 >> v2:172.21.15.195:3300/0 conn(0x7f7eb815d200 0x7f7eb80ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- 172.21.15.179:0/2211290519 >> 172.21.15.179:0/2211290519 conn(0x7f7eb80b18d0 msgr2=0x7f7eb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- 172.21.15.179:0/2211290519 shutdown_connections 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- 172.21.15.179:0/2211290519 wait complete. 2024-05-08T22:33:27.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 Processor -- start 2024-05-08T22:33:27.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.367+0000 7f7ebfff4700 1 -- start start 2024-05-08T22:33:27.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb806c550 0x7f7eb8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb815d200 0x7f7eb814a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8159500 con 0x7f7eb815d5e0 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8157cd0 con 0x7f7eb806c550 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8158b30 con 0x7f7eb815d200 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42024/0 (socket says 172.21.15.179:42024) 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 -- 172.21.15.179:0/1641018314 learned_addr learned my addr 172.21.15.179:0/1641018314 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebdd90700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb806c550 0x7f7eb8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebd58f700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb815d200 0x7f7eb814a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 -- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb815d200 msgr2=0x7f7eb814a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb815d200 0x7f7eb814a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 -- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb806c550 msgr2=0x7f7eb8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb806c550 0x7f7eb8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 -- 172.21.15.179:0/1641018314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea8009570 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebe591700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009d30 tx=0x7f7eb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000cc90 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb000f040 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0009e80 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb814a810 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.371+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb8157f20 con 0x7f7eb815d5e0 2024-05-08T22:33:27.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8073290 con 0x7f7eb815d5e0 2024-05-08T22:33:27.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7eb0014480 con 0x7f7eb815d5e0 2024-05-08T22:33:27.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7eae7fc700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 0x7f7e9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7eb0084b00 con 0x7f7eb815d5e0 2024-05-08T22:33:27.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7ebdd90700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 0x7f7e9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.375+0000 7f7ebdd90700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 0x7f7e9c068c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009b70 tx=0x7f7eb4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.383+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7eb0051c40 con 0x7f7eb815d5e0 2024-05-08T22:33:27.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7eb8158c70 con 0x7f7eb815d5e0 2024-05-08T22:33:27.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7eae7fc700 1 -- 172.21.15.179:0/1641018314 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7eb00180a0 con 0x7f7eb815d5e0 2024-05-08T22:33:27.558 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 msgr2=0x7f7e9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 0x7f7e9c068c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009b70 tx=0x7f7eb4009240 comp rx=0 tx=0).stop 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 msgr2=0x7f7eb8152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009d30 tx=0x7f7eb000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 shutdown_connections 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e9c066760 0x7f7e9c068c20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb815d5e0 0x7f7eb8152d30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eb815d200 0x7f7eb814a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 --2- 172.21.15.179:0/1641018314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eb806c550 0x7f7eb8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 >> 172.21.15.179:0/1641018314 conn(0x7f7eb80b18d0 msgr2=0x7f7eb80ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 shutdown_connections 2024-05-08T22:33:27.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.555+0000 7f7ebfff4700 1 -- 172.21.15.179:0/1641018314 wait complete. 2024-05-08T22:33:27.573 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:33:27.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 Processor -- start 2024-05-08T22:33:27.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 -- start start 2024-05-08T22:33:27.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25340b4ff0 0x7f25340d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340d1fe0 con 0x7f25340d18d0 2024-05-08T22:33:27.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340d22c0 con 0x7f253415d2f0 2024-05-08T22:33:27.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2539c8b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253415d6d0 con 0x7f25340b4ff0 2024-05-08T22:33:27.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2533fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25340b4ff0 0x7f25340d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f253948a700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f25340d18d0 0x7f25340b6460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:34224/0 (socket says 172.21.15.179:34224) 2024-05-08T22:33:27.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f2533fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25340b4ff0 0x7f25340d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42032/0 (socket says 172.21.15.179:42032) 2024-05-08T22:33:27.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.687+0000 7f253948a700 1 -- 172.21.15.179:0/2548335000 learned_addr learned my addr 172.21.15.179:0/2548335000 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:27.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2533fff700 1 -- 172.21.15.179:0/2548335000 >> v1:172.21.15.195:6789/0 conn(0x7f253415d2f0 legacy=0x7f25340b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2533fff700 1 -- 172.21.15.179:0/2548335000 >> v1:172.21.15.179:6789/0 conn(0x7f25340d18d0 legacy=0x7f25340b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2533fff700 1 -- 172.21.15.179:0/2548335000 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340d25a0 con 0x7f25340b4ff0 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2533fff700 1 --2- 172.21.15.179:0/2548335000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25340b4ff0 0x7f25340d1390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f251c009840 tx=0x7f251c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7b7cced891643f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f25337fe700 1 -- 172.21.15.179:0/2548335000 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c005e50 con 0x7f25340b4ff0 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f25337fe700 1 -- 172.21.15.179:0/2548335000 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c006030 con 0x7f25340b4ff0 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f25337fe700 1 -- 172.21.15.179:0/2548335000 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0063c0 con 0x7f25340b4ff0 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- 172.21.15.179:0/2548335000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25340b4ff0 msgr2=0x7f25340d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 --2- 172.21.15.179:0/2548335000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25340b4ff0 0x7f25340d1390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f251c009840 tx=0x7f251c009b50 comp rx=0 tx=0).stop 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- 172.21.15.179:0/2548335000 shutdown_connections 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 --2- 172.21.15.179:0/2548335000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25340b4ff0 0x7f25340d1390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- 172.21.15.179:0/2548335000 >> 172.21.15.179:0/2548335000 conn(0x7f25340cfac0 msgr2=0x7f25340cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- 172.21.15.179:0/2548335000 shutdown_connections 2024-05-08T22:33:27.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- 172.21.15.179:0/2548335000 wait complete. 2024-05-08T22:33:27.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 Processor -- start 2024-05-08T22:33:27.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.691+0000 7f2539c8b700 1 -- start start 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25340b4ff0 0x7f25340c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25340d18d0 0x7f25340cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340caf00 con 0x7f253415d2f0 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340ca5d0 con 0x7f25340b4ff0 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340ca840 con 0x7f25340d18d0 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42038/0 (socket says 172.21.15.179:42038) 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 -- 172.21.15.179:0/1387692867 learned_addr learned my addr 172.21.15.179:0/1387692867 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2533fff700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25340d18d0 0x7f25340cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2538c89700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25340b4ff0 0x7f25340c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 -- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25340d18d0 msgr2=0x7f25340cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25340d18d0 0x7f25340cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 -- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25340b4ff0 msgr2=0x7f25340c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25340b4ff0 0x7f25340c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 -- 172.21.15.179:0/1387692867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c009450 con 0x7f253415d2f0 2024-05-08T22:33:27.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f253948a700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f252c00cbf0 tx=0x7f252c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00a9a0 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00ab00 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00ae70 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340cab20 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.695+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25341f3a20 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253404e700 con 0x7f253415d2f0 2024-05-08T22:33:27.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f252c01b3b0 con 0x7f253415d2f0 2024-05-08T22:33:27.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f25317fa700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 0x7f2514068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:27.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f252c083d40 con 0x7f253415d2f0 2024-05-08T22:33:27.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f2538c89700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 0x7f2514068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:27.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.699+0000 7f2538c89700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 0x7f2514068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2524000f60 tx=0x7f2524009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:27.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.707+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f252c050e70 con 0x7f253415d2f0 2024-05-08T22:33:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.875+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2534154710 con 0x7f253415d2f0 2024-05-08T22:33:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.875+0000 7f25317fa700 1 -- 172.21.15.179:0/1387692867 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f252c0170a0 con 0x7f253415d2f0 2024-05-08T22:33:27.880 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:27.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 msgr2=0x7f2514068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 0x7f2514068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2524000f60 tx=0x7f2524009040 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 msgr2=0x7f25340ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f252c00cbf0 tx=0x7f252c00ccd0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 shutdown_connections 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2514066730 0x7f2514068bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253415d2f0 0x7f25340ccb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25340d18d0 0x7f25340cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 --2- 172.21.15.179:0/1387692867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25340b4ff0 0x7f25340c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 >> 172.21.15.179:0/1387692867 conn(0x7f25340cfac0 msgr2=0x7f25340b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 shutdown_connections 2024-05-08T22:33:27.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:27.879+0000 7f2539c8b700 1 -- 172.21.15.179:0/1387692867 wait complete. 2024-05-08T22:33:27.895 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:33:28.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.007+0000 7f064359e700 1 Processor -- start 2024-05-08T22:33:28.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.007+0000 7f064359e700 1 -- start start 2024-05-08T22:33:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f064406eba0 0x7f064406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 0x7f06440cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064415ccb0 con 0x7f064415c8d0 2024-05-08T22:33:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064406be30 con 0x7f064406eba0 2024-05-08T22:33:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440b2e30 con 0x7f064406b980 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f064406eba0 0x7f064406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f064406eba0 0x7f064406b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42042/0 (socket says 172.21.15.179:42042) 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 -- 172.21.15.179:0/1949352729 learned_addr learned my addr 172.21.15.179:0/1949352729 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0642d9d700 1 --2- 172.21.15.179:0/1949352729 >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 0x7f06440cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 -- 172.21.15.179:0/1949352729 >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 msgr2=0x7f06440cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 --2- 172.21.15.179:0/1949352729 >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 0x7f06440cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 -- 172.21.15.179:0/1949352729 >> v1:172.21.15.195:6789/0 conn(0x7f064415c8d0 legacy=0x7f06440cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 -- 172.21.15.179:0/1949352729 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440cb320 con 0x7f064406eba0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0642d9d700 1 --2- 172.21.15.179:0/1949352729 >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 0x7f06440cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f0641d9b700 1 --2- 172.21.15.179:0/1949352729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064406eba0 0x7f064406b440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f064406dea0 tx=0x7f063400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b23dfa9940eb25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064159a700 1 -- 172.21.15.179:0/1949352729 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400f070 con 0x7f064406eba0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064159a700 1 -- 172.21.15.179:0/1949352729 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063400a2e0 con 0x7f064406eba0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064159a700 1 -- 172.21.15.179:0/1949352729 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400ccc0 con 0x7f064406eba0 2024-05-08T22:33:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 -- 172.21.15.179:0/1949352729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064406eba0 msgr2=0x7f064406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.011+0000 7f064359e700 1 --2- 172.21.15.179:0/1949352729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064406eba0 0x7f064406b440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f064406dea0 tx=0x7f063400c330 comp rx=0 tx=0).stop 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/1949352729 shutdown_connections 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 --2- 172.21.15.179:0/1949352729 >> v2:172.21.15.195:3300/0 conn(0x7f064406b980 0x7f06440cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 --2- 172.21.15.179:0/1949352729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064406eba0 0x7f064406b440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/1949352729 >> 172.21.15.179:0/1949352729 conn(0x7f06440ce960 msgr2=0x7f06440ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/1949352729 shutdown_connections 2024-05-08T22:33:28.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/1949352729 wait complete. 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 Processor -- start 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- start start 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f064406b980 0x7f0644152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f064406eba0 0x7f0644142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064414eec0 con 0x7f064415c8d0 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0644142cc0 con 0x7f064406b980 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0644153400 con 0x7f064406eba0 2024-05-08T22:33:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42058/0 (socket says 172.21.15.179:42058) 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 -- 172.21.15.179:0/621348163 learned_addr learned my addr 172.21.15.179:0/621348163 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064259c700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f064406b980 0x7f0644152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0641d9b700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f064406eba0 0x7f0644142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 -- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f064406eba0 msgr2=0x7f0644142780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f064406eba0 0x7f0644142780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 -- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f064406b980 msgr2=0x7f0644152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f064406b980 0x7f0644152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 -- 172.21.15.179:0/621348163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063400ac40 con 0x7f064415c8d0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f0642d9d700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c7c0 tx=0x7f063c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c011800 con 0x7f064415c8d0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c011960 con 0x7f064415c8d0 2024-05-08T22:33:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06441536e0 con 0x7f064415c8d0 2024-05-08T22:33:28.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c011d10 con 0x7f064415c8d0 2024-05-08T22:33:28.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.015+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0644153ca0 con 0x7f064415c8d0 2024-05-08T22:33:28.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644153e90 con 0x7f064415c8d0 2024-05-08T22:33:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f063c021350 con 0x7f064415c8d0 2024-05-08T22:33:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f063affd700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 0x7f0624068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f063c084480 con 0x7f064415c8d0 2024-05-08T22:33:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f064259c700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 0x7f0624068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.019+0000 7f064259c700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 0x7f0624068bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f062c000f60 tx=0x7f062c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:28.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.027+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f063c051500 con 0x7f064415c8d0 2024-05-08T22:33:28.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.207+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f064404e700 con 0x7f064415c8d0 2024-05-08T22:33:28.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.207+0000 7f063affd700 1 -- 172.21.15.179:0/621348163 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f063c054b20 con 0x7f064415c8d0 2024-05-08T22:33:28.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 msgr2=0x7f0624068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 0x7f0624068bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f062c000f60 tx=0x7f062c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 msgr2=0x7f064413f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c7c0 tx=0x7f063c00cb80 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 shutdown_connections 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0624066730 0x7f0624068bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f064415c8d0 0x7f064413f420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f064406eba0 0x7f0644142780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 --2- 172.21.15.179:0/621348163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f064406b980 0x7f0644152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 >> 172.21.15.179:0/621348163 conn(0x7f06440ce960 msgr2=0x7f0644150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 shutdown_connections 2024-05-08T22:33:28.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:28.211+0000 7f064359e700 1 -- 172.21.15.179:0/621348163 wait complete. 2024-05-08T22:33:28.217 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:33:28.228 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:33:28.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:28.228 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T22:33:28.254 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi179.stderr:ceph-fuse[31435]: fuse finished with error 0 and tester_r 0 2024-05-08T22:33:28.255 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:33:34.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:33:34.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:34.357 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T22:33:34.370 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:34.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.531+0000 7fb927fff700 1 Processor -- start 2024-05-08T22:33:34.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.531+0000 7fb927fff700 1 -- start start 2024-05-08T22:33:34.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb92815d1d0 0x7fb92806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb9280ce350 0x7fb9280d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92815d5b0 con 0x7fb92806bb00 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92806c5d0 con 0x7fb9280ce350 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280b6010 con 0x7fb92815d1d0 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9267fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb92806bb00 0x7fb9280cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:34230/0 (socket says 172.21.15.179:34230) 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9267fc700 1 -- 172.21.15.179:0/2858283096 learned_addr learned my addr 172.21.15.179:0/2858283096 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb9280ce350 0x7fb9280d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb926ffd700 1 --2- 172.21.15.179:0/2858283096 >> v2:172.21.15.195:3300/0 conn(0x7fb92815d1d0 0x7fb92806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 -- 172.21.15.179:0/2858283096 >> v2:172.21.15.195:3300/0 conn(0x7fb92815d1d0 msgr2=0x7fb92806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 --2- 172.21.15.179:0/2858283096 >> v2:172.21.15.195:3300/0 conn(0x7fb92815d1d0 0x7fb92806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 -- 172.21.15.179:0/2858283096 >> v1:172.21.15.179:6789/0 conn(0x7fb92806bb00 legacy=0x7fb9280cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 -- 172.21.15.179:0/2858283096 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92806c010 con 0x7fb9280ce350 2024-05-08T22:33:34.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb9277fe700 1 --2- 172.21.15.179:0/2858283096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280d19c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9280677c0 tx=0x7fb918009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2614f2aa3a3bcd80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb925ffb700 1 -- 172.21.15.179:0/2858283096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918011830 con 0x7fb9280ce350 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb925ffb700 1 -- 172.21.15.179:0/2858283096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918011990 con 0x7fb9280ce350 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb925ffb700 1 -- 172.21.15.179:0/2858283096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918011d20 con 0x7fb9280ce350 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- 172.21.15.179:0/2858283096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 msgr2=0x7fb9280d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 --2- 172.21.15.179:0/2858283096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280d19c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9280677c0 tx=0x7fb918009b30 comp rx=0 tx=0).stop 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- 172.21.15.179:0/2858283096 shutdown_connections 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 --2- 172.21.15.179:0/2858283096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280d19c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 --2- 172.21.15.179:0/2858283096 >> v2:172.21.15.195:3300/0 conn(0x7fb92815d1d0 0x7fb92806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- 172.21.15.179:0/2858283096 >> 172.21.15.179:0/2858283096 conn(0x7fb9280d2e70 msgr2=0x7fb9280d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- 172.21.15.179:0/2858283096 shutdown_connections 2024-05-08T22:33:34.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.535+0000 7fb927fff700 1 -- 172.21.15.179:0/2858283096 wait complete. 2024-05-08T22:33:34.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 Processor -- start 2024-05-08T22:33:34.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 -- start start 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 0x7fb9280b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb92815d1d0 0x7fb92814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280cb550 con 0x7fb9280ce350 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280c95a0 con 0x7fb92806bb00 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb927fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280cac20 con 0x7fb92815d1d0 2024-05-08T22:33:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42076/0 (socket says 172.21.15.179:42076) 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 -- 172.21.15.179:0/428466704 learned_addr learned my addr 172.21.15.179:0/428466704 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb926ffd700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 0x7fb9280b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9277fe700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb92815d1d0 0x7fb92814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 -- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb92815d1d0 msgr2=0x7fb92814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb92815d1d0 0x7fb92814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 -- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 msgr2=0x7fb9280b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 0x7fb9280b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb9267fc700 1 -- 172.21.15.179:0/428466704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180094b0 con 0x7fb9280ce350 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.539+0000 7fb926ffd700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 0x7fb9280b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb9267fc700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb92000cb10 tx=0x7fb92000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000a770 con 0x7fb9280ce350 2024-05-08T22:33:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92000a8d0 con 0x7fb9280ce350 2024-05-08T22:33:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000ac40 con 0x7fb9280ce350 2024-05-08T22:33:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280caf00 con 0x7fb9280ce350 2024-05-08T22:33:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280c7dd0 con 0x7fb9280ce350 2024-05-08T22:33:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb92001b3b0 con 0x7fb9280ce350 2024-05-08T22:33:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.543+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9281585d0 con 0x7fb9280ce350 2024-05-08T22:33:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.547+0000 7fb91f7fe700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 0x7fb908068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.547+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb920083ab0 con 0x7fb9280ce350 2024-05-08T22:33:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.547+0000 7fb926ffd700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 0x7fb908068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.547+0000 7fb926ffd700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 0x7fb908068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb910009c40 tx=0x7fb910008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.551+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb920050bd0 con 0x7fb9280ce350 2024-05-08T22:33:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.719+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb92815d5b0 con 0x7fb9280ce350 2024-05-08T22:33:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb91f7fe700 1 -- 172.21.15.179:0/428466704 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7fb92815d5b0 con 0x7fb9280ce350 2024-05-08T22:33:34.723 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:34.723 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 msgr2=0x7fb908068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 0x7fb908068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb910009c40 tx=0x7fb910008040 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 msgr2=0x7fb9280c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb92000cb10 tx=0x7fb92000cb40 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 shutdown_connections 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb908066760 0x7fb908068c20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb92815d1d0 0x7fb92814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9280ce350 0x7fb9280c8ae0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 --2- 172.21.15.179:0/428466704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb92806bb00 0x7fb9280b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 >> 172.21.15.179:0/428466704 conn(0x7fb9280d2e70 msgr2=0x7fb928157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 shutdown_connections 2024-05-08T22:33:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.723+0000 7fb927fff700 1 -- 172.21.15.179:0/428466704 wait complete. 2024-05-08T22:33:34.738 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:34.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.855+0000 7fb385334700 1 Processor -- start 2024-05-08T22:33:34.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.855+0000 7fb385334700 1 -- start start 2024-05-08T22:33:34.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb3801593d0 0x7fb3800cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3800b1510 con 0x7fb3800ce780 2024-05-08T22:33:34.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38015dea0 con 0x7fb3801597b0 2024-05-08T22:33:34.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3800cec20 con 0x7fb3801593d0 2024-05-08T22:33:34.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb377fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb3800ce780 0x7fb38015d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:34236/0 (socket says 172.21.15.179:34236) 2024-05-08T22:33:34.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb377fff700 1 -- 172.21.15.179:0/1098802435 learned_addr learned my addr 172.21.15.179:0/1098802435 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:34.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37ffff700 1 --2- 172.21.15.179:0/1098802435 >> v2:172.21.15.195:3300/0 conn(0x7fb3801593d0 0x7fb3800cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37f7fe700 1 -- 172.21.15.179:0/1098802435 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252053045 0 0) 0x7fb3800b1510 con 0x7fb3800ce780 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37f7fe700 1 -- 172.21.15.179:0/1098802435 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb36c0035d0 con 0x7fb3800ce780 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37ffff700 1 -- 172.21.15.179:0/1098802435 >> v1:172.21.15.195:6789/0 conn(0x7fb3801597b0 legacy=0x7fb38006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37ffff700 1 -- 172.21.15.179:0/1098802435 >> v1:172.21.15.179:6789/0 conn(0x7fb3800ce780 legacy=0x7fb38015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37ffff700 1 -- 172.21.15.179:0/1098802435 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3800cef00 con 0x7fb3801593d0 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37ffff700 1 --2- 172.21.15.179:0/1098802435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3800cc4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb37000a9f0 tx=0x7fb37000ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e9c6e1005a6242 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37f7fe700 1 -- 172.21.15.179:0/1098802435 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000cb70 con 0x7fb3801593d0 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37f7fe700 1 -- 172.21.15.179:0/1098802435 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000ae90 con 0x7fb3801593d0 2024-05-08T22:33:34.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb37f7fe700 1 -- 172.21.15.179:0/1098802435 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370018400 con 0x7fb3801593d0 2024-05-08T22:33:34.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- 172.21.15.179:0/1098802435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 msgr2=0x7fb3800cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 --2- 172.21.15.179:0/1098802435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3800cc4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb37000a9f0 tx=0x7fb37000ad00 comp rx=0 tx=0).stop 2024-05-08T22:33:34.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- 172.21.15.179:0/1098802435 shutdown_connections 2024-05-08T22:33:34.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 --2- 172.21.15.179:0/1098802435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3800cc4e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- 172.21.15.179:0/1098802435 >> 172.21.15.179:0/1098802435 conn(0x7fb380098a70 msgr2=0x7fb3800b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:34.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- 172.21.15.179:0/1098802435 shutdown_connections 2024-05-08T22:33:34.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.859+0000 7fb385334700 1 -- 172.21.15.179:0/1098802435 wait complete. 2024-05-08T22:33:34.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 Processor -- start 2024-05-08T22:33:34.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- start start 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3800ce780 0x7fb380159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3801586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c43c0 con 0x7fb3801597b0 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c5b00 con 0x7fb3801593d0 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38006bf80 con 0x7fb3800ce780 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb37ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3800ce780 0x7fb380159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42078/0 (socket says 172.21.15.179:42078) 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb377fff700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3801586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 -- 172.21.15.179:0/3516772484 learned_addr learned my addr 172.21.15.179:0/3516772484 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 -- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3800ce780 msgr2=0x7fb380159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3800ce780 0x7fb380159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 -- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 msgr2=0x7fb3801586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3801586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 -- 172.21.15.179:0/3516772484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37000a650 con 0x7fb3801597b0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb384b33700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb37800cb50 tx=0x7fb37800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378003c20 con 0x7fb3801597b0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378003d80 con 0x7fb3801597b0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3800c3a20 con 0x7fb3801597b0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800aa20 con 0x7fb3801597b0 2024-05-08T22:33:34.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.863+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3800c3fe0 con 0x7fb3801597b0 2024-05-08T22:33:34.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3800c4500 con 0x7fb3801597b0 2024-05-08T22:33:34.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb37800ab80 con 0x7fb3801597b0 2024-05-08T22:33:34.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb37d7fa700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 0x7fb360068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:34.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb3780844e0 con 0x7fb3801597b0 2024-05-08T22:33:34.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb37ffff700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 0x7fb360068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:34.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.867+0000 7fb37ffff700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 0x7fb360068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb37000a1c0 tx=0x7fb37001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:34.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:34.875+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb3800c4500 con 0x7fb3801597b0 2024-05-08T22:33:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.051+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3800c4500 con 0x7fb3801597b0 2024-05-08T22:33:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.055+0000 7fb37d7fa700 1 -- 172.21.15.179:0/3516772484 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7fb3800c4500 con 0x7fb3801597b0 2024-05-08T22:33:35.056 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:35.056 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 msgr2=0x7fb360068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 0x7fb360068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb37000a1c0 tx=0x7fb37001b040 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 msgr2=0x7fb3800c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb37800cb50 tx=0x7fb37800ce60 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 shutdown_connections 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb360066730 0x7fb360068bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3801593d0 0x7fb3801586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3801597b0 0x7fb3800c6c50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 --2- 172.21.15.179:0/3516772484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3800ce780 0x7fb380159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 >> 172.21.15.179:0/3516772484 conn(0x7fb380098a70 msgr2=0x7fb3800ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 shutdown_connections 2024-05-08T22:33:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.059+0000 7fb385334700 1 -- 172.21.15.179:0/3516772484 wait complete. 2024-05-08T22:33:35.071 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:33:35.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.187+0000 7f0e1ca8b700 1 Processor -- start 2024-05-08T22:33:35.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.187+0000 7f0e1ca8b700 1 -- start start 2024-05-08T22:33:35.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180ce2b0 0x7f0e1806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1815d4a0 con 0x7f0e180d1d20 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1806abb0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180d2500 con 0x7f0e1815d0c0 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180ce2b0 0x7f0e1806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180ce2b0 0x7f0e1806b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42082/0 (socket says 172.21.15.179:42082) 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 -- 172.21.15.179:0/2632744897 learned_addr learned my addr 172.21.15.179:0/2632744897 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e177fe700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e167fc700 1 -- 172.21.15.179:0/2632744897 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072641222 0 0) 0x7f0e1815d4a0 con 0x7f0e180d1d20 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 -- 172.21.15.179:0/2632744897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 msgr2=0x7f0e180b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 -- 172.21.15.179:0/2632744897 >> v1:172.21.15.179:6789/0 conn(0x7f0e180d1d20 legacy=0x7f0e180cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 -- 172.21.15.179:0/2632744897 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180d27e0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e17fff700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e1806b370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009e50 tx=0x7f0e1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3379533076a4959e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:35.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e167fc700 1 -- 172.21.15.179:0/2632744897 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000e070 con 0x7f0e180ce2b0 2024-05-08T22:33:35.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e167fc700 1 -- 172.21.15.179:0/2632744897 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000b9c0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e167fc700 1 -- 172.21.15.179:0/2632744897 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000bd70 con 0x7f0e180ce2b0 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/2632744897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 msgr2=0x7f0e1806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e1806b370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009e50 tx=0x7f0e1000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/2632744897 shutdown_connections 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e1806b370 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/2632744897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/2632744897 >> 172.21.15.179:0/2632744897 conn(0x7f0e180d2ed0 msgr2=0x7f0e180d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/2632744897 shutdown_connections 2024-05-08T22:33:35.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.191+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/2632744897 wait complete. 2024-05-08T22:33:35.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 Processor -- start 2024-05-08T22:33:35.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- start start 2024-05-08T22:33:35.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e18157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e18157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cb090 con 0x7f0e180ce2b0 2024-05-08T22:33:35.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180ca6f0 con 0x7f0e180d1d20 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180ca9d0 con 0x7f0e1815d0c0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42084/0 (socket says 172.21.15.179:42084) 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 -- 172.21.15.179:0/1116995432 learned_addr learned my addr 172.21.15.179:0/1116995432 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e16ffd700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e18157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e17fff700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e18157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 -- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 msgr2=0x7f0e18157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e18157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 -- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 msgr2=0x7f0e18157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e18157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 -- 172.21.15.179:0/1116995432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10009b30 con 0x7f0e180ce2b0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e177fe700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d800 tx=0x7f0e0800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:35.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e080125f0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08012750 con 0x7f0e180ce2b0 2024-05-08T22:33:35.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e180cacb0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08012ac0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.195+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e18155610 con 0x7f0e180ce2b0 2024-05-08T22:33:35.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e180b7cf0 con 0x7f0e180ce2b0 2024-05-08T22:33:35.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e08012c20 con 0x7f0e180ce2b0 2024-05-08T22:33:35.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0dfffff700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:35.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e08084e90 con 0x7f0e180ce2b0 2024-05-08T22:33:35.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0e16ffd700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:35.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.199+0000 7f0e16ffd700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e180c8b90 tx=0x7f0e0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:35.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.207+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e08052080 con 0x7f0e180ce2b0 2024-05-08T22:33:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:35.415+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e1804e700 con 0x7f0e180ce2b0 2024-05-08T22:33:36.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.007+0000 7f0dfffff700 1 -- 172.21.15.179:0/1116995432 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e080556a0 con 0x7f0e180ce2b0 2024-05-08T22:33:36.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 msgr2=0x7f0e04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:36.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e180c8b90 tx=0x7f0e0c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:36.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 msgr2=0x7f0e180c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:36.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d800 tx=0x7f0e0800db10 comp rx=0 tx=0).stop 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 shutdown_connections 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e18157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e18157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 --2- 172.21.15.179:0/1116995432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180ce2b0 0x7f0e180c83d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 >> 172.21.15.179:0/1116995432 conn(0x7f0e180d2ed0 msgr2=0x7f0e180b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 shutdown_connections 2024-05-08T22:33:36.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.011+0000 7f0e1ca8b700 1 -- 172.21.15.179:0/1116995432 wait complete. 2024-05-08T22:33:36.032 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-08T22:33:36.032 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:33:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 Processor -- start 2024-05-08T22:33:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 -- start start 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe84c142890 0x7fe84c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe84c142fd0 0x7fe84c06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c0b22d0 con 0x7fe84c142890 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c0b19e0 con 0x7fe84c142fd0 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c15e640 con 0x7fe84c0b2ca0 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84a59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe84c142890 0x7fe84c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84a59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe84c142890 0x7fe84c0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34726/0 (socket says 172.21.15.179:34726) 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84a59c700 1 -- 172.21.15.179:0/480631892 learned_addr learned my addr 172.21.15.179:0/480631892 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe84ad9d700 1 --2- 172.21.15.179:0/480631892 >> v2:172.21.15.195:3300/0 conn(0x7fe84c142fd0 0x7fe84c06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.155+0000 7fe849d9b700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 -- 172.21.15.179:0/480631892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 msgr2=0x7fe84c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 -- 172.21.15.179:0/480631892 >> v2:172.21.15.195:3300/0 conn(0x7fe84c142fd0 msgr2=0x7fe84c06d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 --2- 172.21.15.179:0/480631892 >> v2:172.21.15.195:3300/0 conn(0x7fe84c142fd0 0x7fe84c06d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 -- 172.21.15.179:0/480631892 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0b1c80 con 0x7fe84c142890 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84a59c700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142890 0x7fe84c0b2760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0b40a0 tx=0x7fe834009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba97016b35896dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84959a700 1 -- 172.21.15.179:0/480631892 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834011830 con 0x7fe84c142890 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84959a700 1 -- 172.21.15.179:0/480631892 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834011990 con 0x7fe84c142890 2024-05-08T22:33:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84959a700 1 -- 172.21.15.179:0/480631892 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834011d20 con 0x7fe84c142890 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 -- 172.21.15.179:0/480631892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142890 msgr2=0x7fe84c0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142890 0x7fe84c0b2760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0b40a0 tx=0x7fe834009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 -- 172.21.15.179:0/480631892 shutdown_connections 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 --2- 172.21.15.179:0/480631892 >> v2:172.21.15.195:3300/0 conn(0x7fe84c142fd0 0x7fe84c06d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 --2- 172.21.15.179:0/480631892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142890 0x7fe84c0b2760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 -- 172.21.15.179:0/480631892 >> 172.21.15.179:0/480631892 conn(0x7fe84c0cea30 msgr2=0x7fe84c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 -- 172.21.15.179:0/480631892 shutdown_connections 2024-05-08T22:33:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.159+0000 7fe84b59e700 1 -- 172.21.15.179:0/480631892 wait complete. 2024-05-08T22:33:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 Processor -- start 2024-05-08T22:33:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 -- start start 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 0x7fe84c132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c130ba0 con 0x7fe84c142fd0 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c130e80 con 0x7fe84c142890 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c131160 con 0x7fe84c0b2ca0 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84a59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 0x7fe84c132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34738/0 (socket says 172.21.15.179:34738) 2024-05-08T22:33:36.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84a59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:52888/0 (socket says 172.21.15.179:52888) 2024-05-08T22:33:36.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84ad9d700 1 -- 172.21.15.179:0/3938145285 learned_addr learned my addr 172.21.15.179:0/3938145285 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 -- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 msgr2=0x7fe84c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 -- 172.21.15.179:0/3938145285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 msgr2=0x7fe84c133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 -- 172.21.15.179:0/3938145285 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009570 con 0x7fe84c142890 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe84ad9d700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.163+0000 7fe849d9b700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 0x7fe84c132b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe84c06b410 tx=0x7fe83c00eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c013680 con 0x7fe84c142890 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c0137e0 con 0x7fe84c142890 2024-05-08T22:33:36.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c131440 con 0x7fe84c142890 2024-05-08T22:33:36.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c013b70 con 0x7fe84c142890 2024-05-08T22:33:36.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c159b90 con 0x7fe84c142890 2024-05-08T22:33:36.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c04e700 con 0x7fe84c142890 2024-05-08T22:33:36.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe83c013cd0 con 0x7fe84c142890 2024-05-08T22:33:36.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 0x7fe82c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:36.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe83c084330 con 0x7fe84c142890 2024-05-08T22:33:36.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.167+0000 7fe84a59c700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 0x7fe82c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:36.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.171+0000 7fe84a59c700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 0x7fe82c068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe834008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:36.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.179+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe83c051330 con 0x7fe84c142890 2024-05-08T22:33:36.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:36.391+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe84c130fc0 con 0x7fe84c142890 2024-05-08T22:33:37.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.015+0000 7fe842ffd700 1 -- 172.21.15.179:0/3938145285 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe84c130fc0 con 0x7fe84c142890 2024-05-08T22:33:37.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 msgr2=0x7fe82c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 0x7fe82c068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe834008030 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 msgr2=0x7fe84c132b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 0x7fe84c132b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe84c06b410 tx=0x7fe83c00eb90 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 shutdown_connections 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe82c066760 0x7fe82c068c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe84c142890 0x7fe84c132b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84c142fd0 0x7fe84c133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 --2- 172.21.15.179:0/3938145285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe84c0b2ca0 0x7fe84c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 >> 172.21.15.179:0/3938145285 conn(0x7fe84c0cea30 msgr2=0x7fe84c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 shutdown_connections 2024-05-08T22:33:37.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.019+0000 7fe84b59e700 1 -- 172.21.15.179:0/3938145285 wait complete. 2024-05-08T22:33:37.037 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:33:37.037 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:37.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.155+0000 7fb002b7f700 1 Processor -- start 2024-05-08T22:33:37.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.155+0000 7fb002b7f700 1 -- start start 2024-05-08T22:33:37.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cc700 0x7faffc0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faffc0cb6e0 0x7faffc06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc06abb0 con 0x7faffc15b1b0 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc06ae90 con 0x7faffc0cb6e0 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15b9c0 con 0x7faffc0cc700 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faffc0cb6e0 0x7faffc06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faffc0cb6e0 0x7faffc06b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34752/0 (socket says 172.21.15.179:34752) 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 -- 172.21.15.179:0/4130795556 learned_addr learned my addr 172.21.15.179:0/4130795556 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb001b7d700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cc700 0x7faffc0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 -- 172.21.15.179:0/4130795556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cc700 msgr2=0x7faffc0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cc700 0x7faffc0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 -- 172.21.15.179:0/4130795556 >> v1:172.21.15.195:6789/0 conn(0x7faffc15b1b0 legacy=0x7faffc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 -- 172.21.15.179:0/4130795556 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc15bca0 con 0x7faffc0cb6e0 2024-05-08T22:33:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb00237e700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc0cb6e0 0x7faffc06b320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faff4009910 tx=0x7faff4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0a3161dc382e3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb000b7b700 1 -- 172.21.15.179:0/4130795556 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400ba60 con 0x7faffc0cb6e0 2024-05-08T22:33:37.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb000b7b700 1 -- 172.21.15.179:0/4130795556 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400bbc0 con 0x7faffc0cb6e0 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb000b7b700 1 -- 172.21.15.179:0/4130795556 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4009e60 con 0x7faffc0cb6e0 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- 172.21.15.179:0/4130795556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc0cb6e0 msgr2=0x7faffc06b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc0cb6e0 0x7faffc06b320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faff4009910 tx=0x7faff4009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- 172.21.15.179:0/4130795556 shutdown_connections 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc0cb6e0 0x7faffc06b320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 --2- 172.21.15.179:0/4130795556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cc700 0x7faffc0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- 172.21.15.179:0/4130795556 >> 172.21.15.179:0/4130795556 conn(0x7faffc0ce9d0 msgr2=0x7faffc0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- 172.21.15.179:0/4130795556 shutdown_connections 2024-05-08T22:33:37.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.159+0000 7fb002b7f700 1 -- 172.21.15.179:0/4130795556 wait complete. 2024-05-08T22:33:37.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 Processor -- start 2024-05-08T22:33:37.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- start start 2024-05-08T22:33:37.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cb6e0 0x7faffc1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 0x7faffc14ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc14e9f0 con 0x7faffc15b1b0 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc1e52a0 con 0x7faffc0cc700 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc1f3eb0 con 0x7faffc0cb6e0 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34758/0 (socket says 172.21.15.179:34758) 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 -- 172.21.15.179:0/1506530394 learned_addr learned my addr 172.21.15.179:0/1506530394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb001b7d700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cb6e0 0x7faffc1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00137c700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 0x7faffc14ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 -- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cb6e0 msgr2=0x7faffc1f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cb6e0 0x7faffc1f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 -- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 msgr2=0x7faffc14ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 0x7faffc14ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 -- 172.21.15.179:0/1506530394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009570 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00137c700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 0x7faffc14ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb00237e700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faffc0b3e80 tx=0x7faff4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4004310 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff401f070 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40036c0 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc1e4690 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.163+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc1e4c50 con 0x7faffc15b1b0 2024-05-08T22:33:37.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc1e5450 con 0x7faffc15b1b0 2024-05-08T22:33:37.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faff4021030 con 0x7faffc15b1b0 2024-05-08T22:33:37.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7faff27fc700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 0x7fafe8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faff4085750 con 0x7faffc15b1b0 2024-05-08T22:33:37.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7fb001b7d700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 0x7fafe8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.167+0000 7fb001b7d700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 0x7fafe8068c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fafec009a00 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.175+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faffc1e5450 con 0x7faffc15b1b0 2024-05-08T22:33:37.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.383+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faffc1e5450 con 0x7faffc15b1b0 2024-05-08T22:33:37.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.383+0000 7faff27fc700 1 -- 172.21.15.179:0/1506530394 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7faffc1e5450 con 0x7faffc15b1b0 2024-05-08T22:33:37.385 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:37.385 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:33:37.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 msgr2=0x7fafe8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 0x7fafe8068c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fafec009a00 tx=0x7fafec008040 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 msgr2=0x7faffc1f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faffc0b3e80 tx=0x7faff4004770 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 shutdown_connections 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fafe8066760 0x7fafe8068c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faffc0cc700 0x7faffc14ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faffc15b1b0 0x7faffc1f0b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 --2- 172.21.15.179:0/1506530394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faffc0cb6e0 0x7faffc1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 >> 172.21.15.179:0/1506530394 conn(0x7faffc0ce9d0 msgr2=0x7faffc06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 shutdown_connections 2024-05-08T22:33:37.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.387+0000 7fb002b7f700 1 -- 172.21.15.179:0/1506530394 wait complete. 2024-05-08T22:33:37.389 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:33:37.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:37.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 Processor -- start 2024-05-08T22:33:37.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- start start 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa96c15d580 0x7fa96c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa96c0d18a0 0x7fa96c0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0d2640 con 0x7fa96c160f30 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c06b3a0 con 0x7fa96c15d580 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c0b6750 con 0x7fa96c0d18a0 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa96c15d580 0x7fa96c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa96c15d580 0x7fa96c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34766/0 (socket says 172.21.15.179:34766) 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 -- 172.21.15.179:0/664753156 learned_addr learned my addr 172.21.15.179:0/664753156 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa962ffd700 1 --2- 172.21.15.179:0/664753156 >> v2:172.21.15.195:3300/0 conn(0x7fa96c0d18a0 0x7fa96c0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 -- 172.21.15.179:0/664753156 >> v2:172.21.15.195:3300/0 conn(0x7fa96c0d18a0 msgr2=0x7fa96c0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 --2- 172.21.15.179:0/664753156 >> v2:172.21.15.195:3300/0 conn(0x7fa96c0d18a0 0x7fa96c0d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 -- 172.21.15.179:0/664753156 >> v1:172.21.15.195:6789/0 conn(0x7fa96c160f30 legacy=0x7fa96c06b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 -- 172.21.15.179:0/664753156 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0d2360 con 0x7fa96c15d580 2024-05-08T22:33:37.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96b7fe700 1 --2- 172.21.15.179:0/664753156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c15d580 0x7fa96c0b6210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0b58e0 tx=0x7fa95c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310fd8e34a91d1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96affd700 1 -- 172.21.15.179:0/664753156 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c004230 con 0x7fa96c15d580 2024-05-08T22:33:37.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96affd700 1 -- 172.21.15.179:0/664753156 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c004390 con 0x7fa96c15d580 2024-05-08T22:33:37.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa96affd700 1 -- 172.21.15.179:0/664753156 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0036c0 con 0x7fa96c15d580 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- 172.21.15.179:0/664753156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c15d580 msgr2=0x7fa96c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 --2- 172.21.15.179:0/664753156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c15d580 0x7fa96c0b6210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0b58e0 tx=0x7fa95c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- 172.21.15.179:0/664753156 shutdown_connections 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 --2- 172.21.15.179:0/664753156 >> v2:172.21.15.195:3300/0 conn(0x7fa96c0d18a0 0x7fa96c0d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 --2- 172.21.15.179:0/664753156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c15d580 0x7fa96c0b6210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- 172.21.15.179:0/664753156 >> 172.21.15.179:0/664753156 conn(0x7fa96c0d2d90 msgr2=0x7fa96c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- 172.21.15.179:0/664753156 shutdown_connections 2024-05-08T22:33:37.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.507+0000 7fa970e2d700 1 -- 172.21.15.179:0/664753156 wait complete. 2024-05-08T22:33:37.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 Processor -- start 2024-05-08T22:33:37.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 -- start start 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa96c0d18a0 0x7fa96c151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 0x7fa96c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c15fa00 con 0x7fa96c160f30 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c15edf0 con 0x7fa96c15d580 2024-05-08T22:33:37.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa970e2d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c15f0d0 con 0x7fa96c0d18a0 2024-05-08T22:33:37.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96b7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa96c0d18a0 0x7fa96c151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34782/0 (socket says 172.21.15.179:34782) 2024-05-08T22:33:37.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa962ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 0x7fa96c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 -- 172.21.15.179:0/2235682255 learned_addr learned my addr 172.21.15.179:0/2235682255 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 -- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa96c0d18a0 msgr2=0x7fa96c151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa96c0d18a0 0x7fa96c151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 -- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 msgr2=0x7fa96c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 0x7fa96c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 -- 172.21.15.179:0/2235682255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c009570 con 0x7fa96c160f30 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa962ffd700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 0x7fa96c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.511+0000 7fa96bfff700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa96400b970 tx=0x7fa96400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96400d860 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96400d9c0 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96400dd70 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c15f3b0 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c151850 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c151f80 con 0x7fa96c160f30 2024-05-08T22:33:37.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa96400e430 con 0x7fa96c160f30 2024-05-08T22:33:37.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.515+0000 7fa968ff9700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 0x7fa94c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.519+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa964084570 con 0x7fa96c160f30 2024-05-08T22:33:37.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.519+0000 7fa96b7fe700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 0x7fa94c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.519+0000 7fa96b7fe700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 0x7fa94c068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0b58e0 tx=0x7fa95c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.523+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa9640515f0 con 0x7fa96c160f30 2024-05-08T22:33:37.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.691+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa96c04e700 con 0x7fa96c160f30 2024-05-08T22:33:37.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.695+0000 7fa968ff9700 1 -- 172.21.15.179:0/2235682255 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7fa96400ded0 con 0x7fa96c160f30 2024-05-08T22:33:37.695 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:37.695 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 msgr2=0x7fa94c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 0x7fa94c068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0b58e0 tx=0x7fa95c008030 comp rx=0 tx=0).stop 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 msgr2=0x7fa96c156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa96400b970 tx=0x7fa96400bc80 comp rx=0 tx=0).stop 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 shutdown_connections 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa94c066730 0x7fa94c068bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa96c15d580 0x7fa96c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa96c160f30 0x7fa96c156630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 --2- 172.21.15.179:0/2235682255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa96c0d18a0 0x7fa96c151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 >> 172.21.15.179:0/2235682255 conn(0x7fa96c0d2d90 msgr2=0x7fa96c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 shutdown_connections 2024-05-08T22:33:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.699+0000 7fa970e2d700 1 -- 172.21.15.179:0/2235682255 wait complete. 2024-05-08T22:33:37.716 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:33:37.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:33:37.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 Processor -- start 2024-05-08T22:33:37.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 -- start start 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f653806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f653806dc20 0x7f653806e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65380cd200 con 0x7f65380ce2a0 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65380cd4e0 con 0x7f653806dc20 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.835+0000 7f653e3a9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380cd7c0 con 0x7f65381434d0 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653dba8700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f65380ce2a0 0x7f65380b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33402/0 (socket says 172.21.15.179:33402) 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653dba8700 1 -- 172.21.15.179:0/2643680081 learned_addr learned my addr 172.21.15.179:0/2643680081 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653cba6700 1 --2- 172.21.15.179:0/2643680081 >> v2:172.21.15.195:3300/0 conn(0x7f653806dc20 0x7f653806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f653806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503980014 0 0) 0x7f65380cd200 con 0x7f65380ce2a0 2024-05-08T22:33:37.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65240035d0 con 0x7f65380ce2a0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195363377 0 0) 0x7f65240035d0 con 0x7f65380ce2a0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 msgr2=0x7f653806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f653806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 >> v2:172.21.15.195:3300/0 conn(0x7f653806dc20 msgr2=0x7f653806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 --2- 172.21.15.179:0/2643680081 >> v2:172.21.15.195:3300/0 conn(0x7f653806dc20 0x7f653806e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380cdaa0 con 0x7f65380ce2a0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f65300019f0 con 0x7f65380ce2a0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 >> v1:172.21.15.179:6789/0 conn(0x7f65380ce2a0 legacy=0x7f65380b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6524003a20 0x7f6524003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f6524004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6524004db0 0x7f6524005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380cd4e0 con 0x7f6524003a20 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380cd7c0 con 0x7f6524004db0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652400dc40 con 0x7f65381434d0 2024-05-08T22:33:37.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6524003a20 0x7f6524003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653dba8700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6524004db0 0x7f6524005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653cba6700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f6524004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 -- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 msgr2=0x7f6524004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65381434d0 0x7f6524004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 -- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6524004db0 msgr2=0x7f6524005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6524004db0 0x7f6524005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 -- 172.21.15.179:0/2643680081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240035d0 con 0x7f6524003a20 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6524003a20 0x7f6524003e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6534009b70 tx=0x7f65240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1697885a1293d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400e6c0 con 0x7f6524003a20 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400e820 con 0x7f6524003a20 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f652ffff700 1 -- 172.21.15.179:0/2643680081 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400eb90 con 0x7f6524003a20 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 -- 172.21.15.179:0/2643680081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6524003a20 msgr2=0x7f6524003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2643680081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6524003a20 0x7f6524003e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6534009b70 tx=0x7f65240058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653dba8700 1 -- 172.21.15.179:0/2643680081 reap_dead start 2024-05-08T22:33:37.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 -- 172.21.15.179:0/2643680081 shutdown_connections 2024-05-08T22:33:37.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 -- 172.21.15.179:0/2643680081 >> 172.21.15.179:0/2643680081 conn(0x7f65380ce9d0 msgr2=0x7f65380cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:37.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 -- 172.21.15.179:0/2643680081 shutdown_connections 2024-05-08T22:33:37.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.839+0000 7f653e3a9700 1 -- 172.21.15.179:0/2643680081 wait complete. 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 Processor -- start 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- start start 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6524003a20 0x7f65381f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 0x7f65380c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65381e2b60 con 0x7f6538159480 2024-05-08T22:33:37.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380c96d0 con 0x7f653813b100 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65381f31d0 con 0x7f6524003a20 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34808/0 (socket says 172.21.15.179:34808) 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 -- 172.21.15.179:0/2304116811 learned_addr learned my addr 172.21.15.179:0/2304116811 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6524003a20 0x7f65381f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653dba8700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 0x7f65380c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 -- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6524003a20 msgr2=0x7f65381f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6524003a20 0x7f65381f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 -- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 msgr2=0x7f65380c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 0x7f65380c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 -- 172.21.15.179:0/2304116811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65381e28c0 con 0x7f6538159480 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653dba8700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 0x7f65380c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653cba6700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65280116f0 tx=0x7f6528011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528015870 con 0x7f6538159480 2024-05-08T22:33:37.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65280159d0 con 0x7f6538159480 2024-05-08T22:33:37.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65380c8da0 con 0x7f6538159480 2024-05-08T22:33:37.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528015d40 con 0x7f6538159480 2024-05-08T22:33:37.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.843+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65380c9360 con 0x7f6538159480 2024-05-08T22:33:37.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65380c9880 con 0x7f6538159480 2024-05-08T22:33:37.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6528013400 con 0x7f6538159480 2024-05-08T22:33:37.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f652dffb700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 0x7f651c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:37.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6528084640 con 0x7f6538159480 2024-05-08T22:33:37.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 0x7f651c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:37.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.847+0000 7f653d3a7700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 0x7f651c068c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65340090b0 tx=0x7f653401e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:37.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:37.855+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65380c9880 con 0x7f6538159480 2024-05-08T22:33:38.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.023+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65381438b0 con 0x7f651c066760 2024-05-08T22:33:38.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.039+0000 7f652dffb700 1 -- 172.21.15.179:0/2304116811 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345168 (secure 0 0 0) 0x7f653400ea10 con 0x7f651c066760 2024-05-08T22:33:38.045 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 msgr2=0x7f651c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 0x7f651c068c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65340090b0 tx=0x7f653401e040 comp rx=0 tx=0).stop 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 msgr2=0x7f65381e2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65280116f0 tx=0x7f6528011a00 comp rx=0 tx=0).stop 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 shutdown_connections 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f651c066760 0x7f651c068c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f653813b100 0x7f65380c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6538159480 0x7f65381e2380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 --2- 172.21.15.179:0/2304116811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6524003a20 0x7f65381f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 >> 172.21.15.179:0/2304116811 conn(0x7f65380ce9d0 msgr2=0x7f653806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 shutdown_connections 2024-05-08T22:33:38.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.047+0000 7f653e3a9700 1 -- 172.21.15.179:0/2304116811 wait complete. 2024-05-08T22:33:38.049 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:33:38.063 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-05-08T22:33:37.382815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235888,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738832,"statfs":{"total":773094113280,"available":772852563968,"internally_reserved":0,"allocated":3416064,"data_stored":2046228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005452"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.932430+0000","last_change":"2024-05-08T22:33:14.533515+0000","last_active":"2024-05-08T22:33:18.932430+0000","last_peered":"2024-05-08T22:33:18.932430+0000","last_clean":"2024-05-08T22:33:18.932430+0000","last_became_active":"2024-05-08T22:33:14.533293+0000","last_became_peered":"2024-05-08T22:33:14.533293+0000","last_unstale":"2024-05-08T22:33:18.932430+0000","last_undegraded":"2024-05-08T22:33:18.932430+0000","last_fullsized":"2024-05-08T22:33:18.932430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312810+0000","last_change":"2024-05-08T22:33:14.919770+0000","last_active":"2024-05-08T22:33:19.312810+0000","last_peered":"2024-05-08T22:33:19.312810+0000","last_clean":"2024-05-08T22:33:19.312810+0000","last_became_active":"2024-05-08T22:33:14.917629+0000","last_became_peered":"2024-05-08T22:33:14.917629+0000","last_unstale":"2024-05-08T22:33:19.312810+0000","last_undegraded":"2024-05-08T22:33:19.312810+0000","last_fullsized":"2024-05-08T22:33:19.312810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311922+0000","last_change":"2024-05-08T22:33:19.007654+0000","last_active":"2024-05-08T22:33:19.311922+0000","last_peered":"2024-05-08T22:33:19.311922+0000","last_clean":"2024-05-08T22:33:19.311922+0000","last_became_active":"2024-05-08T22:33:19.006811+0000","last_became_peered":"2024-05-08T22:33:19.006811+0000","last_unstale":"2024-05-08T22:33:19.311922+0000","last_undegraded":"2024-05-08T22:33:19.311922+0000","last_fullsized":"2024-05-08T22:33:19.311922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:28.811139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944044+0000","last_change":"2024-05-08T22:33:14.528074+0000","last_active":"2024-05-08T22:33:18.944044+0000","last_peered":"2024-05-08T22:33:18.944044+0000","last_clean":"2024-05-08T22:33:18.944044+0000","last_became_active":"2024-05-08T22:33:14.527773+0000","last_became_peered":"2024-05-08T22:33:14.527773+0000","last_unstale":"2024-05-08T22:33:18.944044+0000","last_undegraded":"2024-05-08T22:33:18.944044+0000","last_fullsized":"2024-05-08T22:33:18.944044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:53.046188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931850+0000","last_change":"2024-05-08T22:33:14.932746+0000","last_active":"2024-05-08T22:33:18.931850+0000","last_peered":"2024-05-08T22:33:18.931850+0000","last_clean":"2024-05-08T22:33:18.931850+0000","last_became_active":"2024-05-08T22:33:14.932511+0000","last_became_peered":"2024-05-08T22:33:14.932511+0000","last_unstale":"2024-05-08T22:33:18.931850+0000","last_undegraded":"2024-05-08T22:33:18.931850+0000","last_fullsized":"2024-05-08T22:33:18.931850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:05.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313017+0000","last_change":"2024-05-08T22:33:18.946740+0000","last_active":"2024-05-08T22:33:19.313017+0000","last_peered":"2024-05-08T22:33:19.313017+0000","last_clean":"2024-05-08T22:33:19.313017+0000","last_became_active":"2024-05-08T22:33:18.946369+0000","last_became_peered":"2024-05-08T22:33:18.946369+0000","last_unstale":"2024-05-08T22:33:19.313017+0000","last_undegraded":"2024-05-08T22:33:19.313017+0000","last_fullsized":"2024-05-08T22:33:19.313017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999447+0000","last_change":"2024-05-08T22:33:14.520155+0000","last_active":"2024-05-08T22:33:18.999447+0000","last_peered":"2024-05-08T22:33:18.999447+0000","last_clean":"2024-05-08T22:33:18.999447+0000","last_became_active":"2024-05-08T22:33:14.519845+0000","last_became_peered":"2024-05-08T22:33:14.519845+0000","last_unstale":"2024-05-08T22:33:18.999447+0000","last_undegraded":"2024-05-08T22:33:18.999447+0000","last_fullsized":"2024-05-08T22:33:18.999447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:03.999595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934686+0000","last_change":"2024-05-08T22:33:14.920443+0000","last_active":"2024-05-08T22:33:18.934686+0000","last_peered":"2024-05-08T22:33:18.934686+0000","last_clean":"2024-05-08T22:33:18.934686+0000","last_became_active":"2024-05-08T22:33:14.920034+0000","last_became_peered":"2024-05-08T22:33:14.920034+0000","last_unstale":"2024-05-08T22:33:18.934686+0000","last_undegraded":"2024-05-08T22:33:18.934686+0000","last_fullsized":"2024-05-08T22:33:18.934686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314786+0000","last_change":"2024-05-08T22:33:18.948680+0000","last_active":"2024-05-08T22:33:19.314786+0000","last_peered":"2024-05-08T22:33:19.314786+0000","last_clean":"2024-05-08T22:33:19.314786+0000","last_became_active":"2024-05-08T22:33:18.948252+0000","last_became_peered":"2024-05-08T22:33:18.948252+0000","last_unstale":"2024-05-08T22:33:19.314786+0000","last_undegraded":"2024-05-08T22:33:19.314786+0000","last_fullsized":"2024-05-08T22:33:19.314786+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:20.694843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999183+0000","last_change":"2024-05-08T22:33:14.526338+0000","last_active":"2024-05-08T22:33:18.999183+0000","last_peered":"2024-05-08T22:33:18.999183+0000","last_clean":"2024-05-08T22:33:18.999183+0000","last_became_active":"2024-05-08T22:33:14.520436+0000","last_became_peered":"2024-05-08T22:33:14.520436+0000","last_unstale":"2024-05-08T22:33:18.999183+0000","last_undegraded":"2024-05-08T22:33:18.999183+0000","last_fullsized":"2024-05-08T22:33:18.999183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:15.246381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935315+0000","last_change":"2024-05-08T22:33:14.919811+0000","last_active":"2024-05-08T22:33:18.935315+0000","last_peered":"2024-05-08T22:33:18.935315+0000","last_clean":"2024-05-08T22:33:18.935315+0000","last_became_active":"2024-05-08T22:33:14.919249+0000","last_became_peered":"2024-05-08T22:33:14.919249+0000","last_unstale":"2024-05-08T22:33:18.935315+0000","last_undegraded":"2024-05-08T22:33:18.935315+0000","last_fullsized":"2024-05-08T22:33:18.935315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:34.252236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313998+0000","last_change":"2024-05-08T22:33:18.942435+0000","last_active":"2024-05-08T22:33:19.313998+0000","last_peered":"2024-05-08T22:33:19.313998+0000","last_clean":"2024-05-08T22:33:19.313998+0000","last_became_active":"2024-05-08T22:33:18.942040+0000","last_became_peered":"2024-05-08T22:33:18.942040+0000","last_unstale":"2024-05-08T22:33:19.313998+0000","last_undegraded":"2024-05-08T22:33:19.313998+0000","last_fullsized":"2024-05-08T22:33:19.313998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927785+0000","last_change":"2024-05-08T22:33:14.913632+0000","last_active":"2024-05-08T22:33:18.927785+0000","last_peered":"2024-05-08T22:33:18.927785+0000","last_clean":"2024-05-08T22:33:18.927785+0000","last_became_active":"2024-05-08T22:33:14.913040+0000","last_became_peered":"2024-05-08T22:33:14.913040+0000","last_unstale":"2024-05-08T22:33:18.927785+0000","last_undegraded":"2024-05-08T22:33:18.927785+0000","last_fullsized":"2024-05-08T22:33:18.927785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998991+0000","last_change":"2024-05-08T22:33:14.526397+0000","last_active":"2024-05-08T22:33:18.998991+0000","last_peered":"2024-05-08T22:33:18.998991+0000","last_clean":"2024-05-08T22:33:18.998991+0000","last_became_active":"2024-05-08T22:33:14.519761+0000","last_became_peered":"2024-05-08T22:33:14.519761+0000","last_unstale":"2024-05-08T22:33:18.998991+0000","last_undegraded":"2024-05-08T22:33:18.998991+0000","last_fullsized":"2024-05-08T22:33:18.998991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315399+0000","last_change":"2024-05-08T22:33:18.944230+0000","last_active":"2024-05-08T22:33:19.315399+0000","last_peered":"2024-05-08T22:33:19.315399+0000","last_clean":"2024-05-08T22:33:19.315399+0000","last_became_active":"2024-05-08T22:33:18.943852+0000","last_became_peered":"2024-05-08T22:33:18.943852+0000","last_unstale":"2024-05-08T22:33:19.315399+0000","last_undegraded":"2024-05-08T22:33:19.315399+0000","last_fullsized":"2024-05-08T22:33:19.315399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312132+0000","last_change":"2024-05-08T22:33:14.912752+0000","last_active":"2024-05-08T22:33:19.312132+0000","last_peered":"2024-05-08T22:33:19.312132+0000","last_clean":"2024-05-08T22:33:19.312132+0000","last_became_active":"2024-05-08T22:33:14.912455+0000","last_became_peered":"2024-05-08T22:33:14.912455+0000","last_unstale":"2024-05-08T22:33:19.312132+0000","last_undegraded":"2024-05-08T22:33:19.312132+0000","last_fullsized":"2024-05-08T22:33:19.312132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934257+0000","last_change":"2024-05-08T22:33:14.525254+0000","last_active":"2024-05-08T22:33:18.934257+0000","last_peered":"2024-05-08T22:33:18.934257+0000","last_clean":"2024-05-08T22:33:18.934257+0000","last_became_active":"2024-05-08T22:33:14.524663+0000","last_became_peered":"2024-05-08T22:33:14.524663+0000","last_unstale":"2024-05-08T22:33:18.934257+0000","last_undegraded":"2024-05-08T22:33:18.934257+0000","last_fullsized":"2024-05-08T22:33:18.934257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314812+0000","last_change":"2024-05-08T22:33:18.947596+0000","last_active":"2024-05-08T22:33:19.314812+0000","last_peered":"2024-05-08T22:33:19.314812+0000","last_clean":"2024-05-08T22:33:19.314812+0000","last_became_active":"2024-05-08T22:33:18.947225+0000","last_became_peered":"2024-05-08T22:33:18.947225+0000","last_unstale":"2024-05-08T22:33:19.314812+0000","last_undegraded":"2024-05-08T22:33:19.314812+0000","last_fullsized":"2024-05-08T22:33:19.314812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312379+0000","last_change":"2024-05-08T22:33:14.906568+0000","last_active":"2024-05-08T22:33:19.312379+0000","last_peered":"2024-05-08T22:33:19.312379+0000","last_clean":"2024-05-08T22:33:19.312379+0000","last_became_active":"2024-05-08T22:33:14.906359+0000","last_became_peered":"2024-05-08T22:33:14.906359+0000","last_unstale":"2024-05-08T22:33:19.312379+0000","last_undegraded":"2024-05-08T22:33:19.312379+0000","last_fullsized":"2024-05-08T22:33:19.312379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:47.862614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934403+0000","last_change":"2024-05-08T22:33:14.530133+0000","last_active":"2024-05-08T22:33:18.934403+0000","last_peered":"2024-05-08T22:33:18.934403+0000","last_clean":"2024-05-08T22:33:18.934403+0000","last_became_active":"2024-05-08T22:33:14.529854+0000","last_became_peered":"2024-05-08T22:33:14.529854+0000","last_unstale":"2024-05-08T22:33:18.934403+0000","last_undegraded":"2024-05-08T22:33:18.934403+0000","last_fullsized":"2024-05-08T22:33:18.934403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:59.313380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448755+0000","last_change":"2024-05-08T22:33:19.004993+0000","last_active":"2024-05-08T22:33:19.448755+0000","last_peered":"2024-05-08T22:33:19.448755+0000","last_clean":"2024-05-08T22:33:19.448755+0000","last_became_active":"2024-05-08T22:33:19.004152+0000","last_became_peered":"2024-05-08T22:33:19.004152+0000","last_unstale":"2024-05-08T22:33:19.448755+0000","last_undegraded":"2024-05-08T22:33:19.448755+0000","last_fullsized":"2024-05-08T22:33:19.448755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:39.634020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929034+0000","last_change":"2024-05-08T22:33:14.909377+0000","last_active":"2024-05-08T22:33:18.929034+0000","last_peered":"2024-05-08T22:33:18.929034+0000","last_clean":"2024-05-08T22:33:18.929034+0000","last_became_active":"2024-05-08T22:33:14.909047+0000","last_became_peered":"2024-05-08T22:33:14.909047+0000","last_unstale":"2024-05-08T22:33:18.929034+0000","last_undegraded":"2024-05-08T22:33:18.929034+0000","last_fullsized":"2024-05-08T22:33:18.929034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:32.048397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998562+0000","last_change":"2024-05-08T22:33:14.529685+0000","last_active":"2024-05-08T22:33:18.998562+0000","last_peered":"2024-05-08T22:33:18.998562+0000","last_clean":"2024-05-08T22:33:18.998562+0000","last_became_active":"2024-05-08T22:33:14.529417+0000","last_became_peered":"2024-05-08T22:33:14.529417+0000","last_unstale":"2024-05-08T22:33:18.998562+0000","last_undegraded":"2024-05-08T22:33:18.998562+0000","last_fullsized":"2024-05-08T22:33:18.998562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.786143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314895+0000","last_change":"2024-05-08T22:33:18.948441+0000","last_active":"2024-05-08T22:33:19.314895+0000","last_peered":"2024-05-08T22:33:19.314895+0000","last_clean":"2024-05-08T22:33:19.314895+0000","last_became_active":"2024-05-08T22:33:18.948006+0000","last_became_peered":"2024-05-08T22:33:18.948006+0000","last_unstale":"2024-05-08T22:33:19.314895+0000","last_undegraded":"2024-05-08T22:33:19.314895+0000","last_fullsized":"2024-05-08T22:33:19.314895+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936140+0000","last_change":"2024-05-08T22:33:14.915374+0000","last_active":"2024-05-08T22:33:18.936140+0000","last_peered":"2024-05-08T22:33:18.936140+0000","last_clean":"2024-05-08T22:33:18.936140+0000","last_became_active":"2024-05-08T22:33:14.915035+0000","last_became_peered":"2024-05-08T22:33:14.915035+0000","last_unstale":"2024-05-08T22:33:18.936140+0000","last_undegraded":"2024-05-08T22:33:18.936140+0000","last_fullsized":"2024-05-08T22:33:18.936140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927757+0000","last_change":"2024-05-08T22:33:14.518968+0000","last_active":"2024-05-08T22:33:18.927757+0000","last_peered":"2024-05-08T22:33:18.927757+0000","last_clean":"2024-05-08T22:33:18.927757+0000","last_became_active":"2024-05-08T22:33:14.518318+0000","last_became_peered":"2024-05-08T22:33:14.518318+0000","last_unstale":"2024-05-08T22:33:18.927757+0000","last_undegraded":"2024-05-08T22:33:18.927757+0000","last_fullsized":"2024-05-08T22:33:18.927757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313504+0000","last_change":"2024-05-08T22:33:19.003063+0000","last_active":"2024-05-08T22:33:19.313504+0000","last_peered":"2024-05-08T22:33:19.313504+0000","last_clean":"2024-05-08T22:33:19.313504+0000","last_became_active":"2024-05-08T22:33:19.002557+0000","last_became_peered":"2024-05-08T22:33:19.002557+0000","last_unstale":"2024-05-08T22:33:19.313504+0000","last_undegraded":"2024-05-08T22:33:19.313504+0000","last_fullsized":"2024-05-08T22:33:19.313504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312756+0000","last_change":"2024-05-08T22:33:14.915824+0000","last_active":"2024-05-08T22:33:19.312756+0000","last_peered":"2024-05-08T22:33:19.312756+0000","last_clean":"2024-05-08T22:33:19.312756+0000","last_became_active":"2024-05-08T22:33:14.915554+0000","last_became_peered":"2024-05-08T22:33:14.915554+0000","last_unstale":"2024-05-08T22:33:19.312756+0000","last_undegraded":"2024-05-08T22:33:19.312756+0000","last_fullsized":"2024-05-08T22:33:19.312756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:10.398283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.937194+0000","last_change":"2024-05-08T22:33:14.526863+0000","last_active":"2024-05-08T22:33:18.937194+0000","last_peered":"2024-05-08T22:33:18.937194+0000","last_clean":"2024-05-08T22:33:18.937194+0000","last_became_active":"2024-05-08T22:33:14.526463+0000","last_became_peered":"2024-05-08T22:33:14.526463+0000","last_unstale":"2024-05-08T22:33:18.937194+0000","last_undegraded":"2024-05-08T22:33:18.937194+0000","last_fullsized":"2024-05-08T22:33:18.937194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311963+0000","last_change":"2024-05-08T22:33:19.005291+0000","last_active":"2024-05-08T22:33:19.311963+0000","last_peered":"2024-05-08T22:33:19.311963+0000","last_clean":"2024-05-08T22:33:19.311963+0000","last_became_active":"2024-05-08T22:33:19.004777+0000","last_became_peered":"2024-05-08T22:33:19.004777+0000","last_unstale":"2024-05-08T22:33:19.311963+0000","last_undegraded":"2024-05-08T22:33:19.311963+0000","last_fullsized":"2024-05-08T22:33:19.311963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:36.904973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448630+0000","last_change":"2024-05-08T22:33:14.911246+0000","last_active":"2024-05-08T22:33:19.448630+0000","last_peered":"2024-05-08T22:33:19.448630+0000","last_clean":"2024-05-08T22:33:19.448630+0000","last_became_active":"2024-05-08T22:33:14.910998+0000","last_became_peered":"2024-05-08T22:33:14.910998+0000","last_unstale":"2024-05-08T22:33:19.448630+0000","last_undegraded":"2024-05-08T22:33:19.448630+0000","last_fullsized":"2024-05-08T22:33:19.448630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999587+0000","last_change":"2024-05-08T22:33:14.529748+0000","last_active":"2024-05-08T22:33:18.999587+0000","last_peered":"2024-05-08T22:33:18.999587+0000","last_clean":"2024-05-08T22:33:18.999587+0000","last_became_active":"2024-05-08T22:33:14.529390+0000","last_became_peered":"2024-05-08T22:33:14.529390+0000","last_unstale":"2024-05-08T22:33:18.999587+0000","last_undegraded":"2024-05-08T22:33:18.999587+0000","last_fullsized":"2024-05-08T22:33:18.999587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:04.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.009380+0000","last_change":"2024-05-08T22:33:19.004064+0000","last_active":"2024-05-08T22:33:19.009380+0000","last_peered":"2024-05-08T22:33:19.009380+0000","last_clean":"2024-05-08T22:33:19.009380+0000","last_became_active":"2024-05-08T22:33:19.003638+0000","last_became_peered":"2024-05-08T22:33:19.003638+0000","last_unstale":"2024-05-08T22:33:19.009380+0000","last_undegraded":"2024-05-08T22:33:19.009380+0000","last_fullsized":"2024-05-08T22:33:19.009380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935398+0000","last_change":"2024-05-08T22:33:14.928058+0000","last_active":"2024-05-08T22:33:18.935398+0000","last_peered":"2024-05-08T22:33:18.935398+0000","last_clean":"2024-05-08T22:33:18.935398+0000","last_became_active":"2024-05-08T22:33:14.927408+0000","last_became_peered":"2024-05-08T22:33:14.927408+0000","last_unstale":"2024-05-08T22:33:18.935398+0000","last_undegraded":"2024-05-08T22:33:18.935398+0000","last_fullsized":"2024-05-08T22:33:18.935398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:22.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934334+0000","last_change":"2024-05-08T22:33:14.530667+0000","last_active":"2024-05-08T22:33:18.934334+0000","last_peered":"2024-05-08T22:33:18.934334+0000","last_clean":"2024-05-08T22:33:18.934334+0000","last_became_active":"2024-05-08T22:33:14.530429+0000","last_became_peered":"2024-05-08T22:33:14.530429+0000","last_unstale":"2024-05-08T22:33:18.934334+0000","last_undegraded":"2024-05-08T22:33:18.934334+0000","last_fullsized":"2024-05-08T22:33:18.934334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:38.212588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448536+0000","last_change":"2024-05-08T22:33:18.945810+0000","last_active":"2024-05-08T22:33:19.448536+0000","last_peered":"2024-05-08T22:33:19.448536+0000","last_clean":"2024-05-08T22:33:19.448536+0000","last_became_active":"2024-05-08T22:33:18.945204+0000","last_became_peered":"2024-05-08T22:33:18.945204+0000","last_unstale":"2024-05-08T22:33:19.448536+0000","last_undegraded":"2024-05-08T22:33:19.448536+0000","last_fullsized":"2024-05-08T22:33:19.448536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:26.949422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930747+0000","last_change":"2024-05-08T22:33:14.911989+0000","last_active":"2024-05-08T22:33:18.930747+0000","last_peered":"2024-05-08T22:33:18.930747+0000","last_clean":"2024-05-08T22:33:18.930747+0000","last_became_active":"2024-05-08T22:33:14.911685+0000","last_became_peered":"2024-05-08T22:33:14.911685+0000","last_unstale":"2024-05-08T22:33:18.930747+0000","last_undegraded":"2024-05-08T22:33:18.930747+0000","last_fullsized":"2024-05-08T22:33:18.930747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934973+0000","last_change":"2024-05-08T22:33:14.524541+0000","last_active":"2024-05-08T22:33:18.934973+0000","last_peered":"2024-05-08T22:33:18.934973+0000","last_clean":"2024-05-08T22:33:18.934973+0000","last_became_active":"2024-05-08T22:33:14.524290+0000","last_became_peered":"2024-05-08T22:33:14.524290+0000","last_unstale":"2024-05-08T22:33:18.934973+0000","last_undegraded":"2024-05-08T22:33:18.934973+0000","last_fullsized":"2024-05-08T22:33:18.934973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448609+0000","last_change":"2024-05-08T22:33:18.951903+0000","last_active":"2024-05-08T22:33:19.448609+0000","last_peered":"2024-05-08T22:33:19.448609+0000","last_clean":"2024-05-08T22:33:19.448609+0000","last_became_active":"2024-05-08T22:33:18.951473+0000","last_became_peered":"2024-05-08T22:33:18.951473+0000","last_unstale":"2024-05-08T22:33:19.448609+0000","last_undegraded":"2024-05-08T22:33:19.448609+0000","last_fullsized":"2024-05-08T22:33:19.448609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:19.057699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448450+0000","last_change":"2024-05-08T22:33:14.911724+0000","last_active":"2024-05-08T22:33:19.448450+0000","last_peered":"2024-05-08T22:33:19.448450+0000","last_clean":"2024-05-08T22:33:19.448450+0000","last_became_active":"2024-05-08T22:33:14.911493+0000","last_became_peered":"2024-05-08T22:33:14.911493+0000","last_unstale":"2024-05-08T22:33:19.448450+0000","last_undegraded":"2024-05-08T22:33:19.448450+0000","last_fullsized":"2024-05-08T22:33:19.448450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925211+0000","last_change":"2024-05-08T22:33:14.524989+0000","last_active":"2024-05-08T22:33:18.925211+0000","last_peered":"2024-05-08T22:33:18.925211+0000","last_clean":"2024-05-08T22:33:18.925211+0000","last_became_active":"2024-05-08T22:33:14.524715+0000","last_became_peered":"2024-05-08T22:33:14.524715+0000","last_unstale":"2024-05-08T22:33:18.925211+0000","last_undegraded":"2024-05-08T22:33:18.925211+0000","last_fullsized":"2024-05-08T22:33:18.925211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448775+0000","last_change":"2024-05-08T22:33:18.952088+0000","last_active":"2024-05-08T22:33:19.448775+0000","last_peered":"2024-05-08T22:33:19.448775+0000","last_clean":"2024-05-08T22:33:19.448775+0000","last_became_active":"2024-05-08T22:33:18.951660+0000","last_became_peered":"2024-05-08T22:33:18.951660+0000","last_unstale":"2024-05-08T22:33:19.448775+0000","last_undegraded":"2024-05-08T22:33:19.448775+0000","last_fullsized":"2024-05-08T22:33:19.448775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:42.700357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930121+0000","last_change":"2024-05-08T22:33:14.911625+0000","last_active":"2024-05-08T22:33:18.930121+0000","last_peered":"2024-05-08T22:33:18.930121+0000","last_clean":"2024-05-08T22:33:18.930121+0000","last_became_active":"2024-05-08T22:33:14.911368+0000","last_became_peered":"2024-05-08T22:33:14.911368+0000","last_unstale":"2024-05-08T22:33:18.930121+0000","last_undegraded":"2024-05-08T22:33:18.930121+0000","last_fullsized":"2024-05-08T22:33:18.930121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998588+0000","last_change":"2024-05-08T22:33:14.529960+0000","last_active":"2024-05-08T22:33:18.998588+0000","last_peered":"2024-05-08T22:33:18.998588+0000","last_clean":"2024-05-08T22:33:18.998588+0000","last_became_active":"2024-05-08T22:33:14.529479+0000","last_became_peered":"2024-05-08T22:33:14.529479+0000","last_unstale":"2024-05-08T22:33:18.998588+0000","last_undegraded":"2024-05-08T22:33:18.998588+0000","last_fullsized":"2024-05-08T22:33:18.998588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:49.272820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.010817+0000","last_change":"2024-05-08T22:33:19.005638+0000","last_active":"2024-05-08T22:33:19.010817+0000","last_peered":"2024-05-08T22:33:19.010817+0000","last_clean":"2024-05-08T22:33:19.010817+0000","last_became_active":"2024-05-08T22:33:19.003876+0000","last_became_peered":"2024-05-08T22:33:19.003876+0000","last_unstale":"2024-05-08T22:33:19.010817+0000","last_undegraded":"2024-05-08T22:33:19.010817+0000","last_fullsized":"2024-05-08T22:33:19.010817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312713+0000","last_change":"2024-05-08T22:33:14.926812+0000","last_active":"2024-05-08T22:33:19.312713+0000","last_peered":"2024-05-08T22:33:19.312713+0000","last_clean":"2024-05-08T22:33:19.312713+0000","last_became_active":"2024-05-08T22:33:14.926536+0000","last_became_peered":"2024-05-08T22:33:14.926536+0000","last_unstale":"2024-05-08T22:33:19.312713+0000","last_undegraded":"2024-05-08T22:33:19.312713+0000","last_fullsized":"2024-05-08T22:33:19.312713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:18.111855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934524+0000","last_change":"2024-05-08T22:33:14.523530+0000","last_active":"2024-05-08T22:33:18.934524+0000","last_peered":"2024-05-08T22:33:18.934524+0000","last_clean":"2024-05-08T22:33:18.934524+0000","last_became_active":"2024-05-08T22:33:14.523193+0000","last_became_peered":"2024-05-08T22:33:14.523193+0000","last_unstale":"2024-05-08T22:33:18.934524+0000","last_undegraded":"2024-05-08T22:33:18.934524+0000","last_fullsized":"2024-05-08T22:33:18.934524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998772+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:33:18.998772+0000","last_peered":"2024-05-08T22:33:18.998772+0000","last_clean":"2024-05-08T22:33:18.998772+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:33:18.998772+0000","last_undegraded":"2024-05-08T22:33:18.998772+0000","last_fullsized":"2024-05-08T22:33:18.998772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007908+0000","last_change":"2024-05-08T22:33:19.006286+0000","last_active":"2024-05-08T22:33:19.007908+0000","last_peered":"2024-05-08T22:33:19.007908+0000","last_clean":"2024-05-08T22:33:19.007908+0000","last_became_active":"2024-05-08T22:33:19.005632+0000","last_became_peered":"2024-05-08T22:33:19.005632+0000","last_unstale":"2024-05-08T22:33:19.007908+0000","last_undegraded":"2024-05-08T22:33:19.007908+0000","last_fullsized":"2024-05-08T22:33:19.007908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:39.728431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999358+0000","last_change":"2024-05-08T22:33:14.931567+0000","last_active":"2024-05-08T22:33:18.999358+0000","last_peered":"2024-05-08T22:33:18.999358+0000","last_clean":"2024-05-08T22:33:18.999358+0000","last_became_active":"2024-05-08T22:33:14.931285+0000","last_became_peered":"2024-05-08T22:33:14.931285+0000","last_unstale":"2024-05-08T22:33:18.999358+0000","last_undegraded":"2024-05-08T22:33:18.999358+0000","last_fullsized":"2024-05-08T22:33:18.999358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925484+0000","last_change":"2024-05-08T22:33:14.523189+0000","last_active":"2024-05-08T22:33:18.925484+0000","last_peered":"2024-05-08T22:33:18.925484+0000","last_clean":"2024-05-08T22:33:18.925484+0000","last_became_active":"2024-05-08T22:33:14.522942+0000","last_became_peered":"2024-05-08T22:33:14.522942+0000","last_unstale":"2024-05-08T22:33:18.925484+0000","last_undegraded":"2024-05-08T22:33:18.925484+0000","last_fullsized":"2024-05-08T22:33:18.925484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007291+0000","last_change":"2024-05-08T22:33:19.003092+0000","last_active":"2024-05-08T22:33:19.007291+0000","last_peered":"2024-05-08T22:33:19.007291+0000","last_clean":"2024-05-08T22:33:19.007291+0000","last_became_active":"2024-05-08T22:33:19.002313+0000","last_became_peered":"2024-05-08T22:33:19.002313+0000","last_unstale":"2024-05-08T22:33:19.007291+0000","last_undegraded":"2024-05-08T22:33:19.007291+0000","last_fullsized":"2024-05-08T22:33:19.007291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:17.965361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934771+0000","last_change":"2024-05-08T22:33:14.927660+0000","last_active":"2024-05-08T22:33:18.934771+0000","last_peered":"2024-05-08T22:33:18.934771+0000","last_clean":"2024-05-08T22:33:18.934771+0000","last_became_active":"2024-05-08T22:33:14.925184+0000","last_became_peered":"2024-05-08T22:33:14.925184+0000","last_unstale":"2024-05-08T22:33:18.934771+0000","last_undegraded":"2024-05-08T22:33:18.934771+0000","last_fullsized":"2024-05-08T22:33:18.934771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:55.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934291+0000","last_change":"2024-05-08T22:33:14.524892+0000","last_active":"2024-05-08T22:33:18.934291+0000","last_peered":"2024-05-08T22:33:18.934291+0000","last_clean":"2024-05-08T22:33:18.934291+0000","last_became_active":"2024-05-08T22:33:14.524584+0000","last_became_peered":"2024-05-08T22:33:14.524584+0000","last_unstale":"2024-05-08T22:33:18.934291+0000","last_undegraded":"2024-05-08T22:33:18.934291+0000","last_fullsized":"2024-05-08T22:33:18.934291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953385+0000","last_change":"2024-05-08T22:33:18.947268+0000","last_active":"2024-05-08T22:33:18.953385+0000","last_peered":"2024-05-08T22:33:18.953385+0000","last_clean":"2024-05-08T22:33:18.953385+0000","last_became_active":"2024-05-08T22:33:18.946822+0000","last_became_peered":"2024-05-08T22:33:18.946822+0000","last_unstale":"2024-05-08T22:33:18.953385+0000","last_undegraded":"2024-05-08T22:33:18.953385+0000","last_fullsized":"2024-05-08T22:33:18.953385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999808+0000","last_change":"2024-05-08T22:33:14.925288+0000","last_active":"2024-05-08T22:33:18.999808+0000","last_peered":"2024-05-08T22:33:18.999808+0000","last_clean":"2024-05-08T22:33:18.999808+0000","last_became_active":"2024-05-08T22:33:14.924982+0000","last_became_peered":"2024-05-08T22:33:14.924982+0000","last_unstale":"2024-05-08T22:33:18.999808+0000","last_undegraded":"2024-05-08T22:33:18.999808+0000","last_fullsized":"2024-05-08T22:33:18.999808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925406+0000","last_change":"2024-05-08T22:33:14.531914+0000","last_active":"2024-05-08T22:33:18.925406+0000","last_peered":"2024-05-08T22:33:18.925406+0000","last_clean":"2024-05-08T22:33:18.925406+0000","last_became_active":"2024-05-08T22:33:14.531550+0000","last_became_peered":"2024-05-08T22:33:14.531550+0000","last_unstale":"2024-05-08T22:33:18.925406+0000","last_undegraded":"2024-05-08T22:33:18.925406+0000","last_fullsized":"2024-05-08T22:33:18.925406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:12.564410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006729+0000","last_change":"2024-05-08T22:33:19.005176+0000","last_active":"2024-05-08T22:33:19.006729+0000","last_peered":"2024-05-08T22:33:19.006729+0000","last_clean":"2024-05-08T22:33:19.006729+0000","last_became_active":"2024-05-08T22:33:19.004757+0000","last_became_peered":"2024-05-08T22:33:19.004757+0000","last_unstale":"2024-05-08T22:33:19.006729+0000","last_undegraded":"2024-05-08T22:33:19.006729+0000","last_fullsized":"2024-05-08T22:33:19.006729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.924451+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.922189+0000","last_became_peered":"2024-05-08T22:33:14.922189+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927288+0000","last_change":"2024-05-08T22:33:14.527475+0000","last_active":"2024-05-08T22:33:18.927288+0000","last_peered":"2024-05-08T22:33:18.927288+0000","last_clean":"2024-05-08T22:33:18.927288+0000","last_became_active":"2024-05-08T22:33:14.526627+0000","last_became_peered":"2024-05-08T22:33:14.526627+0000","last_unstale":"2024-05-08T22:33:18.927288+0000","last_undegraded":"2024-05-08T22:33:18.927288+0000","last_fullsized":"2024-05-08T22:33:18.927288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:00.386313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312868+0000","last_change":"2024-05-08T22:33:18.946891+0000","last_active":"2024-05-08T22:33:19.312868+0000","last_peered":"2024-05-08T22:33:19.312868+0000","last_clean":"2024-05-08T22:33:19.312868+0000","last_became_active":"2024-05-08T22:33:18.946534+0000","last_became_peered":"2024-05-08T22:33:18.946534+0000","last_unstale":"2024-05-08T22:33:19.312868+0000","last_undegraded":"2024-05-08T22:33:19.312868+0000","last_fullsized":"2024-05-08T22:33:19.312868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999925+0000","last_change":"2024-05-08T22:33:14.918650+0000","last_active":"2024-05-08T22:33:18.999925+0000","last_peered":"2024-05-08T22:33:18.999925+0000","last_clean":"2024-05-08T22:33:18.999925+0000","last_became_active":"2024-05-08T22:33:14.918389+0000","last_became_peered":"2024-05-08T22:33:14.918389+0000","last_unstale":"2024-05-08T22:33:18.999925+0000","last_undegraded":"2024-05-08T22:33:18.999925+0000","last_fullsized":"2024-05-08T22:33:18.999925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925062+0000","last_change":"2024-05-08T22:33:14.522438+0000","last_active":"2024-05-08T22:33:18.925062+0000","last_peered":"2024-05-08T22:33:18.925062+0000","last_clean":"2024-05-08T22:33:18.925062+0000","last_became_active":"2024-05-08T22:33:14.522094+0000","last_became_peered":"2024-05-08T22:33:14.522094+0000","last_unstale":"2024-05-08T22:33:18.925062+0000","last_undegraded":"2024-05-08T22:33:18.925062+0000","last_fullsized":"2024-05-08T22:33:18.925062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.950611+0000","last_change":"2024-05-08T22:33:18.944271+0000","last_active":"2024-05-08T22:33:18.950611+0000","last_peered":"2024-05-08T22:33:18.950611+0000","last_clean":"2024-05-08T22:33:18.950611+0000","last_became_active":"2024-05-08T22:33:18.943934+0000","last_became_peered":"2024-05-08T22:33:18.943934+0000","last_unstale":"2024-05-08T22:33:18.950611+0000","last_undegraded":"2024-05-08T22:33:18.950611+0000","last_fullsized":"2024-05-08T22:33:18.950611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926385+0000","last_change":"2024-05-08T22:33:14.915516+0000","last_active":"2024-05-08T22:33:18.926385+0000","last_peered":"2024-05-08T22:33:18.926385+0000","last_clean":"2024-05-08T22:33:18.926385+0000","last_became_active":"2024-05-08T22:33:14.915253+0000","last_became_peered":"2024-05-08T22:33:14.915253+0000","last_unstale":"2024-05-08T22:33:18.926385+0000","last_undegraded":"2024-05-08T22:33:18.926385+0000","last_fullsized":"2024-05-08T22:33:18.926385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925993+0000","last_change":"2024-05-08T22:33:14.517759+0000","last_active":"2024-05-08T22:33:18.925993+0000","last_peered":"2024-05-08T22:33:18.925993+0000","last_clean":"2024-05-08T22:33:18.925993+0000","last_became_active":"2024-05-08T22:33:14.517109+0000","last_became_peered":"2024-05-08T22:33:14.517109+0000","last_unstale":"2024-05-08T22:33:18.925993+0000","last_undegraded":"2024-05-08T22:33:18.925993+0000","last_fullsized":"2024-05-08T22:33:18.925993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:13.973941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313006+0000","last_change":"2024-05-08T22:33:19.005763+0000","last_active":"2024-05-08T22:33:19.313006+0000","last_peered":"2024-05-08T22:33:19.313006+0000","last_clean":"2024-05-08T22:33:19.313006+0000","last_became_active":"2024-05-08T22:33:19.004827+0000","last_became_peered":"2024-05-08T22:33:19.004827+0000","last_unstale":"2024-05-08T22:33:19.313006+0000","last_undegraded":"2024-05-08T22:33:19.313006+0000","last_fullsized":"2024-05-08T22:33:19.313006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:24.712864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998964+0000","last_change":"2024-05-08T22:33:14.918080+0000","last_active":"2024-05-08T22:33:18.998964+0000","last_peered":"2024-05-08T22:33:18.998964+0000","last_clean":"2024-05-08T22:33:18.998964+0000","last_became_active":"2024-05-08T22:33:14.917791+0000","last_became_peered":"2024-05-08T22:33:14.917791+0000","last_unstale":"2024-05-08T22:33:18.998964+0000","last_undegraded":"2024-05-08T22:33:18.998964+0000","last_fullsized":"2024-05-08T22:33:18.998964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934850+0000","last_change":"2024-05-08T22:33:14.527314+0000","last_active":"2024-05-08T22:33:18.934850+0000","last_peered":"2024-05-08T22:33:18.934850+0000","last_clean":"2024-05-08T22:33:18.934850+0000","last_became_active":"2024-05-08T22:33:14.526667+0000","last_became_peered":"2024-05-08T22:33:14.526667+0000","last_unstale":"2024-05-08T22:33:18.934850+0000","last_undegraded":"2024-05-08T22:33:18.934850+0000","last_fullsized":"2024-05-08T22:33:18.934850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312923+0000","last_change":"2024-05-08T22:33:19.004038+0000","last_active":"2024-05-08T22:33:19.312923+0000","last_peered":"2024-05-08T22:33:19.312923+0000","last_clean":"2024-05-08T22:33:19.312923+0000","last_became_active":"2024-05-08T22:33:19.002976+0000","last_became_peered":"2024-05-08T22:33:19.002976+0000","last_unstale":"2024-05-08T22:33:19.312923+0000","last_undegraded":"2024-05-08T22:33:19.312923+0000","last_fullsized":"2024-05-08T22:33:19.312923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.506642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930214+0000","last_change":"2024-05-08T22:33:14.924563+0000","last_active":"2024-05-08T22:33:18.930214+0000","last_peered":"2024-05-08T22:33:18.930214+0000","last_clean":"2024-05-08T22:33:18.930214+0000","last_became_active":"2024-05-08T22:33:14.924299+0000","last_became_peered":"2024-05-08T22:33:14.924299+0000","last_unstale":"2024-05-08T22:33:18.930214+0000","last_undegraded":"2024-05-08T22:33:18.930214+0000","last_fullsized":"2024-05-08T22:33:18.930214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931836+0000","last_change":"2024-05-08T22:33:14.531490+0000","last_active":"2024-05-08T22:33:18.931836+0000","last_peered":"2024-05-08T22:33:18.931836+0000","last_clean":"2024-05-08T22:33:18.931836+0000","last_became_active":"2024-05-08T22:33:14.530820+0000","last_became_peered":"2024-05-08T22:33:14.530820+0000","last_unstale":"2024-05-08T22:33:18.931836+0000","last_undegraded":"2024-05-08T22:33:18.931836+0000","last_fullsized":"2024-05-08T22:33:18.931836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:59.224955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006802+0000","last_change":"2024-05-08T22:33:19.005429+0000","last_active":"2024-05-08T22:33:19.006802+0000","last_peered":"2024-05-08T22:33:19.006802+0000","last_clean":"2024-05-08T22:33:19.006802+0000","last_became_active":"2024-05-08T22:33:19.004941+0000","last_became_peered":"2024-05-08T22:33:19.004941+0000","last_unstale":"2024-05-08T22:33:19.006802+0000","last_undegraded":"2024-05-08T22:33:19.006802+0000","last_fullsized":"2024-05-08T22:33:19.006802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999232+0000","last_change":"2024-05-08T22:33:14.919406+0000","last_active":"2024-05-08T22:33:18.999232+0000","last_peered":"2024-05-08T22:33:18.999232+0000","last_clean":"2024-05-08T22:33:18.999232+0000","last_became_active":"2024-05-08T22:33:14.919204+0000","last_became_peered":"2024-05-08T22:33:14.919204+0000","last_unstale":"2024-05-08T22:33:18.999232+0000","last_undegraded":"2024-05-08T22:33:18.999232+0000","last_fullsized":"2024-05-08T22:33:18.999232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:24.279224+0000","last_change":"2024-05-08T22:33:14.523404+0000","last_active":"2024-05-08T22:33:24.279224+0000","last_peered":"2024-05-08T22:33:24.279224+0000","last_clean":"2024-05-08T22:33:24.279224+0000","last_became_active":"2024-05-08T22:33:14.522325+0000","last_became_peered":"2024-05-08T22:33:14.522325+0000","last_unstale":"2024-05-08T22:33:24.279224+0000","last_undegraded":"2024-05-08T22:33:24.279224+0000","last_fullsized":"2024-05-08T22:33:24.279224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448558+0000","last_change":"2024-05-08T22:33:18.950110+0000","last_active":"2024-05-08T22:33:19.448558+0000","last_peered":"2024-05-08T22:33:19.448558+0000","last_clean":"2024-05-08T22:33:19.448558+0000","last_became_active":"2024-05-08T22:33:18.949555+0000","last_became_peered":"2024-05-08T22:33:18.949555+0000","last_unstale":"2024-05-08T22:33:19.448558+0000","last_undegraded":"2024-05-08T22:33:19.448558+0000","last_fullsized":"2024-05-08T22:33:19.448558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:15.389269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926732+0000","last_change":"2024-05-08T22:33:14.908111+0000","last_active":"2024-05-08T22:33:18.926732+0000","last_peered":"2024-05-08T22:33:18.926732+0000","last_clean":"2024-05-08T22:33:18.926732+0000","last_became_active":"2024-05-08T22:33:14.907276+0000","last_became_peered":"2024-05-08T22:33:14.907276+0000","last_unstale":"2024-05-08T22:33:18.926732+0000","last_undegraded":"2024-05-08T22:33:18.926732+0000","last_fullsized":"2024-05-08T22:33:18.926732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:53.154720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934191+0000","last_change":"2024-05-08T22:33:14.521905+0000","last_active":"2024-05-08T22:33:18.934191+0000","last_peered":"2024-05-08T22:33:18.934191+0000","last_clean":"2024-05-08T22:33:18.934191+0000","last_became_active":"2024-05-08T22:33:14.521518+0000","last_became_peered":"2024-05-08T22:33:14.521518+0000","last_unstale":"2024-05-08T22:33:18.934191+0000","last_undegraded":"2024-05-08T22:33:18.934191+0000","last_fullsized":"2024-05-08T22:33:18.934191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315185+0000","last_change":"2024-05-08T22:33:18.939252+0000","last_active":"2024-05-08T22:33:19.315185+0000","last_peered":"2024-05-08T22:33:19.315185+0000","last_clean":"2024-05-08T22:33:19.315185+0000","last_became_active":"2024-05-08T22:33:18.938564+0000","last_became_peered":"2024-05-08T22:33:18.938564+0000","last_unstale":"2024-05-08T22:33:19.315185+0000","last_undegraded":"2024-05-08T22:33:19.315185+0000","last_fullsized":"2024-05-08T22:33:19.315185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:42.775813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931955+0000","last_change":"2024-05-08T22:33:14.932581+0000","last_active":"2024-05-08T22:33:18.931955+0000","last_peered":"2024-05-08T22:33:18.931955+0000","last_clean":"2024-05-08T22:33:18.931955+0000","last_became_active":"2024-05-08T22:33:14.932332+0000","last_became_peered":"2024-05-08T22:33:14.932332+0000","last_unstale":"2024-05-08T22:33:18.931955+0000","last_undegraded":"2024-05-08T22:33:18.931955+0000","last_fullsized":"2024-05-08T22:33:18.931955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:49:02.412782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935272+0000","last_change":"2024-05-08T22:33:14.525414+0000","last_active":"2024-05-08T22:33:18.935272+0000","last_peered":"2024-05-08T22:33:18.935272+0000","last_clean":"2024-05-08T22:33:18.935272+0000","last_became_active":"2024-05-08T22:33:14.524869+0000","last_became_peered":"2024-05-08T22:33:14.524869+0000","last_unstale":"2024-05-08T22:33:18.935272+0000","last_undegraded":"2024-05-08T22:33:18.935272+0000","last_fullsized":"2024-05-08T22:33:18.935272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312533+0000","last_change":"2024-05-08T22:33:18.948135+0000","last_active":"2024-05-08T22:33:19.312533+0000","last_peered":"2024-05-08T22:33:19.312533+0000","last_clean":"2024-05-08T22:33:19.312533+0000","last_became_active":"2024-05-08T22:33:18.947623+0000","last_became_peered":"2024-05-08T22:33:18.947623+0000","last_unstale":"2024-05-08T22:33:19.312533+0000","last_undegraded":"2024-05-08T22:33:19.312533+0000","last_fullsized":"2024-05-08T22:33:19.312533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:55.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935185+0000","last_change":"2024-05-08T22:33:14.925264+0000","last_active":"2024-05-08T22:33:18.935185+0000","last_peered":"2024-05-08T22:33:18.935185+0000","last_clean":"2024-05-08T22:33:18.935185+0000","last_became_active":"2024-05-08T22:33:14.924936+0000","last_became_peered":"2024-05-08T22:33:14.924936+0000","last_unstale":"2024-05-08T22:33:18.935185+0000","last_undegraded":"2024-05-08T22:33:18.935185+0000","last_fullsized":"2024-05-08T22:33:18.935185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927536+0000","last_change":"2024-05-08T22:33:14.519535+0000","last_active":"2024-05-08T22:33:18.927536+0000","last_peered":"2024-05-08T22:33:18.927536+0000","last_clean":"2024-05-08T22:33:18.927536+0000","last_became_active":"2024-05-08T22:33:14.519264+0000","last_became_peered":"2024-05-08T22:33:14.519264+0000","last_unstale":"2024-05-08T22:33:18.927536+0000","last_undegraded":"2024-05-08T22:33:18.927536+0000","last_fullsized":"2024-05-08T22:33:18.927536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:19.007144+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:19.006007+0000","last_became_peered":"2024-05-08T22:33:19.006007+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:50.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.011031+0000","last_change":"2024-05-08T22:33:19.006206+0000","last_active":"2024-05-08T22:33:19.011031+0000","last_peered":"2024-05-08T22:33:19.011031+0000","last_clean":"2024-05-08T22:33:19.011031+0000","last_became_active":"2024-05-08T22:33:19.004897+0000","last_became_peered":"2024-05-08T22:33:19.004897+0000","last_unstale":"2024-05-08T22:33:19.011031+0000","last_undegraded":"2024-05-08T22:33:19.011031+0000","last_fullsized":"2024-05-08T22:33:19.011031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448478+0000","last_change":"2024-05-08T22:33:14.906510+0000","last_active":"2024-05-08T22:33:19.448478+0000","last_peered":"2024-05-08T22:33:19.448478+0000","last_clean":"2024-05-08T22:33:19.448478+0000","last_became_active":"2024-05-08T22:33:14.906227+0000","last_became_peered":"2024-05-08T22:33:14.906227+0000","last_unstale":"2024-05-08T22:33:19.448478+0000","last_undegraded":"2024-05-08T22:33:19.448478+0000","last_fullsized":"2024-05-08T22:33:19.448478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935784+0000","last_change":"2024-05-08T22:33:14.523994+0000","last_active":"2024-05-08T22:33:18.935784+0000","last_peered":"2024-05-08T22:33:18.935784+0000","last_clean":"2024-05-08T22:33:18.935784+0000","last_became_active":"2024-05-08T22:33:14.523710+0000","last_became_peered":"2024-05-08T22:33:14.523710+0000","last_unstale":"2024-05-08T22:33:18.935784+0000","last_undegraded":"2024-05-08T22:33:18.935784+0000","last_fullsized":"2024-05-08T22:33:18.935784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:56.994588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944980+0000","last_change":"2024-05-08T22:33:18.944209+0000","last_active":"2024-05-08T22:33:18.944980+0000","last_peered":"2024-05-08T22:33:18.944980+0000","last_clean":"2024-05-08T22:33:18.944980+0000","last_became_active":"2024-05-08T22:33:18.943824+0000","last_became_peered":"2024-05-08T22:33:18.943824+0000","last_unstale":"2024-05-08T22:33:18.944980+0000","last_undegraded":"2024-05-08T22:33:18.944980+0000","last_fullsized":"2024-05-08T22:33:18.944980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:04.454019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312284+0000","last_change":"2024-05-08T22:33:14.912166+0000","last_active":"2024-05-08T22:33:19.312284+0000","last_peered":"2024-05-08T22:33:19.312284+0000","last_clean":"2024-05-08T22:33:19.312284+0000","last_became_active":"2024-05-08T22:33:14.911941+0000","last_became_peered":"2024-05-08T22:33:14.911941+0000","last_unstale":"2024-05-08T22:33:19.312284+0000","last_undegraded":"2024-05-08T22:33:19.312284+0000","last_fullsized":"2024-05-08T22:33:19.312284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943398+0000","last_change":"2024-05-08T22:33:14.524713+0000","last_active":"2024-05-08T22:33:18.943398+0000","last_peered":"2024-05-08T22:33:18.943398+0000","last_clean":"2024-05-08T22:33:18.943398+0000","last_became_active":"2024-05-08T22:33:14.523688+0000","last_became_peered":"2024-05-08T22:33:14.523688+0000","last_unstale":"2024-05-08T22:33:18.943398+0000","last_undegraded":"2024-05-08T22:33:18.943398+0000","last_fullsized":"2024-05-08T22:33:18.943398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:12.770341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928004+0000","last_change":"2024-05-08T22:33:14.518843+0000","last_active":"2024-05-08T22:33:18.928004+0000","last_peered":"2024-05-08T22:33:18.928004+0000","last_clean":"2024-05-08T22:33:18.928004+0000","last_became_active":"2024-05-08T22:33:14.518002+0000","last_became_peered":"2024-05-08T22:33:14.518002+0000","last_unstale":"2024-05-08T22:33:18.928004+0000","last_undegraded":"2024-05-08T22:33:18.928004+0000","last_fullsized":"2024-05-08T22:33:18.928004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931587+0000","last_change":"2024-05-08T22:33:14.914102+0000","last_active":"2024-05-08T22:33:18.931587+0000","last_peered":"2024-05-08T22:33:18.931587+0000","last_clean":"2024-05-08T22:33:18.931587+0000","last_became_active":"2024-05-08T22:33:14.913777+0000","last_became_peered":"2024-05-08T22:33:14.913777+0000","last_unstale":"2024-05-08T22:33:18.931587+0000","last_undegraded":"2024-05-08T22:33:18.931587+0000","last_fullsized":"2024-05-08T22:33:18.931587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953813+0000","last_change":"2024-05-08T22:33:18.949165+0000","last_active":"2024-05-08T22:33:18.953813+0000","last_peered":"2024-05-08T22:33:18.953813+0000","last_clean":"2024-05-08T22:33:18.953813+0000","last_became_active":"2024-05-08T22:33:18.948660+0000","last_became_peered":"2024-05-08T22:33:18.948660+0000","last_unstale":"2024-05-08T22:33:18.953813+0000","last_undegraded":"2024-05-08T22:33:18.953813+0000","last_fullsized":"2024-05-08T22:33:18.953813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:44.316596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943131+0000","last_change":"2024-05-08T22:33:14.532226+0000","last_active":"2024-05-08T22:33:18.943131+0000","last_peered":"2024-05-08T22:33:18.943131+0000","last_clean":"2024-05-08T22:33:18.943131+0000","last_became_active":"2024-05-08T22:33:14.531943+0000","last_became_peered":"2024-05-08T22:33:14.531943+0000","last_unstale":"2024-05-08T22:33:18.943131+0000","last_undegraded":"2024-05-08T22:33:18.943131+0000","last_fullsized":"2024-05-08T22:33:18.943131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:51.052408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928212+0000","last_change":"2024-05-08T22:33:14.907843+0000","last_active":"2024-05-08T22:33:18.928212+0000","last_peered":"2024-05-08T22:33:18.928212+0000","last_clean":"2024-05-08T22:33:18.928212+0000","last_became_active":"2024-05-08T22:33:14.907511+0000","last_became_peered":"2024-05-08T22:33:14.907511+0000","last_unstale":"2024-05-08T22:33:18.928212+0000","last_undegraded":"2024-05-08T22:33:18.928212+0000","last_fullsized":"2024-05-08T22:33:18.928212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:38.769965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312725+0000","last_change":"2024-05-08T22:33:18.952988+0000","last_active":"2024-05-08T22:33:19.312725+0000","last_peered":"2024-05-08T22:33:19.312725+0000","last_clean":"2024-05-08T22:33:19.312725+0000","last_became_active":"2024-05-08T22:33:18.948797+0000","last_became_peered":"2024-05-08T22:33:18.948797+0000","last_unstale":"2024-05-08T22:33:19.312725+0000","last_undegraded":"2024-05-08T22:33:19.312725+0000","last_fullsized":"2024-05-08T22:33:19.312725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:10.379649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926181+0000","last_change":"2024-05-08T22:33:14.915829+0000","last_active":"2024-05-08T22:33:18.926181+0000","last_peered":"2024-05-08T22:33:18.926181+0000","last_clean":"2024-05-08T22:33:18.926181+0000","last_became_active":"2024-05-08T22:33:14.915496+0000","last_became_peered":"2024-05-08T22:33:14.915496+0000","last_unstale":"2024-05-08T22:33:18.926181+0000","last_undegraded":"2024-05-08T22:33:18.926181+0000","last_fullsized":"2024-05-08T22:33:18.926181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925052+0000","last_change":"2024-05-08T22:33:14.930927+0000","last_active":"2024-05-08T22:33:18.925052+0000","last_peered":"2024-05-08T22:33:18.925052+0000","last_clean":"2024-05-08T22:33:18.925052+0000","last_became_active":"2024-05-08T22:33:14.930569+0000","last_became_peered":"2024-05-08T22:33:14.930569+0000","last_unstale":"2024-05-08T22:33:18.925052+0000","last_undegraded":"2024-05-08T22:33:18.925052+0000","last_fullsized":"2024-05-08T22:33:18.925052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934767+0000","last_change":"2024-05-08T22:33:14.928385+0000","last_active":"2024-05-08T22:33:18.934767+0000","last_peered":"2024-05-08T22:33:18.934767+0000","last_clean":"2024-05-08T22:33:18.934767+0000","last_became_active":"2024-05-08T22:33:14.928162+0000","last_became_peered":"2024-05-08T22:33:14.928162+0000","last_unstale":"2024-05-08T22:33:18.934767+0000","last_undegraded":"2024-05-08T22:33:18.934767+0000","last_fullsized":"2024-05-08T22:33:18.934767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:11.717668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931430+0000","last_change":"2024-05-08T22:33:14.916147+0000","last_active":"2024-05-08T22:33:18.931430+0000","last_peered":"2024-05-08T22:33:18.931430+0000","last_clean":"2024-05-08T22:33:18.931430+0000","last_became_active":"2024-05-08T22:33:14.915762+0000","last_became_peered":"2024-05-08T22:33:14.915762+0000","last_unstale":"2024-05-08T22:33:18.931430+0000","last_undegraded":"2024-05-08T22:33:18.931430+0000","last_fullsized":"2024-05-08T22:33:18.931430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:28.668944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313453+0000","last_change":"2024-05-08T22:33:14.912915+0000","last_active":"2024-05-08T22:33:19.313453+0000","last_peered":"2024-05-08T22:33:19.313453+0000","last_clean":"2024-05-08T22:33:19.313453+0000","last_became_active":"2024-05-08T22:33:14.912698+0000","last_became_peered":"2024-05-08T22:33:14.912698+0000","last_unstale":"2024-05-08T22:33:19.313453+0000","last_undegraded":"2024-05-08T22:33:19.313453+0000","last_fullsized":"2024-05-08T22:33:19.313453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448664+0000","last_change":"2024-05-08T22:33:14.907107+0000","last_active":"2024-05-08T22:33:19.448664+0000","last_peered":"2024-05-08T22:33:19.448664+0000","last_clean":"2024-05-08T22:33:19.448664+0000","last_became_active":"2024-05-08T22:33:14.906444+0000","last_became_peered":"2024-05-08T22:33:14.906444+0000","last_unstale":"2024-05-08T22:33:19.448664+0000","last_undegraded":"2024-05-08T22:33:19.448664+0000","last_fullsized":"2024-05-08T22:33:19.448664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929966+0000","last_change":"2024-05-08T22:33:14.913584+0000","last_active":"2024-05-08T22:33:18.929966+0000","last_peered":"2024-05-08T22:33:18.929966+0000","last_clean":"2024-05-08T22:33:18.929966+0000","last_became_active":"2024-05-08T22:33:14.913304+0000","last_became_peered":"2024-05-08T22:33:14.913304+0000","last_unstale":"2024-05-08T22:33:18.929966+0000","last_undegraded":"2024-05-08T22:33:18.929966+0000","last_fullsized":"2024-05-08T22:33:18.929966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312698+0000","last_change":"2024-05-08T22:33:14.905976+0000","last_active":"2024-05-08T22:33:19.312698+0000","last_peered":"2024-05-08T22:33:19.312698+0000","last_clean":"2024-05-08T22:33:19.312698+0000","last_became_active":"2024-05-08T22:33:14.905734+0000","last_became_peered":"2024-05-08T22:33:14.905734+0000","last_unstale":"2024-05-08T22:33:19.312698+0000","last_undegraded":"2024-05-08T22:33:19.312698+0000","last_fullsized":"2024-05-08T22:33:19.312698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935694+0000","last_change":"2024-05-08T22:33:14.927777+0000","last_active":"2024-05-08T22:33:18.935694+0000","last_peered":"2024-05-08T22:33:18.935694+0000","last_clean":"2024-05-08T22:33:18.935694+0000","last_became_active":"2024-05-08T22:33:14.927297+0000","last_became_peered":"2024-05-08T22:33:14.927297+0000","last_unstale":"2024-05-08T22:33:18.935694+0000","last_undegraded":"2024-05-08T22:33:18.935694+0000","last_fullsized":"2024-05-08T22:33:18.935694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:16.838261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312806+0000","last_change":"2024-05-08T22:33:14.911565+0000","last_active":"2024-05-08T22:33:19.312806+0000","last_peered":"2024-05-08T22:33:19.312806+0000","last_clean":"2024-05-08T22:33:19.312806+0000","last_became_active":"2024-05-08T22:33:14.909251+0000","last_became_peered":"2024-05-08T22:33:14.909251+0000","last_unstale":"2024-05-08T22:33:19.312806+0000","last_undegraded":"2024-05-08T22:33:19.312806+0000","last_fullsized":"2024-05-08T22:33:19.312806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934680+0000","last_change":"2024-05-08T22:33:14.925382+0000","last_active":"2024-05-08T22:33:18.934680+0000","last_peered":"2024-05-08T22:33:18.934680+0000","last_clean":"2024-05-08T22:33:18.934680+0000","last_became_active":"2024-05-08T22:33:14.925133+0000","last_became_peered":"2024-05-08T22:33:14.925133+0000","last_unstale":"2024-05-08T22:33:18.934680+0000","last_undegraded":"2024-05-08T22:33:18.934680+0000","last_fullsized":"2024-05-08T22:33:18.934680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931302+0000","last_change":"2024-05-08T22:33:14.911744+0000","last_active":"2024-05-08T22:33:18.931302+0000","last_peered":"2024-05-08T22:33:18.931302+0000","last_clean":"2024-05-08T22:33:18.931302+0000","last_became_active":"2024-05-08T22:33:14.910853+0000","last_became_peered":"2024-05-08T22:33:14.910853+0000","last_unstale":"2024-05-08T22:33:18.931302+0000","last_undegraded":"2024-05-08T22:33:18.931302+0000","last_fullsized":"2024-05-08T22:33:18.931302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928361+0000","last_change":"2024-05-08T22:33:14.914067+0000","last_active":"2024-05-08T22:33:18.928361+0000","last_peered":"2024-05-08T22:33:18.928361+0000","last_clean":"2024-05-08T22:33:18.928361+0000","last_became_active":"2024-05-08T22:33:14.913755+0000","last_became_peered":"2024-05-08T22:33:14.913755+0000","last_unstale":"2024-05-08T22:33:18.928361+0000","last_undegraded":"2024-05-08T22:33:18.928361+0000","last_fullsized":"2024-05-08T22:33:18.928361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999511+0000","last_change":"2024-05-08T22:33:14.922252+0000","last_active":"2024-05-08T22:33:18.999511+0000","last_peered":"2024-05-08T22:33:18.999511+0000","last_clean":"2024-05-08T22:33:18.999511+0000","last_became_active":"2024-05-08T22:33:14.921949+0000","last_became_peered":"2024-05-08T22:33:14.921949+0000","last_unstale":"2024-05-08T22:33:18.999511+0000","last_undegraded":"2024-05-08T22:33:18.999511+0000","last_fullsized":"2024-05-08T22:33:18.999511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:12.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925280+0000","last_change":"2024-05-08T22:33:14.916689+0000","last_active":"2024-05-08T22:33:18.925280+0000","last_peered":"2024-05-08T22:33:18.925280+0000","last_clean":"2024-05-08T22:33:18.925280+0000","last_became_active":"2024-05-08T22:33:14.916200+0000","last_became_peered":"2024-05-08T22:33:14.916200+0000","last_unstale":"2024-05-08T22:33:18.925280+0000","last_undegraded":"2024-05-08T22:33:18.925280+0000","last_fullsized":"2024-05-08T22:33:18.925280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936475+0000","last_change":"2024-05-08T22:33:14.925461+0000","last_active":"2024-05-08T22:33:18.936475+0000","last_peered":"2024-05-08T22:33:18.936475+0000","last_clean":"2024-05-08T22:33:18.936475+0000","last_became_active":"2024-05-08T22:33:14.924786+0000","last_became_peered":"2024-05-08T22:33:14.924786+0000","last_unstale":"2024-05-08T22:33:18.936475+0000","last_undegraded":"2024-05-08T22:33:18.936475+0000","last_fullsized":"2024-05-08T22:33:18.936475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935606+0000","last_change":"2024-05-08T22:33:14.918139+0000","last_active":"2024-05-08T22:33:18.935606+0000","last_peered":"2024-05-08T22:33:18.935606+0000","last_clean":"2024-05-08T22:33:18.935606+0000","last_became_active":"2024-05-08T22:33:14.914720+0000","last_became_peered":"2024-05-08T22:33:14.914720+0000","last_unstale":"2024-05-08T22:33:18.935606+0000","last_undegraded":"2024-05-08T22:33:18.935606+0000","last_fullsized":"2024-05-08T22:33:18.935606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448502+0000","last_change":"2024-05-08T22:33:14.905986+0000","last_active":"2024-05-08T22:33:19.448502+0000","last_peered":"2024-05-08T22:33:19.448502+0000","last_clean":"2024-05-08T22:33:19.448502+0000","last_became_active":"2024-05-08T22:33:14.905613+0000","last_became_peered":"2024-05-08T22:33:14.905613+0000","last_unstale":"2024-05-08T22:33:19.448502+0000","last_undegraded":"2024-05-08T22:33:19.448502+0000","last_fullsized":"2024-05-08T22:33:19.448502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930058+0000","last_change":"2024-05-08T22:33:14.911270+0000","last_active":"2024-05-08T22:33:18.930058+0000","last_peered":"2024-05-08T22:33:18.930058+0000","last_clean":"2024-05-08T22:33:18.930058+0000","last_became_active":"2024-05-08T22:33:14.910966+0000","last_became_peered":"2024-05-08T22:33:14.910966+0000","last_unstale":"2024-05-08T22:33:18.930058+0000","last_undegraded":"2024-05-08T22:33:18.930058+0000","last_fullsized":"2024-05-08T22:33:18.930058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:40.907481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312110+0000","last_change":"2024-05-08T22:33:14.917778+0000","last_active":"2024-05-08T22:33:19.312110+0000","last_peered":"2024-05-08T22:33:19.312110+0000","last_clean":"2024-05-08T22:33:19.312110+0000","last_became_active":"2024-05-08T22:33:14.917531+0000","last_became_peered":"2024-05-08T22:33:14.917531+0000","last_unstale":"2024-05-08T22:33:19.312110+0000","last_undegraded":"2024-05-08T22:33:19.312110+0000","last_fullsized":"2024-05-08T22:33:19.312110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926531+0000","last_change":"2024-05-08T22:33:14.916563+0000","last_active":"2024-05-08T22:33:18.926531+0000","last_peered":"2024-05-08T22:33:18.926531+0000","last_clean":"2024-05-08T22:33:18.926531+0000","last_became_active":"2024-05-08T22:33:14.916271+0000","last_became_peered":"2024-05-08T22:33:14.916271+0000","last_unstale":"2024-05-08T22:33:18.926531+0000","last_undegraded":"2024-05-08T22:33:18.926531+0000","last_fullsized":"2024-05-08T22:33:18.926531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927948+0000","last_change":"2024-05-08T22:33:14.931269+0000","last_active":"2024-05-08T22:33:18.927948+0000","last_peered":"2024-05-08T22:33:18.927948+0000","last_clean":"2024-05-08T22:33:18.927948+0000","last_became_active":"2024-05-08T22:33:14.931026+0000","last_became_peered":"2024-05-08T22:33:14.931026+0000","last_unstale":"2024-05-08T22:33:18.927948+0000","last_undegraded":"2024-05-08T22:33:18.927948+0000","last_fullsized":"2024-05-08T22:33:18.927948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929869+0000","last_change":"2024-05-08T22:33:14.928697+0000","last_active":"2024-05-08T22:33:18.929869+0000","last_peered":"2024-05-08T22:33:18.929869+0000","last_clean":"2024-05-08T22:33:18.929869+0000","last_became_active":"2024-05-08T22:33:14.928400+0000","last_became_peered":"2024-05-08T22:33:14.928400+0000","last_unstale":"2024-05-08T22:33:18.929869+0000","last_undegraded":"2024-05-08T22:33:18.929869+0000","last_fullsized":"2024-05-08T22:33:18.929869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999653+0000","last_change":"2024-05-08T22:33:14.931677+0000","last_active":"2024-05-08T22:33:18.999653+0000","last_peered":"2024-05-08T22:33:18.999653+0000","last_clean":"2024-05-08T22:33:18.999653+0000","last_became_active":"2024-05-08T22:33:14.931437+0000","last_became_peered":"2024-05-08T22:33:14.931437+0000","last_unstale":"2024-05-08T22:33:18.999653+0000","last_undegraded":"2024-05-08T22:33:18.999653+0000","last_fullsized":"2024-05-08T22:33:18.999653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:07.861504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:14.924165+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:14.923910+0000","last_became_peered":"2024-05-08T22:33:14.923910+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:09.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.910595+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.910354+0000","last_became_peered":"2024-05-08T22:33:14.910354+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:29.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934575+0000","last_change":"2024-05-08T22:33:14.924832+0000","last_active":"2024-05-08T22:33:18.934575+0000","last_peered":"2024-05-08T22:33:18.934575+0000","last_clean":"2024-05-08T22:33:18.934575+0000","last_became_active":"2024-05-08T22:33:14.924207+0000","last_became_peered":"2024-05-08T22:33:14.924207+0000","last_unstale":"2024-05-08T22:33:18.934575+0000","last_undegraded":"2024-05-08T22:33:18.934575+0000","last_fullsized":"2024-05-08T22:33:18.934575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998856+0000","last_change":"2024-05-08T22:33:14.922269+0000","last_active":"2024-05-08T22:33:18.998856+0000","last_peered":"2024-05-08T22:33:18.998856+0000","last_clean":"2024-05-08T22:33:18.998856+0000","last_became_active":"2024-05-08T22:33:14.921959+0000","last_became_peered":"2024-05-08T22:33:14.921959+0000","last_unstale":"2024-05-08T22:33:18.998856+0000","last_undegraded":"2024-05-08T22:33:18.998856+0000","last_fullsized":"2024-05-08T22:33:18.998856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934594+0000","last_change":"2024-05-08T22:33:14.930737+0000","last_active":"2024-05-08T22:33:18.934594+0000","last_peered":"2024-05-08T22:33:18.934594+0000","last_clean":"2024-05-08T22:33:18.934594+0000","last_became_active":"2024-05-08T22:33:14.930465+0000","last_became_peered":"2024-05-08T22:33:14.930465+0000","last_unstale":"2024-05-08T22:33:18.934594+0000","last_undegraded":"2024-05-08T22:33:18.934594+0000","last_fullsized":"2024-05-08T22:33:18.934594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:03.737555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930293+0000","last_change":"2024-05-08T22:33:14.911063+0000","last_active":"2024-05-08T22:33:18.930293+0000","last_peered":"2024-05-08T22:33:18.930293+0000","last_clean":"2024-05-08T22:33:18.930293+0000","last_became_active":"2024-05-08T22:33:14.910801+0000","last_became_peered":"2024-05-08T22:33:14.910801+0000","last_unstale":"2024-05-08T22:33:18.930293+0000","last_undegraded":"2024-05-08T22:33:18.930293+0000","last_fullsized":"2024-05-08T22:33:18.930293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:50.282359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448710+0000","last_change":"2024-05-08T22:33:14.916640+0000","last_active":"2024-05-08T22:33:19.448710+0000","last_peered":"2024-05-08T22:33:19.448710+0000","last_clean":"2024-05-08T22:33:19.448710+0000","last_became_active":"2024-05-08T22:33:14.916171+0000","last_became_peered":"2024-05-08T22:33:14.916171+0000","last_unstale":"2024-05-08T22:33:19.448710+0000","last_undegraded":"2024-05-08T22:33:19.448710+0000","last_fullsized":"2024-05-08T22:33:19.448710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999564+0000","last_change":"2024-05-08T22:33:14.925286+0000","last_active":"2024-05-08T22:33:18.999564+0000","last_peered":"2024-05-08T22:33:18.999564+0000","last_clean":"2024-05-08T22:33:18.999564+0000","last_became_active":"2024-05-08T22:33:14.924846+0000","last_became_peered":"2024-05-08T22:33:14.924846+0000","last_unstale":"2024-05-08T22:33:18.999564+0000","last_undegraded":"2024-05-08T22:33:18.999564+0000","last_fullsized":"2024-05-08T22:33:18.999564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:15.808109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":868352,"data_stored":697988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":274432,"data_stored":107128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":282624,"data_stored":107164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":698044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:33:38.070 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:33:38.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 Processor -- start 2024-05-08T22:33:38.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 -- start start 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa6301434d0 0x7fa63006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63006ed50 0x7fa6300b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6300cd200 con 0x7fa63006dc20 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6300cd4e0 con 0x7fa63006ed50 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63756f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6300cd7c0 con 0x7fa6301434d0 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa636d6e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63006ed50 0x7fa6300b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa636d6e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63006ed50 0x7fa6300b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34814/0 (socket says 172.21.15.179:34814) 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa636d6e700 1 -- 172.21.15.179:0/2413937797 learned_addr learned my addr 172.21.15.179:0/2413937797 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:38.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.183+0000 7fa63656d700 1 --2- 172.21.15.179:0/2413937797 >> v2:172.21.15.195:3300/0 conn(0x7fa6301434d0 0x7fa63006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa636d6e700 1 -- 172.21.15.179:0/2413937797 >> v2:172.21.15.195:3300/0 conn(0x7fa6301434d0 msgr2=0x7fa63006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa636d6e700 1 --2- 172.21.15.179:0/2413937797 >> v2:172.21.15.195:3300/0 conn(0x7fa6301434d0 0x7fa63006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa636d6e700 1 -- 172.21.15.179:0/2413937797 >> v1:172.21.15.179:6789/0 conn(0x7fa63006dc20 legacy=0x7fa63006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa636d6e700 1 -- 172.21.15.179:0/2413937797 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300cdaa0 con 0x7fa63006ed50 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63556b700 1 -- 172.21.15.179:0/2413937797 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919603830 0 0) 0x7fa6300cd200 con 0x7fa63006dc20 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa636d6e700 1 --2- 172.21.15.179:0/2413937797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6300b19e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa630070800 tx=0x7fa62800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed862665ce711143 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63556b700 1 -- 172.21.15.179:0/2413937797 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628010070 con 0x7fa63006ed50 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63556b700 1 -- 172.21.15.179:0/2413937797 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800dce0 con 0x7fa63006ed50 2024-05-08T22:33:38.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63556b700 1 -- 172.21.15.179:0/2413937797 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800b310 con 0x7fa63006ed50 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- 172.21.15.179:0/2413937797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 msgr2=0x7fa6300b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 --2- 172.21.15.179:0/2413937797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6300b19e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa630070800 tx=0x7fa62800d430 comp rx=0 tx=0).stop 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- 172.21.15.179:0/2413937797 shutdown_connections 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 --2- 172.21.15.179:0/2413937797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6300b19e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 --2- 172.21.15.179:0/2413937797 >> v2:172.21.15.195:3300/0 conn(0x7fa6301434d0 0x7fa63006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- 172.21.15.179:0/2413937797 >> 172.21.15.179:0/2413937797 conn(0x7fa6300ce9d0 msgr2=0x7fa6300cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- 172.21.15.179:0/2413937797 shutdown_connections 2024-05-08T22:33:38.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- 172.21.15.179:0/2413937797 wait complete. 2024-05-08T22:33:38.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 Processor -- start 2024-05-08T22:33:38.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.187+0000 7fa63756f700 1 -- start start 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6301434d0 0x7fa63013cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6300c4330 0x7fa63015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300c5830 con 0x7fa63006ed50 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6301ee1d0 con 0x7fa6301434d0 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300c48d0 con 0x7fa6300c4330 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34818/0 (socket says 172.21.15.179:34818) 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 -- 172.21.15.179:0/2850401820 learned_addr learned my addr 172.21.15.179:0/2850401820 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa635d6c700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6301434d0 0x7fa63013cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa636d6e700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6300c4330 0x7fa63015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 -- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6300c4330 msgr2=0x7fa63015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6300c4330 0x7fa63015c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 -- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6301434d0 msgr2=0x7fa63013cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6301434d0 0x7fa63013cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 -- 172.21.15.179:0/2850401820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62800bd40 con 0x7fa63006ed50 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63656d700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa62000e870 tx=0x7fa62000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6200095c0 con 0x7fa63006ed50 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62000ee70 con 0x7fa63006ed50 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6300c4bb0 con 0x7fa63006ed50 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000ca70 con 0x7fa63006ed50 2024-05-08T22:33:38.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6300c5170 con 0x7fa63006ed50 2024-05-08T22:33:38.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa62000cbd0 con 0x7fa63006ed50 2024-05-08T22:33:38.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.191+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa630067740 con 0x7fa63006ed50 2024-05-08T22:33:38.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.195+0000 7fa62effd700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 0x7fa610068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.195+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa6200844e0 con 0x7fa63006ed50 2024-05-08T22:33:38.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.195+0000 7fa635d6c700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 0x7fa610068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.195+0000 7fa635d6c700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 0x7fa610068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa630159c60 tx=0x7fa618009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.203+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa6200514c0 con 0x7fa63006ed50 2024-05-08T22:33:38.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.367+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6301438b0 con 0x7fa610066730 2024-05-08T22:33:38.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.387+0000 7fa62effd700 1 -- 172.21.15.179:0/2850401820 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345168 (secure 0 0 0) 0x7fa618001480 con 0x7fa610066730 2024-05-08T22:33:38.393 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 msgr2=0x7fa610068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 0x7fa610068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa630159c60 tx=0x7fa618009040 comp rx=0 tx=0).stop 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 msgr2=0x7fa6301edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa62000e870 tx=0x7fa62000ec30 comp rx=0 tx=0).stop 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 shutdown_connections 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa610066730 0x7fa610068bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6301434d0 0x7fa63013cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6300c4330 0x7fa63015c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 --2- 172.21.15.179:0/2850401820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63006ed50 0x7fa6301edc90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 >> 172.21.15.179:0/2850401820 conn(0x7fa6300ce9d0 msgr2=0x7fa63006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 shutdown_connections 2024-05-08T22:33:38.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.395+0000 7fa63756f700 1 -- 172.21.15.179:0/2850401820 wait complete. 2024-05-08T22:33:38.397 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:33:38.412 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-05-08T22:33:37.382815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235888,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738832,"statfs":{"total":773094113280,"available":772852563968,"internally_reserved":0,"allocated":3416064,"data_stored":2046228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005452"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.932430+0000","last_change":"2024-05-08T22:33:14.533515+0000","last_active":"2024-05-08T22:33:18.932430+0000","last_peered":"2024-05-08T22:33:18.932430+0000","last_clean":"2024-05-08T22:33:18.932430+0000","last_became_active":"2024-05-08T22:33:14.533293+0000","last_became_peered":"2024-05-08T22:33:14.533293+0000","last_unstale":"2024-05-08T22:33:18.932430+0000","last_undegraded":"2024-05-08T22:33:18.932430+0000","last_fullsized":"2024-05-08T22:33:18.932430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312810+0000","last_change":"2024-05-08T22:33:14.919770+0000","last_active":"2024-05-08T22:33:19.312810+0000","last_peered":"2024-05-08T22:33:19.312810+0000","last_clean":"2024-05-08T22:33:19.312810+0000","last_became_active":"2024-05-08T22:33:14.917629+0000","last_became_peered":"2024-05-08T22:33:14.917629+0000","last_unstale":"2024-05-08T22:33:19.312810+0000","last_undegraded":"2024-05-08T22:33:19.312810+0000","last_fullsized":"2024-05-08T22:33:19.312810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311922+0000","last_change":"2024-05-08T22:33:19.007654+0000","last_active":"2024-05-08T22:33:19.311922+0000","last_peered":"2024-05-08T22:33:19.311922+0000","last_clean":"2024-05-08T22:33:19.311922+0000","last_became_active":"2024-05-08T22:33:19.006811+0000","last_became_peered":"2024-05-08T22:33:19.006811+0000","last_unstale":"2024-05-08T22:33:19.311922+0000","last_undegraded":"2024-05-08T22:33:19.311922+0000","last_fullsized":"2024-05-08T22:33:19.311922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:28.811139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944044+0000","last_change":"2024-05-08T22:33:14.528074+0000","last_active":"2024-05-08T22:33:18.944044+0000","last_peered":"2024-05-08T22:33:18.944044+0000","last_clean":"2024-05-08T22:33:18.944044+0000","last_became_active":"2024-05-08T22:33:14.527773+0000","last_became_peered":"2024-05-08T22:33:14.527773+0000","last_unstale":"2024-05-08T22:33:18.944044+0000","last_undegraded":"2024-05-08T22:33:18.944044+0000","last_fullsized":"2024-05-08T22:33:18.944044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:53.046188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931850+0000","last_change":"2024-05-08T22:33:14.932746+0000","last_active":"2024-05-08T22:33:18.931850+0000","last_peered":"2024-05-08T22:33:18.931850+0000","last_clean":"2024-05-08T22:33:18.931850+0000","last_became_active":"2024-05-08T22:33:14.932511+0000","last_became_peered":"2024-05-08T22:33:14.932511+0000","last_unstale":"2024-05-08T22:33:18.931850+0000","last_undegraded":"2024-05-08T22:33:18.931850+0000","last_fullsized":"2024-05-08T22:33:18.931850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:05.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313017+0000","last_change":"2024-05-08T22:33:18.946740+0000","last_active":"2024-05-08T22:33:19.313017+0000","last_peered":"2024-05-08T22:33:19.313017+0000","last_clean":"2024-05-08T22:33:19.313017+0000","last_became_active":"2024-05-08T22:33:18.946369+0000","last_became_peered":"2024-05-08T22:33:18.946369+0000","last_unstale":"2024-05-08T22:33:19.313017+0000","last_undegraded":"2024-05-08T22:33:19.313017+0000","last_fullsized":"2024-05-08T22:33:19.313017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999447+0000","last_change":"2024-05-08T22:33:14.520155+0000","last_active":"2024-05-08T22:33:18.999447+0000","last_peered":"2024-05-08T22:33:18.999447+0000","last_clean":"2024-05-08T22:33:18.999447+0000","last_became_active":"2024-05-08T22:33:14.519845+0000","last_became_peered":"2024-05-08T22:33:14.519845+0000","last_unstale":"2024-05-08T22:33:18.999447+0000","last_undegraded":"2024-05-08T22:33:18.999447+0000","last_fullsized":"2024-05-08T22:33:18.999447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:03.999595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934686+0000","last_change":"2024-05-08T22:33:14.920443+0000","last_active":"2024-05-08T22:33:18.934686+0000","last_peered":"2024-05-08T22:33:18.934686+0000","last_clean":"2024-05-08T22:33:18.934686+0000","last_became_active":"2024-05-08T22:33:14.920034+0000","last_became_peered":"2024-05-08T22:33:14.920034+0000","last_unstale":"2024-05-08T22:33:18.934686+0000","last_undegraded":"2024-05-08T22:33:18.934686+0000","last_fullsized":"2024-05-08T22:33:18.934686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314786+0000","last_change":"2024-05-08T22:33:18.948680+0000","last_active":"2024-05-08T22:33:19.314786+0000","last_peered":"2024-05-08T22:33:19.314786+0000","last_clean":"2024-05-08T22:33:19.314786+0000","last_became_active":"2024-05-08T22:33:18.948252+0000","last_became_peered":"2024-05-08T22:33:18.948252+0000","last_unstale":"2024-05-08T22:33:19.314786+0000","last_undegraded":"2024-05-08T22:33:19.314786+0000","last_fullsized":"2024-05-08T22:33:19.314786+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:20.694843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999183+0000","last_change":"2024-05-08T22:33:14.526338+0000","last_active":"2024-05-08T22:33:18.999183+0000","last_peered":"2024-05-08T22:33:18.999183+0000","last_clean":"2024-05-08T22:33:18.999183+0000","last_became_active":"2024-05-08T22:33:14.520436+0000","last_became_peered":"2024-05-08T22:33:14.520436+0000","last_unstale":"2024-05-08T22:33:18.999183+0000","last_undegraded":"2024-05-08T22:33:18.999183+0000","last_fullsized":"2024-05-08T22:33:18.999183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:15.246381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935315+0000","last_change":"2024-05-08T22:33:14.919811+0000","last_active":"2024-05-08T22:33:18.935315+0000","last_peered":"2024-05-08T22:33:18.935315+0000","last_clean":"2024-05-08T22:33:18.935315+0000","last_became_active":"2024-05-08T22:33:14.919249+0000","last_became_peered":"2024-05-08T22:33:14.919249+0000","last_unstale":"2024-05-08T22:33:18.935315+0000","last_undegraded":"2024-05-08T22:33:18.935315+0000","last_fullsized":"2024-05-08T22:33:18.935315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:34.252236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313998+0000","last_change":"2024-05-08T22:33:18.942435+0000","last_active":"2024-05-08T22:33:19.313998+0000","last_peered":"2024-05-08T22:33:19.313998+0000","last_clean":"2024-05-08T22:33:19.313998+0000","last_became_active":"2024-05-08T22:33:18.942040+0000","last_became_peered":"2024-05-08T22:33:18.942040+0000","last_unstale":"2024-05-08T22:33:19.313998+0000","last_undegraded":"2024-05-08T22:33:19.313998+0000","last_fullsized":"2024-05-08T22:33:19.313998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927785+0000","last_change":"2024-05-08T22:33:14.913632+0000","last_active":"2024-05-08T22:33:18.927785+0000","last_peered":"2024-05-08T22:33:18.927785+0000","last_clean":"2024-05-08T22:33:18.927785+0000","last_became_active":"2024-05-08T22:33:14.913040+0000","last_became_peered":"2024-05-08T22:33:14.913040+0000","last_unstale":"2024-05-08T22:33:18.927785+0000","last_undegraded":"2024-05-08T22:33:18.927785+0000","last_fullsized":"2024-05-08T22:33:18.927785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998991+0000","last_change":"2024-05-08T22:33:14.526397+0000","last_active":"2024-05-08T22:33:18.998991+0000","last_peered":"2024-05-08T22:33:18.998991+0000","last_clean":"2024-05-08T22:33:18.998991+0000","last_became_active":"2024-05-08T22:33:14.519761+0000","last_became_peered":"2024-05-08T22:33:14.519761+0000","last_unstale":"2024-05-08T22:33:18.998991+0000","last_undegraded":"2024-05-08T22:33:18.998991+0000","last_fullsized":"2024-05-08T22:33:18.998991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315399+0000","last_change":"2024-05-08T22:33:18.944230+0000","last_active":"2024-05-08T22:33:19.315399+0000","last_peered":"2024-05-08T22:33:19.315399+0000","last_clean":"2024-05-08T22:33:19.315399+0000","last_became_active":"2024-05-08T22:33:18.943852+0000","last_became_peered":"2024-05-08T22:33:18.943852+0000","last_unstale":"2024-05-08T22:33:19.315399+0000","last_undegraded":"2024-05-08T22:33:19.315399+0000","last_fullsized":"2024-05-08T22:33:19.315399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312132+0000","last_change":"2024-05-08T22:33:14.912752+0000","last_active":"2024-05-08T22:33:19.312132+0000","last_peered":"2024-05-08T22:33:19.312132+0000","last_clean":"2024-05-08T22:33:19.312132+0000","last_became_active":"2024-05-08T22:33:14.912455+0000","last_became_peered":"2024-05-08T22:33:14.912455+0000","last_unstale":"2024-05-08T22:33:19.312132+0000","last_undegraded":"2024-05-08T22:33:19.312132+0000","last_fullsized":"2024-05-08T22:33:19.312132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934257+0000","last_change":"2024-05-08T22:33:14.525254+0000","last_active":"2024-05-08T22:33:18.934257+0000","last_peered":"2024-05-08T22:33:18.934257+0000","last_clean":"2024-05-08T22:33:18.934257+0000","last_became_active":"2024-05-08T22:33:14.524663+0000","last_became_peered":"2024-05-08T22:33:14.524663+0000","last_unstale":"2024-05-08T22:33:18.934257+0000","last_undegraded":"2024-05-08T22:33:18.934257+0000","last_fullsized":"2024-05-08T22:33:18.934257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314812+0000","last_change":"2024-05-08T22:33:18.947596+0000","last_active":"2024-05-08T22:33:19.314812+0000","last_peered":"2024-05-08T22:33:19.314812+0000","last_clean":"2024-05-08T22:33:19.314812+0000","last_became_active":"2024-05-08T22:33:18.947225+0000","last_became_peered":"2024-05-08T22:33:18.947225+0000","last_unstale":"2024-05-08T22:33:19.314812+0000","last_undegraded":"2024-05-08T22:33:19.314812+0000","last_fullsized":"2024-05-08T22:33:19.314812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312379+0000","last_change":"2024-05-08T22:33:14.906568+0000","last_active":"2024-05-08T22:33:19.312379+0000","last_peered":"2024-05-08T22:33:19.312379+0000","last_clean":"2024-05-08T22:33:19.312379+0000","last_became_active":"2024-05-08T22:33:14.906359+0000","last_became_peered":"2024-05-08T22:33:14.906359+0000","last_unstale":"2024-05-08T22:33:19.312379+0000","last_undegraded":"2024-05-08T22:33:19.312379+0000","last_fullsized":"2024-05-08T22:33:19.312379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:47.862614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934403+0000","last_change":"2024-05-08T22:33:14.530133+0000","last_active":"2024-05-08T22:33:18.934403+0000","last_peered":"2024-05-08T22:33:18.934403+0000","last_clean":"2024-05-08T22:33:18.934403+0000","last_became_active":"2024-05-08T22:33:14.529854+0000","last_became_peered":"2024-05-08T22:33:14.529854+0000","last_unstale":"2024-05-08T22:33:18.934403+0000","last_undegraded":"2024-05-08T22:33:18.934403+0000","last_fullsized":"2024-05-08T22:33:18.934403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:59.313380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448755+0000","last_change":"2024-05-08T22:33:19.004993+0000","last_active":"2024-05-08T22:33:19.448755+0000","last_peered":"2024-05-08T22:33:19.448755+0000","last_clean":"2024-05-08T22:33:19.448755+0000","last_became_active":"2024-05-08T22:33:19.004152+0000","last_became_peered":"2024-05-08T22:33:19.004152+0000","last_unstale":"2024-05-08T22:33:19.448755+0000","last_undegraded":"2024-05-08T22:33:19.448755+0000","last_fullsized":"2024-05-08T22:33:19.448755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:39.634020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929034+0000","last_change":"2024-05-08T22:33:14.909377+0000","last_active":"2024-05-08T22:33:18.929034+0000","last_peered":"2024-05-08T22:33:18.929034+0000","last_clean":"2024-05-08T22:33:18.929034+0000","last_became_active":"2024-05-08T22:33:14.909047+0000","last_became_peered":"2024-05-08T22:33:14.909047+0000","last_unstale":"2024-05-08T22:33:18.929034+0000","last_undegraded":"2024-05-08T22:33:18.929034+0000","last_fullsized":"2024-05-08T22:33:18.929034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:32.048397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998562+0000","last_change":"2024-05-08T22:33:14.529685+0000","last_active":"2024-05-08T22:33:18.998562+0000","last_peered":"2024-05-08T22:33:18.998562+0000","last_clean":"2024-05-08T22:33:18.998562+0000","last_became_active":"2024-05-08T22:33:14.529417+0000","last_became_peered":"2024-05-08T22:33:14.529417+0000","last_unstale":"2024-05-08T22:33:18.998562+0000","last_undegraded":"2024-05-08T22:33:18.998562+0000","last_fullsized":"2024-05-08T22:33:18.998562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.786143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314895+0000","last_change":"2024-05-08T22:33:18.948441+0000","last_active":"2024-05-08T22:33:19.314895+0000","last_peered":"2024-05-08T22:33:19.314895+0000","last_clean":"2024-05-08T22:33:19.314895+0000","last_became_active":"2024-05-08T22:33:18.948006+0000","last_became_peered":"2024-05-08T22:33:18.948006+0000","last_unstale":"2024-05-08T22:33:19.314895+0000","last_undegraded":"2024-05-08T22:33:19.314895+0000","last_fullsized":"2024-05-08T22:33:19.314895+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936140+0000","last_change":"2024-05-08T22:33:14.915374+0000","last_active":"2024-05-08T22:33:18.936140+0000","last_peered":"2024-05-08T22:33:18.936140+0000","last_clean":"2024-05-08T22:33:18.936140+0000","last_became_active":"2024-05-08T22:33:14.915035+0000","last_became_peered":"2024-05-08T22:33:14.915035+0000","last_unstale":"2024-05-08T22:33:18.936140+0000","last_undegraded":"2024-05-08T22:33:18.936140+0000","last_fullsized":"2024-05-08T22:33:18.936140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927757+0000","last_change":"2024-05-08T22:33:14.518968+0000","last_active":"2024-05-08T22:33:18.927757+0000","last_peered":"2024-05-08T22:33:18.927757+0000","last_clean":"2024-05-08T22:33:18.927757+0000","last_became_active":"2024-05-08T22:33:14.518318+0000","last_became_peered":"2024-05-08T22:33:14.518318+0000","last_unstale":"2024-05-08T22:33:18.927757+0000","last_undegraded":"2024-05-08T22:33:18.927757+0000","last_fullsized":"2024-05-08T22:33:18.927757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313504+0000","last_change":"2024-05-08T22:33:19.003063+0000","last_active":"2024-05-08T22:33:19.313504+0000","last_peered":"2024-05-08T22:33:19.313504+0000","last_clean":"2024-05-08T22:33:19.313504+0000","last_became_active":"2024-05-08T22:33:19.002557+0000","last_became_peered":"2024-05-08T22:33:19.002557+0000","last_unstale":"2024-05-08T22:33:19.313504+0000","last_undegraded":"2024-05-08T22:33:19.313504+0000","last_fullsized":"2024-05-08T22:33:19.313504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312756+0000","last_change":"2024-05-08T22:33:14.915824+0000","last_active":"2024-05-08T22:33:19.312756+0000","last_peered":"2024-05-08T22:33:19.312756+0000","last_clean":"2024-05-08T22:33:19.312756+0000","last_became_active":"2024-05-08T22:33:14.915554+0000","last_became_peered":"2024-05-08T22:33:14.915554+0000","last_unstale":"2024-05-08T22:33:19.312756+0000","last_undegraded":"2024-05-08T22:33:19.312756+0000","last_fullsized":"2024-05-08T22:33:19.312756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:10.398283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.937194+0000","last_change":"2024-05-08T22:33:14.526863+0000","last_active":"2024-05-08T22:33:18.937194+0000","last_peered":"2024-05-08T22:33:18.937194+0000","last_clean":"2024-05-08T22:33:18.937194+0000","last_became_active":"2024-05-08T22:33:14.526463+0000","last_became_peered":"2024-05-08T22:33:14.526463+0000","last_unstale":"2024-05-08T22:33:18.937194+0000","last_undegraded":"2024-05-08T22:33:18.937194+0000","last_fullsized":"2024-05-08T22:33:18.937194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311963+0000","last_change":"2024-05-08T22:33:19.005291+0000","last_active":"2024-05-08T22:33:19.311963+0000","last_peered":"2024-05-08T22:33:19.311963+0000","last_clean":"2024-05-08T22:33:19.311963+0000","last_became_active":"2024-05-08T22:33:19.004777+0000","last_became_peered":"2024-05-08T22:33:19.004777+0000","last_unstale":"2024-05-08T22:33:19.311963+0000","last_undegraded":"2024-05-08T22:33:19.311963+0000","last_fullsized":"2024-05-08T22:33:19.311963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:36.904973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448630+0000","last_change":"2024-05-08T22:33:14.911246+0000","last_active":"2024-05-08T22:33:19.448630+0000","last_peered":"2024-05-08T22:33:19.448630+0000","last_clean":"2024-05-08T22:33:19.448630+0000","last_became_active":"2024-05-08T22:33:14.910998+0000","last_became_peered":"2024-05-08T22:33:14.910998+0000","last_unstale":"2024-05-08T22:33:19.448630+0000","last_undegraded":"2024-05-08T22:33:19.448630+0000","last_fullsized":"2024-05-08T22:33:19.448630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999587+0000","last_change":"2024-05-08T22:33:14.529748+0000","last_active":"2024-05-08T22:33:18.999587+0000","last_peered":"2024-05-08T22:33:18.999587+0000","last_clean":"2024-05-08T22:33:18.999587+0000","last_became_active":"2024-05-08T22:33:14.529390+0000","last_became_peered":"2024-05-08T22:33:14.529390+0000","last_unstale":"2024-05-08T22:33:18.999587+0000","last_undegraded":"2024-05-08T22:33:18.999587+0000","last_fullsized":"2024-05-08T22:33:18.999587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:04.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.009380+0000","last_change":"2024-05-08T22:33:19.004064+0000","last_active":"2024-05-08T22:33:19.009380+0000","last_peered":"2024-05-08T22:33:19.009380+0000","last_clean":"2024-05-08T22:33:19.009380+0000","last_became_active":"2024-05-08T22:33:19.003638+0000","last_became_peered":"2024-05-08T22:33:19.003638+0000","last_unstale":"2024-05-08T22:33:19.009380+0000","last_undegraded":"2024-05-08T22:33:19.009380+0000","last_fullsized":"2024-05-08T22:33:19.009380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935398+0000","last_change":"2024-05-08T22:33:14.928058+0000","last_active":"2024-05-08T22:33:18.935398+0000","last_peered":"2024-05-08T22:33:18.935398+0000","last_clean":"2024-05-08T22:33:18.935398+0000","last_became_active":"2024-05-08T22:33:14.927408+0000","last_became_peered":"2024-05-08T22:33:14.927408+0000","last_unstale":"2024-05-08T22:33:18.935398+0000","last_undegraded":"2024-05-08T22:33:18.935398+0000","last_fullsized":"2024-05-08T22:33:18.935398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:22.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934334+0000","last_change":"2024-05-08T22:33:14.530667+0000","last_active":"2024-05-08T22:33:18.934334+0000","last_peered":"2024-05-08T22:33:18.934334+0000","last_clean":"2024-05-08T22:33:18.934334+0000","last_became_active":"2024-05-08T22:33:14.530429+0000","last_became_peered":"2024-05-08T22:33:14.530429+0000","last_unstale":"2024-05-08T22:33:18.934334+0000","last_undegraded":"2024-05-08T22:33:18.934334+0000","last_fullsized":"2024-05-08T22:33:18.934334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:38.212588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448536+0000","last_change":"2024-05-08T22:33:18.945810+0000","last_active":"2024-05-08T22:33:19.448536+0000","last_peered":"2024-05-08T22:33:19.448536+0000","last_clean":"2024-05-08T22:33:19.448536+0000","last_became_active":"2024-05-08T22:33:18.945204+0000","last_became_peered":"2024-05-08T22:33:18.945204+0000","last_unstale":"2024-05-08T22:33:19.448536+0000","last_undegraded":"2024-05-08T22:33:19.448536+0000","last_fullsized":"2024-05-08T22:33:19.448536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:26.949422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930747+0000","last_change":"2024-05-08T22:33:14.911989+0000","last_active":"2024-05-08T22:33:18.930747+0000","last_peered":"2024-05-08T22:33:18.930747+0000","last_clean":"2024-05-08T22:33:18.930747+0000","last_became_active":"2024-05-08T22:33:14.911685+0000","last_became_peered":"2024-05-08T22:33:14.911685+0000","last_unstale":"2024-05-08T22:33:18.930747+0000","last_undegraded":"2024-05-08T22:33:18.930747+0000","last_fullsized":"2024-05-08T22:33:18.930747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934973+0000","last_change":"2024-05-08T22:33:14.524541+0000","last_active":"2024-05-08T22:33:18.934973+0000","last_peered":"2024-05-08T22:33:18.934973+0000","last_clean":"2024-05-08T22:33:18.934973+0000","last_became_active":"2024-05-08T22:33:14.524290+0000","last_became_peered":"2024-05-08T22:33:14.524290+0000","last_unstale":"2024-05-08T22:33:18.934973+0000","last_undegraded":"2024-05-08T22:33:18.934973+0000","last_fullsized":"2024-05-08T22:33:18.934973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448609+0000","last_change":"2024-05-08T22:33:18.951903+0000","last_active":"2024-05-08T22:33:19.448609+0000","last_peered":"2024-05-08T22:33:19.448609+0000","last_clean":"2024-05-08T22:33:19.448609+0000","last_became_active":"2024-05-08T22:33:18.951473+0000","last_became_peered":"2024-05-08T22:33:18.951473+0000","last_unstale":"2024-05-08T22:33:19.448609+0000","last_undegraded":"2024-05-08T22:33:19.448609+0000","last_fullsized":"2024-05-08T22:33:19.448609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:19.057699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448450+0000","last_change":"2024-05-08T22:33:14.911724+0000","last_active":"2024-05-08T22:33:19.448450+0000","last_peered":"2024-05-08T22:33:19.448450+0000","last_clean":"2024-05-08T22:33:19.448450+0000","last_became_active":"2024-05-08T22:33:14.911493+0000","last_became_peered":"2024-05-08T22:33:14.911493+0000","last_unstale":"2024-05-08T22:33:19.448450+0000","last_undegraded":"2024-05-08T22:33:19.448450+0000","last_fullsized":"2024-05-08T22:33:19.448450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925211+0000","last_change":"2024-05-08T22:33:14.524989+0000","last_active":"2024-05-08T22:33:18.925211+0000","last_peered":"2024-05-08T22:33:18.925211+0000","last_clean":"2024-05-08T22:33:18.925211+0000","last_became_active":"2024-05-08T22:33:14.524715+0000","last_became_peered":"2024-05-08T22:33:14.524715+0000","last_unstale":"2024-05-08T22:33:18.925211+0000","last_undegraded":"2024-05-08T22:33:18.925211+0000","last_fullsized":"2024-05-08T22:33:18.925211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448775+0000","last_change":"2024-05-08T22:33:18.952088+0000","last_active":"2024-05-08T22:33:19.448775+0000","last_peered":"2024-05-08T22:33:19.448775+0000","last_clean":"2024-05-08T22:33:19.448775+0000","last_became_active":"2024-05-08T22:33:18.951660+0000","last_became_peered":"2024-05-08T22:33:18.951660+0000","last_unstale":"2024-05-08T22:33:19.448775+0000","last_undegraded":"2024-05-08T22:33:19.448775+0000","last_fullsized":"2024-05-08T22:33:19.448775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:42.700357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930121+0000","last_change":"2024-05-08T22:33:14.911625+0000","last_active":"2024-05-08T22:33:18.930121+0000","last_peered":"2024-05-08T22:33:18.930121+0000","last_clean":"2024-05-08T22:33:18.930121+0000","last_became_active":"2024-05-08T22:33:14.911368+0000","last_became_peered":"2024-05-08T22:33:14.911368+0000","last_unstale":"2024-05-08T22:33:18.930121+0000","last_undegraded":"2024-05-08T22:33:18.930121+0000","last_fullsized":"2024-05-08T22:33:18.930121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998588+0000","last_change":"2024-05-08T22:33:14.529960+0000","last_active":"2024-05-08T22:33:18.998588+0000","last_peered":"2024-05-08T22:33:18.998588+0000","last_clean":"2024-05-08T22:33:18.998588+0000","last_became_active":"2024-05-08T22:33:14.529479+0000","last_became_peered":"2024-05-08T22:33:14.529479+0000","last_unstale":"2024-05-08T22:33:18.998588+0000","last_undegraded":"2024-05-08T22:33:18.998588+0000","last_fullsized":"2024-05-08T22:33:18.998588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:49.272820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.010817+0000","last_change":"2024-05-08T22:33:19.005638+0000","last_active":"2024-05-08T22:33:19.010817+0000","last_peered":"2024-05-08T22:33:19.010817+0000","last_clean":"2024-05-08T22:33:19.010817+0000","last_became_active":"2024-05-08T22:33:19.003876+0000","last_became_peered":"2024-05-08T22:33:19.003876+0000","last_unstale":"2024-05-08T22:33:19.010817+0000","last_undegraded":"2024-05-08T22:33:19.010817+0000","last_fullsized":"2024-05-08T22:33:19.010817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312713+0000","last_change":"2024-05-08T22:33:14.926812+0000","last_active":"2024-05-08T22:33:19.312713+0000","last_peered":"2024-05-08T22:33:19.312713+0000","last_clean":"2024-05-08T22:33:19.312713+0000","last_became_active":"2024-05-08T22:33:14.926536+0000","last_became_peered":"2024-05-08T22:33:14.926536+0000","last_unstale":"2024-05-08T22:33:19.312713+0000","last_undegraded":"2024-05-08T22:33:19.312713+0000","last_fullsized":"2024-05-08T22:33:19.312713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:18.111855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934524+0000","last_change":"2024-05-08T22:33:14.523530+0000","last_active":"2024-05-08T22:33:18.934524+0000","last_peered":"2024-05-08T22:33:18.934524+0000","last_clean":"2024-05-08T22:33:18.934524+0000","last_became_active":"2024-05-08T22:33:14.523193+0000","last_became_peered":"2024-05-08T22:33:14.523193+0000","last_unstale":"2024-05-08T22:33:18.934524+0000","last_undegraded":"2024-05-08T22:33:18.934524+0000","last_fullsized":"2024-05-08T22:33:18.934524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998772+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:33:18.998772+0000","last_peered":"2024-05-08T22:33:18.998772+0000","last_clean":"2024-05-08T22:33:18.998772+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:33:18.998772+0000","last_undegraded":"2024-05-08T22:33:18.998772+0000","last_fullsized":"2024-05-08T22:33:18.998772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007908+0000","last_change":"2024-05-08T22:33:19.006286+0000","last_active":"2024-05-08T22:33:19.007908+0000","last_peered":"2024-05-08T22:33:19.007908+0000","last_clean":"2024-05-08T22:33:19.007908+0000","last_became_active":"2024-05-08T22:33:19.005632+0000","last_became_peered":"2024-05-08T22:33:19.005632+0000","last_unstale":"2024-05-08T22:33:19.007908+0000","last_undegraded":"2024-05-08T22:33:19.007908+0000","last_fullsized":"2024-05-08T22:33:19.007908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:39.728431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999358+0000","last_change":"2024-05-08T22:33:14.931567+0000","last_active":"2024-05-08T22:33:18.999358+0000","last_peered":"2024-05-08T22:33:18.999358+0000","last_clean":"2024-05-08T22:33:18.999358+0000","last_became_active":"2024-05-08T22:33:14.931285+0000","last_became_peered":"2024-05-08T22:33:14.931285+0000","last_unstale":"2024-05-08T22:33:18.999358+0000","last_undegraded":"2024-05-08T22:33:18.999358+0000","last_fullsized":"2024-05-08T22:33:18.999358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925484+0000","last_change":"2024-05-08T22:33:14.523189+0000","last_active":"2024-05-08T22:33:18.925484+0000","last_peered":"2024-05-08T22:33:18.925484+0000","last_clean":"2024-05-08T22:33:18.925484+0000","last_became_active":"2024-05-08T22:33:14.522942+0000","last_became_peered":"2024-05-08T22:33:14.522942+0000","last_unstale":"2024-05-08T22:33:18.925484+0000","last_undegraded":"2024-05-08T22:33:18.925484+0000","last_fullsized":"2024-05-08T22:33:18.925484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007291+0000","last_change":"2024-05-08T22:33:19.003092+0000","last_active":"2024-05-08T22:33:19.007291+0000","last_peered":"2024-05-08T22:33:19.007291+0000","last_clean":"2024-05-08T22:33:19.007291+0000","last_became_active":"2024-05-08T22:33:19.002313+0000","last_became_peered":"2024-05-08T22:33:19.002313+0000","last_unstale":"2024-05-08T22:33:19.007291+0000","last_undegraded":"2024-05-08T22:33:19.007291+0000","last_fullsized":"2024-05-08T22:33:19.007291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:17.965361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934771+0000","last_change":"2024-05-08T22:33:14.927660+0000","last_active":"2024-05-08T22:33:18.934771+0000","last_peered":"2024-05-08T22:33:18.934771+0000","last_clean":"2024-05-08T22:33:18.934771+0000","last_became_active":"2024-05-08T22:33:14.925184+0000","last_became_peered":"2024-05-08T22:33:14.925184+0000","last_unstale":"2024-05-08T22:33:18.934771+0000","last_undegraded":"2024-05-08T22:33:18.934771+0000","last_fullsized":"2024-05-08T22:33:18.934771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:55.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934291+0000","last_change":"2024-05-08T22:33:14.524892+0000","last_active":"2024-05-08T22:33:18.934291+0000","last_peered":"2024-05-08T22:33:18.934291+0000","last_clean":"2024-05-08T22:33:18.934291+0000","last_became_active":"2024-05-08T22:33:14.524584+0000","last_became_peered":"2024-05-08T22:33:14.524584+0000","last_unstale":"2024-05-08T22:33:18.934291+0000","last_undegraded":"2024-05-08T22:33:18.934291+0000","last_fullsized":"2024-05-08T22:33:18.934291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953385+0000","last_change":"2024-05-08T22:33:18.947268+0000","last_active":"2024-05-08T22:33:18.953385+0000","last_peered":"2024-05-08T22:33:18.953385+0000","last_clean":"2024-05-08T22:33:18.953385+0000","last_became_active":"2024-05-08T22:33:18.946822+0000","last_became_peered":"2024-05-08T22:33:18.946822+0000","last_unstale":"2024-05-08T22:33:18.953385+0000","last_undegraded":"2024-05-08T22:33:18.953385+0000","last_fullsized":"2024-05-08T22:33:18.953385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999808+0000","last_change":"2024-05-08T22:33:14.925288+0000","last_active":"2024-05-08T22:33:18.999808+0000","last_peered":"2024-05-08T22:33:18.999808+0000","last_clean":"2024-05-08T22:33:18.999808+0000","last_became_active":"2024-05-08T22:33:14.924982+0000","last_became_peered":"2024-05-08T22:33:14.924982+0000","last_unstale":"2024-05-08T22:33:18.999808+0000","last_undegraded":"2024-05-08T22:33:18.999808+0000","last_fullsized":"2024-05-08T22:33:18.999808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925406+0000","last_change":"2024-05-08T22:33:14.531914+0000","last_active":"2024-05-08T22:33:18.925406+0000","last_peered":"2024-05-08T22:33:18.925406+0000","last_clean":"2024-05-08T22:33:18.925406+0000","last_became_active":"2024-05-08T22:33:14.531550+0000","last_became_peered":"2024-05-08T22:33:14.531550+0000","last_unstale":"2024-05-08T22:33:18.925406+0000","last_undegraded":"2024-05-08T22:33:18.925406+0000","last_fullsized":"2024-05-08T22:33:18.925406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:12.564410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006729+0000","last_change":"2024-05-08T22:33:19.005176+0000","last_active":"2024-05-08T22:33:19.006729+0000","last_peered":"2024-05-08T22:33:19.006729+0000","last_clean":"2024-05-08T22:33:19.006729+0000","last_became_active":"2024-05-08T22:33:19.004757+0000","last_became_peered":"2024-05-08T22:33:19.004757+0000","last_unstale":"2024-05-08T22:33:19.006729+0000","last_undegraded":"2024-05-08T22:33:19.006729+0000","last_fullsized":"2024-05-08T22:33:19.006729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.924451+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.922189+0000","last_became_peered":"2024-05-08T22:33:14.922189+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927288+0000","last_change":"2024-05-08T22:33:14.527475+0000","last_active":"2024-05-08T22:33:18.927288+0000","last_peered":"2024-05-08T22:33:18.927288+0000","last_clean":"2024-05-08T22:33:18.927288+0000","last_became_active":"2024-05-08T22:33:14.526627+0000","last_became_peered":"2024-05-08T22:33:14.526627+0000","last_unstale":"2024-05-08T22:33:18.927288+0000","last_undegraded":"2024-05-08T22:33:18.927288+0000","last_fullsized":"2024-05-08T22:33:18.927288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:00.386313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312868+0000","last_change":"2024-05-08T22:33:18.946891+0000","last_active":"2024-05-08T22:33:19.312868+0000","last_peered":"2024-05-08T22:33:19.312868+0000","last_clean":"2024-05-08T22:33:19.312868+0000","last_became_active":"2024-05-08T22:33:18.946534+0000","last_became_peered":"2024-05-08T22:33:18.946534+0000","last_unstale":"2024-05-08T22:33:19.312868+0000","last_undegraded":"2024-05-08T22:33:19.312868+0000","last_fullsized":"2024-05-08T22:33:19.312868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999925+0000","last_change":"2024-05-08T22:33:14.918650+0000","last_active":"2024-05-08T22:33:18.999925+0000","last_peered":"2024-05-08T22:33:18.999925+0000","last_clean":"2024-05-08T22:33:18.999925+0000","last_became_active":"2024-05-08T22:33:14.918389+0000","last_became_peered":"2024-05-08T22:33:14.918389+0000","last_unstale":"2024-05-08T22:33:18.999925+0000","last_undegraded":"2024-05-08T22:33:18.999925+0000","last_fullsized":"2024-05-08T22:33:18.999925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925062+0000","last_change":"2024-05-08T22:33:14.522438+0000","last_active":"2024-05-08T22:33:18.925062+0000","last_peered":"2024-05-08T22:33:18.925062+0000","last_clean":"2024-05-08T22:33:18.925062+0000","last_became_active":"2024-05-08T22:33:14.522094+0000","last_became_peered":"2024-05-08T22:33:14.522094+0000","last_unstale":"2024-05-08T22:33:18.925062+0000","last_undegraded":"2024-05-08T22:33:18.925062+0000","last_fullsized":"2024-05-08T22:33:18.925062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.950611+0000","last_change":"2024-05-08T22:33:18.944271+0000","last_active":"2024-05-08T22:33:18.950611+0000","last_peered":"2024-05-08T22:33:18.950611+0000","last_clean":"2024-05-08T22:33:18.950611+0000","last_became_active":"2024-05-08T22:33:18.943934+0000","last_became_peered":"2024-05-08T22:33:18.943934+0000","last_unstale":"2024-05-08T22:33:18.950611+0000","last_undegraded":"2024-05-08T22:33:18.950611+0000","last_fullsized":"2024-05-08T22:33:18.950611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926385+0000","last_change":"2024-05-08T22:33:14.915516+0000","last_active":"2024-05-08T22:33:18.926385+0000","last_peered":"2024-05-08T22:33:18.926385+0000","last_clean":"2024-05-08T22:33:18.926385+0000","last_became_active":"2024-05-08T22:33:14.915253+0000","last_became_peered":"2024-05-08T22:33:14.915253+0000","last_unstale":"2024-05-08T22:33:18.926385+0000","last_undegraded":"2024-05-08T22:33:18.926385+0000","last_fullsized":"2024-05-08T22:33:18.926385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925993+0000","last_change":"2024-05-08T22:33:14.517759+0000","last_active":"2024-05-08T22:33:18.925993+0000","last_peered":"2024-05-08T22:33:18.925993+0000","last_clean":"2024-05-08T22:33:18.925993+0000","last_became_active":"2024-05-08T22:33:14.517109+0000","last_became_peered":"2024-05-08T22:33:14.517109+0000","last_unstale":"2024-05-08T22:33:18.925993+0000","last_undegraded":"2024-05-08T22:33:18.925993+0000","last_fullsized":"2024-05-08T22:33:18.925993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:13.973941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313006+0000","last_change":"2024-05-08T22:33:19.005763+0000","last_active":"2024-05-08T22:33:19.313006+0000","last_peered":"2024-05-08T22:33:19.313006+0000","last_clean":"2024-05-08T22:33:19.313006+0000","last_became_active":"2024-05-08T22:33:19.004827+0000","last_became_peered":"2024-05-08T22:33:19.004827+0000","last_unstale":"2024-05-08T22:33:19.313006+0000","last_undegraded":"2024-05-08T22:33:19.313006+0000","last_fullsized":"2024-05-08T22:33:19.313006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:24.712864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998964+0000","last_change":"2024-05-08T22:33:14.918080+0000","last_active":"2024-05-08T22:33:18.998964+0000","last_peered":"2024-05-08T22:33:18.998964+0000","last_clean":"2024-05-08T22:33:18.998964+0000","last_became_active":"2024-05-08T22:33:14.917791+0000","last_became_peered":"2024-05-08T22:33:14.917791+0000","last_unstale":"2024-05-08T22:33:18.998964+0000","last_undegraded":"2024-05-08T22:33:18.998964+0000","last_fullsized":"2024-05-08T22:33:18.998964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934850+0000","last_change":"2024-05-08T22:33:14.527314+0000","last_active":"2024-05-08T22:33:18.934850+0000","last_peered":"2024-05-08T22:33:18.934850+0000","last_clean":"2024-05-08T22:33:18.934850+0000","last_became_active":"2024-05-08T22:33:14.526667+0000","last_became_peered":"2024-05-08T22:33:14.526667+0000","last_unstale":"2024-05-08T22:33:18.934850+0000","last_undegraded":"2024-05-08T22:33:18.934850+0000","last_fullsized":"2024-05-08T22:33:18.934850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312923+0000","last_change":"2024-05-08T22:33:19.004038+0000","last_active":"2024-05-08T22:33:19.312923+0000","last_peered":"2024-05-08T22:33:19.312923+0000","last_clean":"2024-05-08T22:33:19.312923+0000","last_became_active":"2024-05-08T22:33:19.002976+0000","last_became_peered":"2024-05-08T22:33:19.002976+0000","last_unstale":"2024-05-08T22:33:19.312923+0000","last_undegraded":"2024-05-08T22:33:19.312923+0000","last_fullsized":"2024-05-08T22:33:19.312923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.506642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930214+0000","last_change":"2024-05-08T22:33:14.924563+0000","last_active":"2024-05-08T22:33:18.930214+0000","last_peered":"2024-05-08T22:33:18.930214+0000","last_clean":"2024-05-08T22:33:18.930214+0000","last_became_active":"2024-05-08T22:33:14.924299+0000","last_became_peered":"2024-05-08T22:33:14.924299+0000","last_unstale":"2024-05-08T22:33:18.930214+0000","last_undegraded":"2024-05-08T22:33:18.930214+0000","last_fullsized":"2024-05-08T22:33:18.930214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931836+0000","last_change":"2024-05-08T22:33:14.531490+0000","last_active":"2024-05-08T22:33:18.931836+0000","last_peered":"2024-05-08T22:33:18.931836+0000","last_clean":"2024-05-08T22:33:18.931836+0000","last_became_active":"2024-05-08T22:33:14.530820+0000","last_became_peered":"2024-05-08T22:33:14.530820+0000","last_unstale":"2024-05-08T22:33:18.931836+0000","last_undegraded":"2024-05-08T22:33:18.931836+0000","last_fullsized":"2024-05-08T22:33:18.931836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:59.224955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006802+0000","last_change":"2024-05-08T22:33:19.005429+0000","last_active":"2024-05-08T22:33:19.006802+0000","last_peered":"2024-05-08T22:33:19.006802+0000","last_clean":"2024-05-08T22:33:19.006802+0000","last_became_active":"2024-05-08T22:33:19.004941+0000","last_became_peered":"2024-05-08T22:33:19.004941+0000","last_unstale":"2024-05-08T22:33:19.006802+0000","last_undegraded":"2024-05-08T22:33:19.006802+0000","last_fullsized":"2024-05-08T22:33:19.006802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999232+0000","last_change":"2024-05-08T22:33:14.919406+0000","last_active":"2024-05-08T22:33:18.999232+0000","last_peered":"2024-05-08T22:33:18.999232+0000","last_clean":"2024-05-08T22:33:18.999232+0000","last_became_active":"2024-05-08T22:33:14.919204+0000","last_became_peered":"2024-05-08T22:33:14.919204+0000","last_unstale":"2024-05-08T22:33:18.999232+0000","last_undegraded":"2024-05-08T22:33:18.999232+0000","last_fullsized":"2024-05-08T22:33:18.999232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:24.279224+0000","last_change":"2024-05-08T22:33:14.523404+0000","last_active":"2024-05-08T22:33:24.279224+0000","last_peered":"2024-05-08T22:33:24.279224+0000","last_clean":"2024-05-08T22:33:24.279224+0000","last_became_active":"2024-05-08T22:33:14.522325+0000","last_became_peered":"2024-05-08T22:33:14.522325+0000","last_unstale":"2024-05-08T22:33:24.279224+0000","last_undegraded":"2024-05-08T22:33:24.279224+0000","last_fullsized":"2024-05-08T22:33:24.279224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448558+0000","last_change":"2024-05-08T22:33:18.950110+0000","last_active":"2024-05-08T22:33:19.448558+0000","last_peered":"2024-05-08T22:33:19.448558+0000","last_clean":"2024-05-08T22:33:19.448558+0000","last_became_active":"2024-05-08T22:33:18.949555+0000","last_became_peered":"2024-05-08T22:33:18.949555+0000","last_unstale":"2024-05-08T22:33:19.448558+0000","last_undegraded":"2024-05-08T22:33:19.448558+0000","last_fullsized":"2024-05-08T22:33:19.448558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:15.389269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926732+0000","last_change":"2024-05-08T22:33:14.908111+0000","last_active":"2024-05-08T22:33:18.926732+0000","last_peered":"2024-05-08T22:33:18.926732+0000","last_clean":"2024-05-08T22:33:18.926732+0000","last_became_active":"2024-05-08T22:33:14.907276+0000","last_became_peered":"2024-05-08T22:33:14.907276+0000","last_unstale":"2024-05-08T22:33:18.926732+0000","last_undegraded":"2024-05-08T22:33:18.926732+0000","last_fullsized":"2024-05-08T22:33:18.926732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:53.154720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934191+0000","last_change":"2024-05-08T22:33:14.521905+0000","last_active":"2024-05-08T22:33:18.934191+0000","last_peered":"2024-05-08T22:33:18.934191+0000","last_clean":"2024-05-08T22:33:18.934191+0000","last_became_active":"2024-05-08T22:33:14.521518+0000","last_became_peered":"2024-05-08T22:33:14.521518+0000","last_unstale":"2024-05-08T22:33:18.934191+0000","last_undegraded":"2024-05-08T22:33:18.934191+0000","last_fullsized":"2024-05-08T22:33:18.934191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315185+0000","last_change":"2024-05-08T22:33:18.939252+0000","last_active":"2024-05-08T22:33:19.315185+0000","last_peered":"2024-05-08T22:33:19.315185+0000","last_clean":"2024-05-08T22:33:19.315185+0000","last_became_active":"2024-05-08T22:33:18.938564+0000","last_became_peered":"2024-05-08T22:33:18.938564+0000","last_unstale":"2024-05-08T22:33:19.315185+0000","last_undegraded":"2024-05-08T22:33:19.315185+0000","last_fullsized":"2024-05-08T22:33:19.315185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:42.775813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931955+0000","last_change":"2024-05-08T22:33:14.932581+0000","last_active":"2024-05-08T22:33:18.931955+0000","last_peered":"2024-05-08T22:33:18.931955+0000","last_clean":"2024-05-08T22:33:18.931955+0000","last_became_active":"2024-05-08T22:33:14.932332+0000","last_became_peered":"2024-05-08T22:33:14.932332+0000","last_unstale":"2024-05-08T22:33:18.931955+0000","last_undegraded":"2024-05-08T22:33:18.931955+0000","last_fullsized":"2024-05-08T22:33:18.931955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:49:02.412782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935272+0000","last_change":"2024-05-08T22:33:14.525414+0000","last_active":"2024-05-08T22:33:18.935272+0000","last_peered":"2024-05-08T22:33:18.935272+0000","last_clean":"2024-05-08T22:33:18.935272+0000","last_became_active":"2024-05-08T22:33:14.524869+0000","last_became_peered":"2024-05-08T22:33:14.524869+0000","last_unstale":"2024-05-08T22:33:18.935272+0000","last_undegraded":"2024-05-08T22:33:18.935272+0000","last_fullsized":"2024-05-08T22:33:18.935272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312533+0000","last_change":"2024-05-08T22:33:18.948135+0000","last_active":"2024-05-08T22:33:19.312533+0000","last_peered":"2024-05-08T22:33:19.312533+0000","last_clean":"2024-05-08T22:33:19.312533+0000","last_became_active":"2024-05-08T22:33:18.947623+0000","last_became_peered":"2024-05-08T22:33:18.947623+0000","last_unstale":"2024-05-08T22:33:19.312533+0000","last_undegraded":"2024-05-08T22:33:19.312533+0000","last_fullsized":"2024-05-08T22:33:19.312533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:55.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935185+0000","last_change":"2024-05-08T22:33:14.925264+0000","last_active":"2024-05-08T22:33:18.935185+0000","last_peered":"2024-05-08T22:33:18.935185+0000","last_clean":"2024-05-08T22:33:18.935185+0000","last_became_active":"2024-05-08T22:33:14.924936+0000","last_became_peered":"2024-05-08T22:33:14.924936+0000","last_unstale":"2024-05-08T22:33:18.935185+0000","last_undegraded":"2024-05-08T22:33:18.935185+0000","last_fullsized":"2024-05-08T22:33:18.935185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927536+0000","last_change":"2024-05-08T22:33:14.519535+0000","last_active":"2024-05-08T22:33:18.927536+0000","last_peered":"2024-05-08T22:33:18.927536+0000","last_clean":"2024-05-08T22:33:18.927536+0000","last_became_active":"2024-05-08T22:33:14.519264+0000","last_became_peered":"2024-05-08T22:33:14.519264+0000","last_unstale":"2024-05-08T22:33:18.927536+0000","last_undegraded":"2024-05-08T22:33:18.927536+0000","last_fullsized":"2024-05-08T22:33:18.927536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:19.007144+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:19.006007+0000","last_became_peered":"2024-05-08T22:33:19.006007+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:50.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.011031+0000","last_change":"2024-05-08T22:33:19.006206+0000","last_active":"2024-05-08T22:33:19.011031+0000","last_peered":"2024-05-08T22:33:19.011031+0000","last_clean":"2024-05-08T22:33:19.011031+0000","last_became_active":"2024-05-08T22:33:19.004897+0000","last_became_peered":"2024-05-08T22:33:19.004897+0000","last_unstale":"2024-05-08T22:33:19.011031+0000","last_undegraded":"2024-05-08T22:33:19.011031+0000","last_fullsized":"2024-05-08T22:33:19.011031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448478+0000","last_change":"2024-05-08T22:33:14.906510+0000","last_active":"2024-05-08T22:33:19.448478+0000","last_peered":"2024-05-08T22:33:19.448478+0000","last_clean":"2024-05-08T22:33:19.448478+0000","last_became_active":"2024-05-08T22:33:14.906227+0000","last_became_peered":"2024-05-08T22:33:14.906227+0000","last_unstale":"2024-05-08T22:33:19.448478+0000","last_undegraded":"2024-05-08T22:33:19.448478+0000","last_fullsized":"2024-05-08T22:33:19.448478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935784+0000","last_change":"2024-05-08T22:33:14.523994+0000","last_active":"2024-05-08T22:33:18.935784+0000","last_peered":"2024-05-08T22:33:18.935784+0000","last_clean":"2024-05-08T22:33:18.935784+0000","last_became_active":"2024-05-08T22:33:14.523710+0000","last_became_peered":"2024-05-08T22:33:14.523710+0000","last_unstale":"2024-05-08T22:33:18.935784+0000","last_undegraded":"2024-05-08T22:33:18.935784+0000","last_fullsized":"2024-05-08T22:33:18.935784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:56.994588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944980+0000","last_change":"2024-05-08T22:33:18.944209+0000","last_active":"2024-05-08T22:33:18.944980+0000","last_peered":"2024-05-08T22:33:18.944980+0000","last_clean":"2024-05-08T22:33:18.944980+0000","last_became_active":"2024-05-08T22:33:18.943824+0000","last_became_peered":"2024-05-08T22:33:18.943824+0000","last_unstale":"2024-05-08T22:33:18.944980+0000","last_undegraded":"2024-05-08T22:33:18.944980+0000","last_fullsized":"2024-05-08T22:33:18.944980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:04.454019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312284+0000","last_change":"2024-05-08T22:33:14.912166+0000","last_active":"2024-05-08T22:33:19.312284+0000","last_peered":"2024-05-08T22:33:19.312284+0000","last_clean":"2024-05-08T22:33:19.312284+0000","last_became_active":"2024-05-08T22:33:14.911941+0000","last_became_peered":"2024-05-08T22:33:14.911941+0000","last_unstale":"2024-05-08T22:33:19.312284+0000","last_undegraded":"2024-05-08T22:33:19.312284+0000","last_fullsized":"2024-05-08T22:33:19.312284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943398+0000","last_change":"2024-05-08T22:33:14.524713+0000","last_active":"2024-05-08T22:33:18.943398+0000","last_peered":"2024-05-08T22:33:18.943398+0000","last_clean":"2024-05-08T22:33:18.943398+0000","last_became_active":"2024-05-08T22:33:14.523688+0000","last_became_peered":"2024-05-08T22:33:14.523688+0000","last_unstale":"2024-05-08T22:33:18.943398+0000","last_undegraded":"2024-05-08T22:33:18.943398+0000","last_fullsized":"2024-05-08T22:33:18.943398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:12.770341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928004+0000","last_change":"2024-05-08T22:33:14.518843+0000","last_active":"2024-05-08T22:33:18.928004+0000","last_peered":"2024-05-08T22:33:18.928004+0000","last_clean":"2024-05-08T22:33:18.928004+0000","last_became_active":"2024-05-08T22:33:14.518002+0000","last_became_peered":"2024-05-08T22:33:14.518002+0000","last_unstale":"2024-05-08T22:33:18.928004+0000","last_undegraded":"2024-05-08T22:33:18.928004+0000","last_fullsized":"2024-05-08T22:33:18.928004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931587+0000","last_change":"2024-05-08T22:33:14.914102+0000","last_active":"2024-05-08T22:33:18.931587+0000","last_peered":"2024-05-08T22:33:18.931587+0000","last_clean":"2024-05-08T22:33:18.931587+0000","last_became_active":"2024-05-08T22:33:14.913777+0000","last_became_peered":"2024-05-08T22:33:14.913777+0000","last_unstale":"2024-05-08T22:33:18.931587+0000","last_undegraded":"2024-05-08T22:33:18.931587+0000","last_fullsized":"2024-05-08T22:33:18.931587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953813+0000","last_change":"2024-05-08T22:33:18.949165+0000","last_active":"2024-05-08T22:33:18.953813+0000","last_peered":"2024-05-08T22:33:18.953813+0000","last_clean":"2024-05-08T22:33:18.953813+0000","last_became_active":"2024-05-08T22:33:18.948660+0000","last_became_peered":"2024-05-08T22:33:18.948660+0000","last_unstale":"2024-05-08T22:33:18.953813+0000","last_undegraded":"2024-05-08T22:33:18.953813+0000","last_fullsized":"2024-05-08T22:33:18.953813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:44.316596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943131+0000","last_change":"2024-05-08T22:33:14.532226+0000","last_active":"2024-05-08T22:33:18.943131+0000","last_peered":"2024-05-08T22:33:18.943131+0000","last_clean":"2024-05-08T22:33:18.943131+0000","last_became_active":"2024-05-08T22:33:14.531943+0000","last_became_peered":"2024-05-08T22:33:14.531943+0000","last_unstale":"2024-05-08T22:33:18.943131+0000","last_undegraded":"2024-05-08T22:33:18.943131+0000","last_fullsized":"2024-05-08T22:33:18.943131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:51.052408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928212+0000","last_change":"2024-05-08T22:33:14.907843+0000","last_active":"2024-05-08T22:33:18.928212+0000","last_peered":"2024-05-08T22:33:18.928212+0000","last_clean":"2024-05-08T22:33:18.928212+0000","last_became_active":"2024-05-08T22:33:14.907511+0000","last_became_peered":"2024-05-08T22:33:14.907511+0000","last_unstale":"2024-05-08T22:33:18.928212+0000","last_undegraded":"2024-05-08T22:33:18.928212+0000","last_fullsized":"2024-05-08T22:33:18.928212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:38.769965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312725+0000","last_change":"2024-05-08T22:33:18.952988+0000","last_active":"2024-05-08T22:33:19.312725+0000","last_peered":"2024-05-08T22:33:19.312725+0000","last_clean":"2024-05-08T22:33:19.312725+0000","last_became_active":"2024-05-08T22:33:18.948797+0000","last_became_peered":"2024-05-08T22:33:18.948797+0000","last_unstale":"2024-05-08T22:33:19.312725+0000","last_undegraded":"2024-05-08T22:33:19.312725+0000","last_fullsized":"2024-05-08T22:33:19.312725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:10.379649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926181+0000","last_change":"2024-05-08T22:33:14.915829+0000","last_active":"2024-05-08T22:33:18.926181+0000","last_peered":"2024-05-08T22:33:18.926181+0000","last_clean":"2024-05-08T22:33:18.926181+0000","last_became_active":"2024-05-08T22:33:14.915496+0000","last_became_peered":"2024-05-08T22:33:14.915496+0000","last_unstale":"2024-05-08T22:33:18.926181+0000","last_undegraded":"2024-05-08T22:33:18.926181+0000","last_fullsized":"2024-05-08T22:33:18.926181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925052+0000","last_change":"2024-05-08T22:33:14.930927+0000","last_active":"2024-05-08T22:33:18.925052+0000","last_peered":"2024-05-08T22:33:18.925052+0000","last_clean":"2024-05-08T22:33:18.925052+0000","last_became_active":"2024-05-08T22:33:14.930569+0000","last_became_peered":"2024-05-08T22:33:14.930569+0000","last_unstale":"2024-05-08T22:33:18.925052+0000","last_undegraded":"2024-05-08T22:33:18.925052+0000","last_fullsized":"2024-05-08T22:33:18.925052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934767+0000","last_change":"2024-05-08T22:33:14.928385+0000","last_active":"2024-05-08T22:33:18.934767+0000","last_peered":"2024-05-08T22:33:18.934767+0000","last_clean":"2024-05-08T22:33:18.934767+0000","last_became_active":"2024-05-08T22:33:14.928162+0000","last_became_peered":"2024-05-08T22:33:14.928162+0000","last_unstale":"2024-05-08T22:33:18.934767+0000","last_undegraded":"2024-05-08T22:33:18.934767+0000","last_fullsized":"2024-05-08T22:33:18.934767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:11.717668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931430+0000","last_change":"2024-05-08T22:33:14.916147+0000","last_active":"2024-05-08T22:33:18.931430+0000","last_peered":"2024-05-08T22:33:18.931430+0000","last_clean":"2024-05-08T22:33:18.931430+0000","last_became_active":"2024-05-08T22:33:14.915762+0000","last_became_peered":"2024-05-08T22:33:14.915762+0000","last_unstale":"2024-05-08T22:33:18.931430+0000","last_undegraded":"2024-05-08T22:33:18.931430+0000","last_fullsized":"2024-05-08T22:33:18.931430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:28.668944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313453+0000","last_change":"2024-05-08T22:33:14.912915+0000","last_active":"2024-05-08T22:33:19.313453+0000","last_peered":"2024-05-08T22:33:19.313453+0000","last_clean":"2024-05-08T22:33:19.313453+0000","last_became_active":"2024-05-08T22:33:14.912698+0000","last_became_peered":"2024-05-08T22:33:14.912698+0000","last_unstale":"2024-05-08T22:33:19.313453+0000","last_undegraded":"2024-05-08T22:33:19.313453+0000","last_fullsized":"2024-05-08T22:33:19.313453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448664+0000","last_change":"2024-05-08T22:33:14.907107+0000","last_active":"2024-05-08T22:33:19.448664+0000","last_peered":"2024-05-08T22:33:19.448664+0000","last_clean":"2024-05-08T22:33:19.448664+0000","last_became_active":"2024-05-08T22:33:14.906444+0000","last_became_peered":"2024-05-08T22:33:14.906444+0000","last_unstale":"2024-05-08T22:33:19.448664+0000","last_undegraded":"2024-05-08T22:33:19.448664+0000","last_fullsized":"2024-05-08T22:33:19.448664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929966+0000","last_change":"2024-05-08T22:33:14.913584+0000","last_active":"2024-05-08T22:33:18.929966+0000","last_peered":"2024-05-08T22:33:18.929966+0000","last_clean":"2024-05-08T22:33:18.929966+0000","last_became_active":"2024-05-08T22:33:14.913304+0000","last_became_peered":"2024-05-08T22:33:14.913304+0000","last_unstale":"2024-05-08T22:33:18.929966+0000","last_undegraded":"2024-05-08T22:33:18.929966+0000","last_fullsized":"2024-05-08T22:33:18.929966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312698+0000","last_change":"2024-05-08T22:33:14.905976+0000","last_active":"2024-05-08T22:33:19.312698+0000","last_peered":"2024-05-08T22:33:19.312698+0000","last_clean":"2024-05-08T22:33:19.312698+0000","last_became_active":"2024-05-08T22:33:14.905734+0000","last_became_peered":"2024-05-08T22:33:14.905734+0000","last_unstale":"2024-05-08T22:33:19.312698+0000","last_undegraded":"2024-05-08T22:33:19.312698+0000","last_fullsized":"2024-05-08T22:33:19.312698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935694+0000","last_change":"2024-05-08T22:33:14.927777+0000","last_active":"2024-05-08T22:33:18.935694+0000","last_peered":"2024-05-08T22:33:18.935694+0000","last_clean":"2024-05-08T22:33:18.935694+0000","last_became_active":"2024-05-08T22:33:14.927297+0000","last_became_peered":"2024-05-08T22:33:14.927297+0000","last_unstale":"2024-05-08T22:33:18.935694+0000","last_undegraded":"2024-05-08T22:33:18.935694+0000","last_fullsized":"2024-05-08T22:33:18.935694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:16.838261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312806+0000","last_change":"2024-05-08T22:33:14.911565+0000","last_active":"2024-05-08T22:33:19.312806+0000","last_peered":"2024-05-08T22:33:19.312806+0000","last_clean":"2024-05-08T22:33:19.312806+0000","last_became_active":"2024-05-08T22:33:14.909251+0000","last_became_peered":"2024-05-08T22:33:14.909251+0000","last_unstale":"2024-05-08T22:33:19.312806+0000","last_undegraded":"2024-05-08T22:33:19.312806+0000","last_fullsized":"2024-05-08T22:33:19.312806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934680+0000","last_change":"2024-05-08T22:33:14.925382+0000","last_active":"2024-05-08T22:33:18.934680+0000","last_peered":"2024-05-08T22:33:18.934680+0000","last_clean":"2024-05-08T22:33:18.934680+0000","last_became_active":"2024-05-08T22:33:14.925133+0000","last_became_peered":"2024-05-08T22:33:14.925133+0000","last_unstale":"2024-05-08T22:33:18.934680+0000","last_undegraded":"2024-05-08T22:33:18.934680+0000","last_fullsized":"2024-05-08T22:33:18.934680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931302+0000","last_change":"2024-05-08T22:33:14.911744+0000","last_active":"2024-05-08T22:33:18.931302+0000","last_peered":"2024-05-08T22:33:18.931302+0000","last_clean":"2024-05-08T22:33:18.931302+0000","last_became_active":"2024-05-08T22:33:14.910853+0000","last_became_peered":"2024-05-08T22:33:14.910853+0000","last_unstale":"2024-05-08T22:33:18.931302+0000","last_undegraded":"2024-05-08T22:33:18.931302+0000","last_fullsized":"2024-05-08T22:33:18.931302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928361+0000","last_change":"2024-05-08T22:33:14.914067+0000","last_active":"2024-05-08T22:33:18.928361+0000","last_peered":"2024-05-08T22:33:18.928361+0000","last_clean":"2024-05-08T22:33:18.928361+0000","last_became_active":"2024-05-08T22:33:14.913755+0000","last_became_peered":"2024-05-08T22:33:14.913755+0000","last_unstale":"2024-05-08T22:33:18.928361+0000","last_undegraded":"2024-05-08T22:33:18.928361+0000","last_fullsized":"2024-05-08T22:33:18.928361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999511+0000","last_change":"2024-05-08T22:33:14.922252+0000","last_active":"2024-05-08T22:33:18.999511+0000","last_peered":"2024-05-08T22:33:18.999511+0000","last_clean":"2024-05-08T22:33:18.999511+0000","last_became_active":"2024-05-08T22:33:14.921949+0000","last_became_peered":"2024-05-08T22:33:14.921949+0000","last_unstale":"2024-05-08T22:33:18.999511+0000","last_undegraded":"2024-05-08T22:33:18.999511+0000","last_fullsized":"2024-05-08T22:33:18.999511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:12.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925280+0000","last_change":"2024-05-08T22:33:14.916689+0000","last_active":"2024-05-08T22:33:18.925280+0000","last_peered":"2024-05-08T22:33:18.925280+0000","last_clean":"2024-05-08T22:33:18.925280+0000","last_became_active":"2024-05-08T22:33:14.916200+0000","last_became_peered":"2024-05-08T22:33:14.916200+0000","last_unstale":"2024-05-08T22:33:18.925280+0000","last_undegraded":"2024-05-08T22:33:18.925280+0000","last_fullsized":"2024-05-08T22:33:18.925280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936475+0000","last_change":"2024-05-08T22:33:14.925461+0000","last_active":"2024-05-08T22:33:18.936475+0000","last_peered":"2024-05-08T22:33:18.936475+0000","last_clean":"2024-05-08T22:33:18.936475+0000","last_became_active":"2024-05-08T22:33:14.924786+0000","last_became_peered":"2024-05-08T22:33:14.924786+0000","last_unstale":"2024-05-08T22:33:18.936475+0000","last_undegraded":"2024-05-08T22:33:18.936475+0000","last_fullsized":"2024-05-08T22:33:18.936475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935606+0000","last_change":"2024-05-08T22:33:14.918139+0000","last_active":"2024-05-08T22:33:18.935606+0000","last_peered":"2024-05-08T22:33:18.935606+0000","last_clean":"2024-05-08T22:33:18.935606+0000","last_became_active":"2024-05-08T22:33:14.914720+0000","last_became_peered":"2024-05-08T22:33:14.914720+0000","last_unstale":"2024-05-08T22:33:18.935606+0000","last_undegraded":"2024-05-08T22:33:18.935606+0000","last_fullsized":"2024-05-08T22:33:18.935606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448502+0000","last_change":"2024-05-08T22:33:14.905986+0000","last_active":"2024-05-08T22:33:19.448502+0000","last_peered":"2024-05-08T22:33:19.448502+0000","last_clean":"2024-05-08T22:33:19.448502+0000","last_became_active":"2024-05-08T22:33:14.905613+0000","last_became_peered":"2024-05-08T22:33:14.905613+0000","last_unstale":"2024-05-08T22:33:19.448502+0000","last_undegraded":"2024-05-08T22:33:19.448502+0000","last_fullsized":"2024-05-08T22:33:19.448502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930058+0000","last_change":"2024-05-08T22:33:14.911270+0000","last_active":"2024-05-08T22:33:18.930058+0000","last_peered":"2024-05-08T22:33:18.930058+0000","last_clean":"2024-05-08T22:33:18.930058+0000","last_became_active":"2024-05-08T22:33:14.910966+0000","last_became_peered":"2024-05-08T22:33:14.910966+0000","last_unstale":"2024-05-08T22:33:18.930058+0000","last_undegraded":"2024-05-08T22:33:18.930058+0000","last_fullsized":"2024-05-08T22:33:18.930058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:40.907481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312110+0000","last_change":"2024-05-08T22:33:14.917778+0000","last_active":"2024-05-08T22:33:19.312110+0000","last_peered":"2024-05-08T22:33:19.312110+0000","last_clean":"2024-05-08T22:33:19.312110+0000","last_became_active":"2024-05-08T22:33:14.917531+0000","last_became_peered":"2024-05-08T22:33:14.917531+0000","last_unstale":"2024-05-08T22:33:19.312110+0000","last_undegraded":"2024-05-08T22:33:19.312110+0000","last_fullsized":"2024-05-08T22:33:19.312110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926531+0000","last_change":"2024-05-08T22:33:14.916563+0000","last_active":"2024-05-08T22:33:18.926531+0000","last_peered":"2024-05-08T22:33:18.926531+0000","last_clean":"2024-05-08T22:33:18.926531+0000","last_became_active":"2024-05-08T22:33:14.916271+0000","last_became_peered":"2024-05-08T22:33:14.916271+0000","last_unstale":"2024-05-08T22:33:18.926531+0000","last_undegraded":"2024-05-08T22:33:18.926531+0000","last_fullsized":"2024-05-08T22:33:18.926531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927948+0000","last_change":"2024-05-08T22:33:14.931269+0000","last_active":"2024-05-08T22:33:18.927948+0000","last_peered":"2024-05-08T22:33:18.927948+0000","last_clean":"2024-05-08T22:33:18.927948+0000","last_became_active":"2024-05-08T22:33:14.931026+0000","last_became_peered":"2024-05-08T22:33:14.931026+0000","last_unstale":"2024-05-08T22:33:18.927948+0000","last_undegraded":"2024-05-08T22:33:18.927948+0000","last_fullsized":"2024-05-08T22:33:18.927948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929869+0000","last_change":"2024-05-08T22:33:14.928697+0000","last_active":"2024-05-08T22:33:18.929869+0000","last_peered":"2024-05-08T22:33:18.929869+0000","last_clean":"2024-05-08T22:33:18.929869+0000","last_became_active":"2024-05-08T22:33:14.928400+0000","last_became_peered":"2024-05-08T22:33:14.928400+0000","last_unstale":"2024-05-08T22:33:18.929869+0000","last_undegraded":"2024-05-08T22:33:18.929869+0000","last_fullsized":"2024-05-08T22:33:18.929869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999653+0000","last_change":"2024-05-08T22:33:14.931677+0000","last_active":"2024-05-08T22:33:18.999653+0000","last_peered":"2024-05-08T22:33:18.999653+0000","last_clean":"2024-05-08T22:33:18.999653+0000","last_became_active":"2024-05-08T22:33:14.931437+0000","last_became_peered":"2024-05-08T22:33:14.931437+0000","last_unstale":"2024-05-08T22:33:18.999653+0000","last_undegraded":"2024-05-08T22:33:18.999653+0000","last_fullsized":"2024-05-08T22:33:18.999653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:07.861504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:14.924165+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:14.923910+0000","last_became_peered":"2024-05-08T22:33:14.923910+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:09.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.910595+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.910354+0000","last_became_peered":"2024-05-08T22:33:14.910354+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:29.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934575+0000","last_change":"2024-05-08T22:33:14.924832+0000","last_active":"2024-05-08T22:33:18.934575+0000","last_peered":"2024-05-08T22:33:18.934575+0000","last_clean":"2024-05-08T22:33:18.934575+0000","last_became_active":"2024-05-08T22:33:14.924207+0000","last_became_peered":"2024-05-08T22:33:14.924207+0000","last_unstale":"2024-05-08T22:33:18.934575+0000","last_undegraded":"2024-05-08T22:33:18.934575+0000","last_fullsized":"2024-05-08T22:33:18.934575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998856+0000","last_change":"2024-05-08T22:33:14.922269+0000","last_active":"2024-05-08T22:33:18.998856+0000","last_peered":"2024-05-08T22:33:18.998856+0000","last_clean":"2024-05-08T22:33:18.998856+0000","last_became_active":"2024-05-08T22:33:14.921959+0000","last_became_peered":"2024-05-08T22:33:14.921959+0000","last_unstale":"2024-05-08T22:33:18.998856+0000","last_undegraded":"2024-05-08T22:33:18.998856+0000","last_fullsized":"2024-05-08T22:33:18.998856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934594+0000","last_change":"2024-05-08T22:33:14.930737+0000","last_active":"2024-05-08T22:33:18.934594+0000","last_peered":"2024-05-08T22:33:18.934594+0000","last_clean":"2024-05-08T22:33:18.934594+0000","last_became_active":"2024-05-08T22:33:14.930465+0000","last_became_peered":"2024-05-08T22:33:14.930465+0000","last_unstale":"2024-05-08T22:33:18.934594+0000","last_undegraded":"2024-05-08T22:33:18.934594+0000","last_fullsized":"2024-05-08T22:33:18.934594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:03.737555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930293+0000","last_change":"2024-05-08T22:33:14.911063+0000","last_active":"2024-05-08T22:33:18.930293+0000","last_peered":"2024-05-08T22:33:18.930293+0000","last_clean":"2024-05-08T22:33:18.930293+0000","last_became_active":"2024-05-08T22:33:14.910801+0000","last_became_peered":"2024-05-08T22:33:14.910801+0000","last_unstale":"2024-05-08T22:33:18.930293+0000","last_undegraded":"2024-05-08T22:33:18.930293+0000","last_fullsized":"2024-05-08T22:33:18.930293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:50.282359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448710+0000","last_change":"2024-05-08T22:33:14.916640+0000","last_active":"2024-05-08T22:33:19.448710+0000","last_peered":"2024-05-08T22:33:19.448710+0000","last_clean":"2024-05-08T22:33:19.448710+0000","last_became_active":"2024-05-08T22:33:14.916171+0000","last_became_peered":"2024-05-08T22:33:14.916171+0000","last_unstale":"2024-05-08T22:33:19.448710+0000","last_undegraded":"2024-05-08T22:33:19.448710+0000","last_fullsized":"2024-05-08T22:33:19.448710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999564+0000","last_change":"2024-05-08T22:33:14.925286+0000","last_active":"2024-05-08T22:33:18.999564+0000","last_peered":"2024-05-08T22:33:18.999564+0000","last_clean":"2024-05-08T22:33:18.999564+0000","last_became_active":"2024-05-08T22:33:14.924846+0000","last_became_peered":"2024-05-08T22:33:14.924846+0000","last_unstale":"2024-05-08T22:33:18.999564+0000","last_undegraded":"2024-05-08T22:33:18.999564+0000","last_fullsized":"2024-05-08T22:33:18.999564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:15.808109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":868352,"data_stored":697988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":274432,"data_stored":107128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":282624,"data_stored":107164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":698044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:33:38.417 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:33:38.417 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T22:33:38.417 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T22:33:38.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T22:33:38.418 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T22:33:38.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 Processor -- start 2024-05-08T22:33:38.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 -- start start 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 0x7f92a40507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f92a40dfc20 0x7f92a40335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a4050d10 con 0x7f92a40e0040 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40d6b70 con 0x7f92a40dfc20 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92abfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40e0bc0 con 0x7f92a404ff30 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f92a40dfc20 0x7f92a40335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f92a40dfc20 0x7f92a40335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34826/0 (socket says 172.21.15.179:34826) 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 -- 172.21.15.179:0/2697638648 learned_addr learned my addr 172.21.15.179:0/2697638648 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aaffd700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 0x7f92a40507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 -- 172.21.15.179:0/2697638648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 msgr2=0x7f92a40507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 0x7f92a40507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 -- 172.21.15.179:0/2697638648 >> v1:172.21.15.195:6789/0 conn(0x7f92a40e0040 legacy=0x7f92a4033b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 -- 172.21.15.179:0/2697638648 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40e0600 con 0x7f92a40dfc20 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aaffd700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 0x7f92a40507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.535+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a40dfc20 0x7f92a40335c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f929c009910 tx=0x7f929c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ef574d84dd97c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92a9ffb700 1 -- 172.21.15.179:0/2697638648 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00ba60 con 0x7f92a40dfc20 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92a9ffb700 1 -- 172.21.15.179:0/2697638648 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00bbc0 con 0x7f92a40dfc20 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92a9ffb700 1 -- 172.21.15.179:0/2697638648 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c009e60 con 0x7f92a40dfc20 2024-05-08T22:33:38.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- 172.21.15.179:0/2697638648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a40dfc20 msgr2=0x7f92a40335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a40dfc20 0x7f92a40335c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f929c009910 tx=0x7f929c009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- 172.21.15.179:0/2697638648 shutdown_connections 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a40dfc20 0x7f92a40335c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- 172.21.15.179:0/2697638648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a404ff30 0x7f92a40507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- 172.21.15.179:0/2697638648 >> 172.21.15.179:0/2697638648 conn(0x7f92a4054b70 msgr2=0x7f92a4054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- 172.21.15.179:0/2697638648 shutdown_connections 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- 172.21.15.179:0/2697638648 wait complete. 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 Processor -- start 2024-05-08T22:33:38.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- start start 2024-05-08T22:33:38.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 0x7f92a40d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a40e0040 0x7f92a4052fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a404d1a0 con 0x7f92a404ff30 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40d33e0 con 0x7f92a40dfc20 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92abfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40d2e20 con 0x7f92a40e0040 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34830/0 (socket says 172.21.15.179:34830) 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 -- 172.21.15.179:0/1587830916 learned_addr learned my addr 172.21.15.179:0/1587830916 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92ab7fe700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a40e0040 0x7f92a4052fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 0x7f92a40d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 -- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a40e0040 msgr2=0x7f92a4052fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a40e0040 0x7f92a4052fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 -- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 msgr2=0x7f92a40d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 0x7f92a40d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aaffd700 1 -- 172.21.15.179:0/1587830916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c009570 con 0x7f92a404ff30 2024-05-08T22:33:38.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.539+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 0x7f92a40d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92aaffd700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e810 tx=0x7f92a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0004230 con 0x7f92a404ff30 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a00043b0 con 0x7f92a404ff30 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40d3100 con 0x7f92a404ff30 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a00036c0 con 0x7f92a404ff30 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a40d2040 con 0x7f92a404ff30 2024-05-08T22:33:38.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a404cc90 con 0x7f92a404ff30 2024-05-08T22:33:38.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f92a0003820 con 0x7f92a404ff30 2024-05-08T22:33:38.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 0x7f9298068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.543+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f92a0083d50 con 0x7f92a404ff30 2024-05-08T22:33:38.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.547+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 0x7f9298068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.547+0000 7f92aa7fc700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 0x7f9298068c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f929c000c00 tx=0x7f929c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.551+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f92a404cc90 con 0x7f92a404ff30 2024-05-08T22:33:38.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.759+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f92a404cc90 con 0x7f92a404ff30 2024-05-08T22:33:38.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.763+0000 7f92937fe700 1 -- 172.21.15.179:0/1587830916 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164890 (secure 0 0 0) 0x7f92a404cc90 con 0x7f92a404ff30 2024-05-08T22:33:38.765 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:38.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 msgr2=0x7f9298068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 0x7f9298068c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f929c000c00 tx=0x7f929c008030 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 msgr2=0x7f92a40d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e810 tx=0x7f92a000eb20 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 shutdown_connections 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9298066760 0x7f9298068c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a40dfc20 0x7f92a40d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92a40e0040 0x7f92a4052fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 --2- 172.21.15.179:0/1587830916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a404ff30 0x7f92a40d1160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 >> 172.21.15.179:0/1587830916 conn(0x7f92a4054b70 msgr2=0x7f92a40d8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 shutdown_connections 2024-05-08T22:33:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.767+0000 7f92abfff700 1 -- 172.21.15.179:0/1587830916 wait complete. 2024-05-08T22:33:38.779 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":25951},{"type":"v1","addr":"172.21.15.195:6833","nonce":25951}]},"active_addr":"172.21.15.195:6833/25951","active_change":"2024-05-08T22:33:03.361806+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":453841741}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":497690178}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2120567300}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":3213766961}]}]} 2024-05-08T22:33:38.781 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T22:33:38.781 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T22:33:38.782 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:38.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 Processor -- start 2024-05-08T22:33:38.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- start start 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8c0048210 0x7fe8c00eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 0x7fe8c0052f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c004fe20 con 0x7fe8c0033fb0 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c004fb80 con 0x7fe8c0048210 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00502b0 con 0x7fe8c0047cc0 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe8c0033fb0 0x7fe8c00eb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33412/0 (socket says 172.21.15.179:33412) 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c7fff700 1 -- 172.21.15.179:0/1880469421 learned_addr learned my addr 172.21.15.179:0/1880469421 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/1880469421 >> v2:172.21.15.195:3300/0 conn(0x7fe8c0048210 0x7fe8c00eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cce77700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 0x7fe8c0052f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c6ffd700 1 -- 172.21.15.179:0/1880469421 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648193742 0 0) 0x7fe8c004fe20 con 0x7fe8c0033fb0 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c6ffd700 1 -- 172.21.15.179:0/1880469421 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b00035d0 con 0x7fe8c0033fb0 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 -- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 msgr2=0x7fe8c0052f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 0x7fe8c0052f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 -- 172.21.15.179:0/1880469421 >> v1:172.21.15.179:6789/0 conn(0x7fe8c0033fb0 legacy=0x7fe8c00eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 -- 172.21.15.179:0/1880469421 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00ebe70 con 0x7fe8c0048210 2024-05-08T22:33:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cce77700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 0x7fe8c0052f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00eb930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8c804f480 tx=0x7fe8c804f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b3505969f5cadd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c6ffd700 1 -- 172.21.15.179:0/1880469421 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8067250 con 0x7fe8c0048210 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c6ffd700 1 -- 172.21.15.179:0/1880469421 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80673d0 con 0x7fe8c0048210 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8c6ffd700 1 -- 172.21.15.179:0/1880469421 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8067960 con 0x7fe8c0048210 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 msgr2=0x7fe8c00eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00eb930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8c804f480 tx=0x7fe8c804f710 comp rx=0 tx=0).stop 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- 172.21.15.179:0/1880469421 shutdown_connections 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0047cc0 0x7fe8c0052f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 --2- 172.21.15.179:0/1880469421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00eb930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- 172.21.15.179:0/1880469421 >> 172.21.15.179:0/1880469421 conn(0x7fe8c0054b70 msgr2=0x7fe8c0054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.899+0000 7fe8cd678700 1 -- 172.21.15.179:0/1880469421 shutdown_connections 2024-05-08T22:33:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- 172.21.15.179:0/1880469421 wait complete. 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 Processor -- start 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- start start 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 0x7fe8c004abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00db020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c004c6d0 con 0x7fe8c0047cc0 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c004b500 con 0x7fe8c0048210 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c004bd70 con 0x7fe8c0033fb0 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34842/0 (socket says 172.21.15.179:34842) 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 -- 172.21.15.179:0/2478492650 learned_addr learned my addr 172.21.15.179:0/2478492650 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cce77700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c7fff700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 0x7fe8c004abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 -- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 msgr2=0x7fe8c004abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 0x7fe8c004abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 -- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 msgr2=0x7fe8c00db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 -- 172.21.15.179:0/2478492650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0094b0 con 0x7fe8c0047cc0 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c7fff700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 0x7fe8c004abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cce77700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c77fe700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8074ec0 tx=0x7fe8c80739d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8066eb0 con 0x7fe8c0047cc0 2024-05-08T22:33:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c004c010 con 0x7fe8c0047cc0 2024-05-08T22:33:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c00e7c50 con 0x7fe8c0047cc0 2024-05-08T22:33:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8066150 con 0x7fe8c0047cc0 2024-05-08T22:33:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.903+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80664c0 con 0x7fe8c0047cc0 2024-05-08T22:33:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.907+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a8005310 con 0x7fe8c0047cc0 2024-05-08T22:33:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.907+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8c8066620 con 0x7fe8c0047cc0 2024-05-08T22:33:38.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.907+0000 7fe8c4ff9700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 0x7fe8b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:38.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.907+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8c8099070 con 0x7fe8c0047cc0 2024-05-08T22:33:38.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.907+0000 7fe8c7fff700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 0x7fe8b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:38.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.911+0000 7fe8c7fff700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 0x7fe8b4068bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005930 tx=0x7fe8b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:38.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:38.915+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe8c80cc310 con 0x7fe8c0047cc0 2024-05-08T22:33:39.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.083+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8a8005890 con 0x7fe8c0047cc0 2024-05-08T22:33:39.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8c4ff9700 1 -- 172.21.15.179:0/2478492650 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7fe8c80cf930 con 0x7fe8c0047cc0 2024-05-08T22:33:39.087 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:39.087 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:39.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 msgr2=0x7fe8b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 0x7fe8b4068bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005930 tx=0x7fe8b8009040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 msgr2=0x7fe8c00e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8074ec0 tx=0x7fe8c80739d0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 shutdown_connections 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe8b4066730 0x7fe8b4068bf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0048210 0x7fe8c00db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c0047cc0 0x7fe8c00e4920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 --2- 172.21.15.179:0/2478492650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0033fb0 0x7fe8c004abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 >> 172.21.15.179:0/2478492650 conn(0x7fe8c0054b70 msgr2=0x7fe8c00e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 shutdown_connections 2024-05-08T22:33:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.087+0000 7fe8cd678700 1 -- 172.21.15.179:0/2478492650 wait complete. 2024-05-08T22:33:39.102 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T22:33:39.102 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:39.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 Processor -- start 2024-05-08T22:33:39.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 -- start start 2024-05-08T22:33:39.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 0x7f14680cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146806abb0 con 0x7f14680cc420 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.215+0000 7f146f0de700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14680cbba0 con 0x7f14680cb1b0 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146815c2f0 con 0x7f146806de60 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146d8db700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 0x7f14680cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53000/0 (socket says 172.21.15.179:53000) 2024-05-08T22:33:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 -- 172.21.15.179:0/3141825491 learned_addr learned my addr 172.21.15.179:0/3141825491 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 -- 172.21.15.179:0/3141825491 >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 msgr2=0x7f14680cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 --2- 172.21.15.179:0/3141825491 >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 0x7f14680cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 -- 172.21.15.179:0/3141825491 >> v1:172.21.15.195:6789/0 conn(0x7f14680cc420 legacy=0x7f14680cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 -- 172.21.15.179:0/3141825491 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146815b770 con 0x7f146806de60 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146d8db700 1 --2- 172.21.15.179:0/3141825491 >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 0x7f14680cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146e8dd700 1 --2- 172.21.15.179:0/3141825491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1460009910 tx=0x7f1460009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd07ffd05c6e91f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146d0da700 1 -- 172.21.15.179:0/3141825491 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011830 con 0x7f146806de60 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146d0da700 1 -- 172.21.15.179:0/3141825491 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460011990 con 0x7f146806de60 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146d0da700 1 -- 172.21.15.179:0/3141825491 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011d40 con 0x7f146806de60 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- 172.21.15.179:0/3141825491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 msgr2=0x7f146806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 --2- 172.21.15.179:0/3141825491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1460009910 tx=0x7f1460009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- 172.21.15.179:0/3141825491 shutdown_connections 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 --2- 172.21.15.179:0/3141825491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f146806de60 0x7f146806e2b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 --2- 172.21.15.179:0/3141825491 >> v2:172.21.15.195:3300/0 conn(0x7f14680cb1b0 0x7f14680cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- 172.21.15.179:0/3141825491 >> 172.21.15.179:0/3141825491 conn(0x7f14680ce9d0 msgr2=0x7f14680cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- 172.21.15.179:0/3141825491 shutdown_connections 2024-05-08T22:33:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.219+0000 7f146f0de700 1 -- 172.21.15.179:0/3141825491 wait complete. 2024-05-08T22:33:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 Processor -- start 2024-05-08T22:33:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- start start 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14680cb1b0 0x7f146813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14680cc420 0x7f14680ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146806bac0 con 0x7f146806de60 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468154330 con 0x7f14680cb1b0 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14681520a0 con 0x7f14680cc420 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34850/0 (socket says 172.21.15.179:34850) 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 -- 172.21.15.179:0/302381811 learned_addr learned my addr 172.21.15.179:0/302381811 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146d8db700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14680cb1b0 0x7f146813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e8dd700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14680cc420 0x7f14680ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 -- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14680cc420 msgr2=0x7f14680ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14680cc420 0x7f14680ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 -- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14680cb1b0 msgr2=0x7f146813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14680cb1b0 0x7f146813ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 -- 172.21.15.179:0/302381811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464009a70 con 0x7f146806de60 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146e0dc700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f145800c500 tx=0x7f145800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580052d0 con 0x7f146806de60 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1460009570 con 0x7f146806de60 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1468152660 con 0x7f146806de60 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580011c0 con 0x7f146806de60 2024-05-08T22:33:39.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.223+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458006880 con 0x7f146806de60 2024-05-08T22:33:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1448005310 con 0x7f146806de60 2024-05-08T22:33:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14580069e0 con 0x7f146806de60 2024-05-08T22:33:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f145effd700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 0x7f1454068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1458083890 con 0x7f146806de60 2024-05-08T22:33:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f146d8db700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 0x7f1454068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.227+0000 7f146d8db700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 0x7f1454068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1464009fd0 tx=0x7f146400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.235+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f14580507f0 con 0x7f146806de60 2024-05-08T22:33:39.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.407+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14480055c0 con 0x7f146806de60 2024-05-08T22:33:39.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.411+0000 7f145effd700 1 -- 172.21.15.179:0/302381811 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f1458053e10 con 0x7f146806de60 2024-05-08T22:33:39.413 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:39.413 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:39.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 msgr2=0x7f1454068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 0x7f1454068bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1464009fd0 tx=0x7f146400b040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 msgr2=0x7f1468153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f145800c500 tx=0x7f145800c810 comp rx=0 tx=0).stop 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 shutdown_connections 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1454066730 0x7f1454068bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14680cb1b0 0x7f146813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14680cc420 0x7f14680ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 --2- 172.21.15.179:0/302381811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f146806de60 0x7f1468153cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 >> 172.21.15.179:0/302381811 conn(0x7f14680ce9d0 msgr2=0x7f1468070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 shutdown_connections 2024-05-08T22:33:39.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.415+0000 7f146f0de700 1 -- 172.21.15.179:0/302381811 wait complete. 2024-05-08T22:33:39.432 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T22:33:39.433 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T22:33:39.433 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T22:33:39.433 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T22:33:39.434 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T22:33:39.434 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T22:33:39.435 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T22:33:39.435 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T22:33:39.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 Processor -- start 2024-05-08T22:33:39.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 -- start start 2024-05-08T22:33:39.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 Processor -- start 2024-05-08T22:33:39.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 0x7f444c06c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0b2be0 con 0x7f444c159640 2024-05-08T22:33:39.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c153ad0 con 0x7f444c06bbc0 2024-05-08T22:33:39.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4450d53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c06c580 con 0x7f444c1536f0 2024-05-08T22:33:39.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 -- start start 2024-05-08T22:33:39.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4449d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 0x7f444c06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4449d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 0x7f444c06c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34866/0 (socket says 172.21.15.179:34866) 2024-05-08T22:33:39.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f4449d9b700 1 -- 172.21.15.179:0/923941004 learned_addr learned my addr 172.21.15.179:0/923941004 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7ff9ee9e4700 1 Processor -- start 2024-05-08T22:33:39.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7f444a59c700 1 --2- 172.21.15.179:0/923941004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7ff9ee9e4700 1 -- start start 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7eff8006e180 0x7eff8006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff800cc340 con 0x7eff800b25d0 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8006b540 con 0x7eff8006dda0 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff875dd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800b3110 con 0x7eff8006e180 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444a59c700 1 -- 172.21.15.179:0/923941004 >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 msgr2=0x7f444c06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444a59c700 1 --2- 172.21.15.179:0/923941004 >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 0x7f444c06c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444959a700 1 -- 172.21.15.179:0/923941004 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234795486 0 0) 0x7f443c001110 con 0x7f444c159640 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444a59c700 1 -- 172.21.15.179:0/923941004 >> v1:172.21.15.179:6789/0 conn(0x7f444c159640 legacy=0x7f444c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444a59c700 1 -- 172.21.15.179:0/923941004 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0b1970 con 0x7f444c1536f0 2024-05-08T22:33:39.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.583+0000 7eff84b78700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7eff8006e180 0x7eff8006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7eff8006e180 0x7eff8006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34882/0 (socket says 172.21.15.179:34882) 2024-05-08T22:33:39.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 -- 172.21.15.179:0/2532563956 learned_addr learned my addr 172.21.15.179:0/2532563956 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 Processor -- start 2024-05-08T22:33:39.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 -- start start 2024-05-08T22:33:39.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444a59c700 1 --2- 172.21.15.179:0/923941004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c0b26a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4440009850 tx=0x7f4440009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602c5c077fec1b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 -- 172.21.15.179:0/2532563956 >> v1:172.21.15.195:6789/0 conn(0x7eff8006dda0 legacy=0x7eff800cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 -- 172.21.15.179:0/2532563956 >> v1:172.21.15.179:6789/0 conn(0x7eff800b25d0 legacy=0x7eff800b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 -- 172.21.15.179:0/2532563956 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8006ac50 con 0x7eff8006e180 2024-05-08T22:33:39.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff84b78700 1 --2- 172.21.15.179:0/2532563956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff8006e180 0x7eff8006e5e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff70009910 tx=0x7eff70009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b418a21cacb6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff9e806c2d0 0x7ff9e80d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80d1550 con 0x7ff9e80b8b40 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80d1790 con 0x7ff9e815cff0 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e80d1a70 con 0x7ff9e806c2d0 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444959a700 1 -- 172.21.15.179:0/923941004 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440011830 con 0x7f444c1536f0 2024-05-08T22:33:39.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444959a700 1 -- 172.21.15.179:0/923941004 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440011990 con 0x7f444c1536f0 2024-05-08T22:33:39.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f444959a700 1 -- 172.21.15.179:0/923941004 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440011d40 con 0x7f444c1536f0 2024-05-08T22:33:39.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff77fff700 1 -- 172.21.15.179:0/2532563956 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70004230 con 0x7eff8006e180 2024-05-08T22:33:39.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff77fff700 1 -- 172.21.15.179:0/2532563956 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70004390 con 0x7eff8006e180 2024-05-08T22:33:39.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff77fff700 1 -- 172.21.15.179:0/2532563956 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70009e60 con 0x7eff8006e180 2024-05-08T22:33:39.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ed9e2700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ff9e815cff0 0x7ff9e80d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38288/0 (socket says 172.21.15.179:38288) 2024-05-08T22:33:39.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ed9e2700 1 -- 172.21.15.179:0/757355683 learned_addr learned my addr 172.21.15.179:0/757355683 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ed1e1700 1 --1- 172.21.15.179:0/757355683 >> v1:172.21.15.179:6789/0 conn(0x7ff9e80b8b40 0x7ff9e806abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33452/0 (socket says 172.21.15.179:33452) 2024-05-08T22:33:39.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 Processor -- start 2024-05-08T22:33:39.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 -- start start 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee1e3700 1 --2- 172.21.15.179:0/757355683 >> v2:172.21.15.195:3300/0 conn(0x7ff9e806c2d0 0x7ff9e80d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 Processor -- start 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276880988 0 0) 0x7ff9d8001110 con 0x7ff9e80b8b40 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d40035d0 con 0x7ff9e80b8b40 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665649542 0 0) 0x7ff9e80d1790 con 0x7ff9e815cff0 2024-05-08T22:33:39.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d8001110 con 0x7ff9e815cff0 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/757355683 >> v1:172.21.15.195:6789/0 conn(0x7ff9e815cff0 legacy=0x7ff9e80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/757355683 >> v1:172.21.15.179:6789/0 conn(0x7ff9e80b8b40 legacy=0x7ff9e806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/757355683 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80d1d50 con 0x7ff9e806c2d0 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fec4015f610 0x7fec400cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cf5f0 con 0x7fec401439b0 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400703c0 con 0x7fec4015f610 2024-05-08T22:33:39.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46d24700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400700e0 con 0x7fec400cf1b0 2024-05-08T22:33:39.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 -- start start 2024-05-08T22:33:39.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec45521700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fec4015f610 0x7fec400cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 -- 172.21.15.179:0/2532563956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff8006e180 msgr2=0x7eff8006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 --2- 172.21.15.179:0/2532563956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff8006e180 0x7eff8006e5e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff70009910 tx=0x7eff70009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:39.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee1e3700 1 --2- 172.21.15.179:0/757355683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 0x7ff9e80d2830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009d90 tx=0x7ff9e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5478f2c64f6a409 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 -- 172.21.15.179:0/923941004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 msgr2=0x7f444c0b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 --2- 172.21.15.179:0/923941004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c0b26a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4440009850 tx=0x7f4440009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:39.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53038/0 (socket says 172.21.15.179:53038) 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 -- 172.21.15.179:0/2031168632 learned_addr learned my addr 172.21.15.179:0/2031168632 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 -- 172.21.15.179:0/2532563956 shutdown_connections 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 --2- 172.21.15.179:0/2532563956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff8006e180 0x7eff8006e5e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 -- 172.21.15.179:0/2532563956 >> 172.21.15.179:0/2532563956 conn(0x7eff800ce960 msgr2=0x7eff800ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000e070 con 0x7ff9e806c2d0 2024-05-08T22:33:39.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000bb20 con 0x7ff9e806c2d0 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ec9e0700 1 -- 172.21.15.179:0/757355683 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000be90 con 0x7ff9e806c2d0 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 -- 172.21.15.179:0/923941004 shutdown_connections 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 --2- 172.21.15.179:0/923941004 >> v2:172.21.15.179:3300/0 conn(0x7f444c06bbc0 0x7f444c06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 --2- 172.21.15.179:0/923941004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c0b26a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 -- 172.21.15.179:0/923941004 >> 172.21.15.179:0/923941004 conn(0x7f444c070d30 msgr2=0x7f444c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1cf81616b0 0x7f1cf80ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 0x7f1cf806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf80b4eb0 con 0x7f1cf80ce0c0 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80aeca0 con 0x7f1cf81616b0 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cfce6f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80b42a0 con 0x7f1cf80cec30 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 -- 172.21.15.179:0/2031168632 >> v2:172.21.15.195:3300/0 conn(0x7fec4015f610 msgr2=0x7fec400cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 --2- 172.21.15.179:0/2031168632 >> v2:172.21.15.195:3300/0 conn(0x7fec4015f610 0x7fec400cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 -- 172.21.15.179:0/2031168632 >> v1:172.21.15.195:6789/0 conn(0x7fec401439b0 legacy=0x7fec4006f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 -- 172.21.15.179:0/2031168632 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4015eb30 con 0x7fec400cf1b0 2024-05-08T22:33:39.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 -- 172.21.15.179:0/923941004 shutdown_connections 2024-05-08T22:33:39.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1cf81616b0 0x7f1cf80ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1cf81616b0 0x7f1cf80ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34892/0 (socket says 172.21.15.179:34892) 2024-05-08T22:33:39.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 -- 172.21.15.179:0/3653196985 learned_addr learned my addr 172.21.15.179:0/3653196985 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f4450d53700 1 -- 172.21.15.179:0/923941004 wait complete. 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf940481e0 0x7fcf940eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/3653196985 >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 0x7f1cf806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf94052f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9404fdf0 con 0x7fcf94033f80 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9404fb50 con 0x7fcf940481e0 2024-05-08T22:33:39.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcfa239b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94050280 con 0x7fcf94047c90 2024-05-08T22:33:39.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fec46523700 1 --2- 172.21.15.179:0/2031168632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009910 tx=0x7fec3c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abc7521ea6dd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/757355683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 msgr2=0x7ff9e80d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 --2- 172.21.15.179:0/757355683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 0x7ff9e80d2830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009d90 tx=0x7ff9e000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:39.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 -- 172.21.15.179:0/3653196985 >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 msgr2=0x7f1cf806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 --2- 172.21.15.179:0/3653196985 >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 0x7f1cf806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 -- 172.21.15.179:0/3653196985 >> v1:172.21.15.195:6789/0 conn(0x7f1cf80ce0c0 legacy=0x7f1cf80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 -- 172.21.15.179:0/3653196985 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80b4580 con 0x7f1cf81616b0 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/3653196985 >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 0x7f1cf806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/757355683 shutdown_connections 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 --2- 172.21.15.179:0/757355683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 0x7ff9e80d2830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/757355683 >> 172.21.15.179:0/757355683 conn(0x7ff9e80d3410 msgr2=0x7ff9e80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcf9bfff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fcf94033f80 0x7fcf940eb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38314/0 (socket says 172.21.15.179:38314) 2024-05-08T22:33:39.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcf9bfff700 1 -- 172.21.15.179:0/3249383503 learned_addr learned my addr 172.21.15.179:0/3249383503 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7fcf9b7fe700 1 --2- 172.21.15.179:0/3249383503 >> v2:172.21.15.195:3300/0 conn(0x7fcf940481e0 0x7fcf940eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa0938700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf94052f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7f1cf659c700 1 --2- 172.21.15.179:0/3653196985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf81616b0 0x7f1cf80ae760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8070ae0 tx=0x7f1cf0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514242f9656e6292 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cf559a700 1 -- 172.21.15.179:0/3653196985 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0004230 con 0x7f1cf81616b0 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cf559a700 1 -- 172.21.15.179:0/3653196985 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0004390 con 0x7f1cf81616b0 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9b7fe700 1 -- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 msgr2=0x7fcf94052f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9b7fe700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf94052f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9b7fe700 1 -- 172.21.15.179:0/3249383503 >> v1:172.21.15.195:6789/0 conn(0x7fcf94033f80 legacy=0x7fcf940eb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9affd700 1 -- 172.21.15.179:0/3249383503 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661036560 0 0) 0x7fcf9404fdf0 con 0x7fcf94033f80 2024-05-08T22:33:39.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9b7fe700 1 -- 172.21.15.179:0/3249383503 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940ebe40 con 0x7fcf940481e0 2024-05-08T22:33:39.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cf559a700 1 -- 172.21.15.179:0/3653196985 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00036c0 con 0x7f1cf81616b0 2024-05-08T22:33:39.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9b7fe700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf940eb900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009850 tx=0x7fcf90009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0212f19c4e721c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 Processor -- start 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 -- 172.21.15.179:0/2532563956 shutdown_connections 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.587+0000 7eff875dd700 1 -- 172.21.15.179:0/2532563956 wait complete. 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 -- start start 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9affd700 1 -- 172.21.15.179:0/3249383503 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90011830 con 0x7fcf940481e0 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9affd700 1 -- 172.21.15.179:0/3249383503 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf90011990 con 0x7fcf940481e0 2024-05-08T22:33:39.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcf9affd700 1 -- 172.21.15.179:0/3249383503 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90011d40 con 0x7fcf940481e0 2024-05-08T22:33:39.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 Processor -- start 2024-05-08T22:33:39.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 -- start start 2024-05-08T22:33:39.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 -- 172.21.15.179:0/3653196985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf81616b0 msgr2=0x7f1cf80ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/3653196985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf81616b0 0x7f1cf80ae760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8070ae0 tx=0x7f1cf0009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 -- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 msgr2=0x7fcf940eb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf940eb900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009850 tx=0x7fcf90009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:39.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 -- 172.21.15.179:0/3653196985 shutdown_connections 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8006dda0 0x7eff801fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8006e180 0x7eff800ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 0x7eff800aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015c510 con 0x7eff800b25d0 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800acc00 con 0x7eff8006e180 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff800acee0 con 0x7eff8006dda0 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec44d20700 1 -- 172.21.15.179:0/2031168632 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011830 con 0x7fec400cf1b0 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec44d20700 1 -- 172.21.15.179:0/2031168632 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c011990 con 0x7fec400cf1b0 2024-05-08T22:33:39.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec44d20700 1 -- 172.21.15.179:0/2031168632 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011d40 con 0x7fec400cf1b0 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/757355683 shutdown_connections 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 0x7f444c1f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f444c159640 0x7f444c0a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0af470 con 0x7f444c06bbc0 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0a85e0 con 0x7f444c159640 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4450d53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1f7b00 con 0x7f444c1536f0 2024-05-08T22:33:39.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 0x7f444c1f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/3653196985 >> v2:172.21.15.195:3300/0 conn(0x7f1cf80cec30 0x7f1cf806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 0x7f444c1f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34916/0 (socket says 172.21.15.179:34916) 2024-05-08T22:33:39.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 -- 172.21.15.179:0/3088399269 learned_addr learned my addr 172.21.15.179:0/3088399269 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85379700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8006dda0 0x7eff801fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 0x7eff800aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85379700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8006dda0 0x7eff801fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53066/0 (socket says 172.21.15.179:53066) 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 0x7eff800aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34900/0 (socket says 172.21.15.179:34900) 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85379700 1 -- 172.21.15.179:0/3524556278 learned_addr learned my addr 172.21.15.179:0/3524556278 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff84b78700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8006e180 0x7eff800ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f4449d9b700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/3653196985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf81616b0 0x7f1cf80ae760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444ad9d700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f444c159640 0x7f444c0a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 -- 172.21.15.179:0/3653196985 >> 172.21.15.179:0/3653196985 conn(0x7f1cf8098a70 msgr2=0x7f1cf80b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 -- 172.21.15.179:0/3653196985 shutdown_connections 2024-05-08T22:33:39.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f1cfce6f700 1 -- 172.21.15.179:0/3653196985 wait complete. 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 -- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8006dda0 msgr2=0x7eff801fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8006dda0 0x7eff801fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 -- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8006e180 msgr2=0x7eff800ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8006e180 0x7eff800ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009570 con 0x7eff800b25d0 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 -- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 msgr2=0x7f444c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f444c1536f0 0x7f444c1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 -- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f444c159640 msgr2=0x7f444c0a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f444c159640 0x7f444c0a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400094b0 con 0x7f444c06bbc0 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 -- 172.21.15.179:0/3249383503 shutdown_connections 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf94052f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 --2- 172.21.15.179:0/3249383503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf940eb900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fcfa239b700 1 -- 172.21.15.179:0/3249383503 >> 172.21.15.179:0/3249383503 conn(0x7fcf94054b40 msgr2=0x7fcf94054f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 -- 172.21.15.179:0/2031168632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 msgr2=0x7fec4006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 --2- 172.21.15.179:0/2031168632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009910 tx=0x7fec3c009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff85b7a700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 0x7eff800aa340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff7800be00 tx=0x7eff78009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7f444a59c700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 0x7f444c1f75c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44400090b0 tx=0x7f4440004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 -- 172.21.15.179:0/2031168632 shutdown_connections 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 --2- 172.21.15.179:0/2031168632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec4006b440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 --2- 172.21.15.179:0/2031168632 >> v2:172.21.15.195:3300/0 conn(0x7fec4015f610 0x7fec400cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7fec46d24700 1 -- 172.21.15.179:0/2031168632 >> 172.21.15.179:0/2031168632 conn(0x7fec400cfb50 msgr2=0x7fec400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800cb90 con 0x7eff800b25d0 2024-05-08T22:33:39.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800ccf0 con 0x7eff800b25d0 2024-05-08T22:33:39.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78014680 con 0x7eff800b25d0 2024-05-08T22:33:39.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fec46d24700 1 -- 172.21.15.179:0/2031168632 shutdown_connections 2024-05-08T22:33:39.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fec46d24700 1 -- 172.21.15.179:0/2031168632 wait complete. 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 Processor -- start 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 -- start start 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf80ce0c0 0x7f1cf81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fcfa239b700 1 -- 172.21.15.179:0/3249383503 shutdown_connections 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fcfa239b700 1 -- 172.21.15.179:0/3249383503 wait complete. 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff800ad1c0 con 0x7eff800b25d0 2024-05-08T22:33:39.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff800ad780 con 0x7eff800b25d0 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.591+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/757355683 wait complete. 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 Processor -- start 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 -- start start 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444001a990 con 0x7f444c06bbc0 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444001f070 con 0x7f444c06bbc0 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444001ad30 con 0x7f444c06bbc0 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c0a8a70 con 0x7f444c06bbc0 2024-05-08T22:33:39.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c0a7cb0 con 0x7f444c06bbc0 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf80cec30 0x7f1cf815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf81616b0 0x7f1cf80cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80ca2c0 con 0x7f1cf80ce0c0 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf806c0a0 con 0x7f1cf81616b0 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cfce6f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80ca510 con 0x7f1cf80cec30 2024-05-08T22:33:39.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff7801c360 con 0x7eff800b25d0 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff8006eb20 con 0x7eff800b25d0 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7eff64066760 0x7eff64068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 0x7ff9e8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e80b8b40 0x7ff9e8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 0x7ff9e8153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e814e760 con 0x7ff9e815cff0 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8148aa0 con 0x7ff9e806c2d0 2024-05-08T22:33:39.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee9e4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80c93e0 con 0x7ff9e80b8b40 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf80ce0c0 0x7f1cf81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf80ce0c0 0x7f1cf81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34922/0 (socket says 172.21.15.179:34922) 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf659c700 1 -- 172.21.15.179:0/1344395643 learned_addr learned my addr 172.21.15.179:0/1344395643 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf81616b0 0x7f1cf80cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f25ab00e700 1 Processor -- start 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff78083ea0 con 0x7eff800b25d0 2024-05-08T22:33:39.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7eff6406fda0 0x7eff640721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff640728b0 con 0x7eff6406fda0 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff78089430 con 0x7eff800b25d0 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff85379700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7eff64066760 0x7eff64068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fec46d24700 1 Processor -- start 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf5d9b700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf80cec30 0x7f1cf815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 -- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf80cec30 msgr2=0x7f1cf815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf80cec30 0x7f1cf815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 -- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf80ce0c0 msgr2=0x7f1cf81527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1cf80ce0c0 0x7f1cf81527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009570 con 0x7f1cf81616b0 2024-05-08T22:33:39.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fcfa239b700 1 Processor -- start 2024-05-08T22:33:39.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7eff84b78700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7eff6406fda0 0x7eff640721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f1cf6d9d700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf81616b0 0x7f1cf80cbfa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800cdf0 tx=0x7f1ce800ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fec46d24700 1 -- start start 2024-05-08T22:33:39.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7fcfa239b700 1 -- start start 2024-05-08T22:33:39.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ed1e1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e80b8b40 0x7ff9e8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 0x7ff9e8153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 0x7ff9e8153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34924/0 (socket says 172.21.15.179:34924) 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/2772358317 learned_addr learned my addr 172.21.15.179:0/2772358317 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e80b8b40 msgr2=0x7ff9e8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e80b8b40 0x7ff9e8153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 msgr2=0x7ff9e8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e806c2d0 0x7ff9e8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009a70 con 0x7ff9e815cff0 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f444001b430 con 0x7f444c06bbc0 2024-05-08T22:33:39.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800ad40 con 0x7f1cf81616b0 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9ee1e3700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 0x7ff9e8153b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000b370 tx=0x7ff9e0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.595+0000 7f25ab00e700 1 -- start start 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25ab00e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25a41536f0 0x7f25a40b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25ab00e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25a4159d50 0x7f25a406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25ab00e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40b2be0 con 0x7f25a406bbc0 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25ab00e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a406c5b0 con 0x7f25a4159d50 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25ab00e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a4153ad0 con 0x7f25a41536f0 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25a980b700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f25a406bbc0 0x7f25a4159640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38326/0 (socket says 172.21.15.179:38326) 2024-05-08T22:33:39.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25a980b700 1 -- 172.21.15.179:0/3116354150 learned_addr learned my addr 172.21.15.179:0/3116354150 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa80d700 1 --2- 172.21.15.179:0/3116354150 >> v2:172.21.15.179:3300/0 conn(0x7f25a4159d50 0x7f25a406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce8015440 con 0x7f1cf81616b0 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80ca7f0 con 0x7f1cf81616b0 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf8153430 con 0x7f1cf81616b0 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 0x7fcf940d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf9404d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf9404e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9404c080 con 0x7fcf94033f80 2024-05-08T22:33:39.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9404c360 con 0x7fcf940481e0 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9404c640 con 0x7fcf94047c90 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7eff85379700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7eff64066760 0x7eff64068c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff7c001940 tx=0x7eff7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7eff84b78700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7eff6406fda0 0x7eff640721e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0003c40 con 0x7ff9e815cff0 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e814e110 con 0x7ff9e815cff0 2024-05-08T22:33:39.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e80c96c0 con 0x7ff9e815cff0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f444c0b1ea0 con 0x7f444c06bbc0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec40134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec401439b0 0x7fec40142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 0x7fec40128b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4012fae0 con 0x7fec4015f610 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4012fd30 con 0x7fec401439b0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46d24700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec40130010 con 0x7fec400cf1b0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46523700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 0x7fec40128b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec40134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46523700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 0x7fec40128b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34968/0 (socket says 172.21.15.179:34968) 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46523700 1 -- 172.21.15.179:0/651001794 learned_addr learned my addr 172.21.15.179:0/651001794 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45521700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec401439b0 0x7fec40142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3116354150 >> v2:172.21.15.195:3300/0 conn(0x7f25a41536f0 0x7f25a40b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 -- 172.21.15.179:0/3116354150 >> v2:172.21.15.179:3300/0 conn(0x7f25a4159d50 msgr2=0x7f25a406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3116354150 >> v2:172.21.15.179:3300/0 conn(0x7f25a4159d50 0x7f25a406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25a900a700 1 -- 172.21.15.179:0/3116354150 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407726107 0 0) 0x7f25940049b0 con 0x7f25a406bbc0 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 -- 172.21.15.179:0/3116354150 >> v1:172.21.15.195:6789/0 conn(0x7f25a406bbc0 legacy=0x7f25a4159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 -- 172.21.15.179:0/3116354150 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40b1970 con 0x7f25a41536f0 2024-05-08T22:33:39.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3116354150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40b26a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f25a40b14c0 tx=0x7f25a000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78305e627e497380 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800aea0 con 0x7f1cf81616b0 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa0938700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf9404e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 0x7fcf940d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 0x7fcf940d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34954/0 (socket says 172.21.15.179:34954) 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 -- 172.21.15.179:0/875920303 learned_addr learned my addr 172.21.15.179:0/875920303 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 -- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 msgr2=0x7fcf9404d120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf94047c90 0x7fcf9404d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 -- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 msgr2=0x7fcf9404e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf940481e0 0x7fcf9404e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900094b0 con 0x7fcf94033f80 2024-05-08T22:33:39.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf9bfff700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 0x7fcf940d5ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c02ce60 tx=0x7fcf9c04e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e001f070 con 0x7ff9e815cff0 2024-05-08T22:33:39.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00098f0 con 0x7ff9e815cff0 2024-05-08T22:33:39.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1cdc000fc0 con 0x7f1cf81616b0 2024-05-08T22:33:39.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f443affd700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4434066880 0x7f4434068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7eff640728b0 con 0x7eff6406fda0 2024-05-08T22:33:39.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c066be0 con 0x7fcf94033f80 2024-05-08T22:33:39.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f4449d9b700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4434066880 0x7f4434068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 -- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec401439b0 msgr2=0x7fec40142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec401439b0 0x7fec40142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 -- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 msgr2=0x7fec40128b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 0x7fec40128b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009570 con 0x7fec400cf1b0 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45521700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec401439b0 0x7fec40142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec46523700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec4015f610 0x7fec40128b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fec45d22700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec40134930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c480 tx=0x7fec3000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ce80157f0 con 0x7f1cf81616b0 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c066d80 con 0x7fcf94033f80 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c066150 con 0x7fcf94033f80 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9404c920 con 0x7fcf94033f80 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf940d6400 con 0x7fcf94033f80 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf9404c4d0 con 0x7fcf94033f80 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9d0000fc0 con 0x7ff9e815cff0 2024-05-08T22:33:39.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9e0004030 con 0x7ff9e815cff0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f44400852f0 con 0x7f444c06bbc0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f443affd700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f443406fcd0 0x7f4434072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44340727e0 con 0x7f443406fcd0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f444ad9d700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f443406fcd0 0x7f4434072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f4449d9b700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4434066880 0x7f4434068d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4444008fd0 tx=0x7f444400c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f444ad9d700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f443406fcd0 0x7f4434072110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f64729ab700 1 Processor -- start 2024-05-08T22:33:39.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f64729ab700 1 -- start start 2024-05-08T22:33:39.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1ceeffd700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1cd8066780 0x7f1cd8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f444001ae90 con 0x7f444c06bbc0 2024-05-08T22:33:39.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9cc066730 0x7ff9cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ce8084fe0 con 0x7f1cf81616b0 2024-05-08T22:33:39.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff9e0085c00 con 0x7ff9e815cff0 2024-05-08T22:33:39.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1cf659c700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1cd8066780 0x7f1cd8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1ceeffd700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7f1cd806fdc0 0x7f1cd8072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9ed9e2700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9cc066730 0x7ff9cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cd80728d0 con 0x7f1cd806fdc0 2024-05-08T22:33:39.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ce8085480 con 0x7f1cf81616b0 2024-05-08T22:33:39.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7ff9cc06fd70 0x7ff9cc0721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9cc072880 con 0x7ff9cc06fd70 2024-05-08T22:33:39.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9e008a240 con 0x7ff9e815cff0 2024-05-08T22:33:39.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300052d0 con 0x7fec400cf1b0 2024-05-08T22:33:39.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.599+0000 7f25a900a700 1 -- 172.21.15.179:0/3116354150 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000dc70 con 0x7f25a41536f0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30005430 con 0x7fec400cf1b0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec401302f0 con 0x7fec400cf1b0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300055a0 con 0x7fec400cf1b0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec40134e70 con 0x7fec400cf1b0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9ed1e1700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7ff9cc06fd70 0x7ff9cc0721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf9c0662b0 con 0x7fcf94033f80 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1cf5d9b700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7f1cd806fdc0 0x7f1cd8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1cf659c700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1cd8066780 0x7f1cd8068c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009150 tx=0x7f1cf0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf80066730 0x7fcf80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcf9c0fdc70 con 0x7fcf94033f80 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7fcf8006fcf0 0x7fcf80072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf80072800 con 0x7fcf8006fcf0 2024-05-08T22:33:39.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf9c100330 con 0x7fcf94033f80 2024-05-08T22:33:39.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9ed1e1700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7ff9cc06fd70 0x7ff9cc0721b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9ed9e2700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9cc066730 0x7ff9cc068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4001630 tx=0x7ff9e40079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f1cf5d9b700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7f1cd806fdc0 0x7f1cd8072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64729ab700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f646c0cc740 0x7f646c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64729ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f646c160cd0 0x7f646c161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64729ab700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c143ac0 con 0x7f646c1436e0 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64729ab700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c06c760 con 0x7f646c160cd0 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64729ab700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c1616a0 con 0x7f646c0cc740 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f646c160cd0 0x7f646c161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64711a8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f646c0cc740 0x7f646c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f646c160cd0 0x7f646c161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34970/0 (socket says 172.21.15.179:34970) 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 -- 172.21.15.179:0/3720980472 learned_addr learned my addr 172.21.15.179:0/3720980472 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcfa0938700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7fcf8006fcf0 0x7fcf80072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf9b7fe700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf80066730 0x7fcf80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcfa0938700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7fcf8006fcf0 0x7fcf80072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f25a900a700 1 -- 172.21.15.179:0/3116354150 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000ddd0 con 0x7f25a41536f0 2024-05-08T22:33:39.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f25a900a700 1 -- 172.21.15.179:0/3116354150 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0010670 con 0x7f25a41536f0 2024-05-08T22:33:39.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f44340727e0 con 0x7f443406fcd0 2024-05-08T22:33:39.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec4006fd50 con 0x7fec400cf1b0 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec300038e0 con 0x7fec400cf1b0 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 -- 172.21.15.179:0/3720980472 >> v2:172.21.15.195:3300/0 conn(0x7f646c0cc740 msgr2=0x7f646c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 --2- 172.21.15.179:0/3720980472 >> v2:172.21.15.195:3300/0 conn(0x7f646c0cc740 0x7f646c06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 -- 172.21.15.179:0/3720980472 >> v1:172.21.15.195:6789/0 conn(0x7f646c1436e0 legacy=0x7f646c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 -- 172.21.15.179:0/3720980472 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0ccd00 con 0x7f646c160cd0 2024-05-08T22:33:39.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff9cc072880 con 0x7ff9cc06fd70 2024-05-08T22:33:39.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64721aa700 1 --2- 172.21.15.179:0/3720980472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c160cd0 0x7f646c161160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f646c0b3ba0 tx=0x7f646400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9384f6d5b235b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64709a7700 1 -- 172.21.15.179:0/3720980472 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400e070 con 0x7f646c160cd0 2024-05-08T22:33:39.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f64709a7700 1 -- 172.21.15.179:0/3720980472 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646400b910 con 0x7f646c160cd0 2024-05-08T22:33:39.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fcf80072800 con 0x7fcf8006fcf0 2024-05-08T22:33:39.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f25ab00e700 1 -- 172.21.15.179:0/3116354150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 msgr2=0x7f25a40b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3116354150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40b26a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f25a40b14c0 tx=0x7f25a000bdd0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fcf9b7fe700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf80066730 0x7fcf80068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009850 tx=0x7fcf90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- 172.21.15.179:0/3116354150 shutdown_connections 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3116354150 >> v2:172.21.15.179:3300/0 conn(0x7f25a4159d50 0x7f25a406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3116354150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40b26a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- 172.21.15.179:0/3116354150 >> 172.21.15.179:0/3116354150 conn(0x7f25a4070d30 msgr2=0x7f25a4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1cd80728d0 con 0x7f1cd806fdc0 2024-05-08T22:33:39.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- 172.21.15.179:0/3116354150 shutdown_connections 2024-05-08T22:33:39.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- 172.21.15.179:0/3116354150 wait complete. 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec24066800 0x7fec24068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fec300837c0 con 0x7fec400cf1b0 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec45521700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec24066800 0x7fec24068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.603+0000 7fec367fc700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fec2406fe40 0x7fec24072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec24072950 con 0x7fec2406fe40 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec30085de0 con 0x7fec400cf1b0 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fec46523700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fec2406fe40 0x7fec24072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fec46523700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fec2406fe40 0x7fec24072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 Processor -- start 2024-05-08T22:33:39.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- start start 2024-05-08T22:33:39.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fec24072950 con 0x7fec2406fe40 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25a406bbc0 0x7f25a4153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 0x7f25a41f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40a74d0 con 0x7f25a4159d50 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41f79a0 con 0x7f25a41536f0 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25ab00e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41f7c80 con 0x7f25a406bbc0 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25aa80d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 0x7f25a41f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25aa80d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 0x7f25a41f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34986/0 (socket says 172.21.15.179:34986) 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25aa80d700 1 -- 172.21.15.179:0/3258409248 learned_addr learned my addr 172.21.15.179:0/3258409248 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25a980b700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25a406bbc0 0x7f25a4153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7fec45521700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec24066800 0x7fec24068cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec40067370 tx=0x7fec3800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 -- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25a406bbc0 msgr2=0x7f25a4153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25a406bbc0 0x7f25a4153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 -- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 msgr2=0x7f25a40a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25a41536f0 0x7f25a40a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a000b750 con 0x7f25a4159d50 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f25aa80d700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 0x7f25a41f6e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d80 tx=0x7f259c00e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64709a7700 1 -- 172.21.15.179:0/3720980472 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400e070 con 0x7f646c160cd0 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 -- 172.21.15.179:0/3720980472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c160cd0 msgr2=0x7f646c161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 --2- 172.21.15.179:0/3720980472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c160cd0 0x7f646c161160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f646c0b3ba0 tx=0x7f646400b330 comp rx=0 tx=0).stop 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 -- 172.21.15.179:0/3720980472 shutdown_connections 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 --2- 172.21.15.179:0/3720980472 >> v2:172.21.15.195:3300/0 conn(0x7f646c0cc740 0x7f646c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 --2- 172.21.15.179:0/3720980472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c160cd0 0x7f646c161160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 -- 172.21.15.179:0/3720980472 >> 172.21.15.179:0/3720980472 conn(0x7f646c0ceb70 msgr2=0x7f646c0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 -- 172.21.15.179:0/3720980472 shutdown_connections 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.607+0000 7f64729ab700 1 -- 172.21.15.179:0/3720980472 wait complete. 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 Processor -- start 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- start start 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f646c0cc740 0x7f646c13c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 0x7f646c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646c160cd0 0x7f646c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c15c0e0 con 0x7f646c1436e0 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c13c6d0 con 0x7f646c160cd0 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c155220 con 0x7f646c0cc740 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 0x7f646c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 0x7f646c15f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34994/0 (socket says 172.21.15.179:34994) 2024-05-08T22:33:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 -- 172.21.15.179:0/3016281507 learned_addr learned my addr 172.21.15.179:0/3016281507 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64721aa700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646c160cd0 0x7f646c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64719a9700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f646c0cc740 0x7f646c13c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 -- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f646c0cc740 msgr2=0x7f646c13c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f646c0cc740 0x7f646c13c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 -- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646c160cd0 msgr2=0x7f646c158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646c160cd0 0x7f646c158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 -- 172.21.15.179:0/3016281507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464009b30 con 0x7f646c1436e0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64721aa700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646c160cd0 0x7f646c158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64711a8700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 0x7f646c15f990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f645c00e9d0 tx=0x7f645c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c009720 con 0x7f646c1436e0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c00c850 con 0x7f646c1436e0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c155500 con 0x7f646c1436e0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00cbc0 con 0x7f646c1436e0 2024-05-08T22:33:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.611+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c155ac0 con 0x7f646c1436e0 2024-05-08T22:33:39.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c009c20 con 0x7f25a4159d50 2024-05-08T22:33:39.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c00edd0 con 0x7f25a4159d50 2024-05-08T22:33:39.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0136a0 con 0x7f25a4159d50 2024-05-08T22:33:39.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f645c00cd20 con 0x7f646c1436e0 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f64627fc700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64580668c0 0x7f6458068d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f645c0844f0 con 0x7f646c1436e0 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a40a7720 con 0x7f25a4159d50 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a4156790 con 0x7f25a4159d50 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25a40b1ea0 con 0x7f25a4159d50 2024-05-08T22:33:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f64719a9700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64580668c0 0x7f6458068d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f64719a9700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64580668c0 0x7f6458068d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f64680018a0 tx=0x7f6468009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f259c013800 con 0x7f25a4159d50 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.615+0000 7f259affd700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2590066870 0x7f2590068d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f259c084f70 con 0x7f25a4159d50 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f259affd700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7f259006fd40 0x7f2590072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2590066870 0x7f2590068d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2590072850 con 0x7f259006fd40 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259c087620 con 0x7f25a4159d50 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f25a980b700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7f259006fd40 0x7f2590072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f25a980b700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7f259006fd40 0x7f2590072180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f25aa00c700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2590066870 0x7f2590068d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b2c0 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.619+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2590072850 con 0x7f259006fd40 2024-05-08T22:33:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7f64729ab700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f646c158890 0x7f646c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f646c06bd00 con 0x7f646c158890 2024-05-08T22:33:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7f64721aa700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f646c158890 0x7f646c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7f64721aa700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f646c158890 0x7f646c0c3df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff800ad050 con 0x7eff6406fda0 2024-05-08T22:33:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f64640094d0 con 0x7f646c158890 2024-05-08T22:33:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.627+0000 7eff75ffb700 1 -- 172.21.15.179:0/3524556278 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7eff800ad050 con 0x7eff6406fda0 2024-05-08T22:33:39.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9d0002d60 con 0x7ff9cc06fd70 2024-05-08T22:33:39.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1cdc002d60 con 0x7f1cd806fdc0 2024-05-08T22:33:39.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7eff6406fda0 msgr2=0x7eff640721e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7eff6406fda0 0x7eff640721e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7eff64066760 msgr2=0x7eff64068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7eff64066760 0x7eff64068c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff7c001940 tx=0x7eff7c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 msgr2=0x7eff800aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 --2- 172.21.15.179:0/3524556278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff800b25d0 0x7eff800aa340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff7800be00 tx=0x7eff78009ed0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f444c0b1bc0 con 0x7f443406fcd0 2024-05-08T22:33:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff85b7a700 1 -- 172.21.15.179:0/3524556278 reap_dead start 2024-05-08T22:33:39.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7f1ceeffd700 1 -- 172.21.15.179:0/1344395643 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1cdc002d60 con 0x7f1cd806fdc0 2024-05-08T22:33:39.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 shutdown_connections 2024-05-08T22:33:39.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 >> 172.21.15.179:0/3524556278 conn(0x7eff800ce960 msgr2=0x7eff8006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 shutdown_connections 2024-05-08T22:33:39.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7eff875dd700 1 -- 172.21.15.179:0/3524556278 wait complete. 2024-05-08T22:33:39.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7ff9de7fc700 1 -- 172.21.15.179:0/2772358317 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9d0002d60 con 0x7ff9cc06fd70 2024-05-08T22:33:39.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.631+0000 7f443affd700 1 -- 172.21.15.179:0/3088399269 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f443c001480 con 0x7f443406fcd0 2024-05-08T22:33:39.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f443406fcd0 msgr2=0x7f4434072110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f443406fcd0 0x7f4434072110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4434066880 msgr2=0x7f4434068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4434066880 0x7f4434068d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4444008fd0 tx=0x7f444400c040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 msgr2=0x7f444c1f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 --2- 172.21.15.179:0/3088399269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444c06bbc0 0x7f444c1f75c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44400090b0 tx=0x7f4440004770 comp rx=0 tx=0).stop 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f444ad9d700 1 -- 172.21.15.179:0/3088399269 reap_dead start 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 shutdown_connections 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 >> 172.21.15.179:0/3088399269 conn(0x7f444c070d30 msgr2=0x7f444c06b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 shutdown_connections 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f4450d53700 1 -- 172.21.15.179:0/3088399269 wait complete. 2024-05-08T22:33:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7f1cd806fdc0 msgr2=0x7f1cd8072200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7f1cd806fdc0 0x7f1cd8072200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1cd8066780 msgr2=0x7f1cd8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1cd8066780 0x7f1cd8068c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009150 tx=0x7f1cf0008030 comp rx=0 tx=0).stop 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf81616b0 msgr2=0x7f1cf80cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 --2- 172.21.15.179:0/1344395643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf81616b0 0x7f1cf80cbfa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800cdf0 tx=0x7f1ce800ce20 comp rx=0 tx=0).stop 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7ff9cc06fd70 msgr2=0x7ff9cc0721b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7ff9ee9e4700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7ff9cc06fd70 0x7ff9cc0721b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cf6d9d700 1 -- 172.21.15.179:0/1344395643 reap_dead start 2024-05-08T22:33:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 shutdown_connections 2024-05-08T22:33:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.635+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 >> 172.21.15.179:0/1344395643 conn(0x7f1cf8098a70 msgr2=0x7f1cf80b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9cc066730 msgr2=0x7ff9cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9cc066730 0x7ff9cc068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4001630 tx=0x7ff9e40079e0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 msgr2=0x7ff9e8153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 --2- 172.21.15.179:0/2772358317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e815cff0 0x7ff9e8153b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000b370 tx=0x7ff9e0004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee1e3700 1 -- 172.21.15.179:0/2772358317 reap_dead start 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 shutdown_connections 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 >> 172.21.15.179:0/2772358317 conn(0x7ff9e80d3410 msgr2=0x7ff9e80b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 shutdown_connections 2024-05-08T22:33:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 shutdown_connections 2024-05-08T22:33:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7f1cfce6f700 1 -- 172.21.15.179:0/1344395643 wait complete. 2024-05-08T22:33:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7ff9ee9e4700 1 -- 172.21.15.179:0/2772358317 wait complete. 2024-05-08T22:33:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25a40b1bc0 con 0x7f259006fd40 2024-05-08T22:33:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7f259affd700 1 -- 172.21.15.179:0/3258409248 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2594001910 con 0x7f259006fd40 2024-05-08T22:33:39.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.639+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf9404f510 con 0x7fcf8006fcf0 2024-05-08T22:33:39.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7f259006fd40 msgr2=0x7f2590072180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7f259006fd40 0x7f2590072180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2590066870 msgr2=0x7f2590068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2590066870 0x7f2590068d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b2c0 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 msgr2=0x7f25a41f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 --2- 172.21.15.179:0/3258409248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a4159d50 0x7f25a41f6e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d80 tx=0x7f259c00e4c0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25aa80d700 1 -- 172.21.15.179:0/3258409248 reap_dead start 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 shutdown_connections 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 >> 172.21.15.179:0/3258409248 conn(0x7f25a4070d30 msgr2=0x7f25a40afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcf98ff9700 1 -- 172.21.15.179:0/875920303 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf88009ad0 con 0x7fcf8006fcf0 2024-05-08T22:33:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 shutdown_connections 2024-05-08T22:33:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7f25ab00e700 1 -- 172.21.15.179:0/3258409248 wait complete. 2024-05-08T22:33:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7fcf8006fcf0 msgr2=0x7fcf80072130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7fcf8006fcf0 0x7fcf80072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf80066730 msgr2=0x7fcf80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf80066730 0x7fcf80068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009850 tx=0x7fcf90008030 comp rx=0 tx=0).stop 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 msgr2=0x7fcf940d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 --2- 172.21.15.179:0/875920303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf94033f80 0x7fcf940d5ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c02ce60 tx=0x7fcf9c04e990 comp rx=0 tx=0).stop 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa0938700 1 -- 172.21.15.179:0/875920303 reap_dead start 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 shutdown_connections 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 >> 172.21.15.179:0/875920303 conn(0x7fcf94054b40 msgr2=0x7fcf94007320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 shutdown_connections 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.643+0000 7fcfa239b700 1 -- 172.21.15.179:0/875920303 wait complete. 2024-05-08T22:33:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec4015f0f0 con 0x7fec2406fe40 2024-05-08T22:33:39.679 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:39.680 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T22:33:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec367fc700 1 -- 172.21.15.179:0/651001794 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec3c006d40 con 0x7fec2406fe40 2024-05-08T22:33:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fec2406fe40 msgr2=0x7fec24072280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fec2406fe40 0x7fec24072280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.681 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:39.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T22:33:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec24066800 msgr2=0x7fec24068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec24066800 0x7fec24068cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec40067370 tx=0x7fec3800b040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 msgr2=0x7fec40134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 --2- 172.21.15.179:0/651001794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec400cf1b0 0x7fec40134930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c480 tx=0x7fec3000c790 comp rx=0 tx=0).stop 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46523700 1 -- 172.21.15.179:0/651001794 reap_dead start 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.647+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 shutdown_connections 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.651+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 >> 172.21.15.179:0/651001794 conn(0x7fec400cfb50 msgr2=0x7fec400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.651+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 shutdown_connections 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.651+0000 7fec46d24700 1 -- 172.21.15.179:0/651001794 wait complete. 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f646c06ba20 con 0x7f646c158890 2024-05-08T22:33:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64627fc700 1 -- 172.21.15.179:0/3016281507 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6464003a40 con 0x7f646c158890 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f646c158890 msgr2=0x7f646c0c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f646c158890 0x7f646c0c3df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64580668c0 msgr2=0x7f6458068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64580668c0 0x7f6458068d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f64680018a0 tx=0x7f6468009040 comp rx=0 tx=0).stop 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 msgr2=0x7f646c15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 --2- 172.21.15.179:0/3016281507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f646c1436e0 0x7f646c15f990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f645c00e9d0 tx=0x7f645c00ece0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64721aa700 1 -- 172.21.15.179:0/3016281507 reap_dead start 2024-05-08T22:33:39.685 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:39.686 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T22:33:39.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 shutdown_connections 2024-05-08T22:33:39.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 >> 172.21.15.179:0/3016281507 conn(0x7f646c0ceb70 msgr2=0x7f646c15e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 shutdown_connections 2024-05-08T22:33:39.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.655+0000 7f64729ab700 1 -- 172.21.15.179:0/3016281507 wait complete. 2024-05-08T22:33:39.687 INFO:teuthology.orchestra.run.smithi179.stdout:60129542153 2024-05-08T22:33:39.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T22:33:39.688 INFO:teuthology.orchestra.run.smithi179.stdout:64424509450 2024-05-08T22:33:39.688 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T22:33:39.689 INFO:teuthology.orchestra.run.smithi179.stdout:64424509449 2024-05-08T22:33:39.689 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T22:33:39.690 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:39.690 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T22:33:39.691 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:39.692 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T22:33:39.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 Processor -- start 2024-05-08T22:33:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 -- start start 2024-05-08T22:33:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc01406c2d0 0x7fc0140b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140b2be0 con 0x7fc0141536f0 2024-05-08T22:33:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140b1970 con 0x7fc014159640 2024-05-08T22:33:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc01beab700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc014153ad0 con 0x7fc01406c2d0 2024-05-08T22:33:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 Processor -- start 2024-05-08T22:33:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc019c47700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc0141536f0 0x7fc01406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33466/0 (socket says 172.21.15.179:33466) 2024-05-08T22:33:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc019c47700 1 -- 172.21.15.179:0/369703230 learned_addr learned my addr 172.21.15.179:0/369703230 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f1dff30e700 1 Processor -- start 2024-05-08T22:33:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 -- start start 2024-05-08T22:33:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538860384 0 0) 0x7fc010001040 con 0x7fc0141536f0 2024-05-08T22:33:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc019446700 1 --2- 172.21.15.179:0/369703230 >> v2:172.21.15.195:3300/0 conn(0x7fc01406c2d0 0x7fc0140b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0000035d0 con 0x7fc0141536f0 2024-05-08T22:33:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f1dff30e700 1 -- start start 2024-05-08T22:33:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261992739 0 0) 0x7fc0140b1970 con 0x7fc014159640 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc010001040 con 0x7fc014159640 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2685717769 0 0) 0x7fc0000035d0 con 0x7fc0141536f0 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f00d01536f0 0x7f00d00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 >> v2:172.21.15.195:3300/0 conn(0x7fc01406c2d0 msgr2=0x7fc0140b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 --2- 172.21.15.179:0/369703230 >> v2:172.21.15.195:3300/0 conn(0x7fc01406c2d0 0x7fc0140b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f634ef86700 1 Processor -- start 2024-05-08T22:33:39.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 >> v1:172.21.15.195:6789/0 conn(0x7fc014159640 legacy=0x7fc014159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d00b2be0 con 0x7f00d006bbc0 2024-05-08T22:33:39.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d006c5b0 con 0x7f00d01536f0 2024-05-08T22:33:39.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f00d4f1d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0153ad0 con 0x7f00d0159d50 2024-05-08T22:33:39.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01415a3b0 con 0x7fc0141536f0 2024-05-08T22:33:39.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.879+0000 7f634ef86700 1 -- start start 2024-05-08T22:33:39.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fc010001730 con 0x7fc0141536f0 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1df81536f0 0x7f1df80b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80b2be0 con 0x7f1df8071820 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80b1970 con 0x7f1df8159d50 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df8153ad0 con 0x7f1df81536f0 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 >> v1:172.21.15.179:6789/0 conn(0x7fc0141536f0 legacy=0x7fc01406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00cdd9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f00d006bbc0 0x7f00d0159640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33468/0 (socket says 172.21.15.179:33468) 2024-05-08T22:33:39.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00cdd9b700 1 -- 172.21.15.179:0/537218158 learned_addr learned my addr 172.21.15.179:0/537218158 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc000003c50 0x7fc000004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ce59c700 1 --2- 172.21.15.179:0/537218158 >> v2:172.21.15.179:3300/0 conn(0x7f00d01536f0 0x7f00d00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfc8a9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1df8071820 0x7f1df8159640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33482/0 (socket says 172.21.15.179:33482) 2024-05-08T22:33:39.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfc8a9700 1 -- 172.21.15.179:0/1343057811 learned_addr learned my addr 172.21.15.179:0/1343057811 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 --2- 172.21.15.179:0/537218158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 0x7fc000004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfd0aa700 1 --2- 172.21.15.179:0/1343057811 >> v2:172.21.15.195:3300/0 conn(0x7f1df81536f0 0x7f1df80b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0000052a0 0x7fc00000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019c47700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc000003c50 0x7fc000004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 0x7fc000004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00000c1d0 con 0x7fc000003c50 2024-05-08T22:33:39.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00000c340 con 0x7fc000004870 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00000c500 con 0x7fc0000052a0 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634ef86700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 0x7f634806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634ef86700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f63480b25d0 0x7f63480b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634ef86700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63480cc340 con 0x7f634806dda0 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634ef86700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634806b540 con 0x7f63480b25d0 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634ef86700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634806ac50 con 0x7f634806e180 2024-05-08T22:33:39.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc01a448700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0000052a0 0x7fc00000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfd0aa700 1 -- 172.21.15.179:0/1343057811 >> v1:172.21.15.195:6789/0 conn(0x7f1df8159d50 legacy=0x7f1df815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfd0aa700 1 -- 172.21.15.179:0/1343057811 >> v1:172.21.15.179:6789/0 conn(0x7f1df8071820 legacy=0x7f1df8159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 -- 172.21.15.179:0/537218158 >> v2:172.21.15.179:3300/0 conn(0x7f00d01536f0 msgr2=0x7f00d00b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 --2- 172.21.15.179:0/537218158 >> v2:172.21.15.179:3300/0 conn(0x7f00d01536f0 0x7f00d00b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfd0aa700 1 -- 172.21.15.179:0/1343057811 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df815a3b0 con 0x7f1df81536f0 2024-05-08T22:33:39.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 -- 172.21.15.179:0/537218158 >> v1:172.21.15.179:6789/0 conn(0x7f00d006bbc0 legacy=0x7f00d0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 -- 172.21.15.179:0/537218158 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00b1970 con 0x7f00d0159d50 2024-05-08T22:33:39.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1deffff700 1 -- 172.21.15.179:0/1343057811 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083860931 0 0) 0x7f1df4001110 con 0x7f1df8071820 2024-05-08T22:33:39.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f63480b25d0 0x7f63480b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f63480b25d0 0x7f63480b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35018/0 (socket says 172.21.15.179:35018) 2024-05-08T22:33:39.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 -- 172.21.15.179:0/387288001 learned_addr learned my addr 172.21.15.179:0/387288001 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00ced9d700 1 --2- 172.21.15.179:0/537218158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d006c070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009850 tx=0x7f00c0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff7fc7d8be37a7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 -- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0000052a0 msgr2=0x7fc00000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0000052a0 0x7fc00000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 -- 172.21.15.179:0/369703230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc000003c50 msgr2=0x7fc000004050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dfd0aa700 1 --2- 172.21.15.179:0/1343057811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df81536f0 0x7f1df80b26a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009d90 tx=0x7f1df000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a4c826480d8e43e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc000003c50 0x7fc000004050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc01a448700 1 -- 172.21.15.179:0/369703230 reap_dead start 2024-05-08T22:33:39.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 -- 172.21.15.179:0/369703230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000035d0 con 0x7fc000004870 2024-05-08T22:33:39.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f6347fff700 1 --2- 172.21.15.179:0/387288001 >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 0x7f634806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 -- 172.21.15.179:0/387288001 >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 msgr2=0x7f634806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1deffff700 1 -- 172.21.15.179:0/1343057811 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000e070 con 0x7f1df81536f0 2024-05-08T22:33:39.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00cd59a700 1 -- 172.21.15.179:0/537218158 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0011830 con 0x7f00d0159d50 2024-05-08T22:33:39.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00cd59a700 1 -- 172.21.15.179:0/537218158 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c0011990 con 0x7f00d0159d50 2024-05-08T22:33:39.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00cd59a700 1 -- 172.21.15.179:0/537218158 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0011d20 con 0x7f00d0159d50 2024-05-08T22:33:39.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f63477fe700 1 -- 172.21.15.179:0/387288001 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530779392 0 0) 0x7f6340001490 con 0x7f634806dda0 2024-05-08T22:33:39.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1deffff700 1 -- 172.21.15.179:0/1343057811 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df000bb90 con 0x7f1df81536f0 2024-05-08T22:33:39.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1deffff700 1 -- 172.21.15.179:0/1343057811 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0003a40 con 0x7f1df81536f0 2024-05-08T22:33:39.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 --2- 172.21.15.179:0/387288001 >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 0x7f634806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 -- 172.21.15.179:0/387288001 >> v1:172.21.15.179:6789/0 conn(0x7f634806dda0 legacy=0x7f63480cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc019446700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 0x7fc000004cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc004007b30 tx=0x7fc0000058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f92f229b26596530 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 -- 172.21.15.179:0/387288001 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634806aef0 con 0x7f63480b25d0 2024-05-08T22:33:39.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 -- 172.21.15.179:0/537218158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 msgr2=0x7f00d006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/537218158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d006c070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009850 tx=0x7f00c0009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:39.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fb058953700 1 Processor -- start 2024-05-08T22:33:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fb058953700 1 -- start start 2024-05-08T22:33:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f6347fff700 1 --2- 172.21.15.179:0/387288001 >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 0x7f634806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f634d523700 1 --2- 172.21.15.179:0/387288001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63480b25d0 0x7f63480b2a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f63480ce930 tx=0x7f6338009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef195477938f586f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f85653d9700 1 Processor -- start 2024-05-08T22:33:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f85653d9700 1 -- start start 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- 172.21.15.179:0/1343057811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df81536f0 msgr2=0x7f1df80b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 --2- 172.21.15.179:0/1343057811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df81536f0 0x7f1df80b26a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009d90 tx=0x7f1df000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004019070 con 0x7fc000004870 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004011040 con 0x7fc000004870 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fc018c45700 1 -- 172.21.15.179:0/369703230 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400ab00 con 0x7fc000004870 2024-05-08T22:33:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- 172.21.15.179:0/1343057811 shutdown_connections 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 --2- 172.21.15.179:0/1343057811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df81536f0 0x7f1df80b26a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f1dff30e700 1 -- 172.21.15.179:0/1343057811 >> 172.21.15.179:0/1343057811 conn(0x7f1df8070d30 msgr2=0x7f1df8071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 -- 172.21.15.179:0/537218158 shutdown_connections 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/537218158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d006c070 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/537218158 >> v2:172.21.15.179:3300/0 conn(0x7f00d01536f0 0x7f00d00b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7f00d4f1d700 1 -- 172.21.15.179:0/537218158 >> 172.21.15.179:0/537218158 conn(0x7f00d0070d30 msgr2=0x7f00d0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fb058953700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 0x7fb05406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.883+0000 7fb058953700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cc340 con 0x7fb0540b25d0 2024-05-08T22:33:39.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb058953700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05406b540 con 0x7fb05406dda0 2024-05-08T22:33:39.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb058953700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05406ac50 con 0x7fb05406e180 2024-05-08T22:33:39.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- 172.21.15.179:0/537218158 shutdown_connections 2024-05-08T22:33:39.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- 172.21.15.179:0/537218158 wait complete. 2024-05-08T22:33:39.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb052d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb0540b25d0 0x7fb0540b2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33494/0 (socket says 172.21.15.179:33494) 2024-05-08T22:33:39.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb052d9d700 1 -- 172.21.15.179:0/2875568852 learned_addr learned my addr 172.21.15.179:0/2875568852 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f85653d9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8560150bc0 0x7f856015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f63477fe700 1 -- 172.21.15.179:0/387288001 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800b8a0 con 0x7f63480b25d0 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f85653d9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856015e660 con 0x7f85600ad740 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f85653d9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85600b1cb0 con 0x7f85600adb90 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f85653d9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85600b2930 con 0x7f8560150bc0 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 -- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 msgr2=0x7fc000004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 0x7fc000004cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc004007b30 tx=0x7fc0000058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 -- 172.21.15.179:0/369703230 shutdown_connections 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 --2- 172.21.15.179:0/369703230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc000004870 0x7fc000004cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 -- 172.21.15.179:0/369703230 >> 172.21.15.179:0/369703230 conn(0x7fc014070d30 msgr2=0x7fc014071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb051d9b700 1 --2- 172.21.15.179:0/2875568852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 0x7fb05406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158927004 0 0) 0x7fb04c001490 con 0x7fb0540b25d0 2024-05-08T22:33:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- 172.21.15.179:0/1343057811 shutdown_connections 2024-05-08T22:33:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f63477fe700 1 -- 172.21.15.179:0/387288001 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633800ba00 con 0x7f63480b25d0 2024-05-08T22:33:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb03c0035d0 con 0x7fb0540b25d0 2024-05-08T22:33:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855e7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f85600ad740 0x7f856015d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33510/0 (socket says 172.21.15.179:33510) 2024-05-08T22:33:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855e7fc700 1 -- 172.21.15.179:0/3893343545 learned_addr learned my addr 172.21.15.179:0/3893343545 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855effd700 1 --2- 172.21.15.179:0/3893343545 >> v2:172.21.15.195:3300/0 conn(0x7f8560150bc0 0x7f856015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215524479 0 0) 0x7fb05406b540 con 0x7fb05406dda0 2024-05-08T22:33:39.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855dffb700 1 -- 172.21.15.179:0/3893343545 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285098272 0 0) 0x7f856015e660 con 0x7f85600ad740 2024-05-08T22:33:39.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855dffb700 1 -- 172.21.15.179:0/3893343545 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85480035d0 con 0x7f85600ad740 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c001490 con 0x7fb05406dda0 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb051d9b700 1 -- 172.21.15.179:0/2875568852 >> v1:172.21.15.195:6789/0 conn(0x7fb05406dda0 legacy=0x7fb0540cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb051d9b700 1 -- 172.21.15.179:0/2875568852 >> v1:172.21.15.179:6789/0 conn(0x7fb0540b25d0 legacy=0x7fb0540b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855effd700 1 -- 172.21.15.179:0/3893343545 >> v1:172.21.15.195:6789/0 conn(0x7f85600adb90 legacy=0x7f856015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855effd700 1 -- 172.21.15.179:0/3893343545 >> v1:172.21.15.179:6789/0 conn(0x7f85600ad740 legacy=0x7f856015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855effd700 1 -- 172.21.15.179:0/3893343545 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600b1f90 con 0x7f8560150bc0 2024-05-08T22:33:39.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855dffb700 1 -- 172.21.15.179:0/3893343545 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1602158201 0 0) 0x7f85480035d0 con 0x7f85600ad740 2024-05-08T22:33:39.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb051d9b700 1 -- 172.21.15.179:0/2875568852 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05406aef0 con 0x7fb05406e180 2024-05-08T22:33:39.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- 172.21.15.179:0/1343057811 wait complete. 2024-05-08T22:33:39.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 -- 172.21.15.179:0/387288001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63480b25d0 msgr2=0x7f63480b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 --2- 172.21.15.179:0/387288001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63480b25d0 0x7f63480b2a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f63480ce930 tx=0x7f6338009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f855effd700 1 --2- 172.21.15.179:0/3893343545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8560150bc0 0x7f856015ac70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8554009880 tx=0x7f8554009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45655d82ee50e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 -- 172.21.15.179:0/369703230 shutdown_connections 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 -- 172.21.15.179:0/387288001 shutdown_connections 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 --2- 172.21.15.179:0/387288001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63480b25d0 0x7f63480b2a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 --2- 172.21.15.179:0/387288001 >> v2:172.21.15.195:3300/0 conn(0x7f634806e180 0x7f634806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 -- 172.21.15.179:0/387288001 >> 172.21.15.179:0/387288001 conn(0x7f63480ce960 msgr2=0x7f63480ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 Processor -- start 2024-05-08T22:33:39.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb051d9b700 1 --2- 172.21.15.179:0/2875568852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 0x7fb05406e5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb048009880 tx=0x7fb048009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f123aa1cf463a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f634ef86700 1 -- 172.21.15.179:0/387288001 shutdown_connections 2024-05-08T22:33:39.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048011830 con 0x7fb05406e180 2024-05-08T22:33:39.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048011990 con 0x7fb05406e180 2024-05-08T22:33:39.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fb05159a700 1 -- 172.21.15.179:0/2875568852 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048011d00 con 0x7fb05406e180 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 Processor -- start 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- start start 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1df8071820 0x7f1df80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 0x7f1df80af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815a5d0 con 0x7f1df81536f0 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8148a20 con 0x7f1df8159d50 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dff30e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8148d00 con 0x7f1df8071820 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35040/0 (socket says 172.21.15.179:35040) 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 -- 172.21.15.179:0/3068188342 learned_addr learned my addr 172.21.15.179:0/3068188342 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfd8ab700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 0x7f1df80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- start start 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d01536f0 0x7f00d00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d01f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0154ef0 con 0x7f00d006bbc0 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0154cf0 con 0x7f00d01536f0 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00d4f1d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01f6bb0 con 0x7f00d0159d50 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35026/0 (socket says 172.21.15.179:35026) 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 -- 172.21.15.179:0/2286800843 learned_addr learned my addr 172.21.15.179:0/2286800843 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 -- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 msgr2=0x7f00d01f6670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d01f6670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 -- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d01536f0 msgr2=0x7f00d00ccb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d01536f0 0x7f00d00ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f00ce59c700 1 -- 172.21.15.179:0/2286800843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00094b0 con 0x7f00d006bbc0 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 -- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1df8071820 msgr2=0x7f1df80cd920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1df8071820 0x7f1df80cd920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7f1dfc8a9700 1 -- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 msgr2=0x7f1df80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dfc8a9700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 0x7f1df80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dfc8a9700 1 -- 172.21.15.179:0/3068188342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0009a70 con 0x7f1df81536f0 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dfd8ab700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 0x7f1df80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f855dffb700 1 -- 172.21.15.179:0/3893343545 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580019f0 con 0x7f8560150bc0 2024-05-08T22:33:39.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00ce59c700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e760 tx=0x7f00c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f855dffb700 1 -- 172.21.15.179:0/3893343545 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8554010590 con 0x7f8560150bc0 2024-05-08T22:33:39.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dfc8a9700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c4a0 tx=0x7f1df400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 -- 172.21.15.179:0/2875568852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 msgr2=0x7fb05406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 --2- 172.21.15.179:0/2875568852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 0x7fb05406e5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb048009880 tx=0x7fb048009b90 comp rx=0 tx=0).stop 2024-05-08T22:33:39.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 -- 172.21.15.179:0/369703230 wait complete. 2024-05-08T22:33:39.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.887+0000 7fc01beab700 1 Processor -- start 2024-05-08T22:33:39.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 -- start start 2024-05-08T22:33:39.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 -- 172.21.15.179:0/2875568852 shutdown_connections 2024-05-08T22:33:39.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 --2- 172.21.15.179:0/2875568852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb05406e180 0x7fb05406e5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 -- 172.21.15.179:0/2875568852 >> 172.21.15.179:0/2875568852 conn(0x7fb0540ce960 msgr2=0x7fb0540ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f85653d9700 1 -- 172.21.15.179:0/3893343545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8560150bc0 msgr2=0x7f856015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f85653d9700 1 --2- 172.21.15.179:0/3893343545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8560150bc0 0x7f856015ac70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8554009880 tx=0x7f8554009b90 comp rx=0 tx=0).stop 2024-05-08T22:33:39.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f634ef86700 1 -- 172.21.15.179:0/387288001 wait complete. 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f85653d9700 1 -- 172.21.15.179:0/3893343545 shutdown_connections 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f85653d9700 1 --2- 172.21.15.179:0/3893343545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8560150bc0 0x7f856015ac70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f85653d9700 1 -- 172.21.15.179:0/3893343545 >> 172.21.15.179:0/3893343545 conn(0x7f85600ce960 msgr2=0x7f85600ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004ca0 con 0x7f1df81536f0 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4004e00 con 0x7f1df81536f0 2024-05-08T22:33:39.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df8148fe0 con 0x7f1df81536f0 2024-05-08T22:33:39.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4007310 con 0x7f1df81536f0 2024-05-08T22:33:39.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80cde60 con 0x7f1df81536f0 2024-05-08T22:33:39.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 -- 172.21.15.179:0/2875568852 shutdown_connections 2024-05-08T22:33:39.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fb058953700 1 -- 172.21.15.179:0/2875568852 wait complete. 2024-05-08T22:33:39.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4013610 con 0x7f00d006bbc0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4013770 con 0x7f00d006bbc0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4013ae0 con 0x7f00d006bbc0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d0167390 con 0x7f00d006bbc0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d0167950 con 0x7f00d006bbc0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 0x7fc0140cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc01406b420 0x7fc014202e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 0x7fc0142033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01414f3e0 con 0x7fc01406b420 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140ccbc0 con 0x7fc01406b8a0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01beab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc014203a50 con 0x7fc000004870 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01a448700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 0x7fc0142033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01a448700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 0x7fc0142033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52608/0 (socket says 172.21.15.179:52608) 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc01a448700 1 -- 172.21.15.179:0/3622939229 learned_addr learned my addr 172.21.15.179:0/3622939229 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 0x7fc0140cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 -- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 msgr2=0x7fc0142033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 0x7fc0142033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 -- 172.21.15.179:0/3622939229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc01406b420 msgr2=0x7fc014202e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc01406b420 0x7fc014202e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 -- 172.21.15.179:0/3622939229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014203d30 con 0x7fc000004870 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7fc019c47700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 0x7fc0140cc680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b0620 tx=0x7fc010007600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d004e700 con 0x7f00d006bbc0 2024-05-08T22:33:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f634ef86700 1 Processor -- start 2024-05-08T22:33:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.891+0000 7f634ef86700 1 -- start start 2024-05-08T22:33:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 Processor -- start 2024-05-08T22:33:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00c4013c40 con 0x7f00d006bbc0 2024-05-08T22:33:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1df4007470 con 0x7f1df81536f0 2024-05-08T22:33:39.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f85653d9700 1 -- 172.21.15.179:0/3893343545 shutdown_connections 2024-05-08T22:33:39.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 Processor -- start 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df815a7d0 con 0x7f1df81536f0 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dedffb700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 0x7f1ddc068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1df4083f90 con 0x7f1df81536f0 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 0x7f63480c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634806e180 0x7f634815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 0x7f63480c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480c9f90 con 0x7f634806dda0 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480c9380 con 0x7f634806e180 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480c9660 con 0x7f63480b25d0 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634cd22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 0x7f63480c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 -- start start 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634cd22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 0x7f63480c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35052/0 (socket says 172.21.15.179:35052) 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634cd22700 1 -- 172.21.15.179:0/47824523 learned_addr learned my addr 172.21.15.179:0/47824523 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dfd0aa700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 0x7f1ddc068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f85653d9700 1 -- 172.21.15.179:0/3893343545 wait complete. 2024-05-08T22:33:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f00beffd700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 0x7f00b0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 0x7f63480c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00c4084090 con 0x7f00d006bbc0 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 -- start start 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b840c2170 0x7f4b840cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b84161af0 con 0x7f4b8406c2d0 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b840ca670 con 0x7f4b840cda40 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b840cded0 con 0x7f4b840c2170 2024-05-08T22:33:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010004890 con 0x7fc000004870 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f6347fff700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634806e180 0x7f634815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0100049f0 con 0x7fc000004870 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010004d80 con 0x7fc000004870 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb05406dda0 0x7fb0540ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb05406e180 0x7fb0540ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05414d9f0 con 0x7fb05406e180 2024-05-08T22:33:39.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0141518c0 con 0x7fc000004870 2024-05-08T22:33:39.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05414d760 con 0x7fb05406dda0 2024-05-08T22:33:39.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb058953700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540abc00 con 0x7fb0540b25d0 2024-05-08T22:33:39.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc014151e80 con 0x7fc000004870 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 -- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634806e180 msgr2=0x7f634815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634806e180 0x7f634815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 -- 172.21.15.179:0/47824523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 msgr2=0x7f63480c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 0x7f63480c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 -- 172.21.15.179:0/47824523 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338009570 con 0x7f63480b25d0 2024-05-08T22:33:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53172/0 (socket says 172.21.15.179:53172) 2024-05-08T22:33:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 -- 172.21.15.179:0/1235623412 learned_addr learned my addr 172.21.15.179:0/1235623412 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f1dfd0aa700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 0x7f1ddc068cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b370 tx=0x7f1df0011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634d523700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 0x7f63480c8980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6338016000 tx=0x7f6338004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 -- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb05406dda0 msgr2=0x7fb0540ad1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb05406dda0 0x7fb0540ad1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 -- 172.21.15.179:0/1235623412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb05406e180 msgr2=0x7fb0540ab6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb05406e180 0x7fb0540ab6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 -- 172.21.15.179:0/1235623412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480094b0 con 0x7fb0540b25d0 2024-05-08T22:33:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fb052d9d700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00cc50 tx=0x7fb04c00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b840c2170 0x7f4b840cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b840c2170 0x7f4b840cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52622/0 (socket says 172.21.15.179:52622) 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 -- 172.21.15.179:0/3563935764 learned_addr learned my addr 172.21.15.179:0/3563935764 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 -- 172.21.15.179:0/3563935764 >> v1:172.21.15.195:6789/0 conn(0x7f4b840cda40 legacy=0x7f4b8406bbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 -- 172.21.15.179:0/3563935764 >> v1:172.21.15.179:6789/0 conn(0x7f4b8406c2d0 legacy=0x7f4b8406cfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 -- 172.21.15.179:0/3563935764 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840ca950 con 0x7f4b840c2170 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b89754700 1 --2- 172.21.15.179:0/3563935764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840c2170 0x7f4b840cd500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009910 tx=0x7f4b80009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22c6df93d22412b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b7bfff700 1 -- 172.21.15.179:0/3563935764 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80004230 con 0x7f4b840c2170 2024-05-08T22:33:39.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f85653d9700 1 Processor -- start 2024-05-08T22:33:39.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b7bfff700 1 -- 172.21.15.179:0/3563935764 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b80004390 con 0x7f4b840c2170 2024-05-08T22:33:39.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b7bfff700 1 -- 172.21.15.179:0/3563935764 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80009e60 con 0x7f4b840c2170 2024-05-08T22:33:39.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f00cdd9b700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 0x7f00b0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014151720 con 0x7fc000004870 2024-05-08T22:33:39.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f85653d9700 1 -- start start 2024-05-08T22:33:39.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7febe0d49700 1 Processor -- start 2024-05-08T22:33:39.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338003c90 con 0x7f63480b25d0 2024-05-08T22:33:39.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7febe0d49700 1 -- start start 2024-05-08T22:33:39.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63480c9940 con 0x7f63480b25d0 2024-05-08T22:33:39.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634815fd00 con 0x7f63480b25d0 2024-05-08T22:33:39.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633801f070 con 0x7f63480b25d0 2024-05-08T22:33:39.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800bd90 con 0x7f63480b25d0 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/3563935764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840c2170 msgr2=0x7f4b840cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/3563935764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840c2170 0x7f4b840cd500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009910 tx=0x7f4b80009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/3563935764 shutdown_connections 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/3563935764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840c2170 0x7f4b840cd500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/3563935764 >> 172.21.15.179:0/3563935764 conn(0x7f4b84070e50 msgr2=0x7f4b84071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00a9e0 con 0x7fb0540b25d0 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c00ab40 con 0x7fb0540b25d0 2024-05-08T22:33:39.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c012400 con 0x7fb0540b25d0 2024-05-08T22:33:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.895+0000 7f00cdd9b700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 0x7f00b0068cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ad730 tx=0x7f00c8001550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 0x7f85600ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 0x7f8560151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 0x7f8560157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8560157570 con 0x7f85600ad740 2024-05-08T22:33:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600ad490 con 0x7f85600adb90 2024-05-08T22:33:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600abfd0 con 0x7f8560156b80 2024-05-08T22:33:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540a7a10 con 0x7fb0540b25d0 2024-05-08T22:33:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540a7fc0 con 0x7fb0540b25d0 2024-05-08T22:33:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 0x7febdc06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7febdc15d3f0 0x7febdc06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc15d950 con 0x7febdc15d3f0 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc15dc30 con 0x7febdc15cf70 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febe0d49700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0b1af0 con 0x7febdc159270 2024-05-08T22:33:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 0x7f85600ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 0x7f85600ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35060/0 (socket says 172.21.15.179:35060) 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855effd700 1 -- 172.21.15.179:0/3058058178 learned_addr learned my addr 172.21.15.179:0/3058058178 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 0x7f8560151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855f7fe700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 0x7f8560157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdaffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 0x7febdc06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7febdc15d3f0 0x7febdc06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7febdc15d3f0 0x7febdc06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35072/0 (socket says 172.21.15.179:35072) 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 -- 172.21.15.179:0/1335432836 learned_addr learned my addr 172.21.15.179:0/1335432836 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdaffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 0x7febdc06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52664/0 (socket says 172.21.15.179:52664) 2024-05-08T22:33:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6328005310 con 0x7f63480b25d0 2024-05-08T22:33:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc010006110 con 0x7fc000004870 2024-05-08T22:33:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fc00a7fc700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 0x7fbff8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc010083f10 con 0x7fc000004870 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 -- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 msgr2=0x7f8560157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 0x7f8560157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 -- 172.21.15.179:0/3058058178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 msgr2=0x7f85600ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 0x7f85600ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 -- 172.21.15.179:0/3058058178 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540094b0 con 0x7f85600adb90 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855f7fe700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 0x7f8560157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/3563935764 shutdown_connections 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 -- 172.21.15.179:0/1335432836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 msgr2=0x7febdc0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 --2- 172.21.15.179:0/1335432836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 -- 172.21.15.179:0/1335432836 >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 msgr2=0x7febdc06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 --2- 172.21.15.179:0/1335432836 >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 0x7febdc06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 -- 172.21.15.179:0/1335432836 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc0b1dd0 con 0x7febdc15d3f0 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7febdbfff700 1 --2- 172.21.15.179:0/1335432836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc06b980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febdc072030 tx=0x7febcc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c264cb9bc8acf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f855e7fc700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 0x7f8560151380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f855800cca0 tx=0x7f855800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/3563935764 wait complete. 2024-05-08T22:33:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fc019446700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 0x7fbff8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6338026480 con 0x7f63480b25d0 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855800ab80 con 0x7f85600adb90 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- 172.21.15.179:0/3058058178 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85601518c0 con 0x7f85600adb90 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855800ace0 con 0x7f85600adb90 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558007310 con 0x7f85600adb90 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- 172.21.15.179:0/3058058178 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85601f38c0 con 0x7f85600adb90 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f63457fa700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 0x7f6330068d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fc019446700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 0x7fbff8068c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc004007b30 tx=0x7fc004012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6338021070 con 0x7f63480b25d0 2024-05-08T22:33:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 Processor -- start 2024-05-08T22:33:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f4b8b9b8700 1 -- start start 2024-05-08T22:33:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb038005310 con 0x7fb0540b25d0 2024-05-08T22:33:39.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f634cd22700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 0x7f6330068d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f85653d9700 1 -- 172.21.15.179:0/3058058178 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856004e700 con 0x7f85600adb90 2024-05-08T22:33:39.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.899+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f00c4051180 con 0x7f00d006bbc0 2024-05-08T22:33:39.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b8406c2d0 0x7f4b84164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840cda40 0x7f4b840c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f634cd22700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 0x7f6330068d30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f634000e270 tx=0x7f6340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84164c80 con 0x7f4b840c2170 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8414efe0 con 0x7f4b840cda40 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b8b9b8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840c9100 con 0x7f4b8406c2d0 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35088/0 (socket says 172.21.15.179:35088) 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 -- 172.21.15.179:0/4140269785 learned_addr learned my addr 172.21.15.179:0/4140269785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b89f55700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840cda40 0x7f4b840c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 -- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b8406c2d0 msgr2=0x7f4b84164740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b8406c2d0 0x7f4b84164740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 -- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840cda40 msgr2=0x7f4b840c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840cda40 0x7f4b840c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 -- 172.21.15.179:0/4140269785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009570 con 0x7f4b840c2170 2024-05-08T22:33:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f4b88f53700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400b580 tx=0x7f4b7400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8558007470 con 0x7f85600adb90 2024-05-08T22:33:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f854f7fe700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 0x7f8544068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f85580845b0 con 0x7f85600adb90 2024-05-08T22:33:39.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f855effd700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 0x7f8544068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.903+0000 7f855effd700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 0x7f8544068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8554013040 tx=0x7f855401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febda7fc700 1 -- 172.21.15.179:0/1335432836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011830 con 0x7febdc15d3f0 2024-05-08T22:33:39.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74011830 con 0x7f4b840c2170 2024-05-08T22:33:39.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febda7fc700 1 -- 172.21.15.179:0/1335432836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc011990 con 0x7febdc15d3f0 2024-05-08T22:33:39.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febda7fc700 1 -- 172.21.15.179:0/1335432836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011d20 con 0x7febdc15d3f0 2024-05-08T22:33:39.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74011990 con 0x7f4b840c2170 2024-05-08T22:33:39.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74011d20 con 0x7f4b840c2170 2024-05-08T22:33:39.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b841636a0 con 0x7f4b840c2170 2024-05-08T22:33:39.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1df815a7d0 con 0x7f1df81536f0 2024-05-08T22:33:39.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b84163c60 con 0x7f4b840c2170 2024-05-08T22:33:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- 172.21.15.179:0/1335432836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 msgr2=0x7febdc06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- 172.21.15.179:0/1335432836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc06b980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febdc072030 tx=0x7febcc009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- 172.21.15.179:0/1335432836 shutdown_connections 2024-05-08T22:33:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- 172.21.15.179:0/1335432836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc06b980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- 172.21.15.179:0/1335432836 >> v2:172.21.15.195:3300/0 conn(0x7febdc15cf70 0x7febdc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- 172.21.15.179:0/1335432836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- 172.21.15.179:0/1335432836 >> 172.21.15.179:0/1335432836 conn(0x7febdc0ce960 msgr2=0x7febdc0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- 172.21.15.179:0/1335432836 shutdown_connections 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- 172.21.15.179:0/1335432836 wait complete. 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 Processor -- start 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- start start 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc14d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febdc15cf70 0x7febdc0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc145bf0 con 0x7febdc15d3f0 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc144fe0 con 0x7febdc15cf70 2024-05-08T22:33:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febe0d49700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1452c0 con 0x7febdc159270 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febdbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febdbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35098/0 (socket says 172.21.15.179:35098) 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febdbfff700 1 -- 172.21.15.179:0/2408150072 learned_addr learned my addr 172.21.15.179:0/2408150072 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febdaffd700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febdc15cf70 0x7febdc0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7febdb7fe700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc14d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b84067770 con 0x7f4b840c2170 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b7401a400 con 0x7f4b840c2170 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b74084550 con 0x7f4b840c2170 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b89754700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.907+0000 7f4b89754700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 0x7f4b68068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009910 tx=0x7f4b80008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 -- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 msgr2=0x7febdc14d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc14d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 -- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febdc15cf70 msgr2=0x7febdc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febdc15cf70 0x7febdc0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 -- 172.21.15.179:0/2408150072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009570 con 0x7febdc15d3f0 2024-05-08T22:33:39.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc010051070 con 0x7fc000004870 2024-05-08T22:33:39.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febdbfff700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febcc014040 tx=0x7febcc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb04c01b3b0 con 0x7fb0540b25d0 2024-05-08T22:33:39.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8558051710 con 0x7f85600adb90 2024-05-08T22:33:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7fb042ffd700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 0x7fb034068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7fb05259c700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 0x7fb034068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b74051540 con 0x7f4b840c2170 2024-05-08T22:33:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7fb05259c700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 0x7fb034068c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb0440015b0 tx=0x7fb0440078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc004450 con 0x7febdc15d3f0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc01e070 con 0x7febdc15d3f0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc1455a0 con 0x7febdc15d3f0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc159f20 con 0x7febdc15d3f0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.911+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc003900 con 0x7febdc15d3f0 2024-05-08T22:33:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.915+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb04c055090 con 0x7fb0540b25d0 2024-05-08T22:33:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.915+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6338053500 con 0x7f63480b25d0 2024-05-08T22:33:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.915+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febcc003a60 con 0x7febdc15d3f0 2024-05-08T22:33:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.915+0000 7febc3fff700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 0x7febc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.915+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7febcc085380 con 0x7febdc15d3f0 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.919+0000 7febdb7fe700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 0x7febc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.919+0000 7febdb7fe700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 0x7febc8068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b3270 tx=0x7febd400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.919+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb04c0513d0 con 0x7fb0540b25d0 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.927+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc145d60 con 0x7febdc15d3f0 2024-05-08T22:33:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:39.935+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7febcc0525b0 con 0x7febdc15d3f0 2024-05-08T22:33:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.223+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f00d00b3020 con 0x7f00d006bbc0 2024-05-08T22:33:40.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.223+0000 7f00beffd700 1 -- 172.21.15.179:0/2286800843 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00c40547a0 con 0x7f00d006bbc0 2024-05-08T22:33:40.225 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:40.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.227+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4b84073290 con 0x7f4b840c2170 2024-05-08T22:33:40.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.227+0000 7f4b79ffb700 1 -- 172.21.15.179:0/4140269785 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b74054b60 con 0x7f4b840c2170 2024-05-08T22:33:40.229 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 msgr2=0x7f4b68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 0x7f4b68068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009910 tx=0x7f4b80008030 comp rx=0 tx=0).stop 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 msgr2=0x7f4b8414eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400b580 tx=0x7f4b7400b890 comp rx=0 tx=0).stop 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 msgr2=0x7f00b0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 0x7f00b0068cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ad730 tx=0x7f00c8001550 comp rx=0 tx=0).stop 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 msgr2=0x7f00d0166cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e760 tx=0x7f00c400eb20 comp rx=0 tx=0).stop 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6328005890 con 0x7f63480b25d0 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 shutdown_connections 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b68066730 0x7f4b68068bf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840cda40 0x7f4b840c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b840c2170 0x7f4b8414eaa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 --2- 172.21.15.179:0/4140269785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b8406c2d0 0x7f4b84164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 >> 172.21.15.179:0/4140269785 conn(0x7f4b84070e50 msgr2=0x7f4b840b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 shutdown_connections 2024-05-08T22:33:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00b0066800 0x7f00b0068cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d01536f0 0x7f00d00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d0159d50 0x7f00d01f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 shutdown_connections 2024-05-08T22:33:40.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 --2- 172.21.15.179:0/2286800843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006bbc0 0x7f00d0166cd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 >> 172.21.15.179:0/2286800843 conn(0x7f00d0070d30 msgr2=0x7f00d00afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f4b8b9b8700 1 -- 172.21.15.179:0/4140269785 wait complete. 2024-05-08T22:33:40.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1df80b2be0 con 0x7f1df81536f0 2024-05-08T22:33:40.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 shutdown_connections 2024-05-08T22:33:40.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f00d4f1d700 1 -- 172.21.15.179:0/2286800843 wait complete. 2024-05-08T22:33:40.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f63457fa700 1 -- 172.21.15.179:0/47824523 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6338056b20 con 0x7f63480b25d0 2024-05-08T22:33:40.235 INFO:teuthology.orchestra.run.smithi179.stdout:64424509448 2024-05-08T22:33:40.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f1dedffb700 1 -- 172.21.15.179:0/3068188342 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1df80b2be0 con 0x7f1df81536f0 2024-05-08T22:33:40.235 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 msgr2=0x7f6330068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 0x7f6330068d30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f634000e270 tx=0x7f6340009040 comp rx=0 tx=0).stop 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 msgr2=0x7f63480c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.231+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 0x7f63480c8980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6338016000 tx=0x7f6338004770 comp rx=0 tx=0).stop 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 shutdown_connections 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6330066870 0x7f6330068d30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634806e180 0x7f634815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63480b25d0 0x7f63480c8980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 --2- 172.21.15.179:0/47824523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f634806dda0 0x7f63480c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 >> 172.21.15.179:0/47824523 conn(0x7f63480ce960 msgr2=0x7f6348162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 shutdown_connections 2024-05-08T22:33:40.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f634ef86700 1 -- 172.21.15.179:0/47824523 wait complete. 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 msgr2=0x7f1ddc068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 0x7f1ddc068cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b370 tx=0x7f1df0011000 comp rx=0 tx=0).stop 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 msgr2=0x7f1df8145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c4a0 tx=0x7f1df400c860 comp rx=0 tx=0).stop 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 shutdown_connections 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ddc066800 0x7f1ddc068cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1df8159d50 0x7f1df80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df81536f0 0x7f1df8145b10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 --2- 172.21.15.179:0/3068188342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1df8071820 0x7f1df80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 >> 172.21.15.179:0/3068188342 conn(0x7f1df8070d30 msgr2=0x7f1df806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 shutdown_connections 2024-05-08T22:33:40.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f1dff30e700 1 -- 172.21.15.179:0/3068188342 wait complete. 2024-05-08T22:33:40.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb038005890 con 0x7fb0540b25d0 2024-05-08T22:33:40.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc0140b2be0 con 0x7fc000004870 2024-05-08T22:33:40.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7fb042ffd700 1 -- 172.21.15.179:0/1235623412 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb04c021090 con 0x7fb0540b25d0 2024-05-08T22:33:40.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7fc00a7fc700 1 -- 172.21.15.179:0/3622939229 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0140b2be0 con 0x7fc000004870 2024-05-08T22:33:40.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f85653d9700 1 -- 172.21.15.179:0/3058058178 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f85600ac110 con 0x7f85600adb90 2024-05-08T22:33:40.240 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:40.241 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:40.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.235+0000 7f854f7fe700 1 -- 172.21.15.179:0/3058058178 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8558018090 con 0x7f85600adb90 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 msgr2=0x7fbff8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 0x7fbff8068c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc004007b30 tx=0x7fc004012000 comp rx=0 tx=0).stop 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 msgr2=0x7fc0140cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 0x7fc0140cc680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b0620 tx=0x7fc010007600 comp rx=0 tx=0).stop 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 shutdown_connections 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbff8066780 0x7fbff8068c40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc01406b8a0 0x7fc0142033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc01406b420 0x7fc014202e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 --2- 172.21.15.179:0/3622939229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc000004870 0x7fc0140cc680 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 >> 172.21.15.179:0/3622939229 conn(0x7fc014070d30 msgr2=0x7fc01406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 shutdown_connections 2024-05-08T22:33:40.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.239+0000 7fc01beab700 1 -- 172.21.15.179:0/3622939229 wait complete. 2024-05-08T22:33:40.243 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:40.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 msgr2=0x7f8544068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 0x7f8544068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8554013040 tx=0x7f855401c040 comp rx=0 tx=0).stop 2024-05-08T22:33:40.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 msgr2=0x7f8560151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 0x7f8560151380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f855800cca0 tx=0x7f855800a060 comp rx=0 tx=0).stop 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 shutdown_connections 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8544066730 0x7f8544068bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85600adb90 0x7f8560151380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8560156b80 0x7f8560157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 --2- 172.21.15.179:0/3058058178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85600ad740 0x7f85600ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 >> 172.21.15.179:0/3058058178 conn(0x7f85600ce960 msgr2=0x7f8560070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 shutdown_connections 2024-05-08T22:33:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7f854d7fa700 1 -- 172.21.15.179:0/3058058178 wait complete. 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 msgr2=0x7fb034068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 0x7fb034068c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb0440015b0 tx=0x7fb0440078f0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 msgr2=0x7fb0540a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00cc50 tx=0x7fb04c00cd30 comp rx=0 tx=0).stop 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 shutdown_connections 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb034066780 0x7fb034068c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0540b25d0 0x7fb0540a74d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb05406e180 0x7fb0540ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 --2- 172.21.15.179:0/1235623412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb05406dda0 0x7fb0540ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.247+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 >> 172.21.15.179:0/1235623412 conn(0x7fb0540ce960 msgr2=0x7fb0540cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.251+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 shutdown_connections 2024-05-08T22:33:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.251+0000 7fb058953700 1 -- 172.21.15.179:0/1235623412 wait complete. 2024-05-08T22:33:40.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.4 2024-05-08T22:33:40.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-05-08T22:33:40.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.1 2024-05-08T22:33:40.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.0 2024-05-08T22:33:40.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.6 2024-05-08T22:33:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.259+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7febdc04e700 con 0x7febdc15d3f0 2024-05-08T22:33:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.259+0000 7febc3fff700 1 -- 172.21.15.179:0/2408150072 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febcc015020 con 0x7febdc15d3f0 2024-05-08T22:33:40.263 INFO:teuthology.orchestra.run.smithi179.stdout:64424509447 2024-05-08T22:33:40.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.7 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 msgr2=0x7febc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 0x7febc8068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b3270 tx=0x7febd400d040 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 msgr2=0x7febdc0cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febcc014040 tx=0x7febcc004770 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 shutdown_connections 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7febc8066730 0x7febc8068bf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febdc15cf70 0x7febdc0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7febdc15d3f0 0x7febdc0cca60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 --2- 172.21.15.179:0/2408150072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febdc159270 0x7febdc14d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 >> 172.21.15.179:0/2408150072 conn(0x7febdc0ce960 msgr2=0x7febdc159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:40.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 shutdown_connections 2024-05-08T22:33:40.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:40.263+0000 7febe0d49700 1 -- 172.21.15.179:0/2408150072 wait complete. 2024-05-08T22:33:40.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.2 2024-05-08T22:33:40.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.3 2024-05-08T22:33:41.261 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T22:33:41.262 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T22:33:41.262 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T22:33:41.263 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T22:33:41.264 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T22:33:41.264 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T22:33:41.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T22:33:41.273 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T22:33:41.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 Processor -- start 2024-05-08T22:33:41.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 -- start start 2024-05-08T22:33:41.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1740159d50 0x7f17400b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400b2be0 con 0x7f17401536f0 2024-05-08T22:33:41.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400b1970 con 0x7f174006c2d0 2024-05-08T22:33:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1746cb4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1740153ad0 con 0x7f1740159d50 2024-05-08T22:33:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1744a50700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f17401536f0 0x7f174006bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33522/0 (socket says 172.21.15.179:33522) 2024-05-08T22:33:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1744a50700 1 -- 172.21.15.179:0/2846785297 learned_addr learned my addr 172.21.15.179:0/2846785297 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1745251700 1 --2- 172.21.15.179:0/2846785297 >> v2:172.21.15.179:3300/0 conn(0x7f1740159d50 0x7f17400b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1745251700 1 -- 172.21.15.179:0/2846785297 >> v1:172.21.15.195:6789/0 conn(0x7f174006c2d0 legacy=0x7f1740159640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1745251700 1 -- 172.21.15.179:0/2846785297 >> v1:172.21.15.179:6789/0 conn(0x7f17401536f0 legacy=0x7f174006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.403+0000 7f1745251700 1 -- 172.21.15.179:0/2846785297 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174015a3b0 con 0x7f1740159d50 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1745251700 1 --2- 172.21.15.179:0/2846785297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1740159d50 0x7f17400b26a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1730009850 tx=0x7f1730009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a748f5dab371a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f173f7fe700 1 -- 172.21.15.179:0/2846785297 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730004230 con 0x7f1740159d50 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f173f7fe700 1 -- 172.21.15.179:0/2846785297 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1730004390 con 0x7f1740159d50 2024-05-08T22:33:41.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f173f7fe700 1 -- 172.21.15.179:0/2846785297 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300036c0 con 0x7f1740159d50 2024-05-08T22:33:41.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- 172.21.15.179:0/2846785297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1740159d50 msgr2=0x7f17400b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 --2- 172.21.15.179:0/2846785297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1740159d50 0x7f17400b26a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1730009850 tx=0x7f1730009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:41.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- 172.21.15.179:0/2846785297 shutdown_connections 2024-05-08T22:33:41.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 --2- 172.21.15.179:0/2846785297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1740159d50 0x7f17400b26a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- 172.21.15.179:0/2846785297 >> 172.21.15.179:0/2846785297 conn(0x7f1740070d30 msgr2=0x7f1740071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- 172.21.15.179:0/2846785297 shutdown_connections 2024-05-08T22:33:41.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- 172.21.15.179:0/2846785297 wait complete. 2024-05-08T22:33:41.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 Processor -- start 2024-05-08T22:33:41.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.407+0000 7f1746cb4700 1 -- start start 2024-05-08T22:33:41.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f174006c2d0 0x7f17401481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1740159d50 0x7f1740154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400cdc40 con 0x7f17401536f0 2024-05-08T22:33:41.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740154f60 con 0x7f1740159d50 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400cd5f0 con 0x7f174006c2d0 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35106/0 (socket says 172.21.15.179:35106) 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 -- 172.21.15.179:0/4254796140 learned_addr learned my addr 172.21.15.179:0/4254796140 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1744a50700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f174006c2d0 0x7f17401481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1745251700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1740159d50 0x7f1740154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 -- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f174006c2d0 msgr2=0x7f17401481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f174006c2d0 0x7f17401481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 -- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1740159d50 msgr2=0x7f1740154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1740159d50 0x7f1740154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 -- 172.21.15.179:0/4254796140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300094b0 con 0x7f17401536f0 2024-05-08T22:33:41.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173ffff700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173400b520 tx=0x7f173400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734011830 con 0x7f17401536f0 2024-05-08T22:33:41.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 -- 172.21.15.179:0/4254796140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17400cd8d0 con 0x7f17401536f0 2024-05-08T22:33:41.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f1746cb4700 1 -- 172.21.15.179:0/4254796140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1740148710 con 0x7f17401536f0 2024-05-08T22:33:41.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17340119d0 con 0x7f17401536f0 2024-05-08T22:33:41.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.411+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734011d80 con 0x7f17401536f0 2024-05-08T22:33:41.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f173401a400 con 0x7f17401536f0 2024-05-08T22:33:41.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f173d7fa700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 0x7f1724068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1734084540 con 0x7f17401536f0 2024-05-08T22:33:41.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f1744a50700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 0x7f1724068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f1744a50700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 0x7f1724068cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1738007380 tx=0x7f1738001c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f1746cb4700 1 -- 172.21.15.179:0/4254796140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c005310 con 0x7f17401536f0 2024-05-08T22:33:41.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f961316a700 1 Processor -- start 2024-05-08T22:33:41.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f961316a700 1 -- start start 2024-05-08T22:33:41.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7fe2e176a700 1 Processor -- start 2024-05-08T22:33:41.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7fe2e176a700 1 -- start start 2024-05-08T22:33:41.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f9e290ae700 1 Processor -- start 2024-05-08T22:33:41.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.415+0000 7f9e290ae700 1 -- start start 2024-05-08T22:33:41.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 Processor -- start 2024-05-08T22:33:41.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 -- start start 2024-05-08T22:33:41.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2e176a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe2dc15d340 0x7fe2dc161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2e176a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0cd6d0 con 0x7fe2dc15cf60 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2e176a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc161b30 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2e176a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cd970 con 0x7fe2dc15d340 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f960c06e180 0x7f960c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f960c0b25d0 0x7f960c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c06b540 con 0x7f960c06e180 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c06ac50 con 0x7f960c0b25d0 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f961316a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06aef0 con 0x7f960c06dda0 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53216/0 (socket says 172.21.15.179:53216) 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 -- 172.21.15.179:0/3268298424 learned_addr learned my addr 172.21.15.179:0/3268298424 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612969700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.195:3300/0 conn(0x7f960c0b25d0 0x7f960c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2da7fc700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fe2dc0b4f70 0x7fe2dc160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38368/0 (socket says 172.21.15.179:38368) 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2daffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe2dc15cf60 0x7fe2dc0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33532/0 (socket says 172.21.15.179:33532) 2024-05-08T22:33:41.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2da7fc700 1 -- 172.21.15.179:0/1296285512 learned_addr learned my addr 172.21.15.179:0/1296285512 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:41.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9611967700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.179:3300/0 conn(0x7f960c06e180 0x7f960c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 Processor -- start 2024-05-08T22:33:41.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 -- start start 2024-05-08T22:33:41.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e290ae700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9e240ad740 0x7f9e240adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e290ae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2406b440 0x7f9e2406b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e290ae700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e240ae0d0 con 0x7f9e24150bc0 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e290ae700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240b2930 con 0x7f9e240ad740 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e290ae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240b1cb0 con 0x7f9e2406b440 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e22d9d700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f9e24150bc0 0x7f9e2415ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38382/0 (socket says 172.21.15.179:38382) 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e22d9d700 1 -- 172.21.15.179:0/1318404549 learned_addr learned my addr 172.21.15.179:0/1318404549 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 --2- 172.21.15.179:0/1318404549 >> v2:172.21.15.195:3300/0 conn(0x7f9e240ad740 0x7f9e240adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2359e700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2406b440 0x7f9e2406b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a3406ee20 0x7f7a340b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3406dc40 0x7f7a3406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a34159fe0 con 0x7f7a3414fde0 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b1970 con 0x7f7a3406ee20 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a388d1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3406e5f0 con 0x7f7a3406dc40 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f66f00b3180 0x7f66f0165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00cc9e0 con 0x7f66f015f0c0 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00b37a0 con 0x7f66f00ad9d0 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66f4d8f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00b35b0 con 0x7f66f00b3180 2024-05-08T22:33:41.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f66f00b3180 0x7f66f0165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 -- 172.21.15.179:0/3268298424 >> v2:172.21.15.195:3300/0 conn(0x7f960c0b25d0 msgr2=0x7f960c0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.195:3300/0 conn(0x7f960c0b25d0 0x7f960c0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 -- 172.21.15.179:0/3268298424 >> v2:172.21.15.179:3300/0 conn(0x7f960c06e180 msgr2=0x7f960c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.179:3300/0 conn(0x7f960c06e180 0x7f960c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 -- 172.21.15.179:0/3268298424 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c06b1d0 con 0x7f960c06dda0 2024-05-08T22:33:41.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66eeffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f66f015f0c0 0x7f66f00b2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33546/0 (socket says 172.21.15.179:33546) 2024-05-08T22:33:41.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66eeffd700 1 -- 172.21.15.179:0/1167407694 learned_addr learned my addr 172.21.15.179:0/1167407694 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:41.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a3406ee20 0x7f7a340b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a3406ee20 0x7f7a340b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52744/0 (socket says 172.21.15.179:52744) 2024-05-08T22:33:41.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 -- 172.21.15.179:0/2622919779 learned_addr learned my addr 172.21.15.179:0/2622919779 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596353013 0 0) 0x7fe2dc161b30 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f66f00b3180 0x7f66f0165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35112/0 (socket says 172.21.15.179:35112) 2024-05-08T22:33:41.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c40035d0 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/1296285512 >> v2:172.21.15.195:3300/0 conn(0x7fe2dc15d340 0x7fe2dc161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9612168700 1 --2- 172.21.15.179:0/3268298424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f960c06d400 tx=0x7f9608009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839a4332fafcd847 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e21d9b700 1 -- 172.21.15.179:0/1318404549 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038608518 0 0) 0x7f9e14001110 con 0x7f9e24150bc0 2024-05-08T22:33:41.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e21d9b700 1 -- 172.21.15.179:0/1318404549 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e0c0035d0 con 0x7f9e24150bc0 2024-05-08T22:33:41.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a32d9d700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3406dc40 0x7f7a3406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 -- 172.21.15.179:0/1167407694 >> v1:172.21.15.195:6789/0 conn(0x7f66f00ad9d0 legacy=0x7f66f015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 -- 172.21.15.179:0/1167407694 >> v1:172.21.15.179:6789/0 conn(0x7f66f015f0c0 legacy=0x7f66f00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 -- 172.21.15.179:0/1167407694 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f006ed90 con 0x7f66f00b3180 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 -- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2406b440 msgr2=0x7f9e2406b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2406b440 0x7f9e2406b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 -- 172.21.15.179:0/1318404549 >> v1:172.21.15.195:6789/0 conn(0x7f9e24150bc0 legacy=0x7f9e2415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 -- 172.21.15.179:0/1318404549 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240b1f90 con 0x7f9e240ad740 2024-05-08T22:33:41.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f66effff700 1 --2- 172.21.15.179:0/1167407694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f0165d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009d90 tx=0x7f66e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f684a4152fdfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.1 v1:172.21.15.195:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165097066 0 0) 0x7fe2c40035d0 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 >> v2:172.21.15.195:3300/0 conn(0x7fe2dc15d340 msgr2=0x7fe2dc161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 --2- 172.21.15.179:0/1296285512 >> v2:172.21.15.195:3300/0 conn(0x7fe2dc15d340 0x7fe2dc161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 >> v1:172.21.15.179:6789/0 conn(0x7fe2dc15cf60 legacy=0x7fe2dc0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 --> v1:172.21.15.195:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0cdc50 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.1 v1:172.21.15.195:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe2d40019f0 con 0x7fe2dc0b4f70 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 -- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3406dc40 msgr2=0x7f7a3406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3406dc40 0x7f7a3406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 -- 172.21.15.179:0/2622919779 >> v1:172.21.15.179:6789/0 conn(0x7f7a3414fde0 legacy=0x7f7a34159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 -- 172.21.15.179:0/2622919779 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3415a2c0 con 0x7f7a3406ee20 2024-05-08T22:33:41.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 >> v1:172.21.15.195:6789/0 conn(0x7fe2dc0b4f70 legacy=0x7fe2dc160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c4003ac0 0x7fe2c4003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4004870 0x7fe2c4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 0x7fe2c400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cd970 con 0x7fe2c4005230 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400c1d0 con 0x7fe2c4004870 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400c390 con 0x7fe2c4003ac0 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9611166700 1 -- 172.21.15.179:0/3268298424 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011830 con 0x7f960c06dda0 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9611166700 1 -- 172.21.15.179:0/3268298424 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608011990 con 0x7f960c06dda0 2024-05-08T22:33:41.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9611166700 1 -- 172.21.15.179:0/3268298424 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011d20 con 0x7f960c06dda0 2024-05-08T22:33:41.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 0x7fe2c400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f7a3259c700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406ee20 0x7f7a340b2380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009e50 tx=0x7f7a2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b02f605b2f27e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2da7fc700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4004870 0x7fe2c4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7fe2daffd700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c4003ac0 0x7fe2c4003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.419+0000 7f9e2259c700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e240adb90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009dc0 tx=0x7f9e1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b1a13574d752e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a3159a700 1 -- 172.21.15.179:0/2622919779 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00e070 con 0x7f7a3406ee20 2024-05-08T22:33:41.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a3159a700 1 -- 172.21.15.179:0/2622919779 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00b910 con 0x7f7a3406ee20 2024-05-08T22:33:41.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a3159a700 1 -- 172.21.15.179:0/2622919779 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00bcc0 con 0x7f7a3406ee20 2024-05-08T22:33:41.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9e21d9b700 1 -- 172.21.15.179:0/1318404549 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00e070 con 0x7f9e240ad740 2024-05-08T22:33:41.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- 172.21.15.179:0/3268298424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 msgr2=0x7f960c0cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 --2- 172.21.15.179:0/3268298424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f960c06d400 tx=0x7f9608009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 -- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c4003ac0 msgr2=0x7fe2c4003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c4003ac0 0x7fe2c4003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 -- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4004870 msgr2=0x7fe2c4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4004870 0x7fe2c4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 -- 172.21.15.179:0/1296285512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40035d0 con 0x7fe2c4005230 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 -- 172.21.15.179:0/1296285512 reap_dead start 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- 172.21.15.179:0/3268298424 shutdown_connections 2024-05-08T22:33:41.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.195:3300/0 conn(0x7f960c0b25d0 0x7f960c0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 --2- 172.21.15.179:0/3268298424 >> v2:172.21.15.179:3300/0 conn(0x7f960c06e180 0x7f960c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 --2- 172.21.15.179:0/3268298424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06dda0 0x7f960c0cbc30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- 172.21.15.179:0/3268298424 >> 172.21.15.179:0/3268298424 conn(0x7f960c0ce960 msgr2=0x7f960c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 0x7fe2c400ba30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009b70 tx=0x7fe2c40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59756a34f14b34ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9e21d9b700 1 -- 172.21.15.179:0/1318404549 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00bae0 con 0x7f9e240ad740 2024-05-08T22:33:41.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9e21d9b700 1 -- 172.21.15.179:0/1318404549 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00be50 con 0x7f9e240ad740 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406ee20 msgr2=0x7f7a340b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406ee20 0x7f7a340b2380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009e50 tx=0x7f7a2c00b330 comp rx=0 tx=0).stop 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- 172.21.15.179:0/2622919779 shutdown_connections 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3406dc40 0x7f7a3406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 --2- 172.21.15.179:0/2622919779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406ee20 0x7f7a340b2380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- 172.21.15.179:0/2622919779 >> 172.21.15.179:0/2622919779 conn(0x7f7a340ce960 msgr2=0x7f7a340ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 Processor -- start 2024-05-08T22:33:41.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 -- start start 2024-05-08T22:33:41.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- 172.21.15.179:0/2622919779 shutdown_connections 2024-05-08T22:33:41.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- 172.21.15.179:0/2622919779 wait complete. 2024-05-08T22:33:41.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9e290ae700 1 -- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 msgr2=0x7f9e240adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e240adb90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009dc0 tx=0x7f9e1c00b330 comp rx=0 tx=0).stop 2024-05-08T22:33:41.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1734051630 con 0x7f17401536f0 2024-05-08T22:33:41.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9cf815cf60 0x7f9cf80ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9cf80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- 172.21.15.179:0/3268298424 shutdown_connections 2024-05-08T22:33:41.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- 172.21.15.179:0/3268298424 wait complete. 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80cd6d0 con 0x7f9cf80b4da0 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b5230 con 0x7f9cf815cf60 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfd52c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8161b30 con 0x7f9cf815d340 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfcd2b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9cf80b4da0 0x7f9cf8160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33566/0 (socket says 172.21.15.179:33566) 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfcd2b700 1 -- 172.21.15.179:0/2747530093 learned_addr learned my addr 172.21.15.179:0/2747530093 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf77fe700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9cf80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/2747530093 >> v2:172.21.15.195:3300/0 conn(0x7f9cf815cf60 0x7f9cf80ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 Processor -- start 2024-05-08T22:33:41.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563152961 0 0) 0x7f9cf80cd6d0 con 0x7f9cf80b4da0 2024-05-08T22:33:41.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce00035d0 con 0x7f9cf80b4da0 2024-05-08T22:33:41.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004020 con 0x7fe2c4005230 2024-05-08T22:33:41.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0004180 con 0x7fe2c4005230 2024-05-08T22:33:41.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7fe2d9ffb700 1 -- 172.21.15.179:0/1296285512 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00044f0 con 0x7fe2c4005230 2024-05-08T22:33:41.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f961316a700 1 -- start start 2024-05-08T22:33:41.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 Processor -- start 2024-05-08T22:33:41.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f7a388d1700 1 -- start start 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329849372 0 0) 0x7f9ce00035d0 con 0x7f9cf80b4da0 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 msgr2=0x7f9cf80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9cf80b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 >> v2:172.21.15.195:3300/0 conn(0x7f9cf815cf60 msgr2=0x7f9cf80ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 --2- 172.21.15.179:0/2747530093 >> v2:172.21.15.195:3300/0 conn(0x7f9cf815cf60 0x7f9cf80ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80cd970 con 0x7f9cf80b4da0 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9ce8001670 con 0x7f9cf80b4da0 2024-05-08T22:33:41.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 >> v1:172.21.15.179:6789/0 conn(0x7f9cf80b4da0 legacy=0x7f9cf8160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ce0003a20 0x7f9ce0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9ce0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0004db0 0x7f9ce0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b5230 con 0x7f9ce0003a20 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8161b30 con 0x7f9ce0004db0 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce000ba30 con 0x7f9cf815d340 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ce0003a20 0x7f9ce0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf77fe700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9ce0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cfcd2b700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0004db0 0x7f9ce0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.423+0000 7f9cf7fff700 1 -- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 msgr2=0x7f9ce0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf815d340 0x7f9ce0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf7fff700 1 -- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0004db0 msgr2=0x7f9ce0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0004db0 0x7f9ce0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf7fff700 1 -- 172.21.15.179:0/2747530093 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80cd970 con 0x7f9ce0003a20 2024-05-08T22:33:41.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 msgr2=0x7fe2c400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 0x7fe2c400ba30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009b70 tx=0x7fe2c40058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f960c06dda0 0x7f960c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06e180 0x7f960c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c15f340 con 0x7f960c0b25d0 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c15d660 con 0x7f960c06dda0 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c15d8b0 con 0x7f960c06e180 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3406ee20 0x7f7a341558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- 172.21.15.179:0/1296285512 shutdown_connections 2024-05-08T22:33:41.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 --2- 172.21.15.179:0/1296285512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c4005230 0x7fe2c400ba30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- 172.21.15.179:0/1296285512 >> 172.21.15.179:0/1296285512 conn(0x7fe2dc0cf860 msgr2=0x7fe2dc0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3414fde0 0x7f7a34152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9e290ae700 1 -- 172.21.15.179:0/1318404549 shutdown_connections 2024-05-08T22:33:41.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e2406b440 0x7f9e2406b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1318404549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e240adb90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9e290ae700 1 -- 172.21.15.179:0/1318404549 >> 172.21.15.179:0/1318404549 conn(0x7f9e240ce960 msgr2=0x7f9e240ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34156ce0 con 0x7f7a3406ee20 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34155e00 con 0x7f7a3406dc40 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34156060 con 0x7f7a3414fde0 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35148/0 (socket says 172.21.15.179:35148) 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 -- 172.21.15.179:0/921750128 learned_addr learned my addr 172.21.15.179:0/921750128 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9611967700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06e180 0x7f960c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612168700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f960c06dda0 0x7f960c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ce0003a20 0x7f9ce0003e40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e170 tx=0x7f9ce00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dc5361465d62ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3406ee20 0x7f7a341558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a32d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3414fde0 0x7f7a34152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52804/0 (socket says 172.21.15.179:52804) 2024-05-08T22:33:41.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 -- 172.21.15.179:0/1613034700 learned_addr learned my addr 172.21.15.179:0/1613034700 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- 172.21.15.179:0/1296285512 shutdown_connections 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 -- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06e180 msgr2=0x7f960c1432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06e180 0x7f960c1432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 -- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f960c06dda0 msgr2=0x7f960c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f960c06dda0 0x7f960c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 -- 172.21.15.179:0/921750128 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009570 con 0x7f960c0b25d0 2024-05-08T22:33:41.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- 172.21.15.179:0/1296285512 wait complete. 2024-05-08T22:33:41.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000ec90 con 0x7f9ce0003a20 2024-05-08T22:33:41.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000f040 con 0x7f9ce0003a20 2024-05-08T22:33:41.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cf6ffd700 1 -- 172.21.15.179:0/2747530093 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0014480 con 0x7f9ce0003a20 2024-05-08T22:33:41.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 -- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3414fde0 msgr2=0x7f7a34152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3414fde0 0x7f7a34152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 -- 172.21.15.179:0/1613034700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3406ee20 msgr2=0x7f7a341558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3406ee20 0x7f7a341558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 -- 172.21.15.179:0/1613034700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009b30 con 0x7f7a3406dc40 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- 172.21.15.179:0/2747530093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ce0003a20 msgr2=0x7f9ce0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/2747530093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ce0003a20 0x7f9ce0003e40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e170 tx=0x7f9ce00058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfcd2b700 1 -- 172.21.15.179:0/2747530093 reap_dead start 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- 172.21.15.179:0/2747530093 shutdown_connections 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- 172.21.15.179:0/2747530093 >> 172.21.15.179:0/2747530093 conn(0x7f9cf80cf860 msgr2=0x7f9cf80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- 172.21.15.179:0/2747530093 shutdown_connections 2024-05-08T22:33:41.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- 172.21.15.179:0/2747530093 wait complete. 2024-05-08T22:33:41.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a3259c700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00b370 tx=0x7f7a2c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9612969700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f960400efc0 tx=0x7f960400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 Processor -- start 2024-05-08T22:33:41.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400cb20 con 0x7f960c0b25d0 2024-05-08T22:33:41.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0099b0 con 0x7f7a3406dc40 2024-05-08T22:33:41.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- start start 2024-05-08T22:33:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c004250 con 0x7f7a3406dc40 2024-05-08T22:33:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0045c0 con 0x7f7a3406dc40 2024-05-08T22:33:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c15db90 con 0x7f960c0b25d0 2024-05-08T22:33:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400f040 con 0x7f960c0b25d0 2024-05-08T22:33:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c15f850 con 0x7f960c0b25d0 2024-05-08T22:33:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34156340 con 0x7f7a3406dc40 2024-05-08T22:33:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34156900 con 0x7f7a3406dc40 2024-05-08T22:33:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400cec0 con 0x7f960c0b25d0 2024-05-08T22:33:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 0x7fe2dc1e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2dc1e8690 0x7fe2dc153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc1e8ec0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc155870 con 0x7fe2c4005230 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2e176a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c7b10 con 0x7fe2dc1e8690 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35168/0 (socket says 172.21.15.179:35168) 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 -- 172.21.15.179:0/2576304456 learned_addr learned my addr 172.21.15.179:0/2576304456 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2daffd700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 0x7fe2dc1e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3404e700 con 0x7f7a3406dc40 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 -- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2dc1e8690 msgr2=0x7fe2dc153630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2dc1e8690 0x7fe2dc153630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 -- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 msgr2=0x7fe2dc1e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 0x7fe2dc1e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 -- 172.21.15.179:0/2576304456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc1e8be0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2daffd700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 0x7fe2dc1e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:41.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7fe2db7fe700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0d2690 tx=0x7fe2d00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- 172.21.15.179:0/1318404549 shutdown_connections 2024-05-08T22:33:41.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0021070 con 0x7fe2dc154eb0 2024-05-08T22:33:41.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00036c0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0003a70 con 0x7fe2dc154eb0 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc0c74c0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc1545b0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f66ee7fc700 1 -- 172.21.15.179:0/1167407694 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800e070 con 0x7f66f00b3180 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f66ee7fc700 1 -- 172.21.15.179:0/1167407694 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e800bbd0 con 0x7f66f00b3180 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f66ee7fc700 1 -- 172.21.15.179:0/1167407694 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8012430 con 0x7f66f00b3180 2024-05-08T22:33:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a2c004720 con 0x7f7a3406dc40 2024-05-08T22:33:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- 172.21.15.179:0/1318404549 wait complete. 2024-05-08T22:33:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 Processor -- start 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.427+0000 7f9cfd52c700 1 -- start start 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf81f35b0 0x7f9cf80c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cf80c5b00 0x7f9cf81f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f6e50 con 0x7f9cf80c5b00 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f3190 con 0x7f9ce0003a20 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cfd52c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f28c0 con 0x7f9cf81f35b0 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf77fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf81f35b0 0x7f9cf80c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52824/0 (socket says 172.21.15.179:52824) 2024-05-08T22:33:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 -- 172.21.15.179:0/1825696248 learned_addr learned my addr 172.21.15.179:0/1825696248 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 -- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf81f35b0 msgr2=0x7f9cf80c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf81f35b0 0x7f9cf80c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 -- 172.21.15.179:0/1825696248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cf80c5b00 msgr2=0x7f9cf81f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cf80c5b00 0x7f9cf81f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 -- 172.21.15.179:0/1825696248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80c5f80 con 0x7f9ce0003a20 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f66f4d8f700 1 -- 172.21.15.179:0/1167407694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 msgr2=0x7f66f0165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/1167407694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f0165d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009d90 tx=0x7f66e800b330 comp rx=0 tx=0).stop 2024-05-08T22:33:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f7a22ffd700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 0x7f7a14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7a2c085980 con 0x7f7a3406dc40 2024-05-08T22:33:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9cf7fff700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004730 tx=0x7f9cf0009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f7a31d9b700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 0x7f7a14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 Processor -- start 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- start start 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e2414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 0x7f9e2414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24153140 con 0x7f9e2406b440 2024-05-08T22:33:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24153420 con 0x7f9e240ad740 2024-05-08T22:33:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e290ae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24153700 con 0x7f9e24150bc0 2024-05-08T22:33:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f7a31d9b700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 0x7f7a14068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a340ce450 tx=0x7f7a24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.431+0000 7f9e22d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35186/0 (socket says 172.21.15.179:35186) 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 -- 172.21.15.179:0/1591940837 learned_addr learned my addr 172.21.15.179:0/1591940837 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e2359e700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 0x7f9e2414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e2414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 -- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 msgr2=0x7f9e2414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 0x7f9e2414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 -- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 msgr2=0x7f9e2414df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e2414df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 -- 172.21.15.179:0/1591940837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c009a70 con 0x7f9e2406b440 2024-05-08T22:33:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc005310 con 0x7fe2dc154eb0 2024-05-08T22:33:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2d0003bd0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e22d9d700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e14001df0 tx=0x7f9e14013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7fe2c37fe700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 0x7fe2c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14018440 con 0x7f9e2406b440 2024-05-08T22:33:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe2d0085290 con 0x7fe2dc154eb0 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e241539e0 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e140185a0 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14018930 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241541e0 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e2359e700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 0x7f9e2414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2404e700 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e14018a90 con 0x7f9e2406b440 2024-05-08T22:33:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e0b7fe700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 0x7f9e10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.435+0000 7f9e2259c700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 0x7f9e10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7fe2daffd700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 0x7fe2c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9e2259c700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 0x7f9e10068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00b370 tx=0x7f9e1c01d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- 172.21.15.179:0/1167407694 shutdown_connections 2024-05-08T22:33:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/1167407694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f0165d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- 172.21.15.179:0/1167407694 >> 172.21.15.179:0/1167407694 conn(0x7f66f00ce960 msgr2=0x7f66f00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9604014480 con 0x7f960c0b25d0 2024-05-08T22:33:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c1623f0 con 0x7f960c0b25d0 2024-05-08T22:33:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- 172.21.15.179:0/1167407694 shutdown_connections 2024-05-08T22:33:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- 172.21.15.179:0/1167407694 wait complete. 2024-05-08T22:33:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7fe2daffd700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 0x7fe2c8068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0641b0 tx=0x7fe2cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9602ffd700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 0x7f95f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9604084fb0 con 0x7f960c0b25d0 2024-05-08T22:33:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9612168700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 0x7f95f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 Processor -- start 2024-05-08T22:33:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9612168700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 0x7f95f0068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9608009c30 tx=0x7f960801d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- start start 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f00ad9d0 0x7f66f01517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f015f0c0 0x7f66f014fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f015b040 con 0x7f66f00b3180 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01531f0 con 0x7f66f00ad9d0 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66f4d8f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f015b2a0 con 0x7f66f015f0c0 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35188/0 (socket says 172.21.15.179:35188) 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 -- 172.21.15.179:0/385164614 learned_addr learned my addr 172.21.15.179:0/385164614 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66effff700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f015f0c0 0x7f66f014fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66ef7fe700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f00ad9d0 0x7f66f01517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 -- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f015f0c0 msgr2=0x7f66f014fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f015f0c0 0x7f66f014fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 -- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f00ad9d0 msgr2=0x7f66f01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f00ad9d0 0x7f66f01517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 -- 172.21.15.179:0/385164614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009a70 con 0x7f66f00b3180 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f66eeffd700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001eb0 tx=0x7f66e000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe2d00524c0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.439+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e1400f050 con 0x7f9e2406b440 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0007000 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf0029650 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0029a00 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf81f6b70 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81f2db0 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9cf0029b60 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 0x7f9ce4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9cf0086280 con 0x7f9ce0003a20 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf77fe700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 0x7f9ce4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.443+0000 7f9cf77fe700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 0x7f9ce4068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80c9ae0 tx=0x7f9cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.447+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7a2c052970 con 0x7f7a3406dc40 2024-05-08T22:33:41.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.447+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e1405a630 con 0x7f9e2406b440 2024-05-08T22:33:41.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf81f32d0 con 0x7f9ce0003a20 2024-05-08T22:33:41.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 Processor -- start 2024-05-08T22:33:41.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 -- start start 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 0x7f9acc06bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9acc0ae0c0 0x7f9acc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc15a150 con 0x7f9acc06d240 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc06c4e0 con 0x7f9acc06bbc0 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9acb59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc15a3c0 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9aca59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 0x7f9acc06bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9aca59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 0x7f9acc06bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35200/0 (socket says 172.21.15.179:35200) 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.451+0000 7f9aca59c700 1 -- 172.21.15.179:0/847104565 learned_addr learned my addr 172.21.15.179:0/847104565 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 --2- 172.21.15.179:0/847104565 >> v2:172.21.15.195:3300/0 conn(0x7f9acc0ae0c0 0x7f9acc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0005d90 con 0x7f66f00b3180 2024-05-08T22:33:41.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00070d0 con 0x7f66f00b3180 2024-05-08T22:33:41.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f015b580 con 0x7f66f00b3180 2024-05-08T22:33:41.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0007440 con 0x7f66f00b3180 2024-05-08T22:33:41.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f0150220 con 0x7f66f00b3180 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 -- 172.21.15.179:0/847104565 >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 msgr2=0x7f9acc06bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 --2- 172.21.15.179:0/847104565 >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 0x7f9acc06bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 -- 172.21.15.179:0/847104565 >> v1:172.21.15.179:6789/0 conn(0x7f9acc06d240 legacy=0x7f9acc0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 -- 172.21.15.179:0/847104565 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc15a6a0 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acad9d700 1 --2- 172.21.15.179:0/847104565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc0ae0c0 0x7f9acc0b2c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9acc070cc0 tx=0x7f9ac0010e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93880f994585383 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9ac959a700 1 -- 172.21.15.179:0/847104565 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000bde0 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9ac959a700 1 -- 172.21.15.179:0/847104565 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0018430 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9ac959a700 1 -- 172.21.15.179:0/847104565 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00187c0 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9cf81f32d0 con 0x7f9ce0003a20 2024-05-08T22:33:41.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9604051f10 con 0x7f960c0b25d0 2024-05-08T22:33:41.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66e00075a0 con 0x7f66f00b3180 2024-05-08T22:33:41.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 0x7f66dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66e00836e0 con 0x7f66f00b3180 2024-05-08T22:33:41.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f66ef7fe700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 0x7f66dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f66ef7fe700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 0x7f66dc068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000f60 tx=0x7f66e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.463+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f01536d0 con 0x7f66f00b3180 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 -- 172.21.15.179:0/847104565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc0ae0c0 msgr2=0x7f9acc0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 --2- 172.21.15.179:0/847104565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc0ae0c0 0x7f9acc0b2c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9acc070cc0 tx=0x7f9ac0010e70 comp rx=0 tx=0).stop 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 -- 172.21.15.179:0/847104565 shutdown_connections 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 --2- 172.21.15.179:0/847104565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc0ae0c0 0x7f9acc0b2c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 --2- 172.21.15.179:0/847104565 >> v2:172.21.15.179:3300/0 conn(0x7f9acc06bbc0 0x7f9acc06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 -- 172.21.15.179:0/847104565 >> 172.21.15.179:0/847104565 conn(0x7f9acc070d30 msgr2=0x7f9acc071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 -- 172.21.15.179:0/847104565 shutdown_connections 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.455+0000 7f9acb59e700 1 -- 172.21.15.179:0/847104565 wait complete. 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 Processor -- start 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 -- start start 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 0x7f9acc14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 0x7f9acc152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9acc0ae0c0 0x7f9acc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc15bfd0 con 0x7f9acc06d240 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1500d0 con 0x7f9acc06bbc0 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc15b350 con 0x7f9acc0ae0c0 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9aca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 0x7f9acc14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 0x7f9acc152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9aca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 0x7f9acc14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52840/0 (socket says 172.21.15.179:52840) 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9aca59c700 1 -- 172.21.15.179:0/3364280021 learned_addr learned my addr 172.21.15.179:0/3364280021 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9acad9d700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9acc0ae0c0 0x7f9acc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 -- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9acc0ae0c0 msgr2=0x7f9acc153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9acc0ae0c0 0x7f9acc153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 -- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 msgr2=0x7f9acc14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 0x7f9acc14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.459+0000 7f9ac9d9b700 1 -- 172.21.15.179:0/3364280021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00107f0 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.463+0000 7f9ac9d9b700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 0x7f9acc152ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b16d0 tx=0x7f9ac400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.467+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4003a20 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.467+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac4003b80 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.467+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc15b630 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.467+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc15bbf0 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400a8a0 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ac400aa40 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9abaffd700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 0x7f9ab4068d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9aca59c700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 0x7f9ab4068d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab0005310 con 0x7f9acc06d240 2024-05-08T22:33:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.471+0000 7f9aca59c700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 0x7f9ab4068d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9acc150c80 tx=0x7f9abc0092e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:41.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.475+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f66f01536d0 con 0x7f66f00b3180 2024-05-08T22:33:41.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.475+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ac4099750 con 0x7f9acc06d240 2024-05-08T22:33:41.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.483+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9ac4095390 con 0x7f9acc06d240 2024-05-08T22:33:41.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.735+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe2bc005890 con 0x7fe2dc154eb0 2024-05-08T22:33:41.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.735+0000 7f1746cb4700 1 -- 172.21.15.179:0/4254796140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f171c0055c0 con 0x7f17401536f0 2024-05-08T22:33:41.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.735+0000 7fe2c37fe700 1 -- 172.21.15.179:0/2576304456 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2d0055ae0 con 0x7fe2dc154eb0 2024-05-08T22:33:41.735 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:41.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7f173d7fa700 1 -- 172.21.15.179:0/4254796140 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1734015030 con 0x7f17401536f0 2024-05-08T22:33:41.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 msgr2=0x7fe2c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 0x7fe2c8068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0641b0 tx=0x7fe2cc009040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 msgr2=0x7fe2dc155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0d2690 tx=0x7fe2d00047e0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.738 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:41.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 shutdown_connections 2024-05-08T22:33:41.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2c8066730 0x7fe2c8068bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2dc154eb0 0x7fe2dc155330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2dc1e8690 0x7fe2dc153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 --2- 172.21.15.179:0/2576304456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c4005230 0x7fe2dc1e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 >> 172.21.15.179:0/2576304456 conn(0x7fe2dc0cf860 msgr2=0x7fe2dc06aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 shutdown_connections 2024-05-08T22:33:41.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.739+0000 7fe2e176a700 1 -- 172.21.15.179:0/2576304456 wait complete. 2024-05-08T22:33:41.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 msgr2=0x7f1724068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 0x7f1724068cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1738007380 tx=0x7f1738001c30 comp rx=0 tx=0).stop 2024-05-08T22:33:41.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 msgr2=0x7f17400ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173400b520 tx=0x7f173400b8e0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 shutdown_connections 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1724066800 0x7f1724068cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1740159d50 0x7f1740154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17401536f0 0x7f17400ce0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 --2- 172.21.15.179:0/4254796140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f174006c2d0 0x7f17401481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 >> 172.21.15.179:0/4254796140 conn(0x7f1740070d30 msgr2=0x7f174006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 shutdown_connections 2024-05-08T22:33:41.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.747+0000 7f172effd700 1 -- 172.21.15.179:0/4254796140 wait complete. 2024-05-08T22:33:41.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.7 2024-05-08T22:33:41.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.759+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7a34156a90 con 0x7f7a3406dc40 2024-05-08T22:33:41.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.759+0000 7f7a22ffd700 1 -- 172.21.15.179:0/1613034700 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a34156a90 con 0x7f7a3406dc40 2024-05-08T22:33:41.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.4 2024-05-08T22:33:41.761 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:41.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e2406e980 con 0x7f9e2406b440 2024-05-08T22:33:41.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f9e0b7fe700 1 -- 172.21.15.179:0/1591940837 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e14026070 con 0x7f9e2406b440 2024-05-08T22:33:41.763 INFO:teuthology.orchestra.run.smithi179.stdout:64424509448 2024-05-08T22:33:41.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 msgr2=0x7f7a14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 0x7f7a14068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a340ce450 tx=0x7f7a24009040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 msgr2=0x7f7a340cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00b370 tx=0x7f7a2c004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 shutdown_connections 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066730 0x7f7a14068bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a3414fde0 0x7f7a34152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3406ee20 0x7f7a341558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 --2- 172.21.15.179:0/1613034700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a3406dc40 0x7f7a340cd710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 >> 172.21.15.179:0/1613034700 conn(0x7f7a340ce960 msgr2=0x7f7a3415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 shutdown_connections 2024-05-08T22:33:41.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.763+0000 7f7a388d1700 1 -- 172.21.15.179:0/1613034700 wait complete. 2024-05-08T22:33:41.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 msgr2=0x7f9e10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 0x7f9e10068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00b370 tx=0x7f9e1c01d040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 msgr2=0x7f9e241424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e14001df0 tx=0x7f9e14013520 comp rx=0 tx=0).stop 2024-05-08T22:33:41.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 shutdown_connections 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e10066730 0x7f9e10068bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e240ad740 0x7f9e2414df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e24150bc0 0x7f9e2414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 --2- 172.21.15.179:0/1591940837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e2406b440 0x7f9e241424c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 >> 172.21.15.179:0/1591940837 conn(0x7f9e240ce960 msgr2=0x7f9e240cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 shutdown_connections 2024-05-08T22:33:41.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.767+0000 7f9e290ae700 1 -- 172.21.15.179:0/1591940837 wait complete. 2024-05-08T22:33:41.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.771+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9cf81f32d0 con 0x7f9ce0003a20 2024-05-08T22:33:41.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.771+0000 7f9cf4ff9700 1 -- 172.21.15.179:0/1825696248 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9cf81f32d0 con 0x7f9ce0003a20 2024-05-08T22:33:41.771 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:41.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.771+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f66f01536d0 con 0x7f66f00b3180 2024-05-08T22:33:41.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.771+0000 7f66d7fff700 1 -- 172.21.15.179:0/385164614 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66f01536d0 con 0x7f66f00b3180 2024-05-08T22:33:41.772 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 msgr2=0x7f66dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 0x7f66dc068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000f60 tx=0x7f66e4009040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 msgr2=0x7f66f015ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001eb0 tx=0x7f66e000c7b0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 shutdown_connections 2024-05-08T22:33:41.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66dc066730 0x7f66dc068bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f015f0c0 0x7f66f014fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66f00b3180 0x7f66f015ab00 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001eb0 tx=0x7f66e000c7b0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 --2- 172.21.15.179:0/385164614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f00ad9d0 0x7f66f01517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 >> 172.21.15.179:0/385164614 conn(0x7f66f00ce960 msgr2=0x7f66f006cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 shutdown_connections 2024-05-08T22:33:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f66f4d8f700 1 -- 172.21.15.179:0/385164614 wait complete. 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 msgr2=0x7f9ce4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 0x7f9ce4068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80c9ae0 tx=0x7f9cec008040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 msgr2=0x7f9cf8158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004730 tx=0x7f9cf0009750 comp rx=0 tx=0).stop 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 shutdown_connections 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ce4066730 0x7f9ce4068bf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cf80c5b00 0x7f9cf81f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9cf81f35b0 0x7f9cf80c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 --2- 172.21.15.179:0/1825696248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ce0003a20 0x7f9cf8158ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 >> 172.21.15.179:0/1825696248 conn(0x7f9cf80cf860 msgr2=0x7f9cf80ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 shutdown_connections 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9cfd52c700 1 -- 172.21.15.179:0/1825696248 wait complete. 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f960c04e700 con 0x7f960c0b25d0 2024-05-08T22:33:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.775+0000 7f9602ffd700 1 -- 172.21.15.179:0/921750128 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9604018030 con 0x7f960c0b25d0 2024-05-08T22:33:41.778 INFO:teuthology.orchestra.run.smithi179.stdout:60129542151 2024-05-08T22:33:41.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.1 2024-05-08T22:33:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 msgr2=0x7f95f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 0x7f95f0068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9608009c30 tx=0x7f960801d040 comp rx=0 tx=0).stop 2024-05-08T22:33:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 msgr2=0x7f960c145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f960400efc0 tx=0x7f960400c0c0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 shutdown_connections 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f95f0066730 0x7f95f0068bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f960c0b25d0 0x7f960c145c10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f960c06e180 0x7f960c1432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 --2- 172.21.15.179:0/921750128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f960c06dda0 0x7f960c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 >> 172.21.15.179:0/921750128 conn(0x7f960c0ce960 msgr2=0x7f960c1618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 shutdown_connections 2024-05-08T22:33:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.779+0000 7f961316a700 1 -- 172.21.15.179:0/921750128 wait complete. 2024-05-08T22:33:41.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.0 2024-05-08T22:33:41.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.6 2024-05-08T22:33:41.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-05-08T22:33:41.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.2 2024-05-08T22:33:41.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.795+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9ab00055c0 con 0x7f9acc06d240 2024-05-08T22:33:41.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.795+0000 7f9abaffd700 1 -- 172.21.15.179:0/3364280021 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ac40989b0 con 0x7f9acc06d240 2024-05-08T22:33:41.796 INFO:teuthology.orchestra.run.smithi179.stdout:64424509447 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 msgr2=0x7f9ab4068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 0x7f9ab4068d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9acc150c80 tx=0x7f9abc0092e0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 msgr2=0x7f9acc152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 0x7f9acc152ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b16d0 tx=0x7f9ac400ce90 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 shutdown_connections 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ab40668c0 0x7f9ab4068d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9acc0ae0c0 0x7f9acc153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc06d240 0x7f9acc152ed0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 --2- 172.21.15.179:0/3364280021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9acc06bbc0 0x7f9acc14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 >> 172.21.15.179:0/3364280021 conn(0x7f9acc070d30 msgr2=0x7f9acc06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 shutdown_connections 2024-05-08T22:33:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:41.799+0000 7f9acb59e700 1 -- 172.21.15.179:0/3364280021 wait complete. 2024-05-08T22:33:41.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.3 2024-05-08T22:33:42.756 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T22:33:42.762 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T22:33:42.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T22:33:42.782 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T22:33:42.790 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T22:33:42.792 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T22:33:42.794 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T22:33:42.809 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.911+0000 7fa688016700 1 Processor -- start 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.911+0000 7fa688016700 1 -- start start 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 0x7fa68006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa6800ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa6800adea0 0x7fa68006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68006c560 con 0x7fa680161280 2024-05-08T22:33:42.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa680164fe0 con 0x7fa6800adea0 2024-05-08T22:33:42.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa688016700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801652c0 con 0x7fa68006c100 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6855b1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa6800ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6855b1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa6800ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53302/0 (socket says 172.21.15.179:53302) 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6855b1700 1 -- 172.21.15.179:0/167960850 learned_addr learned my addr 172.21.15.179:0/167960850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 --2- 172.21.15.179:0/167960850 >> v2:172.21.15.195:3300/0 conn(0x7fa6800adea0 0x7fa68006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa685db2700 1 --2- 172.21.15.179:0/167960850 >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 0x7fa68006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 -- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 msgr2=0x7fa6800ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 --2- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa6800ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 -- 172.21.15.179:0/167960850 >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 msgr2=0x7fa68006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 --2- 172.21.15.179:0/167960850 >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 0x7fa68006bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 -- 172.21.15.179:0/167960850 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6801655a0 con 0x7fa6800adea0 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa685db2700 1 --2- 172.21.15.179:0/167960850 >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 0x7fa68006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:42.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa6865b3700 1 --2- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6800adea0 0x7fa68006cd70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa6800b1610 tx=0x7fa67800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b3a09a251c8aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa684db0700 1 -- 172.21.15.179:0/167960850 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800e070 con 0x7fa6800adea0 2024-05-08T22:33:42.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa684db0700 1 -- 172.21.15.179:0/167960850 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67800bb20 con 0x7fa6800adea0 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.915+0000 7fa684db0700 1 -- 172.21.15.179:0/167960850 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678012430 con 0x7fa6800adea0 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6800adea0 msgr2=0x7fa68006cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 --2- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6800adea0 0x7fa68006cd70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa6800b1610 tx=0x7fa67800b330 comp rx=0 tx=0).stop 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- 172.21.15.179:0/167960850 shutdown_connections 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 --2- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6800adea0 0x7fa68006cd70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 --2- 172.21.15.179:0/167960850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa6800ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 --2- 172.21.15.179:0/167960850 >> v2:172.21.15.179:3300/0 conn(0x7fa680161280 0x7fa68006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- 172.21.15.179:0/167960850 >> 172.21.15.179:0/167960850 conn(0x7fa680070d30 msgr2=0x7fa680071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- 172.21.15.179:0/167960850 shutdown_connections 2024-05-08T22:33:42.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- 172.21.15.179:0/167960850 wait complete. 2024-05-08T22:33:42.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 Processor -- start 2024-05-08T22:33:42.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.919+0000 7fa688016700 1 -- start start 2024-05-08T22:33:42.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f29af03a700 1 Processor -- start 2024-05-08T22:33:42.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 Processor -- start 2024-05-08T22:33:42.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 -- start start 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa68014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6800adea0 0x7fa68014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 0x7fa680165fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa680151560 con 0x7fa6800adea0 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015e870 con 0x7fa680161280 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015eb50 con 0x7fa68006c100 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa68014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6865b3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 0x7fa680165fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6865b3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 0x7fa680165fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52856/0 (socket says 172.21.15.179:52856) 2024-05-08T22:33:42.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6865b3700 1 -- 172.21.15.179:0/3536411042 learned_addr learned my addr 172.21.15.179:0/3536411042 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f29af03a700 1 -- start start 2024-05-08T22:33:42.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f318c15aed0 0x7f318c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318c0cec90 con 0x7f318c0d05f0 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318c0ceee0 con 0x7f318c0ce0d0 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6855b1700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6800adea0 0x7fa68014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 -- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 msgr2=0x7fa680165fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 0x7fa680165fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 -- 172.21.15.179:0/3536411042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6800adea0 msgr2=0x7fa68014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6800adea0 0x7fa68014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 -- 172.21.15.179:0/3536411042 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009a70 con 0x7fa68006c100 2024-05-08T22:33:42.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa685db2700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa68014cf10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c7f0 tx=0x7fa67000cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31922fd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c15b2b0 con 0x7f318c15aed0 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f318c15aed0 0x7f318c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f318c15aed0 0x7f318c0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35244/0 (socket says 172.21.15.179:35244) 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 -- 172.21.15.179:0/3687117868 learned_addr learned my addr 172.21.15.179:0/3687117868 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 -- 172.21.15.179:0/3687117868 >> v1:172.21.15.195:6789/0 conn(0x7f318c0ce0d0 legacy=0x7f318c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 -- 172.21.15.179:0/3687117868 >> v1:172.21.15.179:6789/0 conn(0x7f318c0d05f0 legacy=0x7f318c0cd9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 -- 172.21.15.179:0/3687117868 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c0cf1c0 con 0x7f318c15aed0 2024-05-08T22:33:42.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f31912fb700 1 --2- 172.21.15.179:0/3687117868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c15aed0 0x7f318c0d00b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3184009e50 tx=0x7f318400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4c4b223682ded4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670004220 con 0x7fa68006c100 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670004380 con 0x7fa68006c100 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700036c0 con 0x7fa68006c100 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 -- 172.21.15.179:0/3536411042 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68015ee30 con 0x7fa68006c100 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7fa688016700 1 -- 172.21.15.179:0/3536411042 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68014d450 con 0x7fa68006c100 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f3183fff700 1 -- 172.21.15.179:0/3687117868 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184003c40 con 0x7f318c15aed0 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f3183fff700 1 -- 172.21.15.179:0/3687117868 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3184003da0 con 0x7f318c15aed0 2024-05-08T22:33:42.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f3183fff700 1 -- 172.21.15.179:0/3687117868 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318400bbc0 con 0x7f318c15aed0 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.923+0000 7f29ae038700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f29a8143620 0x7f29a80ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:38420/0 (socket says 172.21.15.179:38420) 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae038700 1 -- 172.21.15.179:0/2089765440 learned_addr learned my addr 172.21.15.179:0/2089765440 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29af03a700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80ca8b0 0x7f29a80cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29af03a700 1 --2- 172.21.15.179:0/2089765440 >> v2:172.21.15.179:3300/0 conn(0x7f29a80cdfc0 0x7f29a80c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80c4a30 con 0x7f29a8143620 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80ce7f0 con 0x7f29a80cdfc0 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80cead0 con 0x7f29a80ca8b0 2024-05-08T22:33:42.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 --2- 172.21.15.179:0/2089765440 >> v2:172.21.15.179:3300/0 conn(0x7f29a80cdfc0 0x7f29a80c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- 172.21.15.179:0/3687117868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c15aed0 msgr2=0x7f318c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 --2- 172.21.15.179:0/3687117868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c15aed0 0x7f318c0d00b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3184009e50 tx=0x7f318400b330 comp rx=0 tx=0).stop 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ad837700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80ca8b0 0x7f29a80cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 -- 172.21.15.179:0/2089765440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80ca8b0 msgr2=0x7f29a80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80ca8b0 0x7f29a80cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 -- 172.21.15.179:0/2089765440 >> v1:172.21.15.195:6789/0 conn(0x7f29a8143620 legacy=0x7f29a80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 -- 172.21.15.179:0/2089765440 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80cedb0 con 0x7f29a80cdfc0 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- 172.21.15.179:0/3687117868 shutdown_connections 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 --2- 172.21.15.179:0/3687117868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c15aed0 0x7f318c0d00b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- 172.21.15.179:0/3687117868 >> 172.21.15.179:0/3687117868 conn(0x7f318c0b2940 msgr2=0x7f318c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- 172.21.15.179:0/3687117868 shutdown_connections 2024-05-08T22:33:42.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- 172.21.15.179:0/3687117868 wait complete. 2024-05-08T22:33:42.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa670003820 con 0x7fa68006c100 2024-05-08T22:33:42.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7fa6767fc700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 0x7fa664068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa670083f00 con 0x7fa68006c100 2024-05-08T22:33:42.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7fa6855b1700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 0x7fa664068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7fa688016700 1 -- 172.21.15.179:0/3536411042 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68015f350 con 0x7fa68006c100 2024-05-08T22:33:42.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ae839700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a80cdfc0 0x7f29a80c51a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f29a0014a00 tx=0x7f29a0014d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276b744ee19475ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ad036700 1 -- 172.21.15.179:0/2089765440 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0014ea0 con 0x7f29a80cdfc0 2024-05-08T22:33:42.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ad036700 1 -- 172.21.15.179:0/2089765440 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000fd50 con 0x7f29a80cdfc0 2024-05-08T22:33:42.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 Processor -- start 2024-05-08T22:33:42.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f29ad036700 1 -- 172.21.15.179:0/2089765440 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0013b20 con 0x7f29a80cdfc0 2024-05-08T22:33:42.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.927+0000 7f31922fd700 1 -- start start 2024-05-08T22:33:42.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a80cdfc0 msgr2=0x7f29a80c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a80cdfc0 0x7f29a80c51a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f29a0014a00 tx=0x7f29a0014d10 comp rx=0 tx=0).stop 2024-05-08T22:33:42.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7fa6855b1700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 0x7fa664068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa68015d710 tx=0x7fa67c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f318c0ce0d0 0x7f318c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f318c15aed0 0x7f318c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0c6c60 con 0x7f318c0d05f0 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c158a60 con 0x7f318c0ce0d0 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31922fd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c06c680 con 0x7f318c15aed0 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35268/0 (socket says 172.21.15.179:35268) 2024-05-08T22:33:42.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 -- 172.21.15.179:0/2812227450 learned_addr learned my addr 172.21.15.179:0/2812227450 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f31912fb700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f318c0ce0d0 0x7f318c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 shutdown_connections 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80ca8b0 0x7f29a80cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- 172.21.15.179:0/2089765440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a80cdfc0 0x7f29a80c51a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 >> 172.21.15.179:0/2089765440 conn(0x7f29a80cc5d0 msgr2=0x7f29a80cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 shutdown_connections 2024-05-08T22:33:42.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- 172.21.15.179:0/2089765440 wait complete. 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 Processor -- start 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 -- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f318c15aed0 msgr2=0x7f318c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f318c15aed0 0x7f318c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 -- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f318c0ce0d0 msgr2=0x7f318c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f318c0ce0d0 0x7f318c0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 -- 172.21.15.179:0/2812227450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184009b30 con 0x7f318c0d05f0 2024-05-08T22:33:42.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f3190afa700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b520 tx=0x7f317c00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- start start 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a80ca8b0 0x7f29a80b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80cdfc0 0x7f29a80c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c7120 con 0x7f29a8143620 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c7400 con 0x7f29a80ca8b0 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.931+0000 7f29af03a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c76e0 con 0x7f29a80cdfc0 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35272/0 (socket says 172.21.15.179:35272) 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 -- 172.21.15.179:0/239708254 learned_addr learned my addr 172.21.15.179:0/239708254 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ad837700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80cdfc0 0x7f29a80c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae038700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a80ca8b0 0x7f29a80b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 -- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80cdfc0 msgr2=0x7f29a80c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80cdfc0 0x7f29a80c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 -- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a80ca8b0 msgr2=0x7f29a80b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a80ca8b0 0x7f29a80b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 -- 172.21.15.179:0/239708254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0014660 con 0x7f29a8143620 2024-05-08T22:33:42.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29ae839700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29a0014d50 tx=0x7f29a0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c011830 con 0x7f318c0d05f0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c011990 con 0x7f318c0d05f0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f31922fd700 1 -- 172.21.15.179:0/2812227450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c06c960 con 0x7f318c0d05f0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c011d40 con 0x7f318c0d05f0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f31922fd700 1 -- 172.21.15.179:0/2812227450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c06cf20 con 0x7f318c0d05f0 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0014ea0 con 0x7f29a8143620 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0013b00 con 0x7f29a8143620 2024-05-08T22:33:42.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a80c79c0 con 0x7f29a8143620 2024-05-08T22:33:42.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a8141020 con 0x7f29a8143620 2024-05-08T22:33:42.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f317c01a400 con 0x7f318c0d05f0 2024-05-08T22:33:42.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f31922fd700 1 -- 172.21.15.179:0/2812227450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c067740 con 0x7f318c0d05f0 2024-05-08T22:33:42.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 0x7f3170068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0013eb0 con 0x7f29a8143620 2024-05-08T22:33:42.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.935+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f317c084480 con 0x7f318c0d05f0 2024-05-08T22:33:42.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.939+0000 7f31912fb700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 0x7f3170068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.939+0000 7f31912fb700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 0x7f3170068cc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f80 tx=0x7f3184014000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.939+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2990005310 con 0x7f29a8143620 2024-05-08T22:33:42.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.939+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29a0023040 con 0x7f29a8143620 2024-05-08T22:33:42.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7f299effd700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 0x7f298c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa68015f350 con 0x7fa68006c100 2024-05-08T22:33:42.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29a008e020 con 0x7f29a8143620 2024-05-08T22:33:42.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7f29ae038700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 0x7f298c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f317c051570 con 0x7f318c0d05f0 2024-05-08T22:33:42.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.943+0000 7f29ae038700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 0x7f298c068cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005930 tx=0x7f29a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.955+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f29a00bfcf0 con 0x7f29a8143620 2024-05-08T22:33:42.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.967+0000 7facef59e700 1 Processor -- start 2024-05-08T22:33:42.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.967+0000 7facef59e700 1 -- start start 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 0x7facf006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facf006c100 0x7facf00cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf00b1ca0 con 0x7facf0169b90 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf006c570 con 0x7facf006c100 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00b1f80 con 0x7facf0169f70 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facf006c100 0x7facf00cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facf006c100 0x7facf00cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35286/0 (socket says 172.21.15.179:35286) 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 -- 172.21.15.179:0/3358990415 learned_addr learned my addr 172.21.15.179:0/3358990415 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facee59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7facf0169b90 0x7facf0089590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33588/0 (socket says 172.21.15.179:33588) 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facedd9b700 1 --2- 172.21.15.179:0/3358990415 >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 0x7facf006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faced59a700 1 -- 172.21.15.179:0/3358990415 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289154598 0 0) 0x7facf00b1ca0 con 0x7facf0169b90 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faced59a700 1 -- 172.21.15.179:0/3358990415 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd80035d0 con 0x7facf0169b90 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 -- 172.21.15.179:0/3358990415 >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 msgr2=0x7facf006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 --2- 172.21.15.179:0/3358990415 >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 0x7facf006bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 -- 172.21.15.179:0/3358990415 >> v1:172.21.15.179:6789/0 conn(0x7facf0169b90 legacy=0x7facf0089590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 -- 172.21.15.179:0/3358990415 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00b2260 con 0x7facf006c100 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facedd9b700 1 --2- 172.21.15.179:0/3358990415 >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 0x7facf006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:42.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 Processor -- start 2024-05-08T22:33:42.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faceed9d700 1 --2- 172.21.15.179:0/3358990415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf006c100 0x7facf00cbd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7facf006f8e0 tx=0x7face8014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470aa713fc94d740 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 -- start start 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faced59a700 1 -- 172.21.15.179:0/3358990415 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8014e70 con 0x7facf006c100 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faced59a700 1 -- 172.21.15.179:0/3358990415 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8013af0 con 0x7facf006c100 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7faced59a700 1 -- 172.21.15.179:0/3358990415 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8013ea0 con 0x7facf006c100 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- 172.21.15.179:0/3358990415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf006c100 msgr2=0x7facf00cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 --2- 172.21.15.179:0/3358990415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf006c100 0x7facf00cbd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7facf006f8e0 tx=0x7face8014ce0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb3406bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb340b3620 0x7fcb3415de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3406ed00 con 0x7fcb3415d120 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb341611a0 con 0x7fcb340b3620 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7fcb392a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34161950 con 0x7fcb3406bbc0 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- 172.21.15.179:0/3358990415 shutdown_connections 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 --2- 172.21.15.179:0/3358990415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf006c100 0x7facf00cbd50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb3406bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 --2- 172.21.15.179:0/3358990415 >> v2:172.21.15.195:3300/0 conn(0x7facf0169f70 0x7facf006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb340b3620 0x7fcb3415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb340b3620 0x7fcb3415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52892/0 (socket says 172.21.15.179:52892) 2024-05-08T22:33:42.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 -- 172.21.15.179:0/3452394195 learned_addr learned my addr 172.21.15.179:0/3452394195 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb3406bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53348/0 (socket says 172.21.15.179:53348) 2024-05-08T22:33:42.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.971+0000 7facef59e700 1 -- 172.21.15.179:0/3358990415 >> 172.21.15.179:0/3358990415 conn(0x7facf0070d30 msgr2=0x7facf0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 -- 172.21.15.179:0/3358990415 shutdown_connections 2024-05-08T22:33:42.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 -- 172.21.15.179:0/3358990415 wait complete. 2024-05-08T22:33:42.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 -- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 msgr2=0x7fcb3406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 --2- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb3406bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 -- 172.21.15.179:0/3452394195 >> v1:172.21.15.195:6789/0 conn(0x7fcb3415d120 legacy=0x7fcb34160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 -- 172.21.15.179:0/3452394195 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3406c4e0 con 0x7fcb340b3620 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb337fe700 1 --2- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3415de40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009910 tx=0x7fcb24009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2545a9e06c11a5ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb32ffd700 1 -- 172.21.15.179:0/3452394195 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400ba60 con 0x7fcb340b3620 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb32ffd700 1 -- 172.21.15.179:0/3452394195 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2400bbc0 con 0x7fcb340b3620 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 -- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 msgr2=0x7fcb3415de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3415de40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009910 tx=0x7fcb24009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 -- 172.21.15.179:0/3452394195 shutdown_connections 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3415de40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3452394195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb3406bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 -- 172.21.15.179:0/3452394195 >> 172.21.15.179:0/3452394195 conn(0x7fcb34070d30 msgr2=0x7fcb34071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 -- 172.21.15.179:0/3452394195 shutdown_connections 2024-05-08T22:33:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fcb392a0700 1 -- 172.21.15.179:0/3452394195 wait complete. 2024-05-08T22:33:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7fda163e7700 1 Processor -- start 2024-05-08T22:33:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 Processor -- start 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 -- start start 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facf006c100 0x7facf0088bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.975+0000 7facef59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facf0169f70 0x7facf0167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35296/0 (socket says 172.21.15.179:35296) 2024-05-08T22:33:42.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 Processor -- start 2024-05-08T22:33:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 -- 172.21.15.179:0/3193020682 learned_addr learned my addr 172.21.15.179:0/3193020682 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf0089130 con 0x7facf0169b90 2024-05-08T22:33:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf01ff700 con 0x7facf006c100 2024-05-08T22:33:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf016b280 con 0x7facf0169f70 2024-05-08T22:33:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 -- start start 2024-05-08T22:33:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facee59c700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facf006c100 0x7facf0088bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7faceed9d700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facf0169f70 0x7facf0167eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 -- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facf0169f70 msgr2=0x7facf0167eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facf0169f70 0x7facf0167eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 -- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facf006c100 msgr2=0x7facf0088bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facf006c100 0x7facf0088bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8014660 con 0x7facf0169b90 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facedd9b700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7face400e880 tx=0x7face400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4013610 con 0x7facf0169b90 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4013770 con 0x7facf0169b90 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4013b00 con 0x7facf0169b90 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf016acc0 con 0x7facf0169b90 2024-05-08T22:33:42.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf016bb10 con 0x7facf0169b90 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fda10159d50 0x7fda1006c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda100b2be0 con 0x7fda1006bbc0 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda10153ad0 con 0x7fda10159d50 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda163e7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1006c580 con 0x7fda101536f0 2024-05-08T22:33:42.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 -- start start 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53374/0 (socket says 172.21.15.179:53374) 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 -- 172.21.15.179:0/384056048 learned_addr learned my addr 172.21.15.179:0/384056048 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 -- 172.21.15.179:0/384056048 >> v2:172.21.15.195:3300/0 conn(0x7fda10159d50 msgr2=0x7fda1006c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 --2- 172.21.15.179:0/384056048 >> v2:172.21.15.195:3300/0 conn(0x7fda10159d50 0x7fda1006c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 -- 172.21.15.179:0/384056048 >> v1:172.21.15.179:6789/0 conn(0x7fda1006bbc0 legacy=0x7fda10159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 -- 172.21.15.179:0/384056048 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100b1970 con 0x7fda101536f0 2024-05-08T22:33:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fda153e5700 1 --2- 172.21.15.179:0/384056048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda08009bd0 tx=0x7fda0800d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac519a3691833533 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3414b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb3415d120 0x7fcb340cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34155ce0 con 0x7fcb3415d120 2024-05-08T22:33:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340cba70 con 0x7fcb340b3620 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb392a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340cbd50 con 0x7fcb3406bbc0 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53384/0 (socket says 172.21.15.179:53384) 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb337fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3414b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 -- 172.21.15.179:0/3844900700 learned_addr learned my addr 172.21.15.179:0/3844900700 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb38a9f700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb3415d120 0x7fcb340cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 -- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 msgr2=0x7fcb3414b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3414b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 -- 172.21.15.179:0/3844900700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb3415d120 msgr2=0x7fcb340cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb3415d120 0x7fcb340cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.979+0000 7fcb33fff700 1 -- 172.21.15.179:0/3844900700 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009570 con 0x7fcb3406bbc0 2024-05-08T22:33:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf00ce780 con 0x7facf0169b90 2024-05-08T22:33:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7face4013c60 con 0x7facf0169b90 2024-05-08T22:33:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 Processor -- start 2024-05-08T22:33:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb33fff700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e760 tx=0x7fcb2800ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 -- start start 2024-05-08T22:33:42.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facdeffd700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 0x7facd8069050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7face4083f50 con 0x7facf0169b90 2024-05-08T22:33:42.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facee59c700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 0x7facd8069050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800c070 con 0x7fcb3406bbc0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28013610 con 0x7fcb3406bbc0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28013980 con 0x7fcb3406bbc0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda07fff700 1 -- 172.21.15.179:0/384056048 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0800dcd0 con 0x7fda101536f0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda07fff700 1 -- 172.21.15.179:0/384056048 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0800de30 con 0x7fda101536f0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda07fff700 1 -- 172.21.15.179:0/384056048 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08003c80 con 0x7fda101536f0 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 0x7f0da40ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 0x7f0da406e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340cc030 con 0x7fcb3406bbc0 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3415e020 con 0x7fcb3406bbc0 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0da406ed80 0x7f0da40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40ce780 con 0x7f0da406ed80 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da406f1b0 con 0x7f0da4159400 2024-05-08T22:33:42.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da9466700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da406d8b0 con 0x7f0da40ce230 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 0x7f0da406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 0x7f0da406e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53400/0 (socket says 172.21.15.179:53400) 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da37fe700 1 -- 172.21.15.179:0/1363152109 learned_addr learned my addr 172.21.15.179:0/1363152109 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da3fff700 1 --2- 172.21.15.179:0/1363152109 >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 0x7f0da40ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 --2- 172.21.15.179:0/1363152109 >> v2:172.21.15.179:3300/0 conn(0x7f0da406ed80 0x7f0da40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 -- 172.21.15.179:0/1363152109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 msgr2=0x7f0da406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7facee59c700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 0x7facd8069050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7face0007fd0 tx=0x7face0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 --2- 172.21.15.179:0/1363152109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 0x7f0da406e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 -- 172.21.15.179:0/1363152109 >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 msgr2=0x7f0da40ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 --2- 172.21.15.179:0/1363152109 >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 0x7f0da40ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 -- 172.21.15.179:0/384056048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 msgr2=0x7fda100b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 --2- 172.21.15.179:0/384056048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fda08009bd0 tx=0x7fda0800d3c0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 -- 172.21.15.179:0/1363152109 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da406db90 con 0x7f0da406ed80 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da3fff700 1 --2- 172.21.15.179:0/1363152109 >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 0x7f0da40ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 -- 172.21.15.179:0/384056048 shutdown_connections 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 --2- 172.21.15.179:0/384056048 >> v2:172.21.15.195:3300/0 conn(0x7fda10159d50 0x7fda1006c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 --2- 172.21.15.179:0/384056048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda101536f0 0x7fda100b26a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 -- 172.21.15.179:0/384056048 >> 172.21.15.179:0/384056048 conn(0x7fda10070d30 msgr2=0x7fda10071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 -- 172.21.15.179:0/384056048 shutdown_connections 2024-05-08T22:33:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7fda163e7700 1 -- 172.21.15.179:0/384056048 wait complete. 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.983+0000 7f0da8c65700 1 --2- 172.21.15.179:0/1363152109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da406ed80 0x7f0da40cc500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0da4072a60 tx=0x7f0d94009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ca0c7f70ce9c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 Processor -- start 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 -- start start 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 0x7f669415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669415f810 con 0x7f6694144830 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66940cb8b0 con 0x7f66940cb2f0 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6698b3c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940ce3a0 con 0x7f669406c2d0 2024-05-08T22:33:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f66937fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6694144830 0x7f669406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33592/0 (socket says 172.21.15.179:33592) 2024-05-08T22:33:42.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f66937fe700 1 -- 172.21.15.179:0/172646584 learned_addr learned my addr 172.21.15.179:0/172646584 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:42.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 Processor -- start 2024-05-08T22:33:42.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 -- start start 2024-05-08T22:33:42.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb28013ae0 con 0x7fcb3406bbc0 2024-05-08T22:33:42.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb30ff9700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 0x7fcb20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:42.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6692ffd700 1 --2- 172.21.15.179:0/172646584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 0x7f669415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcb280842e0 con 0x7fcb3406bbc0 2024-05-08T22:33:42.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb337fe700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 0x7fcb20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:42.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb340719a0 con 0x7fcb3406bbc0 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda1006bbc0 0x7fda1014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda10159d50 0x7fda1006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101551b0 con 0x7fda101536f0 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10155c60 con 0x7fda10159d50 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda163e7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda101553c0 con 0x7fda1006bbc0 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35322/0 (socket says 172.21.15.179:35322) 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 -- 172.21.15.179:0/2339776319 learned_addr learned my addr 172.21.15.179:0/2339776319 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda153e5700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda1006bbc0 0x7fda1014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda15be6700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda10159d50 0x7fda1006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6692ffd700 1 -- 172.21.15.179:0/172646584 >> v1:172.21.15.195:6789/0 conn(0x7f66940cb2f0 legacy=0x7f669415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6692ffd700 1 -- 172.21.15.179:0/172646584 >> v1:172.21.15.179:6789/0 conn(0x7f6694144830 legacy=0x7f669406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6692ffd700 1 -- 172.21.15.179:0/172646584 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940cbb90 con 0x7f669406c2d0 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 -- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda1006bbc0 msgr2=0x7fda1014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda1006bbc0 0x7fda1014e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 -- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda10159d50 msgr2=0x7fda1006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda10159d50 0x7fda1006e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 -- 172.21.15.179:0/2339776319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08009830 con 0x7fda101536f0 2024-05-08T22:33:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fcb337fe700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 0x7fcb20068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b3a60 tx=0x7fcb24008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7fda14be4700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c4e0 tx=0x7fda0000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.987+0000 7f6692ffd700 1 --2- 172.21.15.179:0/172646584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 0x7f669415f2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6688009800 tx=0x7f6688009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57697cdf3f65d75e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da2ffd700 1 -- 172.21.15.179:0/1363152109 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94011830 con 0x7f0da406ed80 2024-05-08T22:33:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da2ffd700 1 -- 172.21.15.179:0/1363152109 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94011990 con 0x7f0da406ed80 2024-05-08T22:33:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da2ffd700 1 -- 172.21.15.179:0/1363152109 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94011d20 con 0x7f0da406ed80 2024-05-08T22:33:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7face4051060 con 0x7facf0169b90 2024-05-08T22:33:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 -- 172.21.15.179:0/1363152109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da406ed80 msgr2=0x7f0da40cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 --2- 172.21.15.179:0/1363152109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da406ed80 0x7f0da40cc500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0da4072a60 tx=0x7f0d94009b30 comp rx=0 tx=0).stop 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 -- 172.21.15.179:0/1363152109 shutdown_connections 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 --2- 172.21.15.179:0/1363152109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da406ed80 0x7f0da40cc500 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 --2- 172.21.15.179:0/1363152109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da40ce230 0x7f0da406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 --2- 172.21.15.179:0/1363152109 >> v2:172.21.15.195:3300/0 conn(0x7f0da4159400 0x7f0da40ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 -- 172.21.15.179:0/1363152109 >> 172.21.15.179:0/1363152109 conn(0x7f0da4070d30 msgr2=0x7f0da4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 -- 172.21.15.179:0/1363152109 shutdown_connections 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.991+0000 7f0da9466700 1 -- 172.21.15.179:0/1363152109 wait complete. 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 Processor -- start 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 -- start start 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0da406ed80 0x7f0da4151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da4159400 0x7f0da415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415b4b0 con 0x7f0da40ce230 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da4154bd0 con 0x7f0da406ed80 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da9466700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da4154eb0 con 0x7f0da4159400 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35328/0 (socket says 172.21.15.179:35328) 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 -- 172.21.15.179:0/2073891177 learned_addr learned my addr 172.21.15.179:0/2073891177 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da3fff700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0da406ed80 0x7f0da4151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da8c65700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da4159400 0x7f0da415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 -- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da4159400 msgr2=0x7f0da415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da4159400 0x7f0da415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 -- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0da406ed80 msgr2=0x7f0da4151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0da406ed80 0x7f0da4151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 -- 172.21.15.179:0/2073891177 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980094b0 con 0x7f0da40ce230 2024-05-08T22:33:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.995+0000 7f0da37fe700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c011850 tx=0x7f0d9c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00011970 con 0x7fda101536f0 2024-05-08T22:33:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00011af0 con 0x7fda101536f0 2024-05-08T22:33:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda101556a0 con 0x7fda101536f0 2024-05-08T22:33:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00011ea0 con 0x7fda101536f0 2024-05-08T22:33:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1013b070 con 0x7fda101536f0 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7f66927fc700 1 -- 172.21.15.179:0/172646584 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688004230 con 0x7f669406c2d0 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7f66927fc700 1 -- 172.21.15.179:0/172646584 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688004390 con 0x7f669406c2d0 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7f66927fc700 1 -- 172.21.15.179:0/172646584 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880036c0 con 0x7f669406c2d0 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7f6698b3c700 1 -- 172.21.15.179:0/172646584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 msgr2=0x7f669415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7f6698b3c700 1 --2- 172.21.15.179:0/172646584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 0x7f669415f2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6688009800 tx=0x7f6688009b10 comp rx=0 tx=0).stop 2024-05-08T22:33:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:42.999+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb28051240 con 0x7fcb3406bbc0 2024-05-08T22:33:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 -- 172.21.15.179:0/172646584 shutdown_connections 2024-05-08T22:33:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 --2- 172.21.15.179:0/172646584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f669406c2d0 0x7f669415f2d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 -- 172.21.15.179:0/172646584 >> 172.21.15.179:0/172646584 conn(0x7f6694070d30 msgr2=0x7f6694071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda000193f0 con 0x7fda101536f0 2024-05-08T22:33:43.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7fda05ffb700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 0x7fd9f4068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda00087c30 con 0x7fda101536f0 2024-05-08T22:33:43.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 -- 172.21.15.179:0/172646584 shutdown_connections 2024-05-08T22:33:43.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7fda153e5700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 0x7fd9f4068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7fda153e5700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 0x7fd9f4068c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda08000f80 tx=0x7fda08012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 -- 172.21.15.179:0/172646584 wait complete. 2024-05-08T22:33:43.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.003+0000 7f6698b3c700 1 Processor -- start 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 -- start start 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 0x7f66940b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f66937fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f66937fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52936/0 (socket says 172.21.15.179:52936) 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:53442/0 (socket says 172.21.15.179:53442) 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 0x7f66940b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f66937fe700 1 -- 172.21.15.179:0/4141946147 learned_addr learned my addr 172.21.15.179:0/4141946147 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66941ffab0 con 0x7f6694144830 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940abf80 con 0x7f669406c2d0 2024-05-08T22:33:43.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940ac260 con 0x7f66940cb2f0 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 -- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 msgr2=0x7f66940b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 -- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 msgr2=0x7f669413f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880094b0 con 0x7f6694144830 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6692ffd700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f66937fe700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:43.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.007+0000 7f6693fff700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 0x7f66940b2c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6684000f60 tx=0x7f668400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c019040 con 0x7f0da40ce230 2024-05-08T22:33:43.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d940094b0 con 0x7f0da40ce230 2024-05-08T22:33:43.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da415b9e0 con 0x7f0da40ce230 2024-05-08T22:33:43.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1004e700 con 0x7fda101536f0 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c017ae0 con 0x7f0da40ce230 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c017e50 con 0x7f0da40ce230 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d9c028410 con 0x7f0da40ce230 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 0x7f0d90068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.011+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d9c08d760 con 0x7f0da40ce230 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.015+0000 7f0da3fff700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 0x7f0d90068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.015+0000 7f0da3fff700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 0x7f0d90068c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d00 tx=0x7f0d98009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840054e0 con 0x7f6694144830 2024-05-08T22:33:43.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66940ac540 con 0x7f6694144830 2024-05-08T22:33:43.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d84005310 con 0x7f0da40ce230 2024-05-08T22:33:43.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669413fa60 con 0x7f6694144830 2024-05-08T22:33:43.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66840036a0 con 0x7f6694144830 2024-05-08T22:33:43.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684003a50 con 0x7f6694144830 2024-05-08T22:33:43.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66941ffbf0 con 0x7f6694144830 2024-05-08T22:33:43.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.019+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda00054d90 con 0x7fda101536f0 2024-05-08T22:33:43.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.027+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0d9c05a950 con 0x7f0da40ce230 2024-05-08T22:33:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.031+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6684003bb0 con 0x7f6694144830 2024-05-08T22:33:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.031+0000 7f667bfff700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 0x7f667c068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.031+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6684083d70 con 0x7f6694144830 2024-05-08T22:33:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.031+0000 7f66937fe700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 0x7f667c068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.031+0000 7f66937fe700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 0x7f667c068e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f668c0021a0 tx=0x7f668c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.035+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6684050e60 con 0x7f6694144830 2024-05-08T22:33:43.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.247+0000 7f31922fd700 1 -- 172.21.15.179:0/2812227450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f318c04e700 con 0x7f318c0d05f0 2024-05-08T22:33:43.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.247+0000 7f3181ffb700 1 -- 172.21.15.179:0/2812227450 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f317c054b90 con 0x7f318c0d05f0 2024-05-08T22:33:43.249 INFO:teuthology.orchestra.run.smithi179.stdout:64424509450 2024-05-08T22:33:43.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 msgr2=0x7f3170068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 0x7f3170068cc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f80 tx=0x7f3184014000 comp rx=0 tx=0).stop 2024-05-08T22:33:43.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 msgr2=0x7f318c158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b520 tx=0x7f317c00b830 comp rx=0 tx=0).stop 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 shutdown_connections 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3170066800 0x7f3170068cc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f318c15aed0 0x7f318c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c0d05f0 0x7f318c158520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 --2- 172.21.15.179:0/2812227450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f318c0ce0d0 0x7f318c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 >> 172.21.15.179:0/2812227450 conn(0x7f318c0b2940 msgr2=0x7f318c0c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 shutdown_connections 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7f31777fe700 1 -- 172.21.15.179:0/2812227450 wait complete. 2024-05-08T22:33:43.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7fa688016700 1 -- 172.21.15.179:0/3536411042 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa68015f350 con 0x7fa68006c100 2024-05-08T22:33:43.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.251+0000 7fa6767fc700 1 -- 172.21.15.179:0/3536411042 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa68015f350 con 0x7fa68006c100 2024-05-08T22:33:43.255 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:43.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 msgr2=0x7fa664068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 0x7fa664068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa68015d710 tx=0x7fa67c008040 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 msgr2=0x7fa68014cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa68014cf10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c7f0 tx=0x7fa67000cb00 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 shutdown_connections 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa664066730 0x7fa664068bf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa680161280 0x7fa680165fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6800adea0 0x7fa68014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 --2- 172.21.15.179:0/3536411042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa68006c100 0x7fa68014cf10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 >> 172.21.15.179:0/3536411042 conn(0x7fa680070d30 msgr2=0x7fa68006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 shutdown_connections 2024-05-08T22:33:43.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.259+0000 7fa663fff700 1 -- 172.21.15.179:0/3536411042 wait complete. 2024-05-08T22:33:43.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.1 2024-05-08T22:33:43.266 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.271+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f29900055c0 con 0x7f29a8143620 2024-05-08T22:33:43.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f299effd700 1 -- 172.21.15.179:0/239708254 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29a005b110 con 0x7f29a8143620 2024-05-08T22:33:43.274 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:43.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-05-08T22:33:43.275 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 msgr2=0x7f298c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 0x7f298c068cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005930 tx=0x7f29a4009040 comp rx=0 tx=0).stop 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 msgr2=0x7f29a80c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29a0014d50 tx=0x7f29a0004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 shutdown_connections 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f298c066800 0x7f298c068cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8143620 0x7f29a80c81c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a80cdfc0 0x7f29a80c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 --2- 172.21.15.179:0/239708254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a80ca8b0 0x7f29a80b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 >> 172.21.15.179:0/239708254 conn(0x7f29a80cc5d0 msgr2=0x7f29a815a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 shutdown_connections 2024-05-08T22:33:43.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.275+0000 7f29af03a700 1 -- 172.21.15.179:0/239708254 wait complete. 2024-05-08T22:33:43.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-05-08T22:33:43.286 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f669404e700 con 0x7f6694144830 2024-05-08T22:33:43.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7facf004e700 con 0x7facf0169b90 2024-05-08T22:33:43.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f667bfff700 1 -- 172.21.15.179:0/4141946147 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6684017070 con 0x7f6694144830 2024-05-08T22:33:43.315 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:43.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7facdeffd700 1 -- 172.21.15.179:0/3193020682 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7face4017090 con 0x7facf0169b90 2024-05-08T22:33:43.315 INFO:teuthology.orchestra.run.smithi179.stdout:60129542153 2024-05-08T22:33:43.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb340cc220 con 0x7fcb3406bbc0 2024-05-08T22:33:43.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 msgr2=0x7f667c068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 0x7f667c068e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f668c0021a0 tx=0x7f668c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 msgr2=0x7f66940b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 0x7f66940b2c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6684000f60 tx=0x7f668400c820 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 shutdown_connections 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f667c066940 0x7f667c068e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6694144830 0x7f66940b2c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66940cb2f0 0x7f66940b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 --2- 172.21.15.179:0/4141946147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f669406c2d0 0x7f669413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 >> 172.21.15.179:0/4141946147 conn(0x7f6694070d30 msgr2=0x7f669406b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 shutdown_connections 2024-05-08T22:33:43.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.315+0000 7f6698b3c700 1 -- 172.21.15.179:0/4141946147 wait complete. 2024-05-08T22:33:43.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb30ff9700 1 -- 172.21.15.179:0/3844900700 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb28054860 con 0x7fcb3406bbc0 2024-05-08T22:33:43.319 INFO:teuthology.orchestra.run.smithi179.stdout:64424509449 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 msgr2=0x7facd8069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 0x7facd8069050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7face0007fd0 tx=0x7face0007de0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 msgr2=0x7facf016a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7face400e880 tx=0x7face400eb90 comp rx=0 tx=0).stop 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 shutdown_connections 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facd8066b90 0x7facd8069050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facf0169f70 0x7facf0167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0169b90 0x7facf016a350 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 --2- 172.21.15.179:0/3193020682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facf006c100 0x7facf0088bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 >> 172.21.15.179:0/3193020682 conn(0x7facf0070d30 msgr2=0x7facf0165340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 shutdown_connections 2024-05-08T22:33:43.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7facef59e700 1 -- 172.21.15.179:0/3193020682 wait complete. 2024-05-08T22:33:43.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 msgr2=0x7fcb20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 0x7fcb20068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b3a60 tx=0x7fcb24008030 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 msgr2=0x7fcb340cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e760 tx=0x7fcb2800ea70 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 shutdown_connections 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb20066730 0x7fcb20068bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb340b3620 0x7fcb3414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb3415d120 0x7fcb340cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 --2- 172.21.15.179:0/3844900700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb3406bbc0 0x7fcb340cb390 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e760 tx=0x7fcb2800ea70 comp rx=0 tx=0).stop 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 >> 172.21.15.179:0/3844900700 conn(0x7fcb34070d30 msgr2=0x7fcb340ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 shutdown_connections 2024-05-08T22:33:43.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.319+0000 7fcb392a0700 1 -- 172.21.15.179:0/3844900700 wait complete. 2024-05-08T22:33:43.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2024-05-08T22:33:43.329 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.2 2024-05-08T22:33:43.336 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.335+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fda100b3020 con 0x7fda101536f0 2024-05-08T22:33:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.335+0000 7fda05ffb700 1 -- 172.21.15.179:0/2339776319 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda000583b0 con 0x7fda101536f0 2024-05-08T22:33:43.337 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.335+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0d84005890 con 0x7f0da40ce230 2024-05-08T22:33:43.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-05-08T22:33:43.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da0ff9700 1 -- 172.21.15.179:0/2073891177 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d9c05df70 con 0x7f0da40ce230 2024-05-08T22:33:43.338 INFO:teuthology.orchestra.run.smithi179.stdout:60129542152 2024-05-08T22:33:43.338 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 msgr2=0x7fd9f4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 0x7fd9f4068c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda08000f80 tx=0x7fda08012000 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 msgr2=0x7fda10154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c4e0 tx=0x7fda0000c7f0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 shutdown_connections 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9f4066780 0x7fd9f4068c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda10159d50 0x7fda1006e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda101536f0 0x7fda10154c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 --2- 172.21.15.179:0/2339776319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda1006bbc0 0x7fda1014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 >> 172.21.15.179:0/2339776319 conn(0x7fda10070d30 msgr2=0x7fda100ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 shutdown_connections 2024-05-08T22:33:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7fda163e7700 1 -- 172.21.15.179:0/2339776319 wait complete. 2024-05-08T22:33:43.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 msgr2=0x7f0d90068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 0x7f0d90068c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d00 tx=0x7f0d98009240 comp rx=0 tx=0).stop 2024-05-08T22:33:43.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 msgr2=0x7f0da41409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c011850 tx=0x7f0d9c011b60 comp rx=0 tx=0).stop 2024-05-08T22:33:43.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 shutdown_connections 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d900667b0 0x7f0d90068c70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0da4159400 0x7f0da415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da40ce230 0x7f0da41409e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 --2- 172.21.15.179:0/2073891177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0da406ed80 0x7f0da4151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 >> 172.21.15.179:0/2073891177 conn(0x7f0da4070d30 msgr2=0x7f0da406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 shutdown_connections 2024-05-08T22:33:43.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.339+0000 7f0da9466700 1 -- 172.21.15.179:0/2073891177 wait complete. 2024-05-08T22:33:43.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-05-08T22:33:43.351 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-05-08T22:33:43.355 DEBUG:teuthology.parallel:result is None 2024-05-08T22:33:43.355 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T22:33:43.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:33:43.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.455+0000 7fbe711d2700 1 Processor -- start 2024-05-08T22:33:43.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.455+0000 7fbe711d2700 1 -- start start 2024-05-08T22:33:43.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe640363d0 0x7fbe640576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe64009cb0 con 0x7fbe64038f20 2024-05-08T22:33:43.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe64006950 con 0x7fbe64035ff0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe64006c30 con 0x7fbe640363d0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe709d1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe640363d0 0x7fbe640576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbe64038f20 0x7fbe640e2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33602/0 (socket says 172.21.15.179:33602) 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6b7fe700 1 -- 172.21.15.179:0/2388064917 learned_addr learned my addr 172.21.15.179:0/2388064917 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe709d1700 1 -- 172.21.15.179:0/2388064917 >> v1:172.21.15.195:6789/0 conn(0x7fbe64035ff0 legacy=0x7fbe64038810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe709d1700 1 -- 172.21.15.179:0/2388064917 >> v1:172.21.15.179:6789/0 conn(0x7fbe64038f20 legacy=0x7fbe640e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe709d1700 1 -- 172.21.15.179:0/2388064917 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64007e30 con 0x7fbe640363d0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6affd700 1 -- 172.21.15.179:0/2388064917 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489512115 0 0) 0x7fbe64009cb0 con 0x7fbe64038f20 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe709d1700 1 --2- 172.21.15.179:0/2388064917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640576b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c04f480 tx=0x7fbe6c04f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ae450693f5cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6affd700 1 -- 172.21.15.179:0/2388064917 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c067250 con 0x7fbe640363d0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6affd700 1 -- 172.21.15.179:0/2388064917 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c0673b0 con 0x7fbe640363d0 2024-05-08T22:33:43.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe6affd700 1 -- 172.21.15.179:0/2388064917 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c067960 con 0x7fbe640363d0 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- 172.21.15.179:0/2388064917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 msgr2=0x7fbe640576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 --2- 172.21.15.179:0/2388064917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640576b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c04f480 tx=0x7fbe6c04f710 comp rx=0 tx=0).stop 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- 172.21.15.179:0/2388064917 shutdown_connections 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 --2- 172.21.15.179:0/2388064917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640576b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- 172.21.15.179:0/2388064917 >> 172.21.15.179:0/2388064917 conn(0x7fbe64020120 msgr2=0x7fbe64056d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- 172.21.15.179:0/2388064917 shutdown_connections 2024-05-08T22:33:43.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- 172.21.15.179:0/2388064917 wait complete. 2024-05-08T22:33:43.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 Processor -- start 2024-05-08T22:33:43.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.459+0000 7fbe711d2700 1 -- start start 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe64035ff0 0x7fbe640cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe64038f20 0x7fbe640e5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640e29e0 con 0x7fbe640363d0 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640e2080 con 0x7fbe64035ff0 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64038710 con 0x7fbe64038f20 2024-05-08T22:33:43.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35358/0 (socket says 172.21.15.179:35358) 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 -- 172.21.15.179:0/1119332135 learned_addr learned my addr 172.21.15.179:0/1119332135 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe709d1700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe64038f20 0x7fbe640e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6bfff700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe64035ff0 0x7fbe640cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 -- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe64038f20 msgr2=0x7fbe640e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe64038f20 0x7fbe640e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 -- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe64035ff0 msgr2=0x7fbe640cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe64035ff0 0x7fbe640cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 -- 172.21.15.179:0/1119332135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640e2320 con 0x7fbe640363d0 2024-05-08T22:33:43.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe6b7fe700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c580 tx=0x7fbe5800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004ca0 con 0x7fbe640363d0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe58004e20 con 0x7fbe640363d0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe640e2600 con 0x7fbe640363d0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007310 con 0x7fbe640363d0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe640e0ad0 con 0x7fbe640363d0 2024-05-08T22:33:43.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.463+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe640e2b20 con 0x7fbe640363d0 2024-05-08T22:33:43.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.467+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe58007470 con 0x7fbe640363d0 2024-05-08T22:33:43.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.467+0000 7fbe68ff9700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 0x7fbe50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.467+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe58084090 con 0x7fbe640363d0 2024-05-08T22:33:43.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.467+0000 7fbe6bfff700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 0x7fbe50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.467+0000 7fbe6bfff700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 0x7fbe50068bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009980 tx=0x7fbe600090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.475+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbe640e2b20 con 0x7fbe640363d0 2024-05-08T22:33:43.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.627+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe640e2b20 con 0x7fbe50066730 2024-05-08T22:33:43.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.647+0000 7fbe68ff9700 1 -- 172.21.15.179:0/1119332135 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345273 (secure 0 0 0) 0x7fbe640e2b20 con 0x7fbe50066730 2024-05-08T22:33:43.654 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 msgr2=0x7fbe50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 0x7fbe50068bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009980 tx=0x7fbe600090e0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 msgr2=0x7fbe640e0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c580 tx=0x7fbe5800c890 comp rx=0 tx=0).stop 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 shutdown_connections 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe50066730 0x7fbe50068bf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe64038f20 0x7fbe640e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640363d0 0x7fbe640e0590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 --2- 172.21.15.179:0/1119332135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe64035ff0 0x7fbe640cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 >> 172.21.15.179:0/1119332135 conn(0x7fbe64020120 msgr2=0x7fbe6400c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 shutdown_connections 2024-05-08T22:33:43.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.655+0000 7fbe711d2700 1 -- 172.21.15.179:0/1119332135 wait complete. 2024-05-08T22:33:43.658 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:33:43.671 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-08T22:33:43.385624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235888,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738832,"statfs":{"total":773094113280,"available":772852563968,"internally_reserved":0,"allocated":3416064,"data_stored":2046228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005565"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.932430+0000","last_change":"2024-05-08T22:33:14.533515+0000","last_active":"2024-05-08T22:33:18.932430+0000","last_peered":"2024-05-08T22:33:18.932430+0000","last_clean":"2024-05-08T22:33:18.932430+0000","last_became_active":"2024-05-08T22:33:14.533293+0000","last_became_peered":"2024-05-08T22:33:14.533293+0000","last_unstale":"2024-05-08T22:33:18.932430+0000","last_undegraded":"2024-05-08T22:33:18.932430+0000","last_fullsized":"2024-05-08T22:33:18.932430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312810+0000","last_change":"2024-05-08T22:33:14.919770+0000","last_active":"2024-05-08T22:33:19.312810+0000","last_peered":"2024-05-08T22:33:19.312810+0000","last_clean":"2024-05-08T22:33:19.312810+0000","last_became_active":"2024-05-08T22:33:14.917629+0000","last_became_peered":"2024-05-08T22:33:14.917629+0000","last_unstale":"2024-05-08T22:33:19.312810+0000","last_undegraded":"2024-05-08T22:33:19.312810+0000","last_fullsized":"2024-05-08T22:33:19.312810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311922+0000","last_change":"2024-05-08T22:33:19.007654+0000","last_active":"2024-05-08T22:33:19.311922+0000","last_peered":"2024-05-08T22:33:19.311922+0000","last_clean":"2024-05-08T22:33:19.311922+0000","last_became_active":"2024-05-08T22:33:19.006811+0000","last_became_peered":"2024-05-08T22:33:19.006811+0000","last_unstale":"2024-05-08T22:33:19.311922+0000","last_undegraded":"2024-05-08T22:33:19.311922+0000","last_fullsized":"2024-05-08T22:33:19.311922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:28.811139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944044+0000","last_change":"2024-05-08T22:33:14.528074+0000","last_active":"2024-05-08T22:33:18.944044+0000","last_peered":"2024-05-08T22:33:18.944044+0000","last_clean":"2024-05-08T22:33:18.944044+0000","last_became_active":"2024-05-08T22:33:14.527773+0000","last_became_peered":"2024-05-08T22:33:14.527773+0000","last_unstale":"2024-05-08T22:33:18.944044+0000","last_undegraded":"2024-05-08T22:33:18.944044+0000","last_fullsized":"2024-05-08T22:33:18.944044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:53.046188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931850+0000","last_change":"2024-05-08T22:33:14.932746+0000","last_active":"2024-05-08T22:33:18.931850+0000","last_peered":"2024-05-08T22:33:18.931850+0000","last_clean":"2024-05-08T22:33:18.931850+0000","last_became_active":"2024-05-08T22:33:14.932511+0000","last_became_peered":"2024-05-08T22:33:14.932511+0000","last_unstale":"2024-05-08T22:33:18.931850+0000","last_undegraded":"2024-05-08T22:33:18.931850+0000","last_fullsized":"2024-05-08T22:33:18.931850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:05.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313017+0000","last_change":"2024-05-08T22:33:18.946740+0000","last_active":"2024-05-08T22:33:19.313017+0000","last_peered":"2024-05-08T22:33:19.313017+0000","last_clean":"2024-05-08T22:33:19.313017+0000","last_became_active":"2024-05-08T22:33:18.946369+0000","last_became_peered":"2024-05-08T22:33:18.946369+0000","last_unstale":"2024-05-08T22:33:19.313017+0000","last_undegraded":"2024-05-08T22:33:19.313017+0000","last_fullsized":"2024-05-08T22:33:19.313017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999447+0000","last_change":"2024-05-08T22:33:14.520155+0000","last_active":"2024-05-08T22:33:18.999447+0000","last_peered":"2024-05-08T22:33:18.999447+0000","last_clean":"2024-05-08T22:33:18.999447+0000","last_became_active":"2024-05-08T22:33:14.519845+0000","last_became_peered":"2024-05-08T22:33:14.519845+0000","last_unstale":"2024-05-08T22:33:18.999447+0000","last_undegraded":"2024-05-08T22:33:18.999447+0000","last_fullsized":"2024-05-08T22:33:18.999447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:03.999595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934686+0000","last_change":"2024-05-08T22:33:14.920443+0000","last_active":"2024-05-08T22:33:18.934686+0000","last_peered":"2024-05-08T22:33:18.934686+0000","last_clean":"2024-05-08T22:33:18.934686+0000","last_became_active":"2024-05-08T22:33:14.920034+0000","last_became_peered":"2024-05-08T22:33:14.920034+0000","last_unstale":"2024-05-08T22:33:18.934686+0000","last_undegraded":"2024-05-08T22:33:18.934686+0000","last_fullsized":"2024-05-08T22:33:18.934686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314786+0000","last_change":"2024-05-08T22:33:18.948680+0000","last_active":"2024-05-08T22:33:19.314786+0000","last_peered":"2024-05-08T22:33:19.314786+0000","last_clean":"2024-05-08T22:33:19.314786+0000","last_became_active":"2024-05-08T22:33:18.948252+0000","last_became_peered":"2024-05-08T22:33:18.948252+0000","last_unstale":"2024-05-08T22:33:19.314786+0000","last_undegraded":"2024-05-08T22:33:19.314786+0000","last_fullsized":"2024-05-08T22:33:19.314786+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:20.694843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999183+0000","last_change":"2024-05-08T22:33:14.526338+0000","last_active":"2024-05-08T22:33:18.999183+0000","last_peered":"2024-05-08T22:33:18.999183+0000","last_clean":"2024-05-08T22:33:18.999183+0000","last_became_active":"2024-05-08T22:33:14.520436+0000","last_became_peered":"2024-05-08T22:33:14.520436+0000","last_unstale":"2024-05-08T22:33:18.999183+0000","last_undegraded":"2024-05-08T22:33:18.999183+0000","last_fullsized":"2024-05-08T22:33:18.999183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:15.246381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935315+0000","last_change":"2024-05-08T22:33:14.919811+0000","last_active":"2024-05-08T22:33:18.935315+0000","last_peered":"2024-05-08T22:33:18.935315+0000","last_clean":"2024-05-08T22:33:18.935315+0000","last_became_active":"2024-05-08T22:33:14.919249+0000","last_became_peered":"2024-05-08T22:33:14.919249+0000","last_unstale":"2024-05-08T22:33:18.935315+0000","last_undegraded":"2024-05-08T22:33:18.935315+0000","last_fullsized":"2024-05-08T22:33:18.935315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:34.252236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313998+0000","last_change":"2024-05-08T22:33:18.942435+0000","last_active":"2024-05-08T22:33:19.313998+0000","last_peered":"2024-05-08T22:33:19.313998+0000","last_clean":"2024-05-08T22:33:19.313998+0000","last_became_active":"2024-05-08T22:33:18.942040+0000","last_became_peered":"2024-05-08T22:33:18.942040+0000","last_unstale":"2024-05-08T22:33:19.313998+0000","last_undegraded":"2024-05-08T22:33:19.313998+0000","last_fullsized":"2024-05-08T22:33:19.313998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927785+0000","last_change":"2024-05-08T22:33:14.913632+0000","last_active":"2024-05-08T22:33:18.927785+0000","last_peered":"2024-05-08T22:33:18.927785+0000","last_clean":"2024-05-08T22:33:18.927785+0000","last_became_active":"2024-05-08T22:33:14.913040+0000","last_became_peered":"2024-05-08T22:33:14.913040+0000","last_unstale":"2024-05-08T22:33:18.927785+0000","last_undegraded":"2024-05-08T22:33:18.927785+0000","last_fullsized":"2024-05-08T22:33:18.927785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998991+0000","last_change":"2024-05-08T22:33:14.526397+0000","last_active":"2024-05-08T22:33:18.998991+0000","last_peered":"2024-05-08T22:33:18.998991+0000","last_clean":"2024-05-08T22:33:18.998991+0000","last_became_active":"2024-05-08T22:33:14.519761+0000","last_became_peered":"2024-05-08T22:33:14.519761+0000","last_unstale":"2024-05-08T22:33:18.998991+0000","last_undegraded":"2024-05-08T22:33:18.998991+0000","last_fullsized":"2024-05-08T22:33:18.998991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315399+0000","last_change":"2024-05-08T22:33:18.944230+0000","last_active":"2024-05-08T22:33:19.315399+0000","last_peered":"2024-05-08T22:33:19.315399+0000","last_clean":"2024-05-08T22:33:19.315399+0000","last_became_active":"2024-05-08T22:33:18.943852+0000","last_became_peered":"2024-05-08T22:33:18.943852+0000","last_unstale":"2024-05-08T22:33:19.315399+0000","last_undegraded":"2024-05-08T22:33:19.315399+0000","last_fullsized":"2024-05-08T22:33:19.315399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312132+0000","last_change":"2024-05-08T22:33:14.912752+0000","last_active":"2024-05-08T22:33:19.312132+0000","last_peered":"2024-05-08T22:33:19.312132+0000","last_clean":"2024-05-08T22:33:19.312132+0000","last_became_active":"2024-05-08T22:33:14.912455+0000","last_became_peered":"2024-05-08T22:33:14.912455+0000","last_unstale":"2024-05-08T22:33:19.312132+0000","last_undegraded":"2024-05-08T22:33:19.312132+0000","last_fullsized":"2024-05-08T22:33:19.312132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934257+0000","last_change":"2024-05-08T22:33:14.525254+0000","last_active":"2024-05-08T22:33:18.934257+0000","last_peered":"2024-05-08T22:33:18.934257+0000","last_clean":"2024-05-08T22:33:18.934257+0000","last_became_active":"2024-05-08T22:33:14.524663+0000","last_became_peered":"2024-05-08T22:33:14.524663+0000","last_unstale":"2024-05-08T22:33:18.934257+0000","last_undegraded":"2024-05-08T22:33:18.934257+0000","last_fullsized":"2024-05-08T22:33:18.934257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314812+0000","last_change":"2024-05-08T22:33:18.947596+0000","last_active":"2024-05-08T22:33:19.314812+0000","last_peered":"2024-05-08T22:33:19.314812+0000","last_clean":"2024-05-08T22:33:19.314812+0000","last_became_active":"2024-05-08T22:33:18.947225+0000","last_became_peered":"2024-05-08T22:33:18.947225+0000","last_unstale":"2024-05-08T22:33:19.314812+0000","last_undegraded":"2024-05-08T22:33:19.314812+0000","last_fullsized":"2024-05-08T22:33:19.314812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312379+0000","last_change":"2024-05-08T22:33:14.906568+0000","last_active":"2024-05-08T22:33:19.312379+0000","last_peered":"2024-05-08T22:33:19.312379+0000","last_clean":"2024-05-08T22:33:19.312379+0000","last_became_active":"2024-05-08T22:33:14.906359+0000","last_became_peered":"2024-05-08T22:33:14.906359+0000","last_unstale":"2024-05-08T22:33:19.312379+0000","last_undegraded":"2024-05-08T22:33:19.312379+0000","last_fullsized":"2024-05-08T22:33:19.312379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:47.862614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934403+0000","last_change":"2024-05-08T22:33:14.530133+0000","last_active":"2024-05-08T22:33:18.934403+0000","last_peered":"2024-05-08T22:33:18.934403+0000","last_clean":"2024-05-08T22:33:18.934403+0000","last_became_active":"2024-05-08T22:33:14.529854+0000","last_became_peered":"2024-05-08T22:33:14.529854+0000","last_unstale":"2024-05-08T22:33:18.934403+0000","last_undegraded":"2024-05-08T22:33:18.934403+0000","last_fullsized":"2024-05-08T22:33:18.934403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:59.313380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448755+0000","last_change":"2024-05-08T22:33:19.004993+0000","last_active":"2024-05-08T22:33:19.448755+0000","last_peered":"2024-05-08T22:33:19.448755+0000","last_clean":"2024-05-08T22:33:19.448755+0000","last_became_active":"2024-05-08T22:33:19.004152+0000","last_became_peered":"2024-05-08T22:33:19.004152+0000","last_unstale":"2024-05-08T22:33:19.448755+0000","last_undegraded":"2024-05-08T22:33:19.448755+0000","last_fullsized":"2024-05-08T22:33:19.448755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:39.634020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929034+0000","last_change":"2024-05-08T22:33:14.909377+0000","last_active":"2024-05-08T22:33:18.929034+0000","last_peered":"2024-05-08T22:33:18.929034+0000","last_clean":"2024-05-08T22:33:18.929034+0000","last_became_active":"2024-05-08T22:33:14.909047+0000","last_became_peered":"2024-05-08T22:33:14.909047+0000","last_unstale":"2024-05-08T22:33:18.929034+0000","last_undegraded":"2024-05-08T22:33:18.929034+0000","last_fullsized":"2024-05-08T22:33:18.929034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:32.048397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998562+0000","last_change":"2024-05-08T22:33:14.529685+0000","last_active":"2024-05-08T22:33:18.998562+0000","last_peered":"2024-05-08T22:33:18.998562+0000","last_clean":"2024-05-08T22:33:18.998562+0000","last_became_active":"2024-05-08T22:33:14.529417+0000","last_became_peered":"2024-05-08T22:33:14.529417+0000","last_unstale":"2024-05-08T22:33:18.998562+0000","last_undegraded":"2024-05-08T22:33:18.998562+0000","last_fullsized":"2024-05-08T22:33:18.998562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.786143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314895+0000","last_change":"2024-05-08T22:33:18.948441+0000","last_active":"2024-05-08T22:33:19.314895+0000","last_peered":"2024-05-08T22:33:19.314895+0000","last_clean":"2024-05-08T22:33:19.314895+0000","last_became_active":"2024-05-08T22:33:18.948006+0000","last_became_peered":"2024-05-08T22:33:18.948006+0000","last_unstale":"2024-05-08T22:33:19.314895+0000","last_undegraded":"2024-05-08T22:33:19.314895+0000","last_fullsized":"2024-05-08T22:33:19.314895+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936140+0000","last_change":"2024-05-08T22:33:14.915374+0000","last_active":"2024-05-08T22:33:18.936140+0000","last_peered":"2024-05-08T22:33:18.936140+0000","last_clean":"2024-05-08T22:33:18.936140+0000","last_became_active":"2024-05-08T22:33:14.915035+0000","last_became_peered":"2024-05-08T22:33:14.915035+0000","last_unstale":"2024-05-08T22:33:18.936140+0000","last_undegraded":"2024-05-08T22:33:18.936140+0000","last_fullsized":"2024-05-08T22:33:18.936140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927757+0000","last_change":"2024-05-08T22:33:14.518968+0000","last_active":"2024-05-08T22:33:18.927757+0000","last_peered":"2024-05-08T22:33:18.927757+0000","last_clean":"2024-05-08T22:33:18.927757+0000","last_became_active":"2024-05-08T22:33:14.518318+0000","last_became_peered":"2024-05-08T22:33:14.518318+0000","last_unstale":"2024-05-08T22:33:18.927757+0000","last_undegraded":"2024-05-08T22:33:18.927757+0000","last_fullsized":"2024-05-08T22:33:18.927757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313504+0000","last_change":"2024-05-08T22:33:19.003063+0000","last_active":"2024-05-08T22:33:19.313504+0000","last_peered":"2024-05-08T22:33:19.313504+0000","last_clean":"2024-05-08T22:33:19.313504+0000","last_became_active":"2024-05-08T22:33:19.002557+0000","last_became_peered":"2024-05-08T22:33:19.002557+0000","last_unstale":"2024-05-08T22:33:19.313504+0000","last_undegraded":"2024-05-08T22:33:19.313504+0000","last_fullsized":"2024-05-08T22:33:19.313504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312756+0000","last_change":"2024-05-08T22:33:14.915824+0000","last_active":"2024-05-08T22:33:19.312756+0000","last_peered":"2024-05-08T22:33:19.312756+0000","last_clean":"2024-05-08T22:33:19.312756+0000","last_became_active":"2024-05-08T22:33:14.915554+0000","last_became_peered":"2024-05-08T22:33:14.915554+0000","last_unstale":"2024-05-08T22:33:19.312756+0000","last_undegraded":"2024-05-08T22:33:19.312756+0000","last_fullsized":"2024-05-08T22:33:19.312756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:10.398283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.937194+0000","last_change":"2024-05-08T22:33:14.526863+0000","last_active":"2024-05-08T22:33:18.937194+0000","last_peered":"2024-05-08T22:33:18.937194+0000","last_clean":"2024-05-08T22:33:18.937194+0000","last_became_active":"2024-05-08T22:33:14.526463+0000","last_became_peered":"2024-05-08T22:33:14.526463+0000","last_unstale":"2024-05-08T22:33:18.937194+0000","last_undegraded":"2024-05-08T22:33:18.937194+0000","last_fullsized":"2024-05-08T22:33:18.937194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311963+0000","last_change":"2024-05-08T22:33:19.005291+0000","last_active":"2024-05-08T22:33:19.311963+0000","last_peered":"2024-05-08T22:33:19.311963+0000","last_clean":"2024-05-08T22:33:19.311963+0000","last_became_active":"2024-05-08T22:33:19.004777+0000","last_became_peered":"2024-05-08T22:33:19.004777+0000","last_unstale":"2024-05-08T22:33:19.311963+0000","last_undegraded":"2024-05-08T22:33:19.311963+0000","last_fullsized":"2024-05-08T22:33:19.311963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:36.904973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448630+0000","last_change":"2024-05-08T22:33:14.911246+0000","last_active":"2024-05-08T22:33:19.448630+0000","last_peered":"2024-05-08T22:33:19.448630+0000","last_clean":"2024-05-08T22:33:19.448630+0000","last_became_active":"2024-05-08T22:33:14.910998+0000","last_became_peered":"2024-05-08T22:33:14.910998+0000","last_unstale":"2024-05-08T22:33:19.448630+0000","last_undegraded":"2024-05-08T22:33:19.448630+0000","last_fullsized":"2024-05-08T22:33:19.448630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999587+0000","last_change":"2024-05-08T22:33:14.529748+0000","last_active":"2024-05-08T22:33:18.999587+0000","last_peered":"2024-05-08T22:33:18.999587+0000","last_clean":"2024-05-08T22:33:18.999587+0000","last_became_active":"2024-05-08T22:33:14.529390+0000","last_became_peered":"2024-05-08T22:33:14.529390+0000","last_unstale":"2024-05-08T22:33:18.999587+0000","last_undegraded":"2024-05-08T22:33:18.999587+0000","last_fullsized":"2024-05-08T22:33:18.999587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:04.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.009380+0000","last_change":"2024-05-08T22:33:19.004064+0000","last_active":"2024-05-08T22:33:19.009380+0000","last_peered":"2024-05-08T22:33:19.009380+0000","last_clean":"2024-05-08T22:33:19.009380+0000","last_became_active":"2024-05-08T22:33:19.003638+0000","last_became_peered":"2024-05-08T22:33:19.003638+0000","last_unstale":"2024-05-08T22:33:19.009380+0000","last_undegraded":"2024-05-08T22:33:19.009380+0000","last_fullsized":"2024-05-08T22:33:19.009380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935398+0000","last_change":"2024-05-08T22:33:14.928058+0000","last_active":"2024-05-08T22:33:18.935398+0000","last_peered":"2024-05-08T22:33:18.935398+0000","last_clean":"2024-05-08T22:33:18.935398+0000","last_became_active":"2024-05-08T22:33:14.927408+0000","last_became_peered":"2024-05-08T22:33:14.927408+0000","last_unstale":"2024-05-08T22:33:18.935398+0000","last_undegraded":"2024-05-08T22:33:18.935398+0000","last_fullsized":"2024-05-08T22:33:18.935398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:22.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934334+0000","last_change":"2024-05-08T22:33:14.530667+0000","last_active":"2024-05-08T22:33:18.934334+0000","last_peered":"2024-05-08T22:33:18.934334+0000","last_clean":"2024-05-08T22:33:18.934334+0000","last_became_active":"2024-05-08T22:33:14.530429+0000","last_became_peered":"2024-05-08T22:33:14.530429+0000","last_unstale":"2024-05-08T22:33:18.934334+0000","last_undegraded":"2024-05-08T22:33:18.934334+0000","last_fullsized":"2024-05-08T22:33:18.934334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:38.212588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448536+0000","last_change":"2024-05-08T22:33:18.945810+0000","last_active":"2024-05-08T22:33:19.448536+0000","last_peered":"2024-05-08T22:33:19.448536+0000","last_clean":"2024-05-08T22:33:19.448536+0000","last_became_active":"2024-05-08T22:33:18.945204+0000","last_became_peered":"2024-05-08T22:33:18.945204+0000","last_unstale":"2024-05-08T22:33:19.448536+0000","last_undegraded":"2024-05-08T22:33:19.448536+0000","last_fullsized":"2024-05-08T22:33:19.448536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:26.949422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930747+0000","last_change":"2024-05-08T22:33:14.911989+0000","last_active":"2024-05-08T22:33:18.930747+0000","last_peered":"2024-05-08T22:33:18.930747+0000","last_clean":"2024-05-08T22:33:18.930747+0000","last_became_active":"2024-05-08T22:33:14.911685+0000","last_became_peered":"2024-05-08T22:33:14.911685+0000","last_unstale":"2024-05-08T22:33:18.930747+0000","last_undegraded":"2024-05-08T22:33:18.930747+0000","last_fullsized":"2024-05-08T22:33:18.930747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934973+0000","last_change":"2024-05-08T22:33:14.524541+0000","last_active":"2024-05-08T22:33:18.934973+0000","last_peered":"2024-05-08T22:33:18.934973+0000","last_clean":"2024-05-08T22:33:18.934973+0000","last_became_active":"2024-05-08T22:33:14.524290+0000","last_became_peered":"2024-05-08T22:33:14.524290+0000","last_unstale":"2024-05-08T22:33:18.934973+0000","last_undegraded":"2024-05-08T22:33:18.934973+0000","last_fullsized":"2024-05-08T22:33:18.934973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448609+0000","last_change":"2024-05-08T22:33:18.951903+0000","last_active":"2024-05-08T22:33:19.448609+0000","last_peered":"2024-05-08T22:33:19.448609+0000","last_clean":"2024-05-08T22:33:19.448609+0000","last_became_active":"2024-05-08T22:33:18.951473+0000","last_became_peered":"2024-05-08T22:33:18.951473+0000","last_unstale":"2024-05-08T22:33:19.448609+0000","last_undegraded":"2024-05-08T22:33:19.448609+0000","last_fullsized":"2024-05-08T22:33:19.448609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:19.057699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448450+0000","last_change":"2024-05-08T22:33:14.911724+0000","last_active":"2024-05-08T22:33:19.448450+0000","last_peered":"2024-05-08T22:33:19.448450+0000","last_clean":"2024-05-08T22:33:19.448450+0000","last_became_active":"2024-05-08T22:33:14.911493+0000","last_became_peered":"2024-05-08T22:33:14.911493+0000","last_unstale":"2024-05-08T22:33:19.448450+0000","last_undegraded":"2024-05-08T22:33:19.448450+0000","last_fullsized":"2024-05-08T22:33:19.448450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925211+0000","last_change":"2024-05-08T22:33:14.524989+0000","last_active":"2024-05-08T22:33:18.925211+0000","last_peered":"2024-05-08T22:33:18.925211+0000","last_clean":"2024-05-08T22:33:18.925211+0000","last_became_active":"2024-05-08T22:33:14.524715+0000","last_became_peered":"2024-05-08T22:33:14.524715+0000","last_unstale":"2024-05-08T22:33:18.925211+0000","last_undegraded":"2024-05-08T22:33:18.925211+0000","last_fullsized":"2024-05-08T22:33:18.925211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448775+0000","last_change":"2024-05-08T22:33:18.952088+0000","last_active":"2024-05-08T22:33:19.448775+0000","last_peered":"2024-05-08T22:33:19.448775+0000","last_clean":"2024-05-08T22:33:19.448775+0000","last_became_active":"2024-05-08T22:33:18.951660+0000","last_became_peered":"2024-05-08T22:33:18.951660+0000","last_unstale":"2024-05-08T22:33:19.448775+0000","last_undegraded":"2024-05-08T22:33:19.448775+0000","last_fullsized":"2024-05-08T22:33:19.448775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:42.700357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930121+0000","last_change":"2024-05-08T22:33:14.911625+0000","last_active":"2024-05-08T22:33:18.930121+0000","last_peered":"2024-05-08T22:33:18.930121+0000","last_clean":"2024-05-08T22:33:18.930121+0000","last_became_active":"2024-05-08T22:33:14.911368+0000","last_became_peered":"2024-05-08T22:33:14.911368+0000","last_unstale":"2024-05-08T22:33:18.930121+0000","last_undegraded":"2024-05-08T22:33:18.930121+0000","last_fullsized":"2024-05-08T22:33:18.930121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998588+0000","last_change":"2024-05-08T22:33:14.529960+0000","last_active":"2024-05-08T22:33:18.998588+0000","last_peered":"2024-05-08T22:33:18.998588+0000","last_clean":"2024-05-08T22:33:18.998588+0000","last_became_active":"2024-05-08T22:33:14.529479+0000","last_became_peered":"2024-05-08T22:33:14.529479+0000","last_unstale":"2024-05-08T22:33:18.998588+0000","last_undegraded":"2024-05-08T22:33:18.998588+0000","last_fullsized":"2024-05-08T22:33:18.998588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:49.272820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.010817+0000","last_change":"2024-05-08T22:33:19.005638+0000","last_active":"2024-05-08T22:33:19.010817+0000","last_peered":"2024-05-08T22:33:19.010817+0000","last_clean":"2024-05-08T22:33:19.010817+0000","last_became_active":"2024-05-08T22:33:19.003876+0000","last_became_peered":"2024-05-08T22:33:19.003876+0000","last_unstale":"2024-05-08T22:33:19.010817+0000","last_undegraded":"2024-05-08T22:33:19.010817+0000","last_fullsized":"2024-05-08T22:33:19.010817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312713+0000","last_change":"2024-05-08T22:33:14.926812+0000","last_active":"2024-05-08T22:33:19.312713+0000","last_peered":"2024-05-08T22:33:19.312713+0000","last_clean":"2024-05-08T22:33:19.312713+0000","last_became_active":"2024-05-08T22:33:14.926536+0000","last_became_peered":"2024-05-08T22:33:14.926536+0000","last_unstale":"2024-05-08T22:33:19.312713+0000","last_undegraded":"2024-05-08T22:33:19.312713+0000","last_fullsized":"2024-05-08T22:33:19.312713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:18.111855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934524+0000","last_change":"2024-05-08T22:33:14.523530+0000","last_active":"2024-05-08T22:33:18.934524+0000","last_peered":"2024-05-08T22:33:18.934524+0000","last_clean":"2024-05-08T22:33:18.934524+0000","last_became_active":"2024-05-08T22:33:14.523193+0000","last_became_peered":"2024-05-08T22:33:14.523193+0000","last_unstale":"2024-05-08T22:33:18.934524+0000","last_undegraded":"2024-05-08T22:33:18.934524+0000","last_fullsized":"2024-05-08T22:33:18.934524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998772+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:33:18.998772+0000","last_peered":"2024-05-08T22:33:18.998772+0000","last_clean":"2024-05-08T22:33:18.998772+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:33:18.998772+0000","last_undegraded":"2024-05-08T22:33:18.998772+0000","last_fullsized":"2024-05-08T22:33:18.998772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007908+0000","last_change":"2024-05-08T22:33:19.006286+0000","last_active":"2024-05-08T22:33:19.007908+0000","last_peered":"2024-05-08T22:33:19.007908+0000","last_clean":"2024-05-08T22:33:19.007908+0000","last_became_active":"2024-05-08T22:33:19.005632+0000","last_became_peered":"2024-05-08T22:33:19.005632+0000","last_unstale":"2024-05-08T22:33:19.007908+0000","last_undegraded":"2024-05-08T22:33:19.007908+0000","last_fullsized":"2024-05-08T22:33:19.007908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:39.728431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999358+0000","last_change":"2024-05-08T22:33:14.931567+0000","last_active":"2024-05-08T22:33:18.999358+0000","last_peered":"2024-05-08T22:33:18.999358+0000","last_clean":"2024-05-08T22:33:18.999358+0000","last_became_active":"2024-05-08T22:33:14.931285+0000","last_became_peered":"2024-05-08T22:33:14.931285+0000","last_unstale":"2024-05-08T22:33:18.999358+0000","last_undegraded":"2024-05-08T22:33:18.999358+0000","last_fullsized":"2024-05-08T22:33:18.999358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925484+0000","last_change":"2024-05-08T22:33:14.523189+0000","last_active":"2024-05-08T22:33:18.925484+0000","last_peered":"2024-05-08T22:33:18.925484+0000","last_clean":"2024-05-08T22:33:18.925484+0000","last_became_active":"2024-05-08T22:33:14.522942+0000","last_became_peered":"2024-05-08T22:33:14.522942+0000","last_unstale":"2024-05-08T22:33:18.925484+0000","last_undegraded":"2024-05-08T22:33:18.925484+0000","last_fullsized":"2024-05-08T22:33:18.925484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007291+0000","last_change":"2024-05-08T22:33:19.003092+0000","last_active":"2024-05-08T22:33:19.007291+0000","last_peered":"2024-05-08T22:33:19.007291+0000","last_clean":"2024-05-08T22:33:19.007291+0000","last_became_active":"2024-05-08T22:33:19.002313+0000","last_became_peered":"2024-05-08T22:33:19.002313+0000","last_unstale":"2024-05-08T22:33:19.007291+0000","last_undegraded":"2024-05-08T22:33:19.007291+0000","last_fullsized":"2024-05-08T22:33:19.007291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:17.965361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934771+0000","last_change":"2024-05-08T22:33:14.927660+0000","last_active":"2024-05-08T22:33:18.934771+0000","last_peered":"2024-05-08T22:33:18.934771+0000","last_clean":"2024-05-08T22:33:18.934771+0000","last_became_active":"2024-05-08T22:33:14.925184+0000","last_became_peered":"2024-05-08T22:33:14.925184+0000","last_unstale":"2024-05-08T22:33:18.934771+0000","last_undegraded":"2024-05-08T22:33:18.934771+0000","last_fullsized":"2024-05-08T22:33:18.934771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:55.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934291+0000","last_change":"2024-05-08T22:33:14.524892+0000","last_active":"2024-05-08T22:33:18.934291+0000","last_peered":"2024-05-08T22:33:18.934291+0000","last_clean":"2024-05-08T22:33:18.934291+0000","last_became_active":"2024-05-08T22:33:14.524584+0000","last_became_peered":"2024-05-08T22:33:14.524584+0000","last_unstale":"2024-05-08T22:33:18.934291+0000","last_undegraded":"2024-05-08T22:33:18.934291+0000","last_fullsized":"2024-05-08T22:33:18.934291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953385+0000","last_change":"2024-05-08T22:33:18.947268+0000","last_active":"2024-05-08T22:33:18.953385+0000","last_peered":"2024-05-08T22:33:18.953385+0000","last_clean":"2024-05-08T22:33:18.953385+0000","last_became_active":"2024-05-08T22:33:18.946822+0000","last_became_peered":"2024-05-08T22:33:18.946822+0000","last_unstale":"2024-05-08T22:33:18.953385+0000","last_undegraded":"2024-05-08T22:33:18.953385+0000","last_fullsized":"2024-05-08T22:33:18.953385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999808+0000","last_change":"2024-05-08T22:33:14.925288+0000","last_active":"2024-05-08T22:33:18.999808+0000","last_peered":"2024-05-08T22:33:18.999808+0000","last_clean":"2024-05-08T22:33:18.999808+0000","last_became_active":"2024-05-08T22:33:14.924982+0000","last_became_peered":"2024-05-08T22:33:14.924982+0000","last_unstale":"2024-05-08T22:33:18.999808+0000","last_undegraded":"2024-05-08T22:33:18.999808+0000","last_fullsized":"2024-05-08T22:33:18.999808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925406+0000","last_change":"2024-05-08T22:33:14.531914+0000","last_active":"2024-05-08T22:33:18.925406+0000","last_peered":"2024-05-08T22:33:18.925406+0000","last_clean":"2024-05-08T22:33:18.925406+0000","last_became_active":"2024-05-08T22:33:14.531550+0000","last_became_peered":"2024-05-08T22:33:14.531550+0000","last_unstale":"2024-05-08T22:33:18.925406+0000","last_undegraded":"2024-05-08T22:33:18.925406+0000","last_fullsized":"2024-05-08T22:33:18.925406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:12.564410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006729+0000","last_change":"2024-05-08T22:33:19.005176+0000","last_active":"2024-05-08T22:33:19.006729+0000","last_peered":"2024-05-08T22:33:19.006729+0000","last_clean":"2024-05-08T22:33:19.006729+0000","last_became_active":"2024-05-08T22:33:19.004757+0000","last_became_peered":"2024-05-08T22:33:19.004757+0000","last_unstale":"2024-05-08T22:33:19.006729+0000","last_undegraded":"2024-05-08T22:33:19.006729+0000","last_fullsized":"2024-05-08T22:33:19.006729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.924451+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.922189+0000","last_became_peered":"2024-05-08T22:33:14.922189+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927288+0000","last_change":"2024-05-08T22:33:14.527475+0000","last_active":"2024-05-08T22:33:18.927288+0000","last_peered":"2024-05-08T22:33:18.927288+0000","last_clean":"2024-05-08T22:33:18.927288+0000","last_became_active":"2024-05-08T22:33:14.526627+0000","last_became_peered":"2024-05-08T22:33:14.526627+0000","last_unstale":"2024-05-08T22:33:18.927288+0000","last_undegraded":"2024-05-08T22:33:18.927288+0000","last_fullsized":"2024-05-08T22:33:18.927288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:00.386313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312868+0000","last_change":"2024-05-08T22:33:18.946891+0000","last_active":"2024-05-08T22:33:19.312868+0000","last_peered":"2024-05-08T22:33:19.312868+0000","last_clean":"2024-05-08T22:33:19.312868+0000","last_became_active":"2024-05-08T22:33:18.946534+0000","last_became_peered":"2024-05-08T22:33:18.946534+0000","last_unstale":"2024-05-08T22:33:19.312868+0000","last_undegraded":"2024-05-08T22:33:19.312868+0000","last_fullsized":"2024-05-08T22:33:19.312868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999925+0000","last_change":"2024-05-08T22:33:14.918650+0000","last_active":"2024-05-08T22:33:18.999925+0000","last_peered":"2024-05-08T22:33:18.999925+0000","last_clean":"2024-05-08T22:33:18.999925+0000","last_became_active":"2024-05-08T22:33:14.918389+0000","last_became_peered":"2024-05-08T22:33:14.918389+0000","last_unstale":"2024-05-08T22:33:18.999925+0000","last_undegraded":"2024-05-08T22:33:18.999925+0000","last_fullsized":"2024-05-08T22:33:18.999925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925062+0000","last_change":"2024-05-08T22:33:14.522438+0000","last_active":"2024-05-08T22:33:18.925062+0000","last_peered":"2024-05-08T22:33:18.925062+0000","last_clean":"2024-05-08T22:33:18.925062+0000","last_became_active":"2024-05-08T22:33:14.522094+0000","last_became_peered":"2024-05-08T22:33:14.522094+0000","last_unstale":"2024-05-08T22:33:18.925062+0000","last_undegraded":"2024-05-08T22:33:18.925062+0000","last_fullsized":"2024-05-08T22:33:18.925062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.950611+0000","last_change":"2024-05-08T22:33:18.944271+0000","last_active":"2024-05-08T22:33:18.950611+0000","last_peered":"2024-05-08T22:33:18.950611+0000","last_clean":"2024-05-08T22:33:18.950611+0000","last_became_active":"2024-05-08T22:33:18.943934+0000","last_became_peered":"2024-05-08T22:33:18.943934+0000","last_unstale":"2024-05-08T22:33:18.950611+0000","last_undegraded":"2024-05-08T22:33:18.950611+0000","last_fullsized":"2024-05-08T22:33:18.950611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926385+0000","last_change":"2024-05-08T22:33:14.915516+0000","last_active":"2024-05-08T22:33:18.926385+0000","last_peered":"2024-05-08T22:33:18.926385+0000","last_clean":"2024-05-08T22:33:18.926385+0000","last_became_active":"2024-05-08T22:33:14.915253+0000","last_became_peered":"2024-05-08T22:33:14.915253+0000","last_unstale":"2024-05-08T22:33:18.926385+0000","last_undegraded":"2024-05-08T22:33:18.926385+0000","last_fullsized":"2024-05-08T22:33:18.926385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925993+0000","last_change":"2024-05-08T22:33:14.517759+0000","last_active":"2024-05-08T22:33:18.925993+0000","last_peered":"2024-05-08T22:33:18.925993+0000","last_clean":"2024-05-08T22:33:18.925993+0000","last_became_active":"2024-05-08T22:33:14.517109+0000","last_became_peered":"2024-05-08T22:33:14.517109+0000","last_unstale":"2024-05-08T22:33:18.925993+0000","last_undegraded":"2024-05-08T22:33:18.925993+0000","last_fullsized":"2024-05-08T22:33:18.925993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:13.973941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313006+0000","last_change":"2024-05-08T22:33:19.005763+0000","last_active":"2024-05-08T22:33:19.313006+0000","last_peered":"2024-05-08T22:33:19.313006+0000","last_clean":"2024-05-08T22:33:19.313006+0000","last_became_active":"2024-05-08T22:33:19.004827+0000","last_became_peered":"2024-05-08T22:33:19.004827+0000","last_unstale":"2024-05-08T22:33:19.313006+0000","last_undegraded":"2024-05-08T22:33:19.313006+0000","last_fullsized":"2024-05-08T22:33:19.313006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:24.712864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998964+0000","last_change":"2024-05-08T22:33:14.918080+0000","last_active":"2024-05-08T22:33:18.998964+0000","last_peered":"2024-05-08T22:33:18.998964+0000","last_clean":"2024-05-08T22:33:18.998964+0000","last_became_active":"2024-05-08T22:33:14.917791+0000","last_became_peered":"2024-05-08T22:33:14.917791+0000","last_unstale":"2024-05-08T22:33:18.998964+0000","last_undegraded":"2024-05-08T22:33:18.998964+0000","last_fullsized":"2024-05-08T22:33:18.998964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934850+0000","last_change":"2024-05-08T22:33:14.527314+0000","last_active":"2024-05-08T22:33:18.934850+0000","last_peered":"2024-05-08T22:33:18.934850+0000","last_clean":"2024-05-08T22:33:18.934850+0000","last_became_active":"2024-05-08T22:33:14.526667+0000","last_became_peered":"2024-05-08T22:33:14.526667+0000","last_unstale":"2024-05-08T22:33:18.934850+0000","last_undegraded":"2024-05-08T22:33:18.934850+0000","last_fullsized":"2024-05-08T22:33:18.934850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312923+0000","last_change":"2024-05-08T22:33:19.004038+0000","last_active":"2024-05-08T22:33:19.312923+0000","last_peered":"2024-05-08T22:33:19.312923+0000","last_clean":"2024-05-08T22:33:19.312923+0000","last_became_active":"2024-05-08T22:33:19.002976+0000","last_became_peered":"2024-05-08T22:33:19.002976+0000","last_unstale":"2024-05-08T22:33:19.312923+0000","last_undegraded":"2024-05-08T22:33:19.312923+0000","last_fullsized":"2024-05-08T22:33:19.312923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.506642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930214+0000","last_change":"2024-05-08T22:33:14.924563+0000","last_active":"2024-05-08T22:33:18.930214+0000","last_peered":"2024-05-08T22:33:18.930214+0000","last_clean":"2024-05-08T22:33:18.930214+0000","last_became_active":"2024-05-08T22:33:14.924299+0000","last_became_peered":"2024-05-08T22:33:14.924299+0000","last_unstale":"2024-05-08T22:33:18.930214+0000","last_undegraded":"2024-05-08T22:33:18.930214+0000","last_fullsized":"2024-05-08T22:33:18.930214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931836+0000","last_change":"2024-05-08T22:33:14.531490+0000","last_active":"2024-05-08T22:33:18.931836+0000","last_peered":"2024-05-08T22:33:18.931836+0000","last_clean":"2024-05-08T22:33:18.931836+0000","last_became_active":"2024-05-08T22:33:14.530820+0000","last_became_peered":"2024-05-08T22:33:14.530820+0000","last_unstale":"2024-05-08T22:33:18.931836+0000","last_undegraded":"2024-05-08T22:33:18.931836+0000","last_fullsized":"2024-05-08T22:33:18.931836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:59.224955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006802+0000","last_change":"2024-05-08T22:33:19.005429+0000","last_active":"2024-05-08T22:33:19.006802+0000","last_peered":"2024-05-08T22:33:19.006802+0000","last_clean":"2024-05-08T22:33:19.006802+0000","last_became_active":"2024-05-08T22:33:19.004941+0000","last_became_peered":"2024-05-08T22:33:19.004941+0000","last_unstale":"2024-05-08T22:33:19.006802+0000","last_undegraded":"2024-05-08T22:33:19.006802+0000","last_fullsized":"2024-05-08T22:33:19.006802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999232+0000","last_change":"2024-05-08T22:33:14.919406+0000","last_active":"2024-05-08T22:33:18.999232+0000","last_peered":"2024-05-08T22:33:18.999232+0000","last_clean":"2024-05-08T22:33:18.999232+0000","last_became_active":"2024-05-08T22:33:14.919204+0000","last_became_peered":"2024-05-08T22:33:14.919204+0000","last_unstale":"2024-05-08T22:33:18.999232+0000","last_undegraded":"2024-05-08T22:33:18.999232+0000","last_fullsized":"2024-05-08T22:33:18.999232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:24.279224+0000","last_change":"2024-05-08T22:33:14.523404+0000","last_active":"2024-05-08T22:33:24.279224+0000","last_peered":"2024-05-08T22:33:24.279224+0000","last_clean":"2024-05-08T22:33:24.279224+0000","last_became_active":"2024-05-08T22:33:14.522325+0000","last_became_peered":"2024-05-08T22:33:14.522325+0000","last_unstale":"2024-05-08T22:33:24.279224+0000","last_undegraded":"2024-05-08T22:33:24.279224+0000","last_fullsized":"2024-05-08T22:33:24.279224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448558+0000","last_change":"2024-05-08T22:33:18.950110+0000","last_active":"2024-05-08T22:33:19.448558+0000","last_peered":"2024-05-08T22:33:19.448558+0000","last_clean":"2024-05-08T22:33:19.448558+0000","last_became_active":"2024-05-08T22:33:18.949555+0000","last_became_peered":"2024-05-08T22:33:18.949555+0000","last_unstale":"2024-05-08T22:33:19.448558+0000","last_undegraded":"2024-05-08T22:33:19.448558+0000","last_fullsized":"2024-05-08T22:33:19.448558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:15.389269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926732+0000","last_change":"2024-05-08T22:33:14.908111+0000","last_active":"2024-05-08T22:33:18.926732+0000","last_peered":"2024-05-08T22:33:18.926732+0000","last_clean":"2024-05-08T22:33:18.926732+0000","last_became_active":"2024-05-08T22:33:14.907276+0000","last_became_peered":"2024-05-08T22:33:14.907276+0000","last_unstale":"2024-05-08T22:33:18.926732+0000","last_undegraded":"2024-05-08T22:33:18.926732+0000","last_fullsized":"2024-05-08T22:33:18.926732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:53.154720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934191+0000","last_change":"2024-05-08T22:33:14.521905+0000","last_active":"2024-05-08T22:33:18.934191+0000","last_peered":"2024-05-08T22:33:18.934191+0000","last_clean":"2024-05-08T22:33:18.934191+0000","last_became_active":"2024-05-08T22:33:14.521518+0000","last_became_peered":"2024-05-08T22:33:14.521518+0000","last_unstale":"2024-05-08T22:33:18.934191+0000","last_undegraded":"2024-05-08T22:33:18.934191+0000","last_fullsized":"2024-05-08T22:33:18.934191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315185+0000","last_change":"2024-05-08T22:33:18.939252+0000","last_active":"2024-05-08T22:33:19.315185+0000","last_peered":"2024-05-08T22:33:19.315185+0000","last_clean":"2024-05-08T22:33:19.315185+0000","last_became_active":"2024-05-08T22:33:18.938564+0000","last_became_peered":"2024-05-08T22:33:18.938564+0000","last_unstale":"2024-05-08T22:33:19.315185+0000","last_undegraded":"2024-05-08T22:33:19.315185+0000","last_fullsized":"2024-05-08T22:33:19.315185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:42.775813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931955+0000","last_change":"2024-05-08T22:33:14.932581+0000","last_active":"2024-05-08T22:33:18.931955+0000","last_peered":"2024-05-08T22:33:18.931955+0000","last_clean":"2024-05-08T22:33:18.931955+0000","last_became_active":"2024-05-08T22:33:14.932332+0000","last_became_peered":"2024-05-08T22:33:14.932332+0000","last_unstale":"2024-05-08T22:33:18.931955+0000","last_undegraded":"2024-05-08T22:33:18.931955+0000","last_fullsized":"2024-05-08T22:33:18.931955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:49:02.412782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935272+0000","last_change":"2024-05-08T22:33:14.525414+0000","last_active":"2024-05-08T22:33:18.935272+0000","last_peered":"2024-05-08T22:33:18.935272+0000","last_clean":"2024-05-08T22:33:18.935272+0000","last_became_active":"2024-05-08T22:33:14.524869+0000","last_became_peered":"2024-05-08T22:33:14.524869+0000","last_unstale":"2024-05-08T22:33:18.935272+0000","last_undegraded":"2024-05-08T22:33:18.935272+0000","last_fullsized":"2024-05-08T22:33:18.935272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312533+0000","last_change":"2024-05-08T22:33:18.948135+0000","last_active":"2024-05-08T22:33:19.312533+0000","last_peered":"2024-05-08T22:33:19.312533+0000","last_clean":"2024-05-08T22:33:19.312533+0000","last_became_active":"2024-05-08T22:33:18.947623+0000","last_became_peered":"2024-05-08T22:33:18.947623+0000","last_unstale":"2024-05-08T22:33:19.312533+0000","last_undegraded":"2024-05-08T22:33:19.312533+0000","last_fullsized":"2024-05-08T22:33:19.312533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:55.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935185+0000","last_change":"2024-05-08T22:33:14.925264+0000","last_active":"2024-05-08T22:33:18.935185+0000","last_peered":"2024-05-08T22:33:18.935185+0000","last_clean":"2024-05-08T22:33:18.935185+0000","last_became_active":"2024-05-08T22:33:14.924936+0000","last_became_peered":"2024-05-08T22:33:14.924936+0000","last_unstale":"2024-05-08T22:33:18.935185+0000","last_undegraded":"2024-05-08T22:33:18.935185+0000","last_fullsized":"2024-05-08T22:33:18.935185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927536+0000","last_change":"2024-05-08T22:33:14.519535+0000","last_active":"2024-05-08T22:33:18.927536+0000","last_peered":"2024-05-08T22:33:18.927536+0000","last_clean":"2024-05-08T22:33:18.927536+0000","last_became_active":"2024-05-08T22:33:14.519264+0000","last_became_peered":"2024-05-08T22:33:14.519264+0000","last_unstale":"2024-05-08T22:33:18.927536+0000","last_undegraded":"2024-05-08T22:33:18.927536+0000","last_fullsized":"2024-05-08T22:33:18.927536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:19.007144+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:19.006007+0000","last_became_peered":"2024-05-08T22:33:19.006007+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:50.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.011031+0000","last_change":"2024-05-08T22:33:19.006206+0000","last_active":"2024-05-08T22:33:19.011031+0000","last_peered":"2024-05-08T22:33:19.011031+0000","last_clean":"2024-05-08T22:33:19.011031+0000","last_became_active":"2024-05-08T22:33:19.004897+0000","last_became_peered":"2024-05-08T22:33:19.004897+0000","last_unstale":"2024-05-08T22:33:19.011031+0000","last_undegraded":"2024-05-08T22:33:19.011031+0000","last_fullsized":"2024-05-08T22:33:19.011031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448478+0000","last_change":"2024-05-08T22:33:14.906510+0000","last_active":"2024-05-08T22:33:19.448478+0000","last_peered":"2024-05-08T22:33:19.448478+0000","last_clean":"2024-05-08T22:33:19.448478+0000","last_became_active":"2024-05-08T22:33:14.906227+0000","last_became_peered":"2024-05-08T22:33:14.906227+0000","last_unstale":"2024-05-08T22:33:19.448478+0000","last_undegraded":"2024-05-08T22:33:19.448478+0000","last_fullsized":"2024-05-08T22:33:19.448478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935784+0000","last_change":"2024-05-08T22:33:14.523994+0000","last_active":"2024-05-08T22:33:18.935784+0000","last_peered":"2024-05-08T22:33:18.935784+0000","last_clean":"2024-05-08T22:33:18.935784+0000","last_became_active":"2024-05-08T22:33:14.523710+0000","last_became_peered":"2024-05-08T22:33:14.523710+0000","last_unstale":"2024-05-08T22:33:18.935784+0000","last_undegraded":"2024-05-08T22:33:18.935784+0000","last_fullsized":"2024-05-08T22:33:18.935784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:56.994588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944980+0000","last_change":"2024-05-08T22:33:18.944209+0000","last_active":"2024-05-08T22:33:18.944980+0000","last_peered":"2024-05-08T22:33:18.944980+0000","last_clean":"2024-05-08T22:33:18.944980+0000","last_became_active":"2024-05-08T22:33:18.943824+0000","last_became_peered":"2024-05-08T22:33:18.943824+0000","last_unstale":"2024-05-08T22:33:18.944980+0000","last_undegraded":"2024-05-08T22:33:18.944980+0000","last_fullsized":"2024-05-08T22:33:18.944980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:04.454019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312284+0000","last_change":"2024-05-08T22:33:14.912166+0000","last_active":"2024-05-08T22:33:19.312284+0000","last_peered":"2024-05-08T22:33:19.312284+0000","last_clean":"2024-05-08T22:33:19.312284+0000","last_became_active":"2024-05-08T22:33:14.911941+0000","last_became_peered":"2024-05-08T22:33:14.911941+0000","last_unstale":"2024-05-08T22:33:19.312284+0000","last_undegraded":"2024-05-08T22:33:19.312284+0000","last_fullsized":"2024-05-08T22:33:19.312284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943398+0000","last_change":"2024-05-08T22:33:14.524713+0000","last_active":"2024-05-08T22:33:18.943398+0000","last_peered":"2024-05-08T22:33:18.943398+0000","last_clean":"2024-05-08T22:33:18.943398+0000","last_became_active":"2024-05-08T22:33:14.523688+0000","last_became_peered":"2024-05-08T22:33:14.523688+0000","last_unstale":"2024-05-08T22:33:18.943398+0000","last_undegraded":"2024-05-08T22:33:18.943398+0000","last_fullsized":"2024-05-08T22:33:18.943398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:12.770341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928004+0000","last_change":"2024-05-08T22:33:14.518843+0000","last_active":"2024-05-08T22:33:18.928004+0000","last_peered":"2024-05-08T22:33:18.928004+0000","last_clean":"2024-05-08T22:33:18.928004+0000","last_became_active":"2024-05-08T22:33:14.518002+0000","last_became_peered":"2024-05-08T22:33:14.518002+0000","last_unstale":"2024-05-08T22:33:18.928004+0000","last_undegraded":"2024-05-08T22:33:18.928004+0000","last_fullsized":"2024-05-08T22:33:18.928004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931587+0000","last_change":"2024-05-08T22:33:14.914102+0000","last_active":"2024-05-08T22:33:18.931587+0000","last_peered":"2024-05-08T22:33:18.931587+0000","last_clean":"2024-05-08T22:33:18.931587+0000","last_became_active":"2024-05-08T22:33:14.913777+0000","last_became_peered":"2024-05-08T22:33:14.913777+0000","last_unstale":"2024-05-08T22:33:18.931587+0000","last_undegraded":"2024-05-08T22:33:18.931587+0000","last_fullsized":"2024-05-08T22:33:18.931587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953813+0000","last_change":"2024-05-08T22:33:18.949165+0000","last_active":"2024-05-08T22:33:18.953813+0000","last_peered":"2024-05-08T22:33:18.953813+0000","last_clean":"2024-05-08T22:33:18.953813+0000","last_became_active":"2024-05-08T22:33:18.948660+0000","last_became_peered":"2024-05-08T22:33:18.948660+0000","last_unstale":"2024-05-08T22:33:18.953813+0000","last_undegraded":"2024-05-08T22:33:18.953813+0000","last_fullsized":"2024-05-08T22:33:18.953813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:44.316596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943131+0000","last_change":"2024-05-08T22:33:14.532226+0000","last_active":"2024-05-08T22:33:18.943131+0000","last_peered":"2024-05-08T22:33:18.943131+0000","last_clean":"2024-05-08T22:33:18.943131+0000","last_became_active":"2024-05-08T22:33:14.531943+0000","last_became_peered":"2024-05-08T22:33:14.531943+0000","last_unstale":"2024-05-08T22:33:18.943131+0000","last_undegraded":"2024-05-08T22:33:18.943131+0000","last_fullsized":"2024-05-08T22:33:18.943131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:51.052408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928212+0000","last_change":"2024-05-08T22:33:14.907843+0000","last_active":"2024-05-08T22:33:18.928212+0000","last_peered":"2024-05-08T22:33:18.928212+0000","last_clean":"2024-05-08T22:33:18.928212+0000","last_became_active":"2024-05-08T22:33:14.907511+0000","last_became_peered":"2024-05-08T22:33:14.907511+0000","last_unstale":"2024-05-08T22:33:18.928212+0000","last_undegraded":"2024-05-08T22:33:18.928212+0000","last_fullsized":"2024-05-08T22:33:18.928212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:38.769965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312725+0000","last_change":"2024-05-08T22:33:18.952988+0000","last_active":"2024-05-08T22:33:19.312725+0000","last_peered":"2024-05-08T22:33:19.312725+0000","last_clean":"2024-05-08T22:33:19.312725+0000","last_became_active":"2024-05-08T22:33:18.948797+0000","last_became_peered":"2024-05-08T22:33:18.948797+0000","last_unstale":"2024-05-08T22:33:19.312725+0000","last_undegraded":"2024-05-08T22:33:19.312725+0000","last_fullsized":"2024-05-08T22:33:19.312725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:10.379649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926181+0000","last_change":"2024-05-08T22:33:14.915829+0000","last_active":"2024-05-08T22:33:18.926181+0000","last_peered":"2024-05-08T22:33:18.926181+0000","last_clean":"2024-05-08T22:33:18.926181+0000","last_became_active":"2024-05-08T22:33:14.915496+0000","last_became_peered":"2024-05-08T22:33:14.915496+0000","last_unstale":"2024-05-08T22:33:18.926181+0000","last_undegraded":"2024-05-08T22:33:18.926181+0000","last_fullsized":"2024-05-08T22:33:18.926181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925052+0000","last_change":"2024-05-08T22:33:14.930927+0000","last_active":"2024-05-08T22:33:18.925052+0000","last_peered":"2024-05-08T22:33:18.925052+0000","last_clean":"2024-05-08T22:33:18.925052+0000","last_became_active":"2024-05-08T22:33:14.930569+0000","last_became_peered":"2024-05-08T22:33:14.930569+0000","last_unstale":"2024-05-08T22:33:18.925052+0000","last_undegraded":"2024-05-08T22:33:18.925052+0000","last_fullsized":"2024-05-08T22:33:18.925052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934767+0000","last_change":"2024-05-08T22:33:14.928385+0000","last_active":"2024-05-08T22:33:18.934767+0000","last_peered":"2024-05-08T22:33:18.934767+0000","last_clean":"2024-05-08T22:33:18.934767+0000","last_became_active":"2024-05-08T22:33:14.928162+0000","last_became_peered":"2024-05-08T22:33:14.928162+0000","last_unstale":"2024-05-08T22:33:18.934767+0000","last_undegraded":"2024-05-08T22:33:18.934767+0000","last_fullsized":"2024-05-08T22:33:18.934767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:11.717668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931430+0000","last_change":"2024-05-08T22:33:14.916147+0000","last_active":"2024-05-08T22:33:18.931430+0000","last_peered":"2024-05-08T22:33:18.931430+0000","last_clean":"2024-05-08T22:33:18.931430+0000","last_became_active":"2024-05-08T22:33:14.915762+0000","last_became_peered":"2024-05-08T22:33:14.915762+0000","last_unstale":"2024-05-08T22:33:18.931430+0000","last_undegraded":"2024-05-08T22:33:18.931430+0000","last_fullsized":"2024-05-08T22:33:18.931430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:28.668944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313453+0000","last_change":"2024-05-08T22:33:14.912915+0000","last_active":"2024-05-08T22:33:19.313453+0000","last_peered":"2024-05-08T22:33:19.313453+0000","last_clean":"2024-05-08T22:33:19.313453+0000","last_became_active":"2024-05-08T22:33:14.912698+0000","last_became_peered":"2024-05-08T22:33:14.912698+0000","last_unstale":"2024-05-08T22:33:19.313453+0000","last_undegraded":"2024-05-08T22:33:19.313453+0000","last_fullsized":"2024-05-08T22:33:19.313453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448664+0000","last_change":"2024-05-08T22:33:14.907107+0000","last_active":"2024-05-08T22:33:19.448664+0000","last_peered":"2024-05-08T22:33:19.448664+0000","last_clean":"2024-05-08T22:33:19.448664+0000","last_became_active":"2024-05-08T22:33:14.906444+0000","last_became_peered":"2024-05-08T22:33:14.906444+0000","last_unstale":"2024-05-08T22:33:19.448664+0000","last_undegraded":"2024-05-08T22:33:19.448664+0000","last_fullsized":"2024-05-08T22:33:19.448664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929966+0000","last_change":"2024-05-08T22:33:14.913584+0000","last_active":"2024-05-08T22:33:18.929966+0000","last_peered":"2024-05-08T22:33:18.929966+0000","last_clean":"2024-05-08T22:33:18.929966+0000","last_became_active":"2024-05-08T22:33:14.913304+0000","last_became_peered":"2024-05-08T22:33:14.913304+0000","last_unstale":"2024-05-08T22:33:18.929966+0000","last_undegraded":"2024-05-08T22:33:18.929966+0000","last_fullsized":"2024-05-08T22:33:18.929966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312698+0000","last_change":"2024-05-08T22:33:14.905976+0000","last_active":"2024-05-08T22:33:19.312698+0000","last_peered":"2024-05-08T22:33:19.312698+0000","last_clean":"2024-05-08T22:33:19.312698+0000","last_became_active":"2024-05-08T22:33:14.905734+0000","last_became_peered":"2024-05-08T22:33:14.905734+0000","last_unstale":"2024-05-08T22:33:19.312698+0000","last_undegraded":"2024-05-08T22:33:19.312698+0000","last_fullsized":"2024-05-08T22:33:19.312698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935694+0000","last_change":"2024-05-08T22:33:14.927777+0000","last_active":"2024-05-08T22:33:18.935694+0000","last_peered":"2024-05-08T22:33:18.935694+0000","last_clean":"2024-05-08T22:33:18.935694+0000","last_became_active":"2024-05-08T22:33:14.927297+0000","last_became_peered":"2024-05-08T22:33:14.927297+0000","last_unstale":"2024-05-08T22:33:18.935694+0000","last_undegraded":"2024-05-08T22:33:18.935694+0000","last_fullsized":"2024-05-08T22:33:18.935694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:16.838261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312806+0000","last_change":"2024-05-08T22:33:14.911565+0000","last_active":"2024-05-08T22:33:19.312806+0000","last_peered":"2024-05-08T22:33:19.312806+0000","last_clean":"2024-05-08T22:33:19.312806+0000","last_became_active":"2024-05-08T22:33:14.909251+0000","last_became_peered":"2024-05-08T22:33:14.909251+0000","last_unstale":"2024-05-08T22:33:19.312806+0000","last_undegraded":"2024-05-08T22:33:19.312806+0000","last_fullsized":"2024-05-08T22:33:19.312806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934680+0000","last_change":"2024-05-08T22:33:14.925382+0000","last_active":"2024-05-08T22:33:18.934680+0000","last_peered":"2024-05-08T22:33:18.934680+0000","last_clean":"2024-05-08T22:33:18.934680+0000","last_became_active":"2024-05-08T22:33:14.925133+0000","last_became_peered":"2024-05-08T22:33:14.925133+0000","last_unstale":"2024-05-08T22:33:18.934680+0000","last_undegraded":"2024-05-08T22:33:18.934680+0000","last_fullsized":"2024-05-08T22:33:18.934680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931302+0000","last_change":"2024-05-08T22:33:14.911744+0000","last_active":"2024-05-08T22:33:18.931302+0000","last_peered":"2024-05-08T22:33:18.931302+0000","last_clean":"2024-05-08T22:33:18.931302+0000","last_became_active":"2024-05-08T22:33:14.910853+0000","last_became_peered":"2024-05-08T22:33:14.910853+0000","last_unstale":"2024-05-08T22:33:18.931302+0000","last_undegraded":"2024-05-08T22:33:18.931302+0000","last_fullsized":"2024-05-08T22:33:18.931302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928361+0000","last_change":"2024-05-08T22:33:14.914067+0000","last_active":"2024-05-08T22:33:18.928361+0000","last_peered":"2024-05-08T22:33:18.928361+0000","last_clean":"2024-05-08T22:33:18.928361+0000","last_became_active":"2024-05-08T22:33:14.913755+0000","last_became_peered":"2024-05-08T22:33:14.913755+0000","last_unstale":"2024-05-08T22:33:18.928361+0000","last_undegraded":"2024-05-08T22:33:18.928361+0000","last_fullsized":"2024-05-08T22:33:18.928361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999511+0000","last_change":"2024-05-08T22:33:14.922252+0000","last_active":"2024-05-08T22:33:18.999511+0000","last_peered":"2024-05-08T22:33:18.999511+0000","last_clean":"2024-05-08T22:33:18.999511+0000","last_became_active":"2024-05-08T22:33:14.921949+0000","last_became_peered":"2024-05-08T22:33:14.921949+0000","last_unstale":"2024-05-08T22:33:18.999511+0000","last_undegraded":"2024-05-08T22:33:18.999511+0000","last_fullsized":"2024-05-08T22:33:18.999511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:12.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925280+0000","last_change":"2024-05-08T22:33:14.916689+0000","last_active":"2024-05-08T22:33:18.925280+0000","last_peered":"2024-05-08T22:33:18.925280+0000","last_clean":"2024-05-08T22:33:18.925280+0000","last_became_active":"2024-05-08T22:33:14.916200+0000","last_became_peered":"2024-05-08T22:33:14.916200+0000","last_unstale":"2024-05-08T22:33:18.925280+0000","last_undegraded":"2024-05-08T22:33:18.925280+0000","last_fullsized":"2024-05-08T22:33:18.925280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936475+0000","last_change":"2024-05-08T22:33:14.925461+0000","last_active":"2024-05-08T22:33:18.936475+0000","last_peered":"2024-05-08T22:33:18.936475+0000","last_clean":"2024-05-08T22:33:18.936475+0000","last_became_active":"2024-05-08T22:33:14.924786+0000","last_became_peered":"2024-05-08T22:33:14.924786+0000","last_unstale":"2024-05-08T22:33:18.936475+0000","last_undegraded":"2024-05-08T22:33:18.936475+0000","last_fullsized":"2024-05-08T22:33:18.936475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935606+0000","last_change":"2024-05-08T22:33:14.918139+0000","last_active":"2024-05-08T22:33:18.935606+0000","last_peered":"2024-05-08T22:33:18.935606+0000","last_clean":"2024-05-08T22:33:18.935606+0000","last_became_active":"2024-05-08T22:33:14.914720+0000","last_became_peered":"2024-05-08T22:33:14.914720+0000","last_unstale":"2024-05-08T22:33:18.935606+0000","last_undegraded":"2024-05-08T22:33:18.935606+0000","last_fullsized":"2024-05-08T22:33:18.935606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448502+0000","last_change":"2024-05-08T22:33:14.905986+0000","last_active":"2024-05-08T22:33:19.448502+0000","last_peered":"2024-05-08T22:33:19.448502+0000","last_clean":"2024-05-08T22:33:19.448502+0000","last_became_active":"2024-05-08T22:33:14.905613+0000","last_became_peered":"2024-05-08T22:33:14.905613+0000","last_unstale":"2024-05-08T22:33:19.448502+0000","last_undegraded":"2024-05-08T22:33:19.448502+0000","last_fullsized":"2024-05-08T22:33:19.448502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930058+0000","last_change":"2024-05-08T22:33:14.911270+0000","last_active":"2024-05-08T22:33:18.930058+0000","last_peered":"2024-05-08T22:33:18.930058+0000","last_clean":"2024-05-08T22:33:18.930058+0000","last_became_active":"2024-05-08T22:33:14.910966+0000","last_became_peered":"2024-05-08T22:33:14.910966+0000","last_unstale":"2024-05-08T22:33:18.930058+0000","last_undegraded":"2024-05-08T22:33:18.930058+0000","last_fullsized":"2024-05-08T22:33:18.930058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:40.907481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312110+0000","last_change":"2024-05-08T22:33:14.917778+0000","last_active":"2024-05-08T22:33:19.312110+0000","last_peered":"2024-05-08T22:33:19.312110+0000","last_clean":"2024-05-08T22:33:19.312110+0000","last_became_active":"2024-05-08T22:33:14.917531+0000","last_became_peered":"2024-05-08T22:33:14.917531+0000","last_unstale":"2024-05-08T22:33:19.312110+0000","last_undegraded":"2024-05-08T22:33:19.312110+0000","last_fullsized":"2024-05-08T22:33:19.312110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926531+0000","last_change":"2024-05-08T22:33:14.916563+0000","last_active":"2024-05-08T22:33:18.926531+0000","last_peered":"2024-05-08T22:33:18.926531+0000","last_clean":"2024-05-08T22:33:18.926531+0000","last_became_active":"2024-05-08T22:33:14.916271+0000","last_became_peered":"2024-05-08T22:33:14.916271+0000","last_unstale":"2024-05-08T22:33:18.926531+0000","last_undegraded":"2024-05-08T22:33:18.926531+0000","last_fullsized":"2024-05-08T22:33:18.926531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927948+0000","last_change":"2024-05-08T22:33:14.931269+0000","last_active":"2024-05-08T22:33:18.927948+0000","last_peered":"2024-05-08T22:33:18.927948+0000","last_clean":"2024-05-08T22:33:18.927948+0000","last_became_active":"2024-05-08T22:33:14.931026+0000","last_became_peered":"2024-05-08T22:33:14.931026+0000","last_unstale":"2024-05-08T22:33:18.927948+0000","last_undegraded":"2024-05-08T22:33:18.927948+0000","last_fullsized":"2024-05-08T22:33:18.927948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929869+0000","last_change":"2024-05-08T22:33:14.928697+0000","last_active":"2024-05-08T22:33:18.929869+0000","last_peered":"2024-05-08T22:33:18.929869+0000","last_clean":"2024-05-08T22:33:18.929869+0000","last_became_active":"2024-05-08T22:33:14.928400+0000","last_became_peered":"2024-05-08T22:33:14.928400+0000","last_unstale":"2024-05-08T22:33:18.929869+0000","last_undegraded":"2024-05-08T22:33:18.929869+0000","last_fullsized":"2024-05-08T22:33:18.929869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999653+0000","last_change":"2024-05-08T22:33:14.931677+0000","last_active":"2024-05-08T22:33:18.999653+0000","last_peered":"2024-05-08T22:33:18.999653+0000","last_clean":"2024-05-08T22:33:18.999653+0000","last_became_active":"2024-05-08T22:33:14.931437+0000","last_became_peered":"2024-05-08T22:33:14.931437+0000","last_unstale":"2024-05-08T22:33:18.999653+0000","last_undegraded":"2024-05-08T22:33:18.999653+0000","last_fullsized":"2024-05-08T22:33:18.999653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:07.861504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:14.924165+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:14.923910+0000","last_became_peered":"2024-05-08T22:33:14.923910+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:09.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.910595+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.910354+0000","last_became_peered":"2024-05-08T22:33:14.910354+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:29.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934575+0000","last_change":"2024-05-08T22:33:14.924832+0000","last_active":"2024-05-08T22:33:18.934575+0000","last_peered":"2024-05-08T22:33:18.934575+0000","last_clean":"2024-05-08T22:33:18.934575+0000","last_became_active":"2024-05-08T22:33:14.924207+0000","last_became_peered":"2024-05-08T22:33:14.924207+0000","last_unstale":"2024-05-08T22:33:18.934575+0000","last_undegraded":"2024-05-08T22:33:18.934575+0000","last_fullsized":"2024-05-08T22:33:18.934575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998856+0000","last_change":"2024-05-08T22:33:14.922269+0000","last_active":"2024-05-08T22:33:18.998856+0000","last_peered":"2024-05-08T22:33:18.998856+0000","last_clean":"2024-05-08T22:33:18.998856+0000","last_became_active":"2024-05-08T22:33:14.921959+0000","last_became_peered":"2024-05-08T22:33:14.921959+0000","last_unstale":"2024-05-08T22:33:18.998856+0000","last_undegraded":"2024-05-08T22:33:18.998856+0000","last_fullsized":"2024-05-08T22:33:18.998856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934594+0000","last_change":"2024-05-08T22:33:14.930737+0000","last_active":"2024-05-08T22:33:18.934594+0000","last_peered":"2024-05-08T22:33:18.934594+0000","last_clean":"2024-05-08T22:33:18.934594+0000","last_became_active":"2024-05-08T22:33:14.930465+0000","last_became_peered":"2024-05-08T22:33:14.930465+0000","last_unstale":"2024-05-08T22:33:18.934594+0000","last_undegraded":"2024-05-08T22:33:18.934594+0000","last_fullsized":"2024-05-08T22:33:18.934594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:03.737555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930293+0000","last_change":"2024-05-08T22:33:14.911063+0000","last_active":"2024-05-08T22:33:18.930293+0000","last_peered":"2024-05-08T22:33:18.930293+0000","last_clean":"2024-05-08T22:33:18.930293+0000","last_became_active":"2024-05-08T22:33:14.910801+0000","last_became_peered":"2024-05-08T22:33:14.910801+0000","last_unstale":"2024-05-08T22:33:18.930293+0000","last_undegraded":"2024-05-08T22:33:18.930293+0000","last_fullsized":"2024-05-08T22:33:18.930293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:50.282359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448710+0000","last_change":"2024-05-08T22:33:14.916640+0000","last_active":"2024-05-08T22:33:19.448710+0000","last_peered":"2024-05-08T22:33:19.448710+0000","last_clean":"2024-05-08T22:33:19.448710+0000","last_became_active":"2024-05-08T22:33:14.916171+0000","last_became_peered":"2024-05-08T22:33:14.916171+0000","last_unstale":"2024-05-08T22:33:19.448710+0000","last_undegraded":"2024-05-08T22:33:19.448710+0000","last_fullsized":"2024-05-08T22:33:19.448710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999564+0000","last_change":"2024-05-08T22:33:14.925286+0000","last_active":"2024-05-08T22:33:18.999564+0000","last_peered":"2024-05-08T22:33:18.999564+0000","last_clean":"2024-05-08T22:33:18.999564+0000","last_became_active":"2024-05-08T22:33:14.924846+0000","last_became_peered":"2024-05-08T22:33:14.924846+0000","last_unstale":"2024-05-08T22:33:18.999564+0000","last_undegraded":"2024-05-08T22:33:18.999564+0000","last_fullsized":"2024-05-08T22:33:18.999564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:15.808109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":868352,"data_stored":697988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":274432,"data_stored":107128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":282624,"data_stored":107164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":698044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:33:43.676 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:33:43.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 Processor -- start 2024-05-08T22:33:43.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 -- start start 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f80440c8d70 0x7f804415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f80440c9230 0x7f80440c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044158f60 con 0x7f8044158b80 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440c9be0 con 0x7f80440c8d70 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804bf43700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804406b860 con 0x7f80440c9230 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f80440c8d70 0x7f804415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8049cdf700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8044158b80 0x7f804406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33606/0 (socket says 172.21.15.179:33606) 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f80440c8d70 0x7f804415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35362/0 (socket says 172.21.15.179:35362) 2024-05-08T22:33:43.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8049cdf700 1 -- 172.21.15.179:0/3515244550 learned_addr learned my addr 172.21.15.179:0/3515244550 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f804a4e0700 1 --2- 172.21.15.179:0/3515244550 >> v2:172.21.15.195:3300/0 conn(0x7f80440c9230 0x7f80440c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 -- 172.21.15.179:0/3515244550 >> v2:172.21.15.195:3300/0 conn(0x7f80440c9230 msgr2=0x7f80440c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 --2- 172.21.15.179:0/3515244550 >> v2:172.21.15.195:3300/0 conn(0x7f80440c9230 0x7f80440c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 -- 172.21.15.179:0/3515244550 >> v1:172.21.15.179:6789/0 conn(0x7f8044158b80 legacy=0x7f804406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8048cdd700 1 -- 172.21.15.179:0/3515244550 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763431135 0 0) 0x7f8044158f60 con 0x7f8044158b80 2024-05-08T22:33:43.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 -- 172.21.15.179:0/3515244550 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804406ac50 con 0x7f80440c8d70 2024-05-08T22:33:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f80494de700 1 --2- 172.21.15.179:0/3515244550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f804415b4d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80440cc740 tx=0x7f804000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fda96b4708cd1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8048cdd700 1 -- 172.21.15.179:0/3515244550 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000e070 con 0x7f80440c8d70 2024-05-08T22:33:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8048cdd700 1 -- 172.21.15.179:0/3515244550 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804000bb20 con 0x7f80440c8d70 2024-05-08T22:33:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.783+0000 7f8048cdd700 1 -- 172.21.15.179:0/3515244550 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040012430 con 0x7f80440c8d70 2024-05-08T22:33:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- 172.21.15.179:0/3515244550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 msgr2=0x7f804415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- 172.21.15.179:0/3515244550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f804415b4d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80440cc740 tx=0x7f804000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- 172.21.15.179:0/3515244550 shutdown_connections 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- 172.21.15.179:0/3515244550 >> v2:172.21.15.195:3300/0 conn(0x7f80440c9230 0x7f80440c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- 172.21.15.179:0/3515244550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f804415b4d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- 172.21.15.179:0/3515244550 >> 172.21.15.179:0/3515244550 conn(0x7f80440cb660 msgr2=0x7f80440cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- 172.21.15.179:0/3515244550 shutdown_connections 2024-05-08T22:33:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- 172.21.15.179:0/3515244550 wait complete. 2024-05-08T22:33:43.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 Processor -- start 2024-05-08T22:33:43.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- start start 2024-05-08T22:33:43.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 0x7f80440c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044158b80 0x7f80440c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c5770 con 0x7f80440c8d70 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c1220 con 0x7f8044158b80 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804bf43700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c0610 con 0x7f80440c9230 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35378/0 (socket says 172.21.15.179:35378) 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 -- 172.21.15.179:0/1567278074 learned_addr learned my addr 172.21.15.179:0/1567278074 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f804a4e0700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044158b80 0x7f80440c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f80494de700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 0x7f80440c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 -- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 msgr2=0x7f80440c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 0x7f80440c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 -- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044158b80 msgr2=0x7f80440c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044158b80 0x7f80440c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 -- 172.21.15.179:0/1567278074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009a70 con 0x7f80440c8d70 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f80494de700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 0x7f80440c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.787+0000 7f8049cdf700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f803400c4e0 tx=0x7f803400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034004ca0 con 0x7f80440c8d70 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034004e20 con 0x7f80440c8d70 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80440c08f0 con 0x7f80440c8d70 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034007310 con 0x7f80440c8d70 2024-05-08T22:33:43.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80440c0eb0 con 0x7f80440c8d70 2024-05-08T22:33:43.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044067770 con 0x7f80440c8d70 2024-05-08T22:33:43.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8034007470 con 0x7f80440c8d70 2024-05-08T22:33:43.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 0x7f8030068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:43.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8034084080 con 0x7f80440c8d70 2024-05-08T22:33:43.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.791+0000 7f80494de700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 0x7f8030068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:43.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.795+0000 7f80494de700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 0x7f8030068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8040014040 tx=0x7f8040011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:43.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.799+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8034051100 con 0x7f80440c8d70 2024-05-08T22:33:43.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.955+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f804406b4f0 con 0x7f8030066730 2024-05-08T22:33:43.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.975+0000 7f803a7fc700 1 -- 172.21.15.179:0/1567278074 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345273 (secure 0 0 0) 0x7f8040003df0 con 0x7f8030066730 2024-05-08T22:33:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 msgr2=0x7f8030068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 0x7f8030068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8040014040 tx=0x7f8040011000 comp rx=0 tx=0).stop 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 msgr2=0x7f80440c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f803400c4e0 tx=0x7f803400c7f0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 shutdown_connections 2024-05-08T22:33:43.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8030066730 0x7f8030068bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044158b80 0x7f80440c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80440c9230 0x7f80440c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 --2- 172.21.15.179:0/1567278074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80440c8d70 0x7f80440c5230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 >> 172.21.15.179:0/1567278074 conn(0x7f80440cb660 msgr2=0x7f80440c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 shutdown_connections 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:43.979+0000 7f804bf43700 1 -- 172.21.15.179:0/1567278074 wait complete. 2024-05-08T22:33:43.982 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:33:43.995 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-08T22:33:43.385624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235888,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738832,"statfs":{"total":773094113280,"available":772852563968,"internally_reserved":0,"allocated":3416064,"data_stored":2046228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005565"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.932430+0000","last_change":"2024-05-08T22:33:14.533515+0000","last_active":"2024-05-08T22:33:18.932430+0000","last_peered":"2024-05-08T22:33:18.932430+0000","last_clean":"2024-05-08T22:33:18.932430+0000","last_became_active":"2024-05-08T22:33:14.533293+0000","last_became_peered":"2024-05-08T22:33:14.533293+0000","last_unstale":"2024-05-08T22:33:18.932430+0000","last_undegraded":"2024-05-08T22:33:18.932430+0000","last_fullsized":"2024-05-08T22:33:18.932430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312810+0000","last_change":"2024-05-08T22:33:14.919770+0000","last_active":"2024-05-08T22:33:19.312810+0000","last_peered":"2024-05-08T22:33:19.312810+0000","last_clean":"2024-05-08T22:33:19.312810+0000","last_became_active":"2024-05-08T22:33:14.917629+0000","last_became_peered":"2024-05-08T22:33:14.917629+0000","last_unstale":"2024-05-08T22:33:19.312810+0000","last_undegraded":"2024-05-08T22:33:19.312810+0000","last_fullsized":"2024-05-08T22:33:19.312810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311922+0000","last_change":"2024-05-08T22:33:19.007654+0000","last_active":"2024-05-08T22:33:19.311922+0000","last_peered":"2024-05-08T22:33:19.311922+0000","last_clean":"2024-05-08T22:33:19.311922+0000","last_became_active":"2024-05-08T22:33:19.006811+0000","last_became_peered":"2024-05-08T22:33:19.006811+0000","last_unstale":"2024-05-08T22:33:19.311922+0000","last_undegraded":"2024-05-08T22:33:19.311922+0000","last_fullsized":"2024-05-08T22:33:19.311922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:28.811139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944044+0000","last_change":"2024-05-08T22:33:14.528074+0000","last_active":"2024-05-08T22:33:18.944044+0000","last_peered":"2024-05-08T22:33:18.944044+0000","last_clean":"2024-05-08T22:33:18.944044+0000","last_became_active":"2024-05-08T22:33:14.527773+0000","last_became_peered":"2024-05-08T22:33:14.527773+0000","last_unstale":"2024-05-08T22:33:18.944044+0000","last_undegraded":"2024-05-08T22:33:18.944044+0000","last_fullsized":"2024-05-08T22:33:18.944044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:53.046188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931850+0000","last_change":"2024-05-08T22:33:14.932746+0000","last_active":"2024-05-08T22:33:18.931850+0000","last_peered":"2024-05-08T22:33:18.931850+0000","last_clean":"2024-05-08T22:33:18.931850+0000","last_became_active":"2024-05-08T22:33:14.932511+0000","last_became_peered":"2024-05-08T22:33:14.932511+0000","last_unstale":"2024-05-08T22:33:18.931850+0000","last_undegraded":"2024-05-08T22:33:18.931850+0000","last_fullsized":"2024-05-08T22:33:18.931850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:05.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313017+0000","last_change":"2024-05-08T22:33:18.946740+0000","last_active":"2024-05-08T22:33:19.313017+0000","last_peered":"2024-05-08T22:33:19.313017+0000","last_clean":"2024-05-08T22:33:19.313017+0000","last_became_active":"2024-05-08T22:33:18.946369+0000","last_became_peered":"2024-05-08T22:33:18.946369+0000","last_unstale":"2024-05-08T22:33:19.313017+0000","last_undegraded":"2024-05-08T22:33:19.313017+0000","last_fullsized":"2024-05-08T22:33:19.313017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999447+0000","last_change":"2024-05-08T22:33:14.520155+0000","last_active":"2024-05-08T22:33:18.999447+0000","last_peered":"2024-05-08T22:33:18.999447+0000","last_clean":"2024-05-08T22:33:18.999447+0000","last_became_active":"2024-05-08T22:33:14.519845+0000","last_became_peered":"2024-05-08T22:33:14.519845+0000","last_unstale":"2024-05-08T22:33:18.999447+0000","last_undegraded":"2024-05-08T22:33:18.999447+0000","last_fullsized":"2024-05-08T22:33:18.999447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:03.999595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934686+0000","last_change":"2024-05-08T22:33:14.920443+0000","last_active":"2024-05-08T22:33:18.934686+0000","last_peered":"2024-05-08T22:33:18.934686+0000","last_clean":"2024-05-08T22:33:18.934686+0000","last_became_active":"2024-05-08T22:33:14.920034+0000","last_became_peered":"2024-05-08T22:33:14.920034+0000","last_unstale":"2024-05-08T22:33:18.934686+0000","last_undegraded":"2024-05-08T22:33:18.934686+0000","last_fullsized":"2024-05-08T22:33:18.934686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314786+0000","last_change":"2024-05-08T22:33:18.948680+0000","last_active":"2024-05-08T22:33:19.314786+0000","last_peered":"2024-05-08T22:33:19.314786+0000","last_clean":"2024-05-08T22:33:19.314786+0000","last_became_active":"2024-05-08T22:33:18.948252+0000","last_became_peered":"2024-05-08T22:33:18.948252+0000","last_unstale":"2024-05-08T22:33:19.314786+0000","last_undegraded":"2024-05-08T22:33:19.314786+0000","last_fullsized":"2024-05-08T22:33:19.314786+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:20.694843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999183+0000","last_change":"2024-05-08T22:33:14.526338+0000","last_active":"2024-05-08T22:33:18.999183+0000","last_peered":"2024-05-08T22:33:18.999183+0000","last_clean":"2024-05-08T22:33:18.999183+0000","last_became_active":"2024-05-08T22:33:14.520436+0000","last_became_peered":"2024-05-08T22:33:14.520436+0000","last_unstale":"2024-05-08T22:33:18.999183+0000","last_undegraded":"2024-05-08T22:33:18.999183+0000","last_fullsized":"2024-05-08T22:33:18.999183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:15.246381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935315+0000","last_change":"2024-05-08T22:33:14.919811+0000","last_active":"2024-05-08T22:33:18.935315+0000","last_peered":"2024-05-08T22:33:18.935315+0000","last_clean":"2024-05-08T22:33:18.935315+0000","last_became_active":"2024-05-08T22:33:14.919249+0000","last_became_peered":"2024-05-08T22:33:14.919249+0000","last_unstale":"2024-05-08T22:33:18.935315+0000","last_undegraded":"2024-05-08T22:33:18.935315+0000","last_fullsized":"2024-05-08T22:33:18.935315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:34.252236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313998+0000","last_change":"2024-05-08T22:33:18.942435+0000","last_active":"2024-05-08T22:33:19.313998+0000","last_peered":"2024-05-08T22:33:19.313998+0000","last_clean":"2024-05-08T22:33:19.313998+0000","last_became_active":"2024-05-08T22:33:18.942040+0000","last_became_peered":"2024-05-08T22:33:18.942040+0000","last_unstale":"2024-05-08T22:33:19.313998+0000","last_undegraded":"2024-05-08T22:33:19.313998+0000","last_fullsized":"2024-05-08T22:33:19.313998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927785+0000","last_change":"2024-05-08T22:33:14.913632+0000","last_active":"2024-05-08T22:33:18.927785+0000","last_peered":"2024-05-08T22:33:18.927785+0000","last_clean":"2024-05-08T22:33:18.927785+0000","last_became_active":"2024-05-08T22:33:14.913040+0000","last_became_peered":"2024-05-08T22:33:14.913040+0000","last_unstale":"2024-05-08T22:33:18.927785+0000","last_undegraded":"2024-05-08T22:33:18.927785+0000","last_fullsized":"2024-05-08T22:33:18.927785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998991+0000","last_change":"2024-05-08T22:33:14.526397+0000","last_active":"2024-05-08T22:33:18.998991+0000","last_peered":"2024-05-08T22:33:18.998991+0000","last_clean":"2024-05-08T22:33:18.998991+0000","last_became_active":"2024-05-08T22:33:14.519761+0000","last_became_peered":"2024-05-08T22:33:14.519761+0000","last_unstale":"2024-05-08T22:33:18.998991+0000","last_undegraded":"2024-05-08T22:33:18.998991+0000","last_fullsized":"2024-05-08T22:33:18.998991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315399+0000","last_change":"2024-05-08T22:33:18.944230+0000","last_active":"2024-05-08T22:33:19.315399+0000","last_peered":"2024-05-08T22:33:19.315399+0000","last_clean":"2024-05-08T22:33:19.315399+0000","last_became_active":"2024-05-08T22:33:18.943852+0000","last_became_peered":"2024-05-08T22:33:18.943852+0000","last_unstale":"2024-05-08T22:33:19.315399+0000","last_undegraded":"2024-05-08T22:33:19.315399+0000","last_fullsized":"2024-05-08T22:33:19.315399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312132+0000","last_change":"2024-05-08T22:33:14.912752+0000","last_active":"2024-05-08T22:33:19.312132+0000","last_peered":"2024-05-08T22:33:19.312132+0000","last_clean":"2024-05-08T22:33:19.312132+0000","last_became_active":"2024-05-08T22:33:14.912455+0000","last_became_peered":"2024-05-08T22:33:14.912455+0000","last_unstale":"2024-05-08T22:33:19.312132+0000","last_undegraded":"2024-05-08T22:33:19.312132+0000","last_fullsized":"2024-05-08T22:33:19.312132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934257+0000","last_change":"2024-05-08T22:33:14.525254+0000","last_active":"2024-05-08T22:33:18.934257+0000","last_peered":"2024-05-08T22:33:18.934257+0000","last_clean":"2024-05-08T22:33:18.934257+0000","last_became_active":"2024-05-08T22:33:14.524663+0000","last_became_peered":"2024-05-08T22:33:14.524663+0000","last_unstale":"2024-05-08T22:33:18.934257+0000","last_undegraded":"2024-05-08T22:33:18.934257+0000","last_fullsized":"2024-05-08T22:33:18.934257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314812+0000","last_change":"2024-05-08T22:33:18.947596+0000","last_active":"2024-05-08T22:33:19.314812+0000","last_peered":"2024-05-08T22:33:19.314812+0000","last_clean":"2024-05-08T22:33:19.314812+0000","last_became_active":"2024-05-08T22:33:18.947225+0000","last_became_peered":"2024-05-08T22:33:18.947225+0000","last_unstale":"2024-05-08T22:33:19.314812+0000","last_undegraded":"2024-05-08T22:33:19.314812+0000","last_fullsized":"2024-05-08T22:33:19.314812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312379+0000","last_change":"2024-05-08T22:33:14.906568+0000","last_active":"2024-05-08T22:33:19.312379+0000","last_peered":"2024-05-08T22:33:19.312379+0000","last_clean":"2024-05-08T22:33:19.312379+0000","last_became_active":"2024-05-08T22:33:14.906359+0000","last_became_peered":"2024-05-08T22:33:14.906359+0000","last_unstale":"2024-05-08T22:33:19.312379+0000","last_undegraded":"2024-05-08T22:33:19.312379+0000","last_fullsized":"2024-05-08T22:33:19.312379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:47.862614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934403+0000","last_change":"2024-05-08T22:33:14.530133+0000","last_active":"2024-05-08T22:33:18.934403+0000","last_peered":"2024-05-08T22:33:18.934403+0000","last_clean":"2024-05-08T22:33:18.934403+0000","last_became_active":"2024-05-08T22:33:14.529854+0000","last_became_peered":"2024-05-08T22:33:14.529854+0000","last_unstale":"2024-05-08T22:33:18.934403+0000","last_undegraded":"2024-05-08T22:33:18.934403+0000","last_fullsized":"2024-05-08T22:33:18.934403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:59.313380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448755+0000","last_change":"2024-05-08T22:33:19.004993+0000","last_active":"2024-05-08T22:33:19.448755+0000","last_peered":"2024-05-08T22:33:19.448755+0000","last_clean":"2024-05-08T22:33:19.448755+0000","last_became_active":"2024-05-08T22:33:19.004152+0000","last_became_peered":"2024-05-08T22:33:19.004152+0000","last_unstale":"2024-05-08T22:33:19.448755+0000","last_undegraded":"2024-05-08T22:33:19.448755+0000","last_fullsized":"2024-05-08T22:33:19.448755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:39.634020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929034+0000","last_change":"2024-05-08T22:33:14.909377+0000","last_active":"2024-05-08T22:33:18.929034+0000","last_peered":"2024-05-08T22:33:18.929034+0000","last_clean":"2024-05-08T22:33:18.929034+0000","last_became_active":"2024-05-08T22:33:14.909047+0000","last_became_peered":"2024-05-08T22:33:14.909047+0000","last_unstale":"2024-05-08T22:33:18.929034+0000","last_undegraded":"2024-05-08T22:33:18.929034+0000","last_fullsized":"2024-05-08T22:33:18.929034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:32.048397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998562+0000","last_change":"2024-05-08T22:33:14.529685+0000","last_active":"2024-05-08T22:33:18.998562+0000","last_peered":"2024-05-08T22:33:18.998562+0000","last_clean":"2024-05-08T22:33:18.998562+0000","last_became_active":"2024-05-08T22:33:14.529417+0000","last_became_peered":"2024-05-08T22:33:14.529417+0000","last_unstale":"2024-05-08T22:33:18.998562+0000","last_undegraded":"2024-05-08T22:33:18.998562+0000","last_fullsized":"2024-05-08T22:33:18.998562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.786143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.314895+0000","last_change":"2024-05-08T22:33:18.948441+0000","last_active":"2024-05-08T22:33:19.314895+0000","last_peered":"2024-05-08T22:33:19.314895+0000","last_clean":"2024-05-08T22:33:19.314895+0000","last_became_active":"2024-05-08T22:33:18.948006+0000","last_became_peered":"2024-05-08T22:33:18.948006+0000","last_unstale":"2024-05-08T22:33:19.314895+0000","last_undegraded":"2024-05-08T22:33:19.314895+0000","last_fullsized":"2024-05-08T22:33:19.314895+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936140+0000","last_change":"2024-05-08T22:33:14.915374+0000","last_active":"2024-05-08T22:33:18.936140+0000","last_peered":"2024-05-08T22:33:18.936140+0000","last_clean":"2024-05-08T22:33:18.936140+0000","last_became_active":"2024-05-08T22:33:14.915035+0000","last_became_peered":"2024-05-08T22:33:14.915035+0000","last_unstale":"2024-05-08T22:33:18.936140+0000","last_undegraded":"2024-05-08T22:33:18.936140+0000","last_fullsized":"2024-05-08T22:33:18.936140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927757+0000","last_change":"2024-05-08T22:33:14.518968+0000","last_active":"2024-05-08T22:33:18.927757+0000","last_peered":"2024-05-08T22:33:18.927757+0000","last_clean":"2024-05-08T22:33:18.927757+0000","last_became_active":"2024-05-08T22:33:14.518318+0000","last_became_peered":"2024-05-08T22:33:14.518318+0000","last_unstale":"2024-05-08T22:33:18.927757+0000","last_undegraded":"2024-05-08T22:33:18.927757+0000","last_fullsized":"2024-05-08T22:33:18.927757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:25.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313504+0000","last_change":"2024-05-08T22:33:19.003063+0000","last_active":"2024-05-08T22:33:19.313504+0000","last_peered":"2024-05-08T22:33:19.313504+0000","last_clean":"2024-05-08T22:33:19.313504+0000","last_became_active":"2024-05-08T22:33:19.002557+0000","last_became_peered":"2024-05-08T22:33:19.002557+0000","last_unstale":"2024-05-08T22:33:19.313504+0000","last_undegraded":"2024-05-08T22:33:19.313504+0000","last_fullsized":"2024-05-08T22:33:19.313504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312756+0000","last_change":"2024-05-08T22:33:14.915824+0000","last_active":"2024-05-08T22:33:19.312756+0000","last_peered":"2024-05-08T22:33:19.312756+0000","last_clean":"2024-05-08T22:33:19.312756+0000","last_became_active":"2024-05-08T22:33:14.915554+0000","last_became_peered":"2024-05-08T22:33:14.915554+0000","last_unstale":"2024-05-08T22:33:19.312756+0000","last_undegraded":"2024-05-08T22:33:19.312756+0000","last_fullsized":"2024-05-08T22:33:19.312756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:10.398283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.937194+0000","last_change":"2024-05-08T22:33:14.526863+0000","last_active":"2024-05-08T22:33:18.937194+0000","last_peered":"2024-05-08T22:33:18.937194+0000","last_clean":"2024-05-08T22:33:18.937194+0000","last_became_active":"2024-05-08T22:33:14.526463+0000","last_became_peered":"2024-05-08T22:33:14.526463+0000","last_unstale":"2024-05-08T22:33:18.937194+0000","last_undegraded":"2024-05-08T22:33:18.937194+0000","last_fullsized":"2024-05-08T22:33:18.937194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.311963+0000","last_change":"2024-05-08T22:33:19.005291+0000","last_active":"2024-05-08T22:33:19.311963+0000","last_peered":"2024-05-08T22:33:19.311963+0000","last_clean":"2024-05-08T22:33:19.311963+0000","last_became_active":"2024-05-08T22:33:19.004777+0000","last_became_peered":"2024-05-08T22:33:19.004777+0000","last_unstale":"2024-05-08T22:33:19.311963+0000","last_undegraded":"2024-05-08T22:33:19.311963+0000","last_fullsized":"2024-05-08T22:33:19.311963+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:36.904973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448630+0000","last_change":"2024-05-08T22:33:14.911246+0000","last_active":"2024-05-08T22:33:19.448630+0000","last_peered":"2024-05-08T22:33:19.448630+0000","last_clean":"2024-05-08T22:33:19.448630+0000","last_became_active":"2024-05-08T22:33:14.910998+0000","last_became_peered":"2024-05-08T22:33:14.910998+0000","last_unstale":"2024-05-08T22:33:19.448630+0000","last_undegraded":"2024-05-08T22:33:19.448630+0000","last_fullsized":"2024-05-08T22:33:19.448630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999587+0000","last_change":"2024-05-08T22:33:14.529748+0000","last_active":"2024-05-08T22:33:18.999587+0000","last_peered":"2024-05-08T22:33:18.999587+0000","last_clean":"2024-05-08T22:33:18.999587+0000","last_became_active":"2024-05-08T22:33:14.529390+0000","last_became_peered":"2024-05-08T22:33:14.529390+0000","last_unstale":"2024-05-08T22:33:18.999587+0000","last_undegraded":"2024-05-08T22:33:18.999587+0000","last_fullsized":"2024-05-08T22:33:18.999587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:04.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.009380+0000","last_change":"2024-05-08T22:33:19.004064+0000","last_active":"2024-05-08T22:33:19.009380+0000","last_peered":"2024-05-08T22:33:19.009380+0000","last_clean":"2024-05-08T22:33:19.009380+0000","last_became_active":"2024-05-08T22:33:19.003638+0000","last_became_peered":"2024-05-08T22:33:19.003638+0000","last_unstale":"2024-05-08T22:33:19.009380+0000","last_undegraded":"2024-05-08T22:33:19.009380+0000","last_fullsized":"2024-05-08T22:33:19.009380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:54.283901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935398+0000","last_change":"2024-05-08T22:33:14.928058+0000","last_active":"2024-05-08T22:33:18.935398+0000","last_peered":"2024-05-08T22:33:18.935398+0000","last_clean":"2024-05-08T22:33:18.935398+0000","last_became_active":"2024-05-08T22:33:14.927408+0000","last_became_peered":"2024-05-08T22:33:14.927408+0000","last_unstale":"2024-05-08T22:33:18.935398+0000","last_undegraded":"2024-05-08T22:33:18.935398+0000","last_fullsized":"2024-05-08T22:33:18.935398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:22.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934334+0000","last_change":"2024-05-08T22:33:14.530667+0000","last_active":"2024-05-08T22:33:18.934334+0000","last_peered":"2024-05-08T22:33:18.934334+0000","last_clean":"2024-05-08T22:33:18.934334+0000","last_became_active":"2024-05-08T22:33:14.530429+0000","last_became_peered":"2024-05-08T22:33:14.530429+0000","last_unstale":"2024-05-08T22:33:18.934334+0000","last_undegraded":"2024-05-08T22:33:18.934334+0000","last_fullsized":"2024-05-08T22:33:18.934334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:38.212588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448536+0000","last_change":"2024-05-08T22:33:18.945810+0000","last_active":"2024-05-08T22:33:19.448536+0000","last_peered":"2024-05-08T22:33:19.448536+0000","last_clean":"2024-05-08T22:33:19.448536+0000","last_became_active":"2024-05-08T22:33:18.945204+0000","last_became_peered":"2024-05-08T22:33:18.945204+0000","last_unstale":"2024-05-08T22:33:19.448536+0000","last_undegraded":"2024-05-08T22:33:19.448536+0000","last_fullsized":"2024-05-08T22:33:19.448536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:26.949422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930747+0000","last_change":"2024-05-08T22:33:14.911989+0000","last_active":"2024-05-08T22:33:18.930747+0000","last_peered":"2024-05-08T22:33:18.930747+0000","last_clean":"2024-05-08T22:33:18.930747+0000","last_became_active":"2024-05-08T22:33:14.911685+0000","last_became_peered":"2024-05-08T22:33:14.911685+0000","last_unstale":"2024-05-08T22:33:18.930747+0000","last_undegraded":"2024-05-08T22:33:18.930747+0000","last_fullsized":"2024-05-08T22:33:18.930747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934973+0000","last_change":"2024-05-08T22:33:14.524541+0000","last_active":"2024-05-08T22:33:18.934973+0000","last_peered":"2024-05-08T22:33:18.934973+0000","last_clean":"2024-05-08T22:33:18.934973+0000","last_became_active":"2024-05-08T22:33:14.524290+0000","last_became_peered":"2024-05-08T22:33:14.524290+0000","last_unstale":"2024-05-08T22:33:18.934973+0000","last_undegraded":"2024-05-08T22:33:18.934973+0000","last_fullsized":"2024-05-08T22:33:18.934973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:52.145181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448609+0000","last_change":"2024-05-08T22:33:18.951903+0000","last_active":"2024-05-08T22:33:19.448609+0000","last_peered":"2024-05-08T22:33:19.448609+0000","last_clean":"2024-05-08T22:33:19.448609+0000","last_became_active":"2024-05-08T22:33:18.951473+0000","last_became_peered":"2024-05-08T22:33:18.951473+0000","last_unstale":"2024-05-08T22:33:19.448609+0000","last_undegraded":"2024-05-08T22:33:19.448609+0000","last_fullsized":"2024-05-08T22:33:19.448609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:19.057699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448450+0000","last_change":"2024-05-08T22:33:14.911724+0000","last_active":"2024-05-08T22:33:19.448450+0000","last_peered":"2024-05-08T22:33:19.448450+0000","last_clean":"2024-05-08T22:33:19.448450+0000","last_became_active":"2024-05-08T22:33:14.911493+0000","last_became_peered":"2024-05-08T22:33:14.911493+0000","last_unstale":"2024-05-08T22:33:19.448450+0000","last_undegraded":"2024-05-08T22:33:19.448450+0000","last_fullsized":"2024-05-08T22:33:19.448450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925211+0000","last_change":"2024-05-08T22:33:14.524989+0000","last_active":"2024-05-08T22:33:18.925211+0000","last_peered":"2024-05-08T22:33:18.925211+0000","last_clean":"2024-05-08T22:33:18.925211+0000","last_became_active":"2024-05-08T22:33:14.524715+0000","last_became_peered":"2024-05-08T22:33:14.524715+0000","last_unstale":"2024-05-08T22:33:18.925211+0000","last_undegraded":"2024-05-08T22:33:18.925211+0000","last_fullsized":"2024-05-08T22:33:18.925211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448775+0000","last_change":"2024-05-08T22:33:18.952088+0000","last_active":"2024-05-08T22:33:19.448775+0000","last_peered":"2024-05-08T22:33:19.448775+0000","last_clean":"2024-05-08T22:33:19.448775+0000","last_became_active":"2024-05-08T22:33:18.951660+0000","last_became_peered":"2024-05-08T22:33:18.951660+0000","last_unstale":"2024-05-08T22:33:19.448775+0000","last_undegraded":"2024-05-08T22:33:19.448775+0000","last_fullsized":"2024-05-08T22:33:19.448775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:42.700357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930121+0000","last_change":"2024-05-08T22:33:14.911625+0000","last_active":"2024-05-08T22:33:18.930121+0000","last_peered":"2024-05-08T22:33:18.930121+0000","last_clean":"2024-05-08T22:33:18.930121+0000","last_became_active":"2024-05-08T22:33:14.911368+0000","last_became_peered":"2024-05-08T22:33:14.911368+0000","last_unstale":"2024-05-08T22:33:18.930121+0000","last_undegraded":"2024-05-08T22:33:18.930121+0000","last_fullsized":"2024-05-08T22:33:18.930121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998588+0000","last_change":"2024-05-08T22:33:14.529960+0000","last_active":"2024-05-08T22:33:18.998588+0000","last_peered":"2024-05-08T22:33:18.998588+0000","last_clean":"2024-05-08T22:33:18.998588+0000","last_became_active":"2024-05-08T22:33:14.529479+0000","last_became_peered":"2024-05-08T22:33:14.529479+0000","last_unstale":"2024-05-08T22:33:18.998588+0000","last_undegraded":"2024-05-08T22:33:18.998588+0000","last_fullsized":"2024-05-08T22:33:18.998588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:49.272820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.010817+0000","last_change":"2024-05-08T22:33:19.005638+0000","last_active":"2024-05-08T22:33:19.010817+0000","last_peered":"2024-05-08T22:33:19.010817+0000","last_clean":"2024-05-08T22:33:19.010817+0000","last_became_active":"2024-05-08T22:33:19.003876+0000","last_became_peered":"2024-05-08T22:33:19.003876+0000","last_unstale":"2024-05-08T22:33:19.010817+0000","last_undegraded":"2024-05-08T22:33:19.010817+0000","last_fullsized":"2024-05-08T22:33:19.010817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312713+0000","last_change":"2024-05-08T22:33:14.926812+0000","last_active":"2024-05-08T22:33:19.312713+0000","last_peered":"2024-05-08T22:33:19.312713+0000","last_clean":"2024-05-08T22:33:19.312713+0000","last_became_active":"2024-05-08T22:33:14.926536+0000","last_became_peered":"2024-05-08T22:33:14.926536+0000","last_unstale":"2024-05-08T22:33:19.312713+0000","last_undegraded":"2024-05-08T22:33:19.312713+0000","last_fullsized":"2024-05-08T22:33:19.312713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:18.111855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934524+0000","last_change":"2024-05-08T22:33:14.523530+0000","last_active":"2024-05-08T22:33:18.934524+0000","last_peered":"2024-05-08T22:33:18.934524+0000","last_clean":"2024-05-08T22:33:18.934524+0000","last_became_active":"2024-05-08T22:33:14.523193+0000","last_became_peered":"2024-05-08T22:33:14.523193+0000","last_unstale":"2024-05-08T22:33:18.934524+0000","last_undegraded":"2024-05-08T22:33:18.934524+0000","last_fullsized":"2024-05-08T22:33:18.934524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998772+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:33:18.998772+0000","last_peered":"2024-05-08T22:33:18.998772+0000","last_clean":"2024-05-08T22:33:18.998772+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:33:18.998772+0000","last_undegraded":"2024-05-08T22:33:18.998772+0000","last_fullsized":"2024-05-08T22:33:18.998772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007908+0000","last_change":"2024-05-08T22:33:19.006286+0000","last_active":"2024-05-08T22:33:19.007908+0000","last_peered":"2024-05-08T22:33:19.007908+0000","last_clean":"2024-05-08T22:33:19.007908+0000","last_became_active":"2024-05-08T22:33:19.005632+0000","last_became_peered":"2024-05-08T22:33:19.005632+0000","last_unstale":"2024-05-08T22:33:19.007908+0000","last_undegraded":"2024-05-08T22:33:19.007908+0000","last_fullsized":"2024-05-08T22:33:19.007908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:39.728431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999358+0000","last_change":"2024-05-08T22:33:14.931567+0000","last_active":"2024-05-08T22:33:18.999358+0000","last_peered":"2024-05-08T22:33:18.999358+0000","last_clean":"2024-05-08T22:33:18.999358+0000","last_became_active":"2024-05-08T22:33:14.931285+0000","last_became_peered":"2024-05-08T22:33:14.931285+0000","last_unstale":"2024-05-08T22:33:18.999358+0000","last_undegraded":"2024-05-08T22:33:18.999358+0000","last_fullsized":"2024-05-08T22:33:18.999358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925484+0000","last_change":"2024-05-08T22:33:14.523189+0000","last_active":"2024-05-08T22:33:18.925484+0000","last_peered":"2024-05-08T22:33:18.925484+0000","last_clean":"2024-05-08T22:33:18.925484+0000","last_became_active":"2024-05-08T22:33:14.522942+0000","last_became_peered":"2024-05-08T22:33:14.522942+0000","last_unstale":"2024-05-08T22:33:18.925484+0000","last_undegraded":"2024-05-08T22:33:18.925484+0000","last_fullsized":"2024-05-08T22:33:18.925484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.007291+0000","last_change":"2024-05-08T22:33:19.003092+0000","last_active":"2024-05-08T22:33:19.007291+0000","last_peered":"2024-05-08T22:33:19.007291+0000","last_clean":"2024-05-08T22:33:19.007291+0000","last_became_active":"2024-05-08T22:33:19.002313+0000","last_became_peered":"2024-05-08T22:33:19.002313+0000","last_unstale":"2024-05-08T22:33:19.007291+0000","last_undegraded":"2024-05-08T22:33:19.007291+0000","last_fullsized":"2024-05-08T22:33:19.007291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:17.965361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934771+0000","last_change":"2024-05-08T22:33:14.927660+0000","last_active":"2024-05-08T22:33:18.934771+0000","last_peered":"2024-05-08T22:33:18.934771+0000","last_clean":"2024-05-08T22:33:18.934771+0000","last_became_active":"2024-05-08T22:33:14.925184+0000","last_became_peered":"2024-05-08T22:33:14.925184+0000","last_unstale":"2024-05-08T22:33:18.934771+0000","last_undegraded":"2024-05-08T22:33:18.934771+0000","last_fullsized":"2024-05-08T22:33:18.934771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:55.225256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934291+0000","last_change":"2024-05-08T22:33:14.524892+0000","last_active":"2024-05-08T22:33:18.934291+0000","last_peered":"2024-05-08T22:33:18.934291+0000","last_clean":"2024-05-08T22:33:18.934291+0000","last_became_active":"2024-05-08T22:33:14.524584+0000","last_became_peered":"2024-05-08T22:33:14.524584+0000","last_unstale":"2024-05-08T22:33:18.934291+0000","last_undegraded":"2024-05-08T22:33:18.934291+0000","last_fullsized":"2024-05-08T22:33:18.934291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953385+0000","last_change":"2024-05-08T22:33:18.947268+0000","last_active":"2024-05-08T22:33:18.953385+0000","last_peered":"2024-05-08T22:33:18.953385+0000","last_clean":"2024-05-08T22:33:18.953385+0000","last_became_active":"2024-05-08T22:33:18.946822+0000","last_became_peered":"2024-05-08T22:33:18.946822+0000","last_unstale":"2024-05-08T22:33:18.953385+0000","last_undegraded":"2024-05-08T22:33:18.953385+0000","last_fullsized":"2024-05-08T22:33:18.953385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:20.669800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999808+0000","last_change":"2024-05-08T22:33:14.925288+0000","last_active":"2024-05-08T22:33:18.999808+0000","last_peered":"2024-05-08T22:33:18.999808+0000","last_clean":"2024-05-08T22:33:18.999808+0000","last_became_active":"2024-05-08T22:33:14.924982+0000","last_became_peered":"2024-05-08T22:33:14.924982+0000","last_unstale":"2024-05-08T22:33:18.999808+0000","last_undegraded":"2024-05-08T22:33:18.999808+0000","last_fullsized":"2024-05-08T22:33:18.999808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925406+0000","last_change":"2024-05-08T22:33:14.531914+0000","last_active":"2024-05-08T22:33:18.925406+0000","last_peered":"2024-05-08T22:33:18.925406+0000","last_clean":"2024-05-08T22:33:18.925406+0000","last_became_active":"2024-05-08T22:33:14.531550+0000","last_became_peered":"2024-05-08T22:33:14.531550+0000","last_unstale":"2024-05-08T22:33:18.925406+0000","last_undegraded":"2024-05-08T22:33:18.925406+0000","last_fullsized":"2024-05-08T22:33:18.925406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:12.564410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006729+0000","last_change":"2024-05-08T22:33:19.005176+0000","last_active":"2024-05-08T22:33:19.006729+0000","last_peered":"2024-05-08T22:33:19.006729+0000","last_clean":"2024-05-08T22:33:19.006729+0000","last_became_active":"2024-05-08T22:33:19.004757+0000","last_became_peered":"2024-05-08T22:33:19.004757+0000","last_unstale":"2024-05-08T22:33:19.006729+0000","last_undegraded":"2024-05-08T22:33:19.006729+0000","last_fullsized":"2024-05-08T22:33:19.006729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:22.142109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.924451+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.922189+0000","last_became_peered":"2024-05-08T22:33:14.922189+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:20.957859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927288+0000","last_change":"2024-05-08T22:33:14.527475+0000","last_active":"2024-05-08T22:33:18.927288+0000","last_peered":"2024-05-08T22:33:18.927288+0000","last_clean":"2024-05-08T22:33:18.927288+0000","last_became_active":"2024-05-08T22:33:14.526627+0000","last_became_peered":"2024-05-08T22:33:14.526627+0000","last_unstale":"2024-05-08T22:33:18.927288+0000","last_undegraded":"2024-05-08T22:33:18.927288+0000","last_fullsized":"2024-05-08T22:33:18.927288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:00.386313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312868+0000","last_change":"2024-05-08T22:33:18.946891+0000","last_active":"2024-05-08T22:33:19.312868+0000","last_peered":"2024-05-08T22:33:19.312868+0000","last_clean":"2024-05-08T22:33:19.312868+0000","last_became_active":"2024-05-08T22:33:18.946534+0000","last_became_peered":"2024-05-08T22:33:18.946534+0000","last_unstale":"2024-05-08T22:33:19.312868+0000","last_undegraded":"2024-05-08T22:33:19.312868+0000","last_fullsized":"2024-05-08T22:33:19.312868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:36.314513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999925+0000","last_change":"2024-05-08T22:33:14.918650+0000","last_active":"2024-05-08T22:33:18.999925+0000","last_peered":"2024-05-08T22:33:18.999925+0000","last_clean":"2024-05-08T22:33:18.999925+0000","last_became_active":"2024-05-08T22:33:14.918389+0000","last_became_peered":"2024-05-08T22:33:14.918389+0000","last_unstale":"2024-05-08T22:33:18.999925+0000","last_undegraded":"2024-05-08T22:33:18.999925+0000","last_fullsized":"2024-05-08T22:33:18.999925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:15.733116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925062+0000","last_change":"2024-05-08T22:33:14.522438+0000","last_active":"2024-05-08T22:33:18.925062+0000","last_peered":"2024-05-08T22:33:18.925062+0000","last_clean":"2024-05-08T22:33:18.925062+0000","last_became_active":"2024-05-08T22:33:14.522094+0000","last_became_peered":"2024-05-08T22:33:14.522094+0000","last_unstale":"2024-05-08T22:33:18.925062+0000","last_undegraded":"2024-05-08T22:33:18.925062+0000","last_fullsized":"2024-05-08T22:33:18.925062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:59.775081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.950611+0000","last_change":"2024-05-08T22:33:18.944271+0000","last_active":"2024-05-08T22:33:18.950611+0000","last_peered":"2024-05-08T22:33:18.950611+0000","last_clean":"2024-05-08T22:33:18.950611+0000","last_became_active":"2024-05-08T22:33:18.943934+0000","last_became_peered":"2024-05-08T22:33:18.943934+0000","last_unstale":"2024-05-08T22:33:18.950611+0000","last_undegraded":"2024-05-08T22:33:18.950611+0000","last_fullsized":"2024-05-08T22:33:18.950611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:34.381096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926385+0000","last_change":"2024-05-08T22:33:14.915516+0000","last_active":"2024-05-08T22:33:18.926385+0000","last_peered":"2024-05-08T22:33:18.926385+0000","last_clean":"2024-05-08T22:33:18.926385+0000","last_became_active":"2024-05-08T22:33:14.915253+0000","last_became_peered":"2024-05-08T22:33:14.915253+0000","last_unstale":"2024-05-08T22:33:18.926385+0000","last_undegraded":"2024-05-08T22:33:18.926385+0000","last_fullsized":"2024-05-08T22:33:18.926385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925993+0000","last_change":"2024-05-08T22:33:14.517759+0000","last_active":"2024-05-08T22:33:18.925993+0000","last_peered":"2024-05-08T22:33:18.925993+0000","last_clean":"2024-05-08T22:33:18.925993+0000","last_became_active":"2024-05-08T22:33:14.517109+0000","last_became_peered":"2024-05-08T22:33:14.517109+0000","last_unstale":"2024-05-08T22:33:18.925993+0000","last_undegraded":"2024-05-08T22:33:18.925993+0000","last_fullsized":"2024-05-08T22:33:18.925993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:13.973941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313006+0000","last_change":"2024-05-08T22:33:19.005763+0000","last_active":"2024-05-08T22:33:19.313006+0000","last_peered":"2024-05-08T22:33:19.313006+0000","last_clean":"2024-05-08T22:33:19.313006+0000","last_became_active":"2024-05-08T22:33:19.004827+0000","last_became_peered":"2024-05-08T22:33:19.004827+0000","last_unstale":"2024-05-08T22:33:19.313006+0000","last_undegraded":"2024-05-08T22:33:19.313006+0000","last_fullsized":"2024-05-08T22:33:19.313006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:24.712864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998964+0000","last_change":"2024-05-08T22:33:14.918080+0000","last_active":"2024-05-08T22:33:18.998964+0000","last_peered":"2024-05-08T22:33:18.998964+0000","last_clean":"2024-05-08T22:33:18.998964+0000","last_became_active":"2024-05-08T22:33:14.917791+0000","last_became_peered":"2024-05-08T22:33:14.917791+0000","last_unstale":"2024-05-08T22:33:18.998964+0000","last_undegraded":"2024-05-08T22:33:18.998964+0000","last_fullsized":"2024-05-08T22:33:18.998964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934850+0000","last_change":"2024-05-08T22:33:14.527314+0000","last_active":"2024-05-08T22:33:18.934850+0000","last_peered":"2024-05-08T22:33:18.934850+0000","last_clean":"2024-05-08T22:33:18.934850+0000","last_became_active":"2024-05-08T22:33:14.526667+0000","last_became_peered":"2024-05-08T22:33:14.526667+0000","last_unstale":"2024-05-08T22:33:18.934850+0000","last_undegraded":"2024-05-08T22:33:18.934850+0000","last_fullsized":"2024-05-08T22:33:18.934850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:40.385273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312923+0000","last_change":"2024-05-08T22:33:19.004038+0000","last_active":"2024-05-08T22:33:19.312923+0000","last_peered":"2024-05-08T22:33:19.312923+0000","last_clean":"2024-05-08T22:33:19.312923+0000","last_became_active":"2024-05-08T22:33:19.002976+0000","last_became_peered":"2024-05-08T22:33:19.002976+0000","last_unstale":"2024-05-08T22:33:19.312923+0000","last_undegraded":"2024-05-08T22:33:19.312923+0000","last_fullsized":"2024-05-08T22:33:19.312923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.506642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930214+0000","last_change":"2024-05-08T22:33:14.924563+0000","last_active":"2024-05-08T22:33:18.930214+0000","last_peered":"2024-05-08T22:33:18.930214+0000","last_clean":"2024-05-08T22:33:18.930214+0000","last_became_active":"2024-05-08T22:33:14.924299+0000","last_became_peered":"2024-05-08T22:33:14.924299+0000","last_unstale":"2024-05-08T22:33:18.930214+0000","last_undegraded":"2024-05-08T22:33:18.930214+0000","last_fullsized":"2024-05-08T22:33:18.930214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931836+0000","last_change":"2024-05-08T22:33:14.531490+0000","last_active":"2024-05-08T22:33:18.931836+0000","last_peered":"2024-05-08T22:33:18.931836+0000","last_clean":"2024-05-08T22:33:18.931836+0000","last_became_active":"2024-05-08T22:33:14.530820+0000","last_became_peered":"2024-05-08T22:33:14.530820+0000","last_unstale":"2024-05-08T22:33:18.931836+0000","last_undegraded":"2024-05-08T22:33:18.931836+0000","last_fullsized":"2024-05-08T22:33:18.931836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:59.224955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.006802+0000","last_change":"2024-05-08T22:33:19.005429+0000","last_active":"2024-05-08T22:33:19.006802+0000","last_peered":"2024-05-08T22:33:19.006802+0000","last_clean":"2024-05-08T22:33:19.006802+0000","last_became_active":"2024-05-08T22:33:19.004941+0000","last_became_peered":"2024-05-08T22:33:19.004941+0000","last_unstale":"2024-05-08T22:33:19.006802+0000","last_undegraded":"2024-05-08T22:33:19.006802+0000","last_fullsized":"2024-05-08T22:33:19.006802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:06.367658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999232+0000","last_change":"2024-05-08T22:33:14.919406+0000","last_active":"2024-05-08T22:33:18.999232+0000","last_peered":"2024-05-08T22:33:18.999232+0000","last_clean":"2024-05-08T22:33:18.999232+0000","last_became_active":"2024-05-08T22:33:14.919204+0000","last_became_peered":"2024-05-08T22:33:14.919204+0000","last_unstale":"2024-05-08T22:33:18.999232+0000","last_undegraded":"2024-05-08T22:33:18.999232+0000","last_fullsized":"2024-05-08T22:33:18.999232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:24.279224+0000","last_change":"2024-05-08T22:33:14.523404+0000","last_active":"2024-05-08T22:33:24.279224+0000","last_peered":"2024-05-08T22:33:24.279224+0000","last_clean":"2024-05-08T22:33:24.279224+0000","last_became_active":"2024-05-08T22:33:14.522325+0000","last_became_peered":"2024-05-08T22:33:14.522325+0000","last_unstale":"2024-05-08T22:33:24.279224+0000","last_undegraded":"2024-05-08T22:33:24.279224+0000","last_fullsized":"2024-05-08T22:33:24.279224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:50.617846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448558+0000","last_change":"2024-05-08T22:33:18.950110+0000","last_active":"2024-05-08T22:33:19.448558+0000","last_peered":"2024-05-08T22:33:19.448558+0000","last_clean":"2024-05-08T22:33:19.448558+0000","last_became_active":"2024-05-08T22:33:18.949555+0000","last_became_peered":"2024-05-08T22:33:18.949555+0000","last_unstale":"2024-05-08T22:33:19.448558+0000","last_undegraded":"2024-05-08T22:33:19.448558+0000","last_fullsized":"2024-05-08T22:33:19.448558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:15.389269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926732+0000","last_change":"2024-05-08T22:33:14.908111+0000","last_active":"2024-05-08T22:33:18.926732+0000","last_peered":"2024-05-08T22:33:18.926732+0000","last_clean":"2024-05-08T22:33:18.926732+0000","last_became_active":"2024-05-08T22:33:14.907276+0000","last_became_peered":"2024-05-08T22:33:14.907276+0000","last_unstale":"2024-05-08T22:33:18.926732+0000","last_undegraded":"2024-05-08T22:33:18.926732+0000","last_fullsized":"2024-05-08T22:33:18.926732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:53.154720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934191+0000","last_change":"2024-05-08T22:33:14.521905+0000","last_active":"2024-05-08T22:33:18.934191+0000","last_peered":"2024-05-08T22:33:18.934191+0000","last_clean":"2024-05-08T22:33:18.934191+0000","last_became_active":"2024-05-08T22:33:14.521518+0000","last_became_peered":"2024-05-08T22:33:14.521518+0000","last_unstale":"2024-05-08T22:33:18.934191+0000","last_undegraded":"2024-05-08T22:33:18.934191+0000","last_fullsized":"2024-05-08T22:33:18.934191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.315185+0000","last_change":"2024-05-08T22:33:18.939252+0000","last_active":"2024-05-08T22:33:19.315185+0000","last_peered":"2024-05-08T22:33:19.315185+0000","last_clean":"2024-05-08T22:33:19.315185+0000","last_became_active":"2024-05-08T22:33:18.938564+0000","last_became_peered":"2024-05-08T22:33:18.938564+0000","last_unstale":"2024-05-08T22:33:19.315185+0000","last_undegraded":"2024-05-08T22:33:19.315185+0000","last_fullsized":"2024-05-08T22:33:19.315185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:42.775813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931955+0000","last_change":"2024-05-08T22:33:14.932581+0000","last_active":"2024-05-08T22:33:18.931955+0000","last_peered":"2024-05-08T22:33:18.931955+0000","last_clean":"2024-05-08T22:33:18.931955+0000","last_became_active":"2024-05-08T22:33:14.932332+0000","last_became_peered":"2024-05-08T22:33:14.932332+0000","last_unstale":"2024-05-08T22:33:18.931955+0000","last_undegraded":"2024-05-08T22:33:18.931955+0000","last_fullsized":"2024-05-08T22:33:18.931955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:49:02.412782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935272+0000","last_change":"2024-05-08T22:33:14.525414+0000","last_active":"2024-05-08T22:33:18.935272+0000","last_peered":"2024-05-08T22:33:18.935272+0000","last_clean":"2024-05-08T22:33:18.935272+0000","last_became_active":"2024-05-08T22:33:14.524869+0000","last_became_peered":"2024-05-08T22:33:14.524869+0000","last_unstale":"2024-05-08T22:33:18.935272+0000","last_undegraded":"2024-05-08T22:33:18.935272+0000","last_fullsized":"2024-05-08T22:33:18.935272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:39.269473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312533+0000","last_change":"2024-05-08T22:33:18.948135+0000","last_active":"2024-05-08T22:33:19.312533+0000","last_peered":"2024-05-08T22:33:19.312533+0000","last_clean":"2024-05-08T22:33:19.312533+0000","last_became_active":"2024-05-08T22:33:18.947623+0000","last_became_peered":"2024-05-08T22:33:18.947623+0000","last_unstale":"2024-05-08T22:33:19.312533+0000","last_undegraded":"2024-05-08T22:33:19.312533+0000","last_fullsized":"2024-05-08T22:33:19.312533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:55.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935185+0000","last_change":"2024-05-08T22:33:14.925264+0000","last_active":"2024-05-08T22:33:18.935185+0000","last_peered":"2024-05-08T22:33:18.935185+0000","last_clean":"2024-05-08T22:33:18.935185+0000","last_became_active":"2024-05-08T22:33:14.924936+0000","last_became_peered":"2024-05-08T22:33:14.924936+0000","last_unstale":"2024-05-08T22:33:18.935185+0000","last_undegraded":"2024-05-08T22:33:18.935185+0000","last_fullsized":"2024-05-08T22:33:18.935185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:46.857960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927536+0000","last_change":"2024-05-08T22:33:14.519535+0000","last_active":"2024-05-08T22:33:18.927536+0000","last_peered":"2024-05-08T22:33:18.927536+0000","last_clean":"2024-05-08T22:33:18.927536+0000","last_became_active":"2024-05-08T22:33:14.519264+0000","last_became_peered":"2024-05-08T22:33:14.519264+0000","last_unstale":"2024-05-08T22:33:18.927536+0000","last_undegraded":"2024-05-08T22:33:18.927536+0000","last_fullsized":"2024-05-08T22:33:18.927536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:59.500931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:19.007144+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:19.006007+0000","last_became_peered":"2024-05-08T22:33:19.006007+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:50.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.011031+0000","last_change":"2024-05-08T22:33:19.006206+0000","last_active":"2024-05-08T22:33:19.011031+0000","last_peered":"2024-05-08T22:33:19.011031+0000","last_clean":"2024-05-08T22:33:19.011031+0000","last_became_active":"2024-05-08T22:33:19.004897+0000","last_became_peered":"2024-05-08T22:33:19.004897+0000","last_unstale":"2024-05-08T22:33:19.011031+0000","last_undegraded":"2024-05-08T22:33:19.011031+0000","last_fullsized":"2024-05-08T22:33:19.011031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:11.289634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448478+0000","last_change":"2024-05-08T22:33:14.906510+0000","last_active":"2024-05-08T22:33:19.448478+0000","last_peered":"2024-05-08T22:33:19.448478+0000","last_clean":"2024-05-08T22:33:19.448478+0000","last_became_active":"2024-05-08T22:33:14.906227+0000","last_became_peered":"2024-05-08T22:33:14.906227+0000","last_unstale":"2024-05-08T22:33:19.448478+0000","last_undegraded":"2024-05-08T22:33:19.448478+0000","last_fullsized":"2024-05-08T22:33:19.448478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935784+0000","last_change":"2024-05-08T22:33:14.523994+0000","last_active":"2024-05-08T22:33:18.935784+0000","last_peered":"2024-05-08T22:33:18.935784+0000","last_clean":"2024-05-08T22:33:18.935784+0000","last_became_active":"2024-05-08T22:33:14.523710+0000","last_became_peered":"2024-05-08T22:33:14.523710+0000","last_unstale":"2024-05-08T22:33:18.935784+0000","last_undegraded":"2024-05-08T22:33:18.935784+0000","last_fullsized":"2024-05-08T22:33:18.935784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:56.994588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.944980+0000","last_change":"2024-05-08T22:33:18.944209+0000","last_active":"2024-05-08T22:33:18.944980+0000","last_peered":"2024-05-08T22:33:18.944980+0000","last_clean":"2024-05-08T22:33:18.944980+0000","last_became_active":"2024-05-08T22:33:18.943824+0000","last_became_peered":"2024-05-08T22:33:18.943824+0000","last_unstale":"2024-05-08T22:33:18.944980+0000","last_undegraded":"2024-05-08T22:33:18.944980+0000","last_fullsized":"2024-05-08T22:33:18.944980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:04.454019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312284+0000","last_change":"2024-05-08T22:33:14.912166+0000","last_active":"2024-05-08T22:33:19.312284+0000","last_peered":"2024-05-08T22:33:19.312284+0000","last_clean":"2024-05-08T22:33:19.312284+0000","last_became_active":"2024-05-08T22:33:14.911941+0000","last_became_peered":"2024-05-08T22:33:14.911941+0000","last_unstale":"2024-05-08T22:33:19.312284+0000","last_undegraded":"2024-05-08T22:33:19.312284+0000","last_fullsized":"2024-05-08T22:33:19.312284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943398+0000","last_change":"2024-05-08T22:33:14.524713+0000","last_active":"2024-05-08T22:33:18.943398+0000","last_peered":"2024-05-08T22:33:18.943398+0000","last_clean":"2024-05-08T22:33:18.943398+0000","last_became_active":"2024-05-08T22:33:14.523688+0000","last_became_peered":"2024-05-08T22:33:14.523688+0000","last_unstale":"2024-05-08T22:33:18.943398+0000","last_undegraded":"2024-05-08T22:33:18.943398+0000","last_fullsized":"2024-05-08T22:33:18.943398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:12.770341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928004+0000","last_change":"2024-05-08T22:33:14.518843+0000","last_active":"2024-05-08T22:33:18.928004+0000","last_peered":"2024-05-08T22:33:18.928004+0000","last_clean":"2024-05-08T22:33:18.928004+0000","last_became_active":"2024-05-08T22:33:14.518002+0000","last_became_peered":"2024-05-08T22:33:14.518002+0000","last_unstale":"2024-05-08T22:33:18.928004+0000","last_undegraded":"2024-05-08T22:33:18.928004+0000","last_fullsized":"2024-05-08T22:33:18.928004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:17:40.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931587+0000","last_change":"2024-05-08T22:33:14.914102+0000","last_active":"2024-05-08T22:33:18.931587+0000","last_peered":"2024-05-08T22:33:18.931587+0000","last_clean":"2024-05-08T22:33:18.931587+0000","last_became_active":"2024-05-08T22:33:14.913777+0000","last_became_peered":"2024-05-08T22:33:14.913777+0000","last_unstale":"2024-05-08T22:33:18.931587+0000","last_undegraded":"2024-05-08T22:33:18.931587+0000","last_fullsized":"2024-05-08T22:33:18.931587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.633541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.953813+0000","last_change":"2024-05-08T22:33:18.949165+0000","last_active":"2024-05-08T22:33:18.953813+0000","last_peered":"2024-05-08T22:33:18.953813+0000","last_clean":"2024-05-08T22:33:18.953813+0000","last_became_active":"2024-05-08T22:33:18.948660+0000","last_became_peered":"2024-05-08T22:33:18.948660+0000","last_unstale":"2024-05-08T22:33:18.953813+0000","last_undegraded":"2024-05-08T22:33:18.953813+0000","last_fullsized":"2024-05-08T22:33:18.953813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:44.316596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.943131+0000","last_change":"2024-05-08T22:33:14.532226+0000","last_active":"2024-05-08T22:33:18.943131+0000","last_peered":"2024-05-08T22:33:18.943131+0000","last_clean":"2024-05-08T22:33:18.943131+0000","last_became_active":"2024-05-08T22:33:14.531943+0000","last_became_peered":"2024-05-08T22:33:14.531943+0000","last_unstale":"2024-05-08T22:33:18.943131+0000","last_undegraded":"2024-05-08T22:33:18.943131+0000","last_fullsized":"2024-05-08T22:33:18.943131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:13.487000+0000","last_clean_scrub_stamp":"2024-05-08T22:33:13.487000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:51.052408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928212+0000","last_change":"2024-05-08T22:33:14.907843+0000","last_active":"2024-05-08T22:33:18.928212+0000","last_peered":"2024-05-08T22:33:18.928212+0000","last_clean":"2024-05-08T22:33:18.928212+0000","last_became_active":"2024-05-08T22:33:14.907511+0000","last_became_peered":"2024-05-08T22:33:14.907511+0000","last_unstale":"2024-05-08T22:33:18.928212+0000","last_undegraded":"2024-05-08T22:33:18.928212+0000","last_fullsized":"2024-05-08T22:33:18.928212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:38.769965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312725+0000","last_change":"2024-05-08T22:33:18.952988+0000","last_active":"2024-05-08T22:33:19.312725+0000","last_peered":"2024-05-08T22:33:19.312725+0000","last_clean":"2024-05-08T22:33:19.312725+0000","last_became_active":"2024-05-08T22:33:18.948797+0000","last_became_peered":"2024-05-08T22:33:18.948797+0000","last_unstale":"2024-05-08T22:33:19.312725+0000","last_undegraded":"2024-05-08T22:33:19.312725+0000","last_fullsized":"2024-05-08T22:33:19.312725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:17.905653+0000","last_clean_scrub_stamp":"2024-05-08T22:33:17.905653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:10.379649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926181+0000","last_change":"2024-05-08T22:33:14.915829+0000","last_active":"2024-05-08T22:33:18.926181+0000","last_peered":"2024-05-08T22:33:18.926181+0000","last_clean":"2024-05-08T22:33:18.926181+0000","last_became_active":"2024-05-08T22:33:14.915496+0000","last_became_peered":"2024-05-08T22:33:14.915496+0000","last_unstale":"2024-05-08T22:33:18.926181+0000","last_undegraded":"2024-05-08T22:33:18.926181+0000","last_fullsized":"2024-05-08T22:33:18.926181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925052+0000","last_change":"2024-05-08T22:33:14.930927+0000","last_active":"2024-05-08T22:33:18.925052+0000","last_peered":"2024-05-08T22:33:18.925052+0000","last_clean":"2024-05-08T22:33:18.925052+0000","last_became_active":"2024-05-08T22:33:14.930569+0000","last_became_peered":"2024-05-08T22:33:14.930569+0000","last_unstale":"2024-05-08T22:33:18.925052+0000","last_undegraded":"2024-05-08T22:33:18.925052+0000","last_fullsized":"2024-05-08T22:33:18.925052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934767+0000","last_change":"2024-05-08T22:33:14.928385+0000","last_active":"2024-05-08T22:33:18.934767+0000","last_peered":"2024-05-08T22:33:18.934767+0000","last_clean":"2024-05-08T22:33:18.934767+0000","last_became_active":"2024-05-08T22:33:14.928162+0000","last_became_peered":"2024-05-08T22:33:14.928162+0000","last_unstale":"2024-05-08T22:33:18.934767+0000","last_undegraded":"2024-05-08T22:33:18.934767+0000","last_fullsized":"2024-05-08T22:33:18.934767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:11.717668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931430+0000","last_change":"2024-05-08T22:33:14.916147+0000","last_active":"2024-05-08T22:33:18.931430+0000","last_peered":"2024-05-08T22:33:18.931430+0000","last_clean":"2024-05-08T22:33:18.931430+0000","last_became_active":"2024-05-08T22:33:14.915762+0000","last_became_peered":"2024-05-08T22:33:14.915762+0000","last_unstale":"2024-05-08T22:33:18.931430+0000","last_undegraded":"2024-05-08T22:33:18.931430+0000","last_fullsized":"2024-05-08T22:33:18.931430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:28.668944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.313453+0000","last_change":"2024-05-08T22:33:14.912915+0000","last_active":"2024-05-08T22:33:19.313453+0000","last_peered":"2024-05-08T22:33:19.313453+0000","last_clean":"2024-05-08T22:33:19.313453+0000","last_became_active":"2024-05-08T22:33:14.912698+0000","last_became_peered":"2024-05-08T22:33:14.912698+0000","last_unstale":"2024-05-08T22:33:19.313453+0000","last_undegraded":"2024-05-08T22:33:19.313453+0000","last_fullsized":"2024-05-08T22:33:19.313453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:21.410357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448664+0000","last_change":"2024-05-08T22:33:14.907107+0000","last_active":"2024-05-08T22:33:19.448664+0000","last_peered":"2024-05-08T22:33:19.448664+0000","last_clean":"2024-05-08T22:33:19.448664+0000","last_became_active":"2024-05-08T22:33:14.906444+0000","last_became_peered":"2024-05-08T22:33:14.906444+0000","last_unstale":"2024-05-08T22:33:19.448664+0000","last_undegraded":"2024-05-08T22:33:19.448664+0000","last_fullsized":"2024-05-08T22:33:19.448664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929966+0000","last_change":"2024-05-08T22:33:14.913584+0000","last_active":"2024-05-08T22:33:18.929966+0000","last_peered":"2024-05-08T22:33:18.929966+0000","last_clean":"2024-05-08T22:33:18.929966+0000","last_became_active":"2024-05-08T22:33:14.913304+0000","last_became_peered":"2024-05-08T22:33:14.913304+0000","last_unstale":"2024-05-08T22:33:18.929966+0000","last_undegraded":"2024-05-08T22:33:18.929966+0000","last_fullsized":"2024-05-08T22:33:18.929966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:45.448508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312698+0000","last_change":"2024-05-08T22:33:14.905976+0000","last_active":"2024-05-08T22:33:19.312698+0000","last_peered":"2024-05-08T22:33:19.312698+0000","last_clean":"2024-05-08T22:33:19.312698+0000","last_became_active":"2024-05-08T22:33:14.905734+0000","last_became_peered":"2024-05-08T22:33:14.905734+0000","last_unstale":"2024-05-08T22:33:19.312698+0000","last_undegraded":"2024-05-08T22:33:19.312698+0000","last_fullsized":"2024-05-08T22:33:19.312698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935694+0000","last_change":"2024-05-08T22:33:14.927777+0000","last_active":"2024-05-08T22:33:18.935694+0000","last_peered":"2024-05-08T22:33:18.935694+0000","last_clean":"2024-05-08T22:33:18.935694+0000","last_became_active":"2024-05-08T22:33:14.927297+0000","last_became_peered":"2024-05-08T22:33:14.927297+0000","last_unstale":"2024-05-08T22:33:18.935694+0000","last_undegraded":"2024-05-08T22:33:18.935694+0000","last_fullsized":"2024-05-08T22:33:18.935694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:16.838261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312806+0000","last_change":"2024-05-08T22:33:14.911565+0000","last_active":"2024-05-08T22:33:19.312806+0000","last_peered":"2024-05-08T22:33:19.312806+0000","last_clean":"2024-05-08T22:33:19.312806+0000","last_became_active":"2024-05-08T22:33:14.909251+0000","last_became_peered":"2024-05-08T22:33:14.909251+0000","last_unstale":"2024-05-08T22:33:19.312806+0000","last_undegraded":"2024-05-08T22:33:19.312806+0000","last_fullsized":"2024-05-08T22:33:19.312806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:54.721070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934680+0000","last_change":"2024-05-08T22:33:14.925382+0000","last_active":"2024-05-08T22:33:18.934680+0000","last_peered":"2024-05-08T22:33:18.934680+0000","last_clean":"2024-05-08T22:33:18.934680+0000","last_became_active":"2024-05-08T22:33:14.925133+0000","last_became_peered":"2024-05-08T22:33:14.925133+0000","last_unstale":"2024-05-08T22:33:18.934680+0000","last_undegraded":"2024-05-08T22:33:18.934680+0000","last_fullsized":"2024-05-08T22:33:18.934680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:41.901584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.931302+0000","last_change":"2024-05-08T22:33:14.911744+0000","last_active":"2024-05-08T22:33:18.931302+0000","last_peered":"2024-05-08T22:33:18.931302+0000","last_clean":"2024-05-08T22:33:18.931302+0000","last_became_active":"2024-05-08T22:33:14.910853+0000","last_became_peered":"2024-05-08T22:33:14.910853+0000","last_unstale":"2024-05-08T22:33:18.931302+0000","last_undegraded":"2024-05-08T22:33:18.931302+0000","last_fullsized":"2024-05-08T22:33:18.931302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.928361+0000","last_change":"2024-05-08T22:33:14.914067+0000","last_active":"2024-05-08T22:33:18.928361+0000","last_peered":"2024-05-08T22:33:18.928361+0000","last_clean":"2024-05-08T22:33:18.928361+0000","last_became_active":"2024-05-08T22:33:14.913755+0000","last_became_peered":"2024-05-08T22:33:14.913755+0000","last_unstale":"2024-05-08T22:33:18.928361+0000","last_undegraded":"2024-05-08T22:33:18.928361+0000","last_fullsized":"2024-05-08T22:33:18.928361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999511+0000","last_change":"2024-05-08T22:33:14.922252+0000","last_active":"2024-05-08T22:33:18.999511+0000","last_peered":"2024-05-08T22:33:18.999511+0000","last_clean":"2024-05-08T22:33:18.999511+0000","last_became_active":"2024-05-08T22:33:14.921949+0000","last_became_peered":"2024-05-08T22:33:14.921949+0000","last_unstale":"2024-05-08T22:33:18.999511+0000","last_undegraded":"2024-05-08T22:33:18.999511+0000","last_fullsized":"2024-05-08T22:33:18.999511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:12.263075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.925280+0000","last_change":"2024-05-08T22:33:14.916689+0000","last_active":"2024-05-08T22:33:18.925280+0000","last_peered":"2024-05-08T22:33:18.925280+0000","last_clean":"2024-05-08T22:33:18.925280+0000","last_became_active":"2024-05-08T22:33:14.916200+0000","last_became_peered":"2024-05-08T22:33:14.916200+0000","last_unstale":"2024-05-08T22:33:18.925280+0000","last_undegraded":"2024-05-08T22:33:18.925280+0000","last_fullsized":"2024-05-08T22:33:18.925280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:16.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.936475+0000","last_change":"2024-05-08T22:33:14.925461+0000","last_active":"2024-05-08T22:33:18.936475+0000","last_peered":"2024-05-08T22:33:18.936475+0000","last_clean":"2024-05-08T22:33:18.936475+0000","last_became_active":"2024-05-08T22:33:14.924786+0000","last_became_peered":"2024-05-08T22:33:14.924786+0000","last_unstale":"2024-05-08T22:33:18.936475+0000","last_undegraded":"2024-05-08T22:33:18.936475+0000","last_fullsized":"2024-05-08T22:33:18.936475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.932849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.935606+0000","last_change":"2024-05-08T22:33:14.918139+0000","last_active":"2024-05-08T22:33:18.935606+0000","last_peered":"2024-05-08T22:33:18.935606+0000","last_clean":"2024-05-08T22:33:18.935606+0000","last_became_active":"2024-05-08T22:33:14.914720+0000","last_became_peered":"2024-05-08T22:33:14.914720+0000","last_unstale":"2024-05-08T22:33:18.935606+0000","last_undegraded":"2024-05-08T22:33:18.935606+0000","last_fullsized":"2024-05-08T22:33:18.935606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448502+0000","last_change":"2024-05-08T22:33:14.905986+0000","last_active":"2024-05-08T22:33:19.448502+0000","last_peered":"2024-05-08T22:33:19.448502+0000","last_clean":"2024-05-08T22:33:19.448502+0000","last_became_active":"2024-05-08T22:33:14.905613+0000","last_became_peered":"2024-05-08T22:33:14.905613+0000","last_unstale":"2024-05-08T22:33:19.448502+0000","last_undegraded":"2024-05-08T22:33:19.448502+0000","last_fullsized":"2024-05-08T22:33:19.448502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930058+0000","last_change":"2024-05-08T22:33:14.911270+0000","last_active":"2024-05-08T22:33:18.930058+0000","last_peered":"2024-05-08T22:33:18.930058+0000","last_clean":"2024-05-08T22:33:18.930058+0000","last_became_active":"2024-05-08T22:33:14.910966+0000","last_became_peered":"2024-05-08T22:33:14.910966+0000","last_unstale":"2024-05-08T22:33:18.930058+0000","last_undegraded":"2024-05-08T22:33:18.930058+0000","last_fullsized":"2024-05-08T22:33:18.930058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:40.907481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312110+0000","last_change":"2024-05-08T22:33:14.917778+0000","last_active":"2024-05-08T22:33:19.312110+0000","last_peered":"2024-05-08T22:33:19.312110+0000","last_clean":"2024-05-08T22:33:19.312110+0000","last_became_active":"2024-05-08T22:33:14.917531+0000","last_became_peered":"2024-05-08T22:33:14.917531+0000","last_unstale":"2024-05-08T22:33:19.312110+0000","last_undegraded":"2024-05-08T22:33:19.312110+0000","last_fullsized":"2024-05-08T22:33:19.312110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:00.591011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.926531+0000","last_change":"2024-05-08T22:33:14.916563+0000","last_active":"2024-05-08T22:33:18.926531+0000","last_peered":"2024-05-08T22:33:18.926531+0000","last_clean":"2024-05-08T22:33:18.926531+0000","last_became_active":"2024-05-08T22:33:14.916271+0000","last_became_peered":"2024-05-08T22:33:14.916271+0000","last_unstale":"2024-05-08T22:33:18.926531+0000","last_undegraded":"2024-05-08T22:33:18.926531+0000","last_fullsized":"2024-05-08T22:33:18.926531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:05.009134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.927948+0000","last_change":"2024-05-08T22:33:14.931269+0000","last_active":"2024-05-08T22:33:18.927948+0000","last_peered":"2024-05-08T22:33:18.927948+0000","last_clean":"2024-05-08T22:33:18.927948+0000","last_became_active":"2024-05-08T22:33:14.931026+0000","last_became_peered":"2024-05-08T22:33:14.931026+0000","last_unstale":"2024-05-08T22:33:18.927948+0000","last_undegraded":"2024-05-08T22:33:18.927948+0000","last_fullsized":"2024-05-08T22:33:18.927948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:23.012089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.929869+0000","last_change":"2024-05-08T22:33:14.928697+0000","last_active":"2024-05-08T22:33:18.929869+0000","last_peered":"2024-05-08T22:33:18.929869+0000","last_clean":"2024-05-08T22:33:18.929869+0000","last_became_active":"2024-05-08T22:33:14.928400+0000","last_became_peered":"2024-05-08T22:33:14.928400+0000","last_unstale":"2024-05-08T22:33:18.929869+0000","last_undegraded":"2024-05-08T22:33:18.929869+0000","last_fullsized":"2024-05-08T22:33:18.929869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:39.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999653+0000","last_change":"2024-05-08T22:33:14.931677+0000","last_active":"2024-05-08T22:33:18.999653+0000","last_peered":"2024-05-08T22:33:18.999653+0000","last_clean":"2024-05-08T22:33:18.999653+0000","last_became_active":"2024-05-08T22:33:14.931437+0000","last_became_peered":"2024-05-08T22:33:14.931437+0000","last_unstale":"2024-05-08T22:33:18.999653+0000","last_undegraded":"2024-05-08T22:33:18.999653+0000","last_fullsized":"2024-05-08T22:33:18.999653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:07.861504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.312183+0000","last_change":"2024-05-08T22:33:14.924165+0000","last_active":"2024-05-08T22:33:19.312183+0000","last_peered":"2024-05-08T22:33:19.312183+0000","last_clean":"2024-05-08T22:33:19.312183+0000","last_became_active":"2024-05-08T22:33:14.923910+0000","last_became_peered":"2024-05-08T22:33:14.923910+0000","last_unstale":"2024-05-08T22:33:19.312183+0000","last_undegraded":"2024-05-08T22:33:19.312183+0000","last_fullsized":"2024-05-08T22:33:19.312183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:09.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999755+0000","last_change":"2024-05-08T22:33:14.910595+0000","last_active":"2024-05-08T22:33:18.999755+0000","last_peered":"2024-05-08T22:33:18.999755+0000","last_clean":"2024-05-08T22:33:18.999755+0000","last_became_active":"2024-05-08T22:33:14.910354+0000","last_became_peered":"2024-05-08T22:33:14.910354+0000","last_unstale":"2024-05-08T22:33:18.999755+0000","last_undegraded":"2024-05-08T22:33:18.999755+0000","last_fullsized":"2024-05-08T22:33:18.999755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:29.520048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934575+0000","last_change":"2024-05-08T22:33:14.924832+0000","last_active":"2024-05-08T22:33:18.934575+0000","last_peered":"2024-05-08T22:33:18.934575+0000","last_clean":"2024-05-08T22:33:18.934575+0000","last_became_active":"2024-05-08T22:33:14.924207+0000","last_became_peered":"2024-05-08T22:33:14.924207+0000","last_unstale":"2024-05-08T22:33:18.934575+0000","last_undegraded":"2024-05-08T22:33:18.934575+0000","last_fullsized":"2024-05-08T22:33:18.934575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:40.963211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.998856+0000","last_change":"2024-05-08T22:33:14.922269+0000","last_active":"2024-05-08T22:33:18.998856+0000","last_peered":"2024-05-08T22:33:18.998856+0000","last_clean":"2024-05-08T22:33:18.998856+0000","last_became_active":"2024-05-08T22:33:14.921959+0000","last_became_peered":"2024-05-08T22:33:14.921959+0000","last_unstale":"2024-05-08T22:33:18.998856+0000","last_undegraded":"2024-05-08T22:33:18.998856+0000","last_fullsized":"2024-05-08T22:33:18.998856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:13.185684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.934594+0000","last_change":"2024-05-08T22:33:14.930737+0000","last_active":"2024-05-08T22:33:18.934594+0000","last_peered":"2024-05-08T22:33:18.934594+0000","last_clean":"2024-05-08T22:33:18.934594+0000","last_became_active":"2024-05-08T22:33:14.930465+0000","last_became_peered":"2024-05-08T22:33:14.930465+0000","last_unstale":"2024-05-08T22:33:18.934594+0000","last_undegraded":"2024-05-08T22:33:18.934594+0000","last_fullsized":"2024-05-08T22:33:18.934594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:03.737555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.930293+0000","last_change":"2024-05-08T22:33:14.911063+0000","last_active":"2024-05-08T22:33:18.930293+0000","last_peered":"2024-05-08T22:33:18.930293+0000","last_clean":"2024-05-08T22:33:18.930293+0000","last_became_active":"2024-05-08T22:33:14.910801+0000","last_became_peered":"2024-05-08T22:33:14.910801+0000","last_unstale":"2024-05-08T22:33:18.930293+0000","last_undegraded":"2024-05-08T22:33:18.930293+0000","last_fullsized":"2024-05-08T22:33:18.930293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:50.282359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:19.448710+0000","last_change":"2024-05-08T22:33:14.916640+0000","last_active":"2024-05-08T22:33:19.448710+0000","last_peered":"2024-05-08T22:33:19.448710+0000","last_clean":"2024-05-08T22:33:19.448710+0000","last_became_active":"2024-05-08T22:33:14.916171+0000","last_became_peered":"2024-05-08T22:33:14.916171+0000","last_unstale":"2024-05-08T22:33:19.448710+0000","last_undegraded":"2024-05-08T22:33:19.448710+0000","last_fullsized":"2024-05-08T22:33:19.448710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:47.681503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-08T22:33:18.999564+0000","last_change":"2024-05-08T22:33:14.925286+0000","last_active":"2024-05-08T22:33:18.999564+0000","last_peered":"2024-05-08T22:33:18.999564+0000","last_clean":"2024-05-08T22:33:18.999564+0000","last_became_active":"2024-05-08T22:33:14.924846+0000","last_became_peered":"2024-05-08T22:33:14.924846+0000","last_unstale":"2024-05-08T22:33:18.999564+0000","last_undegraded":"2024-05-08T22:33:18.999564+0000","last_fullsized":"2024-05-08T22:33:18.999564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:14.496539+0000","last_clean_scrub_stamp":"2024-05-08T22:33:14.496539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:15.808109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":868352,"data_stored":697988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":110834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":274432,"data_stored":107128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":282624,"data_stored":107164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":698044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:33:44.000 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T22:33:44.000 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T22:33:44.000 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T22:33:44.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:33:44.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 Processor -- start 2024-05-08T22:33:44.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 -- start start 2024-05-08T22:33:44.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc0ce810 con 0x7ff7dc15d5b0 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc06afc0 con 0x7ff7dc06b450 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e1b82700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc15d990 con 0x7ff7dc0cde70 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35384/0 (socket says 172.21.15.179:35384) 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 -- 172.21.15.179:0/3390909494 learned_addr learned my addr 172.21.15.179:0/3390909494 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7e0b80700 1 --1- 172.21.15.179:0/3390909494 >> v1:172.21.15.179:6789/0 conn(0x7ff7dc15d5b0 0x7ff7dc0b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33612/0 (socket says 172.21.15.179:33612) 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 -- 172.21.15.179:0/3390909494 >> v1:172.21.15.195:6789/0 conn(0x7ff7dc06b450 legacy=0x7ff7dc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 -- 172.21.15.179:0/3390909494 >> v1:172.21.15.179:6789/0 conn(0x7ff7dc15d5b0 legacy=0x7ff7dc0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.107+0000 7ff7dbfff700 1 -- 172.21.15.179:0/3390909494 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc0d26f0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7dbfff700 1 --2- 172.21.15.179:0/3390909494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009910 tx=0x7ff7cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50ad80b01b36ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7db7fe700 1 -- 172.21.15.179:0/3390909494 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc004230 con 0x7ff7dc0cde70 2024-05-08T22:33:44.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7db7fe700 1 -- 172.21.15.179:0/3390909494 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc004390 con 0x7ff7dc0cde70 2024-05-08T22:33:44.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7db7fe700 1 -- 172.21.15.179:0/3390909494 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc009e60 con 0x7ff7dc0cde70 2024-05-08T22:33:44.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 -- 172.21.15.179:0/3390909494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 msgr2=0x7ff7dc0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/3390909494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009910 tx=0x7ff7cc009c20 comp rx=0 tx=0).stop 2024-05-08T22:33:44.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 -- 172.21.15.179:0/3390909494 shutdown_connections 2024-05-08T22:33:44.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/3390909494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0ce2d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 -- 172.21.15.179:0/3390909494 >> 172.21.15.179:0/3390909494 conn(0x7ff7dc0b4940 msgr2=0x7ff7dc0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 -- 172.21.15.179:0/3390909494 shutdown_connections 2024-05-08T22:33:44.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.111+0000 7ff7e1b82700 1 -- 172.21.15.179:0/3390909494 wait complete. 2024-05-08T22:33:44.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 Processor -- start 2024-05-08T22:33:44.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 -- start start 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7dc06b450 0x7ff7dc160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7dc15d5b0 0x7ff7dc155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c6790 con 0x7ff7dc0cde70 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0caf70 con 0x7ff7dc15d5b0 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0cb1c0 con 0x7ff7dc06b450 2024-05-08T22:33:44.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35394/0 (socket says 172.21.15.179:35394) 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 -- 172.21.15.179:0/1510163143 learned_addr learned my addr 172.21.15.179:0/1510163143 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e0b80700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7dc06b450 0x7ff7dc160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1381700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7dc15d5b0 0x7ff7dc155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 -- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7dc06b450 msgr2=0x7ff7dc160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7dc06b450 0x7ff7dc160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 -- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7dc15d5b0 msgr2=0x7ff7dc155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7dc15d5b0 0x7ff7dc155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 -- 172.21.15.179:0/1510163143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009570 con 0x7ff7dc0cde70 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7dbfff700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004000 tx=0x7ff7cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00bca0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc01f070 con 0x7ff7dc0cde70 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc0cb4a0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.115+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0036c0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc0cba60 con 0x7ff7dc0cde70 2024-05-08T22:33:44.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc04e700 con 0x7ff7dc0cde70 2024-05-08T22:33:44.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7cc021030 con 0x7ff7dc0cde70 2024-05-08T22:33:44.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7d97fa700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 0x7ff7c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff7cc0859a0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7e0b80700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 0x7ff7c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.119+0000 7ff7e0b80700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 0x7ff7c8068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000f60 tx=0x7ff7d0007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.127+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff7cc052ab0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.315+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff7dc04e700 con 0x7ff7dc0cde70 2024-05-08T22:33:44.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.315+0000 7ff7d97fa700 1 -- 172.21.15.179:0/1510163143 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff7cc0560d0 con 0x7ff7dc0cde70 2024-05-08T22:33:44.317 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:44.317 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:33:44.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 msgr2=0x7ff7c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 0x7ff7c8068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000f60 tx=0x7ff7d0007f30 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 msgr2=0x7ff7dc0cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004000 tx=0x7ff7cc004770 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 shutdown_connections 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff7c8066730 0x7ff7c8068bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7dc15d5b0 0x7ff7dc155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7dc0cde70 0x7ff7dc0cc260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 --2- 172.21.15.179:0/1510163143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff7dc06b450 0x7ff7dc160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 >> 172.21.15.179:0/1510163143 conn(0x7ff7dc0b4940 msgr2=0x7ff7dc0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 shutdown_connections 2024-05-08T22:33:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.319+0000 7ff7e1b82700 1 -- 172.21.15.179:0/1510163143 wait complete. 2024-05-08T22:33:44.331 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T22:33:44.331 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:44.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 Processor -- start 2024-05-08T22:33:44.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 -- start start 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ef80 0x7f5a50161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5a5006de70 0x7f5a5006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a50165d50 con 0x7f5a50164db0 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5006ad70 con 0x7f5a5006de70 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a56641700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50165230 con 0x7f5a5015ef80 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a54e3e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5a50164db0 0x7f5a5006d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33614/0 (socket says 172.21.15.179:33614) 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a54e3e700 1 -- 172.21.15.179:0/3965309090 learned_addr learned my addr 172.21.15.179:0/3965309090 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 --2- 172.21.15.179:0/3965309090 >> v2:172.21.15.195:3300/0 conn(0x7f5a5006de70 0x7f5a5006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a5563f700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ef80 0x7f5a50161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925998512 0 0) 0x7f5a50165d50 con 0x7f5a50164db0 2024-05-08T22:33:44.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a3c0035d0 con 0x7f5a50164db0 2024-05-08T22:33:44.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 -- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ef80 msgr2=0x7f5a50161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:44.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482157898 0 0) 0x7f5a3c0035d0 con 0x7f5a50164db0 2024-05-08T22:33:44.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ef80 0x7f5a50161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 -- 172.21.15.179:0/3965309090 >> v1:172.21.15.179:6789/0 conn(0x7f5a50164db0 legacy=0x7f5a5006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 -- 172.21.15.179:0/3965309090 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5006e4b0 con 0x7f5a5006de70 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a55e40700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5006de70 0x7f5a5006b200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009d90 tx=0x7f5a4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17dfcd38ecb5d79e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.439+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800e070 con 0x7f5a5006de70 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a48003a40 con 0x7f5a5006de70 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a43fff700 1 -- 172.21.15.179:0/3965309090 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44001670 con 0x7f5a5006de70 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5006de70 msgr2=0x7f5a5006b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5006de70 0x7f5a5006b200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009d90 tx=0x7f5a4800b330 comp rx=0 tx=0).stop 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- 172.21.15.179:0/3965309090 shutdown_connections 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5006de70 0x7f5a5006b200 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- 172.21.15.179:0/3965309090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ef80 0x7f5a50161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- 172.21.15.179:0/3965309090 >> 172.21.15.179:0/3965309090 conn(0x7f5a500ce9d0 msgr2=0x7f5a500cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- 172.21.15.179:0/3965309090 shutdown_connections 2024-05-08T22:33:44.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- 172.21.15.179:0/3965309090 wait complete. 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 Processor -- start 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- start start 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ef80 0x7f5a500cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 0x7f5a5015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5014a1e0 con 0x7f5a5006de70 2024-05-08T22:33:44.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500cc7b0 con 0x7f5a5015ef80 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a56641700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500cca90 con 0x7f5a50164db0 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35402/0 (socket says 172.21.15.179:35402) 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 -- 172.21.15.179:0/2226919750 learned_addr learned my addr 172.21.15.179:0/2226919750 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a54e3e700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ef80 0x7f5a500cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a55e40700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 0x7f5a5015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 -- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 msgr2=0x7f5a5015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 0x7f5a5015fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 -- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ef80 msgr2=0x7f5a500cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ef80 0x7f5a500cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a5563f700 1 -- 172.21.15.179:0/2226919750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48009a70 con 0x7f5a5006de70 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.443+0000 7f5a55e40700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 0x7f5a5015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:44.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a5563f700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00e9d0 tx=0x7f5a4c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c009720 con 0x7f5a5006de70 2024-05-08T22:33:44.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a500ccd70 con 0x7f5a5006de70 2024-05-08T22:33:44.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a500cd330 con 0x7f5a5006de70 2024-05-08T22:33:44.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c00c850 con 0x7f5a5006de70 2024-05-08T22:33:44.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00cbc0 con 0x7f5a5006de70 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30005310 con 0x7f5a5006de70 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5a4c00cd20 con 0x7f5a5006de70 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 0x7f5a34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a4c084560 con 0x7f5a5006de70 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.447+0000 7f5a54e3e700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 0x7f5a34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.451+0000 7f5a54e3e700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 0x7f5a34068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a500aef90 tx=0x7f5a44001e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.455+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5a4c051560 con 0x7f5a5006de70 2024-05-08T22:33:44.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.627+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a30005890 con 0x7f5a5006de70 2024-05-08T22:33:44.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a41ffb700 1 -- 172.21.15.179:0/2226919750 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f5a4c054b80 con 0x7f5a5006de70 2024-05-08T22:33:44.630 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:44.630 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 msgr2=0x7f5a34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 0x7f5a34068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a500aef90 tx=0x7f5a44001e00 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 msgr2=0x7f5a50149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00e9d0 tx=0x7f5a4c00ece0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 shutdown_connections 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5a34066730 0x7f5a34068bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ef80 0x7f5a500cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a50164db0 0x7f5a5015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 --2- 172.21.15.179:0/2226919750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5006de70 0x7f5a50149ca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 >> 172.21.15.179:0/2226919750 conn(0x7f5a500ce9d0 msgr2=0x7f5a500b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 shutdown_connections 2024-05-08T22:33:44.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.631+0000 7f5a56641700 1 -- 172.21.15.179:0/2226919750 wait complete. 2024-05-08T22:33:44.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:33:44.646 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:33:44.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 Processor -- start 2024-05-08T22:33:44.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 -- start start 2024-05-08T22:33:44.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc7180b19e0 0x7fc7180cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc718159700 0x7fc71806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71806cda0 con 0x7fc718159320 2024-05-08T22:33:44.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180cb4c0 con 0x7fc718159700 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71daa7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180b1f10 con 0x7fc7180b19e0 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc718159700 0x7fc71806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71caa5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc718159320 0x7fc7180cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33616/0 (socket says 172.21.15.179:33616) 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71caa5700 1 -- 172.21.15.179:0/1623128346 learned_addr learned my addr 172.21.15.179:0/1623128346 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc717fff700 1 --2- 172.21.15.179:0/1623128346 >> v2:172.21.15.195:3300/0 conn(0x7fc7180b19e0 0x7fc7180cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc7177fe700 1 -- 172.21.15.179:0/1623128346 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161535552 0 0) 0x7fc71806cda0 con 0x7fc718159320 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 -- 172.21.15.179:0/1623128346 >> v2:172.21.15.195:3300/0 conn(0x7fc7180b19e0 msgr2=0x7fc7180cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 --2- 172.21.15.179:0/1623128346 >> v2:172.21.15.195:3300/0 conn(0x7fc7180b19e0 0x7fc7180cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 -- 172.21.15.179:0/1623128346 >> v1:172.21.15.179:6789/0 conn(0x7fc718159320 legacy=0x7fc7180cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 -- 172.21.15.179:0/1623128346 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180b21f0 con 0x7fc718159700 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc71d2a6700 1 --2- 172.21.15.179:0/1623128346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159700 0x7fc71806b440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc71806ef00 tx=0x7fc71000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f710cba203129264 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc7177fe700 1 -- 172.21.15.179:0/1623128346 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000e070 con 0x7fc718159700 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc7177fe700 1 -- 172.21.15.179:0/1623128346 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000bb20 con 0x7fc718159700 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.759+0000 7fc7177fe700 1 -- 172.21.15.179:0/1623128346 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710012430 con 0x7fc718159700 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- 172.21.15.179:0/1623128346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159700 msgr2=0x7fc71806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 --2- 172.21.15.179:0/1623128346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159700 0x7fc71806b440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc71806ef00 tx=0x7fc71000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- 172.21.15.179:0/1623128346 shutdown_connections 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 --2- 172.21.15.179:0/1623128346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159700 0x7fc71806b440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 --2- 172.21.15.179:0/1623128346 >> v2:172.21.15.195:3300/0 conn(0x7fc7180b19e0 0x7fc7180cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- 172.21.15.179:0/1623128346 >> 172.21.15.179:0/1623128346 conn(0x7fc7180cea30 msgr2=0x7fc7180cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- 172.21.15.179:0/1623128346 shutdown_connections 2024-05-08T22:33:44.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- 172.21.15.179:0/1623128346 wait complete. 2024-05-08T22:33:44.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 Processor -- start 2024-05-08T22:33:44.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.763+0000 7fc71daa7700 1 -- start start 2024-05-08T22:33:44.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7180b19e0 0x7fc718145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc718159700 0x7fc71813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181280f0 con 0x7fc718159320 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181466a0 con 0x7fc7180b19e0 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc718127aa0 con 0x7fc718159700 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35418/0 (socket says 172.21.15.179:35418) 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 -- 172.21.15.179:0/664838266 learned_addr learned my addr 172.21.15.179:0/664838266 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71d2a6700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc718159700 0x7fc71813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71caa5700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7180b19e0 0x7fc718145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 -- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc718159700 msgr2=0x7fc71813bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc718159700 0x7fc71813bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 -- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7180b19e0 msgr2=0x7fc718145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7180b19e0 0x7fc718145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 -- 172.21.15.179:0/664838266 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710009a70 con 0x7fc718159320 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc717fff700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc704009fd0 tx=0x7fc70400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040095a0 con 0x7fc718159320 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704013680 con 0x7fc718159320 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc718127d80 con 0x7fc718159320 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704013a10 con 0x7fc718159320 2024-05-08T22:33:44.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718146980 con 0x7fc718159320 2024-05-08T22:33:44.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.767+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71804e700 con 0x7fc718159320 2024-05-08T22:33:44.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.771+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc704013bf0 con 0x7fc718159320 2024-05-08T22:33:44.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.771+0000 7fc7157fa700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 0x7fc6fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:44.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.771+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc7040846d0 con 0x7fc718159320 2024-05-08T22:33:44.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.771+0000 7fc71caa5700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 0x7fc6fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:44.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.771+0000 7fc71caa5700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 0x7fc6fc068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f60 tx=0x7fc70c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:44.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.775+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7040517e0 con 0x7fc718159320 2024-05-08T22:33:44.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.947+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc718127f10 con 0x7fc718159320 2024-05-08T22:33:44.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc7157fa700 1 -- 172.21.15.179:0/664838266 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc704017020 con 0x7fc718159320 2024-05-08T22:33:44.950 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 msgr2=0x7fc6fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 0x7fc6fc068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f60 tx=0x7fc70c007c00 comp rx=0 tx=0).stop 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 msgr2=0x7fc718149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc704009fd0 tx=0x7fc70400ebd0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 shutdown_connections 2024-05-08T22:33:44.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc6fc066730 0x7fc6fc068bf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc718159700 0x7fc71813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc718159320 0x7fc718149ad0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 --2- 172.21.15.179:0/664838266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7180b19e0 0x7fc718145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 >> 172.21.15.179:0/664838266 conn(0x7fc7180cea30 msgr2=0x7fc7180b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 shutdown_connections 2024-05-08T22:33:44.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:44.951+0000 7fc71daa7700 1 -- 172.21.15.179:0/664838266 wait complete. 2024-05-08T22:33:44.964 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:33:45.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.071+0000 7efdeb59e700 1 Processor -- start 2024-05-08T22:33:45.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.071+0000 7efdeb59e700 1 -- start start 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efdec142890 0x7efdec0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0b22d0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0b19e0 con 0x7efdec06d4b0 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec15e640 con 0x7efdec142890 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde9d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efdec0b2ca0 0x7efdec06cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:33626/0 (socket says 172.21.15.179:33626) 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde9d9b700 1 -- 172.21.15.179:0/244412862 learned_addr learned my addr 172.21.15.179:0/244412862 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:45.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdea59c700 1 --2- 172.21.15.179:0/244412862 >> v2:172.21.15.195:3300/0 conn(0x7efdec142890 0x7efdec0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325792145 0 0) 0x7efdec0b22d0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd80035d0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304891738 0 0) 0x7efdd80035d0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdea59c700 1 -- 172.21.15.179:0/244412862 >> v1:172.21.15.195:6789/0 conn(0x7efdec06d4b0 legacy=0x7efdec15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdea59c700 1 -- 172.21.15.179:0/244412862 >> v1:172.21.15.179:6789/0 conn(0x7efdec0b2ca0 legacy=0x7efdec06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdea59c700 1 -- 172.21.15.179:0/244412862 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0b1c80 con 0x7efdec142890 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdea59c700 1 --2- 172.21.15.179:0/244412862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec142890 0x7efdec0b2760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009850 tx=0x7efdd4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce012729a362f24b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc001670 con 0x7efdec142890 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd4004030 con 0x7efdec142890 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efde959a700 1 -- 172.21.15.179:0/244412862 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd40043a0 con 0x7efdec142890 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- 172.21.15.179:0/244412862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec142890 msgr2=0x7efdec0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 --2- 172.21.15.179:0/244412862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec142890 0x7efdec0b2760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009850 tx=0x7efdd4009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- 172.21.15.179:0/244412862 shutdown_connections 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 --2- 172.21.15.179:0/244412862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec142890 0x7efdec0b2760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- 172.21.15.179:0/244412862 >> 172.21.15.179:0/244412862 conn(0x7efdec0cea30 msgr2=0x7efdec0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- 172.21.15.179:0/244412862 shutdown_connections 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.075+0000 7efdeb59e700 1 -- 172.21.15.179:0/244412862 wait complete. 2024-05-08T22:33:45.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 Processor -- start 2024-05-08T22:33:45.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- start start 2024-05-08T22:33:45.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 0x7efdec14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec06be80 0x7efdec1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06c310 con 0x7efdec0b2ca0 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec14f140 con 0x7efdec06d4b0 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec1f5010 con 0x7efdec06be80 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35420/0 (socket says 172.21.15.179:35420) 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 -- 172.21.15.179:0/3831254654 learned_addr learned my addr 172.21.15.179:0/3831254654 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdea59c700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 0x7efdec14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdead9d700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec06be80 0x7efdec1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 -- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec06be80 msgr2=0x7efdec1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec06be80 0x7efdec1f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 -- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 msgr2=0x7efdec14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 0x7efdec14ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 -- 172.21.15.179:0/3831254654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd40094b0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdea59c700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 0x7efdec14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde9d9b700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c540 tx=0x7efddc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc004ca0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec1f52f0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc004e00 con 0x7efdec0b2ca0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc007310 con 0x7efdec0b2ca0 2024-05-08T22:33:45.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.079+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec1f58b0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec06f0a0 con 0x7efdec0b2ca0 2024-05-08T22:33:45.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efddc007470 con 0x7efdec0b2ca0 2024-05-08T22:33:45.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efde2ffd700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efddc084330 con 0x7efdec0b2ca0 2024-05-08T22:33:45.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efdea59c700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.083+0000 7efdea59c700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 0x7efdcc068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efdec06c610 tx=0x7efdd4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.091+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efddc051290 con 0x7efdec0b2ca0 2024-05-08T22:33:45.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.259+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdec04e700 con 0x7efdec0b2ca0 2024-05-08T22:33:45.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.259+0000 7efde2ffd700 1 -- 172.21.15.179:0/3831254654 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7efddc007720 con 0x7efdec0b2ca0 2024-05-08T22:33:45.259 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:45.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 msgr2=0x7efdcc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 0x7efdcc068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efdec06c610 tx=0x7efdd4008030 comp rx=0 tx=0).stop 2024-05-08T22:33:45.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 msgr2=0x7efdec06b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c540 tx=0x7efddc00c900 comp rx=0 tx=0).stop 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 shutdown_connections 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdcc066730 0x7efdcc068bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec06be80 0x7efdec1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0b2ca0 0x7efdec06b940 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 --2- 172.21.15.179:0/3831254654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec06d4b0 0x7efdec14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 >> 172.21.15.179:0/3831254654 conn(0x7efdec0cea30 msgr2=0x7efdec071410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 shutdown_connections 2024-05-08T22:33:45.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.263+0000 7efdeb59e700 1 -- 172.21.15.179:0/3831254654 wait complete. 2024-05-08T22:33:45.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:33:45.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 Processor -- start 2024-05-08T22:33:45.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 -- start start 2024-05-08T22:33:45.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 0x7fd64c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 0x7fd64c166de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c0cf170 con 0x7fd64c0cbd50 2024-05-08T22:33:45.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c0ae750 con 0x7fd64c06e600 2024-05-08T22:33:45.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.395+0000 7fd652821700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0b43e0 con 0x7fd64c0cc670 2024-05-08T22:33:45.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65181f700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd64c0cbd50 0x7fd64c0cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:52962/0 (socket says 172.21.15.179:52962) 2024-05-08T22:33:45.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65101e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 0x7fd64c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65181f700 1 -- 172.21.15.179:0/3277846250 learned_addr learned my addr 172.21.15.179:0/3277846250 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:45.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 --2- 172.21.15.179:0/3277846250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 0x7fd64c166de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 -- 172.21.15.179:0/3277846250 >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 msgr2=0x7fd64c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 --2- 172.21.15.179:0/3277846250 >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 0x7fd64c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 -- 172.21.15.179:0/3277846250 >> v1:172.21.15.195:6789/0 conn(0x7fd64c0cbd50 legacy=0x7fd64c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 -- 172.21.15.179:0/3277846250 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0b46c0 con 0x7fd64c0cc670 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65101e700 1 --2- 172.21.15.179:0/3277846250 >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 0x7fd64c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652020700 1 --2- 172.21.15.179:0/3277846250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 0x7fd64c166de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd644009d90 tx=0x7fd64400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d7ddf1defbc191 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65081d700 1 -- 172.21.15.179:0/3277846250 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400e070 con 0x7fd64c0cc670 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65081d700 1 -- 172.21.15.179:0/3277846250 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64400bb20 con 0x7fd64c0cc670 2024-05-08T22:33:45.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd65081d700 1 -- 172.21.15.179:0/3277846250 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644012430 con 0x7fd64c0cc670 2024-05-08T22:33:45.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 -- 172.21.15.179:0/3277846250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 msgr2=0x7fd64c166de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 --2- 172.21.15.179:0/3277846250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 0x7fd64c166de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd644009d90 tx=0x7fd64400b330 comp rx=0 tx=0).stop 2024-05-08T22:33:45.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 -- 172.21.15.179:0/3277846250 shutdown_connections 2024-05-08T22:33:45.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 --2- 172.21.15.179:0/3277846250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cc670 0x7fd64c166de0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 --2- 172.21.15.179:0/3277846250 >> v2:172.21.15.195:3300/0 conn(0x7fd64c06e600 0x7fd64c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 -- 172.21.15.179:0/3277846250 >> 172.21.15.179:0/3277846250 conn(0x7fd64c098a70 msgr2=0x7fd64c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 -- 172.21.15.179:0/3277846250 shutdown_connections 2024-05-08T22:33:45.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.399+0000 7fd652821700 1 -- 172.21.15.179:0/3277846250 wait complete. 2024-05-08T22:33:45.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 Processor -- start 2024-05-08T22:33:45.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 -- start start 2024-05-08T22:33:45.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cbd50 0x7fd64c0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 0x7fd64c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0c8550 con 0x7fd64c06e600 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0c76e0 con 0x7fd64c0cc670 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.403+0000 7fd652821700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0c7940 con 0x7fd64c0cbd50 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39792/0 (socket says 172.21.15.179:39792) 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 -- 172.21.15.179:0/1751481781 learned_addr learned my addr 172.21.15.179:0/1751481781 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65101e700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cbd50 0x7fd64c0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd652020700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 0x7fd64c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 -- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cbd50 msgr2=0x7fd64c0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cbd50 0x7fd64c0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 -- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 msgr2=0x7fd64c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 0x7fd64c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 -- 172.21.15.179:0/1751481781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0094b0 con 0x7fd64c06e600 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd652020700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 0x7fd64c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd65181f700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd648001eb0 tx=0x7fd64800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648005d90 con 0x7fd64c06e600 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd644009a70 con 0x7fd64c06e600 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480070d0 con 0x7fd64c06e600 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648007440 con 0x7fd64c06e600 2024-05-08T22:33:45.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c0c7f00 con 0x7fd64c06e600 2024-05-08T22:33:45.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.407+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c04e700 con 0x7fd64c06e600 2024-05-08T22:33:45.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.411+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6480075a0 con 0x7fd64c06e600 2024-05-08T22:33:45.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.411+0000 7fd6427fc700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 0x7fd630068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.411+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd648083650 con 0x7fd64c06e600 2024-05-08T22:33:45.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.411+0000 7fd65101e700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 0x7fd630068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.411+0000 7fd65101e700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 0x7fd630068c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0c8bc0 tx=0x7fd63c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.415+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd648050790 con 0x7fd64c06e600 2024-05-08T22:33:45.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.583+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd64c0b0c80 con 0x7fd64c06e600 2024-05-08T22:33:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.583+0000 7fd6427fc700 1 -- 172.21.15.179:0/1751481781 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd648053db0 con 0x7fd64c06e600 2024-05-08T22:33:45.583 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 msgr2=0x7fd630068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 0x7fd630068c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0c8bc0 tx=0x7fd63c009240 comp rx=0 tx=0).stop 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 msgr2=0x7fd64c0c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd648001eb0 tx=0x7fd64800c7b0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 shutdown_connections 2024-05-08T22:33:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd630066760 0x7fd630068c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd64c0cc670 0x7fd64c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd64c0cbd50 0x7fd64c0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 --2- 172.21.15.179:0/1751481781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c06e600 0x7fd64c0c9310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 >> 172.21.15.179:0/1751481781 conn(0x7fd64c098a70 msgr2=0x7fd64c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 shutdown_connections 2024-05-08T22:33:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.587+0000 7fd652821700 1 -- 172.21.15.179:0/1751481781 wait complete. 2024-05-08T22:33:45.599 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:33:45.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 Processor -- start 2024-05-08T22:33:45.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 -- start start 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f380c167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c0b8ee0 con 0x7f380c15d0c0 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c15d4a0 con 0x7f380c0b8a80 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c06abb0 con 0x7f380c0ce2b0 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380affd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f380c15d0c0 0x7f380c0d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47624/0 (socket says 172.21.15.179:47624) 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380affd700 1 -- 172.21.15.179:0/1562839836 learned_addr learned my addr 172.21.15.179:0/1562839836 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.719+0000 7f380b7fe700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f380c167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620360876 0 0) 0x7f380c0b8ee0 con 0x7f380c15d0c0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f40035d0 con 0x7f380c15d0c0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090845156 0 0) 0x7f380c15d4a0 con 0x7f380c0b8a80 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f380c0b8ee0 con 0x7f380c0b8a80 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950649035 0 0) 0x7f37f40035d0 con 0x7f380c15d0c0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 msgr2=0x7f380c167ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f380c167ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 >> v1:172.21.15.195:6789/0 conn(0x7f380c0b8a80 legacy=0x7f380c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0d2500 con 0x7f380c15d0c0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f37fc001670 con 0x7f380c15d0c0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 >> v1:172.21.15.179:6789/0 conn(0x7f380c15d0c0 legacy=0x7f380c0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37f4003be0 0x7f37f4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f4004870 0x7f37f4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f37f4005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c06abb0 con 0x7f37f4003be0 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400bbf0 con 0x7f37f4004870 2024-05-08T22:33:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400bdb0 con 0x7f380c0ce2b0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37f4003be0 0x7f37f4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380a7fc700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f4004870 0x7f37f4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380b7fe700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f37f4005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 -- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 msgr2=0x7f37f4005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f37f4005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 -- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f4004870 msgr2=0x7f37f4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f4004870 0x7f37f4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 -- 172.21.15.179:0/1562839836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40035d0 con 0x7f37f4003be0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380b7fe700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f380c0ce2b0 0x7f37f4005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380affd700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37f4003be0 0x7f37f4003fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c190 tx=0x7f37f40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a2fcb2dc6f1c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ccd0 con 0x7f37f4003be0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc00ce50 con 0x7f37f4003be0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f3809ffb700 1 -- 172.21.15.179:0/1562839836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc005bc0 con 0x7f37f4003be0 2024-05-08T22:33:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 -- 172.21.15.179:0/1562839836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37f4003be0 msgr2=0x7f37f4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 --2- 172.21.15.179:0/1562839836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37f4003be0 0x7f37f4003fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c190 tx=0x7f37f40058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380b7fe700 1 -- 172.21.15.179:0/1562839836 reap_dead start 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 -- 172.21.15.179:0/1562839836 shutdown_connections 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 -- 172.21.15.179:0/1562839836 >> 172.21.15.179:0/1562839836 conn(0x7f380c0d2ed0 msgr2=0x7f380c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 -- 172.21.15.179:0/1562839836 shutdown_connections 2024-05-08T22:33:45.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.723+0000 7f380bfff700 1 -- 172.21.15.179:0/1562839836 wait complete. 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 Processor -- start 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- start start 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4003be0 0x7f380c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f380c0b6e10 0x7f380c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0b77e0 con 0x7f380c0cbd20 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0caf40 con 0x7f380c0b6e10 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0ca6a0 con 0x7f37f4003be0 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39814/0 (socket says 172.21.15.179:39814) 2024-05-08T22:33:45.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 -- 172.21.15.179:0/1091682656 learned_addr learned my addr 172.21.15.179:0/1091682656 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380b7fe700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f380c0b6e10 0x7f380c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380affd700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4003be0 0x7f380c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 -- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4003be0 msgr2=0x7f380c0c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4003be0 0x7f380c0c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 -- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f380c0b6e10 msgr2=0x7f380c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f380c0b6e10 0x7f380c0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 -- 172.21.15.179:0/1091682656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0ca980 con 0x7f380c0cbd20 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380a7fc700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f380000e7b0 tx=0x7f380000eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000052e0 con 0x7f380c0cbd20 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800005440 con 0x7f380c0cbd20 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c0cac60 con 0x7f380c0cbd20 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000038e0 con 0x7f380c0cbd20 2024-05-08T22:33:45.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.727+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c0cc180 con 0x7f380c0cbd20 2024-05-08T22:33:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c15da10 con 0x7f380c0cbd20 2024-05-08T22:33:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3800003a40 con 0x7f380c0cbd20 2024-05-08T22:33:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f37f3fff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 0x7f37f8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3800083be0 con 0x7f380c0cbd20 2024-05-08T22:33:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f380affd700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 0x7f37f8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:45.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.731+0000 7f380affd700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 0x7f37f8068c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37fc015040 tx=0x7f37fc007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:45.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.739+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f380c15da10 con 0x7f380c0cbd20 2024-05-08T22:33:45.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.907+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f380c15da10 con 0x7f380c0cbd20 2024-05-08T22:33:45.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.907+0000 7f37f3fff700 1 -- 172.21.15.179:0/1091682656 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f380c15da10 con 0x7f380c0cbd20 2024-05-08T22:33:45.908 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 msgr2=0x7f37f8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 0x7f37f8068c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37fc015040 tx=0x7f37fc007000 comp rx=0 tx=0).stop 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 msgr2=0x7f380c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f380000e7b0 tx=0x7f380000eac0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 shutdown_connections 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37f8066760 0x7f37f8068c20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f380c0b6e10 0x7f380c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f380c0cbd20 0x7f380c1582c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 --2- 172.21.15.179:0/1091682656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4003be0 0x7f380c0c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:45.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 >> 172.21.15.179:0/1091682656 conn(0x7f380c0d2ed0 msgr2=0x7f380c0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:45.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 shutdown_connections 2024-05-08T22:33:45.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:45.911+0000 7f380bfff700 1 -- 172.21.15.179:0/1091682656 wait complete. 2024-05-08T22:33:45.924 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:46.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 Processor -- start 2024-05-08T22:33:46.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- start start 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25b40ce060 0x7f25b406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25b40add40 0x7f25b406b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4161ac0 con 0x7f25b40cc310 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40ae550 con 0x7f25b40ce060 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b406abb0 con 0x7f25b40add40 2024-05-08T22:33:46.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25b40ce060 0x7f25b406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25b40ce060 0x7f25b406ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39818/0 (socket says 172.21.15.179:39818) 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 -- 172.21.15.179:0/1190872511 learned_addr learned my addr 172.21.15.179:0/1190872511 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b3fff700 1 --2- 172.21.15.179:0/1190872511 >> v2:172.21.15.195:3300/0 conn(0x7f25b40add40 0x7f25b406b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 -- 172.21.15.179:0/1190872511 >> v2:172.21.15.195:3300/0 conn(0x7f25b40add40 msgr2=0x7f25b406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 --2- 172.21.15.179:0/1190872511 >> v2:172.21.15.195:3300/0 conn(0x7f25b40add40 0x7f25b406b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 -- 172.21.15.179:0/1190872511 >> v1:172.21.15.195:6789/0 conn(0x7f25b40cc310 legacy=0x7f25b40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 -- 172.21.15.179:0/1190872511 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40cdaa0 con 0x7f25b40ce060 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b37fe700 1 --2- 172.21.15.179:0/1190872511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ce060 0x7f25b406ce50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25b4067cc0 tx=0x7f25a4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd075d3f08db9402 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b27fc700 1 -- 172.21.15.179:0/1190872511 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400b8a0 con 0x7f25b40ce060 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b27fc700 1 -- 172.21.15.179:0/1190872511 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400ba00 con 0x7f25b40ce060 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b27fc700 1 -- 172.21.15.179:0/1190872511 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400bd90 con 0x7f25b40ce060 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- 172.21.15.179:0/1190872511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ce060 msgr2=0x7f25b406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 --2- 172.21.15.179:0/1190872511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ce060 0x7f25b406ce50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25b4067cc0 tx=0x7f25a4009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- 172.21.15.179:0/1190872511 shutdown_connections 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 --2- 172.21.15.179:0/1190872511 >> v2:172.21.15.195:3300/0 conn(0x7f25b40add40 0x7f25b406b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 --2- 172.21.15.179:0/1190872511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ce060 0x7f25b406ce50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- 172.21.15.179:0/1190872511 >> 172.21.15.179:0/1190872511 conn(0x7f25b40cb660 msgr2=0x7f25b40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- 172.21.15.179:0/1190872511 shutdown_connections 2024-05-08T22:33:46.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.039+0000 7f25b9c08700 1 -- 172.21.15.179:0/1190872511 wait complete. 2024-05-08T22:33:46.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 Processor -- start 2024-05-08T22:33:46.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 -- start start 2024-05-08T22:33:46.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25b40add40 0x7f25b40b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25b40ce060 0x7f25b40c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415ff00 con 0x7f25b40cc310 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b31b0 con 0x7f25b40ce060 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b9c08700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b28f0 con 0x7f25b40add40 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39826/0 (socket says 172.21.15.179:39826) 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 -- 172.21.15.179:0/3986057376 learned_addr learned my addr 172.21.15.179:0/3986057376 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b37fe700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25b40add40 0x7f25b40b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b3fff700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25b40ce060 0x7f25b40c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 -- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25b40add40 msgr2=0x7f25b40b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25b40add40 0x7f25b40b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 -- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25b40ce060 msgr2=0x7f25b40c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25b40ce060 0x7f25b40c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 -- 172.21.15.179:0/3986057376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4009570 con 0x7f25b40cc310 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.043+0000 7f25b2ffd700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00cc50 tx=0x7f25ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac003c20 con 0x7f25b40cc310 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac003d80 con 0x7f25b40cc310 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b40b2b60 con 0x7f25b40cc310 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00aad0 con 0x7f25b40cc310 2024-05-08T22:33:46.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40b3b80 con 0x7f25b40cc310 2024-05-08T22:33:46.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4067770 con 0x7f25b40cc310 2024-05-08T22:33:46.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25ac00ac30 con 0x7f25b40cc310 2024-05-08T22:33:46.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 0x7f2594068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25ac084420 con 0x7f25b40cc310 2024-05-08T22:33:46.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.047+0000 7f25b37fe700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 0x7f2594068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.051+0000 7f25b37fe700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 0x7f2594068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000c00 tx=0x7f25a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.055+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f25ac0514a0 con 0x7f25b40cc310 2024-05-08T22:33:46.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.259+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b40b3360 con 0x7f25b40cc310 2024-05-08T22:33:46.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.259+0000 7f25abfff700 1 -- 172.21.15.179:0/3986057376 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f25ac054ac0 con 0x7f25b40cc310 2024-05-08T22:33:46.261 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:46.261 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:33:46.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 msgr2=0x7f2594068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 0x7f2594068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000c00 tx=0x7f25a4008030 comp rx=0 tx=0).stop 2024-05-08T22:33:46.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 msgr2=0x7f25b40b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00cc50 tx=0x7f25ac00cf60 comp rx=0 tx=0).stop 2024-05-08T22:33:46.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 shutdown_connections 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2594066730 0x7f2594068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25b40ce060 0x7f25b40c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40cc310 0x7f25b40b3640 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 --2- 172.21.15.179:0/3986057376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25b40add40 0x7f25b40b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 >> 172.21.15.179:0/3986057376 conn(0x7f25b40cb660 msgr2=0x7f25b406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 shutdown_connections 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.263+0000 7f25b9c08700 1 -- 172.21.15.179:0/3986057376 wait complete. 2024-05-08T22:33:46.264 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:33:46.275 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:46.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.387+0000 7f2225b2b700 1 Processor -- start 2024-05-08T22:33:46.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.387+0000 7f2225b2b700 1 -- start start 2024-05-08T22:33:46.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f22180e0100 0x7f2218053280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 0x7f22180537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2218006db0 0x7f2218007240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221803a3f0 con 0x7f22180e0100 2024-05-08T22:33:46.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2218051770 con 0x7f22180e04e0 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2218051a50 con 0x7f2218006db0 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f22180e0100 0x7f2218053280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f22180e0100 0x7f2218053280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39834/0 (socket says 172.21.15.179:39834) 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 -- 172.21.15.179:0/3363343771 learned_addr learned my addr 172.21.15.179:0/3363343771 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f221ffff700 1 --2- 172.21.15.179:0/3363343771 >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 0x7f22180537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f222532a700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2218006db0 0x7f2218007240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 -- 172.21.15.179:0/3363343771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2218006db0 msgr2=0x7f2218007240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2218006db0 0x7f2218007240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 -- 172.21.15.179:0/3363343771 >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 msgr2=0x7f22180537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 --2- 172.21.15.179:0/3363343771 >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 0x7f22180537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 -- 172.21.15.179:0/3363343771 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180062d0 con 0x7f22180e0100 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f221ffff700 1 --2- 172.21.15.179:0/3363343771 >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 0x7f22180537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2224b29700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22180e0100 0x7f2218053280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2218036f60 tx=0x7f2214009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8faa12e591a0b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f221f7fe700 1 -- 172.21.15.179:0/3363343771 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221400b8a0 con 0x7f22180e0100 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f221f7fe700 1 -- 172.21.15.179:0/3363343771 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221400ba00 con 0x7f22180e0100 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f221f7fe700 1 -- 172.21.15.179:0/3363343771 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221400bd90 con 0x7f22180e0100 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- 172.21.15.179:0/3363343771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22180e0100 msgr2=0x7f2218053280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22180e0100 0x7f2218053280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2218036f60 tx=0x7f2214009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- 172.21.15.179:0/3363343771 shutdown_connections 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2218006db0 0x7f2218007240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- 172.21.15.179:0/3363343771 >> v2:172.21.15.195:3300/0 conn(0x7f22180e04e0 0x7f22180537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 --2- 172.21.15.179:0/3363343771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22180e0100 0x7f2218053280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- 172.21.15.179:0/3363343771 >> 172.21.15.179:0/3363343771 conn(0x7f221801e770 msgr2=0x7f22180554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.391+0000 7f2225b2b700 1 -- 172.21.15.179:0/3363343771 shutdown_connections 2024-05-08T22:33:46.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 -- 172.21.15.179:0/3363343771 wait complete. 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 Processor -- start 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 -- start start 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22180e0100 0x7f2218032060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 0x7f221801fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180c66e0 con 0x7f2218006db0 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2218176e90 con 0x7f22180e04e0 2024-05-08T22:33:46.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2225b2b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180325a0 con 0x7f22180e0100 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2224b29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2224b29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39848/0 (socket says 172.21.15.179:39848) 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f2224b29700 1 -- 172.21.15.179:0/489416694 learned_addr learned my addr 172.21.15.179:0/489416694 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f221ffff700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22180e0100 0x7f2218032060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.395+0000 7f222532a700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 0x7f221801fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 -- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22180e0100 msgr2=0x7f2218032060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22180e0100 0x7f2218032060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 -- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 msgr2=0x7f221801fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 0x7f221801fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 -- 172.21.15.179:0/489416694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214009570 con 0x7f2218006db0 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f222532a700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 0x7f221801fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2224b29700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221400b0e0 tx=0x7f2214004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140036c0 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221401f070 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22180200c0 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214003a80 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22180c6020 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2214021030 con 0x7f2218006db0 2024-05-08T22:33:46.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2218005890 con 0x7f2218006db0 2024-05-08T22:33:46.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 0x7f2204068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.399+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2214085a00 con 0x7f2218006db0 2024-05-08T22:33:46.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.403+0000 7f221ffff700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 0x7f2204068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.403+0000 7f221ffff700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 0x7f2204068c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f221817a470 tx=0x7f220c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.407+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2214052a70 con 0x7f2218006db0 2024-05-08T22:33:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.587+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f221804b490 con 0x7f2218006db0 2024-05-08T22:33:46.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.587+0000 7f221d7fa700 1 -- 172.21.15.179:0/489416694 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f2214056090 con 0x7f2218006db0 2024-05-08T22:33:46.589 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:46.589 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 msgr2=0x7f2204068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 0x7f2204068c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f221817a470 tx=0x7f220c008040 comp rx=0 tx=0).stop 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 msgr2=0x7f2218176950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221400b0e0 tx=0x7f2214004770 comp rx=0 tx=0).stop 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 shutdown_connections 2024-05-08T22:33:46.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2204066760 0x7f2204068c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22180e04e0 0x7f221801fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22180e0100 0x7f2218032060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 --2- 172.21.15.179:0/489416694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2218006db0 0x7f2218176950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 >> 172.21.15.179:0/489416694 conn(0x7f221801e770 msgr2=0x7f2218049830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 shutdown_connections 2024-05-08T22:33:46.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.591+0000 7f2225b2b700 1 -- 172.21.15.179:0/489416694 wait complete. 2024-05-08T22:33:46.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:33:46.604 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:33:46.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 Processor -- start 2024-05-08T22:33:46.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 -- start start 2024-05-08T22:33:46.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a001606b0 0x7f0a000b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0a00160d20 0x7f0a00161180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0006c560 con 0x7f0a0006b5c0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a000b6600 con 0x7f0a00160d20 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06a44700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000d1a70 con 0x7f0a001606b0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06243700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0a0006b5c0 0x7f0a0006ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47628/0 (socket says 172.21.15.179:47628) 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a06243700 1 -- 172.21.15.179:0/3500438887 learned_addr learned my addr 172.21.15.179:0/3500438887 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a05241700 1 --2- 172.21.15.179:0/3500438887 >> v2:172.21.15.195:3300/0 conn(0x7f0a00160d20 0x7f0a00161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a05a42700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a001606b0 0x7f0a000b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a04a40700 1 -- 172.21.15.179:0/3500438887 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559341661 0 0) 0x7f0a0006c560 con 0x7f0a0006b5c0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.727+0000 7f0a04a40700 1 -- 172.21.15.179:0/3500438887 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ec0035d0 con 0x7f0a0006b5c0 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a05241700 1 -- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a001606b0 msgr2=0x7f0a000b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a05241700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a001606b0 0x7f0a000b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a05241700 1 -- 172.21.15.179:0/3500438887 >> v1:172.21.15.179:6789/0 conn(0x7f0a0006b5c0 legacy=0x7f0a0006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a05241700 1 -- 172.21.15.179:0/3500438887 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a001616c0 con 0x7f0a00160d20 2024-05-08T22:33:46.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a05241700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a00160d20 0x7f0a00161180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009850 tx=0x7f09fc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2a2348703c6f809 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a04a40700 1 -- 172.21.15.179:0/3500438887 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc011830 con 0x7f0a00160d20 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a04a40700 1 -- 172.21.15.179:0/3500438887 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc011990 con 0x7f0a00160d20 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a04a40700 1 -- 172.21.15.179:0/3500438887 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc011d40 con 0x7f0a00160d20 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a00160d20 msgr2=0x7f0a00161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a00160d20 0x7f0a00161180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009850 tx=0x7f09fc009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- 172.21.15.179:0/3500438887 shutdown_connections 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a00160d20 0x7f0a00161180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 --2- 172.21.15.179:0/3500438887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a001606b0 0x7f0a000b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- 172.21.15.179:0/3500438887 >> 172.21.15.179:0/3500438887 conn(0x7f0a000b49a0 msgr2=0x7f0a000b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- 172.21.15.179:0/3500438887 shutdown_connections 2024-05-08T22:33:46.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- 172.21.15.179:0/3500438887 wait complete. 2024-05-08T22:33:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 Processor -- start 2024-05-08T22:33:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.731+0000 7f0a06a44700 1 -- start start 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0006b5c0 0x7f0a001581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 0x7f0a000c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 0x7f0a00140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00158730 con 0x7f0a00160d20 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00157150 con 0x7f0a001606b0 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00157430 con 0x7f0a0006b5c0 2024-05-08T22:33:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06243700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 0x7f0a00140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 0x7f0a000c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06243700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 0x7f0a00140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39860/0 (socket says 172.21.15.179:39860) 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06243700 1 -- 172.21.15.179:0/1496106292 learned_addr learned my addr 172.21.15.179:0/1496106292 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05a42700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0006b5c0 0x7f0a001581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 -- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0006b5c0 msgr2=0x7f0a001581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0006b5c0 0x7f0a001581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 -- 172.21.15.179:0/1496106292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 msgr2=0x7f0a00140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 0x7f0a00140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 -- 172.21.15.179:0/1496106292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0094b0 con 0x7f0a001606b0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a05241700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 0x7f0a000c87d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09fc016000 tx=0x7f09fc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004590 con 0x7f0a001606b0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc0036c0 con 0x7f0a001606b0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc003a70 con 0x7f0a001606b0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a00157710 con 0x7f0a001606b0 2024-05-08T22:33:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.735+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a00157cd0 con 0x7f0a001606b0 2024-05-08T22:33:46.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a000d1be0 con 0x7f0a001606b0 2024-05-08T22:33:46.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09fc003bd0 con 0x7f0a001606b0 2024-05-08T22:33:46.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f09f67fc700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 0x7f09e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f09fc021030 con 0x7f0a001606b0 2024-05-08T22:33:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f0a05a42700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 0x7f09e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.739+0000 7f0a05a42700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 0x7f09e4068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009bf0 tx=0x7f09f0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:46.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.751+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f09fc0529d0 con 0x7f0a001606b0 2024-05-08T22:33:46.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.923+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a0004e700 con 0x7f0a001606b0 2024-05-08T22:33:46.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.927+0000 7f09f67fc700 1 -- 172.21.15.179:0/1496106292 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09fc055ff0 con 0x7f0a001606b0 2024-05-08T22:33:46.926 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:33:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 msgr2=0x7f09e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 0x7f09e4068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009bf0 tx=0x7f09f0009240 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 msgr2=0x7f0a000c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 0x7f0a000c87d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09fc016000 tx=0x7f09fc004770 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 shutdown_connections 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e4066730 0x7f09e4068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a001606b0 0x7f0a000c87d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00160d20 0x7f0a00140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 --2- 172.21.15.179:0/1496106292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0006b5c0 0x7f0a001581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 >> 172.21.15.179:0/1496106292 conn(0x7f0a000b49a0 msgr2=0x7f0a000b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 shutdown_connections 2024-05-08T22:33:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:46.931+0000 7f0a06a44700 1 -- 172.21.15.179:0/1496106292 wait complete. 2024-05-08T22:33:46.946 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:33:47.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.067+0000 7fabaf59e700 1 Processor -- start 2024-05-08T22:33:47.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- start start 2024-05-08T22:33:47.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faba804f7f0 0x7faba80df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 0x7faba8006220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba8054430 con 0x7faba804fbd0 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8053820 con 0x7faba804f7f0 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8053b00 con 0x7faba8053340 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faba804f7f0 0x7faba80df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faba804f7f0 0x7faba80df460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39864/0 (socket says 172.21.15.179:39864) 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 -- 172.21.15.179:0/3868314027 learned_addr learned my addr 172.21.15.179:0/3868314027 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaed9d700 1 --2- 172.21.15.179:0/3868314027 >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 0x7faba8006220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 -- 172.21.15.179:0/3868314027 >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 msgr2=0x7faba8006220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 --2- 172.21.15.179:0/3868314027 >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 0x7faba8006220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 -- 172.21.15.179:0/3868314027 >> v1:172.21.15.195:6789/0 conn(0x7faba804fbd0 legacy=0x7faba804ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 -- 172.21.15.179:0/3868314027 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8053de0 con 0x7faba804f7f0 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaed9d700 1 --2- 172.21.15.179:0/3868314027 >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 0x7faba8006220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:47.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabadd9b700 1 --2- 172.21.15.179:0/3868314027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba804f7f0 0x7faba80df460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faba8055840 tx=0x7faba0007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d64919b75b30a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabad59a700 1 -- 172.21.15.179:0/3868314027 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000f070 con 0x7faba804f7f0 2024-05-08T22:33:47.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabad59a700 1 -- 172.21.15.179:0/3868314027 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0008bf0 con 0x7faba804f7f0 2024-05-08T22:33:47.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabad59a700 1 -- 172.21.15.179:0/3868314027 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000a0d0 con 0x7faba804f7f0 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- 172.21.15.179:0/3868314027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba804f7f0 msgr2=0x7faba80df460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 --2- 172.21.15.179:0/3868314027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba804f7f0 0x7faba80df460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faba8055840 tx=0x7faba0007640 comp rx=0 tx=0).stop 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- 172.21.15.179:0/3868314027 shutdown_connections 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 --2- 172.21.15.179:0/3868314027 >> v2:172.21.15.195:3300/0 conn(0x7faba8053340 0x7faba8006220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 --2- 172.21.15.179:0/3868314027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba804f7f0 0x7faba80df460 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- 172.21.15.179:0/3868314027 >> 172.21.15.179:0/3868314027 conn(0x7faba8054b70 msgr2=0x7faba8054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- 172.21.15.179:0/3868314027 shutdown_connections 2024-05-08T22:33:47.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.071+0000 7fabaf59e700 1 -- 172.21.15.179:0/3868314027 wait complete. 2024-05-08T22:33:47.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 Processor -- start 2024-05-08T22:33:47.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 -- start start 2024-05-08T22:33:47.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba804f7f0 0x7faba80d9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba804fbd0 0x7faba8039020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba8038c00 con 0x7faba8053340 2024-05-08T22:33:47.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba804b4c0 con 0x7faba804f7f0 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaf59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80e2340 con 0x7faba804fbd0 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39876/0 (socket says 172.21.15.179:39876) 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabae59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba804f7f0 0x7faba80d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabadd9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba804fbd0 0x7faba8039020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 -- 172.21.15.179:0/1456120378 learned_addr learned my addr 172.21.15.179:0/1456120378 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 -- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba804fbd0 msgr2=0x7faba8039020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba804fbd0 0x7faba8039020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 -- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba804f7f0 msgr2=0x7faba80d9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba804f7f0 0x7faba80d9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.075+0000 7fabaed9d700 1 -- 172.21.15.179:0/1456120378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0007040 con 0x7faba8053340 2024-05-08T22:33:47.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fabaed9d700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabb0067900 tx=0x7fabb004e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb008e070 con 0x7faba8053340 2024-05-08T22:33:47.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb0067c30 con 0x7faba8053340 2024-05-08T22:33:47.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80e2620 con 0x7faba8053340 2024-05-08T22:33:47.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0066c60 con 0x7faba8053340 2024-05-08T22:33:47.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba8038540 con 0x7faba8053340 2024-05-08T22:33:47.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba8038d40 con 0x7faba8053340 2024-05-08T22:33:47.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabb0066dc0 con 0x7faba8053340 2024-05-08T22:33:47.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.079+0000 7fab9effd700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 0x7fab90068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.083+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabb00fd8f0 con 0x7faba8053340 2024-05-08T22:33:47.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.083+0000 7fabae59c700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 0x7fab90068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.083+0000 7fabae59c700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 0x7fab90068c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faba4000f60 tx=0x7faba4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.087+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faba8038d40 con 0x7faba8053340 2024-05-08T22:33:47.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.255+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faba8038d40 con 0x7faba8053340 2024-05-08T22:33:47.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.255+0000 7fab9effd700 1 -- 172.21.15.179:0/1456120378 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7faba8038d40 con 0x7faba8053340 2024-05-08T22:33:47.257 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:47.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 msgr2=0x7fab90068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 0x7fab90068c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faba4000f60 tx=0x7faba4009040 comp rx=0 tx=0).stop 2024-05-08T22:33:47.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 msgr2=0x7faba80dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabb0067900 tx=0x7fabb004e930 comp rx=0 tx=0).stop 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 shutdown_connections 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fab90066780 0x7fab90068c40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8053340 0x7faba80dc490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba804fbd0 0x7faba8039020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 --2- 172.21.15.179:0/1456120378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba804f7f0 0x7faba80d9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 >> 172.21.15.179:0/1456120378 conn(0x7faba8054b70 msgr2=0x7faba8007080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 shutdown_connections 2024-05-08T22:33:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.259+0000 7fabaf59e700 1 -- 172.21.15.179:0/1456120378 wait complete. 2024-05-08T22:33:47.272 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:33:47.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.387+0000 7f640e9ce700 1 Processor -- start 2024-05-08T22:33:47.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.387+0000 7f640e9ce700 1 -- start start 2024-05-08T22:33:47.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6408142890 0x7f64080b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080b22d0 con 0x7f640806d4b0 2024-05-08T22:33:47.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080b19e0 con 0x7f64080b2ca0 2024-05-08T22:33:47.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640815e640 con 0x7f6408142890 2024-05-08T22:33:47.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e1cd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f640806d4b0 0x7f640815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47642/0 (socket says 172.21.15.179:47642) 2024-05-08T22:33:47.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e1cd700 1 -- 172.21.15.179:0/3413130350 learned_addr learned my addr 172.21.15.179:0/3413130350 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:47.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3413130350 >> v2:172.21.15.195:3300/0 conn(0x7f6408142890 0x7f64080b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640c9ca700 1 -- 172.21.15.179:0/3413130350 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658648828 0 0) 0x7f64080b22d0 con 0x7f640806d4b0 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640c9ca700 1 -- 172.21.15.179:0/3413130350 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f00035d0 con 0x7f640806d4b0 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640d9cc700 1 -- 172.21.15.179:0/3413130350 >> v1:172.21.15.195:6789/0 conn(0x7f64080b2ca0 legacy=0x7f640806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640d9cc700 1 -- 172.21.15.179:0/3413130350 >> v1:172.21.15.179:6789/0 conn(0x7f640806d4b0 legacy=0x7f640815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640d9cc700 1 -- 172.21.15.179:0/3413130350 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080b1c80 con 0x7f6408142890 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3413130350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080b2760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009850 tx=0x7f63f8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a24e9839a2a020cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640c9ca700 1 -- 172.21.15.179:0/3413130350 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8011830 con 0x7f6408142890 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640c9ca700 1 -- 172.21.15.179:0/3413130350 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8011990 con 0x7f6408142890 2024-05-08T22:33:47.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640c9ca700 1 -- 172.21.15.179:0/3413130350 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8011d00 con 0x7f6408142890 2024-05-08T22:33:47.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- 172.21.15.179:0/3413130350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 msgr2=0x7f64080b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3413130350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080b2760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009850 tx=0x7f63f8009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:47.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- 172.21.15.179:0/3413130350 shutdown_connections 2024-05-08T22:33:47.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3413130350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080b2760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- 172.21.15.179:0/3413130350 >> 172.21.15.179:0/3413130350 conn(0x7f64080cea30 msgr2=0x7f64080cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- 172.21.15.179:0/3413130350 shutdown_connections 2024-05-08T22:33:47.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.391+0000 7f640e9ce700 1 -- 172.21.15.179:0/3413130350 wait complete. 2024-05-08T22:33:47.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 Processor -- start 2024-05-08T22:33:47.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 -- start start 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 0x7f64081f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080ad7b0 con 0x7f64080b2ca0 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081f3990 con 0x7f6408142890 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e9ce700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080acba0 con 0x7f640806d4b0 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39880/0 (socket says 172.21.15.179:39880) 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 -- 172.21.15.179:0/3255927249 learned_addr learned my addr 172.21.15.179:0/3255927249 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:47.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 0x7f64081f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640e1cd700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 -- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 msgr2=0x7f64081f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 0x7f64081f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 -- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 msgr2=0x7f64080abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d1cb700 1 -- 172.21.15.179:0/3255927249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80094b0 con 0x7f64080b2ca0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.395+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 0x7f64081f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f640d1cb700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f640400c500 tx=0x7f640400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64040052d0 con 0x7f64080b2ca0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6404005430 con 0x7f64080b2ca0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64040055a0 con 0x7f64080b2ca0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080ace80 con 0x7f64080b2ca0 2024-05-08T22:33:47.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080ad440 con 0x7f64080b2ca0 2024-05-08T22:33:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64040036a0 con 0x7f64080b2ca0 2024-05-08T22:33:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.399+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080ad960 con 0x7f64080b2ca0 2024-05-08T22:33:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.403+0000 7f63fe7fc700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 0x7f63f4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.403+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6404083690 con 0x7f64080b2ca0 2024-05-08T22:33:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.403+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 0x7f63f4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.403+0000 7f640d9cc700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 0x7f63f4068c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63f80090b0 tx=0x7f63f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.411+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64080ad960 con 0x7f64080b2ca0 2024-05-08T22:33:47.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64080ad960 con 0x7f64080b2ca0 2024-05-08T22:33:47.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f63fe7fc700 1 -- 172.21.15.179:0/3255927249 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64080ad960 con 0x7f64080b2ca0 2024-05-08T22:33:47.578 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:33:47.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 msgr2=0x7f63f4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 0x7f63f4068c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63f80090b0 tx=0x7f63f8008030 comp rx=0 tx=0).stop 2024-05-08T22:33:47.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 msgr2=0x7f64080adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f640400c500 tx=0x7f640400c810 comp rx=0 tx=0).stop 2024-05-08T22:33:47.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 shutdown_connections 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f63f4066760 0x7f63f4068c20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6408142890 0x7f64080abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64080b2ca0 0x7f64080adc40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 --2- 172.21.15.179:0/3255927249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f640806d4b0 0x7f64081f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 >> 172.21.15.179:0/3255927249 conn(0x7f64080cea30 msgr2=0x7f640806ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 shutdown_connections 2024-05-08T22:33:47.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.579+0000 7f640e9ce700 1 -- 172.21.15.179:0/3255927249 wait complete. 2024-05-08T22:33:47.597 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:33:47.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.711+0000 7f981e437700 1 Processor -- start 2024-05-08T22:33:47.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.711+0000 7f981e437700 1 -- start start 2024-05-08T22:33:47.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.711+0000 7f981e437700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9818165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9818166090 con 0x7f98180b2de0 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981806dd50 con 0x7f98180b1a50 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981806e9d0 con 0x7f981814ae30 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981dc36700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98180b2de0 0x7f981806abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47650/0 (socket says 172.21.15.179:47650) 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981dc36700 1 -- 172.21.15.179:0/2969101517 learned_addr learned my addr 172.21.15.179:0/2969101517 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9818165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530493889 0 0) 0x7f9818166090 con 0x7f98180b2de0 2024-05-08T22:33:47.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98040035d0 con 0x7f98180b2de0 2024-05-08T22:33:47.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042650354 0 0) 0x7f981806dd50 con 0x7f98180b1a50 2024-05-08T22:33:47.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9818166090 con 0x7f98180b1a50 2024-05-08T22:33:47.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 173468228 0 0) 0x7f98040035d0 con 0x7f98180b2de0 2024-05-08T22:33:47.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 msgr2=0x7f9818165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9818165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 >> v1:172.21.15.195:6789/0 conn(0x7f98180b1a50 legacy=0x7f98180b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981806e030 con 0x7f98180b2de0 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9810001690 con 0x7f98180b2de0 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 >> v1:172.21.15.179:6789/0 conn(0x7f98180b2de0 legacy=0x7f981806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9804003be0 0x7f9804003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9804004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9804004db0 0x7f9804005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981806e9d0 con 0x7f9804003be0 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980400bb30 con 0x7f9804004db0 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980400bcf0 con 0x7f981814ae30 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9804003be0 0x7f9804003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981cc34700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9804004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981dc36700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9804004db0 0x7f9804005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 -- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 msgr2=0x7f9804004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f981814ae30 0x7f9804004870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 -- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9804004db0 msgr2=0x7f9804005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9804004db0 0x7f9804005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 -- 172.21.15.179:0/2969101517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040035d0 con 0x7f9804003be0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981dc36700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9804004db0 0x7f9804005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981d435700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9804003be0 0x7f9804003fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9808009b70 tx=0x7f98040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0bcba144a0458 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800e6c0 con 0x7f9804003be0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980800e820 con 0x7f9804003be0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f980ffff700 1 -- 172.21.15.179:0/2969101517 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800eb90 con 0x7f9804003be0 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- 172.21.15.179:0/2969101517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9804003be0 msgr2=0x7f9804003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 --2- 172.21.15.179:0/2969101517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9804003be0 0x7f9804003fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9808009b70 tx=0x7f98040058a0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981dc36700 1 -- 172.21.15.179:0/2969101517 reap_dead start 2024-05-08T22:33:47.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- 172.21.15.179:0/2969101517 shutdown_connections 2024-05-08T22:33:47.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- 172.21.15.179:0/2969101517 >> 172.21.15.179:0/2969101517 conn(0x7f98180cea30 msgr2=0x7f98180cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- 172.21.15.179:0/2969101517 shutdown_connections 2024-05-08T22:33:47.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.715+0000 7f981e437700 1 -- 172.21.15.179:0/2969101517 wait complete. 2024-05-08T22:33:47.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 Processor -- start 2024-05-08T22:33:47.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 -- start start 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 0x7f9818140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98181499b0 0x7f981813f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818149e60 con 0x7f9818161140 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818140b40 con 0x7f98181499b0 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981e437700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9818140580 con 0x7f9818160d60 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39906/0 (socket says 172.21.15.179:39906) 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 -- 172.21.15.179:0/2299715299 learned_addr learned my addr 172.21.15.179:0/2299715299 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981d435700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 0x7f9818140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981dc36700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98181499b0 0x7f981813f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 -- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 msgr2=0x7f9818140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 0x7f9818140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 -- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98181499b0 msgr2=0x7f981813f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98181499b0 0x7f981813f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 -- 172.21.15.179:0/2299715299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818140860 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981d435700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 0x7f9818140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f981cc34700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98140020b0 tx=0x7f981400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.719+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400bd20 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98181443d0 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981400be80 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814012630 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981813fcd0 con 0x7f9818161140 2024-05-08T22:33:47.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98181448f0 con 0x7f9818161140 2024-05-08T22:33:47.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9814012790 con 0x7f9818161140 2024-05-08T22:33:47.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f980dffb700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 0x7f97fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:47.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9814084b80 con 0x7f9818161140 2024-05-08T22:33:47.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.723+0000 7f981d435700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 0x7f97fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:47.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.727+0000 7f981d435700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 0x7f97fc068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98080090b0 tx=0x7f980801f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:47.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.731+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98181448f0 con 0x7f9818161140 2024-05-08T22:33:47.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.899+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98181448f0 con 0x7f9818161140 2024-05-08T22:33:47.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.899+0000 7f980dffb700 1 -- 172.21.15.179:0/2299715299 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f98181448f0 con 0x7f9818161140 2024-05-08T22:33:47.900 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 msgr2=0x7f97fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 0x7f97fc068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98080090b0 tx=0x7f980801f040 comp rx=0 tx=0).stop 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 msgr2=0x7f9818149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98140020b0 tx=0x7f981400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 shutdown_connections 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97fc066730 0x7f97fc068bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98181499b0 0x7f981813f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9818161140 0x7f9818149470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 --2- 172.21.15.179:0/2299715299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9818160d60 0x7f9818140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:47.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 >> 172.21.15.179:0/2299715299 conn(0x7f98180cea30 msgr2=0x7f98180cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:47.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 shutdown_connections 2024-05-08T22:33:47.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:47.903+0000 7f981e437700 1 -- 172.21.15.179:0/2299715299 wait complete. 2024-05-08T22:33:47.915 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:48.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.023+0000 7f7e9daf6700 1 Processor -- start 2024-05-08T22:33:48.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.023+0000 7f7e9daf6700 1 -- start start 2024-05-08T22:33:48.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e9806de70 0x7f7e98161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 0x7f7e9806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98165d50 con 0x7f7e9815ef80 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9806ad70 con 0x7f7e9806de70 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e98165220 con 0x7f7e98164db0 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e9806de70 0x7f7e98161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e9806de70 0x7f7e98161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39908/0 (socket says 172.21.15.179:39908) 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 -- 172.21.15.179:0/3731869153 learned_addr learned my addr 172.21.15.179:0/3731869153 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9caf4700 1 --1- 172.21.15.179:0/3731869153 >> v1:172.21.15.179:6789/0 conn(0x7f7e9815ef80 0x7f7e9806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47662/0 (socket says 172.21.15.179:47662) 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9d2f5700 1 --2- 172.21.15.179:0/3731869153 >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 0x7f7e9806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 -- 172.21.15.179:0/3731869153 >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 msgr2=0x7f7e9806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 --2- 172.21.15.179:0/3731869153 >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 0x7f7e9806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 -- 172.21.15.179:0/3731869153 >> v1:172.21.15.179:6789/0 conn(0x7f7e9815ef80 legacy=0x7f7e9806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 -- 172.21.15.179:0/3731869153 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9806e4b0 con 0x7f7e9806de70 2024-05-08T22:33:48.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9d2f5700 1 --2- 172.21.15.179:0/3731869153 >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 0x7f7e9806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:48.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e97fff700 1 --2- 172.21.15.179:0/3731869153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9806de70 0x7f7e98161790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7e98070c00 tx=0x7f7e80009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270368d005caffb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e977fe700 1 -- 172.21.15.179:0/3731869153 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000b8a0 con 0x7f7e9806de70 2024-05-08T22:33:48.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e977fe700 1 -- 172.21.15.179:0/3731869153 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8000ba00 con 0x7f7e9806de70 2024-05-08T22:33:48.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e977fe700 1 -- 172.21.15.179:0/3731869153 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000bd90 con 0x7f7e9806de70 2024-05-08T22:33:48.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- 172.21.15.179:0/3731869153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9806de70 msgr2=0x7f7e98161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/3731869153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9806de70 0x7f7e98161790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7e98070c00 tx=0x7f7e80009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- 172.21.15.179:0/3731869153 shutdown_connections 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/3731869153 >> v2:172.21.15.195:3300/0 conn(0x7f7e98164db0 0x7f7e9806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/3731869153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9806de70 0x7f7e98161790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- 172.21.15.179:0/3731869153 >> 172.21.15.179:0/3731869153 conn(0x7f7e980ce9d0 msgr2=0x7f7e980cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- 172.21.15.179:0/3731869153 shutdown_connections 2024-05-08T22:33:48.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.027+0000 7f7e9daf6700 1 -- 172.21.15.179:0/3731869153 wait complete. 2024-05-08T22:33:48.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 Processor -- start 2024-05-08T22:33:48.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- start start 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 0x7f7e980a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e98164db0 0x7f7e980adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981fb130 con 0x7f7e9815ef80 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980ab9a0 con 0x7f7e9806de70 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981fa370 con 0x7f7e98164db0 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39910/0 (socket says 172.21.15.179:39910) 2024-05-08T22:33:48.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 -- 172.21.15.179:0/1021031376 learned_addr learned my addr 172.21.15.179:0/1021031376 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 -- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e98164db0 msgr2=0x7f7e980adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e98164db0 0x7f7e980adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 -- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 msgr2=0x7f7e980a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9caf4700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 0x7f7e980a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 0x7f7e980a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 -- 172.21.15.179:0/1021031376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80009570 con 0x7f7e9815ef80 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9caf4700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 0x7f7e980a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e97fff700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7e98072020 tx=0x7f7e80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800038c0 con 0x7f7e9815ef80 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e981fa650 con 0x7f7e9815ef80 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8001f070 con 0x7f7e9815ef80 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003c60 con 0x7f7e9815ef80 2024-05-08T22:33:48.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.031+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e981fac10 con 0x7f7e9815ef80 2024-05-08T22:33:48.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e80021030 con 0x7f7e9815ef80 2024-05-08T22:33:48.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98067740 con 0x7f7e9815ef80 2024-05-08T22:33:48.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e957fa700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e80085c60 con 0x7f7e9815ef80 2024-05-08T22:33:48.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e9caf4700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.035+0000 7f7e9caf4700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000f60 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.043+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e80052c50 con 0x7f7e9815ef80 2024-05-08T22:33:48.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.247+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e981fae00 con 0x7f7e9815ef80 2024-05-08T22:33:48.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e957fa700 1 -- 172.21.15.179:0/1021031376 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f7e80056270 con 0x7f7e9815ef80 2024-05-08T22:33:48.250 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:48.250 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 msgr2=0x7f7e78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000f60 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 msgr2=0x7f7e980ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7e98072020 tx=0x7f7e80004770 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 shutdown_connections 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e98164db0 0x7f7e980adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e9815ef80 0x7f7e980ab460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 --2- 172.21.15.179:0/1021031376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e9806de70 0x7f7e980a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 >> 172.21.15.179:0/1021031376 conn(0x7f7e980ce9d0 msgr2=0x7f7e9806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 shutdown_connections 2024-05-08T22:33:48.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.251+0000 7f7e9daf6700 1 -- 172.21.15.179:0/1021031376 wait complete. 2024-05-08T22:33:48.254 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:33:48.265 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:33:48.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.371+0000 7ffaa3fff700 1 Processor -- start 2024-05-08T22:33:48.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.371+0000 7ffaa3fff700 1 -- start start 2024-05-08T22:33:48.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa41434d0 0x7ffaa406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffaa406dc20 0x7ffaa406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffaa40ce2a0 0x7ffaa406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40cd200 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40cd4e0 con 0x7ffaa406dc20 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40cd7c0 con 0x7ffaa41434d0 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffaa40ce2a0 0x7ffaa406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffaa40ce2a0 0x7ffaa406eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39912/0 (socket says 172.21.15.179:39912) 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 -- 172.21.15.179:0/1475062570 learned_addr learned my addr 172.21.15.179:0/1475062570 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffa9bfff700 1 --2- 172.21.15.179:0/1475062570 >> v2:172.21.15.195:3300/0 conn(0x7ffaa406dc20 0x7ffaa406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa2ffd700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa41434d0 0x7ffaa406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 -- 172.21.15.179:0/1475062570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa41434d0 msgr2=0x7ffaa406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa41434d0 0x7ffaa406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 -- 172.21.15.179:0/1475062570 >> v2:172.21.15.195:3300/0 conn(0x7ffaa406dc20 msgr2=0x7ffaa406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 --2- 172.21.15.179:0/1475062570 >> v2:172.21.15.195:3300/0 conn(0x7ffaa406dc20 0x7ffaa406e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 -- 172.21.15.179:0/1475062570 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40cdaa0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa37fe700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa406eb80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406b410 tx=0x7ffa9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bd2c076239a84a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa27fc700 1 -- 172.21.15.179:0/1475062570 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00e070 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa27fc700 1 -- 172.21.15.179:0/1475062570 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c00b910 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa27fc700 1 -- 172.21.15.179:0/1475062570 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00bcc0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- 172.21.15.179:0/1475062570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 msgr2=0x7ffaa406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa406eb80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406b410 tx=0x7ffa9c00b330 comp rx=0 tx=0).stop 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- 172.21.15.179:0/1475062570 shutdown_connections 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa406eb80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/1475062570 >> v2:172.21.15.195:3300/0 conn(0x7ffaa406dc20 0x7ffaa406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/1475062570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa41434d0 0x7ffaa406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- 172.21.15.179:0/1475062570 >> 172.21.15.179:0/1475062570 conn(0x7ffaa40ce9d0 msgr2=0x7ffaa40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- 172.21.15.179:0/1475062570 shutdown_connections 2024-05-08T22:33:48.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.375+0000 7ffaa3fff700 1 -- 172.21.15.179:0/1475062570 wait complete. 2024-05-08T22:33:48.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.379+0000 7ffaa3fff700 1 Processor -- start 2024-05-08T22:33:48.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.379+0000 7ffaa3fff700 1 -- start start 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 0x7ffaa41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa415a290 0x7ffaa415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa415ac60 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4159b20 con 0x7ffaa41434d0 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40c3950 con 0x7ffaa415a290 2024-05-08T22:33:48.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39928/0 (socket says 172.21.15.179:39928) 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 -- 172.21.15.179:0/2043272948 learned_addr learned my addr 172.21.15.179:0/2043272948 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9bfff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 0x7ffaa41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa37fe700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa415a290 0x7ffaa415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 -- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa415a290 msgr2=0x7ffaa415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa415a290 0x7ffaa415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 -- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 msgr2=0x7ffaa41595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 0x7ffaa41595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 -- 172.21.15.179:0/2043272948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c009b30 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9bfff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 0x7ffaa41595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa2ffd700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffa94010e90 tx=0x7ffa9400e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400b9e0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94011650 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa40c3ba0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940119e0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa40c4160 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa4067740 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.383+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa94011b40 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.387+0000 7ffa9b7fe700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 0x7ffa84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.387+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa94087140 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.387+0000 7ffa9bfff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 0x7ffa84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.387+0000 7ffa9bfff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 0x7ffa84068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40cb330 tx=0x7ffa8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.391+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa940541c0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.563+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa404e700 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffa9b7fe700 1 -- 172.21.15.179:0/2043272948 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7ffa940577e0 con 0x7ffaa40ce2a0 2024-05-08T22:33:48.566 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:48.566 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 msgr2=0x7ffa84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 0x7ffa84068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40cb330 tx=0x7ffa8c008040 comp rx=0 tx=0).stop 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 msgr2=0x7ffaa40c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffa94010e90 tx=0x7ffa9400e640 comp rx=0 tx=0).stop 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 shutdown_connections 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffa84066730 0x7ffa84068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffaa41434d0 0x7ffaa41595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffaa415a290 0x7ffaa415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 --2- 172.21.15.179:0/2043272948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa40ce2a0 0x7ffaa40c4c40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 >> 172.21.15.179:0/2043272948 conn(0x7ffaa40ce9d0 msgr2=0x7ffaa40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 shutdown_connections 2024-05-08T22:33:48.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.567+0000 7ffaa3fff700 1 -- 172.21.15.179:0/2043272948 wait complete. 2024-05-08T22:33:48.582 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:33:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 Processor -- start 2024-05-08T22:33:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 -- start start 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64c006e840 0x7f64c00b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c015d010 con 0x7f64c00b3880 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c015d2f0 con 0x7f64c006bb50 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64c5018700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c015dc90 con 0x7f64c006e840 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bf7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f64c00b3880 0x7f64c006b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47664/0 (socket says 172.21.15.179:47664) 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64c006e840 0x7f64c00b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bf7fe700 1 -- 172.21.15.179:0/884473868 learned_addr learned my addr 172.21.15.179:0/884473868 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:48.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bffff700 1 -- 172.21.15.179:0/884473868 >> v1:172.21.15.195:6789/0 conn(0x7f64c006bb50 legacy=0x7f64c00b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64beffd700 1 -- 172.21.15.179:0/884473868 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766895276 0 0) 0x7f64c015d010 con 0x7f64c00b3880 2024-05-08T22:33:48.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bffff700 1 -- 172.21.15.179:0/884473868 >> v1:172.21.15.179:6789/0 conn(0x7f64c00b3880 legacy=0x7f64c006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bffff700 1 -- 172.21.15.179:0/884473868 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c015d5d0 con 0x7f64c006e840 2024-05-08T22:33:48.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.691+0000 7f64bffff700 1 --2- 172.21.15.179:0/884473868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c00b3340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009dc0 tx=0x7f64b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5845361aeef0e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64beffd700 1 -- 172.21.15.179:0/884473868 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000e070 con 0x7f64c006e840 2024-05-08T22:33:48.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64beffd700 1 -- 172.21.15.179:0/884473868 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000bb90 con 0x7f64c006e840 2024-05-08T22:33:48.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64beffd700 1 -- 172.21.15.179:0/884473868 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0012430 con 0x7f64c006e840 2024-05-08T22:33:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- 172.21.15.179:0/884473868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 msgr2=0x7f64c00b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 --2- 172.21.15.179:0/884473868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c00b3340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009dc0 tx=0x7f64b000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- 172.21.15.179:0/884473868 shutdown_connections 2024-05-08T22:33:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 --2- 172.21.15.179:0/884473868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c00b3340 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- 172.21.15.179:0/884473868 >> 172.21.15.179:0/884473868 conn(0x7f64c00ce9d0 msgr2=0x7f64c00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- 172.21.15.179:0/884473868 shutdown_connections 2024-05-08T22:33:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- 172.21.15.179:0/884473868 wait complete. 2024-05-08T22:33:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 Processor -- start 2024-05-08T22:33:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.695+0000 7f64c5018700 1 -- start start 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64c006bb50 0x7f64c00c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64c00b3880 0x7f64c0152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01403a0 con 0x7f64c006e840 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01f4e30 con 0x7f64c00b3880 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01f4b50 con 0x7f64c006bb50 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39946/0 (socket says 172.21.15.179:39946) 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 -- 172.21.15.179:0/2679247774 learned_addr learned my addr 172.21.15.179:0/2679247774 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bffff700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64c006bb50 0x7f64c00c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c4817700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64c00b3880 0x7f64c0152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 -- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64c006bb50 msgr2=0x7f64c00c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64c006bb50 0x7f64c00c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 -- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64c00b3880 msgr2=0x7f64c0152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64c00b3880 0x7f64c0152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 -- 172.21.15.179:0/2679247774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b0009a70 con 0x7f64c006e840 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bf7fe700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c480 tx=0x7f64a800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004ca0 con 0x7f64c006e840 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8004e00 con 0x7f64c006e840 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c013fd50 con 0x7f64c006e840 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8007310 con 0x7f64c006e840 2024-05-08T22:33:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.699+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c00c5720 con 0x7f64c006e840 2024-05-08T22:33:48.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c0140550 con 0x7f64c006e840 2024-05-08T22:33:48.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64a8007470 con 0x7f64c006e840 2024-05-08T22:33:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64bcff9700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 0x7f64a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f64a8084100 con 0x7f64c006e840 2024-05-08T22:33:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64bffff700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 0x7f64a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.703+0000 7f64bffff700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 0x7f64a0068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64b0014040 tx=0x7f64b001d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:48.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.707+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64c0140550 con 0x7f64c006e840 2024-05-08T22:33:48.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.919+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64c0140550 con 0x7f64c006e840 2024-05-08T22:33:48.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.919+0000 7f64bcff9700 1 -- 172.21.15.179:0/2679247774 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f64c0140550 con 0x7f64c006e840 2024-05-08T22:33:48.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:33:48.919 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:33:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 msgr2=0x7f64a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 0x7f64a0068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64b0014040 tx=0x7f64b001d040 comp rx=0 tx=0).stop 2024-05-08T22:33:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 msgr2=0x7f64c0152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c480 tx=0x7f64a800c840 comp rx=0 tx=0).stop 2024-05-08T22:33:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 shutdown_connections 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f64a0066730 0x7f64a0068bf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64c00b3880 0x7f64c0152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64c006e840 0x7f64c0152220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 --2- 172.21.15.179:0/2679247774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64c006bb50 0x7f64c00c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 >> 172.21.15.179:0/2679247774 conn(0x7f64c00ce9d0 msgr2=0x7f64c014f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 shutdown_connections 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:48.923+0000 7f64c5018700 1 -- 172.21.15.179:0/2679247774 wait complete. 2024-05-08T22:33:48.923 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:33:49.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:33:14.889801+0000', 'modified': '2024-05-08T22:33:37.005931+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54155}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54155': {'gid': 54155, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/3661147208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 3661147208}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 3661147208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:33:49.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:33:49.337 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:33:49.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 Processor -- start 2024-05-08T22:33:49.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 -- start start 2024-05-08T22:33:49.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3d3c0b25d0 0x7f3d3c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3c0cc340 con 0x7f3d3c06dda0 2024-05-08T22:33:49.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c06b540 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c06ac50 con 0x7f3d3c06e180 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3d3c0b25d0 0x7f3d3c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:38468/0 (socket says 172.21.15.179:38468) 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d33fff700 1 -- 172.21.15.179:0/1847985629 learned_addr learned my addr 172.21.15.179:0/1847985629 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 msgr2=0x7f3d3c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/1847985629 >> v1:172.21.15.195:6789/0 conn(0x7f3d3c06dda0 legacy=0x7f3d3c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/1847985629 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c06aef0 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d33fff700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:49.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c0b2a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009d90 tx=0x7f3d3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcf3abf54f8c119 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d39d9b700 1 -- 172.21.15.179:0/1847985629 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400e070 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d39d9b700 1 -- 172.21.15.179:0/1847985629 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34003a40 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.467+0000 7f3d39d9b700 1 -- 172.21.15.179:0/1847985629 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003df0 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 msgr2=0x7f3d3c0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c0b2a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009d90 tx=0x7f3d3400b330 comp rx=0 tx=0).stop 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- 172.21.15.179:0/1847985629 shutdown_connections 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c0b2a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/1847985629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06e180 0x7f3d3c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- 172.21.15.179:0/1847985629 >> 172.21.15.179:0/1847985629 conn(0x7f3d3c0ce960 msgr2=0x7f3d3c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- 172.21.15.179:0/1847985629 shutdown_connections 2024-05-08T22:33:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- 172.21.15.179:0/1847985629 wait complete. 2024-05-08T22:33:49.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 Processor -- start 2024-05-08T22:33:49.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- start start 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 0x7f3d3c162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c1f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c1f74f0 con 0x7f3d3c06e180 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c1f77d0 con 0x7f3d3c0b25d0 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c1f7ab0 con 0x7f3d3c06dda0 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 0x7f3d3c162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d3ad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c1f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39958/0 (socket says 172.21.15.179:39958) 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.471+0000 7f3d33fff700 1 -- 172.21.15.179:0/72717294 learned_addr learned my addr 172.21.15.179:0/72717294 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 -- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 msgr2=0x7f3d3c162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 0x7f3d3c162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 -- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 msgr2=0x7f3d3c1f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c1f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 -- 172.21.15.179:0/72717294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009570 con 0x7f3d3c06e180 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 0x7f3d3c162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d33fff700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400ebe0 tx=0x7f3d2400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d32ffd700 1 -- 172.21.15.179:0/72717294 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400c850 con 0x7f3d3c06e180 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d34009a70 con 0x7f3d3c06e180 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d32ffd700 1 -- 172.21.15.179:0/72717294 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2400c9b0 con 0x7f3d3c06e180 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c161a40 con 0x7f3d3c06e180 2024-05-08T22:33:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.475+0000 7f3d32ffd700 1 -- 172.21.15.179:0/72717294 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400cd20 con 0x7f3d3c06e180 2024-05-08T22:33:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d32ffd700 1 -- 172.21.15.179:0/72717294 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d24013470 con 0x7f3d3c06e180 2024-05-08T22:33:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d32ffd700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d32ffd700 1 -- 172.21.15.179:0/72717294 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d24084620 con 0x7f3d3c06e180 2024-05-08T22:33:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 0x7f3d1c068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c007fd0 tx=0x7f3d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d44290740 1 Processor -- start 2024-05-08T22:33:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.479+0000 7f3d44290740 1 -- start start 2024-05-08T22:33:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x1457ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x141f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x137ed80 con 0x144ebe0 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x137c510 con 0x1457b00 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1377e00 con 0x1458420 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39974/0 (socket says 172.21.15.179:39974) 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 -- 172.21.15.179:0/2342061188 learned_addr learned my addr 172.21.15.179:0/2342061188 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x1457ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x141f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 -- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 msgr2=0x141f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x141f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 -- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 msgr2=0x1457ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x1457ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 -- 172.21.15.179:0/2342061188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11ea960 con 0x144ebe0 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x141f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d33fff700 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d24051740 tx=0x7f3d24054e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08d57fb5ce4ea15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d1bfff700 1 -- 172.21.15.179:0/2342061188 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24055240 con 0x144ebe0 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d1bfff700 1 -- 172.21.15.179:0/2342061188 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24059070 con 0x144ebe0 2024-05-08T22:33:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d1bfff700 1 -- 172.21.15.179:0/2342061188 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2405d630 con 0x144ebe0 2024-05-08T22:33:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- 172.21.15.179:0/2342061188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 msgr2=0x141f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d24051740 tx=0x7f3d24054e00 comp rx=0 tx=0).stop 2024-05-08T22:33:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- 172.21.15.179:0/2342061188 shutdown_connections 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x141f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x141f470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 --2- 172.21.15.179:0/2342061188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x1457ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- 172.21.15.179:0/2342061188 >> 172.21.15.179:0/2342061188 conn(0x119f4d0 msgr2=0x1439250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- 172.21.15.179:0/2342061188 shutdown_connections 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.483+0000 7f3d44290740 1 -- 172.21.15.179:0/2342061188 wait complete. 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- start start 2024-05-08T22:33:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 Processor -- start 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x140f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x140feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1335730 con 0x144ebe0 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x133b5b0 con 0x1457b00 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1223aa0 con 0x1458420 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 10 client.0 _ll_register_callbacks cb 0x1371610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:33:49.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39980/0 (socket says 172.21.15.179:39980) 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 -- 172.21.15.179:0/54954878 learned_addr learned my addr 172.21.15.179:0/54954878 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d33fff700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x140f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x140feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 -- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 msgr2=0x140feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x140feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 -- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 msgr2=0x140f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x140f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 -- 172.21.15.179:0/54954878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c161a40 con 0x144ebe0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d33fff700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x140f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0072c0 tx=0x7f3d2c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9707d01727c093c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:33:49.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c026030 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d2c027070 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x11ac1e0 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d2c02d470 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 10 client.4893 fetch_fsmap learned FSMap version 9 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.487+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x133e220 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c031020 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f3d2c021410 con 0x144ebe0 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 10 client.4893 fetch_fsmap finished waiting for FSMap version 9 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 10 client.4893 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 20 client.4893 populate_metadata read hostname 'smithi179' 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1450400 0x146b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 4 client.4893 mds_command: new command op to 54155 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 --> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x11f6460 con 0x1450400 2024-05-08T22:33:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d33fff700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1450400 0x146b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 10 client.4893 ms_handle_connect on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:49.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d33fff700 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1450400 0x146b380 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e768e0e90f6bfddb server_cookie=8325f5ab55b1d67f in_seq=0 out_seq=0 2024-05-08T22:33:49.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 1 -- 172.21.15.179:0/54954878 <== mds.0 v2:172.21.15.195:6836/3661147208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x11f6460 con 0x1450400 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 10 client.4893 handle_command_reply: tid=0 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 client.4893 shutdown 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 msgr2=0x144e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0072c0 tx=0x7f3d2c000ed0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 shutdown_connections 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1450400 0x146b380 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x140f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d197fa700 0 client.4893 ms_handle_reset on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:49.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1458420 0x140feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 --2- 172.21.15.179:0/54954878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x144e2e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 >> 172.21.15.179:0/54954878 conn(0x119f4d0 msgr2=0x1438a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 shutdown_connections 2024-05-08T22:33:49.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 1 -- 172.21.15.179:0/54954878 wait complete. 2024-05-08T22:33:49.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.491+0000 7f3d44290740 20 client.4893 trim_cache size 0 max 16384 2024-05-08T22:33:49.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 Processor -- start 2024-05-08T22:33:49.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- start start 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1450400 0x1467170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x14676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x13418d0 con 0x144ebe0 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1339e90 con 0x1457b00 2024-05-08T22:33:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x131e950 con 0x1450400 2024-05-08T22:33:49.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39982/0 (socket says 172.21.15.179:39982) 2024-05-08T22:33:49.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 -- 172.21.15.179:0/1591984436 learned_addr learned my addr 172.21.15.179:0/1591984436 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d33fff700 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1450400 0x1467170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x14676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 -- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1450400 msgr2=0x1467170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1450400 0x1467170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 -- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 msgr2=0x14676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x14676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 -- 172.21.15.179:0/1591984436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1324450 con 0x144ebe0 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c001c20 tx=0x7f3d2c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b055f1e8f3b7e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d30ff9700 1 -- 172.21.15.179:0/1591984436 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c026040 con 0x144ebe0 2024-05-08T22:33:49.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d30ff9700 1 -- 172.21.15.179:0/1591984436 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c031040 con 0x144ebe0 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d30ff9700 1 -- 172.21.15.179:0/1591984436 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c021970 con 0x144ebe0 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- 172.21.15.179:0/1591984436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 msgr2=0x140ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c001c20 tx=0x7f3d2c01b000 comp rx=0 tx=0).stop 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- 172.21.15.179:0/1591984436 shutdown_connections 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1457b00 0x14676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1450400 0x1467170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 --2- 172.21.15.179:0/1591984436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x144ebe0 0x140ff50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- 172.21.15.179:0/1591984436 >> 172.21.15.179:0/1591984436 conn(0x119f4d0 msgr2=0x1460980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- 172.21.15.179:0/1591984436 shutdown_connections 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.515+0000 7f3d44290740 1 -- 172.21.15.179:0/1591984436 wait complete. 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- start start 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 Processor -- start 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x144ebe0 0x14b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1450400 0x140b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x144d560 con 0x1457b00 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x14b9a10 con 0x1450400 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x144d790 con 0x144ebe0 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 10 client.0 _ll_register_callbacks cb 0x7f3d2400ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39984/0 (socket says 172.21.15.179:39984) 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/2538427922 learned_addr learned my addr 172.21.15.179:0/2538427922 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x144ebe0 0x14b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d33fff700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1450400 0x140b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x144ebe0 msgr2=0x14b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x144ebe0 0x14b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1450400 msgr2=0x140b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1450400 0x140b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 -- 172.21.15.179:0/2538427922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x133e220 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3ad9d700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cf5e0 tx=0x7f3d34004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1b383179a30305 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34023070 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d340045a0 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d24059070 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d34004760 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 10 client.4899 fetch_fsmap learned FSMap version 9 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x1318720 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24059070 con 0x1457b00 2024-05-08T22:33:49.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f3d3400b050 con 0x1457b00 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 10 client.4899 fetch_fsmap finished waiting for FSMap version 9 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 10 client.4899 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 20 client.4899 populate_metadata read hostname 'smithi179' 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1458420 0x14ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 4 client.4899 mds_command: new command op to 54155 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 --> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] -- command(tid 0: {"prefix": "status"}) v1 -- 0x121b1e0 con 0x1458420 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.519+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1458420 0x14ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d197fa700 10 client.4899 ms_handle_connect on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3a59c700 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1458420 0x14ba230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be50b04a4d9d13f server_cookie=df14a8858e38be0d in_seq=0 out_seq=0 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d197fa700 1 -- 172.21.15.179:0/2538427922 <== mds.0 v2:172.21.15.195:6836/3661147208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x121b1e0 con 0x1458420 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d197fa700 10 client.4899 handle_command_reply: tid=0 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 client.4899 shutdown 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 msgr2=0x140bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cf5e0 tx=0x7f3d34004b60 comp rx=0 tx=0).stop 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 shutdown_connections 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:6836/3661147208,v1:172.21.15.195:6837/3661147208] conn(0x1458420 0x14ba230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1450400 0x140b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1457b00 0x140bc40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 --2- 172.21.15.179:0/2538427922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x144ebe0 0x14b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d197fa700 0 client.4899 ms_handle_reset on v2:172.21.15.195:6836/3661147208 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 >> 172.21.15.179:0/2538427922 conn(0x119f4d0 msgr2=0x1460980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 shutdown_connections 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 1 -- 172.21.15.179:0/2538427922 wait complete. 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d44290740 20 client.4899 trim_cache size 0 max 16384 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 54155, 2024-05-08T22:33:49.528 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 34.629678143, 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 9, 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 26, 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 26, 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 40.953151521999999 2024-05-08T22:33:49.529 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 msgr2=0x7f3d1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 0x7f3d1c068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c007fd0 tx=0x7f3d2c009040 comp rx=0 tx=0).stop 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 msgr2=0x7f3d3c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400ebe0 tx=0x7f3d2400eef0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 shutdown_connections 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d1c066730 0x7f3d1c068bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d3c0b25d0 0x7f3d3c1f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c06e180 0x7f3d3c159b10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 --2- 172.21.15.179:0/72717294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d3c06dda0 0x7f3d3c162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 >> 172.21.15.179:0/72717294 conn(0x7f3d3c0ce960 msgr2=0x7f3d3c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 shutdown_connections 2024-05-08T22:33:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.523+0000 7f3d3b59e700 1 -- 172.21.15.179:0/72717294 wait complete. 2024-05-08T22:33:49.541 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T22:33:49.551 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T22:33:49.551 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T22:33:49.551 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T22:33:49.551 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-08T22:33:49.574 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-08T22:33:49.575 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T22:33:49.575 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:49.575 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:49.575 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:33:49.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:49.576 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-08T22:33:49.576 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:33:49.576 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T22:33:49.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:49.576 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-05-08T22:33:49.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:49.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-05-08T22:33:49.607 INFO:teuthology.orchestra.run.smithi195.stderr:Cannot find device "ceph-brx" 2024-05-08T22:33:49.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:33:49.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:49.608 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:33:49.616 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T22:33:49.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:49.616 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T22:33:49.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:49.674 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-08T22:33:49.787 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-08T22:33:49.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:33:49.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 Processor -- start 2024-05-08T22:33:49.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 -- start start 2024-05-08T22:33:49.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a840ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0a8415b1b0 0x7f0a840cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a840cb510 0x7f0a8415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840cb9a0 con 0x7f0a840cb510 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8406abb0 con 0x7f0a8415b1b0 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a88aa1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8406ae90 con 0x7f0a840cc700 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a840cb510 0x7f0a8415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a840ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a840cb510 0x7f0a8415e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39986/0 (socket says 172.21.15.179:39986) 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 -- 172.21.15.179:0/142534887 learned_addr learned my addr 172.21.15.179:0/142534887 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/142534887 >> v2:172.21.15.195:3300/0 conn(0x7f0a8415b1b0 0x7f0a840cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 -- 172.21.15.179:0/142534887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 msgr2=0x7f0a840ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 --2- 172.21.15.179:0/142534887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a840ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 -- 172.21.15.179:0/142534887 >> v2:172.21.15.195:3300/0 conn(0x7f0a8415b1b0 msgr2=0x7f0a840cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:49.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 --2- 172.21.15.179:0/142534887 >> v2:172.21.15.195:3300/0 conn(0x7f0a8415b1b0 0x7f0a840cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 -- 172.21.15.179:0/142534887 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8415b9c0 con 0x7f0a840cb510 2024-05-08T22:33:49.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.899+0000 7f0a83fff700 1 --2- 172.21.15.179:0/142534887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a8415e7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a8406c6f0 tx=0x7f0a78009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e056837720adb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a827fc700 1 -- 172.21.15.179:0/142534887 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78011830 con 0x7f0a840cb510 2024-05-08T22:33:49.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a827fc700 1 -- 172.21.15.179:0/142534887 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a78011990 con 0x7f0a840cb510 2024-05-08T22:33:49.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a827fc700 1 -- 172.21.15.179:0/142534887 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78011d20 con 0x7f0a840cb510 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- 172.21.15.179:0/142534887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 msgr2=0x7f0a8415e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/142534887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a8415e7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a8406c6f0 tx=0x7f0a78009bf0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- 172.21.15.179:0/142534887 shutdown_connections 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/142534887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a8415e7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/142534887 >> v2:172.21.15.195:3300/0 conn(0x7f0a8415b1b0 0x7f0a840cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/142534887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a840ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- 172.21.15.179:0/142534887 >> 172.21.15.179:0/142534887 conn(0x7f0a840ce9d0 msgr2=0x7f0a840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- 172.21.15.179:0/142534887 shutdown_connections 2024-05-08T22:33:49.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- 172.21.15.179:0/142534887 wait complete. 2024-05-08T22:33:49.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 Processor -- start 2024-05-08T22:33:49.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 -- start start 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.903+0000 7f0a88aa1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a8415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8415b1b0 0x7f0a84157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84146e10 con 0x7f0a840cb510 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841542e0 con 0x7f0a8415b1b0 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84153660 con 0x7f0a840cc700 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40002/0 (socket says 172.21.15.179:40002) 2024-05-08T22:33:49.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 -- 172.21.15.179:0/3552310201 learned_addr learned my addr 172.21.15.179:0/3552310201 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a8415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a83fff700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8415b1b0 0x7f0a84157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 -- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 msgr2=0x7f0a8415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a8415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 -- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8415b1b0 msgr2=0x7f0a84157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8415b1b0 0x7f0a84157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 -- 172.21.15.179:0/3552310201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78009570 con 0x7f0a840cb510 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a837fe700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00efc0 tx=0x7f0a7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c009ea0 con 0x7f0a840cb510 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c00f040 con 0x7f0a840cb510 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a84153940 con 0x7f0a840cb510 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c014480 con 0x7f0a840cb510 2024-05-08T22:33:49.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a84153f00 con 0x7f0a840cb510 2024-05-08T22:33:49.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84154420 con 0x7f0a840cb510 2024-05-08T22:33:49.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.907+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a7c0145e0 con 0x7f0a840cb510 2024-05-08T22:33:49.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.911+0000 7f0a6bfff700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 0x7f0a70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:49.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.911+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 0x7f0a70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:49.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.911+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a7c084d00 con 0x7f0a840cb510 2024-05-08T22:33:49.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.911+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 0x7f0a70068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a8415aca0 tx=0x7f0a74009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:49.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:49.919+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a84154420 con 0x7f0a840cb510 2024-05-08T22:33:50.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a84154420 con 0x7f0a840cb510 2024-05-08T22:33:50.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a6bfff700 1 -- 172.21.15.179:0/3552310201 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0a84154420 con 0x7f0a840cb510 2024-05-08T22:33:50.130 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 msgr2=0x7f0a70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 0x7f0a70068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a8415aca0 tx=0x7f0a74009240 comp rx=0 tx=0).stop 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 msgr2=0x7f0a84154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00efc0 tx=0x7f0a7c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 shutdown_connections 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a70066730 0x7f0a70068bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a8415b1b0 0x7f0a84157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:50.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a840cc700 0x7f0a8415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:50.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 --2- 172.21.15.179:0/3552310201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a840cb510 0x7f0a84154ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:50.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 >> 172.21.15.179:0/3552310201 conn(0x7f0a840ce9d0 msgr2=0x7f0a840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:50.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 shutdown_connections 2024-05-08T22:33:50.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:50.131+0000 7f0a88aa1700 1 -- 172.21.15.179:0/3552310201 wait complete. 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:33:50.146 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:33:50.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.147 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:33:50.151 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:33:50.152 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42541sec preferred_lft 42541sec 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:50.153 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:33:50.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.153 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr flush dev ceph-brx 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set ceph-brx up 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:33:50.154 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:50.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.194 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:33:50.203 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:33:50.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.205 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:33:50.261 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:33:50.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:33:50.263 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:50.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.301 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:33:50.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.307 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-05-08T22:33:50.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.313 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:50.313 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:50.313 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:33:50.313 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T22:33:50.313 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:50.343 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T22:33:50.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.343 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:50.343 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:50.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T22:33:50.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:33:50.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T22:33:50.344 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T22:33:50.344 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T22:33:50.344 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:50.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.447 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:50.447 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:50.447 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:33:50.447 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:33:50.447 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:50.475 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:33:50.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:50.475 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:50.480 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:33:50.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:50.481 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:50.485 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:50.546 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:50.546 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:50.556 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:50.557 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:50.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.558 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:50.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:33:50.563 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:33:50.607 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:50.626 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:50.626 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:50.641 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:50.642 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:50.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:50.643 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:50.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:33:50.695+0000 7f4fa5ef5640 -1 init, newargv = 0x5595ec0fdf10 newargc=15 2024-05-08T22:33:50.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[33614]: starting ceph client 2024-05-08T22:33:50.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[33614]: starting fuse 2024-05-08T22:33:51.568 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:33:51.571 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:33:51.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:51.810 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:51.810 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:51.866 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:51.867 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:51.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:51.868 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:51.874 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:33:51.875 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:33:51.875 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:33:51.876 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:33:51.877 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:51.877 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:33:51.877 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:33:51.960 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.33614.asok 2024-05-08T22:33:52.144 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33614.asok 2024-05-08T22:33:52.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.145 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33614.asok status 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:33:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "33614", 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 4911, 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 4911 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2626877779 2024-05-08T22:33:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2626877779 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.4911 192.168.0.1:0/2626877779", 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/2626877779", 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 9, 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 26, 2024-05-08T22:33:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:33:52.291 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:33:52.291 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:33:52.291 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:33:52.298 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:33:52.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 Processor -- start 2024-05-08T22:33:52.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- start start 2024-05-08T22:33:52.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e140c8d70 0x7f6e1415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:52.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e14158f60 con 0x7f6e140c9230 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1406b860 con 0x7f6e14158b80 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1406ac50 con 0x7f6e140c8d70 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e18fc1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6e140c9230 0x7f6e1406ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47674/0 (socket says 172.21.15.179:47674) 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e18fc1700 1 -- 172.21.15.179:0/102594733 learned_addr learned my addr 172.21.15.179:0/102594733 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e137fe700 1 --2- 172.21.15.179:0/102594733 >> v2:172.21.15.179:3300/0 conn(0x7f6e140c8d70 0x7f6e1415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:52.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e12ffd700 1 -- 172.21.15.179:0/102594733 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461075068 0 0) 0x7f6e14158f60 con 0x7f6e140c9230 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e12ffd700 1 -- 172.21.15.179:0/102594733 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dfc0035d0 con 0x7f6e140c9230 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e137fe700 1 -- 172.21.15.179:0/102594733 >> v1:172.21.15.195:6789/0 conn(0x7f6e14158b80 legacy=0x7f6e1406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e137fe700 1 -- 172.21.15.179:0/102594733 >> v1:172.21.15.179:6789/0 conn(0x7f6e140c9230 legacy=0x7f6e1406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e137fe700 1 -- 172.21.15.179:0/102594733 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1406af30 con 0x7f6e140c8d70 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e137fe700 1 --2- 172.21.15.179:0/102594733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e1415b4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009850 tx=0x7f6e08009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dd9d43b98999bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:52.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e12ffd700 1 -- 172.21.15.179:0/102594733 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08011830 con 0x7f6e140c8d70 2024-05-08T22:33:52.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e12ffd700 1 -- 172.21.15.179:0/102594733 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08011990 con 0x7f6e140c8d70 2024-05-08T22:33:52.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e12ffd700 1 -- 172.21.15.179:0/102594733 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08011d40 con 0x7f6e140c8d70 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- 172.21.15.179:0/102594733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 msgr2=0x7f6e1415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/102594733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e1415b4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009850 tx=0x7f6e08009b60 comp rx=0 tx=0).stop 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- 172.21.15.179:0/102594733 shutdown_connections 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/102594733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e1415b4d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- 172.21.15.179:0/102594733 >> 172.21.15.179:0/102594733 conn(0x7f6e140cb660 msgr2=0x7f6e140cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:52.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.415+0000 7f6e1aa24700 1 -- 172.21.15.179:0/102594733 shutdown_connections 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 -- 172.21.15.179:0/102594733 wait complete. 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 Processor -- start 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 -- start start 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e140c9230 0x7f6e140ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:52.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e14158b80 0x7f6e14150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14152ce0 con 0x7f6e140c8d70 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c5cb0 con 0x7f6e140c9230 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.419+0000 7f6e1aa24700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c5f00 con 0x7f6e14158b80 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40010/0 (socket says 172.21.15.179:40010) 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 -- 172.21.15.179:0/3806989789 learned_addr learned my addr 172.21.15.179:0/3806989789 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:52.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e137fe700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e140c9230 0x7f6e140ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:52.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e18fc1700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e14158b80 0x7f6e14150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:52.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 -- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e14158b80 msgr2=0x7f6e14150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e14158b80 0x7f6e14150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 -- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e140c9230 msgr2=0x7f6e140ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e140c9230 0x7f6e140ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 -- 172.21.15.179:0/3806989789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e080094b0 con 0x7f6e140c8d70 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e13fff700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c500 tx=0x7f6e0400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e040052d0 con 0x7f6e140c8d70 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04005430 con 0x7f6e140c8d70 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140c61e0 con 0x7f6e140c8d70 2024-05-08T22:33:52.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e040055a0 con 0x7f6e140c8d70 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e140c67a0 con 0x7f6e140c8d70 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e040038e0 con 0x7f6e140c8d70 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e10ff9700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 0x7f6e00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.423+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14067770 con 0x7f6e140c8d70 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.427+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6e04083820 con 0x7f6e140c8d70 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.427+0000 7f6e137fe700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 0x7f6e00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:52.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.427+0000 7f6e137fe700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 0x7f6e00068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e080090b0 tx=0x7f6e08008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:52.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.431+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6e04017080 con 0x7f6e140c8d70 2024-05-08T22:33:52.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e14073290 con 0x7f6e140c8d70 2024-05-08T22:33:52.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e10ff9700 1 -- 172.21.15.179:0/3806989789 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6e040508a0 con 0x7f6e140c8d70 2024-05-08T22:33:52.638 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 msgr2=0x7f6e00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 0x7f6e00068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e080090b0 tx=0x7f6e08008030 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 msgr2=0x7f6e141f0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c500 tx=0x7f6e0400c810 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 shutdown_connections 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e00066730 0x7f6e00068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e140c9230 0x7f6e140ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e14158b80 0x7f6e14150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 --2- 172.21.15.179:0/3806989789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e140c8d70 0x7f6e141f0910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 >> 172.21.15.179:0/3806989789 conn(0x7f6e140cb660 msgr2=0x7f6e14071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 shutdown_connections 2024-05-08T22:33:52.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:52.639+0000 7f6e1aa24700 1 -- 172.21.15.179:0/3806989789 wait complete. 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:33:52.653 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T22:33:52.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.654 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:52.659 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.660 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.661 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:33:52.676 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:33:52.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.678 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:33:52.733 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:33:52.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:33:52.735 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:52.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.774 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:33:52.780 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:33:52.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.780 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-05-08T22:33:52.787 INFO:teuthology.orchestra.run.smithi179.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T22:33:52.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.787 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:52.787 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:52.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:33:52.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T22:33:52.787 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:52.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.815 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T22:33:52.831 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 2e:92:cf:53:24:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.834 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::2c92:cfff:fe53:2421/64 scope link 2024-05-08T22:33:52.835 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.836 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T22:33:52.849 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:52.849 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:52.849 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 2e:92:cf:53:24:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::2c92:cfff:fe53:2421/64 scope link 2024-05-08T22:33:52.850 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:52.851 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T22:33:52.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T22:33:52.852 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:52.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:52.956 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:33:52.956 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:33:52.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.1 up 2024-05-08T22:33:52.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T22:33:52.956 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:33:52.980 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-08T22:33:52.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:52.980 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:52.987 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T22:33:52.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:52.988 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:53.038 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:53.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:53.055 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:53.113 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:53.114 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:53.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:53.115 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:53.121 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:33:53.122 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-05-08T22:33:53.122 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-08T22:33:53.124 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:53.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:53.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:53.191 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:53.193 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:53.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:53.194 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:53.198 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:33:53.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi179.stderr:2024-05-08T22:33:53.207+0000 7f90f882b640 -1 init, newargv = 0x55adcdfe78a0 newargc=15 2024-05-08T22:33:53.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi179.stderr:ceph-fuse[33757]: starting ceph client 2024-05-08T22:33:53.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi179.stderr:ceph-fuse[33757]: starting fuse 2024-05-08T22:33:54.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:33:54.318 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:54.318 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:54.377 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:54.377 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:54.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:54.378 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:33:54.385 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:33:54.385 INFO:teuthology.orchestra.run.smithi179.stdout:53 2024-05-08T22:33:54.386 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:33:54.386 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:33:54.387 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:33:54.388 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:33:54.388 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:33:54.388 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:33:54.388 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.2")) 2024-05-08T22:33:54.388 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:33:54.468 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.2.33757.asok 2024-05-08T22:33:54.655 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33757.asok 2024-05-08T22:33:54.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:54.655 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33757.asok status 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "2", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "33757", 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:33:54.803 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 4929, 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 4929 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.2:0", 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 264374150 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:33:54.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.2:0", 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 264374150 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.4929 192.168.0.2:0/264374150", 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.2:0/264374150", 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 9, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 26, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:33:54.805 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:33:54.812 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:33:54.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.922+0000 7f64ae59a700 1 Processor -- start 2024-05-08T22:33:54.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.922+0000 7f64ae59a700 1 -- start start 2024-05-08T22:33:54.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f64a80b21e0 0x7f64a815d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64a80b5050 0x7f64a80b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a80b6070 con 0x7f64a815ce90 2024-05-08T22:33:54.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a80b5a00 con 0x7f64a80b5050 2024-05-08T22:33:54.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a806abb0 con 0x7f64a80b21e0 2024-05-08T22:33:54.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64a80b5050 0x7f64a80b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64a80b5050 0x7f64a80b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40014/0 (socket says 172.21.15.179:40014) 2024-05-08T22:33:54.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 -- 172.21.15.179:0/2528980692 learned_addr learned my addr 172.21.15.179:0/2528980692 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:54.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64a77fe700 1 --2- 172.21.15.179:0/2528980692 >> v2:172.21.15.195:3300/0 conn(0x7f64a80b21e0 0x7f64a815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 -- 172.21.15.179:0/2528980692 >> v2:172.21.15.195:3300/0 conn(0x7f64a80b21e0 msgr2=0x7f64a815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:54.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 --2- 172.21.15.179:0/2528980692 >> v2:172.21.15.195:3300/0 conn(0x7f64a80b21e0 0x7f64a815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:54.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 -- 172.21.15.179:0/2528980692 >> v1:172.21.15.195:6789/0 conn(0x7f64a815ce90 legacy=0x7f64a80b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:54.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 -- 172.21.15.179:0/2528980692 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a806ae90 con 0x7f64a80b5050 2024-05-08T22:33:54.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64acb37700 1 --2- 172.21.15.179:0/2528980692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80b54c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64a80cf190 tx=0x7f64a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5e04c983a1319b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64a6ffd700 1 -- 172.21.15.179:0/2528980692 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000e070 con 0x7f64a80b5050 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64a6ffd700 1 -- 172.21.15.179:0/2528980692 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a000b910 con 0x7f64a80b5050 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64a6ffd700 1 -- 172.21.15.179:0/2528980692 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000bcc0 con 0x7f64a80b5050 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- 172.21.15.179:0/2528980692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 msgr2=0x7f64a80b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2528980692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80b54c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64a80cf190 tx=0x7f64a000b330 comp rx=0 tx=0).stop 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- 172.21.15.179:0/2528980692 shutdown_connections 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2528980692 >> v2:172.21.15.195:3300/0 conn(0x7f64a80b21e0 0x7f64a815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2528980692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80b54c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:54.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- 172.21.15.179:0/2528980692 >> 172.21.15.179:0/2528980692 conn(0x7f64a80d3400 msgr2=0x7f64a80d3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:54.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- 172.21.15.179:0/2528980692 shutdown_connections 2024-05-08T22:33:54.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.926+0000 7f64ae59a700 1 -- 172.21.15.179:0/2528980692 wait complete. 2024-05-08T22:33:54.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 Processor -- start 2024-05-08T22:33:54.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- start start 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a80b21e0 0x7f64a81536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64a815ce90 0x7f64a80c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a80c73a0 con 0x7f64a80b5050 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a80c7680 con 0x7f64a80b21e0 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a80c7960 con 0x7f64a815ce90 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40028/0 (socket says 172.21.15.179:40028) 2024-05-08T22:33:54.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 -- 172.21.15.179:0/2726441498 learned_addr learned my addr 172.21.15.179:0/2726441498 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a7fff700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a80b21e0 0x7f64a81536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64acb37700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64a815ce90 0x7f64a80c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 -- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64a815ce90 msgr2=0x7f64a80c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64a815ce90 0x7f64a80c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 -- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a80b21e0 msgr2=0x7f64a81536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a80b21e0 0x7f64a81536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 -- 172.21.15.179:0/2726441498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a0009b30 con 0x7f64a80b5050 2024-05-08T22:33:54.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a77fe700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649c009fd0 tx=0x7f649c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0095a0 con 0x7f64a80b5050 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c013680 con 0x7f64a80b5050 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a80c7c40 con 0x7f64a80b5050 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c013a10 con 0x7f64a80b5050 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.930+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a80c8980 con 0x7f64a80b5050 2024-05-08T22:33:54.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a808a000 con 0x7f64a80b5050 2024-05-08T22:33:54.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f649c013b70 con 0x7f64a80b5050 2024-05-08T22:33:54.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64a4ff9700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 0x7f6494068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:54.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f649c0841f0 con 0x7f64a80b5050 2024-05-08T22:33:54.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64a7fff700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 0x7f6494068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:54.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.934+0000 7f64a7fff700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 0x7f6494068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6498000f60 tx=0x7f6498009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:54.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:54.938+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f649c051300 con 0x7f64a80b5050 2024-05-08T22:33:55.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.154+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64a804e7e0 con 0x7f64a80b5050 2024-05-08T22:33:55.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.154+0000 7f64a4ff9700 1 -- 172.21.15.179:0/2726441498 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f649c054920 con 0x7f64a80b5050 2024-05-08T22:33:55.155 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 msgr2=0x7f6494068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 0x7f6494068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6498000f60 tx=0x7f6498009040 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 msgr2=0x7f64a80c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649c009fd0 tx=0x7f649c00eb20 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 shutdown_connections 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6494066730 0x7f6494068bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64a815ce90 0x7f64a80c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a80b5050 0x7f64a80c8440 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 --2- 172.21.15.179:0/2726441498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a80b21e0 0x7f64a81536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:55.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 >> 172.21.15.179:0/2726441498 conn(0x7f64a80d3400 msgr2=0x7f64a80b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:55.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 shutdown_connections 2024-05-08T22:33:55.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:55.158+0000 7f64ae59a700 1 -- 172.21.15.179:0/2726441498 wait complete. 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:33:55.176 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T22:33:55.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.177 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2024-05-08T22:33:55.185 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42543sec preferred_lft 42543sec 2024-05-08T22:33:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run.smithi195.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 02:42:cf:f5:06:76 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:55.187 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:33:55.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.187 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:55.187 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:55.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:33:55.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr flush dev ceph-brx 2024-05-08T22:33:55.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set ceph-brx up 2024-05-08T22:33:55.188 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:33:55.188 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:55.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.231 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:33:55.241 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-05-08T22:33:55.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.242 DEBUG:teuthology.orchestra.run.smithi195:> route 2024-05-08T22:33:55.330 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:33:55.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.331 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:55.331 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:55.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:33:55.332 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:33:55.332 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:33:55.332 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:55.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.374 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-05-08T22:33:55.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.381 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2024-05-08T22:33:55.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.431 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:55.431 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:55.431 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T22:33:55.431 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T22:33:55.431 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:55.461 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T22:33:55.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T22:33:55.461 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:55.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.541 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:55.541 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:55.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.0 up 2024-05-08T22:33:55.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:33:55.541 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:55.567 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-08T22:33:55.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:55.568 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:55.619 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T22:33:55.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:55.619 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:55.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:55.639 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:55.639 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:55.698 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:55.698 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:55.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.699 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:55.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:33:55.705 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-08T22:33:55.707 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:55.723 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:55.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:55.778 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:55.778 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:55.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:55.779 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:55.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:2024-05-08T22:33:55.789+0000 7fc0001a5640 -1 init, newargv = 0x563b07ca2ae0 newargc=15 2024-05-08T22:33:55.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:ceph-fuse[30991]: starting ceph client 2024-05-08T22:33:55.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:ceph-fuse[30991]: starting fuse 2024-05-08T22:33:56.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:56.904 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:56.904 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:56.966 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:56.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:56.967 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:56.972 INFO:teuthology.orchestra.run.smithi195.stdout:52 2024-05-08T22:33:56.973 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> import glob 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> import re 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> import os 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> import subprocess 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> def _find_admin_socket(client_name): 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> files = glob.glob(asok_path) 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi195:> # Given a non-glob path, it better be there 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> if "*" not in asok_path: 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> assert(len(files) == 1) 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> return files[0] 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> for f in files: 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> contents = proc_f.read() 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> if mountpoint in contents: 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> return f 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> print(_find_admin_socket("client.1")) 2024-05-08T22:33:56.974 DEBUG:teuthology.orchestra.run.smithi195:> ' 2024-05-08T22:33:57.056 INFO:teuthology.orchestra.run.smithi195.stdout:/var/run/ceph/ceph-client.1.30991.asok 2024-05-08T22:33:57.239 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.30991.asok 2024-05-08T22:33:57.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.239 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.30991.asok status 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout: "metadata": { 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout: "entity_id": "1", 2024-05-08T22:33:57.383 INFO:teuthology.orchestra.run.smithi195.stdout: "hostname": "smithi195", 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "pid": "30991", 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "root": "/" 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_count": 0, 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_pinned_count": 0, 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "id": 54608, 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "inst": { 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "name": { 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "client", 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "num": 54608 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2024-05-08T22:33:57.384 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1944666760 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1944666760 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "inst_str": "client.54608 192.168.0.1:0/1944666760", 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "addr_str": "192.168.0.1:0/1944666760", 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "inode_count": 1, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_epoch": 9, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch": 26, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "blocklisted": false, 2024-05-08T22:33:57.385 INFO:teuthology.orchestra.run.smithi195.stdout: "fs_name": "cephfs" 2024-05-08T22:33:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-08T22:33:57.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:33:57.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 Processor -- start 2024-05-08T22:33:57.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 -- start start 2024-05-08T22:33:57.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b00d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70b0160f30 0x7f70b01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00d2640 con 0x7f70b015d580 2024-05-08T22:33:57.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b0161900 con 0x7f70b0160f30 2024-05-08T22:33:57.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.510+0000 7f70b70db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b006b3a0 con 0x7f70b00d18a0 2024-05-08T22:33:57.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70b0160f30 0x7f70b01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b58d8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b00d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70b0160f30 0x7f70b01613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:41632/0 (socket says 172.21.15.179:41632) 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 -- 172.21.15.179:0/4195637939 learned_addr learned my addr 172.21.15.179:0/4195637939 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 -- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 msgr2=0x7f70b00d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 --2- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b00d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 -- 172.21.15.179:0/4195637939 >> v1:172.21.15.195:6789/0 conn(0x7f70b015d580 legacy=0x7f70b00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 -- 172.21.15.179:0/4195637939 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b00d2360 con 0x7f70b0160f30 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b68da700 1 --2- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b01613c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009e50 tx=0x7f70a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dbff09c519dce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b50d7700 1 -- 172.21.15.179:0/4195637939 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a800e070 con 0x7f70b0160f30 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b50d7700 1 -- 172.21.15.179:0/4195637939 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a800b910 con 0x7f70b0160f30 2024-05-08T22:33:57.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b50d7700 1 -- 172.21.15.179:0/4195637939 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a800bcc0 con 0x7f70b0160f30 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 -- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 msgr2=0x7f70b01613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 --2- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b01613c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009e50 tx=0x7f70a800b330 comp rx=0 tx=0).stop 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 -- 172.21.15.179:0/4195637939 shutdown_connections 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 --2- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b01613c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 --2- 172.21.15.179:0/4195637939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b00d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 -- 172.21.15.179:0/4195637939 >> 172.21.15.179:0/4195637939 conn(0x7f70b00d2d90 msgr2=0x7f70b00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 -- 172.21.15.179:0/4195637939 shutdown_connections 2024-05-08T22:33:57.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.514+0000 7f70b70db700 1 -- 172.21.15.179:0/4195637939 wait complete. 2024-05-08T22:33:57.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 Processor -- start 2024-05-08T22:33:57.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- start start 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 0x7f70b00b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b0160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00b52a0 con 0x7f70b015d580 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b015fb50 con 0x7f70b0160f30 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00b4200 con 0x7f70b00d18a0 2024-05-08T22:33:57.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b58d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 0x7f70b00b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b58d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 0x7f70b00b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38004/0 (socket says 172.21.15.179:38004) 2024-05-08T22:33:57.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b58d8700 1 -- 172.21.15.179:0/3046080076 learned_addr learned my addr 172.21.15.179:0/3046080076 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:33:57.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b0160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b60d9700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 -- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 msgr2=0x7f70b015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 -- 172.21.15.179:0/3046080076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 msgr2=0x7f70b00b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 0x7f70b00b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 -- 172.21.15.179:0/3046080076 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009b30 con 0x7f70b0160f30 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b68da700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b0160290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f70a8015000 tx=0x7f70a8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:57.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8004840 con 0x7f70b0160f30 2024-05-08T22:33:57.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a80099b0 con 0x7f70b0160f30 2024-05-08T22:33:57.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a801c680 con 0x7f70b0160f30 2024-05-08T22:33:57.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b00b44e0 con 0x7f70b0160f30 2024-05-08T22:33:57.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.518+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00b4aa0 con 0x7f70b0160f30 2024-05-08T22:33:57.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b004e700 con 0x7f70b0160f30 2024-05-08T22:33:57.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70a801c7e0 con 0x7f70b0160f30 2024-05-08T22:33:57.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70a6ffd700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 0x7f7094068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f70a8085c60 con 0x7f70b0160f30 2024-05-08T22:33:57.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70b60d9700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 0x7f7094068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.522+0000 7f70b60d9700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 0x7f7094068c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000f60 tx=0x7f70a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:57.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.534+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f70a8052e70 con 0x7f70b0160f30 2024-05-08T22:33:57.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.746+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70b00cc040 con 0x7f70b0160f30 2024-05-08T22:33:57.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.746+0000 7f70a6ffd700 1 -- 172.21.15.179:0/3046080076 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f70a8056490 con 0x7f70b0160f30 2024-05-08T22:33:57.747 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 msgr2=0x7f7094068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 0x7f7094068c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000f60 tx=0x7f70a0009040 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 msgr2=0x7f70b0160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b0160290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f70a8015000 tx=0x7f70a8004af0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 shutdown_connections 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7094066760 0x7f7094068c20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70b0160f30 0x7f70b0160290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015d580 0x7f70b00b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 --2- 172.21.15.179:0/3046080076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70b00d18a0 0x7f70b015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 >> 172.21.15.179:0/3046080076 conn(0x7f70b00d2d90 msgr2=0x7f70b00b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 shutdown_connections 2024-05-08T22:33:57.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:33:57.750+0000 7f70b70db700 1 -- 172.21.15.179:0/3046080076 wait complete. 2024-05-08T22:33:57.767 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:33:57.767 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:33:57.767 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2024-05-08T22:33:57.767 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T22:33:57.767 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T22:33:57.768 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:33:57.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:33:57.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:33:57.768 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T22:33:57.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.768 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2024-05-08T22:33:57.776 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.777 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42540sec preferred_lft 42540sec 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 02:42:cf:f5:06:76 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether da:65:c7:92:68:e4 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.778 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::4086:28ff:fe48:184c/64 scope link 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run.smithi195.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether da:65:c7:92:68:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::d865:c7ff:fe92:68e4/64 scope link 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.779 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:33:57.794 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-05-08T22:33:57.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.795 DEBUG:teuthology.orchestra.run.smithi195:> route 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:33:57.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:33:57.850 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:57.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.890 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-05-08T22:33:57.896 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T22:33:57.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.896 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2024-05-08T22:33:57.902 INFO:teuthology.orchestra.run.smithi195.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T22:33:57.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.903 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:57.903 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:57.903 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T22:33:57.903 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T22:33:57.903 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:57.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.976 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.987 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:57.988 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 96:b8:86:b0:25:5c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T22:33:57.988 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T22:33:57.988 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.988 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::94b8:86ff:feb0:255c/64 scope link 2024-05-08T22:33:57.988 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:57.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:57.989 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:33:58.003 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 96:b8:86:b0:25:5c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T22:33:58.004 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T22:33:58.004 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:58.004 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::94b8:86ff:feb0:255c/64 scope link 2024-05-08T22:33:58.004 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:33:58.005 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T22:33:58.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:58.005 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:58.005 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T22:33:58.006 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:58.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:58.060 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2024-05-08T22:33:58.060 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2024-05-08T22:33:58.060 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.1 up 2024-05-08T22:33:58.060 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T22:33:58.060 DEBUG:teuthology.orchestra.run.smithi195:> ') 2024-05-08T22:33:58.087 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-08T22:33:58.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:58.087 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T22:33:58.142 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T22:33:58.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:33:58.142 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T22:33:58.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:58.163 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:58.163 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:58.174 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:58.174 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:58.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:58.175 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:58.222 INFO:teuthology.orchestra.run.smithi195.stdout:52 2024-05-08T22:33:58.223 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-05-08T22:33:58.223 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-08T22:33:58.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:58.241 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:58.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:58.253 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:58.254 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:58.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:58.255 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:58.261 INFO:teuthology.orchestra.run.smithi195.stdout:52 2024-05-08T22:33:58.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:2024-05-08T22:33:58.309+0000 7fad08b4f640 -1 init, newargv = 0x559ace786a50 newargc=15 2024-05-08T22:33:58.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:ceph-fuse[31108]: starting ceph client 2024-05-08T22:33:58.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:ceph-fuse[31108]: starting fuse 2024-05-08T22:33:59.362 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2024-05-08T22:33:59.381 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:33:59.381 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:33:59.440 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:33:59.440 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:33:59.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.441 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2024-05-08T22:33:59.448 INFO:teuthology.orchestra.run.smithi195.stdout:52 2024-05-08T22:33:59.448 INFO:teuthology.orchestra.run.smithi195.stdout:53 2024-05-08T22:33:59.448 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> import glob 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> import re 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> import os 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> import subprocess 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> def _find_admin_socket(client_name): 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> files = glob.glob(asok_path) 2024-05-08T22:33:59.449 DEBUG:teuthology.orchestra.run.smithi195:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> # Given a non-glob path, it better be there 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> if "*" not in asok_path: 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> assert(len(files) == 1) 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> return files[0] 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> for f in files: 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> contents = proc_f.read() 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> if mountpoint in contents: 2024-05-08T22:33:59.450 DEBUG:teuthology.orchestra.run.smithi195:> return f 2024-05-08T22:33:59.451 DEBUG:teuthology.orchestra.run.smithi195:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:33:59.451 DEBUG:teuthology.orchestra.run.smithi195:> 2024-05-08T22:33:59.451 DEBUG:teuthology.orchestra.run.smithi195:> print(_find_admin_socket("client.3")) 2024-05-08T22:33:59.451 DEBUG:teuthology.orchestra.run.smithi195:> ' 2024-05-08T22:33:59.531 INFO:teuthology.orchestra.run.smithi195.stdout:/var/run/ceph/ceph-client.3.31108.asok 2024-05-08T22:33:59.716 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.31108.asok 2024-05-08T22:33:59.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.716 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.31108.asok status 2024-05-08T22:33:59.866 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-08T22:33:59.866 INFO:teuthology.orchestra.run.smithi195.stdout: "metadata": { 2024-05-08T22:33:59.866 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "entity_id": "3", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "hostname": "smithi195", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "pid": "31108", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "root": "/" 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_count": 0, 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_pinned_count": 0, 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "id": 4906, 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inst": { 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "name": { 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "client", 2024-05-08T22:33:59.867 INFO:teuthology.orchestra.run.smithi195.stdout: "num": 4906 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.2:0", 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1681970988 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.2:0", 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1681970988 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "inst_str": "client.4906 192.168.0.2:0/1681970988", 2024-05-08T22:33:59.868 INFO:teuthology.orchestra.run.smithi195.stdout: "addr_str": "192.168.0.2:0/1681970988", 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "inode_count": 1, 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_epoch": 9, 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch": 26, 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "blocklisted": false, 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout: "fs_name": "cephfs" 2024-05-08T22:33:59.869 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-08T22:33:59.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.876 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:33:59.876 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:59.884 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:33:59.885 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:59.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:33:59.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.899 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:33:59.900 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:59.906 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:33:59.906 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:59.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.907 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T22:33:59.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.969 DEBUG:teuthology.orchestra.run.smithi195:> stat --file-system '--printf=%T 2024-05-08T22:33:59.969 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:59.978 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2024-05-08T22:33:59.978 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-08T22:33:59.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:33:59.978 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T22:34:00.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:00.036 DEBUG:teuthology.orchestra.run.smithi195:> stat --file-system '--printf=%T 2024-05-08T22:34:00.036 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-08T22:34:00.042 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2024-05-08T22:34:00.042 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-08T22:34:00.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:00.042 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T22:34:00.059 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T22:34:00.076 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:00.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.182+0000 7f251f59e700 1 Processor -- start 2024-05-08T22:34:00.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.182+0000 7f251f59e700 1 -- start start 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25200ce2d0 0x7f252006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252006afc0 con 0x7f252015d5b0 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252015d990 con 0x7f25200cde70 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25200d26f0 con 0x7f25200ce2d0 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251e59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f252015d5b0 0x7f25200b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58656/0 (socket says 172.21.15.179:58656) 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251e59c700 1 -- 172.21.15.179:0/4144941558 learned_addr learned my addr 172.21.15.179:0/4144941558 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:00.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251ed9d700 1 --2- 172.21.15.179:0/4144941558 >> v2:172.21.15.195:3300/0 conn(0x7f25200ce2d0 0x7f252006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251d59a700 1 -- 172.21.15.179:0/4144941558 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999609851 0 0) 0x7f252006afc0 con 0x7f252015d5b0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251d59a700 1 -- 172.21.15.179:0/4144941558 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f250c0035d0 con 0x7f252015d5b0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251ed9d700 1 -- 172.21.15.179:0/4144941558 >> v1:172.21.15.195:6789/0 conn(0x7f25200cde70 legacy=0x7f252006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251ed9d700 1 -- 172.21.15.179:0/4144941558 >> v1:172.21.15.179:6789/0 conn(0x7f252015d5b0 legacy=0x7f25200b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251ed9d700 1 -- 172.21.15.179:0/4144941558 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200d2130 con 0x7f25200ce2d0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251ed9d700 1 --2- 172.21.15.179:0/4144941558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200ce2d0 0x7f252006bb60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2518009d90 tx=0x7f251800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569bb87cb86caa92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251d59a700 1 -- 172.21.15.179:0/4144941558 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251800e070 con 0x7f25200ce2d0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251d59a700 1 -- 172.21.15.179:0/4144941558 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518003a40 con 0x7f25200ce2d0 2024-05-08T22:34:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251d59a700 1 -- 172.21.15.179:0/4144941558 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510001670 con 0x7f25200ce2d0 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- 172.21.15.179:0/4144941558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200ce2d0 msgr2=0x7f252006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 --2- 172.21.15.179:0/4144941558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200ce2d0 0x7f252006bb60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2518009d90 tx=0x7f251800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- 172.21.15.179:0/4144941558 shutdown_connections 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 --2- 172.21.15.179:0/4144941558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200ce2d0 0x7f252006bb60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- 172.21.15.179:0/4144941558 >> 172.21.15.179:0/4144941558 conn(0x7f25200b4940 msgr2=0x7f25200b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- 172.21.15.179:0/4144941558 shutdown_connections 2024-05-08T22:34:00.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.186+0000 7f251f59e700 1 -- 172.21.15.179:0/4144941558 wait complete. 2024-05-08T22:34:00.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 Processor -- start 2024-05-08T22:34:00.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- start start 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200cde70 0x7f25201524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f252015d5b0 0x7f252015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520152a20 con 0x7f25200ce2d0 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520153e60 con 0x7f25200cde70 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25201547e0 con 0x7f252015d5b0 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38006/0 (socket says 172.21.15.179:38006) 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 -- 172.21.15.179:0/788836095 learned_addr learned my addr 172.21.15.179:0/788836095 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251e59c700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200cde70 0x7f25201524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251ed9d700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f252015d5b0 0x7f252015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 -- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f252015d5b0 msgr2=0x7f252015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f252015d5b0 0x7f252015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 -- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200cde70 msgr2=0x7f25201524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200cde70 0x7f25201524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 -- 172.21.15.179:0/788836095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518009a70 con 0x7f25200ce2d0 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251dd9b700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f250800c440 tx=0x7f250800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080052d0 con 0x7f25200ce2d0 2024-05-08T22:34:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508005430 con 0x7f25200ce2d0 2024-05-08T22:34:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252015eb30 con 0x7f25200ce2d0 2024-05-08T22:34:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080055a0 con 0x7f25200ce2d0 2024-05-08T22:34:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.190+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252015f0f0 con 0x7f25200ce2d0 2024-05-08T22:34:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25201549b0 con 0x7f25200ce2d0 2024-05-08T22:34:00.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25080036a0 con 0x7f25200ce2d0 2024-05-08T22:34:00.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f2516ffd700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 0x7f2500068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2508083560 con 0x7f25200ce2d0 2024-05-08T22:34:00.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f251e59c700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 0x7f2500068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.194+0000 7f251e59c700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 0x7f2500068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2510001b70 tx=0x7f2510001b00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.198+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f25201549b0 con 0x7f25200ce2d0 2024-05-08T22:34:00.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.366+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25201549b0 con 0x7f25200ce2d0 2024-05-08T22:34:00.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f2516ffd700 1 -- 172.21.15.179:0/788836095 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f25201549b0 con 0x7f25200ce2d0 2024-05-08T22:34:00.370 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:00.370 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:00.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 msgr2=0x7f2500068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 0x7f2500068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2510001b70 tx=0x7f2510001b00 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 msgr2=0x7f252015aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f250800c440 tx=0x7f250800c750 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 shutdown_connections 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2500066730 0x7f2500068bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f252015d5b0 0x7f252015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25200ce2d0 0x7f252015aeb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 --2- 172.21.15.179:0/788836095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25200cde70 0x7f25201524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 >> 172.21.15.179:0/788836095 conn(0x7f25200b4940 msgr2=0x7f252006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 shutdown_connections 2024-05-08T22:34:00.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.370+0000 7f251f59e700 1 -- 172.21.15.179:0/788836095 wait complete. 2024-05-08T22:34:00.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:00.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:00.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 Processor -- start 2024-05-08T22:34:00.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 -- start start 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796c15d0c0 0x7f796c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 0x7f796c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 0x7f796c160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0ce570 con 0x7f796c15d0c0 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c15d4a0 con 0x7f796c0ce0e0 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972b15700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c06abb0 con 0x7f796c0d1d20 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796c15d0c0 0x7f796c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796c15d0c0 0x7f796c0b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38012/0 (socket says 172.21.15.179:38012) 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 -- 172.21.15.179:0/1787349866 learned_addr learned my addr 172.21.15.179:0/1787349866 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972314700 1 --2- 172.21.15.179:0/1787349866 >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 0x7f796c160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 -- 172.21.15.179:0/1787349866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 msgr2=0x7f796c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971312700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 0x7f796c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 0x7f796c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 -- 172.21.15.179:0/1787349866 >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 msgr2=0x7f796c160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 --2- 172.21.15.179:0/1787349866 >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 0x7f796c160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 -- 172.21.15.179:0/1787349866 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0d2500 con 0x7f796c15d0c0 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7972314700 1 --2- 172.21.15.179:0/1787349866 >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 0x7f796c160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971312700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 0x7f796c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7971b13700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c15d0c0 0x7f796c0b8a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f796c161c40 tx=0x7f796800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1603fa36e630d122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7970b11700 1 -- 172.21.15.179:0/1787349866 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800e070 con 0x7f796c15d0c0 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7970b11700 1 -- 172.21.15.179:0/1787349866 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800b9c0 con 0x7f796c15d0c0 2024-05-08T22:34:00.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.502+0000 7f7970b11700 1 -- 172.21.15.179:0/1787349866 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800bd70 con 0x7f796c15d0c0 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- 172.21.15.179:0/1787349866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c15d0c0 msgr2=0x7f796c0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c15d0c0 0x7f796c0b8a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f796c161c40 tx=0x7f796800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- 172.21.15.179:0/1787349866 shutdown_connections 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- 172.21.15.179:0/1787349866 >> v2:172.21.15.195:3300/0 conn(0x7f796c0ce0e0 0x7f796c160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0d1d20 0x7f796c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- 172.21.15.179:0/1787349866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c15d0c0 0x7f796c0b8a80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- 172.21.15.179:0/1787349866 >> 172.21.15.179:0/1787349866 conn(0x7f796c0d2ed0 msgr2=0x7f796c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- 172.21.15.179:0/1787349866 shutdown_connections 2024-05-08T22:34:00.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- 172.21.15.179:0/1787349866 wait complete. 2024-05-08T22:34:00.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 Processor -- start 2024-05-08T22:34:00.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- start start 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0ce0e0 0x7f796c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 0x7f796c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c160900 con 0x7f796c0d1d20 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c150650 con 0x7f796c15d6a0 2024-05-08T22:34:00.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972b15700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c160340 con 0x7f796c0ce0e0 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971312700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971312700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38020/0 (socket says 172.21.15.179:38020) 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971312700 1 -- 172.21.15.179:0/3904396378 learned_addr learned my addr 172.21.15.179:0/3904396378 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7972314700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 0x7f796c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971b13700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0ce0e0 0x7f796c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971312700 1 -- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0ce0e0 msgr2=0x7f796c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.506+0000 7f7971312700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0ce0e0 0x7f796c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7971312700 1 -- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 msgr2=0x7f796c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7971312700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 0x7f796c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7971312700 1 -- 172.21.15.179:0/3904396378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009b30 con 0x7f796c0d1d20 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7972314700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 0x7f796c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:00.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7971312700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f796000c770 tx=0x7f796000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011800 con 0x7f796c0d1d20 2024-05-08T22:34:00.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960011960 con 0x7f796c0d1d20 2024-05-08T22:34:00.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c160620 con 0x7f796c0d1d20 2024-05-08T22:34:00.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011d10 con 0x7f796c0d1d20 2024-05-08T22:34:00.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c15c1a0 con 0x7f796c0d1d20 2024-05-08T22:34:00.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c04e700 con 0x7f796c0d1d20 2024-05-08T22:34:00.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7960018360 con 0x7f796c0d1d20 2024-05-08T22:34:00.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 0x7f7950068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7960084430 con 0x7f796c0d1d20 2024-05-08T22:34:00.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.510+0000 7f7971b13700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 0x7f7950068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.514+0000 7f7971b13700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 0x7f7950068c20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f796c0d3df0 tx=0x7f7968009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.518+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7960051540 con 0x7f796c0d1d20 2024-05-08T22:34:00.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.682+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f796c0676e0 con 0x7f796c0d1d20 2024-05-08T22:34:00.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.682+0000 7f795e7fc700 1 -- 172.21.15.179:0/3904396378 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7960054b60 con 0x7f796c0d1d20 2024-05-08T22:34:00.684 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:00.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 msgr2=0x7f7950068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 0x7f7950068c20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f796c0d3df0 tx=0x7f7968009000 comp rx=0 tx=0).stop 2024-05-08T22:34:00.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 msgr2=0x7f796c15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f796000c770 tx=0x7f796000cb30 comp rx=0 tx=0).stop 2024-05-08T22:34:00.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 shutdown_connections 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7950066760 0x7f7950068c20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f796c15d6a0 0x7f796c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c0d1d20 0x7f796c15bc60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 --2- 172.21.15.179:0/3904396378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f796c0ce0e0 0x7f796c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 >> 172.21.15.179:0/3904396378 conn(0x7f796c0d2ed0 msgr2=0x7f796c0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 shutdown_connections 2024-05-08T22:34:00.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.686+0000 7f7972b15700 1 -- 172.21.15.179:0/3904396378 wait complete. 2024-05-08T22:34:00.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.818+0000 7ff111461700 1 Processor -- start 2024-05-08T22:34:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.818+0000 7ff111461700 1 -- start start 2024-05-08T22:34:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff10c15b200 0x7ff10c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10c15b5e0 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10c0b1cf0 con 0x7ff10c0ce3a0 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10c0b2590 con 0x7ff10c15b200 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff10c0ae4b0 0x7ff10c164d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58672/0 (socket says 172.21.15.179:58672) 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10b7fe700 1 -- 172.21.15.179:0/44390257 learned_addr learned my addr 172.21.15.179:0/44390257 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 --2- 172.21.15.179:0/44390257 >> v2:172.21.15.195:3300/0 conn(0x7ff10c15b200 0x7ff10c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127374583 0 0) 0x7ff10c15b5e0 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec0035d0 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4058110211 0 0) 0x7ff0ec0035d0 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 >> v2:172.21.15.195:3300/0 conn(0x7ff10c15b200 msgr2=0x7ff10c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 --2- 172.21.15.179:0/44390257 >> v2:172.21.15.195:3300/0 conn(0x7ff10c15b200 0x7ff10c0cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 >> v1:172.21.15.195:6789/0 conn(0x7ff10c0ce3a0 legacy=0x7ff10c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c0ce880 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff1040019f0 con 0x7ff10c0ae4b0 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 >> v1:172.21.15.179:6789/0 conn(0x7ff10c0ae4b0 legacy=0x7ff10c164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 0x7ff0ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec004870 0x7ff0ec004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec005230 0x7ff0ec00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c0b2590 con 0x7ff0ec003a20 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00c0d0 con 0x7ff0ec004870 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00c290 con 0x7ff0ec005230 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 0x7ff0ec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10b7fe700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec005230 0x7ff0ec00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 -- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec005230 msgr2=0x7ff0ec00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10a7fc700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec004870 0x7ff0ec004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec005230 0x7ff0ec00ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 -- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec004870 msgr2=0x7ff0ec004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec004870 0x7ff0ec004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10b7fe700 1 -- 172.21.15.179:0/44390257 reap_dead start 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 -- 172.21.15.179:0/44390257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0035d0 con 0x7ff0ec003a20 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10a7fc700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ec004870 0x7ff0ec004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff10affd700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 0x7ff0ec003e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009b70 tx=0x7ff0ec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f261984d01293804 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00e6c0 con 0x7ff0ec003a20 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00e820 con 0x7ff0ec003a20 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff109ffb700 1 -- 172.21.15.179:0/44390257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00ebb0 con 0x7ff0ec003a20 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 msgr2=0x7ff0ec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 0x7ff0ec003e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009b70 tx=0x7ff0ec0058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- 172.21.15.179:0/44390257 shutdown_connections 2024-05-08T22:34:00.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 --2- 172.21.15.179:0/44390257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0ec003a20 0x7ff0ec003e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- 172.21.15.179:0/44390257 >> 172.21.15.179:0/44390257 conn(0x7ff10c0ceca0 msgr2=0x7ff10c0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:00.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- 172.21.15.179:0/44390257 shutdown_connections 2024-05-08T22:34:00.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.822+0000 7ff111461700 1 -- 172.21.15.179:0/44390257 wait complete. 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 Processor -- start 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 -- start start 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec003a20 0x7ff10c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff10c0af1c0 0x7ff10c0af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c0afbb0 con 0x7ff10c09c630 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c157e10 con 0x7ff10c0af1c0 2024-05-08T22:34:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff111461700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c099bd0 con 0x7ff0ec003a20 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff10a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff10a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38044/0 (socket says 172.21.15.179:38044) 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff10a7fc700 1 -- 172.21.15.179:0/1427601436 learned_addr learned my addr 172.21.15.179:0/1427601436 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff10affd700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec003a20 0x7ff10c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.826+0000 7ff10b7fe700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff10c0af1c0 0x7ff10c0af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 -- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec003a20 msgr2=0x7ff10c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec003a20 0x7ff10c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 -- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff10c0af1c0 msgr2=0x7ff10c0af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff10c0af1c0 0x7ff10c0af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 -- 172.21.15.179:0/1427601436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c099eb0 con 0x7ff10c09c630 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff10a7fc700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d540 tx=0x7ff0f400d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400b070 con 0x7ff10c09c630 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4005540 con 0x7ff10c09c630 2024-05-08T22:34:00.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c09a190 con 0x7ff10c09c630 2024-05-08T22:34:00.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4006ac0 con 0x7ff10c09c630 2024-05-08T22:34:00.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c09a750 con 0x7ff10c09c630 2024-05-08T22:34:00.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c09a8e0 con 0x7ff10c09c630 2024-05-08T22:34:00.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.830+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0f4006c20 con 0x7ff10c09c630 2024-05-08T22:34:00.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.834+0000 7ff1037fe700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 0x7ff0f0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:00.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.834+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0f4084620 con 0x7ff10c09c630 2024-05-08T22:34:00.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.834+0000 7ff10affd700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 0x7ff0f0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:00.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.834+0000 7ff10affd700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 0x7ff0f0068c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0090b0 tx=0x7ff0fc01e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:00.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:00.838+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff0f4051780 con 0x7ff10c09c630 2024-05-08T22:34:01.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.006+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff10c073290 con 0x7ff10c09c630 2024-05-08T22:34:01.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.006+0000 7ff1037fe700 1 -- 172.21.15.179:0/1427601436 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0f4054da0 con 0x7ff10c09c630 2024-05-08T22:34:01.009 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 msgr2=0x7ff0f0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 0x7ff0f0068c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0090b0 tx=0x7ff0fc01e040 comp rx=0 tx=0).stop 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 msgr2=0x7ff10c0c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d540 tx=0x7ff0f400d850 comp rx=0 tx=0).stop 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 shutdown_connections 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff0f0066760 0x7ff0f0068c20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff10c0af1c0 0x7ff10c0af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff10c09c630 0x7ff10c0c3c60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 --2- 172.21.15.179:0/1427601436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0ec003a20 0x7ff10c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 >> 172.21.15.179:0/1427601436 conn(0x7ff10c0ceca0 msgr2=0x7ff10c0ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 shutdown_connections 2024-05-08T22:34:01.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.010+0000 7ff111461700 1 -- 172.21.15.179:0/1427601436 wait complete. 2024-05-08T22:34:01.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:01.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 Processor -- start 2024-05-08T22:34:01.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 -- start start 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc15f4c0 con 0x7f4bfc15f020 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc15f7a0 con 0x7f4bfc0aea70 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4c0083e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0b4610 con 0x7f4bfc0ae690 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4bfa59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4bfc15f020 0x7f4bfc06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58676/0 (socket says 172.21.15.179:58676) 2024-05-08T22:34:01.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4bfa59c700 1 -- 172.21.15.179:0/460549127 learned_addr learned my addr 172.21.15.179:0/460549127 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4bfad9d700 1 --2- 172.21.15.179:0/460549127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870843908 0 0) 0x7f4bfc15f4c0 con 0x7f4bfc15f020 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.146+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be40035d0 con 0x7f4bfc15f020 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bfad9d700 1 -- 172.21.15.179:0/460549127 >> v1:172.21.15.195:6789/0 conn(0x7f4bfc0aea70 legacy=0x7f4bfc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562882305 0 0) 0x7f4bfc15f7a0 con 0x7f4bfc0aea70 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bfad9d700 1 -- 172.21.15.179:0/460549127 >> v1:172.21.15.179:6789/0 conn(0x7f4bfc15f020 legacy=0x7f4bfc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bfad9d700 1 -- 172.21.15.179:0/460549127 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0b4230 con 0x7f4bfc0ae690 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bfad9d700 1 --2- 172.21.15.179:0/460549127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc15eae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009880 tx=0x7f4bec009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5852958a4a667abb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0001670 con 0x7f4bfc0ae690 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec010590 con 0x7f4bfc0ae690 2024-05-08T22:34:01.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4bf9d9b700 1 -- 172.21.15.179:0/460549127 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec010940 con 0x7f4bfc0ae690 2024-05-08T22:34:01.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 -- 172.21.15.179:0/460549127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 msgr2=0x7f4bfc15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 --2- 172.21.15.179:0/460549127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc15eae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009880 tx=0x7f4bec009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:01.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 -- 172.21.15.179:0/460549127 shutdown_connections 2024-05-08T22:34:01.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 --2- 172.21.15.179:0/460549127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc15eae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 -- 172.21.15.179:0/460549127 >> 172.21.15.179:0/460549127 conn(0x7f4bfc098a70 msgr2=0x7f4bfc0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 -- 172.21.15.179:0/460549127 shutdown_connections 2024-05-08T22:34:01.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.150+0000 7f4c0083e700 1 -- 172.21.15.179:0/460549127 wait complete. 2024-05-08T22:34:01.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 Processor -- start 2024-05-08T22:34:01.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 -- start start 2024-05-08T22:34:01.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bfc0aea70 0x7f4bfc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0c6950 con 0x7f4bfc15f020 2024-05-08T22:34:01.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc159a90 con 0x7f4bfc0aea70 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4c0083e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15a330 con 0x7f4bfc0ae690 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4bfb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4bfb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38058/0 (socket says 172.21.15.179:38058) 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4bfb59e700 1 -- 172.21.15.179:0/2061250034 learned_addr learned my addr 172.21.15.179:0/2061250034 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4bfa59c700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bfc0aea70 0x7f4bfc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.154+0000 7f4bfad9d700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 -- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 msgr2=0x7f4bfc0c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc0c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 -- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bfc0aea70 msgr2=0x7f4bfc1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bfc0aea70 0x7f4bfc1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 -- 172.21.15.179:0/2061250034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0094b0 con 0x7f4bfc15f020 2024-05-08T22:34:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfb59e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400cba0 tx=0x7f4bf400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4003e50 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf400a690 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc157ff0 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400aa40 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc159ce0 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc15a4e0 con 0x7f4bfc15f020 2024-05-08T22:34:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4bf400aba0 con 0x7f4bfc15f020 2024-05-08T22:34:01.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 0x7f4be8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4bf4083d80 con 0x7f4bfc15f020 2024-05-08T22:34:01.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.158+0000 7f4bfad9d700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 0x7f4be8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.162+0000 7f4bfad9d700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 0x7f4be8068c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097d0 tx=0x7f4bec010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.166+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4bfc15a4e0 con 0x7f4bfc15f020 2024-05-08T22:34:01.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bfc15a4a0 con 0x7f4bfc15f020 2024-05-08T22:34:01.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4be37fe700 1 -- 172.21.15.179:0/2061250034 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4bf4050e90 con 0x7f4bfc15f020 2024-05-08T22:34:01.343 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:01.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 msgr2=0x7f4be8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 0x7f4be8068c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097d0 tx=0x7f4bec010000 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 msgr2=0x7f4bfc0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400cba0 tx=0x7f4bf400ceb0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 shutdown_connections 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4be8066760 0x7f4be8068c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bfc0aea70 0x7f4bfc1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bfc15f020 0x7f4bfc0c8810 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 --2- 172.21.15.179:0/2061250034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bfc0ae690 0x7f4bfc0c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 >> 172.21.15.179:0/2061250034 conn(0x7f4bfc098a70 msgr2=0x7f4bfc06bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.342+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 shutdown_connections 2024-05-08T22:34:01.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.346+0000 7f4c0083e700 1 -- 172.21.15.179:0/2061250034 wait complete. 2024-05-08T22:34:01.357 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:01.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 Processor -- start 2024-05-08T22:34:01.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 -- start start 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd97414ae30 0x7fd974165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd9740b2de0 0x7fd97406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd974166090 con 0x7fd9740b1a50 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97406e9d0 con 0x7fd9740b2de0 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd979094700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97406dd50 con 0x7fd97414ae30 2024-05-08T22:34:01.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd9740b2de0 0x7fd97406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd9737fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd9740b1a50 0x7fd9740b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58680/0 (socket says 172.21.15.179:58680) 2024-05-08T22:34:01.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd9737fe700 1 -- 172.21.15.179:0/2601580018 learned_addr learned my addr 172.21.15.179:0/2601580018 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:01.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd973fff700 1 --2- 172.21.15.179:0/2601580018 >> v2:172.21.15.195:3300/0 conn(0x7fd97414ae30 0x7fd974165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 -- 172.21.15.179:0/2601580018 >> v2:172.21.15.195:3300/0 conn(0x7fd97414ae30 msgr2=0x7fd974165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 --2- 172.21.15.179:0/2601580018 >> v2:172.21.15.195:3300/0 conn(0x7fd97414ae30 0x7fd974165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 -- 172.21.15.179:0/2601580018 >> v1:172.21.15.179:6789/0 conn(0x7fd9740b1a50 legacy=0x7fd9740b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 -- 172.21.15.179:0/2601580018 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97406e030 con 0x7fd9740b2de0 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.474+0000 7fd978893700 1 --2- 172.21.15.179:0/2601580018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b2de0 0x7fd97406abb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd974072220 tx=0x7fd964009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c70db2c78dfdbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd972ffd700 1 -- 172.21.15.179:0/2601580018 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964011830 con 0x7fd9740b2de0 2024-05-08T22:34:01.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd972ffd700 1 -- 172.21.15.179:0/2601580018 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964011990 con 0x7fd9740b2de0 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- 172.21.15.179:0/2601580018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b2de0 msgr2=0x7fd97406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 --2- 172.21.15.179:0/2601580018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b2de0 0x7fd97406abb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd974072220 tx=0x7fd964009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- 172.21.15.179:0/2601580018 shutdown_connections 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 --2- 172.21.15.179:0/2601580018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b2de0 0x7fd97406abb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 --2- 172.21.15.179:0/2601580018 >> v2:172.21.15.195:3300/0 conn(0x7fd97414ae30 0x7fd974165b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- 172.21.15.179:0/2601580018 >> 172.21.15.179:0/2601580018 conn(0x7fd9740cea30 msgr2=0x7fd9740cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- 172.21.15.179:0/2601580018 shutdown_connections 2024-05-08T22:34:01.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- 172.21.15.179:0/2601580018 wait complete. 2024-05-08T22:34:01.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 Processor -- start 2024-05-08T22:34:01.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 -- start start 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.478+0000 7fd979094700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 0x7fd9741609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd97414ae30 0x7fd974160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740c5980 con 0x7fd9740b1a50 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740c4e00 con 0x7fd97414ae30 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740c50e0 con 0x7fd9740b2de0 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38082/0 (socket says 172.21.15.179:38082) 2024-05-08T22:34:01.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 -- 172.21.15.179:0/2445072990 learned_addr learned my addr 172.21.15.179:0/2445072990 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd978893700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd97414ae30 0x7fd974160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd9737fe700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 0x7fd9741609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 -- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 msgr2=0x7fd9741609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 0x7fd9741609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 -- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd97414ae30 msgr2=0x7fd974160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd97414ae30 0x7fd974160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 -- 172.21.15.179:0/2445072990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640094b0 con 0x7fd9740b1a50 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd9737fe700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 0x7fd9741609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd973fff700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00efc0 tx=0x7fd96c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00cc00 con 0x7fd9740b1a50 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c00f040 con 0x7fd9740b1a50 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9740c53c0 con 0x7fd9740b1a50 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c014480 con 0x7fd9740b1a50 2024-05-08T22:34:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741fad20 con 0x7fd9740b1a50 2024-05-08T22:34:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.482+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97404e700 con 0x7fd9740b1a50 2024-05-08T22:34:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.486+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd96c0145e0 con 0x7fd9740b1a50 2024-05-08T22:34:01.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.486+0000 7fd970ff9700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 0x7fd960068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.486+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd96c084c60 con 0x7fd9740b1a50 2024-05-08T22:34:01.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.486+0000 7fd9737fe700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 0x7fd960068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.486+0000 7fd9737fe700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 0x7fd960068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd9741f4b70 tx=0x7fd968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.490+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd96c051cd0 con 0x7fd9740b1a50 2024-05-08T22:34:01.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.662+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd97406d460 con 0x7fd9740b1a50 2024-05-08T22:34:01.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.662+0000 7fd970ff9700 1 -- 172.21.15.179:0/2445072990 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd96c0552f0 con 0x7fd9740b1a50 2024-05-08T22:34:01.663 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 msgr2=0x7fd960068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 0x7fd960068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd9741f4b70 tx=0x7fd968009040 comp rx=0 tx=0).stop 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 msgr2=0x7fd9741fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00efc0 tx=0x7fd96c00c0c0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 shutdown_connections 2024-05-08T22:34:01.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd960066730 0x7fd960068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd97414ae30 0x7fd974160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9740b2de0 0x7fd9741609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 --2- 172.21.15.179:0/2445072990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9740b1a50 0x7fd9741fa7e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 >> 172.21.15.179:0/2445072990 conn(0x7fd9740cea30 msgr2=0x7fd9740b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 shutdown_connections 2024-05-08T22:34:01.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.666+0000 7fd979094700 1 -- 172.21.15.179:0/2445072990 wait complete. 2024-05-08T22:34:01.678 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:01.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 Processor -- start 2024-05-08T22:34:01.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 -- start start 2024-05-08T22:34:01.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f931c0cc700 0x7f931c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f931c15b1b0 0x7f931c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c06abb0 con 0x7f931c0cb510 2024-05-08T22:34:01.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0cb980 con 0x7f931c15b1b0 2024-05-08T22:34:01.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f9320a54700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c06ae90 con 0x7f931c0cc700 2024-05-08T22:34:01.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f931c0cb510 0x7f931c15e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58692/0 (socket says 172.21.15.179:58692) 2024-05-08T22:34:01.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f931c15b1b0 0x7f931c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931bfff700 1 -- 172.21.15.179:0/1747226921 learned_addr learned my addr 172.21.15.179:0/1747226921 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:01.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931b7fe700 1 --2- 172.21.15.179:0/1747226921 >> v2:172.21.15.195:3300/0 conn(0x7f931c0cc700 0x7f931c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931a7fc700 1 -- 172.21.15.179:0/1747226921 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650688584 0 0) 0x7f931c06abb0 con 0x7f931c0cb510 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931a7fc700 1 -- 172.21.15.179:0/1747226921 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93080035d0 con 0x7f931c0cb510 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931affd700 1 -- 172.21.15.179:0/1747226921 >> v2:172.21.15.195:3300/0 conn(0x7f931c0cc700 msgr2=0x7f931c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931affd700 1 --2- 172.21.15.179:0/1747226921 >> v2:172.21.15.195:3300/0 conn(0x7f931c0cc700 0x7f931c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931affd700 1 -- 172.21.15.179:0/1747226921 >> v1:172.21.15.179:6789/0 conn(0x7f931c0cb510 legacy=0x7f931c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.786+0000 7f931affd700 1 -- 172.21.15.179:0/1747226921 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c15b9c0 con 0x7f931c15b1b0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f931affd700 1 --2- 172.21.15.179:0/1747226921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c15b1b0 0x7f931c0cafd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f931c070800 tx=0x7f930c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155c5b326f9886d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f931a7fc700 1 -- 172.21.15.179:0/1747226921 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00e070 con 0x7f931c15b1b0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f931a7fc700 1 -- 172.21.15.179:0/1747226921 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c00bb20 con 0x7f931c15b1b0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f931a7fc700 1 -- 172.21.15.179:0/1747226921 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00be90 con 0x7f931c15b1b0 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- 172.21.15.179:0/1747226921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c15b1b0 msgr2=0x7f931c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 --2- 172.21.15.179:0/1747226921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c15b1b0 0x7f931c0cafd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f931c070800 tx=0x7f930c00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:01.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- 172.21.15.179:0/1747226921 shutdown_connections 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 --2- 172.21.15.179:0/1747226921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c15b1b0 0x7f931c0cafd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 --2- 172.21.15.179:0/1747226921 >> v2:172.21.15.195:3300/0 conn(0x7f931c0cc700 0x7f931c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- 172.21.15.179:0/1747226921 >> 172.21.15.179:0/1747226921 conn(0x7f931c0ce9d0 msgr2=0x7f931c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- 172.21.15.179:0/1747226921 shutdown_connections 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- 172.21.15.179:0/1747226921 wait complete. 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 Processor -- start 2024-05-08T22:34:01.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.790+0000 7f9320a54700 1 -- start start 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f931c0cb510 0x7f931c0a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f931c15b1b0 0x7f931c0ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a8d70 con 0x7f931c0cc700 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a8160 con 0x7f931c0cb510 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a8440 con 0x7f931c15b1b0 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38112/0 (socket says 172.21.15.179:38112) 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 -- 172.21.15.179:0/24838567 learned_addr learned my addr 172.21.15.179:0/24838567 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931b7fe700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f931c0cb510 0x7f931c0a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931bfff700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f931c15b1b0 0x7f931c0ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 -- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f931c15b1b0 msgr2=0x7f931c0ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f931c15b1b0 0x7f931c0ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 -- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f931c0cb510 msgr2=0x7f931c0a7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f931c0cb510 0x7f931c0a7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 -- 172.21.15.179:0/24838567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c009a70 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f931affd700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f930c00b370 tx=0x7f930c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c004800 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c020070 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0276c0 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0a8720 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.794+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0a7e70 con 0x7f931c0cc700 2024-05-08T22:34:01.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c0a8f20 con 0x7f931c0cc700 2024-05-08T22:34:01.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f930c012430 con 0x7f931c0cc700 2024-05-08T22:34:01.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f9313fff700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 0x7f92fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:01.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f930c022070 con 0x7f931c0cc700 2024-05-08T22:34:01.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f931b7fe700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 0x7f92fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:01.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.798+0000 7f931b7fe700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 0x7f92fc068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9304009b50 tx=0x7f9304009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:01.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.806+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f931c0a8f20 con 0x7f931c0cc700 2024-05-08T22:34:01.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.978+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f931c067740 con 0x7f931c0cc700 2024-05-08T22:34:01.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.978+0000 7f9313fff700 1 -- 172.21.15.179:0/24838567 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17926 (secure 0 0 0) 0x7f930c02c090 con 0x7f931c0cc700 2024-05-08T22:34:01.981 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:01.982 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:33:19.305875+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:01.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 msgr2=0x7f92fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 0x7f92fc068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9304009b50 tx=0x7f9304009240 comp rx=0 tx=0).stop 2024-05-08T22:34:01.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 msgr2=0x7f931c0ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f930c00b370 tx=0x7f930c004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 shutdown_connections 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f92fc066730 0x7f92fc068bf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f931c15b1b0 0x7f931c0ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f931c0cc700 0x7f931c0ac2d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 --2- 172.21.15.179:0/24838567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f931c0cb510 0x7f931c0a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.982+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 >> 172.21.15.179:0/24838567 conn(0x7f931c0ce9d0 msgr2=0x7f931c0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.986+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 shutdown_connections 2024-05-08T22:34:01.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:01.986+0000 7f9320a54700 1 -- 172.21.15.179:0/24838567 wait complete. 2024-05-08T22:34:02.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:02.002 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:02.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 Processor -- start 2024-05-08T22:34:02.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 -- start start 2024-05-08T22:34:02.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe2600e3300 0x7fe260006950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe2600e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe260057880 con 0x7fe2600e7410 2024-05-08T22:34:02.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2600e78a0 con 0x7fe2600e3300 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.118+0000 7fe26759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600074c0 con 0x7fe260006e90 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe266d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe2600e7410 0x7fe260053850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58694/0 (socket says 172.21.15.179:58694) 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe266d9d700 1 -- 172.21.15.179:0/814844601 learned_addr learned my addr 172.21.15.179:0/814844601 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26659c700 1 --2- 172.21.15.179:0/814844601 >> v2:172.21.15.195:3300/0 conn(0x7fe2600e3300 0x7fe260006950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe2600e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012965561 0 0) 0x7fe260057880 con 0x7fe2600e7410 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2500035d0 con 0x7fe2600e7410 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659584011 0 0) 0x7fe2500035d0 con 0x7fe2600e7410 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 msgr2=0x7fe2600e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe2600e6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 >> v2:172.21.15.195:3300/0 conn(0x7fe2600e3300 msgr2=0x7fe260006950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 --2- 172.21.15.179:0/814844601 >> v2:172.21.15.195:3300/0 conn(0x7fe2600e3300 0x7fe260006950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2600077a0 con 0x7fe2600e7410 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe268073d00 con 0x7fe2600e7410 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 >> v1:172.21.15.179:6789/0 conn(0x7fe2600e7410 legacy=0x7fe260053850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe250003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe250003fa0 0x7fe250004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250004db0 0x7fe250005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600e78a0 con 0x7fe250003fa0 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600074c0 con 0x7fe250004db0 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25000ba60 con 0x7fe260006e90 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe250003fa0 0x7fe250004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26659c700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe250003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe266d9d700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250004db0 0x7fe250005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 -- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 msgr2=0x7fe250003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe260006e90 0x7fe250003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 -- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250004db0 msgr2=0x7fe250005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250004db0 0x7fe250005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 -- 172.21.15.179:0/814844601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500035d0 con 0x7fe250003fa0 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe265d9b700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe250003fa0 0x7fe250004870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe258009b70 tx=0x7fe2500058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94abfc10fbb5f3f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800e660 con 0x7fe250003fa0 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25800e7c0 con 0x7fe250003fa0 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26559a700 1 -- 172.21.15.179:0/814844601 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800eb30 con 0x7fe250003fa0 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 -- 172.21.15.179:0/814844601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe250003fa0 msgr2=0x7fe250004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 --2- 172.21.15.179:0/814844601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe250003fa0 0x7fe250004870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe258009b70 tx=0x7fe2500058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe266d9d700 1 -- 172.21.15.179:0/814844601 reap_dead start 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 -- 172.21.15.179:0/814844601 shutdown_connections 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 -- 172.21.15.179:0/814844601 >> 172.21.15.179:0/814844601 conn(0x7fe26003aae0 msgr2=0x7fe26003aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 -- 172.21.15.179:0/814844601 shutdown_connections 2024-05-08T22:34:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.122+0000 7fe26759e700 1 -- 172.21.15.179:0/814844601 wait complete. 2024-05-08T22:34:02.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 Processor -- start 2024-05-08T22:34:02.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 -- start start 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250003fa0 0x7fe2600d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe26004cfa0 0x7fe2600e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26004d550 con 0x7fe26004cb20 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26004d830 con 0x7fe250003fa0 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26004db10 con 0x7fe26004cfa0 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38128/0 (socket says 172.21.15.179:38128) 2024-05-08T22:34:02.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 -- 172.21.15.179:0/2352293282 learned_addr learned my addr 172.21.15.179:0/2352293282 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe26659c700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250003fa0 0x7fe2600d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe266d9d700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe26004cfa0 0x7fe2600e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 -- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe26004cfa0 msgr2=0x7fe2600e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe26004cfa0 0x7fe2600e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 -- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250003fa0 msgr2=0x7fe2600d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250003fa0 0x7fe2600d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.126+0000 7fe265d9b700 1 -- 172.21.15.179:0/2352293282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26004ddf0 con 0x7fe26004cb20 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe265d9b700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2580090b0 tx=0x7fe2580047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800e660 con 0x7fe26004cb20 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258013d50 con 0x7fe26004cb20 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26004e0d0 con 0x7fe26004cb20 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258004290 con 0x7fe26004cb20 2024-05-08T22:34:02.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2600d08b0 con 0x7fe26004cb20 2024-05-08T22:34:02.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe260005440 con 0x7fe26004cb20 2024-05-08T22:34:02.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.130+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2580043f0 con 0x7fe26004cb20 2024-05-08T22:34:02.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.134+0000 7fe256ffd700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 0x7fe248068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.134+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe258085a20 con 0x7fe26004cb20 2024-05-08T22:34:02.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.134+0000 7fe26659c700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 0x7fe248068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.134+0000 7fe26659c700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 0x7fe248068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009bf0 tx=0x7fe25c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.138+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe258052bb0 con 0x7fe26004cb20 2024-05-08T22:34:02.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.310+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe260054d30 con 0x7fe26004cb20 2024-05-08T22:34:02.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.310+0000 7fe256ffd700 1 -- 172.21.15.179:0/2352293282 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2580561d0 con 0x7fe26004cb20 2024-05-08T22:34:02.311 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 msgr2=0x7fe248068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 0x7fe248068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009bf0 tx=0x7fe25c008040 comp rx=0 tx=0).stop 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 msgr2=0x7fe2600e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2580090b0 tx=0x7fe2580047e0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 shutdown_connections 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe248066730 0x7fe248068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe26004cfa0 0x7fe2600e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26004cb20 0x7fe2600e4dd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 --2- 172.21.15.179:0/2352293282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe250003fa0 0x7fe2600d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 >> 172.21.15.179:0/2352293282 conn(0x7fe26003aae0 msgr2=0x7fe260057c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 shutdown_connections 2024-05-08T22:34:02.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.314+0000 7fe26759e700 1 -- 172.21.15.179:0/2352293282 wait complete. 2024-05-08T22:34:02.326 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:02.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 Processor -- start 2024-05-08T22:34:02.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 -- start start 2024-05-08T22:34:02.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9415d1c0 0x7f8b940d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b940b8a80 0x7f8b940cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b940d0e80 con 0x7f8b940ce300 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b940d2ec0 con 0x7f8b940b8a80 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b9b45c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9406ae50 con 0x7f8b9415d1c0 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b940b8a80 0x7f8b940cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b940b8a80 0x7f8b940cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38136/0 (socket says 172.21.15.179:38136) 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b999f9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8b940ce300 0x7f8b94160d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58706/0 (socket says 172.21.15.179:58706) 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 -- 172.21.15.179:0/1909241723 learned_addr learned my addr 172.21.15.179:0/1909241723 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b991f8700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9415d1c0 0x7f8b940d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 -- 172.21.15.179:0/1909241723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9415d1c0 msgr2=0x7f8b940d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9415d1c0 0x7f8b940d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 -- 172.21.15.179:0/1909241723 >> v1:172.21.15.179:6789/0 conn(0x7f8b940ce300 legacy=0x7f8b94160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 -- 172.21.15.179:0/1909241723 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940d2620 con 0x7f8b940b8a80 2024-05-08T22:34:02.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b989f7700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b940cddc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009910 tx=0x7f8b90009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18826fecd4fafc4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b8bfff700 1 -- 172.21.15.179:0/1909241723 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90005e50 con 0x7f8b940b8a80 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b8bfff700 1 -- 172.21.15.179:0/1909241723 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b90006030 con 0x7f8b940b8a80 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.446+0000 7f8b8bfff700 1 -- 172.21.15.179:0/1909241723 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b900063e0 con 0x7f8b940b8a80 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- 172.21.15.179:0/1909241723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 msgr2=0x7f8b940cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b940cddc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009910 tx=0x7f8b90009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- 172.21.15.179:0/1909241723 shutdown_connections 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b940cddc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/1909241723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b9415d1c0 0x7f8b940d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- 172.21.15.179:0/1909241723 >> 172.21.15.179:0/1909241723 conn(0x7f8b940b4be0 msgr2=0x7f8b940b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- 172.21.15.179:0/1909241723 shutdown_connections 2024-05-08T22:34:02.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- 172.21.15.179:0/1909241723 wait complete. 2024-05-08T22:34:02.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 Processor -- start 2024-05-08T22:34:02.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.450+0000 7f8b9b45c700 1 -- start start 2024-05-08T22:34:02.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b940ce300 0x7f8b941f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b9415d1c0 0x7f8b940c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941453b0 con 0x7f8b940b8a80 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941f66f0 con 0x7f8b9415d1c0 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94145650 con 0x7f8b940ce300 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38142/0 (socket says 172.21.15.179:38142) 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 -- 172.21.15.179:0/2075281223 learned_addr learned my addr 172.21.15.179:0/2075281223 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b989f7700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b940ce300 0x7f8b941f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b999f9700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b9415d1c0 0x7f8b940c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 -- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b940ce300 msgr2=0x7f8b941f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b940ce300 0x7f8b941f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 -- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b9415d1c0 msgr2=0x7f8b940c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b9415d1c0 0x7f8b940c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 -- 172.21.15.179:0/2075281223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b90009520 con 0x7f8b940b8a80 2024-05-08T22:34:02.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b991f8700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400e840 tx=0x7f8b8400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b840095a0 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84013680 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b941f6900 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b840139f0 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b941f6ec0 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.454+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b941f70b0 con 0x7f8b940b8a80 2024-05-08T22:34:02.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.458+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b84013b50 con 0x7f8b940b8a80 2024-05-08T22:34:02.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.458+0000 7f8b89ffb700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 0x7f8b78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.458+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b84084210 con 0x7f8b940b8a80 2024-05-08T22:34:02.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.458+0000 7f8b989f7700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 0x7f8b78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.458+0000 7f8b989f7700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 0x7f8b78068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b90006000 tx=0x7f8b90008070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.466+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b941f70b0 con 0x7f8b940b8a80 2024-05-08T22:34:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b941f70b0 con 0x7f8b940b8a80 2024-05-08T22:34:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b89ffb700 1 -- 172.21.15.179:0/2075281223 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b941f70b0 con 0x7f8b940b8a80 2024-05-08T22:34:02.635 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 msgr2=0x7f8b78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 0x7f8b78068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b90006000 tx=0x7f8b90008070 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 msgr2=0x7f8b941f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.634+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400e840 tx=0x7f8b8400eb50 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 shutdown_connections 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b78066730 0x7f8b78068bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b9415d1c0 0x7f8b940c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b940ce300 0x7f8b941f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 --2- 172.21.15.179:0/2075281223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b940b8a80 0x7f8b941f36d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 >> 172.21.15.179:0/2075281223 conn(0x7f8b940b4be0 msgr2=0x7f8b940b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 shutdown_connections 2024-05-08T22:34:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.638+0000 7f8b9b45c700 1 -- 172.21.15.179:0/2075281223 wait complete. 2024-05-08T22:34:02.650 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:02.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.766+0000 7ff5fa99c700 1 Processor -- start 2024-05-08T22:34:02.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.766+0000 7ff5fa99c700 1 -- start start 2024-05-08T22:34:02.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5f406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f406d7e0 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40cf8c0 con 0x7ff5f40b3880 2024-05-08T22:34:02.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f406d220 con 0x7ff5f415a160 2024-05-08T22:34:02.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5f40cb2a0 0x7ff5f40cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58712/0 (socket says 172.21.15.179:58712) 2024-05-08T22:34:02.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f37fe700 1 -- 172.21.15.179:0/3913156954 learned_addr learned my addr 172.21.15.179:0/3913156954 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f3fff700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5f406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726265441 0 0) 0x7ff5f406d7e0 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5dc0035d0 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3912864466 0 0) 0x7ff5dc0035d0 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 msgr2=0x7ff5f406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5f406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 >> v1:172.21.15.195:6789/0 conn(0x7ff5f40b3880 legacy=0x7ff5f40cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f406d500 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff5e4001670 con 0x7ff5f40cb2a0 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 >> v1:172.21.15.179:6789/0 conn(0x7ff5f40cb2a0 legacy=0x7ff5f40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc003a50 0x7ff5dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5dc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5dc004db0 0x7ff5dc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f406d220 con 0x7ff5dc004db0 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc00bb10 con 0x7ff5dc003a50 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc00bcd0 con 0x7ff5f415a160 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5dc004db0 0x7ff5dc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f37fe700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5dc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f3fff700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc003a50 0x7ff5dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 -- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 msgr2=0x7ff5dc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f415a160 0x7ff5dc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 -- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc003a50 msgr2=0x7ff5dc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc003a50 0x7ff5dc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 -- 172.21.15.179:0/3913156954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0035d0 con 0x7ff5dc004db0 2024-05-08T22:34:02.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5dc004db0 0x7ff5dc005230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e420 tx=0x7ff5dc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7749e73f4cb41976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec001e40 con 0x7ff5dc004db0 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00ee80 con 0x7ff5dc004db0 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f2ffd700 1 -- 172.21.15.179:0/3913156954 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec003c60 con 0x7ff5dc004db0 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- 172.21.15.179:0/3913156954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5dc004db0 msgr2=0x7ff5dc005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/3913156954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5dc004db0 0x7ff5dc005230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e420 tx=0x7ff5dc0058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5f8f39700 1 -- 172.21.15.179:0/3913156954 reap_dead start 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- 172.21.15.179:0/3913156954 shutdown_connections 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- 172.21.15.179:0/3913156954 >> 172.21.15.179:0/3913156954 conn(0x7ff5f40ceca0 msgr2=0x7ff5f40cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- 172.21.15.179:0/3913156954 shutdown_connections 2024-05-08T22:34:02.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.770+0000 7ff5fa99c700 1 -- 172.21.15.179:0/3913156954 wait complete. 2024-05-08T22:34:02.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 Processor -- start 2024-05-08T22:34:02.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- start start 2024-05-08T22:34:02.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc004db0 0x7ff5f41f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 0x7ff5f41485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f41f29c0 con 0x7ff5f4146060 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f41568f0 con 0x7ff5dc004db0 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4149170 con 0x7ff5f4145be0 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38158/0 (socket says 172.21.15.179:38158) 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 -- 172.21.15.179:0/2157111954 learned_addr learned my addr 172.21.15.179:0/2157111954 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f3fff700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc004db0 0x7ff5f41f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f37fe700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 0x7ff5f41485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 -- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 msgr2=0x7ff5f41485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 0x7ff5f41485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 -- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc004db0 msgr2=0x7ff5f41f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc004db0 0x7ff5f41f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 -- 172.21.15.179:0/2157111954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4149450 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f37fe700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 0x7ff5f41485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f8f39700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005040 tx=0x7ff5ec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0055a0 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec005720 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f41f2300 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0116c0 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.774+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f40aed00 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f41f2b00 con 0x7ff5f4146060 2024-05-08T22:34:02.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5ec011820 con 0x7ff5f4146060 2024-05-08T22:34:02.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5f0ff9700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 0x7ff5e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:02.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff5ec085fb0 con 0x7ff5f4146060 2024-05-08T22:34:02.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5f3fff700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 0x7ff5e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:02.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.778+0000 7ff5f3fff700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 0x7ff5e0068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8010ad0 tx=0x7ff5e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:02.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.782+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5f41f2b00 con 0x7ff5f4146060 2024-05-08T22:34:02.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.954+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5f41f2b00 con 0x7ff5f4146060 2024-05-08T22:34:02.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5f0ff9700 1 -- 172.21.15.179:0/2157111954 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5f41f2b00 con 0x7ff5f4146060 2024-05-08T22:34:02.958 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:02.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 msgr2=0x7ff5e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 0x7ff5e0068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8010ad0 tx=0x7ff5e8008040 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 msgr2=0x7ff5f4148b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005040 tx=0x7ff5ec0014a0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 shutdown_connections 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5e0066730 0x7ff5e0068bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f4146060 0x7ff5f4148b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f4145be0 0x7ff5f41485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 --2- 172.21.15.179:0/2157111954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5dc004db0 0x7ff5f41f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 >> 172.21.15.179:0/2157111954 conn(0x7ff5f40ceca0 msgr2=0x7ff5f4159450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 shutdown_connections 2024-05-08T22:34:02.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:02.958+0000 7ff5fa99c700 1 -- 172.21.15.179:0/2157111954 wait complete. 2024-05-08T22:34:02.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:03.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 Processor -- start 2024-05-08T22:34:03.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 -- start start 2024-05-08T22:34:03.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe1a416c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a406dd50 con 0x7fe1a40b1a50 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a406e030 con 0x7fe1a414ae30 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aacb4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a406e9d0 con 0x7fe1a40b2de0 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a94b1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe1a40b1a50 0x7fe1a40b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58720/0 (socket says 172.21.15.179:58720) 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a94b1700 1 -- 172.21.15.179:0/1968344450 learned_addr learned my addr 172.21.15.179:0/1968344450 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aa4b3700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe1a416c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795923660 0 0) 0x7fe1a406dd50 con 0x7fe1a40b1a50 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1900035d0 con 0x7fe1a40b1a50 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2162995874 0 0) 0x7fe1900035d0 con 0x7fe1a40b1a50 2024-05-08T22:34:03.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 msgr2=0x7fe1a416c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe1a416c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 >> v1:172.21.15.195:6789/0 conn(0x7fe1a414ae30 legacy=0x7fe1a4165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a406e310 con 0x7fe1a40b1a50 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe1a0001670 con 0x7fe1a40b1a50 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 >> v1:172.21.15.179:6789/0 conn(0x7fe1a40b1a50 legacy=0x7fe1a40b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe190003a50 0x7fe190003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe190004870 0x7fe190004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe190005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a406e9d0 con 0x7fe190003a50 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19000bb90 con 0x7fe190004870 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19000bd50 con 0x7fe1a40b2de0 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe190003a50 0x7fe190003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aa4b3700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe190005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a94b1700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe190004870 0x7fe190004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 -- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 msgr2=0x7fe190005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe190005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 -- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe190004870 msgr2=0x7fe190004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe190004870 0x7fe190004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 -- 172.21.15.179:0/1968344450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900035d0 con 0x7fe190003a50 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1aa4b3700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1a40b2de0 0x7fe190005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe190003a50 0x7fe190003e70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe1940017e0 tx=0x7fe1900058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d9a4e1644ccda4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1940052d0 con 0x7fe190003a50 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe194005430 con 0x7fe190003a50 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.086+0000 7fe1a8cb0700 1 -- 172.21.15.179:0/1968344450 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1940055a0 con 0x7fe190003a50 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- 172.21.15.179:0/1968344450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe190003a50 msgr2=0x7fe190003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/1968344450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe190003a50 0x7fe190003e70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe1940017e0 tx=0x7fe1900058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aa4b3700 1 -- 172.21.15.179:0/1968344450 reap_dead start 2024-05-08T22:34:03.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- 172.21.15.179:0/1968344450 shutdown_connections 2024-05-08T22:34:03.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- 172.21.15.179:0/1968344450 >> 172.21.15.179:0/1968344450 conn(0x7fe1a40cea30 msgr2=0x7fe1a40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- 172.21.15.179:0/1968344450 shutdown_connections 2024-05-08T22:34:03.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- 172.21.15.179:0/1968344450 wait complete. 2024-05-08T22:34:03.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 Processor -- start 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- start start 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe190003a50 0x7fe1a415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1a4148ae0 0x7fe1a4148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41494a0 con 0x7fe1a40cce90 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a415f5c0 con 0x7fe1a4148ae0 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aacb4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a4145d40 con 0x7fe190003a50 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1a94b1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1a94b1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38174/0 (socket says 172.21.15.179:38174) 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1a94b1700 1 -- 172.21.15.179:0/3864232624 learned_addr learned my addr 172.21.15.179:0/3864232624 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe190003a50 0x7fe1a415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.090+0000 7fe1aa4b3700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1a4148ae0 0x7fe1a4148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 -- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe190003a50 msgr2=0x7fe1a415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe190003a50 0x7fe1a415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 -- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1a4148ae0 msgr2=0x7fe1a4148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1a4148ae0 0x7fe1a4148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 -- 172.21.15.179:0/3864232624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a415f860 con 0x7fe1a40cce90 2024-05-08T22:34:03.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1a94b1700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007290 tx=0x7fe1a0007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000f6c0 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a000f820 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000fbd0 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a415fb40 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a4160100 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a404e700 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.094+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1a000fd30 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.098+0000 7fe19a7fc700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.098+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe1a00843d0 con 0x7fe1a40cce90 2024-05-08T22:34:03.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.098+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.098+0000 7fe1a9cb2700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 0x7fe188068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe194000c00 tx=0x7fe194009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.106+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe1a0051460 con 0x7fe1a40cce90 2024-05-08T22:34:03.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.270+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1a406d460 con 0x7fe1a40cce90 2024-05-08T22:34:03.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.270+0000 7fe19a7fc700 1 -- 172.21.15.179:0/3864232624 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe1a0054a80 con 0x7fe1a40cce90 2024-05-08T22:34:03.270 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 msgr2=0x7fe188068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 0x7fe188068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe194000c00 tx=0x7fe194009030 comp rx=0 tx=0).stop 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 msgr2=0x7fe1a4145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007290 tx=0x7fe1a0007650 comp rx=0 tx=0).stop 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 shutdown_connections 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1a4148ae0 0x7fe1a4148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1a40cce90 0x7fe1a4145800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 --2- 172.21.15.179:0/3864232624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe190003a50 0x7fe1a415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 >> 172.21.15.179:0/3864232624 conn(0x7fe1a40cea30 msgr2=0x7fe1a406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 shutdown_connections 2024-05-08T22:34:03.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.274+0000 7fe1aacb4700 1 -- 172.21.15.179:0/3864232624 wait complete. 2024-05-08T22:34:03.292 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:03.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.402+0000 7f72d5d8c700 1 Processor -- start 2024-05-08T22:34:03.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.402+0000 7f72d5d8c700 1 -- start start 2024-05-08T22:34:03.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f72d00cc700 0x7f72d00ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d006abb0 con 0x7f72d015b1b0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d006ae90 con 0x7f72d00cb6e0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d015b9c0 con 0x7f72d00cc700 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f72d015b1b0 0x7f72d00cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58734/0 (socket says 172.21.15.179:58734) 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cffff700 1 -- 172.21.15.179:0/147829976 learned_addr learned my addr 172.21.15.179:0/147829976 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d4d8a700 1 --2- 172.21.15.179:0/147829976 >> v2:172.21.15.195:3300/0 conn(0x7f72d00cc700 0x7f72d00ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271604433 0 0) 0x7f72d006abb0 con 0x7f72d015b1b0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c00035d0 con 0x7f72d015b1b0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 656814708 0 0) 0x7f72c00035d0 con 0x7f72d015b1b0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 >> v2:172.21.15.195:3300/0 conn(0x7f72d00cc700 msgr2=0x7f72d00ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 --2- 172.21.15.179:0/147829976 >> v2:172.21.15.195:3300/0 conn(0x7f72d00cc700 0x7f72d00ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 >> v1:172.21.15.195:6789/0 conn(0x7f72d00cb6e0 legacy=0x7f72d006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d015bca0 con 0x7f72d015b1b0 2024-05-08T22:34:03.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f72bc001670 con 0x7f72d015b1b0 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 >> v1:172.21.15.179:6789/0 conn(0x7f72d015b1b0 legacy=0x7f72d00cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72c0003a20 0x7f72c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72c0004420 0x7f72c00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72c000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015b9c0 con 0x7f72c0005190 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c000c0d0 con 0x7f72c0004420 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c000c290 con 0x7f72c0003a20 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72c000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d4d8a700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72c0003a20 0x7f72c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cffff700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72c0004420 0x7f72c00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 -- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72c0003a20 msgr2=0x7f72c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72c0003a20 0x7f72c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 -- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72c0004420 msgr2=0x7f72c00048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72c0004420 0x7f72c00048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 -- 172.21.15.179:0/147829976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00035d0 con 0x7f72c0005190 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 -- 172.21.15.179:0/147829976 reap_dead start 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d558b700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72c000ba30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72d00708e0 tx=0x7f72c00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864aeaa227bb0e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8001f20 con 0x7f72c0005190 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800ce30 con 0x7f72c0005190 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72cf7fe700 1 -- 172.21.15.179:0/147829976 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8003c60 con 0x7f72c0005190 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 msgr2=0x7f72c000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72c000ba30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72d00708e0 tx=0x7f72c00058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- 172.21.15.179:0/147829976 shutdown_connections 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/147829976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72c000ba30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- 172.21.15.179:0/147829976 >> 172.21.15.179:0/147829976 conn(0x7f72d00ce9d0 msgr2=0x7f72d00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- 172.21.15.179:0/147829976 shutdown_connections 2024-05-08T22:34:03.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.406+0000 7f72d5d8c700 1 -- 172.21.15.179:0/147829976 wait complete. 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 Processor -- start 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- start start 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d00b23b0 0x7f72d00b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72d014ee10 0x7f72d014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00b2d70 con 0x7f72c0005190 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d014f7d0 con 0x7f72d00b23b0 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0146880 con 0x7f72d014ee10 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38184/0 (socket says 172.21.15.179:38184) 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 -- 172.21.15.179:0/144073657 learned_addr learned my addr 172.21.15.179:0/144073657 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d558b700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72d014ee10 0x7f72d014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72cffff700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d00b23b0 0x7f72d00b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 -- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72d014ee10 msgr2=0x7f72d014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72d014ee10 0x7f72d014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 -- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d00b23b0 msgr2=0x7f72d00b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d00b23b0 0x7f72d00b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 -- 172.21.15.179:0/144073657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d014fa90 con 0x7f72c0005190 2024-05-08T22:34:03.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d4d8a700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72c40119b0 tx=0x7f72c4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400f940 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d014fd70 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d0150330 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400faa0 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.410+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400fe50 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d0067740 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72c401d400 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72cd7fa700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 0x7f72b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72c4084c70 con 0x7f72c0005190 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72cffff700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 0x7f72b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.414+0000 7f72cffff700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 0x7f72b4068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72d00b37b0 tx=0x7f72bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.426+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72c4051d20 con 0x7f72c0005190 2024-05-08T22:34:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.602+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72d004e700 con 0x7f72c0005190 2024-05-08T22:34:03.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72cd7fa700 1 -- 172.21.15.179:0/144073657 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f72c4055340 con 0x7f72c0005190 2024-05-08T22:34:03.606 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:03.606 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":27,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:03.460511+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:03.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 msgr2=0x7f72b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 0x7f72b4068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72d00b37b0 tx=0x7f72bc009040 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 msgr2=0x7f72d0146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72c40119b0 tx=0x7f72c4011d70 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 shutdown_connections 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f72b4066730 0x7f72b4068bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d00b23b0 0x7f72d00b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72d014ee10 0x7f72d014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 --2- 172.21.15.179:0/144073657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c0005190 0x7f72d0146340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 >> 172.21.15.179:0/144073657 conn(0x7f72d00ce9d0 msgr2=0x7f72d006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 shutdown_connections 2024-05-08T22:34:03.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.606+0000 7f72d5d8c700 1 -- 172.21.15.179:0/144073657 wait complete. 2024-05-08T22:34:03.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:03.621 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.734+0000 7f1e165be700 1 Processor -- start 2024-05-08T22:34:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 -- start start 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 0x7f1e100b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e100b2ca0 0x7f1e1006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100b22d0 con 0x7f1e10142fd0 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100b19e0 con 0x7f1e100b2ca0 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e165be700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1015e640 con 0x7f1e10142890 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e14dbb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e100b2ca0 0x7f1e1006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 0x7f1e100b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e15dbd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1e10142fd0 0x7f1e1015d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:42654/0 (socket says 172.21.15.179:42654) 2024-05-08T22:34:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e15dbd700 1 -- 172.21.15.179:0/1765470975 learned_addr learned my addr 172.21.15.179:0/1765470975 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e03fff700 1 -- 172.21.15.179:0/1765470975 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432231107 0 0) 0x7f1e100b22d0 con 0x7f1e10142fd0 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e03fff700 1 -- 172.21.15.179:0/1765470975 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc0035d0 con 0x7f1e10142fd0 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 -- 172.21.15.179:0/1765470975 >> v2:172.21.15.195:3300/0 conn(0x7f1e100b2ca0 msgr2=0x7f1e1006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 --2- 172.21.15.179:0/1765470975 >> v2:172.21.15.195:3300/0 conn(0x7f1e100b2ca0 0x7f1e1006cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 -- 172.21.15.179:0/1765470975 >> v1:172.21.15.195:6789/0 conn(0x7f1e10142fd0 legacy=0x7f1e1015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 -- 172.21.15.179:0/1765470975 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100b1c80 con 0x7f1e10142890 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e155bc700 1 --2- 172.21.15.179:0/1765470975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 0x7f1e100b2760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009850 tx=0x7f1e0c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02143004eea1609 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e03fff700 1 -- 172.21.15.179:0/1765470975 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c011830 con 0x7f1e10142890 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e03fff700 1 -- 172.21.15.179:0/1765470975 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c011990 con 0x7f1e10142890 2024-05-08T22:34:03.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.738+0000 7f1e03fff700 1 -- 172.21.15.179:0/1765470975 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c011d40 con 0x7f1e10142890 2024-05-08T22:34:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- 172.21.15.179:0/1765470975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 msgr2=0x7f1e100b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- 172.21.15.179:0/1765470975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 0x7f1e100b2760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009850 tx=0x7f1e0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- 172.21.15.179:0/1765470975 shutdown_connections 2024-05-08T22:34:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- 172.21.15.179:0/1765470975 >> v2:172.21.15.195:3300/0 conn(0x7f1e100b2ca0 0x7f1e1006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- 172.21.15.179:0/1765470975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e10142890 0x7f1e100b2760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- 172.21.15.179:0/1765470975 >> 172.21.15.179:0/1765470975 conn(0x7f1e100cea30 msgr2=0x7f1e100cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- 172.21.15.179:0/1765470975 shutdown_connections 2024-05-08T22:34:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- 172.21.15.179:0/1765470975 wait complete. 2024-05-08T22:34:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 Processor -- start 2024-05-08T22:34:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- start start 2024-05-08T22:34:03.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e100b2ca0 0x7f1e101ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e10142890 0x7f1e101f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101f2bd0 con 0x7f1e10142fd0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101f2dc0 con 0x7f1e10142890 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e165be700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10135720 con 0x7f1e100b2ca0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e15dbd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e15dbd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38192/0 (socket says 172.21.15.179:38192) 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e15dbd700 1 -- 172.21.15.179:0/476052589 learned_addr learned my addr 172.21.15.179:0/476052589 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e155bc700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e100b2ca0 0x7f1e101ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.742+0000 7f1e14dbb700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e10142890 0x7f1e101f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 -- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e100b2ca0 msgr2=0x7f1e101ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e100b2ca0 0x7f1e101ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 -- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e10142890 msgr2=0x7f1e101f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e10142890 0x7f1e101f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 -- 172.21.15.179:0/476052589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0094b0 con 0x7f1e10142fd0 2024-05-08T22:34:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e15dbd700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800caa0 tx=0x7f1e0800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08007480 con 0x7f1e10142fd0 2024-05-08T22:34:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080075e0 con 0x7f1e10142fd0 2024-05-08T22:34:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e101f3080 con 0x7f1e10142fd0 2024-05-08T22:34:03.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08007970 con 0x7f1e10142fd0 2024-05-08T22:34:03.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e101f3640 con 0x7f1e10142fd0 2024-05-08T22:34:03.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.746+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1004e700 con 0x7f1e10142fd0 2024-05-08T22:34:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.750+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e0801b3b0 con 0x7f1e10142fd0 2024-05-08T22:34:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.750+0000 7f1e01ffb700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 0x7f1df4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.750+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1e08083e80 con 0x7f1e10142fd0 2024-05-08T22:34:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.750+0000 7f1e155bc700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 0x7f1df4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:03.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.750+0000 7f1e155bc700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 0x7f1df4068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0090b0 tx=0x7f1e0c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:03.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.758+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e08050db0 con 0x7f1e10142fd0 2024-05-08T22:34:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e1013db90 con 0x7f1e10142fd0 2024-05-08T22:34:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e01ffb700 1 -- 172.21.15.179:0/476052589 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e080170a0 con 0x7f1e10142fd0 2024-05-08T22:34:03.926 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:03.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 msgr2=0x7f1df4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 0x7f1df4068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0090b0 tx=0x7f1e0c008030 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 msgr2=0x7f1e101351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800caa0 tx=0x7f1e0800cad0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 shutdown_connections 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1df4066730 0x7f1df4068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e10142890 0x7f1e101f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e10142fd0 0x7f1e101351e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 --2- 172.21.15.179:0/476052589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e100b2ca0 0x7f1e101ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 >> 172.21.15.179:0/476052589 conn(0x7f1e100cea30 msgr2=0x7f1e1006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.926+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 shutdown_connections 2024-05-08T22:34:03.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:03.930+0000 7f1e165be700 1 -- 172.21.15.179:0/476052589 wait complete. 2024-05-08T22:34:03.941 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 Processor -- start 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 -- start start 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feedc142fd0 0x7feedc06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0b22d0 con 0x7feedc142890 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0b19e0 con 0x7feedc142fd0 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.062+0000 7feedbfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc15e640 con 0x7feedc143520 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feeda7fc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feedc142fd0 0x7feedc06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:36646/0 (socket says 172.21.15.179:36646) 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 -- 172.21.15.179:0/1877348051 learned_addr learned my addr 172.21.15.179:0/1877348051 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedaffd700 1 --1- 172.21.15.179:0/1877348051 >> v1:172.21.15.195:6789/0 conn(0x7feedc142890 0x7feedc0b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:42668/0 (socket says 172.21.15.179:42668) 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 -- 172.21.15.179:0/1877348051 >> v2:172.21.15.195:3300/0 conn(0x7feedc142fd0 msgr2=0x7feedc06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 --2- 172.21.15.179:0/1877348051 >> v2:172.21.15.195:3300/0 conn(0x7feedc142fd0 0x7feedc06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 -- 172.21.15.179:0/1877348051 >> v1:172.21.15.195:6789/0 conn(0x7feedc142890 legacy=0x7feedc0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 -- 172.21.15.179:0/1877348051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0b1c80 con 0x7feedc143520 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedb7fe700 1 --2- 172.21.15.179:0/1877348051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e50 tx=0x7feed400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8740ea22d095abc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feed9ffb700 1 -- 172.21.15.179:0/1877348051 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400e070 con 0x7feedc143520 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feed9ffb700 1 -- 172.21.15.179:0/1877348051 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400b910 con 0x7feedc143520 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feed9ffb700 1 -- 172.21.15.179:0/1877348051 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400bcc0 con 0x7feedc143520 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 -- 172.21.15.179:0/1877348051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 msgr2=0x7feedc06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 --2- 172.21.15.179:0/1877348051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e50 tx=0x7feed400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:04.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 -- 172.21.15.179:0/1877348051 shutdown_connections 2024-05-08T22:34:04.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 --2- 172.21.15.179:0/1877348051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc06d2e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 --2- 172.21.15.179:0/1877348051 >> v2:172.21.15.195:3300/0 conn(0x7feedc142fd0 0x7feedc06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 -- 172.21.15.179:0/1877348051 >> 172.21.15.179:0/1877348051 conn(0x7feedc0cea30 msgr2=0x7feedc0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 -- 172.21.15.179:0/1877348051 shutdown_connections 2024-05-08T22:34:04.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.066+0000 7feedbfff700 1 -- 172.21.15.179:0/1877348051 wait complete. 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 Processor -- start 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- start start 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feedc142890 0x7feedc0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc12bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0ab990 con 0x7feedc142fd0 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0ac230 con 0x7feedc142890 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b01c0 con 0x7feedc143520 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38198/0 (socket says 172.21.15.179:38198) 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 -- 172.21.15.179:0/3037488365 learned_addr learned my addr 172.21.15.179:0/3037488365 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedaffd700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feedc142890 0x7feedc0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedb7fe700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc12bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 -- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 msgr2=0x7feedc12bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc12bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 -- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feedc142890 msgr2=0x7feedc0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feedc142890 0x7feedc0af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 -- 172.21.15.179:0/3037488365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4009b30 con 0x7feedc142fd0 2024-05-08T22:34:04.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feeda7fc700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9d0 tx=0x7feecc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc009720 con 0x7feedc142fd0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00ee70 con 0x7feedc142fd0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc0abbe0 con 0x7feedc142fd0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00ca50 con 0x7feedc142fd0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.070+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc0afe40 con 0x7feedc142fd0 2024-05-08T22:34:04.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc0ac3a0 con 0x7feedc142fd0 2024-05-08T22:34:04.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feecc00cbb0 con 0x7feedc142fd0 2024-05-08T22:34:04.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feec37fe700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 0x7feec8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feecc0845a0 con 0x7feedc142fd0 2024-05-08T22:34:04.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feedaffd700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 0x7feec8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.074+0000 7feedaffd700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 0x7feec8068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feed0000f60 tx=0x7feed0007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.082+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feecc0514d0 con 0x7feedc142fd0 2024-05-08T22:34:04.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feedc04e700 con 0x7feedc142fd0 2024-05-08T22:34:04.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feec37fe700 1 -- 172.21.15.179:0/3037488365 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7feecc00ce60 con 0x7feedc142fd0 2024-05-08T22:34:04.254 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 msgr2=0x7feec8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 0x7feec8068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feed0000f60 tx=0x7feed0007f30 comp rx=0 tx=0).stop 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 msgr2=0x7feedc159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9d0 tx=0x7feecc00ece0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 shutdown_connections 2024-05-08T22:34:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feec8066730 0x7feec8068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feedc143520 0x7feedc12bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feedc142fd0 0x7feedc159620 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 --2- 172.21.15.179:0/3037488365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feedc142890 0x7feedc0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 >> 172.21.15.179:0/3037488365 conn(0x7feedc0cea30 msgr2=0x7feedc134b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 shutdown_connections 2024-05-08T22:34:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.254+0000 7feedbfff700 1 -- 172.21.15.179:0/3037488365 wait complete. 2024-05-08T22:34:04.270 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:04.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 Processor -- start 2024-05-08T22:34:04.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 -- start start 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb2240b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb2240d2b10 0x7fb22406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240d1040 con 0x7fb22415d160 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22406c100 con 0x7fb2240d2b10 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb22b749700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22406c330 con 0x7fb2241614c0 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb2294e5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb22415d160 0x7fb224160db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58746/0 (socket says 172.21.15.179:58746) 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb2294e5700 1 -- 172.21.15.179:0/2781764251 learned_addr learned my addr 172.21.15.179:0/2781764251 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:04.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb228ce4700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb2240b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2781764251 >> v2:172.21.15.195:3300/0 conn(0x7fb2240d2b10 0x7fb22406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790920305 0 0) 0x7fb2240d1040 con 0x7fb22415d160 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2100035d0 con 0x7fb22415d160 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637295738 0 0) 0x7fb2100035d0 con 0x7fb22415d160 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 msgr2=0x7fb2240b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb2240b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 >> v2:172.21.15.195:3300/0 conn(0x7fb2240d2b10 msgr2=0x7fb22406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 --2- 172.21.15.179:0/2781764251 >> v2:172.21.15.195:3300/0 conn(0x7fb2240d2b10 0x7fb22406abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22406c610 con 0x7fb22415d160 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fb214001670 con 0x7fb22415d160 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 >> v1:172.21.15.179:6789/0 conn(0x7fb22415d160 legacy=0x7fb224160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb210003a20 0x7fb210003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb210004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb210004db0 0x7fb210005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22406c100 con 0x7fb210004db0 2024-05-08T22:34:04.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22406c330 con 0x7fb210003a20 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.386+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21000ba30 con 0x7fb2241614c0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb210004db0 0x7fb210005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb228ce4700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb210004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb2294e5700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb210003a20 0x7fb210003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 -- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 msgr2=0x7fb210004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241614c0 0x7fb210004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 -- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb210003a20 msgr2=0x7fb210003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb210003a20 0x7fb210003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 -- 172.21.15.179:0/2781764251 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100035d0 con 0x7fb210004db0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb210004db0 0x7fb210005230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e170 tx=0x7fb2100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de707c88dc418f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00ec90 con 0x7fb210004db0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00f040 con 0x7fb210004db0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb21bfff700 1 -- 172.21.15.179:0/2781764251 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c014480 con 0x7fb210004db0 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- 172.21.15.179:0/2781764251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb210004db0 msgr2=0x7fb210005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 --2- 172.21.15.179:0/2781764251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb210004db0 0x7fb210005230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e170 tx=0x7fb2100058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb229ce6700 1 -- 172.21.15.179:0/2781764251 reap_dead start 2024-05-08T22:34:04.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- 172.21.15.179:0/2781764251 shutdown_connections 2024-05-08T22:34:04.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- 172.21.15.179:0/2781764251 >> 172.21.15.179:0/2781764251 conn(0x7fb2240b4af0 msgr2=0x7fb2240b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- 172.21.15.179:0/2781764251 shutdown_connections 2024-05-08T22:34:04.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- 172.21.15.179:0/2781764251 wait complete. 2024-05-08T22:34:04.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 Processor -- start 2024-05-08T22:34:04.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.390+0000 7fb22b749700 1 -- start start 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb210004db0 0x7fb2240cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241f2a60 0x7fb2241f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241f3fe0 con 0x7fb2240c75b0 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240ce1c0 con 0x7fb2241f2a60 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241f3cb0 con 0x7fb210004db0 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38204/0 (socket says 172.21.15.179:38204) 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 -- 172.21.15.179:0/2741826550 learned_addr learned my addr 172.21.15.179:0/2741826550 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb228ce4700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241f2a60 0x7fb2241f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb2294e5700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb210004db0 0x7fb2240cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 -- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb210004db0 msgr2=0x7fb2240cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb210004db0 0x7fb2240cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 -- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241f2a60 msgr2=0x7fb2241f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241f2a60 0x7fb2241f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 -- 172.21.15.179:0/2741826550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2241f3420 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb229ce6700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb21c016000 tx=0x7fb21c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c007000 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c003e50 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2240c82e0 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c014c40 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240c7f70 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224073290 con 0x7fb2240c75b0 2024-05-08T22:34:04.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb21c024030 con 0x7fb2240c75b0 2024-05-08T22:34:04.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 0x7fb208068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.394+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb21c085cb0 con 0x7fb2240c75b0 2024-05-08T22:34:04.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.398+0000 7fb2294e5700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 0x7fb208068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.398+0000 7fb2294e5700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 0x7fb208068c20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb214001eb0 tx=0x7fb214007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.402+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb21c052e10 con 0x7fb2240c75b0 2024-05-08T22:34:04.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.578+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2240d0bf0 con 0x7fb2240c75b0 2024-05-08T22:34:04.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.578+0000 7fb219ffb700 1 -- 172.21.15.179:0/2741826550 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb21c018020 con 0x7fb2240c75b0 2024-05-08T22:34:04.581 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:04.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 msgr2=0x7fb208068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 0x7fb208068c20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb214001eb0 tx=0x7fb214007040 comp rx=0 tx=0).stop 2024-05-08T22:34:04.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 msgr2=0x7fb2240c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb21c016000 tx=0x7fb21c004b60 comp rx=0 tx=0).stop 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 shutdown_connections 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb208066760 0x7fb208068c20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241f2a60 0x7fb2241f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2240c75b0 0x7fb2240c7a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 --2- 172.21.15.179:0/2741826550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb210004db0 0x7fb2240cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 >> 172.21.15.179:0/2741826550 conn(0x7fb2240b4af0 msgr2=0x7fb2240b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 shutdown_connections 2024-05-08T22:34:04.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.582+0000 7fb22b749700 1 -- 172.21.15.179:0/2741826550 wait complete. 2024-05-08T22:34:04.596 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:04.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.714+0000 7facdb59e700 1 Processor -- start 2024-05-08T22:34:04.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.714+0000 7facdb59e700 1 -- start start 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.714+0000 7facdb59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facdc15d0c0 0x7facdc0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc15d4a0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc0ce570 con 0x7facdc15d0c0 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc06abb0 con 0x7facdc0d1d20 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facdc15d0c0 0x7facdc0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7facdc15d0c0 0x7facdc0b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38206/0 (socket says 172.21.15.179:38206) 2024-05-08T22:34:04.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 -- 172.21.15.179:0/2702430328 learned_addr learned my addr 172.21.15.179:0/2702430328 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facd9d9b700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 -- 172.21.15.179:0/2702430328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 msgr2=0x7facdc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 -- 172.21.15.179:0/2702430328 >> v1:172.21.15.195:6789/0 conn(0x7facdc0ce0e0 legacy=0x7facdc160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 -- 172.21.15.179:0/2702430328 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0d2500 con 0x7facdc15d0c0 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facda59c700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc15d0c0 0x7facdc0b8a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faccc009910 tx=0x7faccc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef79b085fe856083 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facd959a700 1 -- 172.21.15.179:0/2702430328 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc011830 con 0x7facdc15d0c0 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facd959a700 1 -- 172.21.15.179:0/2702430328 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc011990 con 0x7facdc15d0c0 2024-05-08T22:34:04.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facd959a700 1 -- 172.21.15.179:0/2702430328 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc011d40 con 0x7facdc15d0c0 2024-05-08T22:34:04.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- 172.21.15.179:0/2702430328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc15d0c0 msgr2=0x7facdc0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc15d0c0 0x7facdc0b8a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faccc009910 tx=0x7faccc009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:04.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- 172.21.15.179:0/2702430328 shutdown_connections 2024-05-08T22:34:04.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 --2- 172.21.15.179:0/2702430328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc15d0c0 0x7facdc0b8a80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- 172.21.15.179:0/2702430328 >> 172.21.15.179:0/2702430328 conn(0x7facdc0d2ed0 msgr2=0x7facdc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- 172.21.15.179:0/2702430328 shutdown_connections 2024-05-08T22:34:04.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.718+0000 7facdb59e700 1 -- 172.21.15.179:0/2702430328 wait complete. 2024-05-08T22:34:04.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 Processor -- start 2024-05-08T22:34:04.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 -- start start 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facdc15d0c0 0x7facdc14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0cc1b0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc15ad70 con 0x7facdc15d0c0 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0cbb30 con 0x7facdc0d1d20 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38210/0 (socket says 172.21.15.179:38210) 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 -- 172.21.15.179:0/401275969 learned_addr learned my addr 172.21.15.179:0/401275969 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facd9d9b700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facdad9d700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facdc15d0c0 0x7facdc14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 -- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 msgr2=0x7facdc0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 -- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facdc15d0c0 msgr2=0x7facdc14aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facdc15d0c0 0x7facdc14aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 -- 172.21.15.179:0/401275969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc009570 con 0x7facdc0ce0e0 2024-05-08T22:34:04.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7facda59c700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7facdc08b290 tx=0x7faccc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.722+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc004510 con 0x7facdc0ce0e0 2024-05-08T22:34:04.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc01e070 con 0x7facdc0ce0e0 2024-05-08T22:34:04.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc003900 con 0x7facdc0ce0e0 2024-05-08T22:34:04.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc0cb320 con 0x7facdc0ce0e0 2024-05-08T22:34:04.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc0cbdd0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc0cc2f0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.726+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faccc003a60 con 0x7facdc0ce0e0 2024-05-08T22:34:04.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.730+0000 7faccaffd700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 0x7facbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:04.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.730+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faccc0852a0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.730+0000 7facd9d9b700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 0x7facbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:04.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.730+0000 7facd9d9b700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 0x7facbc068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facdc14ed20 tx=0x7facd0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:04.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.734+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faccc0522e0 con 0x7facdc0ce0e0 2024-05-08T22:34:04.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.906+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7facdc04e700 con 0x7facdc0ce0e0 2024-05-08T22:34:04.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.906+0000 7faccaffd700 1 -- 172.21.15.179:0/401275969 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7faccc055900 con 0x7facdc0ce0e0 2024-05-08T22:34:04.908 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 msgr2=0x7facbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 0x7facbc068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facdc14ed20 tx=0x7facd0009240 comp rx=0 tx=0).stop 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 msgr2=0x7facdc15a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7facdc08b290 tx=0x7faccc004770 comp rx=0 tx=0).stop 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 shutdown_connections 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7facbc066730 0x7facbc068bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facdc15d0c0 0x7facdc14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facdc0d1d20 0x7facdc0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 --2- 172.21.15.179:0/401275969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facdc0ce0e0 0x7facdc15a830 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 >> 172.21.15.179:0/401275969 conn(0x7facdc0d2ed0 msgr2=0x7facdc0b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 shutdown_connections 2024-05-08T22:34:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:04.910+0000 7facdb59e700 1 -- 172.21.15.179:0/401275969 wait complete. 2024-05-08T22:34:04.923 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:05.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 Processor -- start 2024-05-08T22:34:05.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 -- start start 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b700ce2a0 0x7f4b7006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b700b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7006ddf0 con 0x7f4b701434d0 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7006f010 con 0x7f4b700ce2a0 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b767c2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700cd200 con 0x7f4b7006eb80 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b757c0700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4b701434d0 0x7f4b7006d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58748/0 (socket says 172.21.15.179:58748) 2024-05-08T22:34:05.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b757c0700 1 -- 172.21.15.179:0/2927984122 learned_addr learned my addr 172.21.15.179:0/2927984122 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b74fbf700 1 --2- 172.21.15.179:0/2927984122 >> v2:172.21.15.195:3300/0 conn(0x7f4b700ce2a0 0x7f4b7006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b75fc1700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b700b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b6ffff700 1 -- 172.21.15.179:0/2927984122 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269751051 0 0) 0x7f4b7006ddf0 con 0x7f4b701434d0 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b6ffff700 1 -- 172.21.15.179:0/2927984122 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b580035d0 con 0x7f4b701434d0 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b74fbf700 1 -- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 msgr2=0x7f4b700b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b74fbf700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b700b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b74fbf700 1 -- 172.21.15.179:0/2927984122 >> v1:172.21.15.179:6789/0 conn(0x7f4b701434d0 legacy=0x7f4b7006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.034+0000 7f4b74fbf700 1 -- 172.21.15.179:0/2927984122 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700cd4e0 con 0x7f4b700ce2a0 2024-05-08T22:34:05.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b74fbf700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7006e640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009850 tx=0x7f4b64009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961149a339b0593d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b6ffff700 1 -- 172.21.15.179:0/2927984122 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64011830 con 0x7f4b700ce2a0 2024-05-08T22:34:05.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b6ffff700 1 -- 172.21.15.179:0/2927984122 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64011990 con 0x7f4b700ce2a0 2024-05-08T22:34:05.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b6ffff700 1 -- 172.21.15.179:0/2927984122 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64011d40 con 0x7f4b700ce2a0 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 msgr2=0x7f4b7006e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7006e640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009850 tx=0x7f4b64009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- 172.21.15.179:0/2927984122 shutdown_connections 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b700b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 --2- 172.21.15.179:0/2927984122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7006e640 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- 172.21.15.179:0/2927984122 >> 172.21.15.179:0/2927984122 conn(0x7f4b700ce9d0 msgr2=0x7f4b700cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- 172.21.15.179:0/2927984122 shutdown_connections 2024-05-08T22:34:05.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- 172.21.15.179:0/2927984122 wait complete. 2024-05-08T22:34:05.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 Processor -- start 2024-05-08T22:34:05.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.038+0000 7f4b767c2700 1 -- start start 2024-05-08T22:34:05.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b70159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70159ad0 con 0x7f4b701434d0 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701ef530 con 0x7f4b700ce2a0 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7013b6c0 con 0x7f4b7006eb80 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38222/0 (socket says 172.21.15.179:38222) 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 -- 172.21.15.179:0/3582072361 learned_addr learned my addr 172.21.15.179:0/3582072361 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b757c0700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b70159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b74fbf700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 -- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 msgr2=0x7f4b70159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b70159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 -- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 msgr2=0x7f4b7015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 -- 172.21.15.179:0/3582072361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640094b0 con 0x7f4b701434d0 2024-05-08T22:34:05.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b75fc1700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800efc0 tx=0x7f4b6800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800cad0 con 0x7f4b701434d0 2024-05-08T22:34:05.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6800f040 con 0x7f4b701434d0 2024-05-08T22:34:05.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7013b9a0 con 0x7f4b701434d0 2024-05-08T22:34:05.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800ce70 con 0x7f4b701434d0 2024-05-08T22:34:05.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7013bf60 con 0x7f4b701434d0 2024-05-08T22:34:05.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.042+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b70067740 con 0x7f4b701434d0 2024-05-08T22:34:05.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.046+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b68014480 con 0x7f4b701434d0 2024-05-08T22:34:05.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.046+0000 7f4b6dffb700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 0x7f4b5c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.046+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4b68084c50 con 0x7f4b701434d0 2024-05-08T22:34:05.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.046+0000 7f4b757c0700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 0x7f4b5c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.046+0000 7f4b757c0700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 0x7f4b5c068c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b60007fd0 tx=0x7f4b60007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.054+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b6808a020 con 0x7f4b701434d0 2024-05-08T22:34:05.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.226+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b7004e700 con 0x7f4b701434d0 2024-05-08T22:34:05.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.230+0000 7f4b6dffb700 1 -- 172.21.15.179:0/3582072361 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f4b68018020 con 0x7f4b701434d0 2024-05-08T22:34:05.231 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:05.231 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:05.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.230+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 msgr2=0x7f4b5c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.230+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 0x7f4b5c068c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b60007fd0 tx=0x7f4b60007de0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.230+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 msgr2=0x7f4b7015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.230+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800efc0 tx=0x7f4b6800c0c0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 shutdown_connections 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b5c066760 0x7f4b5c068c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b700ce2a0 0x7f4b7015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b701434d0 0x7f4b7015b8a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 --2- 172.21.15.179:0/3582072361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b7006eb80 0x7f4b70159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 >> 172.21.15.179:0/3582072361 conn(0x7f4b700ce9d0 msgr2=0x7f4b7006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 shutdown_connections 2024-05-08T22:34:05.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.234+0000 7f4b767c2700 1 -- 172.21.15.179:0/3582072361 wait complete. 2024-05-08T22:34:05.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:05.247 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:05.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 Processor -- start 2024-05-08T22:34:05.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 -- start start 2024-05-08T22:34:05.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fccc80cbf00 0x7fccc815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fccc806c550 0x7fccc806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc806e6c0 con 0x7fccc8159eb0 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc80cc660 con 0x7fccc80cbf00 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fcccd6b7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc806cec0 con 0x7fccc806c550 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc6ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fccc8159eb0 0x7fccc815e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60818/0 (socket says 172.21.15.179:60818) 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc6ffd700 1 -- 172.21.15.179:0/2045554919 learned_addr learned my addr 172.21.15.179:0/2045554919 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 --2- 172.21.15.179:0/2045554919 >> v2:172.21.15.179:3300/0 conn(0x7fccc80cbf00 0x7fccc815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc5ffb700 1 -- 172.21.15.179:0/2045554919 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399964064 0 0) 0x7fccc806e6c0 con 0x7fccc8159eb0 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc77fe700 1 --2- 172.21.15.179:0/2045554919 >> v2:172.21.15.195:3300/0 conn(0x7fccc806c550 0x7fccc806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc5ffb700 1 -- 172.21.15.179:0/2045554919 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb40035d0 con 0x7fccc8159eb0 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 -- 172.21.15.179:0/2045554919 >> v2:172.21.15.195:3300/0 conn(0x7fccc806c550 msgr2=0x7fccc806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 --2- 172.21.15.179:0/2045554919 >> v2:172.21.15.195:3300/0 conn(0x7fccc806c550 0x7fccc806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 -- 172.21.15.179:0/2045554919 >> v1:172.21.15.179:6789/0 conn(0x7fccc8159eb0 legacy=0x7fccc815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 -- 172.21.15.179:0/2045554919 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc806eeb0 con 0x7fccc80cbf00 2024-05-08T22:34:05.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc67fc700 1 --2- 172.21.15.179:0/2045554919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80cbf00 0x7fccc815ada0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fccc8067f30 tx=0x7fccb8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c086907582d53870 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc5ffb700 1 -- 172.21.15.179:0/2045554919 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8011830 con 0x7fccc80cbf00 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.362+0000 7fccc5ffb700 1 -- 172.21.15.179:0/2045554919 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb8011990 con 0x7fccc80cbf00 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fccc5ffb700 1 -- 172.21.15.179:0/2045554919 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8011d00 con 0x7fccc80cbf00 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- 172.21.15.179:0/2045554919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80cbf00 msgr2=0x7fccc815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/2045554919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80cbf00 0x7fccc815ada0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fccc8067f30 tx=0x7fccb8009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- 172.21.15.179:0/2045554919 shutdown_connections 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/2045554919 >> v2:172.21.15.195:3300/0 conn(0x7fccc806c550 0x7fccc806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/2045554919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80cbf00 0x7fccc815ada0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- 172.21.15.179:0/2045554919 >> 172.21.15.179:0/2045554919 conn(0x7fccc80cfdc0 msgr2=0x7fccc80d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- 172.21.15.179:0/2045554919 shutdown_connections 2024-05-08T22:34:05.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- 172.21.15.179:0/2045554919 wait complete. 2024-05-08T22:34:05.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 Processor -- start 2024-05-08T22:34:05.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.366+0000 7fcccd6b7700 1 -- start start 2024-05-08T22:34:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80cbf00 0x7fccc80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 0x7fccc80c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c38f0 con 0x7fccc806c550 2024-05-08T22:34:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80ca080 con 0x7fccc8159eb0 2024-05-08T22:34:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c58f0 con 0x7fccc80cbf00 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58976/0 (socket says 172.21.15.179:58976) 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 -- 172.21.15.179:0/273631193 learned_addr learned my addr 172.21.15.179:0/273631193 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc67fc700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80cbf00 0x7fccc80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc77fe700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 0x7fccc80c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 -- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80cbf00 msgr2=0x7fccc80c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80cbf00 0x7fccc80c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 -- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 msgr2=0x7fccc80c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 0x7fccc80c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 -- 172.21.15.179:0/273631193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80094b0 con 0x7fccc806c550 2024-05-08T22:34:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc77fe700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 0x7fccc80c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccc6ffd700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fccb000c4d0 tx=0x7fccb000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0005d90 con 0x7fccc806c550 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb00070d0 con 0x7fccc806c550 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc81f3660 con 0x7fccc806c550 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007440 con 0x7fccc806c550 2024-05-08T22:34:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc81f3c20 con 0x7fccc806c550 2024-05-08T22:34:05.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8073290 con 0x7fccc806c550 2024-05-08T22:34:05.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.370+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccb00075a0 con 0x7fccc806c550 2024-05-08T22:34:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.374+0000 7fccbf7fe700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 0x7fcca8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.374+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fccb0083830 con 0x7fccc806c550 2024-05-08T22:34:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.374+0000 7fccc67fc700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 0x7fcca8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.374+0000 7fccc67fc700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 0x7fcca8068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.378+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fccb0050880 con 0x7fccc806c550 2024-05-08T22:34:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.550+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccc815a320 con 0x7fccc806c550 2024-05-08T22:34:05.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.550+0000 7fccbf7fe700 1 -- 172.21.15.179:0/273631193 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccb0017020 con 0x7fccc806c550 2024-05-08T22:34:05.551 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.550+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 msgr2=0x7fcca8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.550+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 0x7fcca8068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb8008030 comp rx=0 tx=0).stop 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 msgr2=0x7fccc81f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fccb000c4d0 tx=0x7fccb000c7e0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 shutdown_connections 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcca8066730 0x7fcca8068bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8159eb0 0x7fccc80c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80cbf00 0x7fccc80c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 --2- 172.21.15.179:0/273631193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc806c550 0x7fccc81f42b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 >> 172.21.15.179:0/273631193 conn(0x7fccc80cfdc0 msgr2=0x7fccc80c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 shutdown_connections 2024-05-08T22:34:05.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.554+0000 7fcccd6b7700 1 -- 172.21.15.179:0/273631193 wait complete. 2024-05-08T22:34:05.566 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:05.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.678+0000 7efd35541700 1 Processor -- start 2024-05-08T22:34:05.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.678+0000 7efd35541700 1 -- start start 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 0x7efd300b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd3006d4b0 0x7efd3015d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300b22d0 con 0x7efd300b2ca0 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300b19e0 con 0x7efd3006d4b0 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3015e640 con 0x7efd30142890 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd3006d4b0 0x7efd3015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd3006d4b0 0x7efd3015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58992/0 (socket says 172.21.15.179:58992) 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 -- 172.21.15.179:0/3626573498 learned_addr learned my addr 172.21.15.179:0/3626573498 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd2ffff700 1 --2- 172.21.15.179:0/3626573498 >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 0x7efd300b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 -- 172.21.15.179:0/3626573498 >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 msgr2=0x7efd300b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 --2- 172.21.15.179:0/3626573498 >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 0x7efd300b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 -- 172.21.15.179:0/3626573498 >> v1:172.21.15.195:6789/0 conn(0x7efd300b2ca0 legacy=0x7efd3006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 -- 172.21.15.179:0/3626573498 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300b1c80 con 0x7efd3006d4b0 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd2ffff700 1 --2- 172.21.15.179:0/3626573498 >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 0x7efd300b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd34d40700 1 --2- 172.21.15.179:0/3626573498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd3006d4b0 0x7efd3015d830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd300720b0 tx=0x7efd1800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f6b4280b84efd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd2effd700 1 -- 172.21.15.179:0/3626573498 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800dd50 con 0x7efd3006d4b0 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd2effd700 1 -- 172.21.15.179:0/3626573498 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18004030 con 0x7efd3006d4b0 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd2effd700 1 -- 172.21.15.179:0/3626573498 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd180043c0 con 0x7efd3006d4b0 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- 172.21.15.179:0/3626573498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd3006d4b0 msgr2=0x7efd3015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 --2- 172.21.15.179:0/3626573498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd3006d4b0 0x7efd3015d830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd300720b0 tx=0x7efd1800be90 comp rx=0 tx=0).stop 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- 172.21.15.179:0/3626573498 shutdown_connections 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 --2- 172.21.15.179:0/3626573498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd3006d4b0 0x7efd3015d830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 --2- 172.21.15.179:0/3626573498 >> v2:172.21.15.195:3300/0 conn(0x7efd30142890 0x7efd300b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- 172.21.15.179:0/3626573498 >> 172.21.15.179:0/3626573498 conn(0x7efd300cea30 msgr2=0x7efd300cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- 172.21.15.179:0/3626573498 shutdown_connections 2024-05-08T22:34:05.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.682+0000 7efd35541700 1 -- 172.21.15.179:0/3626573498 wait complete. 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 Processor -- start 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- start start 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 0x7efd30141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 0x7efd3014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd30151c30 con 0x7efd300b2ca0 2024-05-08T22:34:05.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd30140aa0 con 0x7efd30142890 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd30140d80 con 0x7efd3006d4b0 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58994/0 (socket says 172.21.15.179:58994) 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 -- 172.21.15.179:0/281366391 learned_addr learned my addr 172.21.15.179:0/281366391 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2ffff700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 0x7efd30141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 -- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 msgr2=0x7efd30141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd34d40700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 0x7efd3014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 0x7efd30141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 -- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 msgr2=0x7efd3014e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 0x7efd3014e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 -- 172.21.15.179:0/281366391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1800b810 con 0x7efd300b2ca0 2024-05-08T22:34:05.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd34d40700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 0x7efd3014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2ffff700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 0x7efd30141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2f7fe700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c4e0 tx=0x7efd2000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20011790 con 0x7efd300b2ca0 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20011910 con 0x7efd300b2ca0 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd30141060 con 0x7efd300b2ca0 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20011cc0 con 0x7efd300b2ca0 2024-05-08T22:34:05.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.686+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd30141620 con 0x7efd300b2ca0 2024-05-08T22:34:05.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30141840 con 0x7efd300b2ca0 2024-05-08T22:34:05.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd2001c460 con 0x7efd300b2ca0 2024-05-08T22:34:05.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd2cff9700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 0x7efd14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efd20084350 con 0x7efd300b2ca0 2024-05-08T22:34:05.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd2ffff700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 0x7efd14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.690+0000 7efd2ffff700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 0x7efd14068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd28007f40 tx=0x7efd28007490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:05.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.698+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efd20051300 con 0x7efd300b2ca0 2024-05-08T22:34:05.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.862+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd3004e700 con 0x7efd300b2ca0 2024-05-08T22:34:05.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.862+0000 7efd2cff9700 1 -- 172.21.15.179:0/281366391 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd20054920 con 0x7efd300b2ca0 2024-05-08T22:34:05.864 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 msgr2=0x7efd14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 0x7efd14068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd28007f40 tx=0x7efd28007490 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 msgr2=0x7efd301516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c4e0 tx=0x7efd2000c8a0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 shutdown_connections 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efd14066730 0x7efd14068bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd30142890 0x7efd3014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300b2ca0 0x7efd301516f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 --2- 172.21.15.179:0/281366391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3006d4b0 0x7efd30141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 >> 172.21.15.179:0/281366391 conn(0x7efd300cea30 msgr2=0x7efd3006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 shutdown_connections 2024-05-08T22:34:05.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.866+0000 7efd35541700 1 -- 172.21.15.179:0/281366391 wait complete. 2024-05-08T22:34:05.879 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:05.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.994+0000 7ff6f9901700 1 Processor -- start 2024-05-08T22:34:05.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.994+0000 7ff6f9901700 1 -- start start 2024-05-08T22:34:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.994+0000 7ff6f9901700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.994+0000 7ff6f9901700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6f40ce2b0 0x7ff6f406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.994+0000 7ff6f9901700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f415d4a0 con 0x7ff6f40d1d20 2024-05-08T22:34:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f406abb0 con 0x7ff6f40ce2b0 2024-05-08T22:34:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40d2500 con 0x7ff6f415d0c0 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f3fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ff6f40d1d20 0x7ff6f40cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60776/0 (socket says 172.21.15.179:60776) 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9100700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6f40ce2b0 0x7ff6f406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f3fff700 1 -- 172.21.15.179:0/248542901 learned_addr learned my addr 172.21.15.179:0/248542901 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 -- 172.21.15.179:0/248542901 >> v2:172.21.15.195:3300/0 conn(0x7ff6f40ce2b0 msgr2=0x7ff6f406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 --2- 172.21.15.179:0/248542901 >> v2:172.21.15.195:3300/0 conn(0x7ff6f40ce2b0 0x7ff6f406b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 -- 172.21.15.179:0/248542901 >> v1:172.21.15.195:6789/0 conn(0x7ff6f40d1d20 legacy=0x7ff6f40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f37fe700 1 -- 172.21.15.179:0/248542901 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866126848 0 0) 0x7ff6f415d4a0 con 0x7ff6f40d1d20 2024-05-08T22:34:05.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 -- 172.21.15.179:0/248542901 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40d27e0 con 0x7ff6f415d0c0 2024-05-08T22:34:06.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f88ff700 1 --2- 172.21.15.179:0/248542901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f40b8a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009850 tx=0x7ff6e8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651a6297ba15b63e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f37fe700 1 -- 172.21.15.179:0/248542901 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011830 con 0x7ff6f415d0c0 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f37fe700 1 -- 172.21.15.179:0/248542901 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8011990 con 0x7ff6f415d0c0 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f37fe700 1 -- 172.21.15.179:0/248542901 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011d40 con 0x7ff6f415d0c0 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- 172.21.15.179:0/248542901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 msgr2=0x7ff6f40b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 --2- 172.21.15.179:0/248542901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f40b8a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009850 tx=0x7ff6e8009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- 172.21.15.179:0/248542901 shutdown_connections 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 --2- 172.21.15.179:0/248542901 >> v2:172.21.15.195:3300/0 conn(0x7ff6f40ce2b0 0x7ff6f406b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 --2- 172.21.15.179:0/248542901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f40b8a80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- 172.21.15.179:0/248542901 >> 172.21.15.179:0/248542901 conn(0x7ff6f40d2ed0 msgr2=0x7ff6f40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- 172.21.15.179:0/248542901 shutdown_connections 2024-05-08T22:34:06.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:05.998+0000 7ff6f9901700 1 -- 172.21.15.179:0/248542901 wait complete. 2024-05-08T22:34:06.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 Processor -- start 2024-05-08T22:34:06.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 -- start start 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6f40ce2b0 0x7ff6f41522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f4156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f41f24d0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40cc8f0 con 0x7ff6f40ce2b0 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9901700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f41462f0 con 0x7ff6f415d0c0 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59000/0 (socket says 172.21.15.179:59000) 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 -- 172.21.15.179:0/597392597 learned_addr learned my addr 172.21.15.179:0/597392597 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f88ff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6f40ce2b0 0x7ff6f41522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f9100700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f4156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 -- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 msgr2=0x7ff6f4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f4156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 -- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6f40ce2b0 msgr2=0x7ff6f41522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6f40ce2b0 0x7ff6f41522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.002+0000 7ff6f3fff700 1 -- 172.21.15.179:0/597392597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80094b0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f3fff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c580 tx=0x7ff6e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4004ca0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4004e20 con 0x7ff6f40d1d20 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f41465d0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4007310 con 0x7ff6f40d1d20 2024-05-08T22:34:06.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f4146b90 con 0x7ff6f40d1d20 2024-05-08T22:34:06.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f4146d80 con 0x7ff6f40d1d20 2024-05-08T22:34:06.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.006+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6e4007470 con 0x7ff6f40d1d20 2024-05-08T22:34:06.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.010+0000 7ff6f17fa700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 0x7ff6e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.010+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff6e40840b0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.010+0000 7ff6f88ff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 0x7ff6e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.010+0000 7ff6f88ff700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 0x7ff6e0068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80090b0 tx=0x7ff6e801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.014+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff6e40510f0 con 0x7ff6f40d1d20 2024-05-08T22:34:06.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6f404e700 con 0x7ff6f40d1d20 2024-05-08T22:34:06.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f17fa700 1 -- 172.21.15.179:0/597392597 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6e4007720 con 0x7ff6f40d1d20 2024-05-08T22:34:06.182 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 msgr2=0x7ff6e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 0x7ff6e0068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80090b0 tx=0x7ff6e801c040 comp rx=0 tx=0).stop 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 msgr2=0x7ff6f40cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c580 tx=0x7ff6e400c890 comp rx=0 tx=0).stop 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 shutdown_connections 2024-05-08T22:34:06.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff6e0066730 0x7ff6e0068bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6f415d0c0 0x7ff6f4156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f40d1d20 0x7ff6f40cbec0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 --2- 172.21.15.179:0/597392597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6f40ce2b0 0x7ff6f41522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 >> 172.21.15.179:0/597392597 conn(0x7ff6f40d2ed0 msgr2=0x7ff6f40b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 shutdown_connections 2024-05-08T22:34:06.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.182+0000 7ff6f9901700 1 -- 172.21.15.179:0/597392597 wait complete. 2024-05-08T22:34:06.198 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:06.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 Processor -- start 2024-05-08T22:34:06.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 -- start start 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ab015d0c0 0x7f2ab00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ab00d1d20 0x7f2ab00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00ce0e0 0x7f2ab0160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00ce570 con 0x7f2ab00d1d20 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab015d4a0 con 0x7f2ab015d0c0 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab5603700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab006abb0 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ab00d1d20 0x7f2ab00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ab00d1d20 0x7f2ab00cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59010/0 (socket says 172.21.15.179:59010) 2024-05-08T22:34:06.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 -- 172.21.15.179:0/2900734195 learned_addr learned my addr 172.21.15.179:0/2900734195 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2ab4e02700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00ce0e0 0x7f2ab0160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaffff700 1 --2- 172.21.15.179:0/2900734195 >> v2:172.21.15.195:3300/0 conn(0x7f2ab015d0c0 0x7f2ab00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 -- 172.21.15.179:0/2900734195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00ce0e0 msgr2=0x7f2ab0160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00ce0e0 0x7f2ab0160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 -- 172.21.15.179:0/2900734195 >> v2:172.21.15.195:3300/0 conn(0x7f2ab015d0c0 msgr2=0x7f2ab00b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/2900734195 >> v2:172.21.15.195:3300/0 conn(0x7f2ab015d0c0 0x7f2ab00b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.306+0000 7f2aaf7fe700 1 -- 172.21.15.179:0/2900734195 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00d2500 con 0x7f2ab00d1d20 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00d1d20 0x7f2ab00cdba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ab008b1c0 tx=0x7f2aa0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a3f452716d3d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2aaeffd700 1 -- 172.21.15.179:0/2900734195 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000b8a0 con 0x7f2ab00d1d20 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2aaeffd700 1 -- 172.21.15.179:0/2900734195 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa000ba00 con 0x7f2ab00d1d20 2024-05-08T22:34:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2aaeffd700 1 -- 172.21.15.179:0/2900734195 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000bd90 con 0x7f2ab00d1d20 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 -- 172.21.15.179:0/2900734195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00d1d20 msgr2=0x7f2ab00cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00d1d20 0x7f2ab00cdba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ab008b1c0 tx=0x7f2aa0009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 -- 172.21.15.179:0/2900734195 shutdown_connections 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00ce0e0 0x7f2ab0160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 --2- 172.21.15.179:0/2900734195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00d1d20 0x7f2ab00cdba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 --2- 172.21.15.179:0/2900734195 >> v2:172.21.15.195:3300/0 conn(0x7f2ab015d0c0 0x7f2ab00b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 -- 172.21.15.179:0/2900734195 >> 172.21.15.179:0/2900734195 conn(0x7f2ab00d2ed0 msgr2=0x7f2ab00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 -- 172.21.15.179:0/2900734195 shutdown_connections 2024-05-08T22:34:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.310+0000 7f2ab5603700 1 -- 172.21.15.179:0/2900734195 wait complete. 2024-05-08T22:34:06.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 Processor -- start 2024-05-08T22:34:06.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- start start 2024-05-08T22:34:06.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00d1d20 0x7f2ab014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 0x7f2ab014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab01f6470 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab014f260 con 0x7f2ab015d0c0 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab014d130 con 0x7f2ab00d1d20 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59016/0 (socket says 172.21.15.179:59016) 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 -- 172.21.15.179:0/1510683448 learned_addr learned my addr 172.21.15.179:0/1510683448 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab4e02700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 0x7f2ab014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00d1d20 0x7f2ab014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 -- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00d1d20 msgr2=0x7f2ab014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00d1d20 0x7f2ab014e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 -- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 msgr2=0x7f2ab014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 0x7f2ab014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 -- 172.21.15.179:0/1510683448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0009570 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab4e02700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 0x7f2ab014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:06.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aaffff700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e760 tx=0x7f2aa400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400c070 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4013610 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab014ef80 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4013980 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.314+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab014d6b0 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.318+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2aa4013ae0 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.318+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00d2a80 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.318+0000 7f2aacff9700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 0x7f2a9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.318+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2aa40841a0 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.322+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 0x7f2a9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.322+0000 7f2aaf7fe700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 0x7f2a9c068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.326+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2aa40511e0 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.494+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2ab004e700 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.498+0000 7f2aacff9700 1 -- 172.21.15.179:0/1510683448 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2aa4054800 con 0x7f2ab00ce0e0 2024-05-08T22:34:06.498 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 msgr2=0x7f2a9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 0x7f2a9c068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa0008030 comp rx=0 tx=0).stop 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 msgr2=0x7f2ab01f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e760 tx=0x7f2aa400ea70 comp rx=0 tx=0).stop 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 shutdown_connections 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a9c066760 0x7f2a9c068c20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ab015d0c0 0x7f2ab014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ab00d1d20 0x7f2ab014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 --2- 172.21.15.179:0/1510683448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab00ce0e0 0x7f2ab01f5f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 >> 172.21.15.179:0/1510683448 conn(0x7f2ab00d2ed0 msgr2=0x7f2ab01527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 shutdown_connections 2024-05-08T22:34:06.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.502+0000 7f2ab5603700 1 -- 172.21.15.179:0/1510683448 wait complete. 2024-05-08T22:34:06.539 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:34:06.540 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:06.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 Processor -- start 2024-05-08T22:34:06.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 -- start start 2024-05-08T22:34:06.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b840cc040 0x7f8b840cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 0x7f8b840b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b840cf630 con 0x7f8b8415b000 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8415b3e0 con 0x7f8b840cc040 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b889ff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b840cf910 con 0x7f8b840d06d0 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b82d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8b8415b000 0x7f8b840cb930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60826/0 (socket says 172.21.15.179:60826) 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b82d9d700 1 -- 172.21.15.179:0/3875360198 learned_addr learned my addr 172.21.15.179:0/3875360198 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/3875360198 >> v2:172.21.15.179:3300/0 conn(0x7f8b840cc040 0x7f8b840cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b8359e700 1 --2- 172.21.15.179:0/3875360198 >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 0x7f8b840b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b8259c700 1 -- 172.21.15.179:0/3875360198 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646009834 0 0) 0x7f8b840cf630 con 0x7f8b8415b000 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.654+0000 7f8b8259c700 1 -- 172.21.15.179:0/3875360198 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b700035d0 con 0x7f8b8415b000 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b7bfff700 1 -- 172.21.15.179:0/3875360198 >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 msgr2=0x7f8b840b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/3875360198 >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 0x7f8b840b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b7bfff700 1 -- 172.21.15.179:0/3875360198 >> v1:172.21.15.179:6789/0 conn(0x7f8b8415b000 legacy=0x7f8b840cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b7bfff700 1 -- 172.21.15.179:0/3875360198 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840cfbf0 con 0x7f8b840cc040 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b8359e700 1 --2- 172.21.15.179:0/3875360198 >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 0x7f8b840b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/3875360198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b840cefc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cf560 tx=0x7f8b6c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a71f2e8f447cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b8259c700 1 -- 172.21.15.179:0/3875360198 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011830 con 0x7f8b840cc040 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b8259c700 1 -- 172.21.15.179:0/3875360198 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c011990 con 0x7f8b840cc040 2024-05-08T22:34:06.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b8259c700 1 -- 172.21.15.179:0/3875360198 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011d20 con 0x7f8b840cc040 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 -- 172.21.15.179:0/3875360198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 msgr2=0x7f8b840cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 --2- 172.21.15.179:0/3875360198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b840cefc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cf560 tx=0x7f8b6c009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 -- 172.21.15.179:0/3875360198 shutdown_connections 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 --2- 172.21.15.179:0/3875360198 >> v2:172.21.15.195:3300/0 conn(0x7f8b840d06d0 0x7f8b840b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 --2- 172.21.15.179:0/3875360198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b840cefc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 -- 172.21.15.179:0/3875360198 >> 172.21.15.179:0/3875360198 conn(0x7f8b840b2780 msgr2=0x7f8b840b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 -- 172.21.15.179:0/3875360198 shutdown_connections 2024-05-08T22:34:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.658+0000 7f8b889ff700 1 -- 172.21.15.179:0/3875360198 wait complete. 2024-05-08T22:34:06.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 Processor -- start 2024-05-08T22:34:06.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 -- start start 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b840d06d0 0x7f8b841573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8415b000 0x7f8b84155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84154710 con 0x7f8b840cc040 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841f0fc0 con 0x7f8b8415b000 2024-05-08T22:34:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b889ff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841f12a0 con 0x7f8b840d06d0 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59030/0 (socket says 172.21.15.179:59030) 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 -- 172.21.15.179:0/1482438434 learned_addr learned my addr 172.21.15.179:0/1482438434 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b8359e700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8415b000 0x7f8b84155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b840d06d0 0x7f8b841573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 -- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b840d06d0 msgr2=0x7f8b841573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b840d06d0 0x7f8b841573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 -- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8415b000 msgr2=0x7f8b84155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.662+0000 7f8b82d9d700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8415b000 0x7f8b84155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b82d9d700 1 -- 172.21.15.179:0/1482438434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c009a70 con 0x7f8b840cc040 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b82d9d700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c4d0 tx=0x7f8b7400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74005d90 con 0x7f8b840cc040 2024-05-08T22:34:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b740070d0 con 0x7f8b840cc040 2024-05-08T22:34:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c0094b0 con 0x7f8b840cc040 2024-05-08T22:34:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74007480 con 0x7f8b840cc040 2024-05-08T22:34:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b841f1b40 con 0x7f8b840cc040 2024-05-08T22:34:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b840cedd0 con 0x7f8b840cc040 2024-05-08T22:34:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b740075e0 con 0x7f8b840cc040 2024-05-08T22:34:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 0x7f8b64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8b740838b0 con 0x7f8b840cc040 2024-05-08T22:34:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.666+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 0x7f8b64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.670+0000 7f8b7bfff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 0x7f8b64068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b8415e9c0 tx=0x7f8b6c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.674+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b740508f0 con 0x7f8b840cc040 2024-05-08T22:34:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.850+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b840b23c0 con 0x7f8b840cc040 2024-05-08T22:34:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.850+0000 7f8b7b7fe700 1 -- 172.21.15.179:0/1482438434 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f8b74053f10 con 0x7f8b840cc040 2024-05-08T22:34:06.852 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:06.852 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 msgr2=0x7f8b64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 0x7f8b64068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b8415e9c0 tx=0x7f8b6c008030 comp rx=0 tx=0).stop 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 msgr2=0x7f8b84158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c4d0 tx=0x7f8b7400c7e0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 shutdown_connections 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b64066730 0x7f8b64068bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8415b000 0x7f8b84155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b840d06d0 0x7f8b841573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 --2- 172.21.15.179:0/1482438434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840cc040 0x7f8b84158060 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 >> 172.21.15.179:0/1482438434 conn(0x7f8b840b2780 msgr2=0x7f8b8406e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 shutdown_connections 2024-05-08T22:34:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.854+0000 7f8b889ff700 1 -- 172.21.15.179:0/1482438434 wait complete. 2024-05-08T22:34:06.873 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:06.873 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:06.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 Processor -- start 2024-05-08T22:34:06.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 -- start start 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11e415d150 0x7f11e406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e40c5ed0 0x7f11e406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e406f500 con 0x7f11e40ae420 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e40ae900 con 0x7f11e415d150 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.986+0000 7f11ea92a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40b1040 con 0x7f11e40c5ed0 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea129700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f11e40ae420 0x7f11e40c4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60832/0 (socket says 172.21.15.179:60832) 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11e415d150 0x7f11e406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea129700 1 -- 172.21.15.179:0/2250118597 learned_addr learned my addr 172.21.15.179:0/2250118597 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9127700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e40c5ed0 0x7f11e406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 -- 172.21.15.179:0/2250118597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e40c5ed0 msgr2=0x7f11e406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:06.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e40c5ed0 0x7f11e406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 -- 172.21.15.179:0/2250118597 >> v1:172.21.15.179:6789/0 conn(0x7f11e40ae420 legacy=0x7f11e40c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e8926700 1 -- 172.21.15.179:0/2250118597 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598330897 0 0) 0x7f11e406f500 con 0x7f11e40ae420 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 -- 172.21.15.179:0/2250118597 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40b1320 con 0x7f11e415d150 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e9928700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e415d150 0x7f11e406ea80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009850 tx=0x7f11e0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e179cac2801ab03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e8926700 1 -- 172.21.15.179:0/2250118597 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0011830 con 0x7f11e415d150 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e8926700 1 -- 172.21.15.179:0/2250118597 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e0011990 con 0x7f11e415d150 2024-05-08T22:34:06.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11e8926700 1 -- 172.21.15.179:0/2250118597 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0011d20 con 0x7f11e415d150 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 -- 172.21.15.179:0/2250118597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e415d150 msgr2=0x7f11e406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e415d150 0x7f11e406ea80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009850 tx=0x7f11e0009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 -- 172.21.15.179:0/2250118597 shutdown_connections 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e40c5ed0 0x7f11e406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 --2- 172.21.15.179:0/2250118597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e415d150 0x7f11e406ea80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 -- 172.21.15.179:0/2250118597 >> 172.21.15.179:0/2250118597 conn(0x7f11e4098a70 msgr2=0x7f11e40cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 -- 172.21.15.179:0/2250118597 shutdown_connections 2024-05-08T22:34:06.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.990+0000 7f11ea92a700 1 -- 172.21.15.179:0/2250118597 wait complete. 2024-05-08T22:34:06.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 Processor -- start 2024-05-08T22:34:06.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 -- start start 2024-05-08T22:34:06.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e40ae420 0x7f11e41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 0x7f11e40ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:06.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4152810 con 0x7f11e40c5ed0 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e406b4f0 con 0x7f11e40ae420 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e406a450 con 0x7f11e415d150 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59050/0 (socket says 172.21.15.179:59050) 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 -- 172.21.15.179:0/277328824 learned_addr learned my addr 172.21.15.179:0/277328824 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9928700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e40ae420 0x7f11e41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea129700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 0x7f11e40ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 -- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 msgr2=0x7f11e40ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 0x7f11e40ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 -- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e40ae420 msgr2=0x7f11e41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:06.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e40ae420 0x7f11e41522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 -- 172.21.15.179:0/277328824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00094b0 con 0x7f11e40c5ed0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea129700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 0x7f11e40ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11e9127700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e760 tx=0x7f11d800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8013610 con 0x7f11e40c5ed0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8013770 con 0x7f11e40c5ed0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8013ae0 con 0x7f11e40c5ed0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.994+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e406a730 con 0x7f11e40c5ed0 2024-05-08T22:34:06.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.998+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e406acf0 con 0x7f11e40c5ed0 2024-05-08T22:34:07.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.998+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11d8013c40 con 0x7f11e40c5ed0 2024-05-08T22:34:07.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.998+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e404e700 con 0x7f11e40c5ed0 2024-05-08T22:34:07.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:06.998+0000 7f11d67fc700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 0x7f11c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.002+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f11d80840e0 con 0x7f11e40c5ed0 2024-05-08T22:34:07.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.002+0000 7f11e9928700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 0x7f11c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.002+0000 7f11e9928700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 0x7f11c8068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11e0014040 tx=0x7f11e0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.010+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11d8051120 con 0x7f11e40c5ed0 2024-05-08T22:34:07.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.178+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11e40cf2b0 con 0x7f11e40c5ed0 2024-05-08T22:34:07.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.178+0000 7f11d67fc700 1 -- 172.21.15.179:0/277328824 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11d8054740 con 0x7f11e40c5ed0 2024-05-08T22:34:07.181 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 msgr2=0x7f11c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 0x7f11c8068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11e0014040 tx=0x7f11e0008030 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 msgr2=0x7f11e40acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e760 tx=0x7f11d800ea70 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 shutdown_connections 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11c8066730 0x7f11c8068bf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11e415d150 0x7f11e40ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11e40c5ed0 0x7f11e40acee0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 --2- 172.21.15.179:0/277328824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11e40ae420 0x7f11e41522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 >> 172.21.15.179:0/277328824 conn(0x7f11e4098a70 msgr2=0x7f11e40c56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 shutdown_connections 2024-05-08T22:34:07.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.182+0000 7f11ea92a700 1 -- 172.21.15.179:0/277328824 wait complete. 2024-05-08T22:34:07.196 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:07.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 Processor -- start 2024-05-08T22:34:07.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 -- start start 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58a015d0c0 0x7f58a00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58a00d1d20 0x7f58a00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a015d4a0 con 0x7f58a00ce0e0 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.310+0000 7f589f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00ce570 con 0x7f58a00d1d20 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a006abb0 con 0x7f58a015d0c0 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58a00d1d20 0x7f58a00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58a00d1d20 0x7f58a00cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59064/0 (socket says 172.21.15.179:59064) 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 -- 172.21.15.179:0/154961492 learned_addr learned my addr 172.21.15.179:0/154961492 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589e59c700 1 --2- 172.21.15.179:0/154961492 >> v2:172.21.15.195:3300/0 conn(0x7f58a015d0c0 0x7f58a00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 -- 172.21.15.179:0/154961492 >> v2:172.21.15.195:3300/0 conn(0x7f58a015d0c0 msgr2=0x7f58a00b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 --2- 172.21.15.179:0/154961492 >> v2:172.21.15.195:3300/0 conn(0x7f58a015d0c0 0x7f58a00b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 -- 172.21.15.179:0/154961492 >> v1:172.21.15.195:6789/0 conn(0x7f58a00ce0e0 legacy=0x7f58a0160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 -- 172.21.15.179:0/154961492 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00d2500 con 0x7f58a00d1d20 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589dd9b700 1 --2- 172.21.15.179:0/154961492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a00d1d20 0x7f58a00cdba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58a008b1c0 tx=0x7f5890009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc121f7fa38770e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589d59a700 1 -- 172.21.15.179:0/154961492 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000b8a0 con 0x7f58a00d1d20 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589d59a700 1 -- 172.21.15.179:0/154961492 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000ba00 con 0x7f58a00d1d20 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589d59a700 1 -- 172.21.15.179:0/154961492 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000bd90 con 0x7f58a00d1d20 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- 172.21.15.179:0/154961492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a00d1d20 msgr2=0x7f58a00cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 --2- 172.21.15.179:0/154961492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a00d1d20 0x7f58a00cdba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58a008b1c0 tx=0x7f5890009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- 172.21.15.179:0/154961492 shutdown_connections 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 --2- 172.21.15.179:0/154961492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a00d1d20 0x7f58a00cdba0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 --2- 172.21.15.179:0/154961492 >> v2:172.21.15.195:3300/0 conn(0x7f58a015d0c0 0x7f58a00b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- 172.21.15.179:0/154961492 >> 172.21.15.179:0/154961492 conn(0x7f58a00d2ed0 msgr2=0x7f58a00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- 172.21.15.179:0/154961492 shutdown_connections 2024-05-08T22:34:07.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.314+0000 7f589f59e700 1 -- 172.21.15.179:0/154961492 wait complete. 2024-05-08T22:34:07.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 Processor -- start 2024-05-08T22:34:07.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- start start 2024-05-08T22:34:07.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a00ce0e0 0x7f58a00cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a00d1d20 0x7f58a00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cc0a0 con 0x7f58a015d0c0 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cb1e0 con 0x7f58a00d1d20 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cb490 con 0x7f58a00ce0e0 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59066/0 (socket says 172.21.15.179:59066) 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 -- 172.21.15.179:0/2818478781 learned_addr learned my addr 172.21.15.179:0/2818478781 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589e59c700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a00ce0e0 0x7f58a00cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589dd9b700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a00d1d20 0x7f58a00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 -- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a00ce0e0 msgr2=0x7f58a00cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a00ce0e0 0x7f58a00cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 -- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a00d1d20 msgr2=0x7f58a00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a00d1d20 0x7f58a00cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 -- 172.21.15.179:0/2818478781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890009570 con 0x7f58a015d0c0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589ed9d700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f589800cc50 tx=0x7f589800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800ad80 con 0x7f58a015d0c0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589800ce50 con 0x7f58a015d0c0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003c60 con 0x7f58a015d0c0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00cb770 con 0x7f58a015d0c0 2024-05-08T22:34:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00cbd30 con 0x7f58a015d0c0 2024-05-08T22:34:07.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.318+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a00cc210 con 0x7f58a015d0c0 2024-05-08T22:34:07.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.322+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58980050d0 con 0x7f58a015d0c0 2024-05-08T22:34:07.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.322+0000 7f588effd700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 0x7f5880068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.322+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f58980845a0 con 0x7f58a015d0c0 2024-05-08T22:34:07.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.322+0000 7f589e59c700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 0x7f5880068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.322+0000 7f589e59c700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 0x7f5880068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f589400b9d0 tx=0x7f5894008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.326+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f58980515e0 con 0x7f58a015d0c0 2024-05-08T22:34:07.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.494+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58a004e700 con 0x7f58a015d0c0 2024-05-08T22:34:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.494+0000 7f588effd700 1 -- 172.21.15.179:0/2818478781 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5898054c00 con 0x7f58a015d0c0 2024-05-08T22:34:07.498 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 msgr2=0x7f5880068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 0x7f5880068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f589400b9d0 tx=0x7f5894008040 comp rx=0 tx=0).stop 2024-05-08T22:34:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 msgr2=0x7f58a00caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f589800cc50 tx=0x7f589800a340 comp rx=0 tx=0).stop 2024-05-08T22:34:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 shutdown_connections 2024-05-08T22:34:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5880066730 0x7f5880068bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a00d1d20 0x7f58a00cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58a015d0c0 0x7f58a00caca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 --2- 172.21.15.179:0/2818478781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a00ce0e0 0x7f58a00cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 >> 172.21.15.179:0/2818478781 conn(0x7f58a00d2ed0 msgr2=0x7f58a00b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 shutdown_connections 2024-05-08T22:34:07.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.502+0000 7f589f59e700 1 -- 172.21.15.179:0/2818478781 wait complete. 2024-05-08T22:34:07.517 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:07.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 Processor -- start 2024-05-08T22:34:07.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 -- start start 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab4167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40d1fe0 con 0x7feab415d2f0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40d22c0 con 0x7feab40b4ff0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feabab84700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab415d6d0 con 0x7feab40d1aa0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9b82700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7feab415d2f0 0x7feab40b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60834/0 (socket says 172.21.15.179:60834) 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9b82700 1 -- 172.21.15.179:0/1442586351 learned_addr learned my addr 172.21.15.179:0/1442586351 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feaba383700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab4167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136255866 0 0) 0x7feab40d1fe0 con 0x7feab415d2f0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa00035d0 con 0x7feab415d2f0 2024-05-08T22:34:07.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641530721 0 0) 0x7feab40d22c0 con 0x7feab40b4ff0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab40d1fe0 con 0x7feab40b4ff0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268744051 0 0) 0x7feaa00035d0 con 0x7feab415d2f0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 msgr2=0x7feab4167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab4167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 >> v1:172.21.15.195:6789/0 conn(0x7feab40b4ff0 legacy=0x7feab40d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40d25a0 con 0x7feab415d2f0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7feaa8001670 con 0x7feab415d2f0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 >> v1:172.21.15.179:6789/0 conn(0x7feab415d2f0 legacy=0x7feab40b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feaa0003be0 0x7feaa0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feaa0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa0004db0 0x7feaa0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab415d6d0 con 0x7feaa0004db0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa000bb30 con 0x7feaa0003be0 2024-05-08T22:34:07.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa000bcf0 con 0x7feab40d1aa0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feaa0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feaba383700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa0004db0 0x7feaa0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9b82700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feaa0003be0 0x7feaa0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 -- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feaa0003be0 msgr2=0x7feaa0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feaa0003be0 0x7feaa0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 -- 172.21.15.179:0/1442586351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa0004db0 msgr2=0x7feaa0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa0004db0 0x7feaa0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.638+0000 7feab9381700 1 -- 172.21.15.179:0/1442586351 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00035d0 con 0x7feab40d1aa0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feaba383700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa0004db0 0x7feaa0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feab9381700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feaa0004870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feab00019c0 tx=0x7feaa00058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f11b5401ade6d662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0001480 con 0x7feab40d1aa0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab00052d0 con 0x7feab40d1aa0 2024-05-08T22:34:07.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feab8b80700 1 -- 172.21.15.179:0/1442586351 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab00036f0 con 0x7feab40d1aa0 2024-05-08T22:34:07.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 -- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 msgr2=0x7feaa0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 --2- 172.21.15.179:0/1442586351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feaa0004870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feab00019c0 tx=0x7feaa00058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feaba383700 1 -- 172.21.15.179:0/1442586351 reap_dead start 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 -- 172.21.15.179:0/1442586351 shutdown_connections 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 -- 172.21.15.179:0/1442586351 >> 172.21.15.179:0/1442586351 conn(0x7feab40cfac0 msgr2=0x7feab40cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 -- 172.21.15.179:0/1442586351 shutdown_connections 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.642+0000 7feabab84700 1 -- 172.21.15.179:0/1442586351 wait complete. 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 Processor -- start 2024-05-08T22:34:07.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 -- start start 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feab41f61e0 0x7feab41468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 0x7feab40cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab4149d60 con 0x7feab40cbbb0 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40cc5a0 con 0x7feab41f61e0 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feabab84700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40c9810 con 0x7feab40d1aa0 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feab9b82700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 0x7feab40cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feab9b82700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab40c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:35380/0 (socket says 172.21.15.179:35380) 2024-05-08T22:34:07.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feab9b82700 1 -- 172.21.15.179:0/2592941470 learned_addr learned my addr 172.21.15.179:0/2592941470 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:07.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feab9381700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feab41f61e0 0x7feab41468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 -- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 msgr2=0x7feab40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab40c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 -- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feab41f61e0 msgr2=0x7feab41468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feab41f61e0 0x7feab41468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 -- 172.21.15.179:0/2592941470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4146e00 con 0x7feab40cbbb0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaba383700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 0x7feab40cc060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feab408b4f0 tx=0x7feaac00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.646+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00e040 con 0x7feab40cbbb0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac00f040 con 0x7feab40cbbb0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab41f6760 con 0x7feab40cbbb0 2024-05-08T22:34:07.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac003a40 con 0x7feab40cbbb0 2024-05-08T22:34:07.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab41f6d00 con 0x7feab40cbbb0 2024-05-08T22:34:07.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab404e700 con 0x7feab40cbbb0 2024-05-08T22:34:07.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.650+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feaac015430 con 0x7feab40cbbb0 2024-05-08T22:34:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.654+0000 7feaa67fc700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 0x7fea98068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.654+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feaac0848f0 con 0x7feab40cbbb0 2024-05-08T22:34:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.654+0000 7feab9b82700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 0x7fea98068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.654+0000 7feab9b82700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 0x7fea98068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feaa8007db0 tx=0x7feaa8001e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.658+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feaac051890 con 0x7feab40cbbb0 2024-05-08T22:34:07.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feab415c2c0 con 0x7feab40cbbb0 2024-05-08T22:34:07.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feaa67fc700 1 -- 172.21.15.179:0/2592941470 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7feaac054eb0 con 0x7feab40cbbb0 2024-05-08T22:34:07.828 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 msgr2=0x7fea98068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 0x7fea98068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feaa8007db0 tx=0x7feaa8001e00 comp rx=0 tx=0).stop 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 msgr2=0x7feab40cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.826+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 0x7feab40cc060 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feab408b4f0 tx=0x7feaac00f6c0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 shutdown_connections 2024-05-08T22:34:07.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea98066760 0x7fea98068c20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feab41f61e0 0x7feab41468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab40cbbb0 0x7feab40cc060 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 --2- 172.21.15.179:0/2592941470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feab40d1aa0 0x7feab40c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 >> 172.21.15.179:0/2592941470 conn(0x7feab40cfac0 msgr2=0x7feab415f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 shutdown_connections 2024-05-08T22:34:07.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.830+0000 7feabab84700 1 -- 172.21.15.179:0/2592941470 wait complete. 2024-05-08T22:34:07.842 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:07.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.962+0000 7fdcad0e2700 1 Processor -- start 2024-05-08T22:34:07.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.962+0000 7fdcad0e2700 1 -- start start 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.962+0000 7fdcad0e2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdca815d0c0 0x7fdca80b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.962+0000 7fdcad0e2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdca80ce2b0 0x7fdca806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.962+0000 7fdcad0e2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca815d4a0 con 0x7fdca80d1d20 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca806abb0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca80d2500 con 0x7fdca815d0c0 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdca80ce2b0 0x7fdca806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdca80d1d20 0x7fdca80cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60850/0 (socket says 172.21.15.179:60850) 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca77fe700 1 -- 172.21.15.179:0/3413865260 learned_addr learned my addr 172.21.15.179:0/3413865260 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca7fff700 1 --2- 172.21.15.179:0/3413865260 >> v2:172.21.15.195:3300/0 conn(0x7fdca815d0c0 0x7fdca80b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 -- 172.21.15.179:0/3413865260 >> v2:172.21.15.195:3300/0 conn(0x7fdca815d0c0 msgr2=0x7fdca80b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 --2- 172.21.15.179:0/3413865260 >> v2:172.21.15.195:3300/0 conn(0x7fdca815d0c0 0x7fdca80b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 -- 172.21.15.179:0/3413865260 >> v1:172.21.15.179:6789/0 conn(0x7fdca80d1d20 legacy=0x7fdca80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 -- 172.21.15.179:0/3413865260 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80d27e0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca6ffd700 1 -- 172.21.15.179:0/3413865260 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696498500 0 0) 0x7fdca815d4a0 con 0x7fdca80d1d20 2024-05-08T22:34:07.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcac8e1700 1 --2- 172.21.15.179:0/3413865260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca806b370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2968b6280d7bcd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca6ffd700 1 -- 172.21.15.179:0/3413865260 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000e070 con 0x7fdca80ce2b0 2024-05-08T22:34:07.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca6ffd700 1 -- 172.21.15.179:0/3413865260 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000bae0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdca6ffd700 1 -- 172.21.15.179:0/3413865260 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000be90 con 0x7fdca80ce2b0 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- 172.21.15.179:0/3413865260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 msgr2=0x7fdca806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/3413865260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca806b370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca000b330 comp rx=0 tx=0).stop 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- 172.21.15.179:0/3413865260 shutdown_connections 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/3413865260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca806b370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/3413865260 >> v2:172.21.15.195:3300/0 conn(0x7fdca815d0c0 0x7fdca80b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- 172.21.15.179:0/3413865260 >> 172.21.15.179:0/3413865260 conn(0x7fdca80d2ed0 msgr2=0x7fdca80d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- 172.21.15.179:0/3413865260 shutdown_connections 2024-05-08T22:34:07.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.966+0000 7fdcad0e2700 1 -- 172.21.15.179:0/3413865260 wait complete. 2024-05-08T22:34:07.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 Processor -- start 2024-05-08T22:34:07.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 -- start start 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca80d1d20 0x7fdca80c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca815d0c0 0x7fdca8148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c6860 con 0x7fdca80ce2b0 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80c7700 con 0x7fdca80d1d20 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcad0e2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80cb940 con 0x7fdca815d0c0 2024-05-08T22:34:07.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59106/0 (socket says 172.21.15.179:59106) 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 -- 172.21.15.179:0/2960630584 learned_addr learned my addr 172.21.15.179:0/2960630584 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdcac8e1700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca815d0c0 0x7fdca8148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca77fe700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca80d1d20 0x7fdca80c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 -- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca815d0c0 msgr2=0x7fdca8148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca815d0c0 0x7fdca8148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 -- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca80d1d20 msgr2=0x7fdca80c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca80d1d20 0x7fdca80c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.970+0000 7fdca7fff700 1 -- 172.21.15.179:0/2960630584 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009a70 con 0x7fdca80ce2b0 2024-05-08T22:34:07.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca7fff700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e840 tx=0x7fdc9800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980095a0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98013680 con 0x7fdca80ce2b0 2024-05-08T22:34:07.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980139f0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca80cbc00 con 0x7fdca80ce2b0 2024-05-08T22:34:07.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca80cc1c0 con 0x7fdca80ce2b0 2024-05-08T22:34:07.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca804e700 con 0x7fdca80ce2b0 2024-05-08T22:34:07.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc98013b50 con 0x7fdca80ce2b0 2024-05-08T22:34:07.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 0x7fdc88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:07.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdc98084b60 con 0x7fdca80ce2b0 2024-05-08T22:34:07.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.974+0000 7fdca77fe700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 0x7fdc88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:07.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.978+0000 7fdca77fe700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 0x7fdc88068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdca80b4740 tx=0x7fdc90007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:07.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:07.982+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc98051ba0 con 0x7fdca80ce2b0 2024-05-08T22:34:08.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.150+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdca80676e0 con 0x7fdca80ce2b0 2024-05-08T22:34:08.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.150+0000 7fdca4ff9700 1 -- 172.21.15.179:0/2960630584 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc98021020 con 0x7fdca80ce2b0 2024-05-08T22:34:08.153 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:08.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 msgr2=0x7fdc88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 0x7fdc88068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdca80b4740 tx=0x7fdc90007e00 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 msgr2=0x7fdca80cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e840 tx=0x7fdc9800eb50 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 shutdown_connections 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc88066730 0x7fdc88068bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca80d1d20 0x7fdca80c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca815d0c0 0x7fdca8148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 --2- 172.21.15.179:0/2960630584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdca80ce2b0 0x7fdca80cc9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.154+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 >> 172.21.15.179:0/2960630584 conn(0x7fdca80d2ed0 msgr2=0x7fdca80d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.158+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 shutdown_connections 2024-05-08T22:34:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.158+0000 7fdcad0e2700 1 -- 172.21.15.179:0/2960630584 wait complete. 2024-05-08T22:34:08.173 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-08T22:34:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 Processor -- start 2024-05-08T22:34:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 -- start start 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 0x7f99c81405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 0x7f99c806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c8140350 con 0x7f99c815e220 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80b3590 con 0x7f99c80cef10 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80b2c60 con 0x7f99c815de40 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c6d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 0x7f99c806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 0x7f99c81405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c5d9b700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f99c815e220 0x7f99c80ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60794/0 (socket says 172.21.15.179:60794) 2024-05-08T22:34:08.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c5d9b700 1 -- 172.21.15.179:0/2886531962 learned_addr learned my addr 172.21.15.179:0/2886531962 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:08.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c559a700 1 -- 172.21.15.179:0/2886531962 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590114874 0 0) 0x7f99c8140350 con 0x7f99c815e220 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c559a700 1 -- 172.21.15.179:0/2886531962 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b40035d0 con 0x7f99c815e220 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 -- 172.21.15.179:0/2886531962 >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 msgr2=0x7f99c806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 --2- 172.21.15.179:0/2886531962 >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 0x7f99c806e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 -- 172.21.15.179:0/2886531962 >> v1:172.21.15.195:6789/0 conn(0x7f99c815e220 legacy=0x7f99c80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 -- 172.21.15.179:0/2886531962 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80b2f40 con 0x7f99c815de40 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c6d9d700 1 --2- 172.21.15.179:0/2886531962 >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 0x7f99c806e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.286+0000 7f99c659c700 1 --2- 172.21.15.179:0/2886531962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 0x7f99c81405e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009850 tx=0x7f99b8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d0ed730fe8dd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c559a700 1 -- 172.21.15.179:0/2886531962 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8011830 con 0x7f99c815de40 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c559a700 1 -- 172.21.15.179:0/2886531962 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8011990 con 0x7f99c815de40 2024-05-08T22:34:08.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c559a700 1 -- 172.21.15.179:0/2886531962 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8011d00 con 0x7f99c815de40 2024-05-08T22:34:08.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 -- 172.21.15.179:0/2886531962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 msgr2=0x7f99c81405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 --2- 172.21.15.179:0/2886531962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 0x7f99c81405e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009850 tx=0x7f99b8009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:08.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 -- 172.21.15.179:0/2886531962 shutdown_connections 2024-05-08T22:34:08.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 --2- 172.21.15.179:0/2886531962 >> v2:172.21.15.179:3300/0 conn(0x7f99c80cef10 0x7f99c806e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 --2- 172.21.15.179:0/2886531962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c815de40 0x7f99c81405e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 -- 172.21.15.179:0/2886531962 >> 172.21.15.179:0/2886531962 conn(0x7f99c80cfb50 msgr2=0x7f99c80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 -- 172.21.15.179:0/2886531962 shutdown_connections 2024-05-08T22:34:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.290+0000 7f99c759e700 1 -- 172.21.15.179:0/2886531962 wait complete. 2024-05-08T22:34:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 Processor -- start 2024-05-08T22:34:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- start start 2024-05-08T22:34:08.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c80cef10 0x7f99c8156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c815e220 0x7f99c8158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81307d0 con 0x7f99c815de40 2024-05-08T22:34:08.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c812fec0 con 0x7f99c815e220 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cab60 con 0x7f99c80cef10 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59128/0 (socket says 172.21.15.179:59128) 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 -- 172.21.15.179:0/3424800272 learned_addr learned my addr 172.21.15.179:0/3424800272 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c6d9d700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c815e220 0x7f99c8158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c659c700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c80cef10 0x7f99c8156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 -- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c80cef10 msgr2=0x7f99c8156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c80cef10 0x7f99c8156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 -- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c815e220 msgr2=0x7f99c8158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:08.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c815e220 0x7f99c8158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 -- 172.21.15.179:0/3424800272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0009a70 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c5d9b700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f99b00010e0 tx=0x7f99b000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0005d90 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b000ce50 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b80094b0 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00072f0 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.294+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81303f0 con 0x7f99c815de40 2024-05-08T22:34:08.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c804e700 con 0x7f99c815de40 2024-05-08T22:34:08.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99b0007450 con 0x7f99c815de40 2024-05-08T22:34:08.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99beffd700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 0x7f99a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:08.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f99b00837f0 con 0x7f99c815de40 2024-05-08T22:34:08.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99c659c700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 0x7f99a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:08.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.298+0000 7f99c659c700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 0x7f99a8068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99b80090b0 tx=0x7f99b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:08.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.306+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f99b0050830 con 0x7f99c815de40 2024-05-08T22:34:08.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:08.542+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f99c80b4d50 con 0x7f99c815de40 2024-05-08T22:34:09.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.514+0000 7f99beffd700 1 -- 172.21.15.179:0/3424800272 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v63) v1 ==== 145+0+0 (secure 0 0 0) 0x7f99b00170d0 con 0x7f99c815de40 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 msgr2=0x7f99a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 0x7f99a8068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99b80090b0 tx=0x7f99b8008030 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 msgr2=0x7f99c8157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f99b00010e0 tx=0x7f99b000c7b0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 shutdown_connections 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99a8066730 0x7f99a8068bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c815e220 0x7f99c8158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c815de40 0x7f99c8157c50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 --2- 172.21.15.179:0/3424800272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c80cef10 0x7f99c8156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 >> 172.21.15.179:0/3424800272 conn(0x7f99c80cfb50 msgr2=0x7f99c80ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 shutdown_connections 2024-05-08T22:34:09.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.522+0000 7f99c759e700 1 -- 172.21.15.179:0/3424800272 wait complete. 2024-05-08T22:34:09.538 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:34:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 Processor -- start 2024-05-08T22:34:09.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 -- start start 2024-05-08T22:34:09.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d40ce2b0 0x7f90d40d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d40cd710 0x7f90d40cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d40d16b0 con 0x7f90d415d130 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d40ce0d0 con 0x7f90d40cd710 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40d1990 con 0x7f90d40ce2b0 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d2d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d40cd710 0x7f90d40cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d259c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f90d415d130 0x7f90d40d2750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60860/0 (socket says 172.21.15.179:60860) 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.662+0000 7f90d259c700 1 -- 172.21.15.179:0/146861592 learned_addr learned my addr 172.21.15.179:0/146861592 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d1d9b700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d40ce2b0 0x7f90d40d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d2d9d700 1 -- 172.21.15.179:0/146861592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d40ce2b0 msgr2=0x7f90d40d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d2d9d700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d40ce2b0 0x7f90d40d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d2d9d700 1 -- 172.21.15.179:0/146861592 >> v1:172.21.15.179:6789/0 conn(0x7f90d415d130 legacy=0x7f90d40d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d2d9d700 1 -- 172.21.15.179:0/146861592 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40d1c70 con 0x7f90d40cd710 2024-05-08T22:34:09.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d2d9d700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40cd710 0x7f90d40cdb90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd756199151c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:09.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d159a700 1 -- 172.21.15.179:0/146861592 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00e070 con 0x7f90d40cd710 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d159a700 1 -- 172.21.15.179:0/146861592 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc00bbd0 con 0x7f90d40cd710 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d159a700 1 -- 172.21.15.179:0/146861592 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc012430 con 0x7f90d40cd710 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 -- 172.21.15.179:0/146861592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40cd710 msgr2=0x7f90d40cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40cd710 0x7f90d40cdb90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 -- 172.21.15.179:0/146861592 shutdown_connections 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40cd710 0x7f90d40cdb90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 --2- 172.21.15.179:0/146861592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d40ce2b0 0x7f90d40d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 -- 172.21.15.179:0/146861592 >> 172.21.15.179:0/146861592 conn(0x7f90d40b49a0 msgr2=0x7f90d40b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 -- 172.21.15.179:0/146861592 shutdown_connections 2024-05-08T22:34:09.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.666+0000 7f90d359e700 1 -- 172.21.15.179:0/146861592 wait complete. 2024-05-08T22:34:09.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 Processor -- start 2024-05-08T22:34:09.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- start start 2024-05-08T22:34:09.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d40cd710 0x7f90d41586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d415d130 0x7f90d4160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4153c50 con 0x7f90d40ce2b0 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4152550 con 0x7f90d40cd710 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4158da0 con 0x7f90d415d130 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59150/0 (socket says 172.21.15.179:59150) 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 -- 172.21.15.179:0/2211961578 learned_addr learned my addr 172.21.15.179:0/2211961578 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:09.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d259c700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d40cd710 0x7f90d41586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d2d9d700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d415d130 0x7f90d4160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 -- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d415d130 msgr2=0x7f90d4160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d415d130 0x7f90d4160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 -- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d40cd710 msgr2=0x7f90d41586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d40cd710 0x7f90d41586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 -- 172.21.15.179:0/2211961578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc009a70 con 0x7f90d40ce2b0 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d1d9b700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e730 tx=0x7f90c400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400c070 con 0x7f90d40ce2b0 2024-05-08T22:34:09.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4004020 con 0x7f90d40ce2b0 2024-05-08T22:34:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d415d690 con 0x7f90d40ce2b0 2024-05-08T22:34:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40043d0 con 0x7f90d40ce2b0 2024-05-08T22:34:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.670+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d4159360 con 0x7f90d40ce2b0 2024-05-08T22:34:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.674+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d404e700 con 0x7f90d40ce2b0 2024-05-08T22:34:09.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.674+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90c40036c0 con 0x7f90d40ce2b0 2024-05-08T22:34:09.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.678+0000 7f90c2ffd700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 0x7f90b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:09.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.678+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f90c4083d90 con 0x7f90d40ce2b0 2024-05-08T22:34:09.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.678+0000 7f90d259c700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 0x7f90b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:09.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.678+0000 7f90d259c700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 0x7f90b4068bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005930 tx=0x7f90c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:09.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.682+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90c4050dd0 con 0x7f90d40ce2b0 2024-05-08T22:34:09.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.854+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f90d40b2bb0 con 0x7f90d40ce2b0 2024-05-08T22:34:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.862+0000 7f90c2ffd700 1 -- 172.21.15.179:0/2211961578 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f90c40543f0 con 0x7f90d40ce2b0 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 msgr2=0x7f90b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 0x7f90b4068bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005930 tx=0x7f90c8009040 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 msgr2=0x7f90d4152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e730 tx=0x7f90c400ea40 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 shutdown_connections 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b4066730 0x7f90b4068bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d415d130 0x7f90d4160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d40ce2b0 0x7f90d4152010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 --2- 172.21.15.179:0/2211961578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d40cd710 0x7f90d41586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 >> 172.21.15.179:0/2211961578 conn(0x7f90d40b49a0 msgr2=0x7f90d40c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 shutdown_connections 2024-05-08T22:34:09.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:09.866+0000 7f90d359e700 1 -- 172.21.15.179:0/2211961578 wait complete. 2024-05-08T22:34:09.881 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:34:09.881 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:34:09.894 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:34:09.913 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:34:09.913 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:09.913 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:09.921 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:09.921 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:09.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:10.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 Processor -- start 2024-05-08T22:34:10.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 -- start start 2024-05-08T22:34:10.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f11c40b25c0 0x7f11c40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11c415d3d0 0x7f11c40adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40ad4c0 con 0x7f11c40ad0e0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c415ae80 con 0x7f11c415d3d0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11ca49e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c41609c0 con 0x7f11c40b25c0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11c415d3d0 0x7f11c40adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11c415d3d0 0x7f11c40adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59154/0 (socket says 172.21.15.179:59154) 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 -- 172.21.15.179:0/1714711989 learned_addr learned my addr 172.21.15.179:0/1714711989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c8c9b700 1 --2- 172.21.15.179:0/1714711989 >> v2:172.21.15.195:3300/0 conn(0x7f11c40b25c0 0x7f11c40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 -- 172.21.15.179:0/1714711989 >> v2:172.21.15.195:3300/0 conn(0x7f11c40b25c0 msgr2=0x7f11c40ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 --2- 172.21.15.179:0/1714711989 >> v2:172.21.15.195:3300/0 conn(0x7f11c40b25c0 0x7f11c40ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 -- 172.21.15.179:0/1714711989 >> v1:172.21.15.195:6789/0 conn(0x7f11c40ad0e0 legacy=0x7f11c40ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 -- 172.21.15.179:0/1714711989 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c415d900 con 0x7f11c415d3d0 2024-05-08T22:34:10.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11c9c9d700 1 --2- 172.21.15.179:0/1714711989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c40adda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11c406e670 tx=0x7f11bc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcea9fe32c0033 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.086+0000 7f11bbfff700 1 -- 172.21.15.179:0/1714711989 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc004230 con 0x7f11c415d3d0 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11bbfff700 1 -- 172.21.15.179:0/1714711989 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc004390 con 0x7f11c415d3d0 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11bbfff700 1 -- 172.21.15.179:0/1714711989 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0036c0 con 0x7f11c415d3d0 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- 172.21.15.179:0/1714711989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 msgr2=0x7f11c40adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- 172.21.15.179:0/1714711989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c40adda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11c406e670 tx=0x7f11bc009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- 172.21.15.179:0/1714711989 shutdown_connections 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- 172.21.15.179:0/1714711989 >> v2:172.21.15.195:3300/0 conn(0x7f11c40b25c0 0x7f11c40ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- 172.21.15.179:0/1714711989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c40adda0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- 172.21.15.179:0/1714711989 >> 172.21.15.179:0/1714711989 conn(0x7f11c40ce9d0 msgr2=0x7f11c40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- 172.21.15.179:0/1714711989 shutdown_connections 2024-05-08T22:34:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- 172.21.15.179:0/1714711989 wait complete. 2024-05-08T22:34:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 Processor -- start 2024-05-08T22:34:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- start start 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40ad0e0 0x7f11c40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40b25c0 0x7f11c41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4157240 con 0x7f11c415d3d0 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4157520 con 0x7f11c40b25c0 2024-05-08T22:34:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.090+0000 7f11ca49e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4157800 con 0x7f11c40ad0e0 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59158/0 (socket says 172.21.15.179:59158) 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 -- 172.21.15.179:0/3405804789 learned_addr learned my addr 172.21.15.179:0/3405804789 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c8c9b700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40b25c0 0x7f11c41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c949c700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40ad0e0 0x7f11c40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 -- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40ad0e0 msgr2=0x7f11c40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40ad0e0 0x7f11c40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 -- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40b25c0 msgr2=0x7f11c41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40b25c0 0x7f11c41582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 -- 172.21.15.179:0/3405804789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc009570 con 0x7f11c415d3d0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11c9c9d700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009c30 tx=0x7f11bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0036e0 con 0x7f11c415d3d0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc00b660 con 0x7f11c415d3d0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00b9d0 con 0x7f11c415d3d0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c4157ae0 con 0x7f11c415d3d0 2024-05-08T22:34:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40cc550 con 0x7f11c415d3d0 2024-05-08T22:34:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4158000 con 0x7f11c415d3d0 2024-05-08T22:34:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.094+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11bc021070 con 0x7f11c415d3d0 2024-05-08T22:34:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.098+0000 7f11b9ffb700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 0x7f11a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.098+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f11bc085ce0 con 0x7f11c415d3d0 2024-05-08T22:34:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.098+0000 7f11c949c700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 0x7f11a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.098+0000 7f11c949c700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 0x7f11a8068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009980 tx=0x7f11b40090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.106+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11c4158000 con 0x7f11c415d3d0 2024-05-08T22:34:10.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.278+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11c4158000 con 0x7f11c415d3d0 2024-05-08T22:34:10.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11b9ffb700 1 -- 172.21.15.179:0/3405804789 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f11c4158000 con 0x7f11c415d3d0 2024-05-08T22:34:10.282 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:10.282 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 msgr2=0x7f11a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 0x7f11a8068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009980 tx=0x7f11b40090e0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 msgr2=0x7f11c4144ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009c30 tx=0x7f11bc004770 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 shutdown_connections 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11a8066730 0x7f11a8068bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11c40b25c0 0x7f11c41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11c415d3d0 0x7f11c4144ea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 --2- 172.21.15.179:0/3405804789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11c40ad0e0 0x7f11c40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 >> 172.21.15.179:0/3405804789 conn(0x7f11c40ce9d0 msgr2=0x7f11c40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 shutdown_connections 2024-05-08T22:34:10.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.282+0000 7f11ca49e700 1 -- 172.21.15.179:0/3405804789 wait complete. 2024-05-08T22:34:10.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:10.298 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:10.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.414+0000 7f557cb60700 1 Processor -- start 2024-05-08T22:34:10.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.414+0000 7f557cb60700 1 -- start start 2024-05-08T22:34:10.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.414+0000 7f557cb60700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 0x7f55780b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55780b22d0 con 0x7f557806d4b0 2024-05-08T22:34:10.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55780b19e0 con 0x7f55780b2ca0 2024-05-08T22:34:10.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557815e640 con 0x7f5578142890 2024-05-08T22:34:10.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f5577fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f557806d4b0 0x7f557815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60868/0 (socket says 172.21.15.179:60868) 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f5577fff700 1 -- 172.21.15.179:0/3965145938 learned_addr learned my addr 172.21.15.179:0/3965145938 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55777fe700 1 --2- 172.21.15.179:0/3965145938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 0x7f55780b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244363065 0 0) 0x7f55780b22d0 con 0x7f557806d4b0 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55640035d0 con 0x7f557806d4b0 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468854952 0 0) 0x7f55780b19e0 con 0x7f55780b2ca0 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55780b22d0 con 0x7f55780b2ca0 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55777fe700 1 -- 172.21.15.179:0/3965145938 >> v1:172.21.15.195:6789/0 conn(0x7f55780b2ca0 legacy=0x7f557806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55777fe700 1 -- 172.21.15.179:0/3965145938 >> v1:172.21.15.179:6789/0 conn(0x7f557806d4b0 legacy=0x7f557815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55777fe700 1 -- 172.21.15.179:0/3965145938 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780b1c80 con 0x7f5578142890 2024-05-08T22:34:10.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55777fe700 1 --2- 172.21.15.179:0/3965145938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 0x7f55780b2760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5568009880 tx=0x7f5568009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6c624683a23f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568011830 con 0x7f5578142890 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5568011990 con 0x7f5578142890 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f55767fc700 1 -- 172.21.15.179:0/3965145938 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568011d20 con 0x7f5578142890 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- 172.21.15.179:0/3965145938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 msgr2=0x7f55780b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 --2- 172.21.15.179:0/3965145938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 0x7f55780b2760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5568009880 tx=0x7f5568009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- 172.21.15.179:0/3965145938 shutdown_connections 2024-05-08T22:34:10.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 --2- 172.21.15.179:0/3965145938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5578142890 0x7f55780b2760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- 172.21.15.179:0/3965145938 >> 172.21.15.179:0/3965145938 conn(0x7f55780cea30 msgr2=0x7f55780cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- 172.21.15.179:0/3965145938 shutdown_connections 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- 172.21.15.179:0/3965145938 wait complete. 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 Processor -- start 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.418+0000 7f557cb60700 1 -- start start 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 0x7f557814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55780b2ca0 0x7f5578168e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781689d0 con 0x7f5578142890 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578167dc0 con 0x7f55780b2ca0 2024-05-08T22:34:10.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55781680a0 con 0x7f557806d4b0 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59174/0 (socket says 172.21.15.179:59174) 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 -- 172.21.15.179:0/1644975232 learned_addr learned my addr 172.21.15.179:0/1644975232 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f55777fe700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 0x7f557814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5576ffd700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55780b2ca0 0x7f5578168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 -- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 msgr2=0x7f557814ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 0x7f557814ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 -- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55780b2ca0 msgr2=0x7f5578168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55780b2ca0 0x7f5578168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 -- 172.21.15.179:0/1644975232 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680094b0 con 0x7f5578142890 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f55777fe700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 0x7f557814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f5577fff700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f557000cbc0 tx=0x7f557000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000a980 con 0x7f5578142890 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5578168380 con 0x7f5578142890 2024-05-08T22:34:10.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781693a0 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557000aae0 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000ae50 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c005310 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.422+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f557001c440 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.426+0000 7f556ffff700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 0x7f5558068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.426+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5570083ee0 con 0x7f5578142890 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.426+0000 7f55777fe700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 0x7f5558068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.426+0000 7f55777fe700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 0x7f5558068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55680090b0 tx=0x7f556801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.430+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5570050ec0 con 0x7f5578142890 2024-05-08T22:34:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.598+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f555c005b40 con 0x7f5578142890 2024-05-08T22:34:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f556ffff700 1 -- 172.21.15.179:0/1644975232 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55700544e0 con 0x7f5578142890 2024-05-08T22:34:10.602 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:10.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 msgr2=0x7f5558068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 0x7f5558068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55680090b0 tx=0x7f556801c040 comp rx=0 tx=0).stop 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 msgr2=0x7f5578070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f557000cbc0 tx=0x7f557000cca0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 shutdown_connections 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5558066730 0x7f5558068bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55780b2ca0 0x7f5578168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5578142890 0x7f5578070da0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 --2- 172.21.15.179:0/1644975232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557806d4b0 0x7f557814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 >> 172.21.15.179:0/1644975232 conn(0x7f55780cea30 msgr2=0x7f5578136a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 shutdown_connections 2024-05-08T22:34:10.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.602+0000 7f557cb60700 1 -- 172.21.15.179:0/1644975232 wait complete. 2024-05-08T22:34:10.617 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:10.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.730+0000 7f99b2f92700 1 Processor -- start 2024-05-08T22:34:10.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.730+0000 7f99b2f92700 1 -- start start 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 0x7f99ac0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99ac0b2ca0 0x7f99ac06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0b22d0 con 0x7f99ac142fd0 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0b19e0 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac15e640 con 0x7f99ac142890 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99ac0b2ca0 0x7f99ac06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2791700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f99ac142fd0 0x7f99ac15d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60872/0 (socket says 172.21.15.179:60872) 2024-05-08T22:34:10.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2791700 1 -- 172.21.15.179:0/1547941408 learned_addr learned my addr 172.21.15.179:0/1547941408 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b1f90700 1 --2- 172.21.15.179:0/1547941408 >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 0x7f99ac0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b0f8e700 1 -- 172.21.15.179:0/1547941408 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45155365 0 0) 0x7f99ac0b22d0 con 0x7f99ac142fd0 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b0f8e700 1 -- 172.21.15.179:0/1547941408 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99980035d0 con 0x7f99ac142fd0 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 -- 172.21.15.179:0/1547941408 >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 msgr2=0x7f99ac0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 --2- 172.21.15.179:0/1547941408 >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 0x7f99ac0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 -- 172.21.15.179:0/1547941408 >> v1:172.21.15.179:6789/0 conn(0x7f99ac142fd0 legacy=0x7f99ac15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 -- 172.21.15.179:0/1547941408 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0b1c80 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b1f90700 1 --2- 172.21.15.179:0/1547941408 >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 0x7f99ac0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:10.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b178f700 1 --2- 172.21.15.179:0/1547941408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac06cda0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0720b0 tx=0x7f99a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df847e2ee8e1fc59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b0f8e700 1 -- 172.21.15.179:0/1547941408 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a800e070 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b0f8e700 1 -- 172.21.15.179:0/1547941408 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a800bb20 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b0f8e700 1 -- 172.21.15.179:0/1547941408 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a800be90 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- 172.21.15.179:0/1547941408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 msgr2=0x7f99ac06cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1547941408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac06cda0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0720b0 tx=0x7f99a800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- 172.21.15.179:0/1547941408 shutdown_connections 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1547941408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac06cda0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1547941408 >> v2:172.21.15.195:3300/0 conn(0x7f99ac142890 0x7f99ac0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- 172.21.15.179:0/1547941408 >> 172.21.15.179:0/1547941408 conn(0x7f99ac0cea30 msgr2=0x7f99ac0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- 172.21.15.179:0/1547941408 shutdown_connections 2024-05-08T22:34:10.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.734+0000 7f99b2f92700 1 -- 172.21.15.179:0/1547941408 wait complete. 2024-05-08T22:34:10.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 Processor -- start 2024-05-08T22:34:10.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 -- start start 2024-05-08T22:34:10.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac142890 0x7f99ac154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac142fd0 0x7f99ac137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac15cb20 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac15cdc0 con 0x7f99ac142fd0 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2f92700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac15e080 con 0x7f99ac142890 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59180/0 (socket says 172.21.15.179:59180) 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 -- 172.21.15.179:0/1615764288 learned_addr learned my addr 172.21.15.179:0/1615764288 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b178f700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac142890 0x7f99ac154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b2791700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac142fd0 0x7f99ac137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 -- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac142890 msgr2=0x7f99ac154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac142890 0x7f99ac154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 -- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac142fd0 msgr2=0x7f99ac137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac142fd0 0x7f99ac137080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 -- 172.21.15.179:0/1615764288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009a70 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.738+0000 7f99b1f90700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e880 tx=0x7f99a000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0013610 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a0013770 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac15b680 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0013b00 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac15d340 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac15cf30 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99a0013c60 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 0x7f9990068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f99a00840e0 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.742+0000 7f99b178f700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 0x7f9990068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:10.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.746+0000 7f99b178f700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 0x7f9990068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99a8014040 tx=0x7f99a8011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:10.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.750+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f99a0051090 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.922+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99ac04e700 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.922+0000 7f999e7fc700 1 -- 172.21.15.179:0/1615764288 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99a00546b0 con 0x7f99ac0b2ca0 2024-05-08T22:34:10.923 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:10.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 msgr2=0x7f9990068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 0x7f9990068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99a8014040 tx=0x7f99a8011000 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 msgr2=0x7f99ac15db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e880 tx=0x7f99a000eb90 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 shutdown_connections 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9990066730 0x7f9990068bf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99ac142fd0 0x7f99ac137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99ac142890 0x7f99ac154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 --2- 172.21.15.179:0/1615764288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac0b2ca0 0x7f99ac15db40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 >> 172.21.15.179:0/1615764288 conn(0x7f99ac0cea30 msgr2=0x7f99ac0ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:10.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 shutdown_connections 2024-05-08T22:34:10.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:10.926+0000 7f99b2f92700 1 -- 172.21.15.179:0/1615764288 wait complete. 2024-05-08T22:34:10.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:11.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.058+0000 7f76da1e6700 1 Processor -- start 2024-05-08T22:34:11.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.058+0000 7f76da1e6700 1 -- start start 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76d40b8a80 0x7f76d40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 0x7f76d406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40b8ee0 con 0x7f76d415d0c0 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40ce570 con 0x7f76d40b8a80 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d415d4a0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d91e4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f76d415d0c0 0x7f76d40d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60880/0 (socket says 172.21.15.179:60880) 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d91e4700 1 -- 172.21.15.179:0/309521599 learned_addr learned my addr 172.21.15.179:0/309521599 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d89e3700 1 --2- 172.21.15.179:0/309521599 >> v2:172.21.15.195:3300/0 conn(0x7f76d40b8a80 0x7f76d40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 --2- 172.21.15.179:0/309521599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 0x7f76d406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76cbfff700 1 -- 172.21.15.179:0/309521599 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044332233 0 0) 0x7f76d40b8ee0 con 0x7f76d415d0c0 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76cbfff700 1 -- 172.21.15.179:0/309521599 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c00035d0 con 0x7f76d415d0c0 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 -- 172.21.15.179:0/309521599 >> v2:172.21.15.195:3300/0 conn(0x7f76d40b8a80 msgr2=0x7f76d40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 --2- 172.21.15.179:0/309521599 >> v2:172.21.15.195:3300/0 conn(0x7f76d40b8a80 0x7f76d40cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 -- 172.21.15.179:0/309521599 >> v1:172.21.15.179:6789/0 conn(0x7f76d415d0c0 legacy=0x7f76d40d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 -- 172.21.15.179:0/309521599 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d406abb0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76d99e5700 1 --2- 172.21.15.179:0/309521599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 0x7f76d406b370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009d90 tx=0x7f76cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3289d1272215858 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76cbfff700 1 -- 172.21.15.179:0/309521599 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00e070 con 0x7f76d40ce0e0 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76cbfff700 1 -- 172.21.15.179:0/309521599 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00bb20 con 0x7f76d40ce0e0 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76cbfff700 1 -- 172.21.15.179:0/309521599 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc012430 con 0x7f76d40ce0e0 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- 172.21.15.179:0/309521599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 msgr2=0x7f76d406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 --2- 172.21.15.179:0/309521599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 0x7f76d406b370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009d90 tx=0x7f76cc00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:11.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- 172.21.15.179:0/309521599 shutdown_connections 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 --2- 172.21.15.179:0/309521599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40ce0e0 0x7f76d406b370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 --2- 172.21.15.179:0/309521599 >> v2:172.21.15.195:3300/0 conn(0x7f76d40b8a80 0x7f76d40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- 172.21.15.179:0/309521599 >> 172.21.15.179:0/309521599 conn(0x7f76d40d2ed0 msgr2=0x7f76d40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- 172.21.15.179:0/309521599 shutdown_connections 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.062+0000 7f76da1e6700 1 -- 172.21.15.179:0/309521599 wait complete. 2024-05-08T22:34:11.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 Processor -- start 2024-05-08T22:34:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 -- start start 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40b8a80 0x7f76d414e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76d415d0c0 0x7f76d40c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40c82c0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d414ec20 con 0x7f76d415d0c0 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76da1e6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d414e6a0 con 0x7f76d40b8a80 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59188/0 (socket says 172.21.15.179:59188) 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 -- 172.21.15.179:0/535753412 learned_addr learned my addr 172.21.15.179:0/535753412 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d91e4700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40b8a80 0x7f76d414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d99e5700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76d415d0c0 0x7f76d40c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 -- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40b8a80 msgr2=0x7f76d414e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40b8a80 0x7f76d414e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 -- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76d415d0c0 msgr2=0x7f76d40c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76d415d0c0 0x7f76d40c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 -- 172.21.15.179:0/535753412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc009a70 con 0x7f76d40ce0e0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.066+0000 7f76d89e3700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e9d0 tx=0x7f76d000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0009720 con 0x7f76d40ce0e0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000ee70 con 0x7f76d40ce0e0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d40c7740 con 0x7f76d40ce0e0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000ca50 con 0x7f76d40ce0e0 2024-05-08T22:34:11.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d40c7d00 con 0x7f76d40ce0e0 2024-05-08T22:34:11.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76d000cbb0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d404e700 con 0x7f76d40ce0e0 2024-05-08T22:34:11.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 0x7f76b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f76d00849b0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.070+0000 7f76d91e4700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 0x7f76b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.074+0000 7f76d91e4700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 0x7f76b8068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76c4007fd0 tx=0x7f76c4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.078+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76d0051830 con 0x7f76d40ce0e0 2024-05-08T22:34:11.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.246+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76d415a6b0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.246+0000 7f76c9ffb700 1 -- 172.21.15.179:0/535753412 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76d00210a0 con 0x7f76d40ce0e0 2024-05-08T22:34:11.249 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 msgr2=0x7f76b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 0x7f76b8068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76c4007fd0 tx=0x7f76c4007de0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 msgr2=0x7f76d40c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e9d0 tx=0x7f76d000ece0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 shutdown_connections 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76b8066730 0x7f76b8068bf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76d415d0c0 0x7f76d40c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76d40ce0e0 0x7f76d40c8e10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 --2- 172.21.15.179:0/535753412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76d40b8a80 0x7f76d414e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 >> 172.21.15.179:0/535753412 conn(0x7f76d40d2ed0 msgr2=0x7f76d406bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 shutdown_connections 2024-05-08T22:34:11.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.250+0000 7f76da1e6700 1 -- 172.21.15.179:0/535753412 wait complete. 2024-05-08T22:34:11.268 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:11.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.382+0000 7ff2e359e700 1 Processor -- start 2024-05-08T22:34:11.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.382+0000 7ff2e359e700 1 -- start start 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e4142890 0x7ff2e40b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2e406d4b0 0x7ff2e415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e40b22d0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e40b19e0 con 0x7ff2e406d4b0 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e415e640 con 0x7ff2e4142890 2024-05-08T22:34:11.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2e406d4b0 0x7ff2e415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2e406d4b0 0x7ff2e415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59192/0 (socket says 172.21.15.179:59192) 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 -- 172.21.15.179:0/502156724 learned_addr learned my addr 172.21.15.179:0/502156724 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 -- 172.21.15.179:0/502156724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e4142890 msgr2=0x7ff2e40b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e259c700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e4142890 0x7ff2e40b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e4142890 0x7ff2e40b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 -- 172.21.15.179:0/502156724 >> v1:172.21.15.195:6789/0 conn(0x7ff2e40b2ca0 legacy=0x7ff2e406cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 -- 172.21.15.179:0/502156724 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40b1c80 con 0x7ff2e406d4b0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e2d9d700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e406d4b0 0x7ff2e415d830 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009e50 tx=0x7ff2dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a138dc78b5541a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e1d9b700 1 -- 172.21.15.179:0/502156724 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00e070 con 0x7ff2e406d4b0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e1d9b700 1 -- 172.21.15.179:0/502156724 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc003a40 con 0x7ff2e406d4b0 2024-05-08T22:34:11.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e1d9b700 1 -- 172.21.15.179:0/502156724 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc003df0 con 0x7ff2e406d4b0 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- 172.21.15.179:0/502156724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e406d4b0 msgr2=0x7ff2e415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e406d4b0 0x7ff2e415d830 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009e50 tx=0x7ff2dc00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- 172.21.15.179:0/502156724 shutdown_connections 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e406d4b0 0x7ff2e415d830 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 --2- 172.21.15.179:0/502156724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e4142890 0x7ff2e40b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- 172.21.15.179:0/502156724 >> 172.21.15.179:0/502156724 conn(0x7ff2e40cea30 msgr2=0x7ff2e40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- 172.21.15.179:0/502156724 shutdown_connections 2024-05-08T22:34:11.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.386+0000 7ff2e359e700 1 -- 172.21.15.179:0/502156724 wait complete. 2024-05-08T22:34:11.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 Processor -- start 2024-05-08T22:34:11.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- start start 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e406d4b0 0x7ff2e4132180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2e4142890 0x7ff2e412c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e412e0f0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4155110 con 0x7ff2e4142890 2024-05-08T22:34:11.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e412c9b0 con 0x7ff2e406d4b0 2024-05-08T22:34:11.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e259c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e406d4b0 0x7ff2e4132180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59198/0 (socket says 172.21.15.179:59198) 2024-05-08T22:34:11.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 -- 172.21.15.179:0/2021179663 learned_addr learned my addr 172.21.15.179:0/2021179663 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:11.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e2d9d700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2e4142890 0x7ff2e412c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 -- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e406d4b0 msgr2=0x7ff2e4132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e406d4b0 0x7ff2e4132180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 -- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2e4142890 msgr2=0x7ff2e412c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2e4142890 0x7ff2e412c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 -- 172.21.15.179:0/2021179663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc009b30 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2d9d9b700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c550 tx=0x7ff2cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc011790 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc0118f0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e412e320 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc011ca0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.390+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e412e8e0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.394+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e404e700 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.394+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2cc01c460 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.394+0000 7ff2db7fe700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 0x7ff2c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.394+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff2cc084200 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.394+0000 7ff2e259c700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 0x7ff2c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.398+0000 7ff2e259c700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 0x7ff2c4068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40cbcf0 tx=0x7ff2d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.402+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff2cc0511b0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2e4138cb0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2db7fe700 1 -- 172.21.15.179:0/2021179663 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff2cc0547d0 con 0x7ff2e40b2ca0 2024-05-08T22:34:11.574 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 msgr2=0x7ff2c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 0x7ff2c4068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40cbcf0 tx=0x7ff2d4008040 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 msgr2=0x7ff2e4154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c550 tx=0x7ff2cc00c860 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 shutdown_connections 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2c4066730 0x7ff2c4068bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2e4142890 0x7ff2e412c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e40b2ca0 0x7ff2e4154bd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 --2- 172.21.15.179:0/2021179663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2e406d4b0 0x7ff2e4132180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 >> 172.21.15.179:0/2021179663 conn(0x7ff2e40cea30 msgr2=0x7ff2e4137f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 shutdown_connections 2024-05-08T22:34:11.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.574+0000 7ff2e359e700 1 -- 172.21.15.179:0/2021179663 wait complete. 2024-05-08T22:34:11.589 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 Processor -- start 2024-05-08T22:34:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 -- start start 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 0x7fc97406f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc97415f630 0x7fc9740cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97415f1a0 con 0x7fc97406fb80 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9740703e0 con 0x7fc97415f630 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc979ca8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc974070100 con 0x7fc9741439d0 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc97415f630 0x7fc9740cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc978ca6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 0x7fc97406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc9794a7700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fc97406fb80 0x7fc97406b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60840/0 (socket says 172.21.15.179:60840) 2024-05-08T22:34:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc9794a7700 1 -- 172.21.15.179:0/3587555904 learned_addr learned my addr 172.21.15.179:0/3587555904 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 -- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 msgr2=0x7fc97406f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 0x7fc97406f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 -- 172.21.15.179:0/3587555904 >> v1:172.21.15.195:6789/0 conn(0x7fc97406fb80 legacy=0x7fc97406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 -- 172.21.15.179:0/3587555904 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97415eb50 con 0x7fc97415f630 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc978ca6700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 0x7fc97406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.702+0000 7fc973fff700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97415f630 0x7fc9740cec70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc960009850 tx=0x7fc960009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3cf01fb67dd0ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc9737fe700 1 -- 172.21.15.179:0/3587555904 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960011830 con 0x7fc97415f630 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc9737fe700 1 -- 172.21.15.179:0/3587555904 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960011990 con 0x7fc97415f630 2024-05-08T22:34:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc9737fe700 1 -- 172.21.15.179:0/3587555904 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960011d40 con 0x7fc97415f630 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 -- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97415f630 msgr2=0x7fc9740cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97415f630 0x7fc9740cec70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc960009850 tx=0x7fc960009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 -- 172.21.15.179:0/3587555904 shutdown_connections 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97415f630 0x7fc9740cec70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3587555904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9741439d0 0x7fc97406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 -- 172.21.15.179:0/3587555904 >> 172.21.15.179:0/3587555904 conn(0x7fc9740cfb50 msgr2=0x7fc9740cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 -- 172.21.15.179:0/3587555904 shutdown_connections 2024-05-08T22:34:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.706+0000 7fc979ca8700 1 -- 172.21.15.179:0/3587555904 wait complete. 2024-05-08T22:34:11.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 Processor -- start 2024-05-08T22:34:11.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- start start 2024-05-08T22:34:11.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 0x7fc97413b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc97415f630 0x7fc974139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97413a5e0 con 0x7fc9741439d0 2024-05-08T22:34:11.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97413a8c0 con 0x7fc97406fb80 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97413aba0 con 0x7fc97415f630 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59206/0 (socket says 172.21.15.179:59206) 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 -- 172.21.15.179:0/3532457148 learned_addr learned my addr 172.21.15.179:0/3532457148 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc978ca6700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 0x7fc97413b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc9794a7700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc97415f630 0x7fc974139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 -- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc97415f630 msgr2=0x7fc974139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc97415f630 0x7fc974139680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 -- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 msgr2=0x7fc97413b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 0x7fc97413b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 -- 172.21.15.179:0/3532457148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680094b0 con 0x7fc9741439d0 2024-05-08T22:34:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc978ca6700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 0x7fc97413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:11.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc973fff700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc960014040 tx=0x7fc960004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9600036c0 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96001e070 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600094b0 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960003a80 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.710+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97413b160 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97413b380 con 0x7fc9741439d0 2024-05-08T22:34:11.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc960003be0 con 0x7fc9741439d0 2024-05-08T22:34:11.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc9717fa700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 0x7fc958068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:11.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc960020070 con 0x7fc9741439d0 2024-05-08T22:34:11.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc978ca6700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 0x7fc958068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:11.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.714+0000 7fc978ca6700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 0x7fc958068c20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9740ce820 tx=0x7fc968008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:11.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.722+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc960052610 con 0x7fc9741439d0 2024-05-08T22:34:11.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.902+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc97406f640 con 0x7fc9741439d0 2024-05-08T22:34:11.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.902+0000 7fc9717fa700 1 -- 172.21.15.179:0/3532457148 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc960055c30 con 0x7fc9741439d0 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 msgr2=0x7fc958068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 0x7fc958068c20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9740ce820 tx=0x7fc968008040 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 msgr2=0x7fc97415a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc960014040 tx=0x7fc960004770 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 shutdown_connections 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc958066760 0x7fc958068c20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc97415f630 0x7fc974139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9741439d0 0x7fc97415a6b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 --2- 172.21.15.179:0/3532457148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc97406fb80 0x7fc97413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 >> 172.21.15.179:0/3532457148 conn(0x7fc9740cfb50 msgr2=0x7fc97406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 shutdown_connections 2024-05-08T22:34:11.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:11.906+0000 7fc979ca8700 1 -- 172.21.15.179:0/3532457148 wait complete. 2024-05-08T22:34:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:11.919 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:11.920 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:11.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:12.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 Processor -- start 2024-05-08T22:34:12.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 -- start start 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2500ce060 0x7fc25006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc250161ac0 con 0x7fc2500cc310 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500cdaa0 con 0x7fc2500add40 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc257047700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500ae550 con 0x7fc2500ce060 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc24ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc2500cc310 0x7fc2500ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60882/0 (socket says 172.21.15.179:60882) 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc24ffff700 1 -- 172.21.15.179:0/779244731 learned_addr learned my addr 172.21.15.179:0/779244731 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc254de3700 1 --2- 172.21.15.179:0/779244731 >> v2:172.21.15.179:3300/0 conn(0x7fc2500ce060 0x7fc25006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc24f7fe700 1 -- 172.21.15.179:0/779244731 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392351060 0 0) 0x7fc250161ac0 con 0x7fc2500cc310 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc24f7fe700 1 -- 172.21.15.179:0/779244731 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2380035d0 con 0x7fc2500cc310 2024-05-08T22:34:12.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc254de3700 1 -- 172.21.15.179:0/779244731 >> v1:172.21.15.195:6789/0 conn(0x7fc2500add40 legacy=0x7fc25006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc254de3700 1 -- 172.21.15.179:0/779244731 >> v1:172.21.15.179:6789/0 conn(0x7fc2500cc310 legacy=0x7fc2500ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.038+0000 7fc254de3700 1 -- 172.21.15.179:0/779244731 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25006adc0 con 0x7fc2500ce060 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc254de3700 1 --2- 172.21.15.179:0/779244731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500ce060 0x7fc25006ce50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc244009850 tx=0x7fc244009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efed945fedb5a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc24f7fe700 1 -- 172.21.15.179:0/779244731 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244011830 con 0x7fc2500ce060 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc24f7fe700 1 -- 172.21.15.179:0/779244731 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244011990 con 0x7fc2500ce060 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc24f7fe700 1 -- 172.21.15.179:0/779244731 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244011d00 con 0x7fc2500ce060 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- 172.21.15.179:0/779244731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500ce060 msgr2=0x7fc25006ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 --2- 172.21.15.179:0/779244731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500ce060 0x7fc25006ce50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc244009850 tx=0x7fc244009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- 172.21.15.179:0/779244731 shutdown_connections 2024-05-08T22:34:12.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 --2- 172.21.15.179:0/779244731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500ce060 0x7fc25006ce50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- 172.21.15.179:0/779244731 >> 172.21.15.179:0/779244731 conn(0x7fc2500cb660 msgr2=0x7fc2500cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- 172.21.15.179:0/779244731 shutdown_connections 2024-05-08T22:34:12.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- 172.21.15.179:0/779244731 wait complete. 2024-05-08T22:34:12.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 Processor -- start 2024-05-08T22:34:12.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.042+0000 7fc257047700 1 -- start start 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500cc310 0x7fc25015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 0x7fc2500af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25015b960 con 0x7fc2500add40 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500b1fc0 con 0x7fc2500ce060 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c4760 con 0x7fc2500cc310 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59224/0 (socket says 172.21.15.179:59224) 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 -- 172.21.15.179:0/2605258044 learned_addr learned my addr 172.21.15.179:0/2605258044 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc24ffff700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500cc310 0x7fc25015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc2555e4700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 0x7fc2500af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 -- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500cc310 msgr2=0x7fc25015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500cc310 0x7fc25015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 -- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 msgr2=0x7fc2500af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 0x7fc2500af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 -- 172.21.15.179:0/2605258044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2440094b0 con 0x7fc2500add40 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc2555e4700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 0x7fc2500af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc254de3700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc244009ba0 tx=0x7fc244004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244004550 con 0x7fc2500add40 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24401e070 con 0x7fc2500add40 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25015bb80 con 0x7fc2500add40 2024-05-08T22:34:12.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244003900 con 0x7fc2500add40 2024-05-08T22:34:12.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2500c4480 con 0x7fc2500add40 2024-05-08T22:34:12.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.046+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250067770 con 0x7fc2500add40 2024-05-08T22:34:12.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.050+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc244003a60 con 0x7fc2500add40 2024-05-08T22:34:12.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.050+0000 7fc24d7fa700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.050+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc244085520 con 0x7fc2500add40 2024-05-08T22:34:12.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.050+0000 7fc24ffff700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.050+0000 7fc24ffff700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 0x7fc23c068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2500ce4b0 tx=0x7fc240001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.054+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc244052440 con 0x7fc2500add40 2024-05-08T22:34:12.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.230+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc25006abb0 con 0x7fc2500add40 2024-05-08T22:34:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc24d7fa700 1 -- 172.21.15.179:0/2605258044 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7fc244055a60 con 0x7fc2500add40 2024-05-08T22:34:12.235 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:12.235 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 msgr2=0x7fc23c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 0x7fc23c068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2500ce4b0 tx=0x7fc240001a90 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 msgr2=0x7fc25015a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc244009ba0 tx=0x7fc244004770 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 shutdown_connections 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2500ce060 0x7fc2500af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2500cc310 0x7fc25015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 --2- 172.21.15.179:0/2605258044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2500add40 0x7fc25015a970 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 >> 172.21.15.179:0/2605258044 conn(0x7fc2500cb660 msgr2=0x7fc2500ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.234+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 shutdown_connections 2024-05-08T22:34:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.238+0000 7fc257047700 1 -- 172.21.15.179:0/2605258044 wait complete. 2024-05-08T22:34:12.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:12.251 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:12.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.370+0000 7f538d4cc700 1 Processor -- start 2024-05-08T22:34:12.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.370+0000 7f538d4cc700 1 -- start start 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538d4cc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53880ae760 0x7f53880cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538d4cc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880cb560 0x7f53880ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538d4cc700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880ca3a0 con 0x7f53880cd450 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538d4cc700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880cd8e0 con 0x7f53880ae760 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538d4cc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53880ca680 con 0x7f53880cb560 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53880ae760 0x7f53880cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538cccb700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f53880cd450 0x7f53880b4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60854/0 (socket says 172.21.15.179:60854) 2024-05-08T22:34:12.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f537f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880cb560 0x7f53880ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f538cccb700 1 -- 172.21.15.179:0/2051974184 learned_addr learned my addr 172.21.15.179:0/2051974184 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 -- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880cb560 msgr2=0x7f53880ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 --2- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880cb560 0x7f53880ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 -- 172.21.15.179:0/2051974184 >> v1:172.21.15.195:6789/0 conn(0x7f53880cd450 legacy=0x7f53880b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 -- 172.21.15.179:0/2051974184 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880ca960 con 0x7f53880ae760 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f5387fff700 1 --2- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880ae760 0x7f53880cb020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f537800baf0 tx=0x7f537800be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b6e733092dfcd5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f53877fe700 1 -- 172.21.15.179:0/2051974184 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800dc70 con 0x7f53880ae760 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f53877fe700 1 -- 172.21.15.179:0/2051974184 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537800ddd0 con 0x7f53880ae760 2024-05-08T22:34:12.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.374+0000 7f53877fe700 1 -- 172.21.15.179:0/2051974184 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378010690 con 0x7f53880ae760 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880ae760 msgr2=0x7f53880cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 --2- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880ae760 0x7f53880cb020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f537800baf0 tx=0x7f537800be00 comp rx=0 tx=0).stop 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- 172.21.15.179:0/2051974184 shutdown_connections 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 --2- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880cb560 0x7f53880ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 --2- 172.21.15.179:0/2051974184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880ae760 0x7f53880cb020 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- 172.21.15.179:0/2051974184 >> 172.21.15.179:0/2051974184 conn(0x7f5388098a70 msgr2=0x7f53880b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- 172.21.15.179:0/2051974184 shutdown_connections 2024-05-08T22:34:12.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- 172.21.15.179:0/2051974184 wait complete. 2024-05-08T22:34:12.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 Processor -- start 2024-05-08T22:34:12.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.378+0000 7f538d4cc700 1 -- start start 2024-05-08T22:34:12.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880ae760 0x7f53880c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880cd450 0x7f53880b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388146700 con 0x7f53880cb560 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388146950 con 0x7f53880cd450 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5388146c30 con 0x7f53880ae760 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59228/0 (socket says 172.21.15.179:59228) 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 -- 172.21.15.179:0/4129821162 learned_addr learned my addr 172.21.15.179:0/4129821162 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f5387fff700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880ae760 0x7f53880c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538cccb700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880cd450 0x7f53880b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 -- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880ae760 msgr2=0x7f53880c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880ae760 0x7f53880c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 -- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880cd450 msgr2=0x7f53880b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880cd450 0x7f53880b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 -- 172.21.15.179:0/4129821162 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537800b750 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f537f7fe700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f537000e9d0 tx=0x7f537000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370009720 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537000ee70 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5388146f10 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000ca50 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.382+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53880c8fe0 con 0x7f53880cb560 2024-05-08T22:34:12.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538804e700 con 0x7f53880cb560 2024-05-08T22:34:12.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f537000cbb0 con 0x7f53880cb560 2024-05-08T22:34:12.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f53857fa700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5370084440 con 0x7f53880cb560 2024-05-08T22:34:12.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f5387fff700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.386+0000 7f5387fff700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 0x7f5368068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f537800bfd0 tx=0x7f537801b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.394+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5370051480 con 0x7f53880cb560 2024-05-08T22:34:12.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f538806c9b0 con 0x7f53880cb560 2024-05-08T22:34:12.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f53857fa700 1 -- 172.21.15.179:0/4129821162 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f537000ce60 con 0x7f53880cb560 2024-05-08T22:34:12.567 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:12.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 msgr2=0x7f5368068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 0x7f5368068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f537800bfd0 tx=0x7f537801b040 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 msgr2=0x7f53880b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.566+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f537000e9d0 tx=0x7f537000ece0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 shutdown_connections 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53880cd450 0x7f53880b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53880cb560 0x7f53880b0e60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 --2- 172.21.15.179:0/4129821162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53880ae760 0x7f53880c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 >> 172.21.15.179:0/4129821162 conn(0x7f5388098a70 msgr2=0x7f53880ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 shutdown_connections 2024-05-08T22:34:12.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.570+0000 7f538d4cc700 1 -- 172.21.15.179:0/4129821162 wait complete. 2024-05-08T22:34:12.582 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:12.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 Processor -- start 2024-05-08T22:34:12.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 -- start start 2024-05-08T22:34:12.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 0x7f79bc0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79bc0ce2b0 0x7f79bc06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc15d4a0 con 0x7f79bc0d1d20 2024-05-08T22:34:12.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc06abb0 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c2014700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0d2500 con 0x7f79bc15d0c0 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1012700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 0x7f79bc0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1813700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79bc0ce2b0 0x7f79bc06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c0811700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f79bc0d1d20 0x7f79bc0cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60868/0 (socket says 172.21.15.179:60868) 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c0811700 1 -- 172.21.15.179:0/572156297 learned_addr learned my addr 172.21.15.179:0/572156297 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79b3fff700 1 -- 172.21.15.179:0/572156297 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228958711 0 0) 0x7f79bc15d4a0 con 0x7f79bc0d1d20 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79b3fff700 1 -- 172.21.15.179:0/572156297 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a80035d0 con 0x7f79bc0d1d20 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1813700 1 -- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 msgr2=0x7f79bc0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1813700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 0x7f79bc0b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1813700 1 -- 172.21.15.179:0/572156297 >> v1:172.21.15.195:6789/0 conn(0x7f79bc0d1d20 legacy=0x7f79bc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1813700 1 -- 172.21.15.179:0/572156297 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0d27e0 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.698+0000 7f79c1012700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 0x7f79bc0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c1813700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0ce2b0 0x7f79bc06b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d90 tx=0x7f79b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d148f793d5cbb3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79b3fff700 1 -- 172.21.15.179:0/572156297 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400e070 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79b3fff700 1 -- 172.21.15.179:0/572156297 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b400bb20 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79b3fff700 1 -- 172.21.15.179:0/572156297 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400be90 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0ce2b0 msgr2=0x7f79bc06b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0ce2b0 0x7f79bc06b370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d90 tx=0x7f79b400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- 172.21.15.179:0/572156297 shutdown_connections 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0ce2b0 0x7f79bc06b370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- 172.21.15.179:0/572156297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc15d0c0 0x7f79bc0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- 172.21.15.179:0/572156297 >> 172.21.15.179:0/572156297 conn(0x7f79bc0d2ed0 msgr2=0x7f79bc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- 172.21.15.179:0/572156297 shutdown_connections 2024-05-08T22:34:12.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- 172.21.15.179:0/572156297 wait complete. 2024-05-08T22:34:12.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 Processor -- start 2024-05-08T22:34:12.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 -- start start 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc0ce2b0 0x7f79bc157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0d1d20 0x7f79bc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.702+0000 7f79c2014700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc15a510 con 0x7f79bc15d0c0 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0cd000 con 0x7f79bc0d1d20 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0ca6c0 con 0x7f79bc0ce2b0 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59230/0 (socket says 172.21.15.179:59230) 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1012700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc0ce2b0 0x7f79bc157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 -- 172.21.15.179:0/2877959290 learned_addr learned my addr 172.21.15.179:0/2877959290 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c0811700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0d1d20 0x7f79bc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 -- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc0ce2b0 msgr2=0x7f79bc157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc0ce2b0 0x7f79bc157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 -- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0d1d20 msgr2=0x7f79bc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0d1d20 0x7f79bc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 -- 172.21.15.179:0/2877959290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80094b0 con 0x7f79bc15d0c0 2024-05-08T22:34:12.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c1813700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f79b4014040 tx=0x7f79b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004800 con 0x7f79bc15d0c0 2024-05-08T22:34:12.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b4009a70 con 0x7f79bc15d0c0 2024-05-08T22:34:12.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4020070 con 0x7f79bc15d0c0 2024-05-08T22:34:12.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40276c0 con 0x7f79bc15d0c0 2024-05-08T22:34:12.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc159f50 con 0x7f79bc15d0c0 2024-05-08T22:34:12.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.706+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc0ca880 con 0x7f79bc15d0c0 2024-05-08T22:34:12.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.710+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f79b4012430 con 0x7f79bc15d0c0 2024-05-08T22:34:12.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.710+0000 7f79b1ffb700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 0x7f79a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:12.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.710+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f79b4022070 con 0x7f79bc15d0c0 2024-05-08T22:34:12.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.710+0000 7f79c1012700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 0x7f79a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:12.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.710+0000 7f79c1012700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 0x7f79a0068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009ce0 tx=0x7f79b8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:12.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.718+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f79b4053040 con 0x7f79bc15d0c0 2024-05-08T22:34:12.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.890+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79bc04e700 con 0x7f79bc15d0c0 2024-05-08T22:34:12.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.890+0000 7f79b1ffb700 1 -- 172.21.15.179:0/2877959290 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79b402c020 con 0x7f79bc15d0c0 2024-05-08T22:34:12.891 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 msgr2=0x7f79a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 0x7f79a0068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009ce0 tx=0x7f79b8009240 comp rx=0 tx=0).stop 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 msgr2=0x7f79bc159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f79b4014040 tx=0x7f79b4004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 shutdown_connections 2024-05-08T22:34:12.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79a0066730 0x7f79a0068bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79bc0d1d20 0x7f79bc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79bc15d0c0 0x7f79bc159460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 --2- 172.21.15.179:0/2877959290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79bc0ce2b0 0x7f79bc157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 >> 172.21.15.179:0/2877959290 conn(0x7f79bc0d2ed0 msgr2=0x7f79bc0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 shutdown_connections 2024-05-08T22:34:12.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:12.894+0000 7f79c2014700 1 -- 172.21.15.179:0/2877959290 wait complete. 2024-05-08T22:34:12.906 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:13.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 Processor -- start 2024-05-08T22:34:13.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 -- start start 2024-05-08T22:34:13.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 0x7f64280ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f642806e090 0x7f642815e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280ccb40 con 0x7f642815eec0 2024-05-08T22:34:13.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642815f350 con 0x7f642806e090 2024-05-08T22:34:13.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e935700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64280cb370 con 0x7f6428158850 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642d132700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f642806e090 0x7f642815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e134700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f642815eec0 0x7f642815b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:60870/0 (socket says 172.21.15.179:60870) 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642d933700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 0x7f64280ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.022+0000 7f642e134700 1 -- 172.21.15.179:0/2420192613 learned_addr learned my addr 172.21.15.179:0/2420192613 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d132700 1 -- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 msgr2=0x7f64280ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d132700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 0x7f64280ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642c931700 1 -- 172.21.15.179:0/2420192613 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181037802 0 0) 0x7f64280ccb40 con 0x7f642815eec0 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d132700 1 -- 172.21.15.179:0/2420192613 >> v1:172.21.15.195:6789/0 conn(0x7f642815eec0 legacy=0x7f642815b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d132700 1 -- 172.21.15.179:0/2420192613 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642806e5c0 con 0x7f642806e090 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d933700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 0x7f64280ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642d132700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642806e090 0x7f642815e980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6410009850 tx=0x7f6410009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6192662c1f8764 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642c931700 1 -- 172.21.15.179:0/2420192613 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410011830 con 0x7f642806e090 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642c931700 1 -- 172.21.15.179:0/2420192613 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6410011990 con 0x7f642806e090 2024-05-08T22:34:13.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642c931700 1 -- 172.21.15.179:0/2420192613 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410011d40 con 0x7f642806e090 2024-05-08T22:34:13.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642806e090 msgr2=0x7f642815e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642806e090 0x7f642815e980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6410009850 tx=0x7f6410009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- 172.21.15.179:0/2420192613 shutdown_connections 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642806e090 0x7f642815e980 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 --2- 172.21.15.179:0/2420192613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6428158850 0x7f64280ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- 172.21.15.179:0/2420192613 >> 172.21.15.179:0/2420192613 conn(0x7f64280cea30 msgr2=0x7f64280cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- 172.21.15.179:0/2420192613 shutdown_connections 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- 172.21.15.179:0/2420192613 wait complete. 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 Processor -- start 2024-05-08T22:34:13.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.026+0000 7f642e935700 1 -- start start 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f642806e090 0x7f64281ee090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642815eec0 0x7f6428142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428143d50 con 0x7f6428158850 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64281eea50 con 0x7f642815eec0 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64281eed30 con 0x7f642806e090 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59236/0 (socket says 172.21.15.179:59236) 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 -- 172.21.15.179:0/3310195409 learned_addr learned my addr 172.21.15.179:0/3310195409 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d933700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f642806e090 0x7f64281ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e134700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642815eec0 0x7f6428142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 -- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f642806e090 msgr2=0x7f64281ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f642806e090 0x7f64281ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 -- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642815eec0 msgr2=0x7f6428142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642815eec0 0x7f6428142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 -- 172.21.15.179:0/3310195409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64180094b0 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642d132700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6410014040 tx=0x7f6410004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410019980 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64100094b0 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641001e070 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410019d20 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64280b1dc0 con 0x7f6428158850 2024-05-08T22:34:13.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.030+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642804e700 con 0x7f6428158850 2024-05-08T22:34:13.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.034+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f641002d450 con 0x7f6428158850 2024-05-08T22:34:13.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.034+0000 7f64267fc700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 0x7f6408068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.034+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6410085500 con 0x7f6428158850 2024-05-08T22:34:13.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.034+0000 7f642d933700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 0x7f6408068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.034+0000 7f642d933700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 0x7f6408068bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6418009ce0 tx=0x7f6418009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.042+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6410052420 con 0x7f6428158850 2024-05-08T22:34:13.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64280ad410 con 0x7f6428158850 2024-05-08T22:34:13.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f64267fc700 1 -- 172.21.15.179:0/3310195409 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6410055a40 con 0x7f6428158850 2024-05-08T22:34:13.219 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:13.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 msgr2=0x7f6408068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 0x7f6408068bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6418009ce0 tx=0x7f6418009240 comp rx=0 tx=0).stop 2024-05-08T22:34:13.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 msgr2=0x7f6428142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6410014040 tx=0x7f6410004770 comp rx=0 tx=0).stop 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 shutdown_connections 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6408066730 0x7f6408068bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f642815eec0 0x7f6428142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6428158850 0x7f6428142630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 --2- 172.21.15.179:0/3310195409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f642806e090 0x7f64281ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.218+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 >> 172.21.15.179:0/3310195409 conn(0x7f64280cea30 msgr2=0x7f64280cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.222+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 shutdown_connections 2024-05-08T22:34:13.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.222+0000 7f642e935700 1 -- 172.21.15.179:0/3310195409 wait complete. 2024-05-08T22:34:13.234 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:13.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 Processor -- start 2024-05-08T22:34:13.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 -- start start 2024-05-08T22:34:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb64c0b0ea0 0x7fb64c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c0b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c0cf0f0 con 0x7fb64c0b1280 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0cee10 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0c4290 con 0x7fb64c0c5d80 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb64c0b0ea0 0x7fb64c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64a7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb64c0b1280 0x7fb64c0b4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60892/0 (socket says 172.21.15.179:60892) 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64a7fc700 1 -- 172.21.15.179:0/757303215 learned_addr learned my addr 172.21.15.179:0/757303215 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64b7fe700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c0b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 -- 172.21.15.179:0/757303215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 msgr2=0x7fb64c0b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c0b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 -- 172.21.15.179:0/757303215 >> v1:172.21.15.179:6789/0 conn(0x7fb64c0b1280 legacy=0x7fb64c0b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 -- 172.21.15.179:0/757303215 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0c4570 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb64affd700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0cf9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009850 tx=0x7fb63c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23374bb179b85d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb649ffb700 1 -- 172.21.15.179:0/757303215 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c011830 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb649ffb700 1 -- 172.21.15.179:0/757303215 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c011990 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.358+0000 7fb649ffb700 1 -- 172.21.15.179:0/757303215 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c011d20 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/757303215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 msgr2=0x7fb64c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0cf9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009850 tx=0x7fb63c009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/757303215 shutdown_connections 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- 172.21.15.179:0/757303215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0cf9d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/757303215 >> 172.21.15.179:0/757303215 conn(0x7fb64c098a70 msgr2=0x7fb64c0cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/757303215 shutdown_connections 2024-05-08T22:34:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/757303215 wait complete. 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 Processor -- start 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- start start 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb64c0b1280 0x7fb64c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b0880 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c159040 con 0x7fb64c0b1280 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0afd80 con 0x7fb64c0c5d80 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59252/0 (socket says 172.21.15.179:59252) 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 -- 172.21.15.179:0/506328518 learned_addr learned my addr 172.21.15.179:0/506328518 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 -- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 msgr2=0x7fb64c144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 -- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb64c0b1280 msgr2=0x7fb64c0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb64c0b1280 0x7fb64c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 -- 172.21.15.179:0/506328518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0094b0 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64affd700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb63c000c00 tx=0x7fb63c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004400 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c01f070 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c0affe0 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003900 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.362+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c0b05a0 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c158e10 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb63c021030 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb6437fe700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 0x7fb62c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb63c0858d0 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb64a7fc700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 0x7fb62c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.366+0000 7fb64a7fc700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 0x7fb62c068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb64c15a9c0 tx=0x7fb634007af0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.374+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb64c158e10 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.542+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb64c158e10 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.542+0000 7fb6437fe700 1 -- 172.21.15.179:0/506328518 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb64c158e10 con 0x7fb64c0b0ea0 2024-05-08T22:34:13.543 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:13.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 msgr2=0x7fb62c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 0x7fb62c068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb64c15a9c0 tx=0x7fb634007af0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 msgr2=0x7fb64c0c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb63c000c00 tx=0x7fb63c004770 comp rx=0 tx=0).stop 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 shutdown_connections 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb62c066730 0x7fb62c068bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb64c0b1280 0x7fb64c0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb64c0c5d80 0x7fb64c144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 --2- 172.21.15.179:0/506328518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c0b0ea0 0x7fb64c0c77a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 >> 172.21.15.179:0/506328518 conn(0x7fb64c098a70 msgr2=0x7fb64c0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 shutdown_connections 2024-05-08T22:34:13.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.546+0000 7fb64bfff700 1 -- 172.21.15.179:0/506328518 wait complete. 2024-05-08T22:34:13.562 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:13.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 Processor -- start 2024-05-08T22:34:13.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 -- start start 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc85c15cf70 0x7fc85c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc85c15d420 0x7fc85c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c15d950 con 0x7fc85c159270 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c15dc30 con 0x7fc85c15cf70 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc8615fa700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0b1af0 con 0x7fc85c15d420 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.674+0000 7fc85b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc85c15cf70 0x7fc85c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc85c15cf70 0x7fc85c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59258/0 (socket says 172.21.15.179:59258) 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 -- 172.21.15.179:0/749317365 learned_addr learned my addr 172.21.15.179:0/749317365 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc85c159270 0x7fc85c0b22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60904/0 (socket says 172.21.15.179:60904) 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc860df9700 1 --2- 172.21.15.179:0/749317365 >> v2:172.21.15.195:3300/0 conn(0x7fc85c15d420 0x7fc85c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 -- 172.21.15.179:0/749317365 >> v2:172.21.15.195:3300/0 conn(0x7fc85c15d420 msgr2=0x7fc85c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 --2- 172.21.15.179:0/749317365 >> v2:172.21.15.195:3300/0 conn(0x7fc85c15d420 0x7fc85c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 -- 172.21.15.179:0/749317365 >> v1:172.21.15.179:6789/0 conn(0x7fc85c159270 legacy=0x7fc85c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85affd700 1 -- 172.21.15.179:0/749317365 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218212844 0 0) 0x7fc85c15d950 con 0x7fc85c159270 2024-05-08T22:34:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 -- 172.21.15.179:0/749317365 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0b1dd0 con 0x7fc85c15cf70 2024-05-08T22:34:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85b7fe700 1 --2- 172.21.15.179:0/749317365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c15cf70 0x7fc85c06b440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc85c06c690 tx=0x7fc84c00ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489487ca7c6d3115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85affd700 1 -- 172.21.15.179:0/749317365 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00cb70 con 0x7fc85c15cf70 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85affd700 1 -- 172.21.15.179:0/749317365 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00ccd0 con 0x7fc85c15cf70 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc85affd700 1 -- 172.21.15.179:0/749317365 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c018620 con 0x7fc85c15cf70 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- 172.21.15.179:0/749317365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c15cf70 msgr2=0x7fc85c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 --2- 172.21.15.179:0/749317365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c15cf70 0x7fc85c06b440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc85c06c690 tx=0x7fc84c00ad90 comp rx=0 tx=0).stop 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- 172.21.15.179:0/749317365 shutdown_connections 2024-05-08T22:34:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 --2- 172.21.15.179:0/749317365 >> v2:172.21.15.195:3300/0 conn(0x7fc85c15d420 0x7fc85c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 --2- 172.21.15.179:0/749317365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c15cf70 0x7fc85c06b440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- 172.21.15.179:0/749317365 >> 172.21.15.179:0/749317365 conn(0x7fc85c0ce960 msgr2=0x7fc85c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- 172.21.15.179:0/749317365 shutdown_connections 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- 172.21.15.179:0/749317365 wait complete. 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 Processor -- start 2024-05-08T22:34:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.678+0000 7fc8615fa700 1 -- start start 2024-05-08T22:34:13.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc85c15cf70 0x7fc85c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc85c15d420 0x7fc85c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1f6de0 con 0x7fc85c159270 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c153f20 con 0x7fc85c15cf70 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c154200 con 0x7fc85c15d420 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59268/0 (socket says 172.21.15.179:59268) 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 -- 172.21.15.179:0/1065039520 learned_addr learned my addr 172.21.15.179:0/1065039520 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85b7fe700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc85c15cf70 0x7fc85c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc860df9700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc85c15d420 0x7fc85c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 -- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc85c15d420 msgr2=0x7fc85c153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc85c15d420 0x7fc85c153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 -- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc85c15cf70 msgr2=0x7fc85c1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc85c15cf70 0x7fc85c1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 -- 172.21.15.179:0/1065039520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c00a710 con 0x7fc85c159270 2024-05-08T22:34:13.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc85bfff700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c580 tx=0x7fc85000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850004ca0 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850004e20 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c1544e0 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850007310 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.682+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c15c470 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc850007470 con 0x7fc85c159270 2024-05-08T22:34:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c04e700 con 0x7fc85c159270 2024-05-08T22:34:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc858ff9700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 0x7fc848068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc850084310 con 0x7fc85c159270 2024-05-08T22:34:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc85b7fe700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 0x7fc848068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.686+0000 7fc85b7fe700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 0x7fc848068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00afd0 tx=0x7fc84c012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:13.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.694+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc8500512c0 con 0x7fc85c159270 2024-05-08T22:34:13.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.870+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc85c159800 con 0x7fc85c159270 2024-05-08T22:34:13.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.870+0000 7fc858ff9700 1 -- 172.21.15.179:0/1065039520 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc850007720 con 0x7fc85c159270 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 msgr2=0x7fc848068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 0x7fc848068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00afd0 tx=0x7fc84c012000 comp rx=0 tx=0).stop 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 msgr2=0x7fc85c15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c580 tx=0x7fc85000c940 comp rx=0 tx=0).stop 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 shutdown_connections 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc848066730 0x7fc848068bf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc85c15cf70 0x7fc85c1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc85c15d420 0x7fc85c153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 --2- 172.21.15.179:0/1065039520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85c159270 0x7fc85c15bf30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 >> 172.21.15.179:0/1065039520 conn(0x7fc85c0ce960 msgr2=0x7fc85c157400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 shutdown_connections 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:13.874+0000 7fc8615fa700 1 -- 172.21.15.179:0/1065039520 wait complete. 2024-05-08T22:34:13.877 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:13.892 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:34:13.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:13.893 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:13.907 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:13.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:34:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:13.908 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:34:13.913 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:13.913 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:13.921 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:13.921 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:13.928 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:14.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 Processor -- start 2024-05-08T22:34:14.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 -- start start 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2930161790 0x7f2930164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2930165d50 con 0x7f293006de70 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293006ad70 con 0x7f293015ef80 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2936665700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293006e4b0 con 0x7f2930161790 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2934e62700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f293006de70 0x7f293006b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60916/0 (socket says 172.21.15.179:60916) 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2934e62700 1 -- 172.21.15.179:0/2320742926 learned_addr learned my addr 172.21.15.179:0/2320742926 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2935e64700 1 --2- 172.21.15.179:0/2320742926 >> v2:172.21.15.195:3300/0 conn(0x7f2930161790 0x7f2930164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909077045 0 0) 0x7f2930165d50 con 0x7f293006de70 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c0035d0 con 0x7f293006de70 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581579484 0 0) 0x7f293006ad70 con 0x7f293015ef80 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2930165d50 con 0x7f293015ef80 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2935e64700 1 -- 172.21.15.179:0/2320742926 >> v1:172.21.15.195:6789/0 conn(0x7f293015ef80 legacy=0x7f293006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261227145 0 0) 0x7f291c0035d0 con 0x7f293006de70 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2935e64700 1 -- 172.21.15.179:0/2320742926 >> v1:172.21.15.179:6789/0 conn(0x7f293006de70 legacy=0x7f293006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2935e64700 1 -- 172.21.15.179:0/2320742926 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29301654b0 con 0x7f2930161790 2024-05-08T22:34:14.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2935e64700 1 --2- 172.21.15.179:0/2320742926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930164db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f292c009850 tx=0x7f292c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed35645f63cfa78f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29280019f0 con 0x7f2930161790 2024-05-08T22:34:14.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c004030 con 0x7f2930161790 2024-05-08T22:34:14.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.042+0000 7f2927fff700 1 -- 172.21.15.179:0/2320742926 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c0043a0 con 0x7f2930161790 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 -- 172.21.15.179:0/2320742926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 msgr2=0x7f2930164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 --2- 172.21.15.179:0/2320742926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930164db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f292c009850 tx=0x7f292c009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 -- 172.21.15.179:0/2320742926 shutdown_connections 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 --2- 172.21.15.179:0/2320742926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930164db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 -- 172.21.15.179:0/2320742926 >> 172.21.15.179:0/2320742926 conn(0x7f29300ce9d0 msgr2=0x7f29300cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 -- 172.21.15.179:0/2320742926 shutdown_connections 2024-05-08T22:34:14.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.046+0000 7f2936665700 1 -- 172.21.15.179:0/2320742926 wait complete. 2024-05-08T22:34:14.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 Processor -- start 2024-05-08T22:34:14.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- start start 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293015ef80 0x7f2930160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29300c8dd0 con 0x7f293006de70 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293014dc90 con 0x7f2930161790 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293014d9b0 con 0x7f293015ef80 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59272/0 (socket says 172.21.15.179:59272) 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 -- 172.21.15.179:0/4290660143 learned_addr learned my addr 172.21.15.179:0/4290660143 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2934e62700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293015ef80 0x7f2930160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935e64700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 -- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293015ef80 msgr2=0x7f2930160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293015ef80 0x7f2930160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 -- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 msgr2=0x7f2930160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 -- 172.21.15.179:0/4290660143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0094b0 con 0x7f293006de70 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2935663700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f292000c520 tx=0x7f292000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29200052d0 con 0x7f293006de70 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2920005450 con 0x7f293006de70 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29300c9000 con 0x7f293006de70 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29200038e0 con 0x7f293006de70 2024-05-08T22:34:14.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.050+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29300c95c0 con 0x7f293006de70 2024-05-08T22:34:14.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2930067740 con 0x7f293006de70 2024-05-08T22:34:14.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2920003a40 con 0x7f293006de70 2024-05-08T22:34:14.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2925ffb700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 0x7f2914068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2920083a30 con 0x7f293006de70 2024-05-08T22:34:14.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2934e62700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 0x7f2914068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.054+0000 7f2934e62700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 0x7f2914068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f29301fa520 tx=0x7f2928001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.062+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f29200508c0 con 0x7f293006de70 2024-05-08T22:34:14.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.238+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f293004e700 con 0x7f293006de70 2024-05-08T22:34:14.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.238+0000 7f2925ffb700 1 -- 172.21.15.179:0/4290660143 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f2920053ee0 con 0x7f293006de70 2024-05-08T22:34:14.242 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:14.242 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 msgr2=0x7f2914068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 0x7f2914068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f29301fa520 tx=0x7f2928001a90 comp rx=0 tx=0).stop 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 msgr2=0x7f293014a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f292000c520 tx=0x7f292000c830 comp rx=0 tx=0).stop 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 shutdown_connections 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2914066730 0x7f2914068bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2930161790 0x7f2930160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293015ef80 0x7f2930160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 --2- 172.21.15.179:0/4290660143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293006de70 0x7f293014a350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 >> 172.21.15.179:0/4290660143 conn(0x7f29300ce9d0 msgr2=0x7f29300cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 shutdown_connections 2024-05-08T22:34:14.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.242+0000 7f2936665700 1 -- 172.21.15.179:0/4290660143 wait complete. 2024-05-08T22:34:14.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:14.257 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:14.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 Processor -- start 2024-05-08T22:34:14.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 -- start start 2024-05-08T22:34:14.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a84158850 0x7f0a840ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a8406e090 0x7f0a8415e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840ccb40 con 0x7f0a8415eec0 2024-05-08T22:34:14.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8415f350 con 0x7f0a8406e090 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a83fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840cb370 con 0x7f0a84158850 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a8406e090 0x7f0a8415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a837fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0a8415eec0 0x7f0a8415b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60926/0 (socket says 172.21.15.179:60926) 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a837fe700 1 -- 172.21.15.179:0/2492231125 learned_addr learned my addr 172.21.15.179:0/2492231125 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a84158850 0x7f0a840ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a81ffb700 1 -- 172.21.15.179:0/2492231125 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777515659 0 0) 0x7f0a840ccb40 con 0x7f0a8415eec0 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a81ffb700 1 -- 172.21.15.179:0/2492231125 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a700035d0 con 0x7f0a8415eec0 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 -- 172.21.15.179:0/2492231125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a84158850 msgr2=0x7f0a840ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a84158850 0x7f0a840ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 -- 172.21.15.179:0/2492231125 >> v1:172.21.15.179:6789/0 conn(0x7f0a8415eec0 legacy=0x7f0a8415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 -- 172.21.15.179:0/2492231125 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8406e5c0 con 0x7f0a8406e090 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a827fc700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a8415e980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009850 tx=0x7f0a6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb90c7d34e33976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a81ffb700 1 -- 172.21.15.179:0/2492231125 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c011830 con 0x7f0a8406e090 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a81ffb700 1 -- 172.21.15.179:0/2492231125 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c011990 con 0x7f0a8406e090 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.374+0000 7f0a81ffb700 1 -- 172.21.15.179:0/2492231125 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c011d40 con 0x7f0a8406e090 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- 172.21.15.179:0/2492231125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 msgr2=0x7f0a8415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a8415e980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009850 tx=0x7f0a6c009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:14.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- 172.21.15.179:0/2492231125 shutdown_connections 2024-05-08T22:34:14.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a8415e980 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- 172.21.15.179:0/2492231125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a84158850 0x7f0a840ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- 172.21.15.179:0/2492231125 >> 172.21.15.179:0/2492231125 conn(0x7f0a840cea30 msgr2=0x7f0a840cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- 172.21.15.179:0/2492231125 shutdown_connections 2024-05-08T22:34:14.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- 172.21.15.179:0/2492231125 wait complete. 2024-05-08T22:34:14.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 Processor -- start 2024-05-08T22:34:14.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- start start 2024-05-08T22:34:14.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84158850 0x7f0a84158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a8415eec0 0x7f0a840b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841402f0 con 0x7f0a8406e090 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8413c7c0 con 0x7f0a84158850 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a83fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8413bd60 con 0x7f0a8415eec0 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a82ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a82ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59282/0 (socket says 172.21.15.179:59282) 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.378+0000 7f0a82ffd700 1 -- 172.21.15.179:0/4082104153 learned_addr learned my addr 172.21.15.179:0/4082104153 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a837fe700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a8415eec0 0x7f0a840b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a827fc700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84158850 0x7f0a84158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 -- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a8415eec0 msgr2=0x7f0a840b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a8415eec0 0x7f0a840b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 -- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84158850 msgr2=0x7f0a84158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84158850 0x7f0a84158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 -- 172.21.15.179:0/4082104153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0094b0 con 0x7f0a8406e090 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a82ffd700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e870 tx=0x7f0a7400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740095a0 con 0x7f0a8406e090 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74013680 con 0x7f0a8406e090 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8415a9e0 con 0x7f0a8406e090 2024-05-08T22:34:14.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740139f0 con 0x7f0a8406e090 2024-05-08T22:34:14.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8413ff10 con 0x7f0a8406e090 2024-05-08T22:34:14.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8415aec0 con 0x7f0a8406e090 2024-05-08T22:34:14.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a74013b50 con 0x7f0a8406e090 2024-05-08T22:34:14.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.382+0000 7f0a7b7fe700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 0x7f0a64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.386+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0a74084340 con 0x7f0a8406e090 2024-05-08T22:34:14.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.386+0000 7f0a827fc700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 0x7f0a64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.386+0000 7f0a827fc700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 0x7f0a64068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0090b0 tx=0x7f0a6c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.390+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a8415aec0 con 0x7f0a8406e090 2024-05-08T22:34:14.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a8413fd70 con 0x7f0a8406e090 2024-05-08T22:34:14.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a7b7fe700 1 -- 172.21.15.179:0/4082104153 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a8413fd70 con 0x7f0a8406e090 2024-05-08T22:34:14.563 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:14.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 msgr2=0x7f0a64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 0x7f0a64068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0090b0 tx=0x7f0a6c01c040 comp rx=0 tx=0).stop 2024-05-08T22:34:14.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 msgr2=0x7f0a840ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e870 tx=0x7f0a7400eb80 comp rx=0 tx=0).stop 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 shutdown_connections 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a64066730 0x7f0a64068bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a84158850 0x7f0a84158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a8415eec0 0x7f0a840b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 --2- 172.21.15.179:0/4082104153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a8406e090 0x7f0a840ab8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 >> 172.21.15.179:0/4082104153 conn(0x7f0a840cea30 msgr2=0x7f0a841576e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 shutdown_connections 2024-05-08T22:34:14.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.562+0000 7f0a83fff700 1 -- 172.21.15.179:0/4082104153 wait complete. 2024-05-08T22:34:14.577 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:14.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.686+0000 7fe0bffff700 1 Processor -- start 2024-05-08T22:34:14.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.686+0000 7fe0bffff700 1 -- start start 2024-05-08T22:34:14.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0c0143600 0x7fe0c00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 0x7fe0c0161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c01615a0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00cad50 con 0x7fe0c0143600 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00cb030 con 0x7fe0c0160c00 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0c0143600 0x7fe0c00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0c0143600 0x7fe0c00cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59290/0 (socket says 172.21.15.179:59290) 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 -- 172.21.15.179:0/3993309353 learned_addr learned my addr 172.21.15.179:0/3993309353 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0b7fff700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 0x7fe0c0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 -- 172.21.15.179:0/3993309353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 msgr2=0x7fe0c0161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 0x7fe0c0161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 -- 172.21.15.179:0/3993309353 >> v1:172.21.15.195:6789/0 conn(0x7fe0c00b2ee0 legacy=0x7fe0c00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 -- 172.21.15.179:0/3993309353 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00cb310 con 0x7fe0c0143600 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0b7fff700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 0x7fe0c0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0beffd700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c0143600 0x7fe0c00cbdf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009910 tx=0x7fe0b0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d149b093386ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0be7fc700 1 -- 172.21.15.179:0/3993309353 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004230 con 0x7fe0c0143600 2024-05-08T22:34:14.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0be7fc700 1 -- 172.21.15.179:0/3993309353 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0004390 con 0x7fe0c0143600 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0be7fc700 1 -- 172.21.15.179:0/3993309353 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0009e60 con 0x7fe0c0143600 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- 172.21.15.179:0/3993309353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c0143600 msgr2=0x7fe0c00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c0143600 0x7fe0c00cbdf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009910 tx=0x7fe0b0009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- 172.21.15.179:0/3993309353 shutdown_connections 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0160c00 0x7fe0c0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 --2- 172.21.15.179:0/3993309353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c0143600 0x7fe0c00cbdf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- 172.21.15.179:0/3993309353 >> 172.21.15.179:0/3993309353 conn(0x7fe0c00b18d0 msgr2=0x7fe0c00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- 172.21.15.179:0/3993309353 shutdown_connections 2024-05-08T22:34:14.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- 172.21.15.179:0/3993309353 wait complete. 2024-05-08T22:34:14.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 Processor -- start 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.690+0000 7fe0bffff700 1 -- start start 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0143600 0x7fe0c006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c0160c00 0x7fe0c006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015ac00 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015aee0 con 0x7fe0c0160c00 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015b1c0 con 0x7fe0c0143600 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59302/0 (socket says 172.21.15.179:59302) 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 -- 172.21.15.179:0/2166311554 learned_addr learned my addr 172.21.15.179:0/2166311554 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:14.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0b7fff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0143600 0x7fe0c006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bf7fe700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c0160c00 0x7fe0c006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 -- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0143600 msgr2=0x7fe0c006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0143600 0x7fe0c006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 -- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c0160c00 msgr2=0x7fe0c006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c0160c00 0x7fe0c006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 -- 172.21.15.179:0/2166311554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0009570 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0beffd700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0004000 tx=0x7fe0b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000bbd0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b001f070 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c015b4a0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00036c0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.694+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c006e070 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c015b9c0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0b0021030 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0bcff9700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe0b00859f0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0b7fff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.698+0000 7fe0b7fff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 0x7fe0a0068bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0161500 tx=0x7fe0a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.706+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe0c015b9c0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.870+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0c015b9c0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bcff9700 1 -- 172.21.15.179:0/2166311554 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0c015b9c0 con 0x7fe0c00b2ee0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 msgr2=0x7fe0a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 0x7fe0a0068bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0161500 tx=0x7fe0a8008040 comp rx=0 tx=0).stop 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 msgr2=0x7fe0c015d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0004000 tx=0x7fe0b0004770 comp rx=0 tx=0).stop 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 shutdown_connections 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0a0066730 0x7fe0a0068bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c0160c00 0x7fe0c006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0c0143600 0x7fe0c006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 --2- 172.21.15.179:0/2166311554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0c00b2ee0 0x7fe0c015d970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 >> 172.21.15.179:0/2166311554 conn(0x7fe0c00b18d0 msgr2=0x7fe0c00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 shutdown_connections 2024-05-08T22:34:14.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:14.874+0000 7fe0bffff700 1 -- 172.21.15.179:0/2166311554 wait complete. 2024-05-08T22:34:14.889 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:15.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.006+0000 7f87b9b4d700 1 Processor -- start 2024-05-08T22:34:15.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.006+0000 7f87b9b4d700 1 -- start start 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40d29b0 0x7f87b40b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87b40cdb20 0x7f87b40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40d1910 con 0x7f87b415d160 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40ce4f0 con 0x7f87b40cdb20 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40b6f90 con 0x7f87b40d29b0 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f87b415d160 0x7f87b40d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60936/0 (socket says 172.21.15.179:60936) 2024-05-08T22:34:15.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 -- 172.21.15.179:0/4061430737 learned_addr learned my addr 172.21.15.179:0/4061430737 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b934c700 1 --2- 172.21.15.179:0/4061430737 >> v2:172.21.15.195:3300/0 conn(0x7f87b40cdb20 0x7f87b40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b3fff700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40d29b0 0x7f87b40b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64936506 0 0) 0x7f87b40d1910 con 0x7f87b415d160 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a40035d0 con 0x7f87b415d160 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408280740 0 0) 0x7f87a40035d0 con 0x7f87b415d160 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40d29b0 msgr2=0x7f87b40b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40d29b0 0x7f87b40b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 >> v2:172.21.15.195:3300/0 conn(0x7f87b40cdb20 msgr2=0x7f87b40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 --2- 172.21.15.179:0/4061430737 >> v2:172.21.15.195:3300/0 conn(0x7f87b40cdb20 0x7f87b40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40d1bf0 con 0x7f87b415d160 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f87a8001670 con 0x7f87b415d160 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 >> v1:172.21.15.179:6789/0 conn(0x7f87b415d160 legacy=0x7f87b40d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a4004410 0x7f87a40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87a4004f60 0x7f87a400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40ce4f0 con 0x7f87a4004f60 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40b6f90 con 0x7f87a4004410 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a400bf70 con 0x7f87a4003a20 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b934c700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87a4004f60 0x7f87a400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b3fff700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a4004410 0x7f87a40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 -- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a4004410 msgr2=0x7f87a40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a4004410 0x7f87a40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 -- 172.21.15.179:0/4061430737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87a4004f60 msgr2=0x7f87a400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87a4004f60 0x7f87a400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 -- 172.21.15.179:0/4061430737 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40035d0 con 0x7f87a4003a20 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87a4003e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c320 tx=0x7f87a40058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79904dd0fb8a365a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800ce60 con 0x7f87a4003a20 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8005980 con 0x7f87a4003a20 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b37fe700 1 -- 172.21.15.179:0/4061430737 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8005d10 con 0x7f87a4003a20 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 msgr2=0x7f87a4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/4061430737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87a4003e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c320 tx=0x7f87a40058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b934c700 1 -- 172.21.15.179:0/4061430737 reap_dead start 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- 172.21.15.179:0/4061430737 shutdown_connections 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- 172.21.15.179:0/4061430737 >> 172.21.15.179:0/4061430737 conn(0x7f87b40d3270 msgr2=0x7f87b40d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- 172.21.15.179:0/4061430737 shutdown_connections 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.010+0000 7f87b9b4d700 1 -- 172.21.15.179:0/4061430737 wait complete. 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 Processor -- start 2024-05-08T22:34:15.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- start start 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87b41f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 0x7f87b40b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40b52e0 con 0x7f87b40b4e60 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c82f0 con 0x7f87b40c8d90 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4157c50 con 0x7f87a4003a20 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b8b4b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87b41f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 0x7f87b40b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59330/0 (socket says 172.21.15.179:59330) 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b8b4b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87b41f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:35642/0 (socket says 172.21.15.179:35642) 2024-05-08T22:34:15.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 -- 172.21.15.179:0/3347612798 learned_addr learned my addr 172.21.15.179:0/3347612798 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 -- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 msgr2=0x7f87b41f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87b41f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 -- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 msgr2=0x7f87b40b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 0x7f87b40b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 -- 172.21.15.179:0/3347612798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40c8590 con 0x7f87b40b4e60 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b3fff700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 0x7f87b40b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b934c700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f87b40b48f0 tx=0x7f87ac0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00e070 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac003d10 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40c8870 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0117a0 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.014+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b41f2ae0 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b40cd5c0 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f87ac011900 con 0x7f87b40b4e60 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b17fa700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f87ac084790 con 0x7f87b40b4e60 2024-05-08T22:34:15.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.018+0000 7f87b8b4b700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87a8008fd0 tx=0x7f87a800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.026+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f87ac051740 con 0x7f87b40b4e60 2024-05-08T22:34:15.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.190+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87b40c8a60 con 0x7f87b40b4e60 2024-05-08T22:34:15.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b17fa700 1 -- 172.21.15.179:0/3347612798 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87ac054d60 con 0x7f87b40b4e60 2024-05-08T22:34:15.194 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 msgr2=0x7f8798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87a8008fd0 tx=0x7f87a800a000 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 msgr2=0x7f87b4146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f87b40b48f0 tx=0x7f87ac0111d0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 shutdown_connections 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b40c8d90 0x7f87b40b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40b4e60 0x7f87b4146b50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 --2- 172.21.15.179:0/3347612798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87a4003a20 0x7f87b41f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 >> 172.21.15.179:0/3347612798 conn(0x7f87b40d3270 msgr2=0x7f87b415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 shutdown_connections 2024-05-08T22:34:15.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.194+0000 7f87b9b4d700 1 -- 172.21.15.179:0/3347612798 wait complete. 2024-05-08T22:34:15.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:15.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 Processor -- start 2024-05-08T22:34:15.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 -- start start 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe85c0b31d0 0x7fe85c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0b4c70 con 0x7fe85c15bcc0 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0b4f10 con 0x7fe85c15c0a0 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c0b55d0 con 0x7fe85c0b31d0 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85a59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe85c15bcc0 0x7fe85c0b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46514/0 (socket says 172.21.15.179:46514) 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85a59c700 1 -- 172.21.15.179:0/552477672 learned_addr learned my addr 172.21.15.179:0/552477672 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:15.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe859d9b700 1 --2- 172.21.15.179:0/552477672 >> v2:172.21.15.195:3300/0 conn(0x7fe85c0b31d0 0x7fe85c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547654257 0 0) 0x7fe85c0b4c70 con 0x7fe85c15bcc0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8480035d0 con 0x7fe85c15bcc0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655069572 0 0) 0x7fe85c0b4f10 con 0x7fe85c15c0a0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c0b4c70 con 0x7fe85c15c0a0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761372134 0 0) 0x7fe8480035d0 con 0x7fe85c15bcc0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 >> v2:172.21.15.195:3300/0 conn(0x7fe85c0b31d0 msgr2=0x7fe85c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 --2- 172.21.15.179:0/552477672 >> v2:172.21.15.195:3300/0 conn(0x7fe85c0b31d0 0x7fe85c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 >> v1:172.21.15.195:6789/0 conn(0x7fe85c15c0a0 legacy=0x7fe85c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0b51f0 con 0x7fe85c15bcc0 2024-05-08T22:34:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.326+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe84c0019f0 con 0x7fe85c15bcc0 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 >> v1:172.21.15.179:6789/0 conn(0x7fe85c15bcc0 legacy=0x7fe85c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 0x7fe848004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe848004870 0x7fe848004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8480052f0 0x7fe84800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0b55d0 con 0x7fe848003c00 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84800c230 con 0x7fe848004870 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84800c3f0 con 0x7fe8480052f0 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe859d9b700 1 --2- 172.21.15.179:0/552477672 >> v2:172.21.15.195:3300/0 conn(0x7fe85c0b31d0 0x7fe85c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 0x7fe848004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85ad9d700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8480052f0 0x7fe84800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe859d9b700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe848004870 0x7fe848004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 -- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8480052f0 msgr2=0x7fe84800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8480052f0 0x7fe84800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 -- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe848004870 msgr2=0x7fe848004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe848004870 0x7fe848004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 -- 172.21.15.179:0/552477672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480035d0 con 0x7fe848003c00 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe859d9b700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe848004870 0x7fe848004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85ad9d700 1 -- 172.21.15.179:0/552477672 reap_dead start 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85a59c700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 0x7fe848004000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00a710 tx=0x7fe8480058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ad8e01a9d71dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0070d0 con 0x7fe848003c00 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c007250 con 0x7fe848003c00 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85959a700 1 -- 172.21.15.179:0/552477672 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c007600 con 0x7fe848003c00 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 -- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 msgr2=0x7fe848004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 0x7fe848004000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00a710 tx=0x7fe8480058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 -- 172.21.15.179:0/552477672 shutdown_connections 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 --2- 172.21.15.179:0/552477672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe848003c00 0x7fe848004000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 -- 172.21.15.179:0/552477672 >> 172.21.15.179:0/552477672 conn(0x7fe85c0cc5d0 msgr2=0x7fe85c0cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 -- 172.21.15.179:0/552477672 shutdown_connections 2024-05-08T22:34:15.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.330+0000 7fe85b59e700 1 -- 172.21.15.179:0/552477672 wait complete. 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 Processor -- start 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 -- start start 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe848003c00 0x7fe85c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 0x7fe85c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c13ccc0 con 0x7fe85c13c300 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c06cc90 con 0x7fe85c06c2d0 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c8950 con 0x7fe848003c00 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34956/0 (socket says 172.21.15.179:34956) 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 -- 172.21.15.179:0/3391081299 learned_addr learned my addr 172.21.15.179:0/3391081299 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85a59c700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe848003c00 0x7fe85c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85ad9d700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 0x7fe85c06c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 -- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe848003c00 msgr2=0x7fe85c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe848003c00 0x7fe85c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 -- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 msgr2=0x7fe85c06c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 0x7fe85c06c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe859d9b700 1 -- 172.21.15.179:0/3391081299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0b5b70 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.334+0000 7fe85ad9d700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 0x7fe85c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe859d9b700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe844011800 tx=0x7fe844011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844015870 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c0c8bf0 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8440159d0 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c0c91b0 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844015d40 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c04e700 con 0x7fe85c13c300 2024-05-08T22:34:15.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe844013400 con 0x7fe85c13c300 2024-05-08T22:34:15.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 0x7fe83c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe844084da0 con 0x7fe85c13c300 2024-05-08T22:34:15.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.338+0000 7fe85a59c700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 0x7fe83c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.342+0000 7fe85a59c700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 0x7fe83c068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000f80 tx=0x7fe84c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.346+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe844051cd0 con 0x7fe85c13c300 2024-05-08T22:34:15.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.514+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe85c0cf830 con 0x7fe85c13c300 2024-05-08T22:34:15.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.514+0000 7fe852ffd700 1 -- 172.21.15.179:0/3391081299 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe8440552f0 con 0x7fe85c13c300 2024-05-08T22:34:15.516 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 msgr2=0x7fe83c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 0x7fe83c068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000f80 tx=0x7fe84c009000 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 msgr2=0x7fe85c13c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe844011800 tx=0x7fe844011b10 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 shutdown_connections 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe83c066730 0x7fe83c068bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe85c06c2d0 0x7fe85c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe85c13c300 0x7fe85c13c780 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 --2- 172.21.15.179:0/3391081299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe848003c00 0x7fe85c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 >> 172.21.15.179:0/3391081299 conn(0x7fe85c0cc5d0 msgr2=0x7fe85c0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 shutdown_connections 2024-05-08T22:34:15.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.518+0000 7fe85b59e700 1 -- 172.21.15.179:0/3391081299 wait complete. 2024-05-08T22:34:15.531 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:15.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 Processor -- start 2024-05-08T22:34:15.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 -- start start 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff04806e180 0x7ff04806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 0x7ff0480b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480cc340 con 0x7ff04806dda0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04806b540 con 0x7ff04806e180 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04d5ea700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04806ac50 con 0x7ff0480b25d0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff047fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff04806dda0 0x7ff0480cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46518/0 (socket says 172.21.15.179:46518) 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff047fff700 1 -- 172.21.15.179:0/4033591768 learned_addr learned my addr 172.21.15.179:0/4033591768 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 --2- 172.21.15.179:0/4033591768 >> v2:172.21.15.195:3300/0 conn(0x7ff04806e180 0x7ff04806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04cde9700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 0x7ff0480b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790685889 0 0) 0x7ff038001110 con 0x7ff04806dda0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0340035d0 con 0x7ff04806dda0 2024-05-08T22:34:15.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228131963 0 0) 0x7ff0340035d0 con 0x7ff04806dda0 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 -- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 msgr2=0x7ff0480b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 0x7ff0480b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 -- 172.21.15.179:0/4033591768 >> v1:172.21.15.179:6789/0 conn(0x7ff04806dda0 legacy=0x7ff0480cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 -- 172.21.15.179:0/4033591768 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04806aef0 con 0x7ff04806e180 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff04cde9700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 0x7ff0480b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:15.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff0477fe700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806e180 0x7ff04806e5e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff030009850 tx=0x7ff030009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336323e91609545c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.638+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038001800 con 0x7ff04806e180 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff030004030 con 0x7ff04806e180 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff046ffd700 1 -- 172.21.15.179:0/4033591768 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300043a0 con 0x7ff04806e180 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806e180 msgr2=0x7ff04806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806e180 0x7ff04806e5e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff030009850 tx=0x7ff030009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- 172.21.15.179:0/4033591768 shutdown_connections 2024-05-08T22:34:15.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0480b25d0 0x7ff0480b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/4033591768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806e180 0x7ff04806e5e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- 172.21.15.179:0/4033591768 >> 172.21.15.179:0/4033591768 conn(0x7ff0480ce960 msgr2=0x7ff0480ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- 172.21.15.179:0/4033591768 shutdown_connections 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- 172.21.15.179:0/4033591768 wait complete. 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 Processor -- start 2024-05-08T22:34:15.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- start start 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806dda0 0x7ff0481fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 0x7ff0481f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048152c40 con 0x7ff0480b25d0 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04814f7d0 con 0x7ff04806dda0 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04d5ea700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04814f4f0 con 0x7ff04806e180 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.642+0000 7ff04cde9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34962/0 (socket says 172.21.15.179:34962) 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 -- 172.21.15.179:0/276582766 learned_addr learned my addr 172.21.15.179:0/276582766 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff047fff700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806dda0 0x7ff0481fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff0477fe700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 0x7ff0481f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 -- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 msgr2=0x7ff0481f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 0x7ff0481f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 -- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806dda0 msgr2=0x7ff0481fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806dda0 0x7ff0481fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 -- 172.21.15.179:0/276582766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009a70 con 0x7ff0480b25d0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff0477fe700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 0x7ff0481f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04cde9700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff040009e60 tx=0x7ff04000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040009cd0 con 0x7ff0480b25d0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000f040 con 0x7ff0480b25d0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0300094b0 con 0x7ff0480b25d0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000ce20 con 0x7ff0480b25d0 2024-05-08T22:34:15.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481fa430 con 0x7ff0480b25d0 2024-05-08T22:34:15.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04804e700 con 0x7ff0480b25d0 2024-05-08T22:34:15.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.646+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff040014480 con 0x7ff0480b25d0 2024-05-08T22:34:15.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.650+0000 7ff044ff9700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 0x7ff028068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.650+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff040084b20 con 0x7ff0480b25d0 2024-05-08T22:34:15.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.650+0000 7ff047fff700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 0x7ff028068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.650+0000 7ff047fff700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 0x7ff028068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0380013a0 tx=0x7ff0380058a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.654+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff040051b60 con 0x7ff0480b25d0 2024-05-08T22:34:15.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.834+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff04806d850 con 0x7ff0480b25d0 2024-05-08T22:34:15.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.834+0000 7ff044ff9700 1 -- 172.21.15.179:0/276582766 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff040055180 con 0x7ff0480b25d0 2024-05-08T22:34:15.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 msgr2=0x7ff028068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 0x7ff028068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0380013a0 tx=0x7ff0380058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 msgr2=0x7ff0480af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff040009e60 tx=0x7ff04000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 shutdown_connections 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff028066730 0x7ff028068bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0480b25d0 0x7ff0480af440 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04806e180 0x7ff0481f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 --2- 172.21.15.179:0/276582766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04806dda0 0x7ff0481fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 >> 172.21.15.179:0/276582766 conn(0x7ff0480ce960 msgr2=0x7ff0480cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 shutdown_connections 2024-05-08T22:34:15.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.838+0000 7ff04d5ea700 1 -- 172.21.15.179:0/276582766 wait complete. 2024-05-08T22:34:15.843 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:15.852 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:15.852 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:15.860 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:15.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 Processor -- start 2024-05-08T22:34:15.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- start start 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa8700d0650 0x7fa8700ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700b5500 con 0x7fa8700b41c0 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700b4630 con 0x7fa8701447c0 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700b48f0 con 0x7fa8700d0650 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa8700d0650 0x7fa8700ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa8700b41c0 0x7fa8700b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46528/0 (socket says 172.21.15.179:46528) 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86ffff700 1 -- 172.21.15.179:0/1651650144 learned_addr learned my addr 172.21.15.179:0/1651650144 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86effd700 1 -- 172.21.15.179:0/1651650144 >> v1:172.21.15.195:6789/0 conn(0x7fa8701447c0 legacy=0x7fa8700cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86effd700 1 -- 172.21.15.179:0/1651650144 >> v1:172.21.15.179:6789/0 conn(0x7fa8700b41c0 legacy=0x7fa8700b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86effd700 1 -- 172.21.15.179:0/1651650144 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700b4bd0 con 0x7fa8700d0650 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86effd700 1 --2- 172.21.15.179:0/1651650144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700d0650 0x7fa8700ce700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa868009850 tx=0x7fa868009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498c2901fb81329e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86e7fc700 1 -- 172.21.15.179:0/1651650144 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868011830 con 0x7fa8700d0650 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86e7fc700 1 -- 172.21.15.179:0/1651650144 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868011990 con 0x7fa8700d0650 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa86e7fc700 1 -- 172.21.15.179:0/1651650144 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868011d20 con 0x7fa8700d0650 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- 172.21.15.179:0/1651650144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700d0650 msgr2=0x7fa8700ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 --2- 172.21.15.179:0/1651650144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700d0650 0x7fa8700ce700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa868009850 tx=0x7fa868009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- 172.21.15.179:0/1651650144 shutdown_connections 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 --2- 172.21.15.179:0/1651650144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700d0650 0x7fa8700ce700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- 172.21.15.179:0/1651650144 >> 172.21.15.179:0/1651650144 conn(0x7fa8700b2840 msgr2=0x7fa8700b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.978+0000 7fa874e2b700 1 -- 172.21.15.179:0/1651650144 shutdown_connections 2024-05-08T22:34:15.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- 172.21.15.179:0/1651650144 wait complete. 2024-05-08T22:34:15.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 Processor -- start 2024-05-08T22:34:15.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- start start 2024-05-08T22:34:15.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8700d0650 0x7fa8700c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8701447c0 0x7fa8700ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870131bc0 con 0x7fa8700b41c0 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87013fa60 con 0x7fa8700d0650 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700caa70 con 0x7fa8701447c0 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34980/0 (socket says 172.21.15.179:34980) 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 -- 172.21.15.179:0/3848268726 learned_addr learned my addr 172.21.15.179:0/3848268726 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86effd700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8700d0650 0x7fa8700c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86ffff700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8701447c0 0x7fa8700ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 -- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8701447c0 msgr2=0x7fa8700ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8701447c0 0x7fa8700ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 -- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8700d0650 msgr2=0x7fa8700c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8700d0650 0x7fa8700c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:15.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 -- 172.21.15.179:0/3848268726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680094b0 con 0x7fa8700b41c0 2024-05-08T22:34:15.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa86f7fe700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa86000cbd0 tx=0x7fa86000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860003c20 con 0x7fa8700b41c0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700cad50 con 0x7fa8700b41c0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860003d80 con 0x7fa8700b41c0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000aae0 con 0x7fa8700b41c0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.982+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700af8d0 con 0x7fa8700b41c0 2024-05-08T22:34:15.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8700caf90 con 0x7fa8700b41c0 2024-05-08T22:34:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa86000ac40 con 0x7fa8700b41c0 2024-05-08T22:34:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa867fff700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 0x7fa854068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa860084530 con 0x7fa8700b41c0 2024-05-08T22:34:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa86effd700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 0x7fa854068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.986+0000 7fa86effd700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 0x7fa854068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa868009fd0 tx=0x7fa868008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:15.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:15.994+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8700caf90 con 0x7fa8700b41c0 2024-05-08T22:34:16.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.166+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8701316b0 con 0x7fa8700b41c0 2024-05-08T22:34:16.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa867fff700 1 -- 172.21.15.179:0/3848268726 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7fa8701316b0 con 0x7fa8700b41c0 2024-05-08T22:34:16.170 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:16.170 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 msgr2=0x7fa854068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 0x7fa854068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa868009fd0 tx=0x7fa868008030 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 msgr2=0x7fa8700af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa86000cbd0 tx=0x7fa86000cee0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 shutdown_connections 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa854066730 0x7fa854068bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8700d0650 0x7fa8700c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8701447c0 0x7fa8700ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 --2- 172.21.15.179:0/3848268726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8700b41c0 0x7fa8700af390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 >> 172.21.15.179:0/3848268726 conn(0x7fa8700b2840 msgr2=0x7fa870142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 shutdown_connections 2024-05-08T22:34:16.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.170+0000 7fa874e2b700 1 -- 172.21.15.179:0/3848268726 wait complete. 2024-05-08T22:34:16.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:16.185 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:16.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.306+0000 7f329cb0d700 1 Processor -- start 2024-05-08T22:34:16.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.306+0000 7f329cb0d700 1 -- start start 2024-05-08T22:34:16.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32980cfa20 0x7f32980b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32980aed90 con 0x7f32980cd980 2024-05-08T22:34:16.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32980b5070 con 0x7f329815e640 2024-05-08T22:34:16.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980aead0 con 0x7f32980cfa20 2024-05-08T22:34:16.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3296ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32980cfa20 0x7f32980b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3297fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f32980cd980 0x7f329806bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46536/0 (socket says 172.21.15.179:46536) 2024-05-08T22:34:16.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3297fff700 1 -- 172.21.15.179:0/3487921841 learned_addr learned my addr 172.21.15.179:0/3487921841 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3296ffd700 1 -- 172.21.15.179:0/3487921841 >> v1:172.21.15.195:6789/0 conn(0x7f329815e640 legacy=0x7f32980cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3296ffd700 1 -- 172.21.15.179:0/3487921841 >> v1:172.21.15.179:6789/0 conn(0x7f32980cd980 legacy=0x7f329806bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3296ffd700 1 -- 172.21.15.179:0/3487921841 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980ccbe0 con 0x7f32980cfa20 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f3296ffd700 1 --2- 172.21.15.179:0/3487921841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f32980b4170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f328c009850 tx=0x7f328c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9f55b7faedb536 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f32967fc700 1 -- 172.21.15.179:0/3487921841 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011830 con 0x7f32980cfa20 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f32967fc700 1 -- 172.21.15.179:0/3487921841 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c011990 con 0x7f32980cfa20 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f32967fc700 1 -- 172.21.15.179:0/3487921841 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011d20 con 0x7f32980cfa20 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- 172.21.15.179:0/3487921841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 msgr2=0x7f32980b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 --2- 172.21.15.179:0/3487921841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f32980b4170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f328c009850 tx=0x7f328c009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:16.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- 172.21.15.179:0/3487921841 shutdown_connections 2024-05-08T22:34:16.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 --2- 172.21.15.179:0/3487921841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f32980b4170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- 172.21.15.179:0/3487921841 >> 172.21.15.179:0/3487921841 conn(0x7f3298098a70 msgr2=0x7f32980b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- 172.21.15.179:0/3487921841 shutdown_connections 2024-05-08T22:34:16.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.310+0000 7f329cb0d700 1 -- 172.21.15.179:0/3487921841 wait complete. 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 Processor -- start 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 -- start start 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32980cd980 0x7f32980b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 0x7f329809c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329809d280 con 0x7f32980cfa20 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298154620 con 0x7f32980cd980 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f329cb0d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32981523e0 con 0x7f329815e640 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f3296ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f3296ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34996/0 (socket says 172.21.15.179:34996) 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f3296ffd700 1 -- 172.21.15.179:0/318342785 learned_addr learned my addr 172.21.15.179:0/318342785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f32977fe700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32980cd980 0x7f32980b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.314+0000 7f3297fff700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 0x7f329809c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 -- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 msgr2=0x7f329809c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 0x7f329809c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 -- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32980cd980 msgr2=0x7f32980b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32980cd980 0x7f32980b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 -- 172.21.15.179:0/318342785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0094b0 con 0x7f32980cfa20 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3297fff700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 0x7f329809c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f3296ffd700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3298066cc0 tx=0x7f328c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c0036c0 con 0x7f32980cfa20 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c003820 con 0x7f32980cfa20 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329809ccd0 con 0x7f32980cfa20 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c003bd0 con 0x7f32980cfa20 2024-05-08T22:34:16.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298153fd0 con 0x7f32980cfa20 2024-05-08T22:34:16.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f328c003d30 con 0x7f32980cfa20 2024-05-08T22:34:16.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.318+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32981547d0 con 0x7f32980cfa20 2024-05-08T22:34:16.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.322+0000 7f327ffff700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 0x7f3284068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.322+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f328c021030 con 0x7f32980cfa20 2024-05-08T22:34:16.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.322+0000 7f32977fe700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 0x7f3284068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.322+0000 7f32977fe700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 0x7f3284068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3288000f60 tx=0x7f3288009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.326+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f32981547d0 con 0x7f32980cfa20 2024-05-08T22:34:16.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f329806ce00 con 0x7f32980cfa20 2024-05-08T22:34:16.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f327ffff700 1 -- 172.21.15.179:0/318342785 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f328c02b0a0 con 0x7f32980cfa20 2024-05-08T22:34:16.494 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:16.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 msgr2=0x7f3284068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 0x7f3284068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3288000f60 tx=0x7f3288009040 comp rx=0 tx=0).stop 2024-05-08T22:34:16.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 msgr2=0x7f3298151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3298066cc0 tx=0x7f328c004770 comp rx=0 tx=0).stop 2024-05-08T22:34:16.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 shutdown_connections 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3284066730 0x7f3284068bf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f329815e640 0x7f329809c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32980cfa20 0x7f3298151ea0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 --2- 172.21.15.179:0/318342785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32980cd980 0x7f32980b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 >> 172.21.15.179:0/318342785 conn(0x7f3298098a70 msgr2=0x7f329815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 shutdown_connections 2024-05-08T22:34:16.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.494+0000 7f329cb0d700 1 -- 172.21.15.179:0/318342785 wait complete. 2024-05-08T22:34:16.509 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:16.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.626+0000 7fc2b3006700 1 Processor -- start 2024-05-08T22:34:16.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.626+0000 7fc2b3006700 1 -- start start 2024-05-08T22:34:16.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc2ac15cfb0 0x7fc2ac0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ac0d0270 con 0x7fc2ac15d390 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ac0b4a20 con 0x7fc2ac0ca970 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ac0b6c30 con 0x7fc2ac15cfb0 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34998/0 (socket says 172.21.15.179:34998) 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 -- 172.21.15.179:0/2044261961 learned_addr learned my addr 172.21.15.179:0/2044261961 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b2805700 1 --1- 172.21.15.179:0/2044261961 >> v1:172.21.15.179:6789/0 conn(0x7fc2ac15d390 0x7fc2ac160cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46550/0 (socket says 172.21.15.179:46550) 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b2004700 1 --2- 172.21.15.179:0/2044261961 >> v2:172.21.15.195:3300/0 conn(0x7fc2ac15cfb0 0x7fc2ac0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 -- 172.21.15.179:0/2044261961 >> v2:172.21.15.195:3300/0 conn(0x7fc2ac15cfb0 msgr2=0x7fc2ac0b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 --2- 172.21.15.179:0/2044261961 >> v2:172.21.15.195:3300/0 conn(0x7fc2ac15cfb0 0x7fc2ac0b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 -- 172.21.15.179:0/2044261961 >> v1:172.21.15.179:6789/0 conn(0x7fc2ac15d390 legacy=0x7fc2ac160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 -- 172.21.15.179:0/2044261961 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac0d07a0 con 0x7fc2ac0ca970 2024-05-08T22:34:16.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1803700 1 --2- 172.21.15.179:0/2044261961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0677c0 tx=0x7fc2a0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8efdd7813e1b92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1002700 1 -- 172.21.15.179:0/2044261961 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000b8a0 con 0x7fc2ac0ca970 2024-05-08T22:34:16.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1002700 1 -- 172.21.15.179:0/2044261961 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a000ba00 con 0x7fc2ac0ca970 2024-05-08T22:34:16.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b1002700 1 -- 172.21.15.179:0/2044261961 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000bd90 con 0x7fc2ac0ca970 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- 172.21.15.179:0/2044261961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac0ca970 msgr2=0x7fc2ac0b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 --2- 172.21.15.179:0/2044261961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0677c0 tx=0x7fc2a0009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- 172.21.15.179:0/2044261961 shutdown_connections 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 --2- 172.21.15.179:0/2044261961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac0ca970 0x7fc2ac0b66f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 --2- 172.21.15.179:0/2044261961 >> v2:172.21.15.195:3300/0 conn(0x7fc2ac15cfb0 0x7fc2ac0b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- 172.21.15.179:0/2044261961 >> 172.21.15.179:0/2044261961 conn(0x7fc2ac0b2770 msgr2=0x7fc2ac0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- 172.21.15.179:0/2044261961 shutdown_connections 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- 172.21.15.179:0/2044261961 wait complete. 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 Processor -- start 2024-05-08T22:34:16.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.630+0000 7fc2b3006700 1 -- start start 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2ac0ca970 0x7fc2ac14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2ac15cfb0 0x7fc2ac15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac1f6be0 con 0x7fc2ac15d390 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac1f5fd0 con 0x7fc2ac15cfb0 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac1f62b0 con 0x7fc2ac0ca970 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35000/0 (socket says 172.21.15.179:35000) 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 -- 172.21.15.179:0/244772215 learned_addr learned my addr 172.21.15.179:0/244772215 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b1803700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2ac15cfb0 0x7fc2ac15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2004700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2ac0ca970 0x7fc2ac14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 -- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2ac0ca970 msgr2=0x7fc2ac14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2ac0ca970 0x7fc2ac14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 -- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2ac15cfb0 msgr2=0x7fc2ac15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2ac15cfb0 0x7fc2ac15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 -- 172.21.15.179:0/244772215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009570 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b2805700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400cb10 tx=0x7fc2a400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400aac0 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a400ac20 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac1f6590 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4007070 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac160230 con 0x7fc2ac15d390 2024-05-08T22:34:16.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.634+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac04e700 con 0x7fc2ac15d390 2024-05-08T22:34:16.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.638+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2a40071d0 con 0x7fc2ac15d390 2024-05-08T22:34:16.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.638+0000 7fc29effd700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 0x7fc290068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.638+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc2a4084100 con 0x7fc2ac15d390 2024-05-08T22:34:16.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.638+0000 7fc2b2004700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 0x7fc290068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.638+0000 7fc2b2004700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 0x7fc290068c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009950 tx=0x7fc2a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.646+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc2a40511f0 con 0x7fc2ac15d390 2024-05-08T22:34:16.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.814+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2ac070470 con 0x7fc2ac15d390 2024-05-08T22:34:16.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.814+0000 7fc29effd700 1 -- 172.21.15.179:0/244772215 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2a4054810 con 0x7fc2ac15d390 2024-05-08T22:34:16.816 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 msgr2=0x7fc290068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 0x7fc290068c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009950 tx=0x7fc2a8008040 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 msgr2=0x7fc2ac0ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400cb10 tx=0x7fc2a400ced0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 shutdown_connections 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc290066760 0x7fc290068c20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2ac15cfb0 0x7fc2ac15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2ac15d390 0x7fc2ac0ac530 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 --2- 172.21.15.179:0/244772215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2ac0ca970 0x7fc2ac14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 >> 172.21.15.179:0/244772215 conn(0x7fc2ac0b2770 msgr2=0x7fc2ac0c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 shutdown_connections 2024-05-08T22:34:16.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.818+0000 7fc2b3006700 1 -- 172.21.15.179:0/244772215 wait complete. 2024-05-08T22:34:16.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:16.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 Processor -- start 2024-05-08T22:34:16.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 -- start start 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff9800b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff9800d1d20 0x7ff9800cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98015d4a0 con 0x7ff9800ce0e0 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9800ce570 con 0x7ff9800d1d20 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff985641700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98006abb0 con 0x7ff98015d0c0 2024-05-08T22:34:16.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff984e40700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff9800ce0e0 0x7ff980160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46554/0 (socket says 172.21.15.179:46554) 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.958+0000 7ff984e40700 1 -- 172.21.15.179:0/2862212535 learned_addr learned my addr 172.21.15.179:0/2862212535 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97f7fe700 1 --2- 172.21.15.179:0/2862212535 >> v2:172.21.15.195:3300/0 conn(0x7ff9800d1d20 0x7ff9800cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97ffff700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff9800b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624274395 0 0) 0x7ff98015d4a0 con 0x7ff9800ce0e0 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff96c0035d0 con 0x7ff9800ce0e0 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276647076 0 0) 0x7ff96c0035d0 con 0x7ff9800ce0e0 2024-05-08T22:34:16.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 msgr2=0x7ff9800b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff9800b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 >> v2:172.21.15.195:3300/0 conn(0x7ff9800d1d20 msgr2=0x7ff9800cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 --2- 172.21.15.179:0/2862212535 >> v2:172.21.15.195:3300/0 conn(0x7ff9800d1d20 0x7ff9800cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800d2500 con 0x7ff9800ce0e0 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff9780019f0 con 0x7ff9800ce0e0 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97ffff700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff9800b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:16.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 >> v1:172.21.15.179:6789/0 conn(0x7ff9800ce0e0 legacy=0x7ff980160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff96c003a20 0x7ff96c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff96c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96c004db0 0x7ff96c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800ce570 con 0x7ff96c004db0 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98006abb0 con 0x7ff96c003a20 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c00ba30 con 0x7ff98015d0c0 2024-05-08T22:34:16.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97f7fe700 1 --2- 172.21.15.179:0/2862212535 >> v2:172.21.15.195:3300/0 conn(0x7ff9800d1d20 0x7ff9800cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96c004db0 0x7ff96c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97ffff700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff96c003a20 0x7ff96c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 -- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 msgr2=0x7ff96c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff98015d0c0 0x7ff96c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 -- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff96c003a20 msgr2=0x7ff96c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff96c003a20 0x7ff96c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 -- 172.21.15.179:0/2862212535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680094b0 con 0x7ff96c004db0 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97ffff700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff96c003a20 0x7ff96c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96c004db0 0x7ff96c005230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97800a830 tx=0x7ff96c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58147bd7f14cb562 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780072d0 con 0x7ff96c004db0 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800ae90 con 0x7ff96c004db0 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff97effd700 1 -- 172.21.15.179:0/2862212535 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978007670 con 0x7ff96c004db0 2024-05-08T22:34:16.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 -- 172.21.15.179:0/2862212535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96c004db0 msgr2=0x7ff96c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 --2- 172.21.15.179:0/2862212535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96c004db0 0x7ff96c005230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97800a830 tx=0x7ff96c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff984e40700 1 -- 172.21.15.179:0/2862212535 reap_dead start 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 -- 172.21.15.179:0/2862212535 shutdown_connections 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 -- 172.21.15.179:0/2862212535 >> 172.21.15.179:0/2862212535 conn(0x7ff9800d2ed0 msgr2=0x7ff9800d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 -- 172.21.15.179:0/2862212535 shutdown_connections 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.962+0000 7ff985641700 1 -- 172.21.15.179:0/2862212535 wait complete. 2024-05-08T22:34:16.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 Processor -- start 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- start start 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 0x7ff9801f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 0x7ff9800c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800ccc00 con 0x7ff9800cc210 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98014cbd0 con 0x7ff9801f24b0 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800c79d0 con 0x7ff9800c7410 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35024/0 (socket says 172.21.15.179:35024) 2024-05-08T22:34:16.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 -- 172.21.15.179:0/1202870013 learned_addr learned my addr 172.21.15.179:0/1202870013 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97f7fe700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 0x7ff9800c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97ffff700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 0x7ff9801f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 -- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 msgr2=0x7ff9800c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 0x7ff9800c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 -- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 msgr2=0x7ff9801f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 0x7ff9801f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 -- 172.21.15.179:0/1202870013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800c7cb0 con 0x7ff9800cc210 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97ffff700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 0x7ff9801f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:16.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97f7fe700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 0x7ff9800c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff984e40700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff978011000 tx=0x7ff98015d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97801f600 con 0x7ff9800cc210 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97801f760 con 0x7ff9800cc210 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800c7f90 con 0x7ff9800cc210 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97801fb10 con 0x7ff9800cc210 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.966+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800ca040 con 0x7ff9800cc210 2024-05-08T22:34:16.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9800ca200 con 0x7ff9800cc210 2024-05-08T22:34:16.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff97801a030 con 0x7ff9800cc210 2024-05-08T22:34:16.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff97cff9700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 0x7ff960068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:16.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff978085210 con 0x7ff9800cc210 2024-05-08T22:34:16.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff97ffff700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 0x7ff960068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:16.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.970+0000 7ff97ffff700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 0x7ff960068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff9680096e0 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:16.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:16.974+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff978052250 con 0x7ff9800cc210 2024-05-08T22:34:17.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff98004e700 con 0x7ff9800cc210 2024-05-08T22:34:17.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff97cff9700 1 -- 172.21.15.179:0/1202870013 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff978055870 con 0x7ff9800cc210 2024-05-08T22:34:17.143 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 msgr2=0x7ff960068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 0x7ff960068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff9680096e0 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 msgr2=0x7ff9800cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.142+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff978011000 tx=0x7ff98015d630 comp rx=0 tx=0).stop 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 shutdown_connections 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff960066730 0x7ff960068bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9800cc210 0x7ff9800cc6c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9800c7410 0x7ff9800c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 --2- 172.21.15.179:0/1202870013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9801f24b0 0x7ff9801f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 >> 172.21.15.179:0/1202870013 conn(0x7ff9800d2ed0 msgr2=0x7ff98006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:17.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 shutdown_connections 2024-05-08T22:34:17.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.146+0000 7ff985641700 1 -- 172.21.15.179:0/1202870013 wait complete. 2024-05-08T22:34:17.158 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:17.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 Processor -- start 2024-05-08T22:34:17.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 -- start start 2024-05-08T22:34:17.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d0415d0c0 0x7f9d040b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9d040ce2b0 0x7f9d0406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0415d4a0 con 0x7f9d040d1d20 2024-05-08T22:34:17.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0406abb0 con 0x7f9d0415d0c0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d08fb8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040d2500 con 0x7f9d040ce2b0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.282+0000 7f9d037fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d0415d0c0 0x7f9d040b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d0415d0c0 0x7f9d040b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35030/0 (socket says 172.21.15.179:35030) 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 -- 172.21.15.179:0/528199668 learned_addr learned my addr 172.21.15.179:0/528199668 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d03fff700 1 --2- 172.21.15.179:0/528199668 >> v2:172.21.15.195:3300/0 conn(0x7f9d040ce2b0 0x7f9d0406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 -- 172.21.15.179:0/528199668 >> v2:172.21.15.195:3300/0 conn(0x7f9d040ce2b0 msgr2=0x7f9d0406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 --2- 172.21.15.179:0/528199668 >> v2:172.21.15.195:3300/0 conn(0x7f9d040ce2b0 0x7f9d0406b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 -- 172.21.15.179:0/528199668 >> v1:172.21.15.195:6789/0 conn(0x7f9d040d1d20 legacy=0x7f9d040cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d027fc700 1 -- 172.21.15.179:0/528199668 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254749272 0 0) 0x7f9d0415d4a0 con 0x7f9d040d1d20 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 -- 172.21.15.179:0/528199668 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040d27e0 con 0x7f9d0415d0c0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d037fe700 1 --2- 172.21.15.179:0/528199668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0415d0c0 0x7f9d040b8a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009910 tx=0x7f9cf8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acfbf718f06d5a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d027fc700 1 -- 172.21.15.179:0/528199668 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8011830 con 0x7f9d0415d0c0 2024-05-08T22:34:17.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d027fc700 1 -- 172.21.15.179:0/528199668 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8011990 con 0x7f9d0415d0c0 2024-05-08T22:34:17.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d027fc700 1 -- 172.21.15.179:0/528199668 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8011d40 con 0x7f9d0415d0c0 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 -- 172.21.15.179:0/528199668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0415d0c0 msgr2=0x7f9d040b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/528199668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0415d0c0 0x7f9d040b8a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009910 tx=0x7f9cf8009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 -- 172.21.15.179:0/528199668 shutdown_connections 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/528199668 >> v2:172.21.15.195:3300/0 conn(0x7f9d040ce2b0 0x7f9d0406b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/528199668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0415d0c0 0x7f9d040b8a80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 -- 172.21.15.179:0/528199668 >> 172.21.15.179:0/528199668 conn(0x7f9d040d2ed0 msgr2=0x7f9d040d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 -- 172.21.15.179:0/528199668 shutdown_connections 2024-05-08T22:34:17.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.286+0000 7f9d08fb8700 1 -- 172.21.15.179:0/528199668 wait complete. 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 Processor -- start 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- start start 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d040ce2b0 0x7f9d040caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d0415d0c0 0x7f9d040cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040c98e0 con 0x7f9d040d1d20 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040c9600 con 0x7f9d040ce2b0 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040c6ba0 con 0x7f9d0415d0c0 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35032/0 (socket says 172.21.15.179:35032) 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 -- 172.21.15.179:0/836697705 learned_addr learned my addr 172.21.15.179:0/836697705 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d037fe700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d040ce2b0 0x7f9d040caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d03fff700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d0415d0c0 0x7f9d040cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 -- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d0415d0c0 msgr2=0x7f9d040cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d0415d0c0 0x7f9d040cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 -- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d040ce2b0 msgr2=0x7f9d040caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d040ce2b0 0x7f9d040caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 -- 172.21.15.179:0/836697705 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009570 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d02ffd700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40011e0 tx=0x7f9cf400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40054e0 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf40036a0 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d040c6e80 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4003a50 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.290+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040caf80 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d040c6a00 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9cf4003bb0 con 0x7f9d040d1d20 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9cebfff700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 0x7f9cf0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9cf40839e0 con 0x7f9d040d1d20 2024-05-08T22:34:17.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9d037fe700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 0x7f9cf0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.294+0000 7f9d037fe700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 0x7f9cf0068c20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8014040 tx=0x7f9cf8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.302+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d040c6a00 con 0x7f9d040d1d20 2024-05-08T22:34:17.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9d040c73a0 con 0x7f9d040d1d20 2024-05-08T22:34:17.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9cebfff700 1 -- 172.21.15.179:0/836697705 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9cf4017020 con 0x7f9d040d1d20 2024-05-08T22:34:17.475 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:17.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 msgr2=0x7f9cf0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 0x7f9cf0068c20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8014040 tx=0x7f9cf8008030 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 msgr2=0x7f9d040c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40011e0 tx=0x7f9cf400ca30 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 shutdown_connections 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9cf0066760 0x7f9cf0068c20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d0415d0c0 0x7f9d040cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d040d1d20 0x7f9d040c60f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 --2- 172.21.15.179:0/836697705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d040ce2b0 0x7f9d040caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 >> 172.21.15.179:0/836697705 conn(0x7f9d040d2ed0 msgr2=0x7f9d040cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:17.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 shutdown_connections 2024-05-08T22:34:17.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.474+0000 7f9d08fb8700 1 -- 172.21.15.179:0/836697705 wait complete. 2024-05-08T22:34:17.490 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:17.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.602+0000 7fef394f2700 1 Processor -- start 2024-05-08T22:34:17.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.602+0000 7fef394f2700 1 -- start start 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.602+0000 7fef394f2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef340b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3415d950 con 0x7fef3415cf70 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3415dc30 con 0x7fef3406bb50 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b1af0 con 0x7fef34159270 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef337fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fef3415cf70 0x7fef3406b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46556/0 (socket says 172.21.15.179:46556) 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef337fe700 1 -- 172.21.15.179:0/2282450426 learned_addr learned my addr 172.21.15.179:0/2282450426 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef33fff700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef340b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297339927 0 0) 0x7fef3415d950 con 0x7fef3415cf70 2024-05-08T22:34:17.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef200046e0 con 0x7fef3415cf70 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231236592 0 0) 0x7fef200046e0 con 0x7fef3415cf70 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 msgr2=0x7fef340b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef340b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 >> v1:172.21.15.195:6789/0 conn(0x7fef3406bb50 legacy=0x7fef34159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef340b1dd0 con 0x7fef3415cf70 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fef240019f0 con 0x7fef3415cf70 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 >> v1:172.21.15.179:6789/0 conn(0x7fef3415cf70 legacy=0x7fef3406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef20004b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef200050a0 0x7fef20005980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef20005ec0 0x7fef20006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b1af0 con 0x7fef20005ec0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2000cba0 con 0x7fef200050a0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20003570 con 0x7fef34159270 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef20005ec0 0x7fef20006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef33fff700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef20004b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef337fe700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef200050a0 0x7fef20005980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 -- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 msgr2=0x7fef20004b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34159270 0x7fef20004b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 -- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef200050a0 msgr2=0x7fef20005980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef200050a0 0x7fef20005980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 -- 172.21.15.179:0/2282450426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200046e0 con 0x7fef20005ec0 2024-05-08T22:34:17.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef20005ec0 0x7fef20006340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fef340ae440 tx=0x7fef200069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9efcbdef4a6d760 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c001390 con 0x7fef20005ec0 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00c660 con 0x7fef20005ec0 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef32ffd700 1 -- 172.21.15.179:0/2282450426 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00ca10 con 0x7fef20005ec0 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- 172.21.15.179:0/2282450426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef20005ec0 msgr2=0x7fef20006340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 --2- 172.21.15.179:0/2282450426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef20005ec0 0x7fef20006340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fef340ae440 tx=0x7fef200069b0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef38cf1700 1 -- 172.21.15.179:0/2282450426 reap_dead start 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- 172.21.15.179:0/2282450426 shutdown_connections 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- 172.21.15.179:0/2282450426 >> 172.21.15.179:0/2282450426 conn(0x7fef340ce960 msgr2=0x7fef340ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- 172.21.15.179:0/2282450426 shutdown_connections 2024-05-08T22:34:17.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.606+0000 7fef394f2700 1 -- 172.21.15.179:0/2282450426 wait complete. 2024-05-08T22:34:17.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 Processor -- start 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- start start 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34148260 0x7fef340aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34148640 0x7fef341561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340a8050 con 0x7fef341ee410 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34154370 con 0x7fef34148260 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef34154650 con 0x7fef34148640 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35046/0 (socket says 172.21.15.179:35046) 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 -- 172.21.15.179:0/4194994064 learned_addr learned my addr 172.21.15.179:0/4194994064 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef33fff700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34148260 0x7fef340aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef337fe700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34148640 0x7fef341561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 -- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34148640 msgr2=0x7fef341561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34148640 0x7fef341561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 -- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34148260 msgr2=0x7fef340aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34148260 0x7fef340aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 -- 172.21.15.179:0/4194994064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34154930 con 0x7fef341ee410 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef38cf1700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef2c013000 tx=0x7fef2c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c001330 con 0x7fef341ee410 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c005070 con 0x7fef341ee410 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c005420 con 0x7fef341ee410 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef34154040 con 0x7fef341ee410 2024-05-08T22:34:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.610+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340a7ce0 con 0x7fef341ee410 2024-05-08T22:34:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34154b20 con 0x7fef341ee410 2024-05-08T22:34:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef2c022020 con 0x7fef341ee410 2024-05-08T22:34:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef30ff9700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 0x7fef14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fef2c084ec0 con 0x7fef341ee410 2024-05-08T22:34:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef33fff700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 0x7fef14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.614+0000 7fef33fff700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 0x7fef14068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009c70 tx=0x7fef1c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:17.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.618+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fef2c0520e0 con 0x7fef341ee410 2024-05-08T22:34:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.798+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef3404e700 con 0x7fef341ee410 2024-05-08T22:34:17.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.798+0000 7fef30ff9700 1 -- 172.21.15.179:0/4194994064 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef2c02a020 con 0x7fef341ee410 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 msgr2=0x7fef14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 0x7fef14068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009c70 tx=0x7fef1c009240 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 msgr2=0x7fef34156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef2c013000 tx=0x7fef2c001230 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 shutdown_connections 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef14066730 0x7fef14068bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef341ee410 0x7fef34156710 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef34148640 0x7fef341561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 --2- 172.21.15.179:0/4194994064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34148260 0x7fef340aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 >> 172.21.15.179:0/4194994064 conn(0x7fef340ce960 msgr2=0x7fef340adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 shutdown_connections 2024-05-08T22:34:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:17.802+0000 7fef394f2700 1 -- 172.21.15.179:0/4194994064 wait complete. 2024-05-08T22:34:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:17.816 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi195.front.sepia.ceph.com... 2024-05-08T22:34:17.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:17.817 DEBUG:teuthology.orchestra.run.smithi195:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-08T22:34:17.851 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:17.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:34:17.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:17.851 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:34:17.879 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:17.879 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:17.887 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:17.887 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:17.895 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:18.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 Processor -- start 2024-05-08T22:34:18.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 -- start start 2024-05-08T22:34:18.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa0c06ed50 0x7ffa0c0b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c0cd200 con 0x7ffa0c06dc20 2024-05-08T22:34:18.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c0cd4e0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa11f0e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c0cd7c0 con 0x7ffa0c06ed50 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa10f0c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa10f0c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35058/0 (socket says 172.21.15.179:35058) 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.002+0000 7ffa10f0c700 1 -- 172.21.15.179:0/2580904258 learned_addr learned my addr 172.21.15.179:0/2580904258 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa1170d700 1 --2- 172.21.15.179:0/2580904258 >> v2:172.21.15.195:3300/0 conn(0x7ffa0c06ed50 0x7ffa0c0b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa10f0c700 1 -- 172.21.15.179:0/2580904258 >> v2:172.21.15.195:3300/0 conn(0x7ffa0c06ed50 msgr2=0x7ffa0c0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa10f0c700 1 --2- 172.21.15.179:0/2580904258 >> v2:172.21.15.195:3300/0 conn(0x7ffa0c06ed50 0x7ffa0c0b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa10f0c700 1 -- 172.21.15.179:0/2580904258 >> v1:172.21.15.195:6789/0 conn(0x7ffa0c06dc20 legacy=0x7ffa0c06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa10f0c700 1 -- 172.21.15.179:0/2580904258 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c0cdaa0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa10f0c700 1 --2- 172.21.15.179:0/2580904258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c06aec0 tx=0x7ffa0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea44992ad264e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa0b7fe700 1 -- 172.21.15.179:0/2580904258 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400e070 con 0x7ffa0c1434d0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa0b7fe700 1 -- 172.21.15.179:0/2580904258 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0400b9c0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa0b7fe700 1 -- 172.21.15.179:0/2580904258 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400bd70 con 0x7ffa0c1434d0 2024-05-08T22:34:18.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- 172.21.15.179:0/2580904258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 msgr2=0x7ffa0c06d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/2580904258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c06aec0 tx=0x7ffa0400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- 172.21.15.179:0/2580904258 shutdown_connections 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/2580904258 >> v2:172.21.15.195:3300/0 conn(0x7ffa0c06ed50 0x7ffa0c0b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/2580904258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c06d6e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- 172.21.15.179:0/2580904258 >> 172.21.15.179:0/2580904258 conn(0x7ffa0c0ce9d0 msgr2=0x7ffa0c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- 172.21.15.179:0/2580904258 shutdown_connections 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- 172.21.15.179:0/2580904258 wait complete. 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 Processor -- start 2024-05-08T22:34:18.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.006+0000 7ffa11f0e700 1 -- start start 2024-05-08T22:34:18.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c06ed50 0x7ffa0c13c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 0x7ffa0c0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0b28e0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1e54f0 con 0x7ffa0c0b2430 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0c4e60 con 0x7ffa0c06ed50 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35060/0 (socket says 172.21.15.179:35060) 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 -- 172.21.15.179:0/330298444 learned_addr learned my addr 172.21.15.179:0/330298444 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa1170d700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 0x7ffa0c0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa10f0c700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c06ed50 0x7ffa0c13c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 -- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c06ed50 msgr2=0x7ffa0c13c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c06ed50 0x7ffa0c13c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 -- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 msgr2=0x7ffa0c0c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 0x7ffa0c0c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 -- 172.21.15.179:0/330298444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009b30 con 0x7ffa0c1434d0 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa1170d700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 0x7ffa0c0c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:18.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa0bfff700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c500 tx=0x7ffa0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000052d0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa00005430 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000055a0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c0c5140 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c1e5740 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.010+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c067740 con 0x7ffa0c1434d0 2024-05-08T22:34:18.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.014+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa000038e0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.014+0000 7ffa097fa700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 0x7ff9f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.014+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffa00083a00 con 0x7ffa0c1434d0 2024-05-08T22:34:18.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.014+0000 7ffa10f0c700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 0x7ff9f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.014+0000 7ffa10f0c700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 0x7ff9f8068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000f80 tx=0x7ffa04009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.018+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa000509a0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.194+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa0c04e700 con 0x7ffa0c1434d0 2024-05-08T22:34:18.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa097fa700 1 -- 172.21.15.179:0/330298444 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7ffa00053fc0 con 0x7ffa0c1434d0 2024-05-08T22:34:18.198 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:18.198 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:18.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 msgr2=0x7ff9f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 0x7ff9f8068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000f80 tx=0x7ffa04009000 comp rx=0 tx=0).stop 2024-05-08T22:34:18.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 msgr2=0x7ffa0c0b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c500 tx=0x7ffa0000c8c0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 shutdown_connections 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9f8066730 0x7ff9f8068bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c0b2430 0x7ffa0c0c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c1434d0 0x7ffa0c0b1ef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 --2- 172.21.15.179:0/330298444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c06ed50 0x7ffa0c13c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 >> 172.21.15.179:0/330298444 conn(0x7ffa0c0ce9d0 msgr2=0x7ffa0c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 shutdown_connections 2024-05-08T22:34:18.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.198+0000 7ffa11f0e700 1 -- 172.21.15.179:0/330298444 wait complete. 2024-05-08T22:34:18.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:18.214 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:18.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 Processor -- start 2024-05-08T22:34:18.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 -- start start 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 0x7f5dcc0eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5dcc047cc0 0x7f5dcc052f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc04fe20 con 0x7f5dcc048210 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc04fb80 con 0x7f5dcc047cc0 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd88ff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0502b0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5dcc047cc0 0x7f5dcc052f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd2ffd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f5dcc048210 0x7f5dcc0eb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:52744/0 (socket says 172.21.15.179:52744) 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd2ffd700 1 -- 172.21.15.179:0/505413002 learned_addr learned my addr 172.21.15.179:0/505413002 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:18.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd37fe700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 0x7f5dcc0eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 -- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 msgr2=0x7f5dcc0eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:18.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 0x7f5dcc0eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 -- 172.21.15.179:0/505413002 >> v1:172.21.15.195:6789/0 conn(0x7f5dcc048210 legacy=0x7f5dcc0eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 -- 172.21.15.179:0/505413002 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0ebe70 con 0x7f5dcc047cc0 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd37fe700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 0x7f5dcc0eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd3fff700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc047cc0 0x7f5dcc052f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dd404f940 tx=0x7f5dd404f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361039076de00d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd27fc700 1 -- 172.21.15.179:0/505413002 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4067250 con 0x7f5dcc047cc0 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.330+0000 7f5dd27fc700 1 -- 172.21.15.179:0/505413002 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd40673b0 con 0x7f5dcc047cc0 2024-05-08T22:34:18.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd27fc700 1 -- 172.21.15.179:0/505413002 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4067960 con 0x7f5dcc047cc0 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc047cc0 msgr2=0x7f5dcc052f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc047cc0 0x7f5dcc052f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dd404f940 tx=0x7f5dd404f710 comp rx=0 tx=0).stop 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- 172.21.15.179:0/505413002 shutdown_connections 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc047cc0 0x7f5dcc052f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/505413002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc033fb0 0x7f5dcc0eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- 172.21.15.179:0/505413002 >> 172.21.15.179:0/505413002 conn(0x7f5dcc054b70 msgr2=0x7f5dcc054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- 172.21.15.179:0/505413002 shutdown_connections 2024-05-08T22:34:18.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- 172.21.15.179:0/505413002 wait complete. 2024-05-08T22:34:18.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 Processor -- start 2024-05-08T22:34:18.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.334+0000 7f5dd88ff700 1 -- start start 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc047cc0 0x7f5dcc039190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc048210 0x7f5dcc0de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc04d940 con 0x7f5dcc033fb0 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc048840 con 0x7f5dcc048210 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc048b20 con 0x7f5dcc047cc0 2024-05-08T22:34:18.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35068/0 (socket says 172.21.15.179:35068) 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 -- 172.21.15.179:0/3520358058 learned_addr learned my addr 172.21.15.179:0/3520358058 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd2ffd700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc047cc0 0x7f5dcc039190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd3fff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc048210 0x7f5dcc0de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 -- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc047cc0 msgr2=0x7f5dcc039190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc047cc0 0x7f5dcc039190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 -- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc048210 msgr2=0x7f5dcc0de0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc048210 0x7f5dcc0de0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 -- 172.21.15.179:0/3520358058 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00094b0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd37fe700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000ebe0 tx=0x7f5dc000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000c850 con 0x7f5dcc033fb0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc000c9b0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc048e00 con 0x7f5dcc033fb0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000cd40 con 0x7f5dcc033fb0 2024-05-08T22:34:18.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc0e7b70 con 0x7f5dcc033fb0 2024-05-08T22:34:18.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc0486a0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.338+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5dc0013470 con 0x7f5dcc033fb0 2024-05-08T22:34:18.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.342+0000 7f5dbffff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 0x7f5db8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.342+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5dc0084600 con 0x7f5dcc033fb0 2024-05-08T22:34:18.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.342+0000 7f5dd2ffd700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 0x7f5db8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.342+0000 7f5dd2ffd700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 0x7f5db8068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc03a380 tx=0x7f5dc8007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.350+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5dcc0486a0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.514+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5dcc0486a0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dbffff700 1 -- 172.21.15.179:0/3520358058 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dcc0486a0 con 0x7f5dcc033fb0 2024-05-08T22:34:18.517 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 msgr2=0x7f5db8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 0x7f5db8068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc03a380 tx=0x7f5dc8007f30 comp rx=0 tx=0).stop 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 msgr2=0x7f5dcc04d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000ebe0 tx=0x7f5dc000eef0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 shutdown_connections 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5db8066730 0x7f5db8068bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dcc048210 0x7f5dcc0de0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dcc047cc0 0x7f5dcc039190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 --2- 172.21.15.179:0/3520358058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dcc033fb0 0x7f5dcc04d360 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 >> 172.21.15.179:0/3520358058 conn(0x7f5dcc054b70 msgr2=0x7f5dcc0353a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 shutdown_connections 2024-05-08T22:34:18.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.518+0000 7f5dd88ff700 1 -- 172.21.15.179:0/3520358058 wait complete. 2024-05-08T22:34:18.533 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:18.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.650+0000 7f78849e0700 1 Processor -- start 2024-05-08T22:34:18.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.650+0000 7f78849e0700 1 -- start start 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 0x7f78800b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 0x7f78800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800d1910 con 0x7f788015d160 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78800ce4f0 con 0x7f78800cdb20 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800b6f90 con 0x7f78800d29b0 2024-05-08T22:34:18.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 0x7f78800b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787f7fe700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f788015d160 0x7f78800d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:52756/0 (socket says 172.21.15.179:52756) 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787f7fe700 1 -- 172.21.15.179:0/3987498104 learned_addr learned my addr 172.21.15.179:0/3987498104 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787ffff700 1 --2- 172.21.15.179:0/3987498104 >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 0x7f78800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787e7fc700 1 -- 172.21.15.179:0/3987498104 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290338510 0 0) 0x7f78800d1910 con 0x7f788015d160 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787e7fc700 1 -- 172.21.15.179:0/3987498104 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c0035d0 con 0x7f788015d160 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 -- 172.21.15.179:0/3987498104 >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 msgr2=0x7f78800cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 --2- 172.21.15.179:0/3987498104 >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 0x7f78800cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 -- 172.21.15.179:0/3987498104 >> v1:172.21.15.195:6789/0 conn(0x7f788015d160 legacy=0x7f78800d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 -- 172.21.15.179:0/3987498104 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800d1bf0 con 0x7f78800d29b0 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787ffff700 1 --2- 172.21.15.179:0/3987498104 >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 0x7f78800cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:18.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787effd700 1 --2- 172.21.15.179:0/3987498104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 0x7f78800b8a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7870009850 tx=0x7f7870009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21364e8363e0603b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787e7fc700 1 -- 172.21.15.179:0/3987498104 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870011830 con 0x7f78800d29b0 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787e7fc700 1 -- 172.21.15.179:0/3987498104 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870011990 con 0x7f78800d29b0 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f787e7fc700 1 -- 172.21.15.179:0/3987498104 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870011d40 con 0x7f78800d29b0 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- 172.21.15.179:0/3987498104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 msgr2=0x7f78800b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 --2- 172.21.15.179:0/3987498104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 0x7f78800b8a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7870009850 tx=0x7f7870009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- 172.21.15.179:0/3987498104 shutdown_connections 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 --2- 172.21.15.179:0/3987498104 >> v2:172.21.15.195:3300/0 conn(0x7f78800cdb20 0x7f78800cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 --2- 172.21.15.179:0/3987498104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78800d29b0 0x7f78800b8a00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- 172.21.15.179:0/3987498104 >> 172.21.15.179:0/3987498104 conn(0x7f78800d3270 msgr2=0x7f78800d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- 172.21.15.179:0/3987498104 shutdown_connections 2024-05-08T22:34:18.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.654+0000 7f78849e0700 1 -- 172.21.15.179:0/3987498104 wait complete. 2024-05-08T22:34:18.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 Processor -- start 2024-05-08T22:34:18.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 -- start start 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78800cdb20 0x7f788014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f788015d160 0x7f78801f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800cba40 con 0x7f78800d29b0 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800cae30 con 0x7f78800cdb20 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f78849e0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800cb110 con 0x7f788015d160 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f787effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f787effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35076/0 (socket says 172.21.15.179:35076) 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f787effd700 1 -- 172.21.15.179:0/672904487 learned_addr learned my addr 172.21.15.179:0/672904487 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f787f7fe700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78800cdb20 0x7f788014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.658+0000 7f787ffff700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f788015d160 0x7f78801f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 -- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f788015d160 msgr2=0x7f78801f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f788015d160 0x7f78801f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 -- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78800cdb20 msgr2=0x7f788014cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78800cdb20 0x7f788014cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 -- 172.21.15.179:0/672904487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878009a70 con 0x7f78800d29b0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787effd700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78700090b0 tx=0x7f7870004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870004480 con 0x7f78800d29b0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787001f070 con 0x7f78800d29b0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78700094b0 con 0x7f78800d29b0 2024-05-08T22:34:18.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870003900 con 0x7f78800d29b0 2024-05-08T22:34:18.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78800cb6d0 con 0x7f78800d29b0 2024-05-08T22:34:18.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7870021030 con 0x7f78800d29b0 2024-05-08T22:34:18.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78800cb860 con 0x7f78800d29b0 2024-05-08T22:34:18.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 0x7f7860068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7870085840 con 0x7f78800d29b0 2024-05-08T22:34:18.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.662+0000 7f787f7fe700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 0x7f7860068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.666+0000 7f787f7fe700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 0x7f7860068bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78680011e0 tx=0x7f7868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.666+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7870052870 con 0x7f78800d29b0 2024-05-08T22:34:18.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7880072c50 con 0x7f78800d29b0 2024-05-08T22:34:18.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f7877fff700 1 -- 172.21.15.179:0/672904487 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7870006ce0 con 0x7f78800d29b0 2024-05-08T22:34:18.838 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:18.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 msgr2=0x7f7860068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 0x7f7860068bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78680011e0 tx=0x7f7868009040 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 msgr2=0x7f78800cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78700090b0 tx=0x7f7870004770 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 shutdown_connections 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7860066730 0x7f7860068bf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f788015d160 0x7f78801f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78800d29b0 0x7f78800cbed0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 --2- 172.21.15.179:0/672904487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78800cdb20 0x7f788014cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.838+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 >> 172.21.15.179:0/672904487 conn(0x7f78800d3270 msgr2=0x7f78800d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.842+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 shutdown_connections 2024-05-08T22:34:18.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.842+0000 7f78849e0700 1 -- 172.21.15.179:0/672904487 wait complete. 2024-05-08T22:34:18.854 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:18.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 Processor -- start 2024-05-08T22:34:18.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 -- start start 2024-05-08T22:34:18.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98a415d000 0x7f98a40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40b47f0 0x7f98a40cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40cd070 con 0x7f98a40ca880 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40b3b00 con 0x7f98a40b47f0 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a9e90700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40b3d10 con 0x7f98a415d000 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40b47f0 0x7f98a40cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a40b47f0 0x7f98a40cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35084/0 (socket says 172.21.15.179:35084) 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 -- 172.21.15.179:0/3419610520 learned_addr learned my addr 172.21.15.179:0/3419610520 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a8e8e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98a415d000 0x7f98a40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 -- 172.21.15.179:0/3419610520 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d000 msgr2=0x7f98a40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 --2- 172.21.15.179:0/3419610520 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d000 0x7f98a40cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 -- 172.21.15.179:0/3419610520 >> v1:172.21.15.195:6789/0 conn(0x7f98a40ca880 legacy=0x7f98a4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.974+0000 7f98a3fff700 1 -- 172.21.15.179:0/3419610520 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40b3ff0 con 0x7f98a40b47f0 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a3fff700 1 --2- 172.21.15.179:0/3419610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40b47f0 0x7f98a40cfdf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f98a40d0c30 tx=0x7f9894009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e641dffc4981d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a37fe700 1 -- 172.21.15.179:0/3419610520 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011830 con 0x7f98a40b47f0 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a37fe700 1 -- 172.21.15.179:0/3419610520 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894011990 con 0x7f98a40b47f0 2024-05-08T22:34:18.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a37fe700 1 -- 172.21.15.179:0/3419610520 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011d20 con 0x7f98a40b47f0 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- 172.21.15.179:0/3419610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40b47f0 msgr2=0x7f98a40cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- 172.21.15.179:0/3419610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40b47f0 0x7f98a40cfdf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f98a40d0c30 tx=0x7f9894009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- 172.21.15.179:0/3419610520 shutdown_connections 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- 172.21.15.179:0/3419610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40b47f0 0x7f98a40cfdf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- 172.21.15.179:0/3419610520 >> v2:172.21.15.195:3300/0 conn(0x7f98a415d000 0x7f98a40cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- 172.21.15.179:0/3419610520 >> 172.21.15.179:0/3419610520 conn(0x7f98a40b27c0 msgr2=0x7f98a40b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- 172.21.15.179:0/3419610520 shutdown_connections 2024-05-08T22:34:18.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- 172.21.15.179:0/3419610520 wait complete. 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 Processor -- start 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- start start 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a40b47f0 0x7f98a41f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 0x7f98a41f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a406d260 con 0x7f98a40ca880 2024-05-08T22:34:18.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40ae150 con 0x7f98a40b47f0 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a9e90700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a406d490 con 0x7f98a415d000 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35090/0 (socket says 172.21.15.179:35090) 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a3fff700 1 -- 172.21.15.179:0/933009863 learned_addr learned my addr 172.21.15.179:0/933009863 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a8e8e700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a40b47f0 0x7f98a41f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.978+0000 7f98a968f700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 0x7f98a41f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 -- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 msgr2=0x7f98a41f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 0x7f98a41f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 -- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a40b47f0 msgr2=0x7f98a41f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a40b47f0 0x7f98a41f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 -- 172.21.15.179:0/933009863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009570 con 0x7f98a40ca880 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a968f700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 0x7f98a41f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a3fff700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9894013000 tx=0x7f9894004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894004550 con 0x7f98a40ca880 2024-05-08T22:34:18.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989401f070 con 0x7f98a40ca880 2024-05-08T22:34:18.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40ad820 con 0x7f98a40ca880 2024-05-08T22:34:18.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894003900 con 0x7f98a40ca880 2024-05-08T22:34:18.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40adde0 con 0x7f98a40ca880 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a4067740 con 0x7f98a40ca880 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9894003a60 con 0x7f98a40ca880 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.982+0000 7f98a17fa700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 0x7f9890068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.986+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9894021070 con 0x7f98a40ca880 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.986+0000 7f98a8e8e700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 0x7f9890068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:18.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.986+0000 7f98a8e8e700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 0x7f9890068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9898009b50 tx=0x7f9898009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:18.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:18.990+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98940529f0 con 0x7f98a40ca880 2024-05-08T22:34:19.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.154+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98a404e700 con 0x7f98a40ca880 2024-05-08T22:34:19.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a17fa700 1 -- 172.21.15.179:0/933009863 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9894015020 con 0x7f98a40ca880 2024-05-08T22:34:19.158 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 msgr2=0x7f9890068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 0x7f9890068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9898009b50 tx=0x7f9898009240 comp rx=0 tx=0).stop 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 msgr2=0x7f98a406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9894013000 tx=0x7f9894004770 comp rx=0 tx=0).stop 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 shutdown_connections 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9890066730 0x7f9890068bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98a415d000 0x7f98a41f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a40ca880 0x7f98a406cd20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 --2- 172.21.15.179:0/933009863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98a40b47f0 0x7f98a41f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 >> 172.21.15.179:0/933009863 conn(0x7f98a40b27c0 msgr2=0x7f98a40c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 shutdown_connections 2024-05-08T22:34:19.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.158+0000 7f98a9e90700 1 -- 172.21.15.179:0/933009863 wait complete. 2024-05-08T22:34:19.173 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:19.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.290+0000 7f04f272b700 1 Processor -- start 2024-05-08T22:34:19.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.290+0000 7f04f272b700 1 -- start start 2024-05-08T22:34:19.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f04ec142890 0x7f04ec0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f04ec06d4b0 0x7f04ec15d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec0b22d0 con 0x7f04ec0b2ca0 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec0b19e0 con 0x7f04ec06d4b0 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec15e640 con 0x7f04ec142890 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f04ec06d4b0 0x7f04ec15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f0f28700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f04ec0b2ca0 0x7f04ec06cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46568/0 (socket says 172.21.15.179:46568) 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f0f28700 1 -- 172.21.15.179:0/2465163348 learned_addr learned my addr 172.21.15.179:0/2465163348 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1729700 1 --2- 172.21.15.179:0/2465163348 >> v2:172.21.15.195:3300/0 conn(0x7f04ec142890 0x7f04ec0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04dffff700 1 -- 172.21.15.179:0/2465163348 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654245087 0 0) 0x7f04ec0b22d0 con 0x7f04ec0b2ca0 2024-05-08T22:34:19.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04dffff700 1 -- 172.21.15.179:0/2465163348 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04d80035d0 con 0x7f04ec0b2ca0 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 -- 172.21.15.179:0/2465163348 >> v2:172.21.15.195:3300/0 conn(0x7f04ec142890 msgr2=0x7f04ec0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 --2- 172.21.15.179:0/2465163348 >> v2:172.21.15.195:3300/0 conn(0x7f04ec142890 0x7f04ec0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 -- 172.21.15.179:0/2465163348 >> v1:172.21.15.179:6789/0 conn(0x7f04ec0b2ca0 legacy=0x7f04ec06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 -- 172.21.15.179:0/2465163348 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec0b1c80 con 0x7f04ec06d4b0 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f1f2a700 1 --2- 172.21.15.179:0/2465163348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec06d4b0 0x7f04ec15d830 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0720b0 tx=0x7f04e400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43890c3ea7098a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04dffff700 1 -- 172.21.15.179:0/2465163348 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4010070 con 0x7f04ec06d4b0 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04dffff700 1 -- 172.21.15.179:0/2465163348 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e400b2e0 con 0x7f04ec06d4b0 2024-05-08T22:34:19.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04dffff700 1 -- 172.21.15.179:0/2465163348 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400dc10 con 0x7f04ec06d4b0 2024-05-08T22:34:19.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- 172.21.15.179:0/2465163348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec06d4b0 msgr2=0x7f04ec15d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 --2- 172.21.15.179:0/2465163348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec06d4b0 0x7f04ec15d830 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0720b0 tx=0x7f04e400d330 comp rx=0 tx=0).stop 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- 172.21.15.179:0/2465163348 shutdown_connections 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 --2- 172.21.15.179:0/2465163348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec06d4b0 0x7f04ec15d830 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 --2- 172.21.15.179:0/2465163348 >> v2:172.21.15.195:3300/0 conn(0x7f04ec142890 0x7f04ec0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- 172.21.15.179:0/2465163348 >> 172.21.15.179:0/2465163348 conn(0x7f04ec0cea30 msgr2=0x7f04ec0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.294+0000 7f04f272b700 1 -- 172.21.15.179:0/2465163348 shutdown_connections 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 -- 172.21.15.179:0/2465163348 wait complete. 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 Processor -- start 2024-05-08T22:34:19.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 -- start start 2024-05-08T22:34:19.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 0x7f04ec149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04ec0b2ca0 0x7f04ec13f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec1595a0 con 0x7f04ec142890 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec13feb0 con 0x7f04ec0b2ca0 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f272b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec1481e0 con 0x7f04ec06d4b0 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f1729700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 0x7f04ec149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f1f2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1729700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 0x7f04ec149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:33662/0 (socket says 172.21.15.179:33662) 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1729700 1 -- 172.21.15.179:0/3306458171 learned_addr learned my addr 172.21.15.179:0/3306458171 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f0f28700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04ec0b2ca0 0x7f04ec13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.298+0000 7f04f1f2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35114/0 (socket says 172.21.15.179:35114) 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 -- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 msgr2=0x7f04ec149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 0x7f04ec149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 -- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04ec0b2ca0 msgr2=0x7f04ec13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04ec0b2ca0 0x7f04ec13f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 -- 172.21.15.179:0/3306458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e400bc80 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f1f2a700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f04e4017000 tx=0x7f04e4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400b450 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4003e50 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec1484c0 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4004490 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec148a80 con 0x7f04ec142890 2024-05-08T22:34:19.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.302+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec04e700 con 0x7f04ec142890 2024-05-08T22:34:19.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.306+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04e40045f0 con 0x7f04ec142890 2024-05-08T22:34:19.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.306+0000 7f04ddffb700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 0x7f04d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.306+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f04e4087960 con 0x7f04ec142890 2024-05-08T22:34:19.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.306+0000 7f04f1729700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 0x7f04d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.306+0000 7f04f1729700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 0x7f04d0068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009c10 tx=0x7f04e8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.314+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f04e4058430 con 0x7f04ec142890 2024-05-08T22:34:19.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.490+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f04ec13b770 con 0x7f04ec142890 2024-05-08T22:34:19.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.490+0000 7f04ddffb700 1 -- 172.21.15.179:0/3306458171 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f04e4022020 con 0x7f04ec142890 2024-05-08T22:34:19.491 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 msgr2=0x7f04d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 0x7f04d0068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009c10 tx=0x7f04e8009240 comp rx=0 tx=0).stop 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 msgr2=0x7f04ec13f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f04e4017000 tx=0x7f04e4004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 shutdown_connections 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04d0066730 0x7f04d0068bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04ec0b2ca0 0x7f04ec13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04ec142890 0x7f04ec13f970 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 --2- 172.21.15.179:0/3306458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04ec06d4b0 0x7f04ec149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 >> 172.21.15.179:0/3306458171 conn(0x7f04ec0cea30 msgr2=0x7f04ec0ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 shutdown_connections 2024-05-08T22:34:19.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.494+0000 7f04f272b700 1 -- 172.21.15.179:0/3306458171 wait complete. 2024-05-08T22:34:19.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 Processor -- start 2024-05-08T22:34:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 -- start start 2024-05-08T22:34:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c15e430 0x7f4e2c0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0cf960 0x7f4e2c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4e2c0cec80 0x7f4e2c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c0aec50 con 0x7f4e2c0cf960 2024-05-08T22:34:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c0b1cb0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c15e810 con 0x7f4e2c15e430 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e29d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0cf960 0x7f4e2c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e29d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0cf960 0x7f4e2c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35122/0 (socket says 172.21.15.179:35122) 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e29d9b700 1 -- 172.21.15.179:0/3781228342 learned_addr learned my addr 172.21.15.179:0/3781228342 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2a59c700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c15e430 0x7f4e2c0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.622+0000 7f4e2ad9d700 1 --2- 172.21.15.179:0/3781228342 >> v2:172.21.15.195:3300/0 conn(0x7f4e2c0cec80 0x7f4e2c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 -- 172.21.15.179:0/3781228342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c15e430 msgr2=0x7f4e2c0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c15e430 0x7f4e2c0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 -- 172.21.15.179:0/3781228342 >> v2:172.21.15.195:3300/0 conn(0x7f4e2c0cec80 msgr2=0x7f4e2c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/3781228342 >> v2:172.21.15.195:3300/0 conn(0x7f4e2c0cec80 0x7f4e2c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 -- 172.21.15.179:0/3781228342 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0cfdc0 con 0x7f4e2c0cf960 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cf960 0x7f4e2c0ce740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0c9710 tx=0x7f4e20009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545a0dc2772613f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2959a700 1 -- 172.21.15.179:0/3781228342 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011830 con 0x7f4e2c0cf960 2024-05-08T22:34:19.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2959a700 1 -- 172.21.15.179:0/3781228342 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20011990 con 0x7f4e2c0cf960 2024-05-08T22:34:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2959a700 1 -- 172.21.15.179:0/3781228342 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011d20 con 0x7f4e2c0cf960 2024-05-08T22:34:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 -- 172.21.15.179:0/3781228342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cf960 msgr2=0x7f4e2c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cf960 0x7f4e2c0ce740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0c9710 tx=0x7f4e20009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 -- 172.21.15.179:0/3781228342 shutdown_connections 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/3781228342 >> v2:172.21.15.195:3300/0 conn(0x7f4e2c0cec80 0x7f4e2c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cf960 0x7f4e2c0ce740 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/3781228342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c15e430 0x7f4e2c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 -- 172.21.15.179:0/3781228342 >> 172.21.15.179:0/3781228342 conn(0x7f4e2c098a70 msgr2=0x7f4e2c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 -- 172.21.15.179:0/3781228342 shutdown_connections 2024-05-08T22:34:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.626+0000 7f4e2b59e700 1 -- 172.21.15.179:0/3781228342 wait complete. 2024-05-08T22:34:19.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 Processor -- start 2024-05-08T22:34:19.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- start start 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c0cf960 0x7f4e2c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2c15e430 0x7f4e2c0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0ae1d0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0aca20 con 0x7f4e2c15e430 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c06e510 con 0x7f4e2c0cf960 2024-05-08T22:34:19.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35138/0 (socket says 172.21.15.179:35138) 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 -- 172.21.15.179:0/1299353701 learned_addr learned my addr 172.21.15.179:0/1299353701 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2ad9d700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2c15e430 0x7f4e2c0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c0cf960 0x7f4e2c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 -- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c0cf960 msgr2=0x7f4e2c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c0cf960 0x7f4e2c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 -- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2c15e430 msgr2=0x7f4e2c0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2c15e430 0x7f4e2c0c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 -- 172.21.15.179:0/1299353701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20009570 con 0x7f4e2c0cec80 2024-05-08T22:34:19.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2a59c700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e840 tx=0x7f4e1c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0095a0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c0ae3e0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c013680 con 0x7f4e2c0cec80 2024-05-08T22:34:19.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0139f0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.630+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c0ae6c0 con 0x7f4e2c0cec80 2024-05-08T22:34:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2c04e700 con 0x7f4e2c0cec80 2024-05-08T22:34:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e1c013b50 con 0x7f4e2c0cec80 2024-05-08T22:34:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e1affd700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 0x7f4e0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4e1c084480 con 0x7f4e2c0cec80 2024-05-08T22:34:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 0x7f4e0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.634+0000 7f4e29d9b700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 0x7f4e0c068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e20008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.642+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4e1c051310 con 0x7f4e2c0cec80 2024-05-08T22:34:19.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.830+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e2c0cf460 con 0x7f4e2c0cec80 2024-05-08T22:34:19.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.830+0000 7f4e1affd700 1 -- 172.21.15.179:0/1299353701 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e1c054930 con 0x7f4e2c0cec80 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.830+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 msgr2=0x7f4e0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 0x7f4e0c068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e20008030 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 msgr2=0x7f4e2c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e840 tx=0x7f4e1c00eb50 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 shutdown_connections 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4e0c066730 0x7f4e0c068bf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e2c15e430 0x7f4e2c0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e2c0cf960 0x7f4e2c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 --2- 172.21.15.179:0/1299353701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e2c0cec80 0x7f4e2c06f020 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 >> 172.21.15.179:0/1299353701 conn(0x7f4e2c098a70 msgr2=0x7f4e2c15afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 shutdown_connections 2024-05-08T22:34:19.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.834+0000 7f4e2b59e700 1 -- 172.21.15.179:0/1299353701 wait complete. 2024-05-08T22:34:19.835 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:19.846 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:19.847 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:19.855 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 Processor -- start 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 -- start start 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c0e11d0 0x7fdd1c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdd1c038820 0x7fdd1c039f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdd1c051010 0x7fdd1c03a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0078e0 con 0x7fdd1c051010 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c006f10 con 0x7fdd1c038820 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd2a300700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c03a9f0 con 0x7fdd1c0e11d0 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd29aff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdd1c051010 0x7fdd1c03a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd29aff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdd1c051010 0x7fdd1c03a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35140/0 (socket says 172.21.15.179:35140) 2024-05-08T22:34:19.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd29aff700 1 -- 172.21.15.179:0/3189439004 learned_addr learned my addr 172.21.15.179:0/3189439004 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:19.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd292fe700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c0e11d0 0x7fdd1c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.970+0000 7fdd28afd700 1 --2- 172.21.15.179:0/3189439004 >> v2:172.21.15.195:3300/0 conn(0x7fdd1c038820 0x7fdd1c039f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 -- 172.21.15.179:0/3189439004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c0e11d0 msgr2=0x7fdd1c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c0e11d0 0x7fdd1c0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 -- 172.21.15.179:0/3189439004 >> v2:172.21.15.195:3300/0 conn(0x7fdd1c038820 msgr2=0x7fdd1c039f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 --2- 172.21.15.179:0/3189439004 >> v2:172.21.15.195:3300/0 conn(0x7fdd1c038820 0x7fdd1c039f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 -- 172.21.15.179:0/3189439004 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0e15b0 con 0x7fdd1c051010 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd29aff700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c051010 0x7fdd1c03a4b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd24068290 tx=0x7fdd2404f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f84a6945dae0db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd1bfff700 1 -- 172.21.15.179:0/3189439004 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24067250 con 0x7fdd1c051010 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd1bfff700 1 -- 172.21.15.179:0/3189439004 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd240673b0 con 0x7fdd1c051010 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd1bfff700 1 -- 172.21.15.179:0/3189439004 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24067960 con 0x7fdd1c051010 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- 172.21.15.179:0/3189439004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c051010 msgr2=0x7fdd1c03a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c051010 0x7fdd1c03a4b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd24068290 tx=0x7fdd2404f740 comp rx=0 tx=0).stop 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- 172.21.15.179:0/3189439004 shutdown_connections 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c051010 0x7fdd1c03a4b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 --2- 172.21.15.179:0/3189439004 >> v2:172.21.15.195:3300/0 conn(0x7fdd1c038820 0x7fdd1c039f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 --2- 172.21.15.179:0/3189439004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c0e11d0 0x7fdd1c0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- 172.21.15.179:0/3189439004 >> 172.21.15.179:0/3189439004 conn(0x7fdd1c054b70 msgr2=0x7fdd1c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- 172.21.15.179:0/3189439004 shutdown_connections 2024-05-08T22:34:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- 172.21.15.179:0/3189439004 wait complete. 2024-05-08T22:34:19.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 Processor -- start 2024-05-08T22:34:19.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.974+0000 7fdd2a300700 1 -- start start 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c051010 0x7fdd1c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 0x7fdd1c0e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0357e0 con 0x7fdd1c038820 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0e37a0 con 0x7fdd1c0e11d0 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c038380 con 0x7fdd1c051010 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35152/0 (socket says 172.21.15.179:35152) 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 -- 172.21.15.179:0/1563428583 learned_addr learned my addr 172.21.15.179:0/1563428583 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd29aff700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 0x7fdd1c0e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd28afd700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c051010 0x7fdd1c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 -- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c051010 msgr2=0x7fdd1c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c051010 0x7fdd1c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 -- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 msgr2=0x7fdd1c0e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 0x7fdd1c0e3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 -- 172.21.15.179:0/1563428583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c037dc0 con 0x7fdd1c038820 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd29aff700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 0x7fdd1c0e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd292fe700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e870 tx=0x7fdd2000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200095a0 con 0x7fdd1c038820 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20013680 con 0x7fdd1c038820 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0380a0 con 0x7fdd1c038820 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200139f0 con 0x7fdd1c038820 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.978+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c0ce0d0 con 0x7fdd1c038820 2024-05-08T22:34:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c0ce2f0 con 0x7fdd1c038820 2024-05-08T22:34:19.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd20013b50 con 0x7fdd1c038820 2024-05-08T22:34:19.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd19ffb700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 0x7fdd08068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:19.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdd200843a0 con 0x7fdd1c038820 2024-05-08T22:34:19.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd28afd700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 0x7fdd08068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:19.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.982+0000 7fdd28afd700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 0x7fdd08068c20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c034ac0 tx=0x7fdd14009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:19.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:19.990+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdd1c0ce2f0 con 0x7fdd1c038820 2024-05-08T22:34:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.162+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd1c0ce2f0 con 0x7fdd1c038820 2024-05-08T22:34:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd19ffb700 1 -- 172.21.15.179:0/1563428583 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7fdd1c0ce2f0 con 0x7fdd1c038820 2024-05-08T22:34:20.167 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:20.167 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 msgr2=0x7fdd08068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 0x7fdd08068c20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c034ac0 tx=0x7fdd14009240 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 msgr2=0x7fdd1c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e870 tx=0x7fdd2000eb80 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 shutdown_connections 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd08066760 0x7fdd08068c20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd1c0e11d0 0x7fdd1c0e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd1c051010 0x7fdd1c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.166+0000 7fdd2a300700 1 --2- 172.21.15.179:0/1563428583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd1c038820 0x7fdd1c0cd550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.170+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 >> 172.21.15.179:0/1563428583 conn(0x7fdd1c054b70 msgr2=0x7fdd1c04a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.170+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 shutdown_connections 2024-05-08T22:34:20.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.170+0000 7fdd2a300700 1 -- 172.21.15.179:0/1563428583 wait complete. 2024-05-08T22:34:20.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:20.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:20.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.306+0000 7f9ea4c5d700 1 Processor -- start 2024-05-08T22:34:20.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.306+0000 7f9ea4c5d700 1 -- start start 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9ea006bb50 0x7f9ea00b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00c5a70 con 0x7f9ea00b18b0 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea006ce80 con 0x7f9ea0159550 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00b1e00 con 0x7f9ea006bb50 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9ea00b18b0 0x7f9ea00b45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46584/0 (socket says 172.21.15.179:46584) 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1823559341 learned_addr learned my addr 172.21.15.179:0/1823559341 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:20.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e96ffd700 1 --2- 172.21.15.179:0/1823559341 >> v2:172.21.15.195:3300/0 conn(0x7f9ea006bb50 0x7f9ea00b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9effd700 1 -- 172.21.15.179:0/1823559341 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301365886 0 0) 0x7f9ea00c5a70 con 0x7f9ea00b18b0 2024-05-08T22:34:20.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9effd700 1 -- 172.21.15.179:0/1823559341 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e8c0035d0 con 0x7f9ea00b18b0 2024-05-08T22:34:20.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e96ffd700 1 -- 172.21.15.179:0/1823559341 >> v1:172.21.15.195:6789/0 conn(0x7f9ea0159550 legacy=0x7f9ea006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e96ffd700 1 -- 172.21.15.179:0/1823559341 >> v1:172.21.15.179:6789/0 conn(0x7f9ea00b18b0 legacy=0x7f9ea00b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e96ffd700 1 -- 172.21.15.179:0/1823559341 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea006c710 con 0x7f9ea006bb50 2024-05-08T22:34:20.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e96ffd700 1 --2- 172.21.15.179:0/1823559341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea00b4070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800a9f0 tx=0x7f9e8800ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca5f82695f7ca71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9effd700 1 -- 172.21.15.179:0/1823559341 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800cb70 con 0x7f9ea006bb50 2024-05-08T22:34:20.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9effd700 1 -- 172.21.15.179:0/1823559341 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800ae90 con 0x7f9ea006bb50 2024-05-08T22:34:20.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9e9effd700 1 -- 172.21.15.179:0/1823559341 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88018400 con 0x7f9ea006bb50 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1823559341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 msgr2=0x7f9ea00b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1823559341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea00b4070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800a9f0 tx=0x7f9e8800ad00 comp rx=0 tx=0).stop 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1823559341 shutdown_connections 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1823559341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea00b4070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1823559341 >> 172.21.15.179:0/1823559341 conn(0x7f9ea0098a70 msgr2=0x7f9ea00cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1823559341 shutdown_connections 2024-05-08T22:34:20.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.310+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1823559341 wait complete. 2024-05-08T22:34:20.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 Processor -- start 2024-05-08T22:34:20.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 -- start start 2024-05-08T22:34:20.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00b18b0 0x7f9ea01587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0157330 con 0x7f9ea0159550 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00acf70 con 0x7f9ea006bb50 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9ea4c5d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0156ce0 con 0x7f9ea00b18b0 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35160/0 (socket says 172.21.15.179:35160) 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1808211820 learned_addr learned my addr 172.21.15.179:0/1808211820 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e96ffd700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00b18b0 0x7f9ea01587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9f7fe700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00b18b0 msgr2=0x7f9ea01587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00b18b0 0x7f9ea01587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 msgr2=0x7f9ea014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1808211820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8800a650 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.314+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cb40 tx=0x7f9e9800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98007500 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98007660 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea0156fc0 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980079f0 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea0158cf0 con 0x7f9ea0159550 2024-05-08T22:34:20.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0156b40 con 0x7f9ea0159550 2024-05-08T22:34:20.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e9801b3b0 con 0x7f9ea0159550 2024-05-08T22:34:20.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e98083f20 con 0x7f9ea0159550 2024-05-08T22:34:20.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.318+0000 7f9e9f7fe700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.322+0000 7f9e9f7fe700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e90000f60 tx=0x7f9e90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.330+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9ea0156b40 con 0x7f9ea0159550 2024-05-08T22:34:20.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.494+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ea0156b40 con 0x7f9ea0159550 2024-05-08T22:34:20.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9e9cff9700 1 -- 172.21.15.179:0/1808211820 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ea0156b40 con 0x7f9ea0159550 2024-05-08T22:34:20.498 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:20.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 msgr2=0x7f9e80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e90000f60 tx=0x7f9e90009040 comp rx=0 tx=0).stop 2024-05-08T22:34:20.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 msgr2=0x7f9ea00ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cb40 tx=0x7f9e9800cb70 comp rx=0 tx=0).stop 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 shutdown_connections 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea0159550 0x7f9ea00ca6d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00b18b0 0x7f9ea01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 --2- 172.21.15.179:0/1808211820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea006bb50 0x7f9ea014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 >> 172.21.15.179:0/1808211820 conn(0x7f9ea0098a70 msgr2=0x7f9ea00cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 shutdown_connections 2024-05-08T22:34:20.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.498+0000 7f9ea4c5d700 1 -- 172.21.15.179:0/1808211820 wait complete. 2024-05-08T22:34:20.513 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:20.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 Processor -- start 2024-05-08T22:34:20.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 -- start start 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f146015d130 0x7f14600d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600d2c90 con 0x7f14600ce2b0 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600d16b0 con 0x7f14600cd710 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466859700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14600d14c0 con 0x7f146015d130 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465056700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f14600ce2b0 0x7f14600d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46598/0 (socket says 172.21.15.179:46598) 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465056700 1 -- 172.21.15.179:0/2774783980 learned_addr learned my addr 172.21.15.179:0/2774783980 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 --2- 172.21.15.179:0/2774783980 >> v2:172.21.15.195:3300/0 conn(0x7f146015d130 0x7f14600d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627481833 0 0) 0x7f14600d2c90 con 0x7f14600ce2b0 2024-05-08T22:34:20.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c0035d0 con 0x7f14600ce2b0 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106911344 0 0) 0x7f144c0035d0 con 0x7f14600ce2b0 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 >> v2:172.21.15.195:3300/0 conn(0x7f146015d130 msgr2=0x7f14600d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 --2- 172.21.15.179:0/2774783980 >> v2:172.21.15.195:3300/0 conn(0x7f146015d130 0x7f14600d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 >> v1:172.21.15.195:6789/0 conn(0x7f14600cd710 legacy=0x7f14600b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600d1990 con 0x7f14600ce2b0 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f145c001670 con 0x7f14600ce2b0 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 >> v1:172.21.15.179:6789/0 conn(0x7f14600ce2b0 legacy=0x7f14600d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f144c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c0043d0 0x7f144c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f144c005090 0x7f144c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00bf70 con 0x7f144c003a20 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c0b0 con 0x7f144c0043d0 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c240 con 0x7f144c005090 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f144c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465056700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c0043d0 0x7f144c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466058700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f144c005090 0x7f144c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 -- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f144c005090 msgr2=0x7f144c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f144c005090 0x7f144c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 -- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c0043d0 msgr2=0x7f144c004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f144c0043d0 0x7f144c004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1465857700 1 -- 172.21.15.179:0/2774783980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0035d0 con 0x7f144c003a20 2024-05-08T22:34:20.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.634+0000 7f1466058700 1 -- 172.21.15.179:0/2774783980 reap_dead start 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1465857700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f144c003e00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1450009ba0 tx=0x7f144c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfd5cc111b59dea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000e690 con 0x7f144c003a20 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145000e7f0 con 0x7f144c003a20 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1464855700 1 -- 172.21.15.179:0/2774783980 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000eb80 con 0x7f144c003a20 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 msgr2=0x7f144c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f144c003e00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1450009ba0 tx=0x7f144c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- 172.21.15.179:0/2774783980 shutdown_connections 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 --2- 172.21.15.179:0/2774783980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f144c003e00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- 172.21.15.179:0/2774783980 >> 172.21.15.179:0/2774783980 conn(0x7f14600b49a0 msgr2=0x7f14600b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- 172.21.15.179:0/2774783980 shutdown_connections 2024-05-08T22:34:20.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- 172.21.15.179:0/2774783980 wait complete. 2024-05-08T22:34:20.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 Processor -- start 2024-05-08T22:34:20.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.638+0000 7f1466859700 1 -- start start 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14601f27e0 0x7f14600cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 0x7f14600b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601f37e0 con 0x7f144c003a20 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1460158c60 con 0x7f1460155db0 2024-05-08T22:34:20.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1460153cb0 con 0x7f14601f27e0 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35184/0 (socket says 172.21.15.179:35184) 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 -- 172.21.15.179:0/4214867596 learned_addr learned my addr 172.21.15.179:0/4214867596 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465056700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14601f27e0 0x7f14600cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466058700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 0x7f14600b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 -- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14601f27e0 msgr2=0x7f14600cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14601f27e0 0x7f14600cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 -- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 msgr2=0x7f14600b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 0x7f14600b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 -- 172.21.15.179:0/4214867596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14601f2c60 con 0x7f144c003a20 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466058700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 0x7f14600b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1465857700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14500090b0 tx=0x7f14500047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000e690 con 0x7f144c003a20 2024-05-08T22:34:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14500038e0 con 0x7f144c003a20 2024-05-08T22:34:20.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14601f2f40 con 0x7f144c003a20 2024-05-08T22:34:20.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450003c90 con 0x7f144c003a20 2024-05-08T22:34:20.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.642+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14601f3500 con 0x7f144c003a20 2024-05-08T22:34:20.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146004e700 con 0x7f144c003a20 2024-05-08T22:34:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1450013470 con 0x7f144c003a20 2024-05-08T22:34:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f14567fc700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 0x7f1444068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1450085af0 con 0x7f144c003a20 2024-05-08T22:34:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f1465056700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 0x7f1444068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.646+0000 7f1465056700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 0x7f1444068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14600c9790 tx=0x7f145c001430 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.654+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1450052bb0 con 0x7f144c003a20 2024-05-08T22:34:20.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.822+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14600b1ff0 con 0x7f144c003a20 2024-05-08T22:34:20.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.822+0000 7f14567fc700 1 -- 172.21.15.179:0/4214867596 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14500561d0 con 0x7f144c003a20 2024-05-08T22:34:20.824 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 msgr2=0x7f1444068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 0x7f1444068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14600c9790 tx=0x7f145c001430 comp rx=0 tx=0).stop 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 msgr2=0x7f1460153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14500090b0 tx=0x7f14500047e0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 shutdown_connections 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1444066730 0x7f1444068bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1460155db0 0x7f14600b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14601f27e0 0x7f14600cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 --2- 172.21.15.179:0/4214867596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f144c003a20 0x7f1460153770 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 >> 172.21.15.179:0/4214867596 conn(0x7f14600b49a0 msgr2=0x7f1460155540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 shutdown_connections 2024-05-08T22:34:20.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.826+0000 7f1466859700 1 -- 172.21.15.179:0/4214867596 wait complete. 2024-05-08T22:34:20.839 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:20.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.958+0000 7f99e2ec2700 1 Processor -- start 2024-05-08T22:34:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.958+0000 7f99e2ec2700 1 -- start start 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99dc159320 0x7f99dc0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc0b19e0 0x7f99dc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc06cda0 con 0x7f99dc159700 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc0b1f10 con 0x7f99dc159320 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0b21f0 con 0x7f99dc0b19e0 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99dc159320 0x7f99dc0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99dc159320 0x7f99dc0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35198/0 (socket says 172.21.15.179:35198) 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 -- 172.21.15.179:0/810338395 learned_addr learned my addr 172.21.15.179:0/810338395 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e16bf700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc0b19e0 0x7f99dc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 -- 172.21.15.179:0/810338395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc0b19e0 msgr2=0x7f99dc0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc0b19e0 0x7f99dc0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 -- 172.21.15.179:0/810338395 >> v1:172.21.15.195:6789/0 conn(0x7f99dc159700 legacy=0x7f99dc06d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 -- 172.21.15.179:0/810338395 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0b24d0 con 0x7f99dc159320 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e1ec0700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159320 0x7f99dc0cadb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009910 tx=0x7f99d8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4d6e6f57e5bce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e0ebe700 1 -- 172.21.15.179:0/810338395 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8011830 con 0x7f99dc159320 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e0ebe700 1 -- 172.21.15.179:0/810338395 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d8011990 con 0x7f99dc159320 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e0ebe700 1 -- 172.21.15.179:0/810338395 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8011d40 con 0x7f99dc159320 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- 172.21.15.179:0/810338395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159320 msgr2=0x7f99dc0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159320 0x7f99dc0cadb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009910 tx=0x7f99d8009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- 172.21.15.179:0/810338395 shutdown_connections 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc0b19e0 0x7f99dc0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/810338395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159320 0x7f99dc0cadb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- 172.21.15.179:0/810338395 >> 172.21.15.179:0/810338395 conn(0x7f99dc0cea30 msgr2=0x7f99dc0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:20.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- 172.21.15.179:0/810338395 shutdown_connections 2024-05-08T22:34:20.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.962+0000 7f99e2ec2700 1 -- 172.21.15.179:0/810338395 wait complete. 2024-05-08T22:34:20.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 Processor -- start 2024-05-08T22:34:20.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 -- start start 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99dc0b19e0 0x7f99dc1419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc159320 0x7f99dc1f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc159ba0 con 0x7f99dc159700 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc144d90 con 0x7f99dc0b19e0 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e2ec2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc146f10 con 0x7f99dc159320 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35210/0 (socket says 172.21.15.179:35210) 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e16bf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc159320 0x7f99dc1f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 -- 172.21.15.179:0/468391773 learned_addr learned my addr 172.21.15.179:0/468391773 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:20.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e1ec0700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99dc0b19e0 0x7f99dc1419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 -- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc159320 msgr2=0x7f99dc1f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc159320 0x7f99dc1f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 -- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99dc0b19e0 msgr2=0x7f99dc1419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99dc0b19e0 0x7f99dc1419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.966+0000 7f99e26c1700 1 -- 172.21.15.179:0/468391773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8009570 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99e26c1700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cbc0 tx=0x7f99d400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4003c20 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d4003d80 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc1471f0 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d400aae0 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc1477b0 con 0x7f99dc159700 2024-05-08T22:34:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc1479d0 con 0x7f99dc159700 2024-05-08T22:34:20.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.970+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99d400ac40 con 0x7f99dc159700 2024-05-08T22:34:20.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.974+0000 7f99d27fc700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:20.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.974+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f99d40843d0 con 0x7f99dc159700 2024-05-08T22:34:20.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.974+0000 7f99e1ec0700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:20.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.974+0000 7f99e1ec0700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000c00 tx=0x7f99d8020040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:20.978+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f99d4051410 con 0x7f99dc159700 2024-05-08T22:34:21.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.150+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99dc04e700 con 0x7f99dc159700 2024-05-08T22:34:21.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.150+0000 7f99d27fc700 1 -- 172.21.15.179:0/468391773 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99d4054a30 con 0x7f99dc159700 2024-05-08T22:34:21.153 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 msgr2=0x7f99c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000c00 tx=0x7f99d8020040 comp rx=0 tx=0).stop 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 msgr2=0x7f99dc1f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cbc0 tx=0x7f99d400ced0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 shutdown_connections 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99dc159700 0x7f99dc1f4b00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99dc159320 0x7f99dc1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 --2- 172.21.15.179:0/468391773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99dc0b19e0 0x7f99dc1419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 >> 172.21.15.179:0/468391773 conn(0x7f99dc0cea30 msgr2=0x7f99dc06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:21.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 shutdown_connections 2024-05-08T22:34:21.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.154+0000 7f99e2ec2700 1 -- 172.21.15.179:0/468391773 wait complete. 2024-05-08T22:34:21.168 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:21.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 Processor -- start 2024-05-08T22:34:21.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- start start 2024-05-08T22:34:21.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5f9c0b1700 0x7f5f9c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0cc720 con 0x7f5f9c06d960 2024-05-08T22:34:21.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c06c1c0 con 0x7f5f9c159860 2024-05-08T22:34:21.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cc980 con 0x7f5f9c0b1700 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5f9c0b1700 0x7f5f9c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5f9c0b1700 0x7f5f9c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35216/0 (socket says 172.21.15.179:35216) 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 -- 172.21.15.179:0/2898702835 learned_addr learned my addr 172.21.15.179:0/2898702835 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 -- 172.21.15.179:0/2898702835 >> v1:172.21.15.195:6789/0 conn(0x7f5f9c159860 legacy=0x7f5f9c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 -- 172.21.15.179:0/2898702835 >> v1:172.21.15.179:6789/0 conn(0x7f5f9c06d960 legacy=0x7f5f9c06f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 -- 172.21.15.179:0/2898702835 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0ccc60 con 0x7f5f9c0b1700 2024-05-08T22:34:21.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa0d57700 1 --2- 172.21.15.179:0/2898702835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c06b440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d150 tx=0x7f5f9400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19cc9b1d28bcaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5f9affd700 1 -- 172.21.15.179:0/2898702835 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94010070 con 0x7f5f9c0b1700 2024-05-08T22:34:21.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5f9affd700 1 -- 172.21.15.179:0/2898702835 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9400dbb0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5f9affd700 1 -- 172.21.15.179:0/2898702835 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94016400 con 0x7f5f9c0b1700 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2898702835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 msgr2=0x7f5f9c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2898702835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c06b440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d150 tx=0x7f5f9400d460 comp rx=0 tx=0).stop 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2898702835 shutdown_connections 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2898702835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c06b440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2898702835 >> 172.21.15.179:0/2898702835 conn(0x7f5f9c098a70 msgr2=0x7f5f9c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2898702835 shutdown_connections 2024-05-08T22:34:21.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.290+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2898702835 wait complete. 2024-05-08T22:34:21.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 Processor -- start 2024-05-08T22:34:21.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 -- start start 2024-05-08T22:34:21.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f9c06d960 0x7f5f9c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f9c159860 0x7f5f9c144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c099710 con 0x7f5f9c0b1700 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c154950 con 0x7f5f9c06d960 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa27ba700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c145010 con 0x7f5f9c159860 2024-05-08T22:34:21.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5f9b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5f9bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f9c06d960 0x7f5f9c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5f9b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35220/0 (socket says 172.21.15.179:35220) 2024-05-08T22:34:21.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5f9b7fe700 1 -- 172.21.15.179:0/2740559994 learned_addr learned my addr 172.21.15.179:0/2740559994 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.294+0000 7f5fa0d57700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f9c159860 0x7f5f9c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 -- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f9c159860 msgr2=0x7f5f9c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f9c159860 0x7f5f9c144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 -- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f9c06d960 msgr2=0x7f5f9c154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f9c06d960 0x7f5f9c154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 -- 172.21.15.179:0/2740559994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9400bd40 con 0x7f5f9c0b1700 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f9b7fe700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c450 tx=0x7f5f8c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0052d0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c005430 con 0x7f5f9c0b1700 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c1452f0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0055a0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c1458b0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.298+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c073290 con 0x7f5f9c0b1700 2024-05-08T22:34:21.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.302+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f8c003900 con 0x7f5f9c0b1700 2024-05-08T22:34:21.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.302+0000 7f5f98ff9700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 0x7f5f88068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.302+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f8c0838a0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.302+0000 7f5f9bfff700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 0x7f5f88068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.302+0000 7f5f9bfff700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 0x7f5f88068c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cb8c0 tx=0x7f5f90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.310+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5f8c0507b0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.478+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f9c0d00f0 con 0x7f5f9c0b1700 2024-05-08T22:34:21.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.478+0000 7f5f98ff9700 1 -- 172.21.15.179:0/2740559994 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f8c017020 con 0x7f5f9c0b1700 2024-05-08T22:34:21.481 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:21.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 msgr2=0x7f5f88068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 0x7f5f88068c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cb8c0 tx=0x7f5f90009040 comp rx=0 tx=0).stop 2024-05-08T22:34:21.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 msgr2=0x7f5f9c144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c450 tx=0x7f5f8c00c760 comp rx=0 tx=0).stop 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 shutdown_connections 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f88066760 0x7f5f88068c20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f9c159860 0x7f5f9c144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f9c0b1700 0x7f5f9c144430 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 --2- 172.21.15.179:0/2740559994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f9c06d960 0x7f5f9c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 >> 172.21.15.179:0/2740559994 conn(0x7f5f9c098a70 msgr2=0x7f5f9c0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 shutdown_connections 2024-05-08T22:34:21.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.482+0000 7f5fa27ba700 1 -- 172.21.15.179:0/2740559994 wait complete. 2024-05-08T22:34:21.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:21.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.610+0000 7ff5c1487700 1 Processor -- start 2024-05-08T22:34:21.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.610+0000 7ff5c1487700 1 -- start start 2024-05-08T22:34:21.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 0x7ff5bc0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0cd6c0 con 0x7ff5bc0ccb50 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b2f40 con 0x7ff5bc0b3360 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0cefd0 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35226/0 (socket says 172.21.15.179:35226) 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5bb7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5bc0ccb50 0x7ff5bc0ccfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46614/0 (socket says 172.21.15.179:46614) 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 -- 172.21.15.179:0/2362663480 learned_addr learned my addr 172.21.15.179:0/2362663480 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5bbfff700 1 --2- 172.21.15.179:0/2362663480 >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 0x7ff5bc0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 -- 172.21.15.179:0/2362663480 >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 msgr2=0x7ff5bc0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 --2- 172.21.15.179:0/2362663480 >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 0x7ff5bc0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 -- 172.21.15.179:0/2362663480 >> v1:172.21.15.179:6789/0 conn(0x7ff5bc0ccb50 legacy=0x7ff5bc0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 -- 172.21.15.179:0/2362663480 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0cded0 con 0x7ff5bc0b3360 2024-05-08T22:34:21.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5bbfff700 1 --2- 172.21.15.179:0/2362663480 >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 0x7ff5bc0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c0c86700 1 --2- 172.21.15.179:0/2362663480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0b2850 tx=0x7ff5b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fc56005af530fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5baffd700 1 -- 172.21.15.179:0/2362663480 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400e070 con 0x7ff5bc0b3360 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5baffd700 1 -- 172.21.15.179:0/2362663480 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400b9c0 con 0x7ff5bc0b3360 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5baffd700 1 -- 172.21.15.179:0/2362663480 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400bd70 con 0x7ff5bc0b3360 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- 172.21.15.179:0/2362663480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b3360 msgr2=0x7ff5bc0b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 --2- 172.21.15.179:0/2362663480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0b2850 tx=0x7ff5b400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- 172.21.15.179:0/2362663480 shutdown_connections 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 --2- 172.21.15.179:0/2362663480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc0b37f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 --2- 172.21.15.179:0/2362663480 >> v2:172.21.15.195:3300/0 conn(0x7ff5bc0ce6b0 0x7ff5bc0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- 172.21.15.179:0/2362663480 >> 172.21.15.179:0/2362663480 conn(0x7ff5bc0b1ed0 msgr2=0x7ff5bc0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- 172.21.15.179:0/2362663480 shutdown_connections 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.614+0000 7ff5c1487700 1 -- 172.21.15.179:0/2362663480 wait complete. 2024-05-08T22:34:21.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 Processor -- start 2024-05-08T22:34:21.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 -- start start 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0ccb50 0x7ff5bc158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15cc40 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15a560 con 0x7ff5bc0b3360 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c1487700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15bfc0 con 0x7ff5bc0ccb50 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5bb7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0ccb50 0x7ff5bc158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35234/0 (socket says 172.21.15.179:35234) 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 -- 172.21.15.179:0/3384262889 learned_addr learned my addr 172.21.15.179:0/3384262889 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 -- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0ccb50 msgr2=0x7ff5bc158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0ccb50 0x7ff5bc158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 -- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc0b3360 msgr2=0x7ff5bc141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.618+0000 7ff5c0c86700 1 -- 172.21.15.179:0/3384262889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b4009b30 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5c0c86700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b370 tx=0x7ff5b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4004880 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40099b0 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc15c2a0 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b401c680 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc15c860 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.622+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc15cd80 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.624 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:34:21.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:34:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.626+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5b401c7e0 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.626+0000 7ff5b8ff9700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 0x7ff5a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.626+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff5b40860d0 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.626+0000 7ff5bbfff700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 0x7ff5a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.626+0000 7ff5bbfff700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 0x7ff5a8068c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000ab60 tx=0x7ff5b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:21.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.630+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5b4053070 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.814+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff5bc15cd80 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.814+0000 7ff5b8ff9700 1 -- 172.21.15.179:0/3384262889 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff5bc15cd80 con 0x7ff5bc0ce6b0 2024-05-08T22:34:21.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 msgr2=0x7ff5a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 0x7ff5a8068c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000ab60 tx=0x7ff5b0008040 comp rx=0 tx=0).stop 2024-05-08T22:34:21.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 msgr2=0x7ff5bc15d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:21.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b370 tx=0x7ff5b4004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 shutdown_connections 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5a8066760 0x7ff5a8068c20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0ce6b0 0x7ff5bc15d3a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0ccb50 0x7ff5bc158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 --2- 172.21.15.179:0/3384262889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc0b3360 0x7ff5bc141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 >> 172.21.15.179:0/3384262889 conn(0x7ff5bc0b1ed0 msgr2=0x7ff5bc1607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 shutdown_connections 2024-05-08T22:34:21.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:21.818+0000 7ff5c1487700 1 -- 172.21.15.179:0/3384262889 wait complete. 2024-05-08T22:34:21.821 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:21.831 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:34:21.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:21.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-08T22:34:21.849 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:21.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:34:21.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:21.849 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:34:21.900 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:21.900 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:21.908 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:21.908 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:21.916 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 Processor -- start 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 -- start start 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdab00ce2d0 0x7fdab006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab006afc0 con 0x7fdab015d5b0 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab015d990 con 0x7fdab00cde70 2024-05-08T22:34:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab5cda700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab00d26f0 con 0x7fdab00ce2d0 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab4cd8700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdab015d5b0 0x7fdab00b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46630/0 (socket says 172.21.15.179:46630) 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab4cd8700 1 -- 172.21.15.179:0/875829185 learned_addr learned my addr 172.21.15.179:0/875829185 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab54d9700 1 --2- 172.21.15.179:0/875829185 >> v2:172.21.15.195:3300/0 conn(0x7fdab00ce2d0 0x7fdab006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdaaf7fe700 1 -- 172.21.15.179:0/875829185 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57230895 0 0) 0x7fdab006afc0 con 0x7fdab015d5b0 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdaaf7fe700 1 -- 172.21.15.179:0/875829185 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda900035d0 con 0x7fdab015d5b0 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab54d9700 1 -- 172.21.15.179:0/875829185 >> v1:172.21.15.195:6789/0 conn(0x7fdab00cde70 legacy=0x7fdab006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab54d9700 1 -- 172.21.15.179:0/875829185 >> v1:172.21.15.179:6789/0 conn(0x7fdab015d5b0 legacy=0x7fdab00b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab54d9700 1 -- 172.21.15.179:0/875829185 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00d2130 con 0x7fdab00ce2d0 2024-05-08T22:34:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdab54d9700 1 --2- 172.21.15.179:0/875829185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab006bb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009d90 tx=0x7fdaa800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd379a4fc60cac78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.034+0000 7fdaaf7fe700 1 -- 172.21.15.179:0/875829185 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800e070 con 0x7fdab00ce2d0 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdaaf7fe700 1 -- 172.21.15.179:0/875829185 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa800bb20 con 0x7fdab00ce2d0 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdaaf7fe700 1 -- 172.21.15.179:0/875829185 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8012430 con 0x7fdab00ce2d0 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- 172.21.15.179:0/875829185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 msgr2=0x7fdab006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 --2- 172.21.15.179:0/875829185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab006bb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009d90 tx=0x7fdaa800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- 172.21.15.179:0/875829185 shutdown_connections 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 --2- 172.21.15.179:0/875829185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab006bb60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- 172.21.15.179:0/875829185 >> 172.21.15.179:0/875829185 conn(0x7fdab00b4940 msgr2=0x7fdab00b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- 172.21.15.179:0/875829185 shutdown_connections 2024-05-08T22:34:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- 172.21.15.179:0/875829185 wait complete. 2024-05-08T22:34:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 Processor -- start 2024-05-08T22:34:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.038+0000 7fdab5cda700 1 -- start start 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab00cde70 0x7fdab0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00708a0 con 0x7fdab015d5b0 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0159350 con 0x7fdab00ce2d0 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00cad70 con 0x7fdab00cde70 2024-05-08T22:34:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35240/0 (socket says 172.21.15.179:35240) 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 -- 172.21.15.179:0/18313144 learned_addr learned my addr 172.21.15.179:0/18313144 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdaaffff700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab4cd8700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab00cde70 0x7fdab0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 -- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab00cde70 msgr2=0x7fdab0155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab00cde70 0x7fdab0155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 -- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 msgr2=0x7fdab01588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab01588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 -- 172.21.15.179:0/18313144 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8009a70 con 0x7fdab015d5b0 2024-05-08T22:34:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab54d9700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b680 tx=0x7fdaa8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8012430 con 0x7fdab015d5b0 2024-05-08T22:34:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8020070 con 0x7fdab015d5b0 2024-05-08T22:34:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004280 con 0x7fdab015d5b0 2024-05-08T22:34:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab0070ab0 con 0x7fdab015d5b0 2024-05-08T22:34:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.042+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab0071070 con 0x7fdab015d5b0 2024-05-08T22:34:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdaa80043e0 con 0x7fdab015d5b0 2024-05-08T22:34:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab004e700 con 0x7fdab015d5b0 2024-05-08T22:34:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdaad7fa700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 0x7fda94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdaa8022070 con 0x7fdab015d5b0 2024-05-08T22:34:22.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdab4cd8700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 0x7fda94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.046+0000 7fdab4cd8700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 0x7fda94068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007fd0 tx=0x7fdaa0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.054+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdaa8053080 con 0x7fdab015d5b0 2024-05-08T22:34:22.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdab00d0e50 con 0x7fdab015d5b0 2024-05-08T22:34:22.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdaad7fa700 1 -- 172.21.15.179:0/18313144 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7fdaa802c090 con 0x7fdab015d5b0 2024-05-08T22:34:22.223 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:22.223 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 msgr2=0x7fda94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 0x7fda94068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007fd0 tx=0x7fdaa0007de0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 msgr2=0x7fdab006e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b680 tx=0x7fdaa8004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 shutdown_connections 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda94066730 0x7fda94068bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab00ce2d0 0x7fdab01588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d5b0 0x7fdab006e880 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.222+0000 7fdab5cda700 1 --2- 172.21.15.179:0/18313144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab00cde70 0x7fdab0155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.226+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 >> 172.21.15.179:0/18313144 conn(0x7fdab00b4940 msgr2=0x7fdab00c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.226+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 shutdown_connections 2024-05-08T22:34:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.226+0000 7fdab5cda700 1 -- 172.21.15.179:0/18313144 wait complete. 2024-05-08T22:34:22.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:22.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:22.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.358+0000 7f0e1d8c5700 1 Processor -- start 2024-05-08T22:34:22.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.358+0000 7f0e1d8c5700 1 -- start start 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180d1d20 0x7f0e180cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1815d4a0 con 0x7f0e180ce0e0 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180ce570 con 0x7f0e180d1d20 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1806abb0 con 0x7f0e1815d0c0 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180d1d20 0x7f0e180cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e180d1d20 0x7f0e180cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35246/0 (socket says 172.21.15.179:35246) 2024-05-08T22:34:22.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 -- 172.21.15.179:0/3680981649 learned_addr learned my addr 172.21.15.179:0/3680981649 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:22.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1c8c3700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e177fe700 1 -- 172.21.15.179:0/3680981649 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685887106 0 0) 0x7f0e1815d4a0 con 0x7f0e180ce0e0 2024-05-08T22:34:22.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e177fe700 1 -- 172.21.15.179:0/3680981649 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e000035d0 con 0x7f0e180ce0e0 2024-05-08T22:34:22.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 -- 172.21.15.179:0/3680981649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 msgr2=0x7f0e180b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:22.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 -- 172.21.15.179:0/3680981649 >> v1:172.21.15.195:6789/0 conn(0x7f0e180ce0e0 legacy=0x7f0e18160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 -- 172.21.15.179:0/3680981649 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180d2500 con 0x7f0e180d1d20 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e17fff700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180d1d20 0x7f0e180cdba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009910 tx=0x7f0e08009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fa07abb633a55a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e177fe700 1 -- 172.21.15.179:0/3680981649 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08011830 con 0x7f0e180d1d20 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e177fe700 1 -- 172.21.15.179:0/3680981649 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08011990 con 0x7f0e180d1d20 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e177fe700 1 -- 172.21.15.179:0/3680981649 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08011d00 con 0x7f0e180d1d20 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/3680981649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180d1d20 msgr2=0x7f0e180cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180d1d20 0x7f0e180cdba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009910 tx=0x7f0e08009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/3680981649 shutdown_connections 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e180d1d20 0x7f0e180cdba0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/3680981649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1815d0c0 0x7f0e180b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/3680981649 >> 172.21.15.179:0/3680981649 conn(0x7f0e180d2ed0 msgr2=0x7f0e180d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/3680981649 shutdown_connections 2024-05-08T22:34:22.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/3680981649 wait complete. 2024-05-08T22:34:22.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 Processor -- start 2024-05-08T22:34:22.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.362+0000 7f0e1d8c5700 1 -- start start 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18153180 con 0x7f0e1815d0c0 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180caa60 con 0x7f0e180d1d20 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cad40 con 0x7f0e180ce0e0 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d0c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1c8c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:39006/0 (socket says 172.21.15.179:39006) 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d0c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35248/0 (socket says 172.21.15.179:35248) 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1c8c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:33782/0 (socket says 172.21.15.179:33782) 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 -- 172.21.15.179:0/2667564543 learned_addr learned my addr 172.21.15.179:0/2667564543 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 -- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 msgr2=0x7f0e18152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 -- 172.21.15.179:0/2667564543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 msgr2=0x7f0e180cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 -- 172.21.15.179:0/2667564543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08009570 con 0x7f0e180d1d20 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d0c4700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1c8c3700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e17fff700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009c60 tx=0x7f0e08004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08004300 con 0x7f0e180d1d20 2024-05-08T22:34:22.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e180cb020 con 0x7f0e180d1d20 2024-05-08T22:34:22.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0801f070 con 0x7f0e180d1d20 2024-05-08T22:34:22.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e080036c0 con 0x7f0e180d1d20 2024-05-08T22:34:22.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.366+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e180cb5e0 con 0x7f0e180d1d20 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e180cb7d0 con 0x7f0e180d1d20 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e08021030 con 0x7f0e180d1d20 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e157fa700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0e08085850 con 0x7f0e180d1d20 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e1c8c3700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.370+0000 7f0e1c8c3700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e1804ed90 tx=0x7f0e0c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.378+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e08052820 con 0x7f0e180d1d20 2024-05-08T22:34:22.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e1804e700 con 0x7f0e180d1d20 2024-05-08T22:34:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e157fa700 1 -- 172.21.15.179:0/2667564543 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e08055e40 con 0x7f0e180d1d20 2024-05-08T22:34:22.551 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:22.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 msgr2=0x7f0e04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e1804ed90 tx=0x7f0e0c009240 comp rx=0 tx=0).stop 2024-05-08T22:34:22.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 msgr2=0x7f0e180c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009c60 tx=0x7f0e08004770 comp rx=0 tx=0).stop 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 shutdown_connections 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e04066730 0x7f0e04068bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e180d1d20 0x7f0e180c9fa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1815d0c0 0x7f0e180cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 --2- 172.21.15.179:0/2667564543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e180ce0e0 0x7f0e18152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.550+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 >> 172.21.15.179:0/2667564543 conn(0x7f0e180d2ed0 msgr2=0x7f0e180b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.554+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 shutdown_connections 2024-05-08T22:34:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.554+0000 7f0e1d8c5700 1 -- 172.21.15.179:0/2667564543 wait complete. 2024-05-08T22:34:22.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:22.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 Processor -- start 2024-05-08T22:34:22.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 -- start start 2024-05-08T22:34:22.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcbd40b1a50 0x7fcbd40b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b2c10 0x7fcbd406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd406dd50 con 0x7fcbd414ae30 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd406e9d0 con 0x7fcbd40b1a50 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbdabce700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40b30c0 con 0x7fcbd40b2c10 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcbd40b1a50 0x7fcbd40b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd9bcc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcbd414ae30 0x7fcbd4165b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46644/0 (socket says 172.21.15.179:46644) 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd9bcc700 1 -- 172.21.15.179:0/3873539126 learned_addr learned my addr 172.21.15.179:0/3873539126 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbda3cd700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b2c10 0x7fcbd406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd8bca700 1 -- 172.21.15.179:0/3873539126 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091906265 0 0) 0x7fcbd406dd50 con 0x7fcbd414ae30 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd8bca700 1 -- 172.21.15.179:0/3873539126 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc00035d0 con 0x7fcbd414ae30 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3873539126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b2c10 msgr2=0x7fcbd406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:22.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b2c10 0x7fcbd406abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3873539126 >> v1:172.21.15.179:6789/0 conn(0x7fcbd414ae30 legacy=0x7fcbd4165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3873539126 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd406e030 con 0x7fcbd40b1a50 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd93cb700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b1a50 0x7fcbd40b26d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009850 tx=0x7fcbc8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449d251cbf6b9cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd8bca700 1 -- 172.21.15.179:0/3873539126 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8011830 con 0x7fcbd40b1a50 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd8bca700 1 -- 172.21.15.179:0/3873539126 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc8011990 con 0x7fcbd40b1a50 2024-05-08T22:34:22.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.686+0000 7fcbd8bca700 1 -- 172.21.15.179:0/3873539126 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8011d40 con 0x7fcbd40b1a50 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- 172.21.15.179:0/3873539126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b1a50 msgr2=0x7fcbd40b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b1a50 0x7fcbd40b26d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009850 tx=0x7fcbc8009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- 172.21.15.179:0/3873539126 shutdown_connections 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b2c10 0x7fcbd406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3873539126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b1a50 0x7fcbd40b26d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- 172.21.15.179:0/3873539126 >> 172.21.15.179:0/3873539126 conn(0x7fcbd40cea30 msgr2=0x7fcbd40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- 172.21.15.179:0/3873539126 shutdown_connections 2024-05-08T22:34:22.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- 172.21.15.179:0/3873539126 wait complete. 2024-05-08T22:34:22.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 Processor -- start 2024-05-08T22:34:22.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.690+0000 7fcbdabce700 1 -- start start 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b1a50 0x7fcbd41ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbd414ae30 0x7fcbd415ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415fae0 con 0x7fcbd40b2c10 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415f1f0 con 0x7fcbd414ae30 2024-05-08T22:34:22.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd41ed220 con 0x7fcbd40b1a50 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35266/0 (socket says 172.21.15.179:35266) 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3823045290 learned_addr learned my addr 172.21.15.179:0/3823045290 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbda3cd700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbd414ae30 0x7fcbd415ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd9bcc700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b1a50 0x7fcbd41ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b1a50 msgr2=0x7fcbd41ecce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b1a50 0x7fcbd41ecce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbd414ae30 msgr2=0x7fcbd415ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbd414ae30 0x7fcbd415ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 -- 172.21.15.179:0/3823045290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80094b0 con 0x7fcbd40b2c10 2024-05-08T22:34:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbd93cb700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80090b0 tx=0x7fcbc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8004380 con 0x7fcbd40b2c10 2024-05-08T22:34:22.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc801f070 con 0x7fcbd40b2c10 2024-05-08T22:34:22.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40ccc00 con 0x7fcbd40b2c10 2024-05-08T22:34:22.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80036c0 con 0x7fcbd40b2c10 2024-05-08T22:34:22.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd415f770 con 0x7fcbd40b2c10 2024-05-08T22:34:22.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd415fc90 con 0x7fcbd40b2c10 2024-05-08T22:34:22.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.694+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcbc8021030 con 0x7fcbd40b2c10 2024-05-08T22:34:22.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.698+0000 7fcbc67fc700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:22.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.698+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcbc8085680 con 0x7fcbd40b2c10 2024-05-08T22:34:22.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.698+0000 7fcbd9bcc700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:22.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.698+0000 7fcbd9bcc700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 0x7fcbb8068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0007fd0 tx=0x7fcbd0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:22.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.702+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcbd415fc90 con 0x7fcbd40b2c10 2024-05-08T22:34:22.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.870+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbd404e700 con 0x7fcbd40b2c10 2024-05-08T22:34:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbc67fc700 1 -- 172.21.15.179:0/3823045290 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbc80527e0 con 0x7fcbd40b2c10 2024-05-08T22:34:22.875 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 msgr2=0x7fcbb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 0x7fcbb8068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0007fd0 tx=0x7fcbd0007de0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 msgr2=0x7fcbd415dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80090b0 tx=0x7fcbc8004770 comp rx=0 tx=0).stop 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 shutdown_connections 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbd414ae30 0x7fcbd415ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd40b2c10 0x7fcbd415dbe0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 --2- 172.21.15.179:0/3823045290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbd40b1a50 0x7fcbd41ecce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 >> 172.21.15.179:0/3823045290 conn(0x7fcbd40cea30 msgr2=0x7fcbd40af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 shutdown_connections 2024-05-08T22:34:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:22.874+0000 7fcbdabce700 1 -- 172.21.15.179:0/3823045290 wait complete. 2024-05-08T22:34:22.889 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:23.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.010+0000 7fc90eb28700 1 Processor -- start 2024-05-08T22:34:23.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.010+0000 7fc90eb28700 1 -- start start 2024-05-08T22:34:23.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc9080b8a80 0x7fc9080cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc9080ce0e0 0x7fc90806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9080b8ee0 con 0x7fc90815d0c0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9080ce570 con 0x7fc9080ce0e0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90815d4a0 con 0x7fc9080b8a80 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90db26700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc90815d0c0 0x7fc9080d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46652/0 (socket says 172.21.15.179:46652) 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90db26700 1 -- 172.21.15.179:0/1383006833 learned_addr learned my addr 172.21.15.179:0/1383006833 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 --2- 172.21.15.179:0/1383006833 >> v2:172.21.15.179:3300/0 conn(0x7fc9080ce0e0 0x7fc90806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90d325700 1 --2- 172.21.15.179:0/1383006833 >> v2:172.21.15.195:3300/0 conn(0x7fc9080b8a80 0x7fc9080cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 -- 172.21.15.179:0/1383006833 >> v2:172.21.15.195:3300/0 conn(0x7fc9080b8a80 msgr2=0x7fc9080cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 --2- 172.21.15.179:0/1383006833 >> v2:172.21.15.195:3300/0 conn(0x7fc9080b8a80 0x7fc9080cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 -- 172.21.15.179:0/1383006833 >> v1:172.21.15.179:6789/0 conn(0x7fc90815d0c0 legacy=0x7fc9080d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90cb24700 1 -- 172.21.15.179:0/1383006833 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773810834 0 0) 0x7fc9080b8ee0 con 0x7fc90815d0c0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 -- 172.21.15.179:0/1383006833 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90806abb0 con 0x7fc9080ce0e0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90e327700 1 --2- 172.21.15.179:0/1383006833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080ce0e0 0x7fc90806b370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc908160b00 tx=0x7fc904009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebb29970863a201 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90cb24700 1 -- 172.21.15.179:0/1383006833 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904011830 con 0x7fc9080ce0e0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90cb24700 1 -- 172.21.15.179:0/1383006833 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904011990 con 0x7fc9080ce0e0 2024-05-08T22:34:23.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90cb24700 1 -- 172.21.15.179:0/1383006833 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904011d40 con 0x7fc9080ce0e0 2024-05-08T22:34:23.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- 172.21.15.179:0/1383006833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080ce0e0 msgr2=0x7fc90806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 --2- 172.21.15.179:0/1383006833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080ce0e0 0x7fc90806b370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc908160b00 tx=0x7fc904009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:23.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- 172.21.15.179:0/1383006833 shutdown_connections 2024-05-08T22:34:23.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 --2- 172.21.15.179:0/1383006833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080ce0e0 0x7fc90806b370 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 --2- 172.21.15.179:0/1383006833 >> v2:172.21.15.195:3300/0 conn(0x7fc9080b8a80 0x7fc9080cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- 172.21.15.179:0/1383006833 >> 172.21.15.179:0/1383006833 conn(0x7fc9080d2ed0 msgr2=0x7fc9080d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- 172.21.15.179:0/1383006833 shutdown_connections 2024-05-08T22:34:23.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.014+0000 7fc90eb28700 1 -- 172.21.15.179:0/1383006833 wait complete. 2024-05-08T22:34:23.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 Processor -- start 2024-05-08T22:34:23.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- start start 2024-05-08T22:34:23.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9080ce0e0 0x7fc908155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 0x7fc90815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908150c80 con 0x7fc9080b8a80 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908154c50 con 0x7fc90815d0c0 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc908154f30 con 0x7fc9080ce0e0 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35276/0 (socket says 172.21.15.179:35276) 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 -- 172.21.15.179:0/2963702553 learned_addr learned my addr 172.21.15.179:0/2963702553 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90d325700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9080ce0e0 0x7fc908155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90e327700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 0x7fc90815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 -- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9080ce0e0 msgr2=0x7fc908155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9080ce0e0 0x7fc908155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 -- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 msgr2=0x7fc90815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 0x7fc90815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 -- 172.21.15.179:0/2963702553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040094b0 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90e327700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 0x7fc90815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90db26700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc90000cc20 tx=0x7fc90000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000aaa0 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90000ac20 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc908155210 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000070d0 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.018+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc908156150 con 0x7fc9080b8a80 2024-05-08T22:34:23.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.022+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90804e700 con 0x7fc9080b8a80 2024-05-08T22:34:23.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.022+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc900007230 con 0x7fc9080b8a80 2024-05-08T22:34:23.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.022+0000 7fc8fe7fc700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.022+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc9000847b0 con 0x7fc9080b8a80 2024-05-08T22:34:23.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.026+0000 7fc90d325700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.026+0000 7fc90d325700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc908150f60 tx=0x7fc8f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.030+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc900051740 con 0x7fc9080b8a80 2024-05-08T22:34:23.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.198+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9080676e0 con 0x7fc9080b8a80 2024-05-08T22:34:23.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.198+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2963702553 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc900018020 con 0x7fc9080b8a80 2024-05-08T22:34:23.200 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 msgr2=0x7fc8ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc908150f60 tx=0x7fc8f8008040 comp rx=0 tx=0).stop 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 msgr2=0x7fc9081545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc90000cc20 tx=0x7fc90000cf30 comp rx=0 tx=0).stop 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 shutdown_connections 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90815d0c0 0x7fc90815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9080ce0e0 0x7fc908155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 --2- 172.21.15.179:0/2963702553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9080b8a80 0x7fc9081545c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 >> 172.21.15.179:0/2963702553 conn(0x7fc9080d2ed0 msgr2=0x7fc9081533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 shutdown_connections 2024-05-08T22:34:23.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.202+0000 7fc90eb28700 1 -- 172.21.15.179:0/2963702553 wait complete. 2024-05-08T22:34:23.215 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:23.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.334+0000 7f2a6df26700 1 Processor -- start 2024-05-08T22:34:23.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.334+0000 7f2a6df26700 1 -- start start 2024-05-08T22:34:23.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.334+0000 7f2a6df26700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 0x7f2a680b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.334+0000 7f2a6df26700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2a680d1d20 0x7f2a680cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6815d4a0 con 0x7f2a680ce0e0 2024-05-08T22:34:23.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a680ce570 con 0x7f2a680d1d20 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6806abb0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2a680d1d20 0x7f2a680cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2a680d1d20 0x7f2a680cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35280/0 (socket says 172.21.15.179:35280) 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 -- 172.21.15.179:0/502993748 learned_addr learned my addr 172.21.15.179:0/502993748 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6cf24700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 0x7f2a680b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 -- 172.21.15.179:0/502993748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 msgr2=0x7f2a680b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 0x7f2a680b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 -- 172.21.15.179:0/502993748 >> v1:172.21.15.179:6789/0 conn(0x7f2a680ce0e0 legacy=0x7f2a68160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 -- 172.21.15.179:0/502993748 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a680d2500 con 0x7f2a680d1d20 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a677fe700 1 -- 172.21.15.179:0/502993748 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352130127 0 0) 0x7f2a6815d4a0 con 0x7f2a680ce0e0 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6cf24700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 0x7f2a680b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a67fff700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a680d1d20 0x7f2a680cdba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009910 tx=0x7f2a58009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f746823b8f09e77a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a677fe700 1 -- 172.21.15.179:0/502993748 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800ba60 con 0x7f2a680d1d20 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a677fe700 1 -- 172.21.15.179:0/502993748 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5800bbc0 con 0x7f2a680d1d20 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a677fe700 1 -- 172.21.15.179:0/502993748 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58009e60 con 0x7f2a680d1d20 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- 172.21.15.179:0/502993748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a680d1d20 msgr2=0x7f2a680cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a680d1d20 0x7f2a680cdba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009910 tx=0x7f2a58009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- 172.21.15.179:0/502993748 shutdown_connections 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a680d1d20 0x7f2a680cdba0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 --2- 172.21.15.179:0/502993748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6815d0c0 0x7f2a680b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- 172.21.15.179:0/502993748 >> 172.21.15.179:0/502993748 conn(0x7f2a680d2ed0 msgr2=0x7f2a680d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- 172.21.15.179:0/502993748 shutdown_connections 2024-05-08T22:34:23.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.338+0000 7f2a6df26700 1 -- 172.21.15.179:0/502993748 wait complete. 2024-05-08T22:34:23.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 Processor -- start 2024-05-08T22:34:23.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 -- start start 2024-05-08T22:34:23.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a680ce0e0 0x7f2a680c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a680d1d20 0x7f2a68146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a68143fe0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a681433d0 con 0x7f2a680d1d20 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a681436b0 con 0x7f2a680ce0e0 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35288/0 (socket says 172.21.15.179:35288) 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 -- 172.21.15.179:0/947368522 learned_addr learned my addr 172.21.15.179:0/947368522 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:23.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a67fff700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a680d1d20 0x7f2a68146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6cf24700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a680ce0e0 0x7f2a680c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 -- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a680ce0e0 msgr2=0x7f2a680c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a680ce0e0 0x7f2a680c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 -- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a680d1d20 msgr2=0x7f2a68146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a680d1d20 0x7f2a68146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 -- 172.21.15.179:0/947368522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58009570 con 0x7f2a6815d0c0 2024-05-08T22:34:23.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6d725700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000cc70 tx=0x7f2a6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60003e50 con 0x7f2a6815d0c0 2024-05-08T22:34:23.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6000ce70 con 0x7f2a6815d0c0 2024-05-08T22:34:23.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a68143990 con 0x7f2a6815d0c0 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.342+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000abf0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.346+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a680c8fe0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.346+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a68144150 con 0x7f2a6815d0c0 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.346+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2a6000ad50 con 0x7f2a6815d0c0 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.346+0000 7f2a657fa700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 0x7f2a54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.350+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2a60084190 con 0x7f2a6815d0c0 2024-05-08T22:34:23.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.350+0000 7f2a6cf24700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 0x7f2a54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.350+0000 7f2a6cf24700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 0x7f2a54068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c920 tx=0x7f2a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.354+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2a600510a0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2a6804e700 con 0x7f2a6815d0c0 2024-05-08T22:34:23.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a657fa700 1 -- 172.21.15.179:0/947368522 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2a600546c0 con 0x7f2a6815d0c0 2024-05-08T22:34:23.527 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:23.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 msgr2=0x7f2a54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 0x7f2a54068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c920 tx=0x7f2a5c008040 comp rx=0 tx=0).stop 2024-05-08T22:34:23.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 msgr2=0x7f2a68144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000cc70 tx=0x7f2a6000a340 comp rx=0 tx=0).stop 2024-05-08T22:34:23.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 shutdown_connections 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2a54066730 0x7f2a54068bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a680d1d20 0x7f2a68146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6815d0c0 0x7f2a68144470 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 --2- 172.21.15.179:0/947368522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a680ce0e0 0x7f2a680c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 >> 172.21.15.179:0/947368522 conn(0x7f2a680d2ed0 msgr2=0x7f2a680ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 shutdown_connections 2024-05-08T22:34:23.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.526+0000 7f2a6df26700 1 -- 172.21.15.179:0/947368522 wait complete. 2024-05-08T22:34:23.543 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:23.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.654+0000 7f38d86dd700 1 Processor -- start 2024-05-08T22:34:23.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.654+0000 7f38d86dd700 1 -- start start 2024-05-08T22:34:23.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d0158d70 0x7f38d015b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38d006bbc0 0x7f38d006c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d006ed60 con 0x7f38d006ceb0 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d006c590 con 0x7f38d006bbc0 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d006d330 con 0x7f38d0158d70 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38d006bbc0 0x7f38d006c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5c78700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f38d006ceb0 0x7f38d006e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46678/0 (socket says 172.21.15.179:46678) 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5c78700 1 -- 172.21.15.179:0/3352594020 learned_addr learned my addr 172.21.15.179:0/3352594020 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6479700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d0158d70 0x7f38d015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 -- 172.21.15.179:0/3352594020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d0158d70 msgr2=0x7f38d015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d0158d70 0x7f38d015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 -- 172.21.15.179:0/3352594020 >> v1:172.21.15.179:6789/0 conn(0x7f38d006ceb0 legacy=0x7f38d006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5477700 1 -- 172.21.15.179:0/3352594020 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380291109 0 0) 0x7f38d006ed60 con 0x7f38d006ceb0 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 -- 172.21.15.179:0/3352594020 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00b1c70 con 0x7f38d006bbc0 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d6c7a700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d006bbc0 0x7f38d006c050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009850 tx=0x7f38cc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb4e83cb3231c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5477700 1 -- 172.21.15.179:0/3352594020 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011830 con 0x7f38d006bbc0 2024-05-08T22:34:23.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5477700 1 -- 172.21.15.179:0/3352594020 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc011990 con 0x7f38d006bbc0 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d5477700 1 -- 172.21.15.179:0/3352594020 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011d40 con 0x7f38d006bbc0 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- 172.21.15.179:0/3352594020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d006bbc0 msgr2=0x7f38d006c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d006bbc0 0x7f38d006c050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009850 tx=0x7f38cc009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- 172.21.15.179:0/3352594020 shutdown_connections 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d006bbc0 0x7f38d006c050 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 --2- 172.21.15.179:0/3352594020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d0158d70 0x7f38d015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- 172.21.15.179:0/3352594020 >> 172.21.15.179:0/3352594020 conn(0x7f38d0070e50 msgr2=0x7f38d0071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- 172.21.15.179:0/3352594020 shutdown_connections 2024-05-08T22:34:23.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.658+0000 7f38d86dd700 1 -- 172.21.15.179:0/3352594020 wait complete. 2024-05-08T22:34:23.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 Processor -- start 2024-05-08T22:34:23.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 -- start start 2024-05-08T22:34:23.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38d006bbc0 0x7f38d0156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d006ceb0 0x7f38d0143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0144a00 con 0x7f38d0158d70 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0143b50 con 0x7f38d006bbc0 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d86dd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0143d80 con 0x7f38d006ceb0 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35298/0 (socket says 172.21.15.179:35298) 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 -- 172.21.15.179:0/1248883114 learned_addr learned my addr 172.21.15.179:0/1248883114 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6479700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38d006bbc0 0x7f38d0156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d5c78700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d006ceb0 0x7f38d0143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 -- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d006ceb0 msgr2=0x7f38d0143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d006ceb0 0x7f38d0143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 -- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38d006bbc0 msgr2=0x7f38d0156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38d006bbc0 0x7f38d0156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 -- 172.21.15.179:0/1248883114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0094b0 con 0x7f38d0158d70 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.662+0000 7f38d6c7a700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f38d0067e10 tx=0x7f38cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004550 con 0x7f38d0158d70 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d0144060 con 0x7f38d0158d70 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc01f070 con 0x7f38d0158d70 2024-05-08T22:34:23.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003900 con 0x7f38d0158d70 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d0144620 con 0x7f38d0158d70 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0067770 con 0x7f38d0158d70 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38cc003a60 con 0x7f38d0158d70 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 0x7f38b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f38cc021070 con 0x7f38d0158d70 2024-05-08T22:34:23.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.666+0000 7f38d6479700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 0x7f38b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:23.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.670+0000 7f38d6479700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 0x7f38b4068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38d00b1aa0 tx=0x7f38c0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:23.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.678+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f38cc052cd0 con 0x7f38d0158d70 2024-05-08T22:34:23.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.862+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38d0144b40 con 0x7f38d0158d70 2024-05-08T22:34:23.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.862+0000 7f38c6ffd700 1 -- 172.21.15.179:0/1248883114 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f38d0144b40 con 0x7f38d0158d70 2024-05-08T22:34:23.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 msgr2=0x7f38b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 0x7f38b4068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38d00b1aa0 tx=0x7f38c0009240 comp rx=0 tx=0).stop 2024-05-08T22:34:23.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 msgr2=0x7f38d0144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:23.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f38d0067e10 tx=0x7f38cc004770 comp rx=0 tx=0).stop 2024-05-08T22:34:23.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 shutdown_connections 2024-05-08T22:34:23.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38b4066730 0x7f38b4068bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38d0158d70 0x7f38d0144e20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38d006ceb0 0x7f38d0143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 --2- 172.21.15.179:0/1248883114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38d006bbc0 0x7f38d0156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:23.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 >> 172.21.15.179:0/1248883114 conn(0x7f38d0070e50 msgr2=0x7f38d00b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:23.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 shutdown_connections 2024-05-08T22:34:23.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.866+0000 7f38d86dd700 1 -- 172.21.15.179:0/1248883114 wait complete. 2024-05-08T22:34:23.869 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:23.878 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:34:23.878 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:23.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:23.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 Processor -- start 2024-05-08T22:34:24.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 -- start start 2024-05-08T22:34:24.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a406efd0 con 0x7f93a41434d0 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a406ddf0 con 0x7f93a406eb80 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a982f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40cd200 con 0x7f93a40ce2a0 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a882d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f93a41434d0 0x7f93a406d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46694/0 (socket says 172.21.15.179:46694) 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a882d700 1 -- 172.21.15.179:0/2144503555 learned_addr learned my addr 172.21.15.179:0/2144503555 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a3fff700 1 --2- 172.21.15.179:0/2144503555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019030156 0 0) 0x7f939c001490 con 0x7f93a41434d0 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938c0035d0 con 0x7f93a41434d0 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789134810 0 0) 0x7f93a406ddf0 con 0x7f93a406eb80 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:23.998+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939c001490 con 0x7f93a406eb80 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a3fff700 1 -- 172.21.15.179:0/2144503555 >> v1:172.21.15.195:6789/0 conn(0x7f93a406eb80 legacy=0x7f93a40b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a3fff700 1 -- 172.21.15.179:0/2144503555 >> v1:172.21.15.179:6789/0 conn(0x7f93a41434d0 legacy=0x7f93a406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a3fff700 1 -- 172.21.15.179:0/2144503555 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a40cd4e0 con 0x7f93a40ce2a0 2024-05-08T22:34:24.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a3fff700 1 --2- 172.21.15.179:0/2144503555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a406e640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9398009880 tx=0x7f9398009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ed7fbb3d0688d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398011830 con 0x7f93a40ce2a0 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398011990 con 0x7f93a40ce2a0 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a37fe700 1 -- 172.21.15.179:0/2144503555 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398011d20 con 0x7f93a40ce2a0 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- 172.21.15.179:0/2144503555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 msgr2=0x7f93a406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 --2- 172.21.15.179:0/2144503555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a406e640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9398009880 tx=0x7f9398009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- 172.21.15.179:0/2144503555 shutdown_connections 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 --2- 172.21.15.179:0/2144503555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a406e640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- 172.21.15.179:0/2144503555 >> 172.21.15.179:0/2144503555 conn(0x7f93a40ce9d0 msgr2=0x7f93a40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- 172.21.15.179:0/2144503555 shutdown_connections 2024-05-08T22:34:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- 172.21.15.179:0/2144503555 wait complete. 2024-05-08T22:34:24.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 Processor -- start 2024-05-08T22:34:24.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- start start 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a40c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 0x7f93a4140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41f3560 con 0x7f93a406eb80 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41f28e0 con 0x7f93a41434d0 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.002+0000 7f93a982f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41f2bc0 con 0x7f93a40ce2a0 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35304/0 (socket says 172.21.15.179:35304) 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 -- 172.21.15.179:0/2818595096 learned_addr learned my addr 172.21.15.179:0/2818595096 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a3fff700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a40c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a902e700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 0x7f93a4140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 -- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 msgr2=0x7f93a40c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a40c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 -- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 msgr2=0x7f93a4140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 0x7f93a4140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 -- 172.21.15.179:0/2818595096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980094b0 con 0x7f93a406eb80 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a902e700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 0x7f93a4140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a882d700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f939c001f30 tx=0x7f939c001f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00a9e0 con 0x7f93a406eb80 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c00ab40 con 0x7f93a406eb80 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a41f2ea0 con 0x7f93a406eb80 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c012400 con 0x7f93a406eb80 2024-05-08T22:34:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a4159970 con 0x7f93a406eb80 2024-05-08T22:34:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a4067740 con 0x7f93a406eb80 2024-05-08T22:34:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f939c01b3b0 con 0x7f93a406eb80 2024-05-08T22:34:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.006+0000 7f93a17fa700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 0x7f9390068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.010+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f939c083f50 con 0x7f93a406eb80 2024-05-08T22:34:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.010+0000 7f93a3fff700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 0x7f9390068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.010+0000 7f93a3fff700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 0x7f9390068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93980090b0 tx=0x7f939801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.014+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f939c050e90 con 0x7f93a406eb80 2024-05-08T22:34:24.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.186+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93a406efd0 con 0x7f93a406eb80 2024-05-08T22:34:24.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.186+0000 7f93a17fa700 1 -- 172.21.15.179:0/2818595096 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f93a406efd0 con 0x7f93a406eb80 2024-05-08T22:34:24.190 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:24.190 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 msgr2=0x7f9390068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 0x7f9390068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93980090b0 tx=0x7f939801c040 comp rx=0 tx=0).stop 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 msgr2=0x7f93a4159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f939c001f30 tx=0x7f939c001f60 comp rx=0 tx=0).stop 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 shutdown_connections 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9390066730 0x7f9390068bf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93a41434d0 0x7f93a4140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93a40ce2a0 0x7f93a40c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 --2- 172.21.15.179:0/2818595096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a406eb80 0x7f93a4159430 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 >> 172.21.15.179:0/2818595096 conn(0x7f93a40ce9d0 msgr2=0x7f93a40cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 shutdown_connections 2024-05-08T22:34:24.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.190+0000 7f93a982f700 1 -- 172.21.15.179:0/2818595096 wait complete. 2024-05-08T22:34:24.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:24.205 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:24.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 Processor -- start 2024-05-08T22:34:24.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 -- start start 2024-05-08T22:34:24.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fccf414ae30 0x7fccf4165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fccf40b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf4166090 con 0x7fccf40b2c10 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf406e9d0 con 0x7fccf414ae30 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40b30c0 con 0x7fccf40b1a50 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf2d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fccf40b2c10 0x7fccf4161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46710/0 (socket says 172.21.15.179:46710) 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf2d9d700 1 -- 172.21.15.179:0/1171311872 learned_addr learned my addr 172.21.15.179:0/1171311872 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fccf40b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf259c700 1 --2- 172.21.15.179:0/1171311872 >> v2:172.21.15.195:3300/0 conn(0x7fccf414ae30 0x7fccf4165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154336155 0 0) 0x7fccf4166090 con 0x7fccf40b2c10 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce00035d0 con 0x7fccf40b2c10 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434386926 0 0) 0x7fcce00035d0 con 0x7fccf40b2c10 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 msgr2=0x7fccf40b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fccf40b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 >> v2:172.21.15.195:3300/0 conn(0x7fccf414ae30 msgr2=0x7fccf4165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 --2- 172.21.15.179:0/1171311872 >> v2:172.21.15.195:3300/0 conn(0x7fccf414ae30 0x7fccf4165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf406dd50 con 0x7fccf40b2c10 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fccec001670 con 0x7fccf40b2c10 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 >> v1:172.21.15.179:6789/0 conn(0x7fccf40b2c10 legacy=0x7fccf4161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce0003a20 0x7fcce0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce0004870 0x7fcce0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fcce0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf406e9d0 con 0x7fcce0004870 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40b30c0 con 0x7fcce0003a20 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce000ba30 con 0x7fccf40b1a50 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce0004870 0x7fcce0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf259c700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce0003a20 0x7fcce0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.322+0000 7fccf2d9d700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fcce0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 -- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 msgr2=0x7fcce0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fcce0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 -- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce0003a20 msgr2=0x7fcce0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce0003a20 0x7fcce0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 -- 172.21.15.179:0/1171311872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00035d0 con 0x7fcce0004870 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf2d9d700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccf40b1a50 0x7fcce0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce0004870 0x7fcce0004cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009b70 tx=0x7fcce00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e73a4a23aad81d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00e6c0 con 0x7fcce0004870 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00e820 con 0x7fcce0004870 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf159a700 1 -- 172.21.15.179:0/1171311872 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00eb90 con 0x7fcce0004870 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- 172.21.15.179:0/1171311872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce0004870 msgr2=0x7fcce0004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 --2- 172.21.15.179:0/1171311872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce0004870 0x7fcce0004cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009b70 tx=0x7fcce00058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf2d9d700 1 -- 172.21.15.179:0/1171311872 reap_dead start 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- 172.21.15.179:0/1171311872 shutdown_connections 2024-05-08T22:34:24.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- 172.21.15.179:0/1171311872 >> 172.21.15.179:0/1171311872 conn(0x7fccf40cea30 msgr2=0x7fccf40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- 172.21.15.179:0/1171311872 shutdown_connections 2024-05-08T22:34:24.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- 172.21.15.179:0/1171311872 wait complete. 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 Processor -- start 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- start start 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce0004870 0x7fccf40addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccf413abc0 0x7fccf413b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf413bbb0 con 0x7fccf40cce90 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4139390 con 0x7fccf413abc0 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf413c020 con 0x7fcce0004870 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35324/0 (socket says 172.21.15.179:35324) 2024-05-08T22:34:24.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf1d9b700 1 -- 172.21.15.179:0/4127929307 learned_addr learned my addr 172.21.15.179:0/4127929307 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.326+0000 7fccf259c700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce0004870 0x7fccf40addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf2d9d700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccf413abc0 0x7fccf413b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 -- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce0004870 msgr2=0x7fccf40addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce0004870 0x7fccf40addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 -- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccf413abc0 msgr2=0x7fccf413b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccf413abc0 0x7fccf413b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 -- 172.21.15.179:0/4127929307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf40cd830 con 0x7fccf40cce90 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf1d9b700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e4b0 tx=0x7fccdc0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc021070 con 0x7fccf40cce90 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc0036c0 con 0x7fccf40cce90 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf413b8d0 con 0x7fccf40cce90 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00e6c0 con 0x7fccf40cce90 2024-05-08T22:34:24.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf4138fb0 con 0x7fccf40cce90 2024-05-08T22:34:24.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.330+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf4138b30 con 0x7fccf40cce90 2024-05-08T22:34:24.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.334+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccdc025070 con 0x7fccf40cce90 2024-05-08T22:34:24.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.334+0000 7fcceaffd700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 0x7fccd4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.334+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fccdc085720 con 0x7fccf40cce90 2024-05-08T22:34:24.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.334+0000 7fccf259c700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 0x7fccd4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.334+0000 7fccf259c700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 0x7fccd4068c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcce4014510 tx=0x7fcce400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.338+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fccf4138b30 con 0x7fccf40cce90 2024-05-08T22:34:24.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.514+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccf4138b30 con 0x7fccf40cce90 2024-05-08T22:34:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.514+0000 7fcceaffd700 1 -- 172.21.15.179:0/4127929307 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccf4138b30 con 0x7fccf40cce90 2024-05-08T22:34:24.515 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 msgr2=0x7fccd4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 0x7fccd4068c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcce4014510 tx=0x7fcce400b040 comp rx=0 tx=0).stop 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 msgr2=0x7fccf40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e4b0 tx=0x7fccdc0047e0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 shutdown_connections 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccd4066760 0x7fccd4068c20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccf413abc0 0x7fccf413b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccf40cce90 0x7fccf40cd2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 --2- 172.21.15.179:0/4127929307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce0004870 0x7fccf40addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 >> 172.21.15.179:0/4127929307 conn(0x7fccf40cea30 msgr2=0x7fccf406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 shutdown_connections 2024-05-08T22:34:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.518+0000 7fccf359e700 1 -- 172.21.15.179:0/4127929307 wait complete. 2024-05-08T22:34:24.531 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:24.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 Processor -- start 2024-05-08T22:34:24.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 -- start start 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c0d1040 con 0x7f1d4c15d160 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c06c100 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d543f4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c06c330 con 0x7f1d4c1614c0 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d52991700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d52991700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35338/0 (socket says 172.21.15.179:35338) 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d52991700 1 -- 172.21.15.179:0/1699120707 learned_addr learned my addr 172.21.15.179:0/1699120707 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.654+0000 7f1d5198f700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d52991700 1 -- 172.21.15.179:0/1699120707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 msgr2=0x7f1d4c0b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d52991700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d52991700 1 -- 172.21.15.179:0/1699120707 >> v1:172.21.15.195:6789/0 conn(0x7f1d4c15d160 legacy=0x7f1d4c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d52991700 1 -- 172.21.15.179:0/1699120707 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c06c610 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d52991700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009e50 tx=0x7f1d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a191ec82619459 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d5118e700 1 -- 172.21.15.179:0/1699120707 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400e070 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d5118e700 1 -- 172.21.15.179:0/1699120707 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400b910 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d5118e700 1 -- 172.21.15.179:0/1699120707 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400bcc0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 -- 172.21.15.179:0/1699120707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 msgr2=0x7f1d4c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009e50 tx=0x7f1d4400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 -- 172.21.15.179:0/1699120707 shutdown_connections 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1699120707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c06abb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 -- 172.21.15.179:0/1699120707 >> 172.21.15.179:0/1699120707 conn(0x7f1d4c0b4af0 msgr2=0x7f1d4c0b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 -- 172.21.15.179:0/1699120707 shutdown_connections 2024-05-08T22:34:24.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.658+0000 7f1d543f4700 1 -- 172.21.15.179:0/1699120707 wait complete. 2024-05-08T22:34:24.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 Processor -- start 2024-05-08T22:34:24.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- start start 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d4c15d160 0x7f1d4c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c153e20 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c154000 con 0x7f1d4c15d160 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1542e0 con 0x7f1d4c1614c0 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35342/0 (socket says 172.21.15.179:35342) 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 -- 172.21.15.179:0/1465685804 learned_addr learned my addr 172.21.15.179:0/1465685804 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52991700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d5198f700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d4c15d160 0x7f1d4c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 -- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 msgr2=0x7f1d4c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 -- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d4c15d160 msgr2=0x7f1d4c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d4c15d160 0x7f1d4c0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 -- 172.21.15.179:0/1465685804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009b30 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d52190700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c500 tx=0x7f1d3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0052d0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c005430 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c1545c0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0055a0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.662+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c0c97e0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c154ae0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d3c0038e0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d42ffd700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 0x7f1d30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1d3c083ac0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d5198f700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 0x7f1d30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.666+0000 7f1d5198f700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 0x7f1d30068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c15f0a0 tx=0x7f1d48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.674+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1d4c154ae0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.842+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d4c154ae0 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.842+0000 7f1d42ffd700 1 -- 172.21.15.179:0/1465685804 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d3c083e60 con 0x7f1d4c0d2b10 2024-05-08T22:34:24.844 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 msgr2=0x7f1d30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 0x7f1d30068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c15f0a0 tx=0x7f1d48008040 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 msgr2=0x7f1d4c0c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c500 tx=0x7f1d3c00c8c0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 shutdown_connections 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d30066730 0x7f1d30068bf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d4c15d160 0x7f1d4c0ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4c1614c0 0x7f1d4c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 --2- 172.21.15.179:0/1465685804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d4c0d2b10 0x7f1d4c0c92a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 >> 172.21.15.179:0/1465685804 conn(0x7f1d4c0b4af0 msgr2=0x7f1d4c06b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 shutdown_connections 2024-05-08T22:34:24.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.846+0000 7f1d543f4700 1 -- 172.21.15.179:0/1465685804 wait complete. 2024-05-08T22:34:24.859 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:24.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.974+0000 7f624114d700 1 Processor -- start 2024-05-08T22:34:24.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.974+0000 7f624114d700 1 -- start start 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f623c0b19e0 0x7f623c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159700 0x7f623c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c06cda0 con 0x7f623c159320 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0cb4c0 con 0x7f623c0b19e0 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0b1f10 con 0x7f623c159700 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f623c0b19e0 0x7f623c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f623c0b19e0 0x7f623c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35352/0 (socket says 172.21.15.179:35352) 2024-05-08T22:34:24.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 -- 172.21.15.179:0/965242282 learned_addr learned my addr 172.21.15.179:0/965242282 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624094c700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159700 0x7f623c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 -- 172.21.15.179:0/965242282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159700 msgr2=0x7f623c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159700 0x7f623c06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 -- 172.21.15.179:0/965242282 >> v1:172.21.15.195:6789/0 conn(0x7f623c159320 legacy=0x7f623c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 -- 172.21.15.179:0/965242282 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0b21f0 con 0x7f623c0b19e0 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623b7fe700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c0cc630 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f622c009910 tx=0x7f622c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5914001ffbd9cb38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623affd700 1 -- 172.21.15.179:0/965242282 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c011830 con 0x7f623c0b19e0 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623affd700 1 -- 172.21.15.179:0/965242282 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c011990 con 0x7f623c0b19e0 2024-05-08T22:34:24.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f623affd700 1 -- 172.21.15.179:0/965242282 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c011d40 con 0x7f623c0b19e0 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- 172.21.15.179:0/965242282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 msgr2=0x7f623c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c0cc630 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f622c009910 tx=0x7f622c009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- 172.21.15.179:0/965242282 shutdown_connections 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159700 0x7f623c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 --2- 172.21.15.179:0/965242282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c0cc630 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- 172.21.15.179:0/965242282 >> 172.21.15.179:0/965242282 conn(0x7f623c0cea30 msgr2=0x7f623c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- 172.21.15.179:0/965242282 shutdown_connections 2024-05-08T22:34:24.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.978+0000 7f624114d700 1 -- 172.21.15.179:0/965242282 wait complete. 2024-05-08T22:34:24.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 Processor -- start 2024-05-08T22:34:24.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- start start 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159320 0x7f623c157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c159700 0x7f623c157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1f3600 con 0x7f623c0b19e0 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1f2a70 con 0x7f623c159700 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c15cae0 con 0x7f623c159320 2024-05-08T22:34:24.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35360/0 (socket says 172.21.15.179:35360) 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 -- 172.21.15.179:0/1998933400 learned_addr learned my addr 172.21.15.179:0/1998933400 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623b7fe700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159320 0x7f623c157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624094c700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c159700 0x7f623c157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 -- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159320 msgr2=0x7f623c157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159320 0x7f623c157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 -- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c159700 msgr2=0x7f623c157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c159700 0x7f623c157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 -- 172.21.15.179:0/1998933400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009570 con 0x7f623c0b19e0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f623bfff700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f622400c500 tx=0x7f622400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240052d0 con 0x7f623c0b19e0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224005430 con 0x7f623c0b19e0 2024-05-08T22:34:24.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c1f3a20 con 0x7f623c0b19e0 2024-05-08T22:34:24.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240055a0 con 0x7f623c0b19e0 2024-05-08T22:34:24.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.982+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c1f2f40 con 0x7f623c0b19e0 2024-05-08T22:34:24.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c1f3740 con 0x7f623c0b19e0 2024-05-08T22:34:24.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62240038e0 con 0x7f623c0b19e0 2024-05-08T22:34:24.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f6238ff9700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 0x7f621c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:24.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6224083830 con 0x7f623c0b19e0 2024-05-08T22:34:24.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f623b7fe700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 0x7f621c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:24.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.986+0000 7f623b7fe700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 0x7f621c068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f622c009910 tx=0x7f622c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:24.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:24.994+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f623c1f3740 con 0x7f623c0b19e0 2024-05-08T22:34:25.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f623c1f3740 con 0x7f623c0b19e0 2024-05-08T22:34:25.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f6238ff9700 1 -- 172.21.15.179:0/1998933400 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f623c1f3740 con 0x7f623c0b19e0 2024-05-08T22:34:25.162 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 msgr2=0x7f621c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 0x7f621c068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f622c009910 tx=0x7f622c008030 comp rx=0 tx=0).stop 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 msgr2=0x7f623c159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f622400c500 tx=0x7f622400c810 comp rx=0 tx=0).stop 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 shutdown_connections 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f621c066730 0x7f621c068bf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f623c159700 0x7f623c157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f623c159320 0x7f623c157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 --2- 172.21.15.179:0/1998933400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f623c0b19e0 0x7f623c159e10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 >> 172.21.15.179:0/1998933400 conn(0x7f623c0cea30 msgr2=0x7f623c06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 shutdown_connections 2024-05-08T22:34:25.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.162+0000 7f624114d700 1 -- 172.21.15.179:0/1998933400 wait complete. 2024-05-08T22:34:25.177 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:25.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.298+0000 7f2c9eda5700 1 Processor -- start 2024-05-08T22:34:25.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.298+0000 7f2c9eda5700 1 -- start start 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c900dfe00 0x7f2c90050ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2c900505c0 0x7f2c90038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 0x7f2c900393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c90055b30 con 0x7f2c900dfe00 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c90054a90 con 0x7f2c900505c0 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90054d70 con 0x7f2c900e01e0 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c900dfe00 0x7f2c90050ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c900dfe00 0x7f2c90050ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49102/0 (socket says 172.21.15.179:49102) 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 -- 172.21.15.179:0/1705463918 learned_addr learned my addr 172.21.15.179:0/1705463918 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9e5a4700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 0x7f2c900393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9d5a2700 1 --2- 172.21.15.179:0/1705463918 >> v2:172.21.15.195:3300/0 conn(0x7f2c900505c0 0x7f2c90038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 -- 172.21.15.179:0/1705463918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 msgr2=0x7f2c900393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 0x7f2c900393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 -- 172.21.15.179:0/1705463918 >> v2:172.21.15.195:3300/0 conn(0x7f2c900505c0 msgr2=0x7f2c90038eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/1705463918 >> v2:172.21.15.195:3300/0 conn(0x7f2c900505c0 0x7f2c90038eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 -- 172.21.15.179:0/1705463918 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c90055050 con 0x7f2c900dfe00 2024-05-08T22:34:25.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9e5a4700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 0x7f2c900393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:25.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90050ca0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c90053b30 tx=0x7f2c88009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fd4c1fa12dc428 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9cda1700 1 -- 172.21.15.179:0/1705463918 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88011830 con 0x7f2c900dfe00 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9cda1700 1 -- 172.21.15.179:0/1705463918 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c88011990 con 0x7f2c900dfe00 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9cda1700 1 -- 172.21.15.179:0/1705463918 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88011d20 con 0x7f2c900dfe00 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- 172.21.15.179:0/1705463918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 msgr2=0x7f2c90050ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90050ca0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c90053b30 tx=0x7f2c88009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- 172.21.15.179:0/1705463918 shutdown_connections 2024-05-08T22:34:25.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900e01e0 0x7f2c900393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/1705463918 >> v2:172.21.15.195:3300/0 conn(0x7f2c900505c0 0x7f2c90038eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/1705463918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90050ca0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- 172.21.15.179:0/1705463918 >> 172.21.15.179:0/1705463918 conn(0x7f2c90055c80 msgr2=0x7f2c90056090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- 172.21.15.179:0/1705463918 shutdown_connections 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.302+0000 7f2c9eda5700 1 -- 172.21.15.179:0/1705463918 wait complete. 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 Processor -- start 2024-05-08T22:34:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 -- start start 2024-05-08T22:34:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 0x7f2c90006b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c900e01e0 0x7f2c900dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900e05f0 con 0x7f2c900dfe00 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900e3270 con 0x7f2c900e01e0 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9eda5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90178ed0 con 0x7f2c900505c0 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49104/0 (socket says 172.21.15.179:49104) 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 -- 172.21.15.179:0/3739162214 learned_addr learned my addr 172.21.15.179:0/3739162214 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 0x7f2c90006b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9e5a4700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c900e01e0 0x7f2c900dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 -- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 msgr2=0x7f2c90006b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 0x7f2c90006b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 -- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c900e01e0 msgr2=0x7f2c900dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c900e01e0 0x7f2c900dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 -- 172.21.15.179:0/3739162214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c88009570 con 0x7f2c900dfe00 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 0x7f2c90006b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.306+0000 7f2c9d5a2700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400e870 tx=0x7f2c9400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940095a0 con 0x7f2c900dfe00 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c94013680 con 0x7f2c900dfe00 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c900e0800 con 0x7f2c900dfe00 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940139f0 con 0x7f2c900dfe00 2024-05-08T22:34:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c900e2f00 con 0x7f2c900dfe00 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c90009b90 con 0x7f2c900dfe00 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c94013b50 con 0x7f2c900dfe00 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 0x7f2c7c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2c94084300 con 0x7f2c900dfe00 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 0x7f2c7c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.310+0000 7f2c9dda3700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 0x7f2c7c068c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000c00 tx=0x7f2c88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.318+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c940512c0 con 0x7f2c900dfe00 2024-05-08T22:34:25.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c900e3420 con 0x7f2c900dfe00 2024-05-08T22:34:25.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c8e7fc700 1 -- 172.21.15.179:0/3739162214 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c940548e0 con 0x7f2c900dfe00 2024-05-08T22:34:25.487 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:25.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 msgr2=0x7f2c7c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 0x7f2c7c068c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000c00 tx=0x7f2c88008030 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 msgr2=0x7f2c90008ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400e870 tx=0x7f2c9400eb80 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 shutdown_connections 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c7c066760 0x7f2c7c068c20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c900e01e0 0x7f2c900dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c900dfe00 0x7f2c90008ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 --2- 172.21.15.179:0/3739162214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2c900505c0 0x7f2c90006b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 >> 172.21.15.179:0/3739162214 conn(0x7f2c90055c80 msgr2=0x7f2c900df140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 shutdown_connections 2024-05-08T22:34:25.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.486+0000 7f2c9eda5700 1 -- 172.21.15.179:0/3739162214 wait complete. 2024-05-08T22:34:25.502 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:25.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.618+0000 7ff6465a1700 1 Processor -- start 2024-05-08T22:34:25.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.618+0000 7ff6465a1700 1 -- start start 2024-05-08T22:34:25.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff640159520 0x7ff6400c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff64006bfc0 0x7ff64006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6400c5d40 con 0x7ff64006b440 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64006ce60 con 0x7ff64006bfc0 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6400b1970 con 0x7ff640159520 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff64006bfc0 0x7ff64006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff64006bfc0 0x7ff64006d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49110/0 (socket says 172.21.15.179:49110) 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 -- 172.21.15.179:0/3450177978 learned_addr learned my addr 172.21.15.179:0/3450177978 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff64559f700 1 --2- 172.21.15.179:0/3450177978 >> v2:172.21.15.195:3300/0 conn(0x7ff640159520 0x7ff6400c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 -- 172.21.15.179:0/3450177978 >> v2:172.21.15.195:3300/0 conn(0x7ff640159520 msgr2=0x7ff6400c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 --2- 172.21.15.179:0/3450177978 >> v2:172.21.15.195:3300/0 conn(0x7ff640159520 0x7ff6400c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 -- 172.21.15.179:0/3450177978 >> v1:172.21.15.195:6789/0 conn(0x7ff64006b440 legacy=0x7ff64006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 -- 172.21.15.179:0/3450177978 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400b1c50 con 0x7ff64006bfc0 2024-05-08T22:34:25.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff645da0700 1 --2- 172.21.15.179:0/3450177978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006bfc0 0x7ff64006d5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff6400990a0 tx=0x7ff63800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ac9121c82a47d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff637fff700 1 -- 172.21.15.179:0/3450177978 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638010070 con 0x7ff64006bfc0 2024-05-08T22:34:25.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff637fff700 1 -- 172.21.15.179:0/3450177978 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800db10 con 0x7ff64006bfc0 2024-05-08T22:34:25.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff637fff700 1 -- 172.21.15.179:0/3450177978 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638016400 con 0x7ff64006bfc0 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- 172.21.15.179:0/3450177978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006bfc0 msgr2=0x7ff64006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 --2- 172.21.15.179:0/3450177978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006bfc0 0x7ff64006d5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff6400990a0 tx=0x7ff63800d430 comp rx=0 tx=0).stop 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- 172.21.15.179:0/3450177978 shutdown_connections 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 --2- 172.21.15.179:0/3450177978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006bfc0 0x7ff64006d5a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 --2- 172.21.15.179:0/3450177978 >> v2:172.21.15.195:3300/0 conn(0x7ff640159520 0x7ff6400c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- 172.21.15.179:0/3450177978 >> 172.21.15.179:0/3450177978 conn(0x7ff640098a70 msgr2=0x7ff6400cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- 172.21.15.179:0/3450177978 shutdown_connections 2024-05-08T22:34:25.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.622+0000 7ff6465a1700 1 -- 172.21.15.179:0/3450177978 wait complete. 2024-05-08T22:34:25.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 Processor -- start 2024-05-08T22:34:25.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 -- start start 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 0x7ff640156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff640159520 0x7ff64015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401e48c0 con 0x7ff64006b440 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401e1e50 con 0x7ff640159520 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff6465a1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401e2410 con 0x7ff64006bfc0 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.626+0000 7ff64559f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49120/0 (socket says 172.21.15.179:49120) 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 -- 172.21.15.179:0/1448428272 learned_addr learned my addr 172.21.15.179:0/1448428272 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff645da0700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff640159520 0x7ff64015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff644d9e700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 0x7ff640156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 -- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 msgr2=0x7ff640156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 0x7ff640156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 -- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff640159520 msgr2=0x7ff64015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff640159520 0x7ff64015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 -- 172.21.15.179:0/1448428272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63800bd40 con 0x7ff64006b440 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff644d9e700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 0x7ff640156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff64559f700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00d830 tx=0x7ff63c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0125f0 con 0x7ff64006b440 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c012750 con 0x7ff64006b440 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c012b00 con 0x7ff64006b440 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64015b850 con 0x7ff64006b440 2024-05-08T22:34:25.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401e20a0 con 0x7ff64006b440 2024-05-08T22:34:25.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff63c012c60 con 0x7ff64006b440 2024-05-08T22:34:25.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64004e700 con 0x7ff64006b440 2024-05-08T22:34:25.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 0x7ff624068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.630+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff63c0847c0 con 0x7ff64006b440 2024-05-08T22:34:25.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.634+0000 7ff644d9e700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 0x7ff624068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.634+0000 7ff644d9e700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 0x7ff624068c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6401f1c20 tx=0x7ff630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.638+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff63c051780 con 0x7ff64006b440 2024-05-08T22:34:25.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff64015c850 con 0x7ff64006b440 2024-05-08T22:34:25.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff635ffb700 1 -- 172.21.15.179:0/1448428272 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff63c054da0 con 0x7ff64006b440 2024-05-08T22:34:25.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 msgr2=0x7ff624068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 0x7ff624068c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6401f1c20 tx=0x7ff630008040 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 msgr2=0x7ff6401e4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00d830 tx=0x7ff63c00db40 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 shutdown_connections 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff624066760 0x7ff624068c20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff640159520 0x7ff64015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff64006bfc0 0x7ff640156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 --2- 172.21.15.179:0/1448428272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff64006b440 0x7ff6401e4380 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.822+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 >> 172.21.15.179:0/1448428272 conn(0x7ff640098a70 msgr2=0x7ff64006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.826+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 shutdown_connections 2024-05-08T22:34:25.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.826+0000 7ff6465a1700 1 -- 172.21.15.179:0/1448428272 wait complete. 2024-05-08T22:34:25.827 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:25.838 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi195.front.sepia.ceph.com... 2024-05-08T22:34:25.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:25.838 DEBUG:teuthology.orchestra.run.smithi195:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-08T22:34:25.855 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:25.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:34:25.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:25.855 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:34:25.863 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:34:25.863 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:25.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 Processor -- start 2024-05-08T22:34:25.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 -- start start 2024-05-08T22:34:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46c406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c4161ac0 con 0x7f46c40ce060 2024-05-08T22:34:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c40cdaa0 con 0x7f46c40cc310 2024-05-08T22:34:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c8dcf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40ae550 con 0x7f46c40add40 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f46c40ce060 0x7f46c40ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43186/0 (socket says 172.21.15.179:43186) 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 -- 172.21.15.179:0/3207827913 learned_addr learned my addr 172.21.15.179:0/3207827913 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c2d9d700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46c406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018066810 0 0) 0x7f46c4161ac0 con 0x7f46c40ce060 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac0035d0 con 0x7f46c40ce060 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260084866 0 0) 0x7f46c40cdaa0 con 0x7f46c40cc310 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c4161ac0 con 0x7f46c40cc310 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570624825 0 0) 0x7f46ac0035d0 con 0x7f46c40ce060 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 msgr2=0x7f46c406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46c406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 >> v1:172.21.15.195:6789/0 conn(0x7f46c40cc310 legacy=0x7f46c406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c406adc0 con 0x7f46c40ce060 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f46b8001670 con 0x7f46c40ce060 2024-05-08T22:34:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 >> v1:172.21.15.179:6789/0 conn(0x7f46c40ce060 legacy=0x7f46c40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46ac003be0 0x7f46ac003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46ac004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46ac004db0 0x7f46ac005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40ae550 con 0x7f46ac003be0 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00bae0 con 0x7f46ac004db0 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00bc50 con 0x7f46c40add40 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46ac003be0 0x7f46ac003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c1d9b700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46ac004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c2d9d700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46ac004db0 0x7f46ac005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 -- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 msgr2=0x7f46ac004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c40add40 0x7f46ac004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 -- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46ac004db0 msgr2=0x7f46ac005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46ac004db0 0x7f46ac005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 -- 172.21.15.179:0/3207827913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0035d0 con 0x7f46ac003be0 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c2d9d700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46ac004db0 0x7f46ac005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c259c700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46ac003be0 0x7f46ac003fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f46b8007fd0 tx=0x7f46ac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3997c35d4f1038f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800cd80 con 0x7f46ac003be0 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8001af0 con 0x7f46ac003be0 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.978+0000 7f46c159a700 1 -- 172.21.15.179:0/3207827913 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8005bc0 con 0x7f46ac003be0 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- 172.21.15.179:0/3207827913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46ac003be0 msgr2=0x7f46ac003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/3207827913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46ac003be0 0x7f46ac003fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f46b8007fd0 tx=0x7f46ac0058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c2d9d700 1 -- 172.21.15.179:0/3207827913 reap_dead start 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- 172.21.15.179:0/3207827913 shutdown_connections 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- 172.21.15.179:0/3207827913 >> 172.21.15.179:0/3207827913 conn(0x7f46c40cb660 msgr2=0x7f46c40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- 172.21.15.179:0/3207827913 shutdown_connections 2024-05-08T22:34:25.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- 172.21.15.179:0/3207827913 wait complete. 2024-05-08T22:34:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 Processor -- start 2024-05-08T22:34:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.982+0000 7f46c8dcf700 1 -- start start 2024-05-08T22:34:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c41fa990 0x7f46c41fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 0x7f46c40acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c414de00 con 0x7f46c40c4200 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c414d4a0 con 0x7f46c40ac860 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41fb2b0 con 0x7f46c41fa990 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49140/0 (socket says 172.21.15.179:49140) 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 -- 172.21.15.179:0/4100507209 learned_addr learned my addr 172.21.15.179:0/4100507209 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c2d9d700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 0x7f46c40acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c259c700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c41fa990 0x7f46c41fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 -- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c41fa990 msgr2=0x7f46c41fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c41fa990 0x7f46c41fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 -- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 msgr2=0x7f46c40acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 0x7f46c40acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 -- 172.21.15.179:0/4100507209 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40c4f80 con 0x7f46c40c4200 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c2d9d700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 0x7f46c40acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c1d9b700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e7b0 tx=0x7f46b400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052e0 con 0x7f46c40c4200 2024-05-08T22:34:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4005460 con 0x7f46c40c4200 2024-05-08T22:34:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c414d740 con 0x7f46c40c4200 2024-05-08T22:34:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40038e0 con 0x7f46c40c4200 2024-05-08T22:34:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40c4bc0 con 0x7f46c40c4200 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.986+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c4067770 con 0x7f46c40c4200 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.990+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46b4003a40 con 0x7f46c40c4200 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.990+0000 7f46b2ffd700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 0x7f46a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.990+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46b4084370 con 0x7f46c40c4200 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.990+0000 7f46c259c700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 0x7f46a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.990+0000 7f46c259c700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 0x7f46a4068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46c4067cc0 tx=0x7f46b800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:25.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:25.994+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f46b4051280 con 0x7f46c40c4200 2024-05-08T22:34:26.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.202+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46c414df40 con 0x7f46c40c4200 2024-05-08T22:34:26.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.206+0000 7f46b2ffd700 1 -- 172.21.15.179:0/4100507209 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f46c414df40 con 0x7f46c40c4200 2024-05-08T22:34:26.206 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:34:26.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 msgr2=0x7f46a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 0x7f46a4068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46c4067cc0 tx=0x7f46b800a000 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 msgr2=0x7f46c40c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e7b0 tx=0x7f46b400eb70 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 shutdown_connections 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46a4066730 0x7f46a4068bf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46c40ac860 0x7f46c40acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46c40c4200 0x7f46c40c4680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 --2- 172.21.15.179:0/4100507209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46c41fa990 0x7f46c41fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 >> 172.21.15.179:0/4100507209 conn(0x7f46c40cb660 msgr2=0x7f46c40ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 shutdown_connections 2024-05-08T22:34:26.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.210+0000 7f46c8dcf700 1 -- 172.21.15.179:0/4100507209 wait complete. 2024-05-08T22:34:26.212 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:34:26.228 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:26.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 Processor -- start 2024-05-08T22:34:26.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- start start 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f43e40ce060 0x7f43e406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f43e40add40 0x7f43e406b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e4161ac0 con 0x7f43e40cc310 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e40ae550 con 0x7f43e40ce060 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e406abb0 con 0x7f43e40add40 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f43e40ce060 0x7f43e406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e2ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f43e40cc310 0x7f43e40ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43188/0 (socket says 172.21.15.179:43188) 2024-05-08T22:34:26.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e2ffd700 1 -- 172.21.15.179:0/2230720032 learned_addr learned my addr 172.21.15.179:0/2230720032 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:26.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e27fc700 1 -- 172.21.15.179:0/2230720032 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695034050 0 0) 0x7f43e4161ac0 con 0x7f43e40cc310 2024-05-08T22:34:26.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e3fff700 1 --2- 172.21.15.179:0/2230720032 >> v2:172.21.15.195:3300/0 conn(0x7f43e40add40 0x7f43e406b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e27fc700 1 -- 172.21.15.179:0/2230720032 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d00035d0 con 0x7f43e40cc310 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 -- 172.21.15.179:0/2230720032 >> v2:172.21.15.195:3300/0 conn(0x7f43e40add40 msgr2=0x7f43e406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 --2- 172.21.15.179:0/2230720032 >> v2:172.21.15.195:3300/0 conn(0x7f43e40add40 0x7f43e406b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 -- 172.21.15.179:0/2230720032 >> v1:172.21.15.179:6789/0 conn(0x7f43e40cc310 legacy=0x7f43e40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 -- 172.21.15.179:0/2230720032 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40cdaa0 con 0x7f43e40ce060 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e37fe700 1 --2- 172.21.15.179:0/2230720032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40ce060 0x7f43e406ce50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f43e4067cc0 tx=0x7f43cc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a353c8a11616d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e27fc700 1 -- 172.21.15.179:0/2230720032 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc011830 con 0x7f43e40ce060 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e27fc700 1 -- 172.21.15.179:0/2230720032 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc011990 con 0x7f43e40ce060 2024-05-08T22:34:26.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e27fc700 1 -- 172.21.15.179:0/2230720032 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc011d00 con 0x7f43e40ce060 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- 172.21.15.179:0/2230720032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40ce060 msgr2=0x7f43e406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 --2- 172.21.15.179:0/2230720032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40ce060 0x7f43e406ce50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f43e4067cc0 tx=0x7f43cc009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- 172.21.15.179:0/2230720032 shutdown_connections 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 --2- 172.21.15.179:0/2230720032 >> v2:172.21.15.195:3300/0 conn(0x7f43e40add40 0x7f43e406b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 --2- 172.21.15.179:0/2230720032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40ce060 0x7f43e406ce50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- 172.21.15.179:0/2230720032 >> 172.21.15.179:0/2230720032 conn(0x7f43e40cb660 msgr2=0x7f43e40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:26.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.338+0000 7f43e9f03700 1 -- 172.21.15.179:0/2230720032 shutdown_connections 2024-05-08T22:34:26.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.342+0000 7f43e9f03700 1 -- 172.21.15.179:0/2230720032 wait complete. 2024-05-08T22:34:26.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.342+0000 7f43e9f03700 1 Processor -- start 2024-05-08T22:34:26.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.342+0000 7f43e9f03700 1 -- start start 2024-05-08T22:34:26.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 0x7f43e414e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 0x7f43e41f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40c47e0 con 0x7f43e40cc310 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40c3880 con 0x7f43e40add40 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40c3b60 con 0x7f43e40ce060 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49164/0 (socket says 172.21.15.179:49164) 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 -- 172.21.15.179:0/4236218375 learned_addr learned my addr 172.21.15.179:0/4236218375 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e37fe700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 0x7f43e414e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e3fff700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 0x7f43e41f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 -- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 msgr2=0x7f43e41f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 0x7f43e41f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 -- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 msgr2=0x7f43e414e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 0x7f43e414e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 -- 172.21.15.179:0/4236218375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0094b0 con 0x7f43e40cc310 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e37fe700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 0x7f43e414e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e3fff700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 0x7f43e41f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e2ffd700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00cb40 tx=0x7f43dc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00a9e0 con 0x7f43e40cc310 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00ab40 con 0x7f43e40cc310 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e40c3e40 con 0x7f43e40cc310 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00ce60 con 0x7f43e40cc310 2024-05-08T22:34:26.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e40c4400 con 0x7f43e40cc310 2024-05-08T22:34:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4067770 con 0x7f43e40cc310 2024-05-08T22:34:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.346+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43dc01b3b0 con 0x7f43e40cc310 2024-05-08T22:34:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.350+0000 7f43dbfff700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 0x7f43c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.350+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f43dc083fa0 con 0x7f43e40cc310 2024-05-08T22:34:26.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.350+0000 7f43e37fe700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 0x7f43c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.350+0000 7f43e37fe700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 0x7f43c4068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000c00 tx=0x7f43cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.358+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f43dc0b5cf0 con 0x7f43e40cc310 2024-05-08T22:34:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.530+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43e40c4920 con 0x7f43e40cc310 2024-05-08T22:34:26.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.534+0000 7f43dbfff700 1 -- 172.21.15.179:0/4236218375 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7f43e40c4920 con 0x7f43e40cc310 2024-05-08T22:34:26.536 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:26.536 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 msgr2=0x7f43c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 0x7f43c4068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000c00 tx=0x7f43cc008030 comp rx=0 tx=0).stop 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 msgr2=0x7f43e40b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00cb40 tx=0x7f43dc00cc20 comp rx=0 tx=0).stop 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 shutdown_connections 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f43c4066730 0x7f43c4068bf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43e40ce060 0x7f43e41f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43e40cc310 0x7f43e40b4160 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 --2- 172.21.15.179:0/4236218375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43e40add40 0x7f43e414e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 >> 172.21.15.179:0/4236218375 conn(0x7f43e40cb660 msgr2=0x7f43e40af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:26.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 shutdown_connections 2024-05-08T22:34:26.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.538+0000 7f43e9f03700 1 -- 172.21.15.179:0/4236218375 wait complete. 2024-05-08T22:34:26.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:33:12.850411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:33:13.884821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T22:33:16.947767+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T22:34:26.551 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:26.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.666+0000 7f48a3fff700 1 Processor -- start 2024-05-08T22:34:26.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.666+0000 7f48a3fff700 1 -- start start 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f48a4159320 0x7f48a40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f48a4159700 0x7f48a406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a406cda0 con 0x7f48a40b19e0 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40ccd40 con 0x7f48a4159700 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40b1f10 con 0x7f48a4159320 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a27fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f48a40b19e0 0x7f48a40cc630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43192/0 (socket says 172.21.15.179:43192) 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f48a4159700 0x7f48a406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a27fc700 1 -- 172.21.15.179:0/3112152629 learned_addr learned my addr 172.21.15.179:0/3112152629 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:26.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a2ffd700 1 --2- 172.21.15.179:0/3112152629 >> v2:172.21.15.195:3300/0 conn(0x7f48a4159320 0x7f48a40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 -- 172.21.15.179:0/3112152629 >> v2:172.21.15.195:3300/0 conn(0x7f48a4159320 msgr2=0x7f48a40cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 --2- 172.21.15.179:0/3112152629 >> v2:172.21.15.195:3300/0 conn(0x7f48a4159320 0x7f48a40cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 -- 172.21.15.179:0/3112152629 >> v1:172.21.15.179:6789/0 conn(0x7f48a40b19e0 legacy=0x7f48a40cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a1ffb700 1 -- 172.21.15.179:0/3112152629 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270065197 0 0) 0x7f48a406cda0 con 0x7f48a40b19e0 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 -- 172.21.15.179:0/3112152629 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40b21f0 con 0x7f48a4159700 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a37fe700 1 --2- 172.21.15.179:0/3112152629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a406b440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48a40720b0 tx=0x7f489c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e7e0c74b22499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a1ffb700 1 -- 172.21.15.179:0/3112152629 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c011830 con 0x7f48a4159700 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a1ffb700 1 -- 172.21.15.179:0/3112152629 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c011990 con 0x7f48a4159700 2024-05-08T22:34:26.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a1ffb700 1 -- 172.21.15.179:0/3112152629 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c011d20 con 0x7f48a4159700 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- 172.21.15.179:0/3112152629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 msgr2=0x7f48a406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3112152629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a406b440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48a40720b0 tx=0x7f489c009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- 172.21.15.179:0/3112152629 shutdown_connections 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3112152629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a406b440 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3112152629 >> v2:172.21.15.195:3300/0 conn(0x7f48a4159320 0x7f48a40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- 172.21.15.179:0/3112152629 >> 172.21.15.179:0/3112152629 conn(0x7f48a40cea30 msgr2=0x7f48a40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:26.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- 172.21.15.179:0/3112152629 shutdown_connections 2024-05-08T22:34:26.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.670+0000 7f48a3fff700 1 -- 172.21.15.179:0/3112152629 wait complete. 2024-05-08T22:34:26.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 Processor -- start 2024-05-08T22:34:26.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 -- start start 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48a40b19e0 0x7f48a4145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48a4159320 0x7f48a414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a4146390 con 0x7f48a4159700 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a415ba30 con 0x7f48a40b19e0 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c9630 con 0x7f48a4159320 2024-05-08T22:34:26.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49178/0 (socket says 172.21.15.179:49178) 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 -- 172.21.15.179:0/3530476369 learned_addr learned my addr 172.21.15.179:0/3530476369 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a2ffd700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48a40b19e0 0x7f48a4145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a27fc700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48a4159320 0x7f48a414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 -- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48a4159320 msgr2=0x7f48a414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48a4159320 0x7f48a414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 -- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48a40b19e0 msgr2=0x7f48a4145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48a40b19e0 0x7f48a4145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 -- 172.21.15.179:0/3530476369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0094b0 con 0x7f48a4159700 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a37fe700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f489c0090b0 tx=0x7f489c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c004590 con 0x7f48a4159700 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c01e070 con 0x7f48a4159700 2024-05-08T22:34:26.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a40c8a20 con 0x7f48a4159700 2024-05-08T22:34:26.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.674+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c003900 con 0x7f48a4159700 2024-05-08T22:34:26.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a40c8fe0 con 0x7f48a4159700 2024-05-08T22:34:26.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a40c97a0 con 0x7f48a4159700 2024-05-08T22:34:26.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f489c003a60 con 0x7f48a4159700 2024-05-08T22:34:26.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f489bfff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 0x7f4884068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:26.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f489c085360 con 0x7f48a4159700 2024-05-08T22:34:26.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.678+0000 7f48a2ffd700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 0x7f4884068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:26.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.682+0000 7f48a2ffd700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 0x7f4884068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48940113e0 tx=0x7f489400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:26.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.686+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f489c052280 con 0x7f48a4159700 2024-05-08T22:34:26.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.858+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48a404e700 con 0x7f48a4159700 2024-05-08T22:34:26.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.858+0000 7f489bfff700 1 -- 172.21.15.179:0/3530476369 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f489c0558a0 con 0x7f48a4159700 2024-05-08T22:34:26.861 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 msgr2=0x7f4884068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 0x7f4884068bf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48940113e0 tx=0x7f489400b040 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 msgr2=0x7f48a414f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f489c0090b0 tx=0x7f489c004770 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 shutdown_connections 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4884066730 0x7f4884068bf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a4159700 0x7f48a414f140 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48a4159320 0x7f48a414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 --2- 172.21.15.179:0/3530476369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48a40b19e0 0x7f48a4145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:26.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 >> 172.21.15.179:0/3530476369 conn(0x7f48a40cea30 msgr2=0x7f48a415d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:26.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 shutdown_connections 2024-05-08T22:34:26.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.862+0000 7f48a3fff700 1 -- 172.21.15.179:0/3530476369 wait complete. 2024-05-08T22:34:26.876 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 Processor -- start 2024-05-08T22:34:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 -- start start 2024-05-08T22:34:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af815d0c0 0x7f9af80b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9af80ce2b0 0x7f9af806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af815d4a0 con 0x7f9af80d1d20 2024-05-08T22:34:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af806abb0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afefa8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80d2500 con 0x7f9af815d0c0 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afe7a7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9af80ce2b0 0x7f9af806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:26.998+0000 7f9afdfa6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af815d0c0 0x7f9af80b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9af80ce2b0 0x7f9af806b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:49498/0 (socket says 172.21.15.179:49498) 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 -- 172.21.15.179:0/2386970255 learned_addr learned my addr 172.21.15.179:0/2386970255 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 -- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af815d0c0 msgr2=0x7f9af80b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 --2- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af815d0c0 0x7f9af80b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 -- 172.21.15.179:0/2386970255 >> v1:172.21.15.195:6789/0 conn(0x7f9af80d1d20 legacy=0x7f9af80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 -- 172.21.15.179:0/2386970255 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80d27e0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afcfa4700 1 -- 172.21.15.179:0/2386970255 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953305618 0 0) 0x7f9af815d4a0 con 0x7f9af80d1d20 2024-05-08T22:34:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afe7a7700 1 --2- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af80ce2b0 0x7f9af806b370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009910 tx=0x7f9af0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9953e1ed3ede3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afcfa4700 1 -- 172.21.15.179:0/2386970255 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011830 con 0x7f9af80ce2b0 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afcfa4700 1 -- 172.21.15.179:0/2386970255 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0011990 con 0x7f9af80ce2b0 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afcfa4700 1 -- 172.21.15.179:0/2386970255 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011d40 con 0x7f9af80ce2b0 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 -- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af80ce2b0 msgr2=0x7f9af806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af80ce2b0 0x7f9af806b370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009910 tx=0x7f9af0009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 -- 172.21.15.179:0/2386970255 shutdown_connections 2024-05-08T22:34:27.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af80ce2b0 0x7f9af806b370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2386970255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af815d0c0 0x7f9af80b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 -- 172.21.15.179:0/2386970255 >> 172.21.15.179:0/2386970255 conn(0x7f9af80d2ed0 msgr2=0x7f9af80d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 -- 172.21.15.179:0/2386970255 shutdown_connections 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.002+0000 7f9afefa8700 1 -- 172.21.15.179:0/2386970255 wait complete. 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 Processor -- start 2024-05-08T22:34:27.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- start start 2024-05-08T22:34:27.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af80d1d20 0x7f9af8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 0x7f9af8157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80cb090 con 0x7f9af80ce2b0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80ca6f0 con 0x7f9af815d0c0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80ca9d0 con 0x7f9af80d1d20 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49182/0 (socket says 172.21.15.179:49182) 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 -- 172.21.15.179:0/2025226112 learned_addr learned my addr 172.21.15.179:0/2025226112 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afe7a7700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 0x7f9af8157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afd7a5700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af80d1d20 0x7f9af8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 -- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af80d1d20 msgr2=0x7f9af8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af80d1d20 0x7f9af8157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 -- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 msgr2=0x7f9af8157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 0x7f9af8157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 -- 172.21.15.179:0/2025226112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0009570 con 0x7f9af80ce2b0 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afe7a7700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 0x7f9af8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:27.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afdfa6700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800e840 tx=0x7f9ae800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80095a0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af80cacb0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8013680 con 0x7f9af80ce2b0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80139f0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.006+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8155610 con 0x7f9af80ce2b0 2024-05-08T22:34:27.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.010+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80b7cf0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.010+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ae8013b50 con 0x7f9af80ce2b0 2024-05-08T22:34:27.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.010+0000 7f9aee7fc700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 0x7f9adc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.010+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ae80843a0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.010+0000 7f9afd7a5700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 0x7f9adc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.014+0000 7f9afd7a5700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 0x7f9adc068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9af4002230 tx=0x7f9af4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.018+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9ae80512c0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.190+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9af804e700 con 0x7f9af80ce2b0 2024-05-08T22:34:27.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.190+0000 7f9aee7fc700 1 -- 172.21.15.179:0/2025226112 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ae80548e0 con 0x7f9af80ce2b0 2024-05-08T22:34:27.192 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 msgr2=0x7f9adc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 0x7f9adc068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9af4002230 tx=0x7f9af4009040 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 msgr2=0x7f9af80c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800e840 tx=0x7f9ae800eb50 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 shutdown_connections 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9adc066730 0x7f9adc068bf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9af815d0c0 0x7f9af8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9af80d1d20 0x7f9af8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 --2- 172.21.15.179:0/2025226112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af80ce2b0 0x7f9af80c83d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 >> 172.21.15.179:0/2025226112 conn(0x7f9af80d2ed0 msgr2=0x7f9af80b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 shutdown_connections 2024-05-08T22:34:27.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.194+0000 7f9afefa8700 1 -- 172.21.15.179:0/2025226112 wait complete. 2024-05-08T22:34:27.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:27.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 Processor -- start 2024-05-08T22:34:27.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 -- start start 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f58700b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587015dce0 con 0x7f58700b3480 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587015dfc0 con 0x7f58700b24d0 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f5877a8f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587006b5e0 con 0x7f587015d700 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f587582b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f58700b3480 0x7f58700b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43202/0 (socket says 172.21.15.179:43202) 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.326+0000 7f587582b700 1 -- 172.21.15.179:0/934541801 learned_addr learned my addr 172.21.15.179:0/934541801 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f587502a700 1 --2- 172.21.15.179:0/934541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f58700b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5874829700 1 -- 172.21.15.179:0/934541801 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581876348 0 0) 0x7f587015dce0 con 0x7f58700b3480 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5874829700 1 -- 172.21.15.179:0/934541801 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c0035d0 con 0x7f58700b3480 2024-05-08T22:34:27.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f587502a700 1 -- 172.21.15.179:0/934541801 >> v1:172.21.15.195:6789/0 conn(0x7f58700b24d0 legacy=0x7f587006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f587502a700 1 -- 172.21.15.179:0/934541801 >> v1:172.21.15.179:6789/0 conn(0x7f58700b3480 legacy=0x7f58700b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f587502a700 1 -- 172.21.15.179:0/934541801 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587015e2a0 con 0x7f587015d700 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f587502a700 1 --2- 172.21.15.179:0/934541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f58700b1f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f586c009880 tx=0x7f586c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96912e2dd0c69fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5874829700 1 -- 172.21.15.179:0/934541801 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011830 con 0x7f587015d700 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5874829700 1 -- 172.21.15.179:0/934541801 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c011990 con 0x7f587015d700 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5874829700 1 -- 172.21.15.179:0/934541801 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c011d20 con 0x7f587015d700 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 -- 172.21.15.179:0/934541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 msgr2=0x7f58700b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 --2- 172.21.15.179:0/934541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f58700b1f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f586c009880 tx=0x7f586c009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 -- 172.21.15.179:0/934541801 shutdown_connections 2024-05-08T22:34:27.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 --2- 172.21.15.179:0/934541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f58700b1f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 -- 172.21.15.179:0/934541801 >> 172.21.15.179:0/934541801 conn(0x7f58700ceca0 msgr2=0x7f58700cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 -- 172.21.15.179:0/934541801 shutdown_connections 2024-05-08T22:34:27.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.330+0000 7f5877a8f700 1 -- 172.21.15.179:0/934541801 wait complete. 2024-05-08T22:34:27.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 Processor -- start 2024-05-08T22:34:27.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- start start 2024-05-08T22:34:27.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58700b3480 0x7f5870150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f587015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587015ab20 con 0x7f58700b24d0 2024-05-08T22:34:27.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587014ed20 con 0x7f58700b3480 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587014ea50 con 0x7f587015d700 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49190/0 (socket says 172.21.15.179:49190) 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 -- 172.21.15.179:0/2920780426 learned_addr learned my addr 172.21.15.179:0/2920780426 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587602c700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f587015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587502a700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58700b3480 0x7f5870150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 -- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 msgr2=0x7f587015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f587015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 -- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58700b3480 msgr2=0x7f5870150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58700b3480 0x7f5870150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 -- 172.21.15.179:0/2920780426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c0094b0 con 0x7f58700b24d0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f587582b700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f586000c700 tx=0x7f586000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860005d90 con 0x7f58700b24d0 2024-05-08T22:34:27.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600070d0 con 0x7f58700b24d0 2024-05-08T22:34:27.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587015ad00 con 0x7f58700b24d0 2024-05-08T22:34:27.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860007480 con 0x7f58700b24d0 2024-05-08T22:34:27.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.334+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587015b2c0 con 0x7f58700b24d0 2024-05-08T22:34:27.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870073290 con 0x7f58700b24d0 2024-05-08T22:34:27.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58600075e0 con 0x7f58700b24d0 2024-05-08T22:34:27.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f58667fc700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 0x7f5854068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5860083a10 con 0x7f58700b24d0 2024-05-08T22:34:27.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f587502a700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 0x7f5854068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.338+0000 7f587502a700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 0x7f5854068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f586c0090b0 tx=0x7f586c010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.346+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5860050920 con 0x7f58700b24d0 2024-05-08T22:34:27.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.514+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f587015d140 con 0x7f58700b24d0 2024-05-08T22:34:27.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f58667fc700 1 -- 172.21.15.179:0/2920780426 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5860053f40 con 0x7f58700b24d0 2024-05-08T22:34:27.518 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:27.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 msgr2=0x7f5854068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 0x7f5854068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f586c0090b0 tx=0x7f586c010000 comp rx=0 tx=0).stop 2024-05-08T22:34:27.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 msgr2=0x7f587014e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f586000c700 tx=0x7f586000ca10 comp rx=0 tx=0).stop 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 shutdown_connections 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5854066730 0x7f5854068bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58700b3480 0x7f5870150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f587015d700 0x7f587015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 --2- 172.21.15.179:0/2920780426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58700b24d0 0x7f587014e510 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 >> 172.21.15.179:0/2920780426 conn(0x7f58700ceca0 msgr2=0x7f5870070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 shutdown_connections 2024-05-08T22:34:27.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.518+0000 7f5877a8f700 1 -- 172.21.15.179:0/2920780426 wait complete. 2024-05-08T22:34:27.533 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T22:34:27.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.650+0000 7f8d3ddb3700 1 Processor -- start 2024-05-08T22:34:27.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.650+0000 7f8d3ddb3700 1 -- start start 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8d3815a300 0x7f8d3815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d380b4650 con 0x7f8d381544e0 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d380b4320 con 0x7f8d3815acc0 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380cd6e0 con 0x7f8d3815a300 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3cdb1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8d381544e0 0x7f8d3806ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43218/0 (socket says 172.21.15.179:43218) 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3cdb1700 1 -- 172.21.15.179:0/1431501321 learned_addr learned my addr 172.21.15.179:0/1431501321 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:27.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d37fff700 1 --2- 172.21.15.179:0/1431501321 >> v2:172.21.15.195:3300/0 conn(0x7f8d3815a300 0x7f8d3815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928585437 0 0) 0x7f8d380b4650 con 0x7f8d381544e0 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d180035d0 con 0x7f8d381544e0 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d37fff700 1 -- 172.21.15.179:0/1431501321 >> v1:172.21.15.195:6789/0 conn(0x7f8d3815acc0 legacy=0x7f8d380b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653137731 0 0) 0x7f8d380b4320 con 0x7f8d3815acc0 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d37fff700 1 -- 172.21.15.179:0/1431501321 >> v1:172.21.15.179:6789/0 conn(0x7f8d381544e0 legacy=0x7f8d3806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d37fff700 1 -- 172.21.15.179:0/1431501321 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3806d2b0 con 0x7f8d3815a300 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d37fff700 1 --2- 172.21.15.179:0/1431501321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d3815a300 0x7f8d3815a780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009850 tx=0x7f8d28009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efdd60ff1ea44ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28011830 con 0x7f8d3815a300 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28011990 con 0x7f8d3815a300 2024-05-08T22:34:27.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d377fe700 1 -- 172.21.15.179:0/1431501321 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28011d40 con 0x7f8d3815a300 2024-05-08T22:34:27.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/1431501321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d3815a300 msgr2=0x7f8d3815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/1431501321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d3815a300 0x7f8d3815a780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009850 tx=0x7f8d28009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:27.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/1431501321 shutdown_connections 2024-05-08T22:34:27.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/1431501321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d3815a300 0x7f8d3815a780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/1431501321 >> 172.21.15.179:0/1431501321 conn(0x7f8d380b18d0 msgr2=0x7f8d380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/1431501321 shutdown_connections 2024-05-08T22:34:27.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.654+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/1431501321 wait complete. 2024-05-08T22:34:27.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 Processor -- start 2024-05-08T22:34:27.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 -- start start 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d381544e0 0x7f8d38145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d3815a300 0x7f8d38155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38155670 con 0x7f8d3815acc0 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38154a60 con 0x7f8d381544e0 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3ddb3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38154d40 con 0x7f8d3815a300 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3d5b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3d5b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49202/0 (socket says 172.21.15.179:49202) 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d37fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d3815a300 0x7f8d38155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3d5b2700 1 -- 172.21.15.179:0/2932625244 learned_addr learned my addr 172.21.15.179:0/2932625244 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.658+0000 7f8d3cdb1700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d381544e0 0x7f8d38145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 -- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d3815a300 msgr2=0x7f8d38155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d3815a300 0x7f8d38155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 -- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d381544e0 msgr2=0x7f8d38145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d381544e0 0x7f8d38145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 -- 172.21.15.179:0/2932625244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280094b0 con 0x7f8d3815acc0 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3d5b2700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8d20007fd0 tx=0x7f8d2000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200054e0 con 0x7f8d3815acc0 2024-05-08T22:34:27.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2000ce30 con 0x7f8d3815acc0 2024-05-08T22:34:27.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200038e0 con 0x7f8d3815acc0 2024-05-08T22:34:27.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d38155020 con 0x7f8d3815acc0 2024-05-08T22:34:27.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d38156040 con 0x7f8d3815acc0 2024-05-08T22:34:27.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d381557e0 con 0x7f8d3815acc0 2024-05-08T22:34:27.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.662+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d20003a40 con 0x7f8d3815acc0 2024-05-08T22:34:27.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.666+0000 7f8d357fa700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 0x7f8d1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.666+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8d20083890 con 0x7f8d3815acc0 2024-05-08T22:34:27.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.666+0000 7f8d3cdb1700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 0x7f8d1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.666+0000 7f8d3cdb1700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 0x7f8d1c068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005cb0 tx=0x7f8d30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.670+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8d20050840 con 0x7f8d3815acc0 2024-05-08T22:34:27.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.838+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8d3804e700 con 0x7f8d3815acc0 2024-05-08T22:34:27.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.838+0000 7f8d357fa700 1 -- 172.21.15.179:0/2932625244 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8d200170a0 con 0x7f8d3815acc0 2024-05-08T22:34:27.841 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 msgr2=0x7f8d1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 0x7f8d1c068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005cb0 tx=0x7f8d30009040 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 msgr2=0x7f8d380b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8d20007fd0 tx=0x7f8d2000c730 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 shutdown_connections 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d1c066730 0x7f8d1c068bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d3815acc0 0x7f8d380b1020 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d3815a300 0x7f8d38155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 --2- 172.21.15.179:0/2932625244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d381544e0 0x7f8d38145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 >> 172.21.15.179:0/2932625244 conn(0x7f8d380b18d0 msgr2=0x7f8d381586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 shutdown_connections 2024-05-08T22:34:27.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.842+0000 7f8d3ddb3700 1 -- 172.21.15.179:0/2932625244 wait complete. 2024-05-08T22:34:27.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 Processor -- start 2024-05-08T22:34:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 -- start start 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3f80cc700 0x7fc3f80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 0x7fc3f80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f806abb0 con 0x7fc3f80cb510 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80cb980 con 0x7fc3f80cc700 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fe60e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f806ae90 con 0x7fc3f815b1b0 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3f80cc700 0x7fc3f80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fce0b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 0x7fc3f80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3f80cc700 0x7fc3f80ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49214/0 (socket says 172.21.15.179:49214) 2024-05-08T22:34:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 -- 172.21.15.179:0/448138328 learned_addr learned my addr 172.21.15.179:0/448138328 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 -- 172.21.15.179:0/448138328 >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 msgr2=0x7fc3f80cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 --2- 172.21.15.179:0/448138328 >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 0x7fc3f80cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 -- 172.21.15.179:0/448138328 >> v1:172.21.15.195:6789/0 conn(0x7fc3f80cb510 legacy=0x7fc3f815e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 -- 172.21.15.179:0/448138328 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f815b9c0 con 0x7fc3f80cc700 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fce0b700 1 --2- 172.21.15.179:0/448138328 >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 0x7fc3f80cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3fd60c700 1 --2- 172.21.15.179:0/448138328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cc700 0x7fc3f80ce380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80b3e50 tx=0x7fc3f0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0b99d60913ad6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3f7fff700 1 -- 172.21.15.179:0/448138328 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0011830 con 0x7fc3f80cc700 2024-05-08T22:34:27.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.970+0000 7fc3f7fff700 1 -- 172.21.15.179:0/448138328 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f0011990 con 0x7fc3f80cc700 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3f7fff700 1 -- 172.21.15.179:0/448138328 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0011d20 con 0x7fc3f80cc700 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- 172.21.15.179:0/448138328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cc700 msgr2=0x7fc3f80ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/448138328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cc700 0x7fc3f80ce380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80b3e50 tx=0x7fc3f0009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- 172.21.15.179:0/448138328 shutdown_connections 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/448138328 >> v2:172.21.15.195:3300/0 conn(0x7fc3f815b1b0 0x7fc3f80cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/448138328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cc700 0x7fc3f80ce380 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- 172.21.15.179:0/448138328 >> 172.21.15.179:0/448138328 conn(0x7fc3f80ce9d0 msgr2=0x7fc3f80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- 172.21.15.179:0/448138328 shutdown_connections 2024-05-08T22:34:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- 172.21.15.179:0/448138328 wait complete. 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 Processor -- start 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- start start 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 0x7fc3f8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3f815b1b0 0x7fc3f815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f0e20 con 0x7fc3f80cb510 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f80ab620 con 0x7fc3f815b1b0 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fe60e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f0ae0 con 0x7fc3f80cc700 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49224/0 (socket says 172.21.15.179:49224) 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 -- 172.21.15.179:0/2491790262 learned_addr learned my addr 172.21.15.179:0/2491790262 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fde0d700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3f815b1b0 0x7fc3f815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fce0b700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 0x7fc3f8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 -- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 msgr2=0x7fc3f8151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 0x7fc3f8151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 -- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3f815b1b0 msgr2=0x7fc3f815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3f815b1b0 0x7fc3f815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 -- 172.21.15.179:0/2491790262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009a70 con 0x7fc3f80cb510 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fce0b700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 0x7fc3f8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.974+0000 7fc3fd60c700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0004510 con 0x7fc3f80cb510 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f001e070 con 0x7fc3f80cb510 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f0009570 con 0x7fc3f80cb510 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00038c0 con 0x7fc3f80cb510 2024-05-08T22:34:27.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f80ab060 con 0x7fc3f80cb510 2024-05-08T22:34:27.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f8067740 con 0x7fc3f80cb510 2024-05-08T22:34:27.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3f0003a20 con 0x7fc3f80cb510 2024-05-08T22:34:27.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 0x7fc3e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:27.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3f0020070 con 0x7fc3f80cb510 2024-05-08T22:34:27.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.978+0000 7fc3fce0b700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 0x7fc3e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:27.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.982+0000 7fc3fce0b700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 0x7fc3e4068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806dca0 tx=0x7fc3ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:27.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:27.986+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc3f00529b0 con 0x7fc3f80cb510 2024-05-08T22:34:28.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f804e700 con 0x7fc3f80cb510 2024-05-08T22:34:28.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3f5ffb700 1 -- 172.21.15.179:0/2491790262 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7fc3f0055fd0 con 0x7fc3f80cb510 2024-05-08T22:34:28.195 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:28.195 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:34:28.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 msgr2=0x7fc3e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 0x7fc3e4068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3f806dca0 tx=0x7fc3ec00b040 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 msgr2=0x7fc3f80aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f0004770 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 shutdown_connections 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc3e4066730 0x7fc3e4068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3f815b1b0 0x7fc3f815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3f80cc700 0x7fc3f8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 --2- 172.21.15.179:0/2491790262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f80cb510 0x7fc3f80aa530 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.194+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 >> 172.21.15.179:0/2491790262 conn(0x7fc3f80ce9d0 msgr2=0x7fc3f80ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.198+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 shutdown_connections 2024-05-08T22:34:28.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.198+0000 7fc3fe60e700 1 -- 172.21.15.179:0/2491790262 wait complete. 2024-05-08T22:34:28.199 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:34:28.210 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:28.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.326+0000 7fa158c71700 1 Processor -- start 2024-05-08T22:34:28.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.326+0000 7fa158c71700 1 -- start start 2024-05-08T22:34:28.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa1540cb6e0 0x7fa154165b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15406abb0 con 0x7fa1540cc700 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15406ae90 con 0x7fa1540ce380 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540ce800 con 0x7fa1540cb6e0 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1537fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa1540cc700 0x7fa15415b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43222/0 (socket says 172.21.15.179:43222) 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1537fe700 1 -- 172.21.15.179:0/1387505999 learned_addr learned my addr 172.21.15.179:0/1387505999 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa153fff700 1 --2- 172.21.15.179:0/1387505999 >> v2:172.21.15.195:3300/0 conn(0x7fa1540cb6e0 0x7fa154165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1527fc700 1 -- 172.21.15.179:0/1387505999 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98268296 0 0) 0x7fa15406abb0 con 0x7fa1540cc700 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1527fc700 1 -- 172.21.15.179:0/1387505999 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1400035d0 con 0x7fa1540cc700 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa153fff700 1 -- 172.21.15.179:0/1387505999 >> v1:172.21.15.195:6789/0 conn(0x7fa1540ce380 legacy=0x7fa1540cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa153fff700 1 -- 172.21.15.179:0/1387505999 >> v1:172.21.15.179:6789/0 conn(0x7fa1540cc700 legacy=0x7fa15415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa153fff700 1 -- 172.21.15.179:0/1387505999 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15415b9c0 con 0x7fa1540cb6e0 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa153fff700 1 --2- 172.21.15.179:0/1387505999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154165b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009d90 tx=0x7fa14c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65af30510a5f1af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1527fc700 1 -- 172.21.15.179:0/1387505999 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00e070 con 0x7fa1540cb6e0 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1527fc700 1 -- 172.21.15.179:0/1387505999 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c003a40 con 0x7fa1540cb6e0 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa1527fc700 1 -- 172.21.15.179:0/1387505999 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144001670 con 0x7fa1540cb6e0 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- 172.21.15.179:0/1387505999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 msgr2=0x7fa154165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 --2- 172.21.15.179:0/1387505999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154165b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009d90 tx=0x7fa14c00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- 172.21.15.179:0/1387505999 shutdown_connections 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 --2- 172.21.15.179:0/1387505999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154165b60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- 172.21.15.179:0/1387505999 >> 172.21.15.179:0/1387505999 conn(0x7fa1540ce9d0 msgr2=0x7fa1540cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- 172.21.15.179:0/1387505999 shutdown_connections 2024-05-08T22:34:28.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.330+0000 7fa158c71700 1 -- 172.21.15.179:0/1387505999 wait complete. 2024-05-08T22:34:28.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 Processor -- start 2024-05-08T22:34:28.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- start start 2024-05-08T22:34:28.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1540ce380 0x7fa15414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541498a0 con 0x7fa1540cc700 2024-05-08T22:34:28.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa154148c90 con 0x7fa1540cb6e0 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154148f70 con 0x7fa1540ce380 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49230/0 (socket says 172.21.15.179:49230) 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 -- 172.21.15.179:0/4055103080 learned_addr learned my addr 172.21.15.179:0/4055103080 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa1537fe700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa153fff700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1540ce380 0x7fa15414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 -- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1540ce380 msgr2=0x7fa15414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1540ce380 0x7fa15414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 -- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 msgr2=0x7fa154147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154147350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 -- 172.21.15.179:0/4055103080 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c009a70 con 0x7fa1540cc700 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa152ffd700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c410 tx=0x7fa13c00c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0052d0 con 0x7fa1540cc700 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c005430 con 0x7fa1540cc700 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa154149250 con 0x7fa1540cc700 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0055a0 con 0x7fa1540cc700 2024-05-08T22:34:28.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.334+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa154147890 con 0x7fa1540cc700 2024-05-08T22:34:28.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa154067740 con 0x7fa1540cc700 2024-05-08T22:34:28.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa13c0036a0 con 0x7fa1540cc700 2024-05-08T22:34:28.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa14bfff700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 0x7fa134068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa13c083630 con 0x7fa1540cc700 2024-05-08T22:34:28.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa1537fe700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 0x7fa134068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.338+0000 7fa1537fe700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 0x7fa134068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa144001eb0 tx=0x7fa1440018c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.346+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa13c0505e0 con 0x7fa1540cc700 2024-05-08T22:34:28.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.514+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa15404e700 con 0x7fa1540cc700 2024-05-08T22:34:28.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa14bfff700 1 -- 172.21.15.179:0/4055103080 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7fa13c053c00 con 0x7fa1540cc700 2024-05-08T22:34:28.518 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:28.518 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 msgr2=0x7fa134068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 0x7fa134068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa144001eb0 tx=0x7fa1440018c0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 msgr2=0x7fa154149d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c410 tx=0x7fa13c00c720 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 shutdown_connections 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa134066730 0x7fa134068bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1540ce380 0x7fa15414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1540cc700 0x7fa154149d30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 --2- 172.21.15.179:0/4055103080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1540cb6e0 0x7fa154147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 >> 172.21.15.179:0/4055103080 conn(0x7fa1540ce9d0 msgr2=0x7fa15415b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 shutdown_connections 2024-05-08T22:34:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.518+0000 7fa158c71700 1 -- 172.21.15.179:0/4055103080 wait complete. 2024-05-08T22:34:28.537 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:34:28.538 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:28.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 Processor -- start 2024-05-08T22:34:28.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 -- start start 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 0x7f071406bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071406afc0 con 0x7f071415d5b0 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071415d990 con 0x7f07140cde70 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071a33f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07140d26f0 con 0x7f07140ce2d0 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071933d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f071415d5b0 0x7f07140b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43236/0 (socket says 172.21.15.179:43236) 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f071933d700 1 -- 172.21.15.179:0/873477016 learned_addr learned my addr 172.21.15.179:0/873477016 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:28.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f0719b3e700 1 --2- 172.21.15.179:0/873477016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 0x7f071406bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559351926 0 0) 0x7f071406afc0 con 0x7f071415d5b0 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07000035d0 con 0x7f071415d5b0 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863087298 0 0) 0x7f071415d990 con 0x7f07140cde70 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071406afc0 con 0x7f07140cde70 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f0719b3e700 1 -- 172.21.15.179:0/873477016 >> v1:172.21.15.195:6789/0 conn(0x7f07140cde70 legacy=0x7f071406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f0719b3e700 1 -- 172.21.15.179:0/873477016 >> v1:172.21.15.179:6789/0 conn(0x7f071415d5b0 legacy=0x7f07140b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024792461 0 0) 0x7f07000035d0 con 0x7f071415d5b0 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.650+0000 7f0719b3e700 1 -- 172.21.15.179:0/873477016 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140d2130 con 0x7f07140ce2d0 2024-05-08T22:34:28.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f0719b3e700 1 --2- 172.21.15.179:0/873477016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 0x7f071406bb60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f070c009dc0 tx=0x7f070c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4afd7ccd64248f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00e070 con 0x7f07140ce2d0 2024-05-08T22:34:28.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c003a40 con 0x7f07140ce2d0 2024-05-08T22:34:28.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f070bfff700 1 -- 172.21.15.179:0/873477016 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710001670 con 0x7f07140ce2d0 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- 172.21.15.179:0/873477016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 msgr2=0x7f071406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 --2- 172.21.15.179:0/873477016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 0x7f071406bb60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f070c009dc0 tx=0x7f070c00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- 172.21.15.179:0/873477016 shutdown_connections 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 --2- 172.21.15.179:0/873477016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140ce2d0 0x7f071406bb60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- 172.21.15.179:0/873477016 >> 172.21.15.179:0/873477016 conn(0x7f07140b4940 msgr2=0x7f07140b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- 172.21.15.179:0/873477016 shutdown_connections 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- 172.21.15.179:0/873477016 wait complete. 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 Processor -- start 2024-05-08T22:34:28.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.654+0000 7f071a33f700 1 -- start start 2024-05-08T22:34:28.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140cde70 0x7f0714153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07140ce2d0 0x7f071415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07140cb4d0 con 0x7f071415d5b0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07141f2c20 con 0x7f07140ce2d0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07141565a0 con 0x7f07140cde70 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49240/0 (socket says 172.21.15.179:49240) 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 -- 172.21.15.179:0/3898539539 learned_addr learned my addr 172.21.15.179:0/3898539539 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071933d700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140cde70 0x7f0714153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0718b3c700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07140ce2d0 0x7f071415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 -- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140cde70 msgr2=0x7f0714153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140cde70 0x7f0714153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 -- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07140ce2d0 msgr2=0x7f071415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07140ce2d0 0x7f071415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 -- 172.21.15.179:0/3898539539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c009a70 con 0x7f071415d5b0 2024-05-08T22:34:28.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0719b3e700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f070c009d90 tx=0x7f070c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0045c0 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c004740 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0714153870 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c01d470 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07140cae80 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.658+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07140cb680 con 0x7f071415d5b0 2024-05-08T22:34:28.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.662+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f070c01d5d0 con 0x7f071415d5b0 2024-05-08T22:34:28.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.662+0000 7f0709ffb700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 0x7f06f8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:28.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.662+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f070c085b90 con 0x7f071415d5b0 2024-05-08T22:34:28.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.662+0000 7f071933d700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 0x7f06f8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:28.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.662+0000 7f071933d700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 0x7f06f8068c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0710000f60 tx=0x7f0710001c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:28.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.670+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f07140cb680 con 0x7f071415d5b0 2024-05-08T22:34:28.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.874+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f071404e700 con 0x7f071415d5b0 2024-05-08T22:34:28.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.874+0000 7f0709ffb700 1 -- 172.21.15.179:0/3898539539 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f070c016090 con 0x7f071415d5b0 2024-05-08T22:34:28.875 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:28.876 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 msgr2=0x7f06f8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 0x7f06f8068c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0710000f60 tx=0x7f0710001c00 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 msgr2=0x7f071415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f070c009d90 tx=0x7f070c004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 shutdown_connections 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06f8066760 0x7f06f8068c20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07140ce2d0 0x7f071415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f071415d5b0 0x7f071415ef20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 --2- 172.21.15.179:0/3898539539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07140cde70 0x7f0714153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 >> 172.21.15.179:0/3898539539 conn(0x7f07140b4940 msgr2=0x7f07140c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 shutdown_connections 2024-05-08T22:34:28.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:28.878+0000 7f071a33f700 1 -- 172.21.15.179:0/3898539539 wait complete. 2024-05-08T22:34:28.881 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:34:28.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:29.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.002+0000 7f7d2759e700 1 Processor -- start 2024-05-08T22:34:29.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.002+0000 7f7d2759e700 1 -- start start 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d281434d0 0x7f7d2806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7d2806dc20 0x7f7d2806e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d280cd200 con 0x7f7d280ce2a0 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280cd4e0 con 0x7f7d2806dc20 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280cd7c0 con 0x7f7d281434d0 2024-05-08T22:34:29.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7d2806dc20 0x7f7d2806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7d2806dc20 0x7f7d2806e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49246/0 (socket says 172.21.15.179:49246) 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 -- 172.21.15.179:0/1747780467 learned_addr learned my addr 172.21.15.179:0/1747780467 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2659c700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d281434d0 0x7f7d2806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 -- 172.21.15.179:0/1747780467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d281434d0 msgr2=0x7f7d2806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d281434d0 0x7f7d2806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 -- 172.21.15.179:0/1747780467 >> v1:172.21.15.195:6789/0 conn(0x7f7d280ce2a0 legacy=0x7f7d280b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 -- 172.21.15.179:0/1747780467 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280cdaa0 con 0x7f7d2806dc20 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d25d9b700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d2806dc20 0x7f7d2806e640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009910 tx=0x7f7d10009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265b7fcb68d8bdf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2559a700 1 -- 172.21.15.179:0/1747780467 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000ba60 con 0x7f7d2806dc20 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2559a700 1 -- 172.21.15.179:0/1747780467 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1000bbc0 con 0x7f7d2806dc20 2024-05-08T22:34:29.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2559a700 1 -- 172.21.15.179:0/1747780467 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10009e60 con 0x7f7d2806dc20 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- 172.21.15.179:0/1747780467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d2806dc20 msgr2=0x7f7d2806e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d2806dc20 0x7f7d2806e640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009910 tx=0x7f7d10009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- 172.21.15.179:0/1747780467 shutdown_connections 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d2806dc20 0x7f7d2806e640 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 --2- 172.21.15.179:0/1747780467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d281434d0 0x7f7d2806d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- 172.21.15.179:0/1747780467 >> 172.21.15.179:0/1747780467 conn(0x7f7d280ce9d0 msgr2=0x7f7d280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- 172.21.15.179:0/1747780467 shutdown_connections 2024-05-08T22:34:29.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.006+0000 7f7d2759e700 1 -- 172.21.15.179:0/1747780467 wait complete. 2024-05-08T22:34:29.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 Processor -- start 2024-05-08T22:34:29.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- start start 2024-05-08T22:34:29.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 0x7f7d280c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d281434d0 0x7f7d280c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c4250 con 0x7f7d281ef5b0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c4530 con 0x7f7d281434d0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c4810 con 0x7f7d280ce2a0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49250/0 (socket says 172.21.15.179:49250) 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d25d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d281434d0 0x7f7d280c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 -- 172.21.15.179:0/2461233976 learned_addr learned my addr 172.21.15.179:0/2461233976 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2659c700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 0x7f7d280c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 -- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 msgr2=0x7f7d280c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 0x7f7d280c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 -- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d281434d0 msgr2=0x7f7d280c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d281434d0 0x7f7d280c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 -- 172.21.15.179:0/2461233976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009570 con 0x7f7d281ef5b0 2024-05-08T22:34:29.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2659c700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 0x7f7d280c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d26d9d700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000cc50 tx=0x7f7d2000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003c20 con 0x7f7d281ef5b0 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20003d80 con 0x7f7d281ef5b0 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d280c4af0 con 0x7f7d281ef5b0 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000ab60 con 0x7f7d281ef5b0 2024-05-08T22:34:29.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.010+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d280c5860 con 0x7f7d281ef5b0 2024-05-08T22:34:29.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28067740 con 0x7f7d281ef5b0 2024-05-08T22:34:29.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d2000acc0 con 0x7f7d281ef5b0 2024-05-08T22:34:29.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d1effd700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 0x7f7d08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7d200844f0 con 0x7f7d281ef5b0 2024-05-08T22:34:29.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d2659c700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 0x7f7d08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.014+0000 7f7d2659c700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 0x7f7d08068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2813c230 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.022+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7d200515e0 con 0x7f7d281ef5b0 2024-05-08T22:34:29.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.230+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d2804e700 con 0x7f7d281ef5b0 2024-05-08T22:34:29.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.230+0000 7f7d1effd700 1 -- 172.21.15.179:0/2461233976 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4745 (secure 0 0 0) 0x7f7d20054c00 con 0x7f7d281ef5b0 2024-05-08T22:34:29.233 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:29.233 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2047185104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2047185104},{"type":"v1","addr":"172.21.15.195:6835","nonce":2047185104}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1434959575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1434959575},{"type":"v1","addr":"172.21.15.179:6833","nonce":1434959575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3749591289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3749591289},{"type":"v1","addr":"172.21.15.179:6835","nonce":3749591289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:33:14.889801+0000","modified":"2024-05-08T22:33:37.005931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54155},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54155":{"gid":54155,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3661147208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3661147208},{"type":"v1","addr":"172.21.15.195:6837","nonce":3661147208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:34:29.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 msgr2=0x7f7d08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 0x7f7d08068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2813c230 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 msgr2=0x7f7d28158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000cc50 tx=0x7f7d2000cf60 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 shutdown_connections 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d08066730 0x7f7d08068bf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d281434d0 0x7f7d280c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d281ef5b0 0x7f7d28158fd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 --2- 172.21.15.179:0/2461233976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d280ce2a0 0x7f7d280c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 >> 172.21.15.179:0/2461233976 conn(0x7f7d280ce9d0 msgr2=0x7f7d280b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 shutdown_connections 2024-05-08T22:34:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.234+0000 7f7d2759e700 1 -- 172.21.15.179:0/2461233976 wait complete. 2024-05-08T22:34:29.237 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 9 2024-05-08T22:34:29.248 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:29.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 Processor -- start 2024-05-08T22:34:29.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 -- start start 2024-05-08T22:34:29.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00000b19e0 0x7f00000b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b3550 0x7f000015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0000159b60 con 0x7f00001539c0 2024-05-08T22:34:29.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000b39c0 con 0x7f00000b19e0 2024-05-08T22:34:29.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0007458700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0000159e40 con 0x7f00000b3550 2024-05-08T22:34:29.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0006c57700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b3550 0x7f000015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0005c55700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00000b19e0 0x7f00000b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0005c55700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00000b19e0 0x7f00000b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:49612/0 (socket says 172.21.15.179:49612) 2024-05-08T22:34:29.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.358+0000 7f0005c55700 1 -- 172.21.15.179:0/844043152 learned_addr learned my addr 172.21.15.179:0/844043152 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005c55700 1 -- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b3550 msgr2=0x7f000015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005c55700 1 --2- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b3550 0x7f000015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005c55700 1 -- 172.21.15.179:0/844043152 >> v1:172.21.15.195:6789/0 conn(0x7f00001539c0 legacy=0x7f000006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005c55700 1 -- 172.21.15.179:0/844043152 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000015a120 con 0x7f00000b19e0 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005c55700 1 --2- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b19e0 0x7f00000b3010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efffc009e50 tx=0x7efffc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39590f04266e62ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005454700 1 -- 172.21.15.179:0/844043152 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00bb10 con 0x7f00000b19e0 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005454700 1 -- 172.21.15.179:0/844043152 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc00bc70 con 0x7f00000b19e0 2024-05-08T22:34:29.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0005454700 1 -- 172.21.15.179:0/844043152 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc011460 con 0x7f00000b19e0 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b19e0 msgr2=0x7f00000b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 --2- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b19e0 0x7f00000b3010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efffc009e50 tx=0x7efffc00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- 172.21.15.179:0/844043152 shutdown_connections 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 --2- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b3550 0x7f000015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 --2- 172.21.15.179:0/844043152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b19e0 0x7f00000b3010 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- 172.21.15.179:0/844043152 >> 172.21.15.179:0/844043152 conn(0x7f00000ce9d0 msgr2=0x7f00000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- 172.21.15.179:0/844043152 shutdown_connections 2024-05-08T22:34:29.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- 172.21.15.179:0/844043152 wait complete. 2024-05-08T22:34:29.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 Processor -- start 2024-05-08T22:34:29.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.362+0000 7f0007458700 1 -- start start 2024-05-08T22:34:29.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 0x7f000013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b3550 0x7f000006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 0x7f000006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000006e790 con 0x7f00001539c0 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000151280 con 0x7f00000b3550 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000006db80 con 0x7f00000b19e0 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006456700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 0x7f000013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 0x7f000006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006456700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 0x7f000013d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45686/0 (socket says 172.21.15.179:45686) 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006456700 1 -- 172.21.15.179:0/605945063 learned_addr learned my addr 172.21.15.179:0/605945063 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0005c55700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b3550 0x7f000006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 -- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 msgr2=0x7f000013d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 0x7f000013d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 -- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b3550 msgr2=0x7f000006cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b3550 0x7f000006cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 -- 172.21.15.179:0/605945063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc009b30 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0006c57700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 0x7f000006d4e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efff800e820 tx=0x7efff800eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8013680 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000006de60 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff8013800 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8013bb0 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000006e420 con 0x7f00001539c0 2024-05-08T22:34:29.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.366+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0000067740 con 0x7f00001539c0 2024-05-08T22:34:29.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.370+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efff8013d10 con 0x7f00001539c0 2024-05-08T22:34:29.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.370+0000 7efff6ffd700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 0x7effec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.370+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efff8084450 con 0x7f00001539c0 2024-05-08T22:34:29.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.370+0000 7f0006456700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 0x7effec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.370+0000 7f0006456700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 0x7effec068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efff0001940 tx=0x7efff0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.374+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efff8051250 con 0x7f00001539c0 2024-05-08T22:34:29.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.546+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f000004e700 con 0x7f00001539c0 2024-05-08T22:34:29.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.550+0000 7efff6ffd700 1 -- 172.21.15.179:0/605945063 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18315 (secure 0 0 0) 0x7efff8054870 con 0x7f00001539c0 2024-05-08T22:34:29.552 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:29.552 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":28,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:04.467432+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:33:12.850411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:33:13.884821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T22:33:16.947767+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:29.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 msgr2=0x7effec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 0x7effec068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efff0001940 tx=0x7efff0009040 comp rx=0 tx=0).stop 2024-05-08T22:34:29.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 msgr2=0x7f000006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 0x7f000006d4e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efff800e820 tx=0x7efff800eb30 comp rx=0 tx=0).stop 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 shutdown_connections 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7effec066730 0x7effec068bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00000b3550 0x7f000006cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00001539c0 0x7f000006d4e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 --2- 172.21.15.179:0/605945063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000b19e0 0x7f000013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 >> 172.21.15.179:0/605945063 conn(0x7f00000ce9d0 msgr2=0x7f00000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 shutdown_connections 2024-05-08T22:34:29.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.554+0000 7f0007458700 1 -- 172.21.15.179:0/605945063 wait complete. 2024-05-08T22:34:29.568 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:34:29.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 Processor -- start 2024-05-08T22:34:29.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 -- start start 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa0159270 0x7fdfa00b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdfa015cf70 0x7fdfa006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdfa015d3f0 0x7fdfa006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa015d950 con 0x7fdfa015cf70 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa015dc30 con 0x7fdfa015d3f0 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdfa4c4a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00b1af0 con 0x7fdfa0159270 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdfa015cf70 0x7fdfa006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdfa015cf70 0x7fdfa006b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49276/0 (socket says 172.21.15.179:49276) 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9effd700 1 -- 172.21.15.179:0/77169490 learned_addr learned my addr 172.21.15.179:0/77169490 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9ffff700 1 --2- 172.21.15.179:0/77169490 >> v2:172.21.15.195:3300/0 conn(0x7fdfa015d3f0 0x7fdfa006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9f7fe700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa0159270 0x7fdfa00b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9effd700 1 -- 172.21.15.179:0/77169490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa0159270 msgr2=0x7fdfa00b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:29.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.682+0000 7fdf9effd700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa0159270 0x7fdfa00b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9effd700 1 -- 172.21.15.179:0/77169490 >> v2:172.21.15.195:3300/0 conn(0x7fdfa015d3f0 msgr2=0x7fdfa006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9effd700 1 --2- 172.21.15.179:0/77169490 >> v2:172.21.15.195:3300/0 conn(0x7fdfa015d3f0 0x7fdfa006b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9effd700 1 -- 172.21.15.179:0/77169490 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00b1dd0 con 0x7fdfa015cf70 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9effd700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa015cf70 0x7fdfa006b440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00b0600 tx=0x7fdf9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ab41ef4e59c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9e7fc700 1 -- 172.21.15.179:0/77169490 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000e070 con 0x7fdfa015cf70 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9e7fc700 1 -- 172.21.15.179:0/77169490 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000b910 con 0x7fdfa015cf70 2024-05-08T22:34:29.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9e7fc700 1 -- 172.21.15.179:0/77169490 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000bcc0 con 0x7fdfa015cf70 2024-05-08T22:34:29.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/77169490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa015cf70 msgr2=0x7fdfa006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa015cf70 0x7fdfa006b440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00b0600 tx=0x7fdf9000b330 comp rx=0 tx=0).stop 2024-05-08T22:34:29.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/77169490 shutdown_connections 2024-05-08T22:34:29.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/77169490 >> v2:172.21.15.195:3300/0 conn(0x7fdfa015d3f0 0x7fdfa006b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa015cf70 0x7fdfa006b440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/77169490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa0159270 0x7fdfa00b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/77169490 >> 172.21.15.179:0/77169490 conn(0x7fdfa00ce960 msgr2=0x7fdfa00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/77169490 shutdown_connections 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/77169490 wait complete. 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 Processor -- start 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- start start 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa015d3f0 0x7fdfa00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa0156f40 0x7fdfa006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0153900 con 0x7fdfa0159270 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0157400 con 0x7fdfa0156f40 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdfa4c4a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa01576e0 con 0x7fdfa015d3f0 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49288/0 (socket says 172.21.15.179:49288) 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9f7fe700 1 -- 172.21.15.179:0/3434837982 learned_addr learned my addr 172.21.15.179:0/3434837982 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9effd700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa015d3f0 0x7fdfa00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.686+0000 7fdf9ffff700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa0156f40 0x7fdfa006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 -- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa015d3f0 msgr2=0x7fdfa00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa015d3f0 0x7fdfa00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 -- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa0156f40 msgr2=0x7fdfa006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa0156f40 0x7fdfa006cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 -- 172.21.15.179:0/3434837982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90009b30 con 0x7fdfa0159270 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf9f7fe700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e760 tx=0x7fdf8800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800c070 con 0x7fdfa0159270 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88013610 con 0x7fdfa0159270 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa01579c0 con 0x7fdfa0159270 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88013980 con 0x7fdfa0159270 2024-05-08T22:34:29.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa0157f80 con 0x7fdfa0159270 2024-05-08T22:34:29.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.690+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa004e700 con 0x7fdfa0159270 2024-05-08T22:34:29.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.694+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf88013ae0 con 0x7fdfa0159270 2024-05-08T22:34:29.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.694+0000 7fdf97fff700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 0x7fdf84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:29.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.694+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdf88084270 con 0x7fdfa0159270 2024-05-08T22:34:29.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.694+0000 7fdf9effd700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 0x7fdf84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:29.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.694+0000 7fdf9effd700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 0x7fdf84068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdfa006e330 tx=0x7fdf90009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:29.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.702+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf88051220 con 0x7fdfa0159270 2024-05-08T22:34:29.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.902+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdfa00b3e70 con 0x7fdfa0159270 2024-05-08T22:34:29.905 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:34:29.902+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:34:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.918+0000 7fdf97fff700 1 -- 172.21.15.179:0/3434837982 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf88054840 con 0x7fdfa0159270 2024-05-08T22:34:29.923 INFO:tasks.ceph.mds.d.smithi195.stderr: -4550> 2024-05-08T22:33:18.578+0000 7f1ac9d4f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:34:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 msgr2=0x7fdf84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 0x7fdf84068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdfa006e330 tx=0x7fdf90009000 comp rx=0 tx=0).stop 2024-05-08T22:34:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 msgr2=0x7fdfa0152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e760 tx=0x7fdf8800ea70 comp rx=0 tx=0).stop 2024-05-08T22:34:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 shutdown_connections 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf84066730 0x7fdf84068bf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa0156f40 0x7fdfa006cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfa015d3f0 0x7fdfa00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 --2- 172.21.15.179:0/3434837982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfa0159270 0x7fdfa0152e30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 >> 172.21.15.179:0/3434837982 conn(0x7fdfa00ce960 msgr2=0x7fdfa00cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 shutdown_connections 2024-05-08T22:34:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:29.922+0000 7fdfa4c4a700 1 -- 172.21.15.179:0/3434837982 wait complete. 2024-05-08T22:34:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:34:29.941 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:34:29.960 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:34:30.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 Processor -- start 2024-05-08T22:34:30.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 -- start start 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4a806c240 0x7fa4a80d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a80d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a80b8c20 con 0x7fa4a80cdca0 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a80b8f00 con 0x7fa4a806c240 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80cf640 con 0x7fa4a80d17a0 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a67fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4a806c240 0x7fa4a80d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a6ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa4a80cdca0 0x7fa4a80ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43242/0 (socket says 172.21.15.179:43242) 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a6ffd700 1 -- 172.21.15.179:0/1968764223 learned_addr learned my addr 172.21.15.179:0/1968764223 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:30.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.058+0000 7fa4a77fe700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a80d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1968764223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 msgr2=0x7fa4a80d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a67fc700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a80d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a5ffb700 1 -- 172.21.15.179:0/1968764223 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096699728 0 0) 0x7fa4a80b8c20 con 0x7fa4a80cdca0 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1968764223 >> v1:172.21.15.179:6789/0 conn(0x7fa4a80cdca0 legacy=0x7fa4a80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1968764223 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a806b760 con 0x7fa4a806c240 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a67fc700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a806c240 0x7fa4a80d1260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa498009850 tx=0x7fa498009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b24c7246a89fe04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:30.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a5ffb700 1 -- 172.21.15.179:0/1968764223 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011830 con 0x7fa4a806c240 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a5ffb700 1 -- 172.21.15.179:0/1968764223 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498011990 con 0x7fa4a806c240 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a5ffb700 1 -- 172.21.15.179:0/1968764223 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011d40 con 0x7fa4a806c240 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1968764223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a806c240 msgr2=0x7fa4a80d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a806c240 0x7fa4a80d1260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa498009850 tx=0x7fa498009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1968764223 shutdown_connections 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a80d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1968764223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a806c240 0x7fa4a80d1260 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1968764223 >> 172.21.15.179:0/1968764223 conn(0x7fa4a80d2ed0 msgr2=0x7fa4a80d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1968764223 shutdown_connections 2024-05-08T22:34:30.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1968764223 wait complete. 2024-05-08T22:34:30.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 Processor -- start 2024-05-08T22:34:30.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.062+0000 7fa4a7fff700 1 -- start start 2024-05-08T22:34:30.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a806c240 0x7fa4a80a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a81f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80b57e0 con 0x7fa4a80cdca0 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80cbb30 con 0x7fa4a806c240 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80b5500 con 0x7fa4a80d17a0 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49304/0 (socket says 172.21.15.179:49304) 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1673218933 learned_addr learned my addr 172.21.15.179:0/1673218933 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a77fe700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a81f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a6ffd700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a806c240 0x7fa4a80a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 msgr2=0x7fa4a81f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a81f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a806c240 msgr2=0x7fa4a80a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a806c240 0x7fa4a80a9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 -- 172.21.15.179:0/1673218933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980094b0 con 0x7fa4a80cdca0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a67fc700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa4980090b0 tx=0x7fa498004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498019a00 con 0x7fa4a80cdca0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a80cbdd0 con 0x7fa4a80cdca0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a80b5120 con 0x7fa4a80cdca0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49801e070 con 0x7fa4a80cdca0 2024-05-08T22:34:30.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.066+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498019d60 con 0x7fa4a80cdca0 2024-05-08T22:34:30.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa49801a420 con 0x7fa4a80cdca0 2024-05-08T22:34:30.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa48f7fe700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 0x7fa494068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fa4980856a0 con 0x7fa4a80cdca0 2024-05-08T22:34:30.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa4a6ffd700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 0x7fa494068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa488005310 con 0x7fa4a80cdca0 2024-05-08T22:34:30.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.070+0000 7fa4a6ffd700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 0x7fa494068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000f60 tx=0x7fa49c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:30.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.074+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa498052570 con 0x7fa4a80cdca0 2024-05-08T22:34:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.286+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa488005b40 con 0x7fa4a80cdca0 2024-05-08T22:34:30.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.922+0000 7fa48f7fe700 1 -- 172.21.15.179:0/1673218933 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4980200a0 con 0x7fa4a80cdca0 2024-05-08T22:34:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.926+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 msgr2=0x7fa494068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.926+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 0x7fa494068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000f60 tx=0x7fa49c007c00 comp rx=0 tx=0).stop 2024-05-08T22:34:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.926+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 msgr2=0x7fa4a81f5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.926+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa4980090b0 tx=0x7fa498004770 comp rx=0 tx=0).stop 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 shutdown_connections 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa494066730 0x7fa494068bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a80d17a0 0x7fa4a81f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a80cdca0 0x7fa4a81f5ec0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 --2- 172.21.15.179:0/1673218933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a806c240 0x7fa4a80a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 >> 172.21.15.179:0/1673218933 conn(0x7fa4a80d2ed0 msgr2=0x7fa4a815d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:30.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 shutdown_connections 2024-05-08T22:34:30.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:30.930+0000 7fa4a7fff700 1 -- 172.21.15.179:0/1673218933 wait complete. 2024-05-08T22:34:30.947 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:34:31.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 Processor -- start 2024-05-08T22:34:31.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 -- start start 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc240e33a0 0x7fdc24058950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc24007280 0x7fdc24007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240578b0 con 0x7fdc2403ebd0 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240576c0 con 0x7fdc24007280 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.074+0000 7fdc31c2c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc24007c50 con 0x7fdc240e33a0 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdc2403ebd0 0x7fdc24056fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43254/0 (socket says 172.21.15.179:43254) 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2bfff700 1 -- 172.21.15.179:0/4086227886 learned_addr learned my addr 172.21.15.179:0/4086227886 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc24007280 0x7fdc24007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc30c2a700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc240e33a0 0x7fdc24058950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2b7fe700 1 -- 172.21.15.179:0/4086227886 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953588226 0 0) 0x7fdc240578b0 con 0x7fdc2403ebd0 2024-05-08T22:34:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 -- 172.21.15.179:0/4086227886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc240e33a0 msgr2=0x7fdc24058950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc240e33a0 0x7fdc24058950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 -- 172.21.15.179:0/4086227886 >> v1:172.21.15.179:6789/0 conn(0x7fdc2403ebd0 legacy=0x7fdc24056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 -- 172.21.15.179:0/4086227886 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24057b90 con 0x7fdc24007280 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc3142b700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc24007280 0x7fdc24007710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c04f480 tx=0x7fdc2c04f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d01ff21c824405d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2b7fe700 1 -- 172.21.15.179:0/4086227886 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c067250 con 0x7fdc24007280 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2b7fe700 1 -- 172.21.15.179:0/4086227886 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c0673d0 con 0x7fdc24007280 2024-05-08T22:34:31.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc2b7fe700 1 -- 172.21.15.179:0/4086227886 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c067960 con 0x7fdc24007280 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 -- 172.21.15.179:0/4086227886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc24007280 msgr2=0x7fdc24007710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc24007280 0x7fdc24007710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c04f480 tx=0x7fdc2c04f710 comp rx=0 tx=0).stop 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 -- 172.21.15.179:0/4086227886 shutdown_connections 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc24007280 0x7fdc24007710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/4086227886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc240e33a0 0x7fdc24058950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 -- 172.21.15.179:0/4086227886 >> 172.21.15.179:0/4086227886 conn(0x7fdc24055a10 msgr2=0x7fdc24055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 -- 172.21.15.179:0/4086227886 shutdown_connections 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.078+0000 7fdc31c2c700 1 -- 172.21.15.179:0/4086227886 wait complete. 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 Processor -- start 2024-05-08T22:34:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- start start 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 0x7fdc240e0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc2403ebd0 0x7fdc24051a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240e04f0 con 0x7fdc240e33a0 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240523b0 con 0x7fdc24007280 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc24052690 con 0x7fdc2403ebd0 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49318/0 (socket says 172.21.15.179:49318) 2024-05-08T22:34:31.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 -- 172.21.15.179:0/159176322 learned_addr learned my addr 172.21.15.179:0/159176322 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc30c2a700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 0x7fdc240e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc2bfff700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc2403ebd0 0x7fdc24051a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 -- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc2403ebd0 msgr2=0x7fdc24051a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc2403ebd0 0x7fdc24051a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 -- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 msgr2=0x7fdc240e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 0x7fdc240e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 -- 172.21.15.179:0/159176322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240e0180 con 0x7fdc240e33a0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc30c2a700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 0x7fdc240e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc3142b700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c074ec0 tx=0x7fdc2c0739d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c066eb0 con 0x7fdc240e33a0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c066150 con 0x7fdc240e33a0 2024-05-08T22:34:31.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c066540 con 0x7fdc240e33a0 2024-05-08T22:34:31.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc240e0ec0 con 0x7fdc240e33a0 2024-05-08T22:34:31.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.082+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc24052180 con 0x7fdc240e33a0 2024-05-08T22:34:31.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc2c0666a0 con 0x7fdc240e33a0 2024-05-08T22:34:31.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc24008070 con 0x7fdc240e33a0 2024-05-08T22:34:31.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc297fa700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 0x7fdc18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:31.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fdc2c099070 con 0x7fdc240e33a0 2024-05-08T22:34:31.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc30c2a700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 0x7fdc18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:31.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.086+0000 7fdc30c2a700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 0x7fdc18068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009ad0 tx=0x7fdc1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:31.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.094+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc2c0cc1e0 con 0x7fdc240e33a0 2024-05-08T22:34:31.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.262+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc24005440 con 0x7fdc240e33a0 2024-05-08T22:34:31.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.934+0000 7fdc297fa700 1 -- 172.21.15.179:0/159176322 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdc2c0cf800 con 0x7fdc240e33a0 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 msgr2=0x7fdc18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 0x7fdc18068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009ad0 tx=0x7fdc1c008040 comp rx=0 tx=0).stop 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 msgr2=0x7fdc2416eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c074ec0 tx=0x7fdc2c0739d0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 shutdown_connections 2024-05-08T22:34:31.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc18066730 0x7fdc18068bf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc240e33a0 0x7fdc2416eba0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc2403ebd0 0x7fdc24051a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 --2- 172.21.15.179:0/159176322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc24007280 0x7fdc240e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 >> 172.21.15.179:0/159176322 conn(0x7fdc24055a10 msgr2=0x7fdc2403c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 shutdown_connections 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:31.938+0000 7fdc31c2c700 1 -- 172.21.15.179:0/159176322 wait complete. 2024-05-08T22:34:31.941 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:34:31.954 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:34:32.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 Processor -- start 2024-05-08T22:34:32.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- start start 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff8d81597f0 0x7ff8d80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 0x7ff8d806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d806b890 con 0x7ff8d80b1af0 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d80cd3e0 con 0x7ff8d81597f0 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80ce7b0 con 0x7ff8d806dd70 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dd300700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff8d80b1af0 0x7ff8d806d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43262/0 (socket says 172.21.15.179:43262) 2024-05-08T22:34:32.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dd300700 1 -- 172.21.15.179:0/4156008835 learned_addr learned my addr 172.21.15.179:0/4156008835 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:32.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 --2- 172.21.15.179:0/4156008835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 0x7ff8d806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8ddb01700 1 --2- 172.21.15.179:0/4156008835 >> v2:172.21.15.195:3300/0 conn(0x7ff8d81597f0 0x7ff8d80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507570864 0 0) 0x7ff8d806b890 con 0x7ff8d80b1af0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c00035d0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 -- 172.21.15.179:0/4156008835 >> v2:172.21.15.195:3300/0 conn(0x7ff8d81597f0 msgr2=0x7ff8d80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 --2- 172.21.15.179:0/4156008835 >> v2:172.21.15.195:3300/0 conn(0x7ff8d81597f0 0x7ff8d80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070016016 0 0) 0x7ff8c00035d0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 -- 172.21.15.179:0/4156008835 >> v1:172.21.15.179:6789/0 conn(0x7ff8d80b1af0 legacy=0x7ff8d806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 -- 172.21.15.179:0/4156008835 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80ced30 con 0x7ff8d806dd70 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8de302700 1 --2- 172.21.15.179:0/4156008835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 0x7ff8d806c000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdf13f99fb73122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000e070 con 0x7ff8d806dd70 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d000bb20 con 0x7ff8d806dd70 2024-05-08T22:34:32.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8dcaff700 1 -- 172.21.15.179:0/4156008835 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000be90 con 0x7ff8d806dd70 2024-05-08T22:34:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- 172.21.15.179:0/4156008835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 msgr2=0x7ff8d806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 --2- 172.21.15.179:0/4156008835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 0x7ff8d806c000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d000b330 comp rx=0 tx=0).stop 2024-05-08T22:34:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- 172.21.15.179:0/4156008835 shutdown_connections 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 --2- 172.21.15.179:0/4156008835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d806dd70 0x7ff8d806c000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 --2- 172.21.15.179:0/4156008835 >> v2:172.21.15.195:3300/0 conn(0x7ff8d81597f0 0x7ff8d80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- 172.21.15.179:0/4156008835 >> 172.21.15.179:0/4156008835 conn(0x7ff8d8098a70 msgr2=0x7ff8d80b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- 172.21.15.179:0/4156008835 shutdown_connections 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.074+0000 7ff8deb03700 1 -- 172.21.15.179:0/4156008835 wait complete. 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 Processor -- start 2024-05-08T22:34:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- start start 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d806dd70 0x7ff8d80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d81597f0 0x7ff8d814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80aef00 con 0x7ff8d80b1af0 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80af1e0 con 0x7ff8d806dd70 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80af4c0 con 0x7ff8d81597f0 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49328/0 (socket says 172.21.15.179:49328) 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 -- 172.21.15.179:0/3277795415 learned_addr learned my addr 172.21.15.179:0/3277795415 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8de302700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d81597f0 0x7ff8d814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8ddb01700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d806dd70 0x7ff8d80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 -- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d81597f0 msgr2=0x7ff8d814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d81597f0 0x7ff8d814d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 -- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d806dd70 msgr2=0x7ff8d80b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d806dd70 0x7ff8d80b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 -- 172.21.15.179:0/3277795415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0009a70 con 0x7ff8d80b1af0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8dd300700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c780 tx=0x7ff8cc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc005d90 con 0x7ff8d80b1af0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0070d0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc007480 con 0x7ff8d80b1af0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d80af7a0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.078+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80b0bf0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.082+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d8067770 con 0x7ff8d80b1af0 2024-05-08T22:34:32.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.082+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8cc0075e0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.082+0000 7ff8d67fc700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:32.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.082+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6311+0+0 (secure 0 0 0) 0x7ff8cc0835b0 con 0x7ff8d80b1af0 2024-05-08T22:34:32.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.082+0000 7ff8ddb01700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:32.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.086+0000 7ff8ddb01700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 0x7ff8c4068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009b50 tx=0x7ff8c8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:32.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.090+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8cc050630 con 0x7ff8d80b1af0 2024-05-08T22:34:32.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.266+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8d80af040 con 0x7ff8d80b1af0 2024-05-08T22:34:32.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.938+0000 7ff8d67fc700 1 -- 172.21.15.179:0/3277795415 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff8d80af040 con 0x7ff8d80b1af0 2024-05-08T22:34:32.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 msgr2=0x7ff8c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 0x7ff8c4068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009b50 tx=0x7ff8c8009240 comp rx=0 tx=0).stop 2024-05-08T22:34:32.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 msgr2=0x7ff8d814d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:32.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c780 tx=0x7ff8cc00ca90 comp rx=0 tx=0).stop 2024-05-08T22:34:32.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 shutdown_connections 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8d81597f0 0x7ff8d814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8d80b1af0 0x7ff8d814d220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 --2- 172.21.15.179:0/3277795415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d806dd70 0x7ff8d80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 >> 172.21.15.179:0/3277795415 conn(0x7ff8d8098a70 msgr2=0x7ff8d80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 shutdown_connections 2024-05-08T22:34:32.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:32.946+0000 7ff8deb03700 1 -- 172.21.15.179:0/3277795415 wait complete. 2024-05-08T22:34:32.948 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:34:32.964 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T22:34:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.094+0000 7f1225de6700 1 Processor -- start 2024-05-08T22:34:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.094+0000 7f1225de6700 1 -- start start 2024-05-08T22:34:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12200b4140 0x7f12201445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12200b48f0 con 0x7f1220143650 2024-05-08T22:34:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12200b4bd0 con 0x7f12200b4140 2024-05-08T22:34:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12200b4eb0 con 0x7f12200cef20 2024-05-08T22:34:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1224de4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1220143650 0x7f12200ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43264/0 (socket says 172.21.15.179:43264) 2024-05-08T22:34:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1224de4700 1 -- 172.21.15.179:0/2511909731 learned_addr learned my addr 172.21.15.179:0/2511909731 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121ffff700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 --2- 172.21.15.179:0/2511909731 >> v2:172.21.15.195:3300/0 conn(0x7f12200b4140 0x7f12201445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121f7fe700 1 -- 172.21.15.179:0/2511909731 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087699903 0 0) 0x7f12200b48f0 con 0x7f1220143650 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121f7fe700 1 -- 172.21.15.179:0/2511909731 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120c0035d0 con 0x7f1220143650 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 -- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 msgr2=0x7f12200b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 -- 172.21.15.179:0/2511909731 >> v1:172.21.15.179:6789/0 conn(0x7f1220143650 legacy=0x7f12200ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 -- 172.21.15.179:0/2511909731 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220143de0 con 0x7f12200b4140 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121ffff700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f12255e5700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201445e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1218009850 tx=0x7f1218009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0e1899a8aa5855 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:33.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121f7fe700 1 -- 172.21.15.179:0/2511909731 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011830 con 0x7f12200b4140 2024-05-08T22:34:33.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121f7fe700 1 -- 172.21.15.179:0/2511909731 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218011990 con 0x7f12200b4140 2024-05-08T22:34:33.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f121f7fe700 1 -- 172.21.15.179:0/2511909731 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011d40 con 0x7f12200b4140 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 msgr2=0x7f12201445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201445e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1218009850 tx=0x7f1218009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- 172.21.15.179:0/2511909731 shutdown_connections 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201445e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 --2- 172.21.15.179:0/2511909731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- 172.21.15.179:0/2511909731 >> 172.21.15.179:0/2511909731 conn(0x7f122006f690 msgr2=0x7f122006faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- 172.21.15.179:0/2511909731 shutdown_connections 2024-05-08T22:34:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.098+0000 7f1225de6700 1 -- 172.21.15.179:0/2511909731 wait complete. 2024-05-08T22:34:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 Processor -- start 2024-05-08T22:34:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 -- start start 2024-05-08T22:34:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12200cd870 con 0x7f1220143650 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12200ccf40 con 0x7f12200b4140 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1225de6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12200cd220 con 0x7f12200cef20 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f12255e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f12255e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49338/0 (socket says 172.21.15.179:49338) 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f12255e5700 1 -- 172.21.15.179:0/2851551170 learned_addr learned my addr 172.21.15.179:0/2851551170 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f1224de4700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.102+0000 7f121ffff700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 -- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 msgr2=0x7f12200c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 -- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 msgr2=0x7f12201f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 -- 172.21.15.179:0/2851551170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080094b0 con 0x7f1220143650 2024-05-08T22:34:33.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f12255e5700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f12200b2780 tx=0x7f1218004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:33.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218004300 con 0x7f1220143650 2024-05-08T22:34:33.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121801f070 con 0x7f1220143650 2024-05-08T22:34:33.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12180036c0 con 0x7f1220143650 2024-05-08T22:34:33.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12180094b0 con 0x7f1220143650 2024-05-08T22:34:33.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12200ccba0 con 0x7f1220143650 2024-05-08T22:34:33.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.106+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12200cda20 con 0x7f1220143650 2024-05-08T22:34:33.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.110+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1218021030 con 0x7f1220143650 2024-05-08T22:34:33.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.110+0000 7f121d7fa700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 0x7f1200068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.110+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f1218085340 con 0x7f1220143650 2024-05-08T22:34:33.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.110+0000 7f1224de4700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 0x7f1200068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.110+0000 7f1224de4700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 0x7f1200068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1210001560 tx=0x7f12100019f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:33.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.118+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f12200cda20 con 0x7f1220143650 2024-05-08T22:34:33.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.298+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f122004e770 con 0x7f1220143650 2024-05-08T22:34:33.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.957+0000 7f121d7fa700 1 -- 172.21.15.179:0/2851551170 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7f12180528c0 con 0x7f1220143650 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 msgr2=0x7f1200068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 0x7f1200068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1210001560 tx=0x7f12100019f0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 msgr2=0x7f12200ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f12200b2780 tx=0x7f1218004770 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 shutdown_connections 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1200066730 0x7f1200068bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1220143650 0x7f12200ca440 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200cef20 0x7f12200c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 --2- 172.21.15.179:0/2851551170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12200b4140 0x7f12201f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 >> 172.21.15.179:0/2851551170 conn(0x7f122006f690 msgr2=0x7f1220161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:33.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 shutdown_connections 2024-05-08T22:34:33.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:33.961+0000 7f1225de6700 1 -- 172.21.15.179:0/2851551170 wait complete. 2024-05-08T22:34:33.964 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data_ec' removed 2024-05-08T22:34:33.978 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:34:33.978 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:34:33.978 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:34:33.978 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:33.978 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:34:33.978 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:34:33.979 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:34:33.979 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:33.979 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:34:33.979 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:34:33.979 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:34:33.979 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:33.979 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:34:33.979 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:34:33.980 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:34:33.980 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:34:33.989 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:34:33.989 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:34:33.990 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:34:33.990 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:34:33.991 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:34:33.992 DEBUG:teuthology.parallel:result is None 2024-05-08T22:34:33.992 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:34:33.993 DEBUG:teuthology.parallel:result is None 2024-05-08T22:34:33.994 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:34:33.994 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:34:33.995 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:34:33.995 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:34:34.038 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:34:34.038 DEBUG:teuthology.parallel:result is None 2024-05-08T22:34:34.038 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:34:34.039 DEBUG:teuthology.parallel:result is None 2024-05-08T22:34:34.039 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:34:34.069 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:34:34.071 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:34:34.123 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:34:34.123 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:34:34.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.145+0000 7f16753fc700 1 Processor -- start 2024-05-08T22:34:34.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.145+0000 7f16753fc700 1 -- start start 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f16753fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 0x7f167006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f16753fc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700cd260 con 0x7f167015a010 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166effd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f167015a010 0x7f167006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43290/0 (socket says 172.21.15.179:43290) 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f16753fc700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700cb060 con 0x7f167006c100 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166effd700 1 -- 172.21.15.179:0/736748994 learned_addr learned my addr 172.21.15.179:0/736748994 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166e7fc700 1 --2- 172.21.15.179:0/736748994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 0x7f167006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167006c590 con 0x7f16700b1b50 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166e7fc700 1 -- 172.21.15.179:0/736748994 >> v1:172.21.15.195:6789/0 conn(0x7f167006c100 legacy=0x7f16700cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166e7fc700 1 -- 172.21.15.179:0/736748994 >> v1:172.21.15.179:6789/0 conn(0x7f167015a010 legacy=0x7f167006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:34.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166e7fc700 1 -- 172.21.15.179:0/736748994 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700cb340 con 0x7f16700b1b50 2024-05-08T22:34:34.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166e7fc700 1 --2- 172.21.15.179:0/736748994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 0x7f167006bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1660009880 tx=0x7f1660009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b715c3a88f6c9132 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:34.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166dffb700 1 -- 172.21.15.179:0/736748994 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000f8a0 con 0x7f16700b1b50 2024-05-08T22:34:34.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166dffb700 1 -- 172.21.15.179:0/736748994 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166000fa00 con 0x7f16700b1b50 2024-05-08T22:34:34.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.149+0000 7f166dffb700 1 -- 172.21.15.179:0/736748994 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000fd90 con 0x7f16700b1b50 2024-05-08T22:34:34.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 msgr2=0x7f167006bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:34.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 --2- 172.21.15.179:0/736748994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 0x7f167006bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1660009880 tx=0x7f1660009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:34.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 shutdown_connections 2024-05-08T22:34:34.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 --2- 172.21.15.179:0/736748994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16700b1b50 0x7f167006bbc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:34.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 >> 172.21.15.179:0/736748994 conn(0x7f1670070e50 msgr2=0x7f1670071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:34.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 shutdown_connections 2024-05-08T22:34:34.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.153+0000 7f16753fc700 1 -- 172.21.15.179:0/736748994 wait complete. 2024-05-08T22:34:34.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 Processor -- start 2024-05-08T22:34:34.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 -- start start 2024-05-08T22:34:34.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 0x7f16700c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:34.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 0x7f16700c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:34.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 0x7f16700c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:34.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16700c4390 con 0x7f16700b1b50 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16700c4670 con 0x7f167006c100 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f16753fc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16700c4950 con 0x7f167015a010 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f166e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 0x7f16700c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 0x7f16700c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f166effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 0x7f16700c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.157+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 0x7f16700c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45810/0 (socket says 172.21.15.179:45810) 2024-05-08T22:34:34.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166f7fe700 1 -- 172.21.15.179:0/3648573320 learned_addr learned my addr 172.21.15.179:0/3648573320 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:34.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 -- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 msgr2=0x7f16700c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 0x7f16700c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 -- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 msgr2=0x7f16700c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 0x7f16700c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 -- 172.21.15.179:0/3648573320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600094b0 con 0x7f16700b1b50 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166effd700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 0x7f16700c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f166e7fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 0x7f16700c6ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1660018000 tx=0x7f1660004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166001d040 con 0x7f16700b1b50 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660003ad0 con 0x7f16700b1b50 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16700c4c30 con 0x7f16700b1b50 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660003e80 con 0x7f16700b1b50 2024-05-08T22:34:34.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167014bec0 con 0x7f16700b1b50 2024-05-08T22:34:34.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167014c050 con 0x7f16700b1b50 2024-05-08T22:34:34.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.161+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1660010470 con 0x7f16700b1b50 2024-05-08T22:34:34.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.165+0000 7f16577fe700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 0x7f165c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:34.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.165+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1660087a40 con 0x7f16700b1b50 2024-05-08T22:34:34.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.165+0000 7f166effd700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 0x7f165c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:34.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.165+0000 7f166effd700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 0x7f165c068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1668005cb0 tx=0x7f1668009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:34.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.173+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1660054f80 con 0x7f16700b1b50 2024-05-08T22:34:34.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:34.353+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1670067770 con 0x7f16700b1b50 2024-05-08T22:34:35.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.157+0000 7f16577fe700 1 -- 172.21.15.179:0/3648573320 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7f16600585a0 con 0x7f16700b1b50 2024-05-08T22:34:35.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 msgr2=0x7f165c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 0x7f165c068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1668005cb0 tx=0x7f1668009040 comp rx=0 tx=0).stop 2024-05-08T22:34:35.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 msgr2=0x7f16700c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 0x7f16700c6ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1660018000 tx=0x7f1660004770 comp rx=0 tx=0).stop 2024-05-08T22:34:35.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 shutdown_connections 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f165c066730 0x7f165c068bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f167015a010 0x7f16700c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16700b1b50 0x7f16700c6ed0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 --2- 172.21.15.179:0/3648573320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f167006c100 0x7f16700c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 >> 172.21.15.179:0/3648573320 conn(0x7f1670070e50 msgr2=0x7f16700b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 shutdown_connections 2024-05-08T22:34:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.161+0000 7f16753fc700 1 -- 172.21.15.179:0/3648573320 wait complete. 2024-05-08T22:34:35.166 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:34:35.180 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:34:35.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.337+0000 7f946a069700 1 Processor -- start 2024-05-08T22:34:35.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.337+0000 7f946a069700 1 -- start start 2024-05-08T22:34:35.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94641539c0 0x7f946406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 0x7f94640b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464159b60 con 0x7f94640b19e0 2024-05-08T22:34:35.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946406b2c0 con 0x7f94641539c0 2024-05-08T22:34:35.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9464159e40 con 0x7f946406bc50 2024-05-08T22:34:35.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94641539c0 0x7f946406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94641539c0 0x7f946406b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40856/0 (socket says 172.21.15.179:40856) 2024-05-08T22:34:35.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 -- 172.21.15.179:0/3881157968 learned_addr learned my addr 172.21.15.179:0/3881157968 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9468866700 1 --2- 172.21.15.179:0/3881157968 >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 0x7f94640b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 -- 172.21.15.179:0/3881157968 >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 msgr2=0x7f94640b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 --2- 172.21.15.179:0/3881157968 >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 0x7f94640b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 -- 172.21.15.179:0/3881157968 >> v1:172.21.15.195:6789/0 conn(0x7f94640b19e0 legacy=0x7f946406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 -- 172.21.15.179:0/3881157968 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946415a120 con 0x7f94641539c0 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9468866700 1 --2- 172.21.15.179:0/3881157968 >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 0x7f94640b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f9469067700 1 --2- 172.21.15.179:0/3881157968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94641539c0 0x7f946406b710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94640708c0 tx=0x7f945400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850012657b71129 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f945bfff700 1 -- 172.21.15.179:0/3881157968 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400dd30 con 0x7f94641539c0 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f945bfff700 1 -- 172.21.15.179:0/3881157968 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400de90 con 0x7f94641539c0 2024-05-08T22:34:35.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f945bfff700 1 -- 172.21.15.179:0/3881157968 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454010690 con 0x7f94641539c0 2024-05-08T22:34:35.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- 172.21.15.179:0/3881157968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94641539c0 msgr2=0x7f946406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 --2- 172.21.15.179:0/3881157968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94641539c0 0x7f946406b710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94640708c0 tx=0x7f945400be90 comp rx=0 tx=0).stop 2024-05-08T22:34:35.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- 172.21.15.179:0/3881157968 shutdown_connections 2024-05-08T22:34:35.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 --2- 172.21.15.179:0/3881157968 >> v2:172.21.15.195:3300/0 conn(0x7f946406bc50 0x7f94640b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 --2- 172.21.15.179:0/3881157968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94641539c0 0x7f946406b710 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- 172.21.15.179:0/3881157968 >> 172.21.15.179:0/3881157968 conn(0x7f94640ce9d0 msgr2=0x7f94640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:35.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- 172.21.15.179:0/3881157968 shutdown_connections 2024-05-08T22:34:35.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.341+0000 7f946a069700 1 -- 172.21.15.179:0/3881157968 wait complete. 2024-05-08T22:34:35.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 Processor -- start 2024-05-08T22:34:35.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 -- start start 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640b19e0 0x7f94640a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94641539c0 0x7f9464158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946415b7b0 con 0x7f946406bc50 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946414fdb0 con 0x7f94640b19e0 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f946a069700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946414f3c0 con 0x7f94641539c0 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.345+0000 7f9469067700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40858/0 (socket says 172.21.15.179:40858) 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 -- 172.21.15.179:0/3363776787 learned_addr learned my addr 172.21.15.179:0/3363776787 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:35.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9468866700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640b19e0 0x7f94640a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469868700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94641539c0 0x7f9464158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 -- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94641539c0 msgr2=0x7f9464158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94641539c0 0x7f9464158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 -- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640b19e0 msgr2=0x7f94640a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640b19e0 0x7f94640a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 -- 172.21.15.179:0/3363776787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009a70 con 0x7f946406bc50 2024-05-08T22:34:35.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9469067700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94640b17a0 tx=0x7f9454004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945401d070 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454010840 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945400b810 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454010bd0 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9464152380 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9464067740 con 0x7f946406bc50 2024-05-08T22:34:35.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94540036c0 con 0x7f946406bc50 2024-05-08T22:34:35.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 0x7f9448068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9454085810 con 0x7f946406bc50 2024-05-08T22:34:35.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.349+0000 7f9468866700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 0x7f9448068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.353+0000 7f9468866700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 0x7f9448068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f946406b670 tx=0x7f946000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.357+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9454052ec0 con 0x7f946406bc50 2024-05-08T22:34:35.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f946404e700 con 0x7f946406bc50 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f9459ffb700 1 -- 172.21.15.179:0/3363776787 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f94540564e0 con 0x7f946406bc50 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:34:35.591 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:34:35.592 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.593 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.594 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.595 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.596 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.597 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.598 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:34:35.599 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:34:35.600 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:34:35.601 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:34:35.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-mds", 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/Ttm+x+AAxAAqusGa1AP61aMt3kZxlwRXg==", 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/TtmLTqAAxAABfNDzGrc+K2llxZai8wwhA==", 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.603 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-osd", 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/Ttmb1GAAxAAl1Dn5HRGxCMWUkxx4Aj3Bw==", 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/Ttmi2iAAxAAYTKSkgum88qmQrkGMISWHA==", 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T22:34:35.604 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/TtmVYCAAxAAn4bMr/5cAER9169FqjIUYQ==", 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCa/TtmtpuAAxAAakZhZhnfzumzxstpJ+cO/g==", 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.605 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:34:35.606 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:34:35.607 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 msgr2=0x7f9448068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 0x7f9448068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f946406b670 tx=0x7f946000b040 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 msgr2=0x7f946414ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.589+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94640b17a0 tx=0x7f9454004770 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 shutdown_connections 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9448066730 0x7f9448068bf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94640b19e0 0x7f94640a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94641539c0 0x7f9464158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 --2- 172.21.15.179:0/3363776787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f946406bc50 0x7f946414ee80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 >> 172.21.15.179:0/3363776787 conn(0x7f94640ce9d0 msgr2=0x7f94640cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 shutdown_connections 2024-05-08T22:34:35.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.593+0000 7f946a069700 1 -- 172.21.15.179:0/3363776787 wait complete. 2024-05-08T22:34:35.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T22:34:35.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 Processor -- start 2024-05-08T22:34:35.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 -- start start 2024-05-08T22:34:35.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc50c0b4f70 0x7fc50c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c0cd6d0 con 0x7fc50c15cf60 2024-05-08T22:34:35.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c161b30 con 0x7fc50c15d340 2024-05-08T22:34:35.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c0cd970 con 0x7fc50c0b4f70 2024-05-08T22:34:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc509d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc50c0b4f70 0x7fc50c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50a59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc50c15cf60 0x7fc50c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38160/0 (socket says 172.21.15.179:38160) 2024-05-08T22:34:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50a59c700 1 -- 172.21.15.179:0/1209291669 learned_addr learned my addr 172.21.15.179:0/1209291669 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671736871 0 0) 0x7fc50c0cd6d0 con 0x7fc50c15cf60 2024-05-08T22:34:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f80035d0 con 0x7fc50c15cf60 2024-05-08T22:34:35.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115248007 0 0) 0x7fc50c161b30 con 0x7fc50c15d340 2024-05-08T22:34:35.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc509d9b700 1 -- 172.21.15.179:0/1209291669 >> v1:172.21.15.195:6789/0 conn(0x7fc50c15d340 legacy=0x7fc50c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc509d9b700 1 -- 172.21.15.179:0/1209291669 >> v1:172.21.15.179:6789/0 conn(0x7fc50c15cf60 legacy=0x7fc50c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc509d9b700 1 -- 172.21.15.179:0/1209291669 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0cdc50 con 0x7fc50c0b4f70 2024-05-08T22:34:35.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc509d9b700 1 --2- 172.21.15.179:0/1209291669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0b4f70 0x7fc50c0ce450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009850 tx=0x7fc4f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fca112a17331ea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4011830 con 0x7fc50c0b4f70 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f4011990 con 0x7fc50c0b4f70 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.721+0000 7fc50959a700 1 -- 172.21.15.179:0/1209291669 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4011d40 con 0x7fc50c0b4f70 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- 172.21.15.179:0/1209291669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0b4f70 msgr2=0x7fc50c0ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 --2- 172.21.15.179:0/1209291669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0b4f70 0x7fc50c0ce450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009850 tx=0x7fc4f4009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- 172.21.15.179:0/1209291669 shutdown_connections 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 --2- 172.21.15.179:0/1209291669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c0b4f70 0x7fc50c0ce450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- 172.21.15.179:0/1209291669 >> 172.21.15.179:0/1209291669 conn(0x7fc50c0cf860 msgr2=0x7fc50c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:35.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- 172.21.15.179:0/1209291669 shutdown_connections 2024-05-08T22:34:35.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- 172.21.15.179:0/1209291669 wait complete. 2024-05-08T22:34:35.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 Processor -- start 2024-05-08T22:34:35.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.725+0000 7fc50b59e700 1 -- start start 2024-05-08T22:34:35.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15cf60 0x7fc50c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c06a320 0x7fc50c06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c06acf0 con 0x7fc50c15d340 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0cd230 con 0x7fc50c06a320 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0cacf0 con 0x7fc50c15cf60 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40864/0 (socket says 172.21.15.179:40864) 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 -- 172.21.15.179:0/3969483172 learned_addr learned my addr 172.21.15.179:0/3969483172 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50a59c700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15cf60 0x7fc50c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50ad9d700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c06a320 0x7fc50c06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 -- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15cf60 msgr2=0x7fc50c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15cf60 0x7fc50c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 -- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c06a320 msgr2=0x7fc50c06a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c06a320 0x7fc50c06a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 -- 172.21.15.179:0/3969483172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f40094b0 con 0x7fc50c15d340 2024-05-08T22:34:35.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc509d9b700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4013000 tx=0x7fc4f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f401a950 con 0x7fc50c15d340 2024-05-08T22:34:35.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f401f070 con 0x7fc50c15d340 2024-05-08T22:34:35.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c0cafd0 con 0x7fc50c15d340 2024-05-08T22:34:35.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f401ad30 con 0x7fc50c15d340 2024-05-08T22:34:35.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c0cb590 con 0x7fc50c15d340 2024-05-08T22:34:35.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4f401b430 con 0x7fc50c15d340 2024-05-08T22:34:35.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.729+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c0c7ff0 con 0x7fc50c15d340 2024-05-08T22:34:35.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.733+0000 7fc502ffd700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 0x7fc4ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:35.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.733+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc4f4084fe0 con 0x7fc50c15d340 2024-05-08T22:34:35.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.733+0000 7fc50a59c700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 0x7fc4ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:35.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.733+0000 7fc50a59c700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 0x7fc4ec068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc504005cb0 tx=0x7fc504009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:35.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.741+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc50c0c7ff0 con 0x7fc50c15d340 2024-05-08T22:34:35.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.969+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc50c0c7ff0 con 0x7fc50c15d340 2024-05-08T22:34:35.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc502ffd700 1 -- 172.21.15.179:0/3969483172 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc50c0c7ff0 con 0x7fc50c15d340 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 msgr2=0x7fc4ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 0x7fc4ec068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc504005cb0 tx=0x7fc504009040 comp rx=0 tx=0).stop 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 msgr2=0x7fc50c1572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4013000 tx=0x7fc4f4004770 comp rx=0 tx=0).stop 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 shutdown_connections 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc4ec066730 0x7fc4ec068bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc50c06a320 0x7fc50c06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50c15d340 0x7fc50c1572a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 --2- 172.21.15.179:0/3969483172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc50c15cf60 0x7fc50c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:35.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 >> 172.21.15.179:0/3969483172 conn(0x7fc50c0cf860 msgr2=0x7fc50c0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:35.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 shutdown_connections 2024-05-08T22:34:35.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:35.977+0000 7fc50b59e700 1 -- 172.21.15.179:0/3969483172 wait complete. 2024-05-08T22:34:35.994 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T22:34:36.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.109+0000 7fbaa8d29700 1 Processor -- start 2024-05-08T22:34:36.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.109+0000 7fbaa8d29700 1 -- start start 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa4159520 0x7fbaa40c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbaa406b440 0x7fbaa406b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbaa406bdc0 0x7fbaa40b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40c5d40 con 0x7fbaa406b440 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa406ce60 con 0x7fbaa406bdc0 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa8d29700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa40b1970 con 0x7fbaa4159520 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbaa406b440 0x7fbaa406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbaa406b440 0x7fbaa406b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40874/0 (socket says 172.21.15.179:40874) 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 -- 172.21.15.179:0/2993615282 learned_addr learned my addr 172.21.15.179:0/2993615282 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa3fff700 1 --2- 172.21.15.179:0/2993615282 >> v2:172.21.15.195:3300/0 conn(0x7fbaa406bdc0 0x7fbaa40b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa37fe700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa4159520 0x7fbaa40c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 -- 172.21.15.179:0/2993615282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa4159520 msgr2=0x7fbaa40c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa4159520 0x7fbaa40c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 -- 172.21.15.179:0/2993615282 >> v2:172.21.15.195:3300/0 conn(0x7fbaa406bdc0 msgr2=0x7fbaa40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 --2- 172.21.15.179:0/2993615282 >> v2:172.21.15.195:3300/0 conn(0x7fbaa406bdc0 0x7fbaa40b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 -- 172.21.15.179:0/2993615282 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40b1c50 con 0x7fbaa406b440 2024-05-08T22:34:36.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fba9affd700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa406b440 0x7fbaa406b880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbaa409d4b0 tx=0x7fba8c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed194ec87ca62b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa2ffd700 1 -- 172.21.15.179:0/2993615282 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c011830 con 0x7fbaa406b440 2024-05-08T22:34:36.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa2ffd700 1 -- 172.21.15.179:0/2993615282 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c011990 con 0x7fbaa406b440 2024-05-08T22:34:36.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.113+0000 7fbaa2ffd700 1 -- 172.21.15.179:0/2993615282 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c011d20 con 0x7fbaa406b440 2024-05-08T22:34:36.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- 172.21.15.179:0/2993615282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa406b440 msgr2=0x7fbaa406b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa406b440 0x7fbaa406b880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbaa409d4b0 tx=0x7fba8c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- 172.21.15.179:0/2993615282 shutdown_connections 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/2993615282 >> v2:172.21.15.195:3300/0 conn(0x7fbaa406bdc0 0x7fbaa40b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa406b440 0x7fbaa406b880 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/2993615282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa4159520 0x7fbaa40c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- 172.21.15.179:0/2993615282 >> 172.21.15.179:0/2993615282 conn(0x7fbaa4098a70 msgr2=0x7fbaa40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- 172.21.15.179:0/2993615282 shutdown_connections 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- 172.21.15.179:0/2993615282 wait complete. 2024-05-08T22:34:36.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 Processor -- start 2024-05-08T22:34:36.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.117+0000 7fbaa8d29700 1 -- start start 2024-05-08T22:34:36.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaa406b440 0x7fbaa4147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 0x7fbaa415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41e2eb0 con 0x7fbaa4159520 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41e2230 con 0x7fbaa406b440 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41e2510 con 0x7fbaa406bdc0 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40876/0 (socket says 172.21.15.179:40876) 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 -- 172.21.15.179:0/846855878 learned_addr learned my addr 172.21.15.179:0/846855878 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa37fe700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaa406b440 0x7fbaa4147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fba9affd700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 0x7fbaa415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 -- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 msgr2=0x7fbaa415ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 0x7fbaa415ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 -- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaa406b440 msgr2=0x7fbaa4147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaa406b440 0x7fbaa4147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 -- 172.21.15.179:0/846855878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009570 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fba9affd700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 0x7fbaa415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa3fff700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00eed0 tx=0x7fba9c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c010040 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c00bd30 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa41e27f0 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00bea0 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa4148100 con 0x7fbaa4159520 2024-05-08T22:34:36.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.121+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa41e2ff0 con 0x7fbaa4159520 2024-05-08T22:34:36.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.125+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba9c014620 con 0x7fbaa4159520 2024-05-08T22:34:36.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.125+0000 7fbaa0ff9700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 0x7fba84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.125+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fba9c084610 con 0x7fbaa4159520 2024-05-08T22:34:36.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.125+0000 7fbaa37fe700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 0x7fba84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.125+0000 7fbaa37fe700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 0x7fba84068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba9400acf0 tx=0x7fba9400a350 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.129+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbaa41e2ff0 con 0x7fbaa4159520 2024-05-08T22:34:36.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.365+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbaa41e2ff0 con 0x7fbaa4159520 2024-05-08T22:34:36.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.373+0000 7fbaa0ff9700 1 -- 172.21.15.179:0/846855878 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbaa41e2ff0 con 0x7fbaa4159520 2024-05-08T22:34:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 msgr2=0x7fba84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 0x7fba84068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba9400acf0 tx=0x7fba9400a350 comp rx=0 tx=0).stop 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 msgr2=0x7fbaa41e32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00eed0 tx=0x7fba9c00b390 comp rx=0 tx=0).stop 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 shutdown_connections 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fba84066730 0x7fba84068bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbaa4159520 0x7fbaa41e32d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbaa406bdc0 0x7fbaa415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 --2- 172.21.15.179:0/846855878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbaa406b440 0x7fbaa4147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 >> 172.21.15.179:0/846855878 conn(0x7fbaa4098a70 msgr2=0x7fbaa40acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 shutdown_connections 2024-05-08T22:34:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.377+0000 7fbaa8d29700 1 -- 172.21.15.179:0/846855878 wait complete. 2024-05-08T22:34:36.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T22:34:36.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 Processor -- start 2024-05-08T22:34:36.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- start start 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb71815e150 0x7fb718160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 0x7fb7180b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71815e5b0 con 0x7fb71806f640 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7180b2ac0 con 0x7fb71815e150 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb718161460 con 0x7fb7180b31f0 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb71815e150 0x7fb718160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb71815e150 0x7fb718160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40884/0 (socket says 172.21.15.179:40884) 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 -- 172.21.15.179:0/3081525448 learned_addr learned my addr 172.21.15.179:0/3081525448 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71ca4c700 1 --2- 172.21.15.179:0/3081525448 >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 0x7fb7180b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 -- 172.21.15.179:0/3081525448 >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 msgr2=0x7fb7180b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 --2- 172.21.15.179:0/3081525448 >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 0x7fb7180b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 -- 172.21.15.179:0/3081525448 >> v1:172.21.15.195:6789/0 conn(0x7fb71806f640 legacy=0x7fb71806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 -- 172.21.15.179:0/3081525448 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718161740 con 0x7fb71815e150 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71ca4c700 1 --2- 172.21.15.179:0/3081525448 >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 0x7fb7180b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb7177fe700 1 --2- 172.21.15.179:0/3081525448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718160c30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7180d0830 tx=0x7fb70c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf328e88a3901fea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb716ffd700 1 -- 172.21.15.179:0/3081525448 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00b8a0 con 0x7fb71815e150 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb716ffd700 1 -- 172.21.15.179:0/3081525448 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00ba00 con 0x7fb71815e150 2024-05-08T22:34:36.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb716ffd700 1 -- 172.21.15.179:0/3081525448 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00bd90 con 0x7fb71815e150 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- 172.21.15.179:0/3081525448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 msgr2=0x7fb718160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3081525448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718160c30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7180d0830 tx=0x7fb70c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- 172.21.15.179:0/3081525448 shutdown_connections 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3081525448 >> v2:172.21.15.195:3300/0 conn(0x7fb7180b31f0 0x7fb7180b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3081525448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718160c30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- 172.21.15.179:0/3081525448 >> 172.21.15.179:0/3081525448 conn(0x7fb7180cfb50 msgr2=0x7fb7180cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- 172.21.15.179:0/3081525448 shutdown_connections 2024-05-08T22:34:36.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.505+0000 7fb71d24d700 1 -- 172.21.15.179:0/3081525448 wait complete. 2024-05-08T22:34:36.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 Processor -- start 2024-05-08T22:34:36.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 -- start start 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 0x7fb71806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 0x7fb71814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71814cbf0 con 0x7fb71815e150 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7181580a0 con 0x7fb7180b31f0 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71d24d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb718158380 con 0x7fb71806f640 2024-05-08T22:34:36.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71ca4c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71ca4c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40892/0 (socket says 172.21.15.179:40892) 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb71ca4c700 1 -- 172.21.15.179:0/3075585045 learned_addr learned my addr 172.21.15.179:0/3075585045 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb717fff700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 0x7fb71806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.509+0000 7fb7177fe700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 0x7fb71814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 -- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 msgr2=0x7fb71806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 0x7fb71806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 -- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 msgr2=0x7fb71814c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 0x7fb71814c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 -- 172.21.15.179:0/3075585045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c009570 con 0x7fb71815e150 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb717fff700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 0x7fb71806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb7177fe700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 0x7fb71814c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71ca4c700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb710007ef0 tx=0x7fb71000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000de60 con 0x7fb71815e150 2024-05-08T22:34:36.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71000f040 con 0x7fb71815e150 2024-05-08T22:34:36.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb718158660 con 0x7fb71815e150 2024-05-08T22:34:36.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710003c80 con 0x7fb71815e150 2024-05-08T22:34:36.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb718158c20 con 0x7fb71815e150 2024-05-08T22:34:36.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb710004030 con 0x7fb71815e150 2024-05-08T22:34:36.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71804e700 con 0x7fb71815e150 2024-05-08T22:34:36.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.513+0000 7fb714ff9700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 0x7fb704068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.517+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb7100840b0 con 0x7fb71815e150 2024-05-08T22:34:36.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.517+0000 7fb717fff700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 0x7fb704068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.517+0000 7fb717fff700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 0x7fb704068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7181f6310 tx=0x7fb708009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.521+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb7100516c0 con 0x7fb71815e150 2024-05-08T22:34:36.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.753+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb7180cf8c0 con 0x7fb71815e150 2024-05-08T22:34:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.757+0000 7fb714ff9700 1 -- 172.21.15.179:0/3075585045 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb710054ce0 con 0x7fb71815e150 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.757+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 msgr2=0x7fb704068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 0x7fb704068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7181f6310 tx=0x7fb708009040 comp rx=0 tx=0).stop 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 msgr2=0x7fb718159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb710007ef0 tx=0x7fb71000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 shutdown_connections 2024-05-08T22:34:36.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb704066730 0x7fb704068bf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7180b31f0 0x7fb71814c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb71815e150 0x7fb718159140 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 --2- 172.21.15.179:0/3075585045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71806f640 0x7fb71806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 >> 172.21.15.179:0/3075585045 conn(0x7fb7180cfb50 msgr2=0x7fb7180cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 shutdown_connections 2024-05-08T22:34:36.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.761+0000 7fb71d24d700 1 -- 172.21.15.179:0/3075585045 wait complete. 2024-05-08T22:34:36.775 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T22:34:36.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 Processor -- start 2024-05-08T22:34:36.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 -- start start 2024-05-08T22:34:36.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 0x7fa2a00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 0x7fa2a00ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa2a015e9e0 0x7fa2a015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a00cbb50 con 0x7fa2a015e9e0 2024-05-08T22:34:36.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a00b3370 con 0x7fa2a00b2a50 2024-05-08T22:34:36.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.893+0000 7fa29f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00caed0 con 0x7fa2a00ca360 2024-05-08T22:34:36.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa2a015e9e0 0x7fa2a015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa2a015e9e0 0x7fa2a015ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40902/0 (socket says 172.21.15.179:40902) 2024-05-08T22:34:36.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 -- 172.21.15.179:0/2857635751 learned_addr learned my addr 172.21.15.179:0/2857635751 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29e59c700 1 --2- 172.21.15.179:0/2857635751 >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 0x7fa2a00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa297fff700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 0x7fa2a00ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 -- 172.21.15.179:0/2857635751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 msgr2=0x7fa2a00ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:36.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 0x7fa2a00ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 -- 172.21.15.179:0/2857635751 >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 msgr2=0x7fa2a00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:36.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 --2- 172.21.15.179:0/2857635751 >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 0x7fa2a00b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 -- 172.21.15.179:0/2857635751 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00cb1b0 con 0x7fa2a015e9e0 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29e59c700 1 --2- 172.21.15.179:0/2857635751 >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 0x7fa2a00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa297fff700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 0x7fa2a00ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29ed9d700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a015e9e0 0x7fa2a015ee70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00668d0 tx=0x7fa29800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbe4a75f921aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29dd9b700 1 -- 172.21.15.179:0/2857635751 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800e070 con 0x7fa2a015e9e0 2024-05-08T22:34:36.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29dd9b700 1 -- 172.21.15.179:0/2857635751 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29800b910 con 0x7fa2a015e9e0 2024-05-08T22:34:36.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29dd9b700 1 -- 172.21.15.179:0/2857635751 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800bcc0 con 0x7fa2a015e9e0 2024-05-08T22:34:36.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 -- 172.21.15.179:0/2857635751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a015e9e0 msgr2=0x7fa2a015ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a015e9e0 0x7fa2a015ee70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00668d0 tx=0x7fa29800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:36.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 -- 172.21.15.179:0/2857635751 shutdown_connections 2024-05-08T22:34:36.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a015e9e0 0x7fa2a015ee70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 --2- 172.21.15.179:0/2857635751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a00ca360 0x7fa2a00ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 --2- 172.21.15.179:0/2857635751 >> v2:172.21.15.195:3300/0 conn(0x7fa2a00b2a50 0x7fa2a00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 -- 172.21.15.179:0/2857635751 >> 172.21.15.179:0/2857635751 conn(0x7fa2a00cfb00 msgr2=0x7fa2a00cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 -- 172.21.15.179:0/2857635751 shutdown_connections 2024-05-08T22:34:36.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.897+0000 7fa29f59e700 1 -- 172.21.15.179:0/2857635751 wait complete. 2024-05-08T22:34:36.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 Processor -- start 2024-05-08T22:34:36.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 -- start start 2024-05-08T22:34:36.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a00b2a50 0x7fa2a00c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a015e9e0 0x7fa2a0152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a0137d50 con 0x7fa2a00ca360 2024-05-08T22:34:36.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00c7c50 con 0x7fa2a00b2a50 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00c7040 con 0x7fa2a015e9e0 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40904/0 (socket says 172.21.15.179:40904) 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 -- 172.21.15.179:0/1657698971 learned_addr learned my addr 172.21.15.179:0/1657698971 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29e59c700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a00b2a50 0x7fa2a00c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29ed9d700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a015e9e0 0x7fa2a0152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 -- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a015e9e0 msgr2=0x7fa2a0152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a015e9e0 0x7fa2a0152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 -- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a00b2a50 msgr2=0x7fa2a00c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a00b2a50 0x7fa2a00c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 -- 172.21.15.179:0/1657698971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900094b0 con 0x7fa2a00ca360 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa297fff700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa28800ebe0 tx=0x7fa28800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28800c850 con 0x7fa2a00ca360 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28800c9b0 con 0x7fa2a00ca360 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28800cd40 con 0x7fa2a00ca360 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.901+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880094b0 con 0x7fa2a00ca360 2024-05-08T22:34:36.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa298009b30 con 0x7fa2a00ca360 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a00b3e50 con 0x7fa2a00ca360 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa288013470 con 0x7fa2a00ca360 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa296ffd700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 0x7fa280068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa288083f40 con 0x7fa2a00ca360 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa29e59c700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 0x7fa280068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.905+0000 7fa29e59c700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 0x7fa280068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa290009480 tx=0x7fa290009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:36.913+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa288051550 con 0x7fa2a00ca360 2024-05-08T22:34:37.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.157+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa2a004e700 con 0x7fa2a00ca360 2024-05-08T22:34:37.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.165+0000 7fa296ffd700 1 -- 172.21.15.179:0/1657698971 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa28800cea0 con 0x7fa2a00ca360 2024-05-08T22:34:37.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 msgr2=0x7fa280068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 0x7fa280068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa290009480 tx=0x7fa290009240 comp rx=0 tx=0).stop 2024-05-08T22:34:37.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 msgr2=0x7fa2a006c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa28800ebe0 tx=0x7fa28800eef0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 shutdown_connections 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa280066730 0x7fa280068bf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2a015e9e0 0x7fa2a0152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a00ca360 0x7fa2a006c930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 --2- 172.21.15.179:0/1657698971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a00b2a50 0x7fa2a00c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 >> 172.21.15.179:0/1657698971 conn(0x7fa2a00cfb00 msgr2=0x7fa2a01426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 shutdown_connections 2024-05-08T22:34:37.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.169+0000 7fa29f59e700 1 -- 172.21.15.179:0/1657698971 wait complete. 2024-05-08T22:34:37.185 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T22:34:37.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 Processor -- start 2024-05-08T22:34:37.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- start start 2024-05-08T22:34:37.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f912806e180 0x7f912806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91280b25d0 0x7f91280b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280cc340 con 0x7f912806dda0 2024-05-08T22:34:37.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912806b540 con 0x7f91280b25d0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912806ac50 con 0x7f912806e180 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91280b25d0 0x7f91280b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91280b25d0 0x7f91280b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40920/0 (socket says 172.21.15.179:40920) 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 -- 172.21.15.179:0/226730329 learned_addr learned my addr 172.21.15.179:0/226730329 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9126ffd700 1 --2- 172.21.15.179:0/226730329 >> v2:172.21.15.195:3300/0 conn(0x7f912806e180 0x7f912806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 -- 172.21.15.179:0/226730329 >> v2:172.21.15.195:3300/0 conn(0x7f912806e180 msgr2=0x7f912806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 --2- 172.21.15.179:0/226730329 >> v2:172.21.15.195:3300/0 conn(0x7f912806e180 0x7f912806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 -- 172.21.15.179:0/226730329 >> v1:172.21.15.195:6789/0 conn(0x7f912806dda0 legacy=0x7f91280cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 -- 172.21.15.179:0/226730329 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912806aef0 con 0x7f91280b25d0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f9127fff700 1 --2- 172.21.15.179:0/226730329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91280b2a00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce930 tx=0x7f9118009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f60b44f8e3970f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f91267fc700 1 -- 172.21.15.179:0/226730329 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800b8a0 con 0x7f91280b25d0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f91267fc700 1 -- 172.21.15.179:0/226730329 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800ba00 con 0x7f91280b25d0 2024-05-08T22:34:37.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f91267fc700 1 -- 172.21.15.179:0/226730329 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800bd90 con 0x7f91280b25d0 2024-05-08T22:34:37.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- 172.21.15.179:0/226730329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 msgr2=0x7f91280b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 --2- 172.21.15.179:0/226730329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91280b2a00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce930 tx=0x7f9118009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.301+0000 7f912cad1700 1 -- 172.21.15.179:0/226730329 shutdown_connections 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 --2- 172.21.15.179:0/226730329 >> v2:172.21.15.195:3300/0 conn(0x7f912806e180 0x7f912806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 --2- 172.21.15.179:0/226730329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91280b2a00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 -- 172.21.15.179:0/226730329 >> 172.21.15.179:0/226730329 conn(0x7f91280ce960 msgr2=0x7f91280ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 -- 172.21.15.179:0/226730329 shutdown_connections 2024-05-08T22:34:37.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 -- 172.21.15.179:0/226730329 wait complete. 2024-05-08T22:34:37.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 Processor -- start 2024-05-08T22:34:37.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.305+0000 7f912cad1700 1 -- start start 2024-05-08T22:34:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912806dda0 0x7f9128156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912806e180 0x7f9128159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91281f7ca0 con 0x7f91280b25d0 2024-05-08T22:34:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91281f7370 con 0x7f912806dda0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91281f7650 con 0x7f912806e180 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40930/0 (socket says 172.21.15.179:40930) 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 -- 172.21.15.179:0/3831583798 learned_addr learned my addr 172.21.15.179:0/3831583798 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9126ffd700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912806e180 0x7f9128159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f91277fe700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912806dda0 0x7f9128156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 -- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912806e180 msgr2=0x7f9128159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912806e180 0x7f9128159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 -- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912806dda0 msgr2=0x7f9128156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912806dda0 0x7f9128156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 -- 172.21.15.179:0/3831583798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009570 con 0x7f91280b25d0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f9127fff700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9118014040 tx=0x7f9118004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180038c0 con 0x7f91280b25d0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911801e070 con 0x7f91280b25d0 2024-05-08T22:34:37.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91281f7930 con 0x7f91280b25d0 2024-05-08T22:34:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118003c80 con 0x7f91280b25d0 2024-05-08T22:34:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91281f7020 con 0x7f91280b25d0 2024-05-08T22:34:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91281f7e10 con 0x7f91280b25d0 2024-05-08T22:34:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.309+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9118011420 con 0x7f91280b25d0 2024-05-08T22:34:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.313+0000 7f911ffff700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 0x7f9108068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.313+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9118020070 con 0x7f91280b25d0 2024-05-08T22:34:37.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.313+0000 7f91277fe700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 0x7f9108068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.313+0000 7f91277fe700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 0x7f9108068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f912815a590 tx=0x7f9110009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.317+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9118052980 con 0x7f91280b25d0 2024-05-08T22:34:37.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.553+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f912804e700 con 0x7f91280b25d0 2024-05-08T22:34:37.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.561+0000 7f911ffff700 1 -- 172.21.15.179:0/3831583798 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9118055fa0 con 0x7f91280b25d0 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 msgr2=0x7f9108068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 0x7f9108068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f912815a590 tx=0x7f9110009040 comp rx=0 tx=0).stop 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 msgr2=0x7f91281595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9118014040 tx=0x7f9118004770 comp rx=0 tx=0).stop 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 shutdown_connections 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9108066760 0x7f9108068c20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91280b25d0 0x7f91281595a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f912806e180 0x7f9128159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 --2- 172.21.15.179:0/3831583798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f912806dda0 0x7f9128156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 >> 172.21.15.179:0/3831583798 conn(0x7f91280ce960 msgr2=0x7f91280cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:37.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 shutdown_connections 2024-05-08T22:34:37.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.565+0000 7f912cad1700 1 -- 172.21.15.179:0/3831583798 wait complete. 2024-05-08T22:34:37.581 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T22:34:37.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 Processor -- start 2024-05-08T22:34:37.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- start start 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b940ad9d0 0x7f9b9415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940b37a0 con 0x7f9b940b3180 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940b3a80 con 0x7f9b9415eef0 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9415f380 con 0x7f9b940ad9d0 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b93fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9b940b3180 0x7f9b940cb9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38176/0 (socket says 172.21.15.179:38176) 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b93fff700 1 -- 172.21.15.179:0/765380376 learned_addr learned my addr 172.21.15.179:0/765380376 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:37.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b937fe700 1 --2- 172.21.15.179:0/765380376 >> v2:172.21.15.195:3300/0 conn(0x7f9b940ad9d0 0x7f9b9415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205200707 0 0) 0x7f9b940b37a0 con 0x7f9b940b3180 2024-05-08T22:34:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b800035d0 con 0x7f9b940b3180 2024-05-08T22:34:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b937fe700 1 -- 172.21.15.179:0/765380376 >> v1:172.21.15.195:6789/0 conn(0x7f9b9415eef0 legacy=0x7f9b940b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457718619 0 0) 0x7f9b800035d0 con 0x7f9b940b3180 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b937fe700 1 -- 172.21.15.179:0/765380376 >> v1:172.21.15.179:6789/0 conn(0x7f9b940b3180 legacy=0x7f9b940cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b937fe700 1 -- 172.21.15.179:0/765380376 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940b3d60 con 0x7f9b940ad9d0 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b937fe700 1 --2- 172.21.15.179:0/765380376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b9415e9b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009850 tx=0x7f9b84009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f22bbaec6173c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0019f0 con 0x7f9b940ad9d0 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84004030 con 0x7f9b940ad9d0 2024-05-08T22:34:37.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b92ffd700 1 -- 172.21.15.179:0/765380376 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840043a0 con 0x7f9b940ad9d0 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- 172.21.15.179:0/765380376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 msgr2=0x7f9b9415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 --2- 172.21.15.179:0/765380376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b9415e9b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009850 tx=0x7f9b84009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- 172.21.15.179:0/765380376 shutdown_connections 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 --2- 172.21.15.179:0/765380376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b9415e9b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- 172.21.15.179:0/765380376 >> 172.21.15.179:0/765380376 conn(0x7f9b940ce960 msgr2=0x7f9b940ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- 172.21.15.179:0/765380376 shutdown_connections 2024-05-08T22:34:37.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.697+0000 7f9b98882700 1 -- 172.21.15.179:0/765380376 wait complete. 2024-05-08T22:34:37.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 Processor -- start 2024-05-08T22:34:37.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 -- start start 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b94140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b940b3180 0x7f9b94153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941414a0 con 0x7f9b9415eef0 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94158d50 con 0x7f9b940ad9d0 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b98882700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94141820 con 0x7f9b940b3180 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40944/0 (socket says 172.21.15.179:40944) 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.701+0000 7f9b93fff700 1 -- 172.21.15.179:0/1688709175 learned_addr learned my addr 172.21.15.179:0/1688709175 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b937fe700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b94140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b8affd700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b940b3180 0x7f9b94153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 -- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b940b3180 msgr2=0x7f9b94153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b940b3180 0x7f9b94153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 -- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 msgr2=0x7f9b94140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b94140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 -- 172.21.15.179:0/1688709175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840094b0 con 0x7f9b9415eef0 2024-05-08T22:34:37.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b93fff700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c001ba0 tx=0x7f9b8c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00aac0 con 0x7f9b9415eef0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00ac20 con 0x7f9b9415eef0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940adf30 con 0x7f9b9415eef0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0070d0 con 0x7f9b9415eef0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b94141de0 con 0x7f9b9415eef0 2024-05-08T22:34:37.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b94142300 con 0x7f9b9415eef0 2024-05-08T22:34:37.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b8c007230 con 0x7f9b9415eef0 2024-05-08T22:34:37.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 0x7f9b74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:37.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.705+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9b8c083d70 con 0x7f9b9415eef0 2024-05-08T22:34:37.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.709+0000 7f9b937fe700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 0x7f9b74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:37.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.709+0000 7f9b937fe700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 0x7f9b74068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9b840090b0 tx=0x7f9b84008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:37.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.713+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9b94142300 con 0x7f9b9415eef0 2024-05-08T22:34:37.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.945+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f9b94142300 con 0x7f9b9415eef0 2024-05-08T22:34:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.953+0000 7f9b90ff9700 1 -- 172.21.15.179:0/1688709175 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v13) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9b94142300 con 0x7f9b9415eef0 2024-05-08T22:34:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 msgr2=0x7f9b74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 0x7f9b74068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9b840090b0 tx=0x7f9b84008030 comp rx=0 tx=0).stop 2024-05-08T22:34:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 msgr2=0x7f9b94153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c001ba0 tx=0x7f9b8c00cf50 comp rx=0 tx=0).stop 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 shutdown_connections 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b74066730 0x7f9b74068bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b9415eef0 0x7f9b94153590 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b940b3180 0x7f9b94153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 --2- 172.21.15.179:0/1688709175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b940ad9d0 0x7f9b94140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 >> 172.21.15.179:0/1688709175 conn(0x7f9b940ce960 msgr2=0x7f9b9415db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 shutdown_connections 2024-05-08T22:34:37.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:37.957+0000 7f9b98882700 1 -- 172.21.15.179:0/1688709175 wait complete. 2024-05-08T22:34:37.973 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 Processor -- start 2024-05-08T22:34:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 -- start start 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12ec15d580 0x7f12ec0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12ec161640 0x7f12ec0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec0d2640 con 0x7f12ec0d18a0 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec06b3a0 con 0x7f12ec15d580 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12eb59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec0b6750 con 0x7f12ec161640 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12ec15d580 0x7f12ec0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e9d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f12ec0d18a0 0x7f12ec160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38188/0 (socket says 172.21.15.179:38188) 2024-05-08T22:34:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e9d9b700 1 -- 172.21.15.179:0/2140007840 learned_addr learned my addr 172.21.15.179:0/2140007840 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ead9d700 1 --2- 172.21.15.179:0/2140007840 >> v2:172.21.15.195:3300/0 conn(0x7f12ec161640 0x7f12ec0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651899774 0 0) 0x7f12ec0d2640 con 0x7f12ec0d18a0 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d40035d0 con 0x7f12ec0d18a0 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 -- 172.21.15.179:0/2140007840 >> v2:172.21.15.195:3300/0 conn(0x7f12ec161640 msgr2=0x7f12ec0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 --2- 172.21.15.179:0/2140007840 >> v2:172.21.15.195:3300/0 conn(0x7f12ec161640 0x7f12ec0d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 -- 172.21.15.179:0/2140007840 >> v1:172.21.15.179:6789/0 conn(0x7f12ec0d18a0 legacy=0x7f12ec160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 -- 172.21.15.179:0/2140007840 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0d2360 con 0x7f12ec15d580 2024-05-08T22:34:38.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002207756 0 0) 0x7f12d40035d0 con 0x7f12ec0d18a0 2024-05-08T22:34:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12ea59c700 1 --2- 172.21.15.179:0/2140007840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec15d580 0x7f12ec0b6210 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0d2270 tx=0x7f12dc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06270dbdc37f5b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011830 con 0x7f12ec15d580 2024-05-08T22:34:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc011990 con 0x7f12ec15d580 2024-05-08T22:34:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.081+0000 7f12e959a700 1 -- 172.21.15.179:0/2140007840 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011d20 con 0x7f12ec15d580 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- 172.21.15.179:0/2140007840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec15d580 msgr2=0x7f12ec0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- 172.21.15.179:0/2140007840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec15d580 0x7f12ec0b6210 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0d2270 tx=0x7f12dc009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- 172.21.15.179:0/2140007840 shutdown_connections 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- 172.21.15.179:0/2140007840 >> v2:172.21.15.195:3300/0 conn(0x7f12ec161640 0x7f12ec0d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- 172.21.15.179:0/2140007840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec15d580 0x7f12ec0b6210 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- 172.21.15.179:0/2140007840 >> 172.21.15.179:0/2140007840 conn(0x7f12ec0d2d90 msgr2=0x7f12ec0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- 172.21.15.179:0/2140007840 shutdown_connections 2024-05-08T22:34:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- 172.21.15.179:0/2140007840 wait complete. 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 Processor -- start 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- start start 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ec0d18a0 0x7f12ec0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ec15d580 0x7f12ec156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0b3da0 con 0x7f12ec161640 2024-05-08T22:34:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1f3ca0 con 0x7f12ec15d580 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12eb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1f36e0 con 0x7f12ec0d18a0 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40952/0 (socket says 172.21.15.179:40952) 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 -- 172.21.15.179:0/4148694620 learned_addr learned my addr 172.21.15.179:0/4148694620 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12e9d9b700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ec15d580 0x7f12ec156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ea59c700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ec0d18a0 0x7f12ec0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 -- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ec0d18a0 msgr2=0x7f12ec0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ec0d18a0 0x7f12ec0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 -- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ec15d580 msgr2=0x7f12ec156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ec15d580 0x7f12ec156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.085+0000 7f12ead9d700 1 -- 172.21.15.179:0/4148694620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0094b0 con 0x7f12ec161640 2024-05-08T22:34:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12ead9d700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f12e400ef90 tx=0x7f12e4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4009b20 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec1f39c0 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400f040 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400cc70 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec0b4700 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec06b810 con 0x7f12ec161640 2024-05-08T22:34:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12e4014480 con 0x7f12ec161640 2024-05-08T22:34:38.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 0x7f12cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f12e40848f0 con 0x7f12ec161640 2024-05-08T22:34:38.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.089+0000 7f12ea59c700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 0x7f12cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.093+0000 7f12ea59c700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 0x7f12cc068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009fd0 tx=0x7f12dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.097+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f12ec06b810 con 0x7f12ec161640 2024-05-08T22:34:38.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.273+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12ec0b3ee0 con 0x7f12ec161640 2024-05-08T22:34:38.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.273+0000 7f12daffd700 1 -- 172.21.15.179:0/4148694620 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12642 (secure 0 0 0) 0x7f12ec0b3ee0 con 0x7f12ec161640 2024-05-08T22:34:38.276 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:38.277 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":37,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:35.155070+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 msgr2=0x7f12cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 0x7f12cc068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009fd0 tx=0x7f12dc008030 comp rx=0 tx=0).stop 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 msgr2=0x7f12ec1f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f12e400ef90 tx=0x7f12e4009f60 comp rx=0 tx=0).stop 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 shutdown_connections 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12cc066730 0x7f12cc068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ec15d580 0x7f12ec156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12ec161640 0x7f12ec1f2c60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 --2- 172.21.15.179:0/4148694620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ec0d18a0 0x7f12ec0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 >> 172.21.15.179:0/4148694620 conn(0x7f12ec0d2d90 msgr2=0x7f12ec06bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 shutdown_connections 2024-05-08T22:34:38.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.277+0000 7f12eb59e700 1 -- 172.21.15.179:0/4148694620 wait complete. 2024-05-08T22:34:38.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:34:38.292 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:38.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 Processor -- start 2024-05-08T22:34:38.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 -- start start 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8be4142b80 0x7f8be40ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be40ca6e0 0x7f8be40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be40cf070 con 0x7f8be4143170 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be40cab20 con 0x7f8be4142b80 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be93b0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be4142f60 con 0x7f8be40ca6e0 2024-05-08T22:34:38.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8be4143170 0x7f8be406c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38198/0 (socket says 172.21.15.179:38198) 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be37fe700 1 -- 172.21.15.179:0/1799606862 learned_addr learned my addr 172.21.15.179:0/1799606862 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be3fff700 1 --2- 172.21.15.179:0/1799606862 >> v2:172.21.15.195:3300/0 conn(0x7f8be4142b80 0x7f8be40ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be8baf700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be40ca6e0 0x7f8be40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be2ffd700 1 -- 172.21.15.179:0/1799606862 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899941607 0 0) 0x7f8be40cf070 con 0x7f8be4143170 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be2ffd700 1 -- 172.21.15.179:0/1799606862 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd00035d0 con 0x7f8be4143170 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be3fff700 1 -- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be40ca6e0 msgr2=0x7f8be40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be3fff700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be40ca6e0 0x7f8be40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be3fff700 1 -- 172.21.15.179:0/1799606862 >> v1:172.21.15.179:6789/0 conn(0x7f8be4143170 legacy=0x7f8be406c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.409+0000 7f8be3fff700 1 -- 172.21.15.179:0/1799606862 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40ce460 con 0x7f8be4142b80 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be3fff700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be4142b80 0x7f8be40ca1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea52b83810391a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be2ffd700 1 -- 172.21.15.179:0/1799606862 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400e070 con 0x7f8be4142b80 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be2ffd700 1 -- 172.21.15.179:0/1799606862 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400bb20 con 0x7f8be4142b80 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be2ffd700 1 -- 172.21.15.179:0/1799606862 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400be90 con 0x7f8be4142b80 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 -- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be4142b80 msgr2=0x7f8be40ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be4142b80 0x7f8be40ca1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 -- 172.21.15.179:0/1799606862 shutdown_connections 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be40ca6e0 0x7f8be40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 --2- 172.21.15.179:0/1799606862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be4142b80 0x7f8be40ca1a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 -- 172.21.15.179:0/1799606862 >> 172.21.15.179:0/1799606862 conn(0x7f8be40cc5d0 msgr2=0x7f8be40cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 -- 172.21.15.179:0/1799606862 shutdown_connections 2024-05-08T22:34:38.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.413+0000 7f8be93b0700 1 -- 172.21.15.179:0/1799606862 wait complete. 2024-05-08T22:34:38.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 Processor -- start 2024-05-08T22:34:38.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 -- start start 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be40ca6e0 0x7f8be41f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be4143170 0x7f8be41f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be406cf60 con 0x7f8be4142b80 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be406c2e0 con 0x7f8be40ca6e0 2024-05-08T22:34:38.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be93b0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be406c5c0 con 0x7f8be4143170 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40956/0 (socket says 172.21.15.179:40956) 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 -- 172.21.15.179:0/2288479831 learned_addr learned my addr 172.21.15.179:0/2288479831 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be8baf700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be4143170 0x7f8be41f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be3fff700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be40ca6e0 0x7f8be41f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 -- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be4143170 msgr2=0x7f8be41f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be4143170 0x7f8be41f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 -- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be40ca6e0 msgr2=0x7f8be41f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be40ca6e0 0x7f8be41f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.417+0000 7f8be37fe700 1 -- 172.21.15.179:0/2288479831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd4009a70 con 0x7f8be4142b80 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be37fe700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c700 tx=0x7f8bcc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc005d90 con 0x7f8be4142b80 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be406c8a0 con 0x7f8be4142b80 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0070d0 con 0x7f8be4142b80 2024-05-08T22:34:38.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be41f20f0 con 0x7f8be4142b80 2024-05-08T22:34:38.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc007440 con 0x7f8be4142b80 2024-05-08T22:34:38.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be404e700 con 0x7f8be4142b80 2024-05-08T22:34:38.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.421+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bcc0075a0 con 0x7f8be4142b80 2024-05-08T22:34:38.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.425+0000 7f8be0ff9700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 0x7f8bc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.425+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8bcc083e00 con 0x7f8be4142b80 2024-05-08T22:34:38.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.425+0000 7f8be3fff700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 0x7f8bc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.425+0000 7f8be3fff700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 0x7f8bc4068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.429+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8bcc051430 con 0x7f8be4142b80 2024-05-08T22:34:38.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8be40669e0 con 0x7f8be4142b80 2024-05-08T22:34:38.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be0ff9700 1 -- 172.21.15.179:0/2288479831 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8be40669e0 con 0x7f8be4142b80 2024-05-08T22:34:38.587 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:38.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 msgr2=0x7f8bc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 0x7f8bc4068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4011000 comp rx=0 tx=0).stop 2024-05-08T22:34:38.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 msgr2=0x7f8be41f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c700 tx=0x7f8bcc00ca10 comp rx=0 tx=0).stop 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 shutdown_connections 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bc4066730 0x7f8bc4068bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8be4143170 0x7f8be41f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be4142b80 0x7f8be41f5940 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 --2- 172.21.15.179:0/2288479831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8be40ca6e0 0x7f8be41f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.585+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 >> 172.21.15.179:0/2288479831 conn(0x7f8be40cc5d0 msgr2=0x7f8be40c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.589+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 shutdown_connections 2024-05-08T22:34:38.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.589+0000 7f8be93b0700 1 -- 172.21.15.179:0/2288479831 wait complete. 2024-05-08T22:34:38.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:34:38.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.705+0000 7f13b3fff700 1 Processor -- start 2024-05-08T22:34:38.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.705+0000 7f13b3fff700 1 -- start start 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b40cd200 con 0x7f13b406dc20 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40cd4e0 con 0x7f13b406ed50 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40cd7c0 con 0x7f13b41434d0 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b37fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:46134/0 (socket says 172.21.15.179:46134) 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b37fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:39776/0 (socket says 172.21.15.179:39776) 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 -- 172.21.15.179:0/3058415012 learned_addr learned my addr 172.21.15.179:0/3058415012 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b1ffb700 1 -- 172.21.15.179:0/3058415012 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845946701 0 0) 0x7f13b40cd200 con 0x7f13b406dc20 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b1ffb700 1 -- 172.21.15.179:0/3058415012 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13a00035d0 con 0x7f13b406dc20 2024-05-08T22:34:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 -- 172.21.15.179:0/3058415012 >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 msgr2=0x7f13b40b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 --2- 172.21.15.179:0/3058415012 >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 -- 172.21.15.179:0/3058415012 >> v1:172.21.15.195:6789/0 conn(0x7f13b406dc20 legacy=0x7f13b406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 -- 172.21.15.179:0/3058415012 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40cdaa0 con 0x7f13b41434d0 2024-05-08T22:34:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b37fe700 1 --2- 172.21.15.179:0/3058415012 >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b2ffd700 1 --2- 172.21.15.179:0/3058415012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009910 tx=0x7f13a4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c66c20e8b07e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b1ffb700 1 -- 172.21.15.179:0/3058415012 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011830 con 0x7f13b41434d0 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b1ffb700 1 -- 172.21.15.179:0/3058415012 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4011990 con 0x7f13b41434d0 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b1ffb700 1 -- 172.21.15.179:0/3058415012 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011d00 con 0x7f13b41434d0 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.709+0000 7f13b3fff700 1 -- 172.21.15.179:0/3058415012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 msgr2=0x7f13b406d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 --2- 172.21.15.179:0/3058415012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009910 tx=0x7f13a4009c20 comp rx=0 tx=0).stop 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 -- 172.21.15.179:0/3058415012 shutdown_connections 2024-05-08T22:34:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 --2- 172.21.15.179:0/3058415012 >> v2:172.21.15.195:3300/0 conn(0x7f13b406ed50 0x7f13b40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 --2- 172.21.15.179:0/3058415012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b41434d0 0x7f13b406d6e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 -- 172.21.15.179:0/3058415012 >> 172.21.15.179:0/3058415012 conn(0x7f13b40ce9d0 msgr2=0x7f13b40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 -- 172.21.15.179:0/3058415012 shutdown_connections 2024-05-08T22:34:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 -- 172.21.15.179:0/3058415012 wait complete. 2024-05-08T22:34:38.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 Processor -- start 2024-05-08T22:34:38.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.713+0000 7f13b3fff700 1 -- start start 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b406ed50 0x7f13b40c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b40ca100 0x7f13b413a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4159a20 con 0x7f13b41434d0 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c9070 con 0x7f13b406ed50 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c9350 con 0x7f13b40ca100 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40964/0 (socket says 172.21.15.179:40964) 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 -- 172.21.15.179:0/2227318010 learned_addr learned my addr 172.21.15.179:0/2227318010 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b37fe700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b40ca100 0x7f13b413a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b2ffd700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b406ed50 0x7f13b40c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 -- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b40ca100 msgr2=0x7f13b413a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b40ca100 0x7f13b413a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 -- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b406ed50 msgr2=0x7f13b40c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b406ed50 0x7f13b40c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 -- 172.21.15.179:0/2227318010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac009a70 con 0x7f13b41434d0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b27fc700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c520 tx=0x7f139c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0052d0 con 0x7f13b41434d0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4009570 con 0x7f13b41434d0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41f3470 con 0x7f13b41434d0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c005430 con 0x7f13b41434d0 2024-05-08T22:34:38.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0055a0 con 0x7f13b41434d0 2024-05-08T22:34:38.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1398005310 con 0x7f13b41434d0 2024-05-08T22:34:38.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.717+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f139c0038a0 con 0x7f13b41434d0 2024-05-08T22:34:38.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.721+0000 7f13abfff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 0x7f1394068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.721+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f139c083270 con 0x7f13b41434d0 2024-05-08T22:34:38.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.721+0000 7f13b2ffd700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 0x7f1394068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.721+0000 7f13b2ffd700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 0x7f1394068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009c60 tx=0x7f13a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.729+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f139c050820 con 0x7f13b41434d0 2024-05-08T22:34:38.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.897+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1398005890 con 0x7f13b41434d0 2024-05-08T22:34:38.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.897+0000 7f13abfff700 1 -- 172.21.15.179:0/2227318010 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+65 (secure 0 0 0) 0x7f139c053e40 con 0x7f13b41434d0 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:38.901 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 msgr2=0x7f1394068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 0x7f1394068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009c60 tx=0x7f13a4008030 comp rx=0 tx=0).stop 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 msgr2=0x7f13b4139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c520 tx=0x7f139c00c830 comp rx=0 tx=0).stop 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 shutdown_connections 2024-05-08T22:34:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1394066730 0x7f1394068bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13b40ca100 0x7f13b413a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13b41434d0 0x7f13b4139c40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 --2- 172.21.15.179:0/2227318010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13b406ed50 0x7f13b40c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 >> 172.21.15.179:0/2227318010 conn(0x7f13b40ce9d0 msgr2=0x7f13b406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 shutdown_connections 2024-05-08T22:34:38.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:38.901+0000 7f13b3fff700 1 -- 172.21.15.179:0/2227318010 wait complete. 2024-05-08T22:34:38.915 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:34:38.915 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:34:39.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 Processor -- start 2024-05-08T22:34:39.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 -- start start 2024-05-08T22:34:39.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40adc40 0x7f39c415e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f39c40b34a0 0x7f39c406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c40b3010 con 0x7f39c406e6d0 2024-05-08T22:34:39.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c406eb40 con 0x7f39c40b34a0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c406bbc0 con 0x7f39c40adc40 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f39c40b34a0 0x7f39c406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c2d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f39c406e6d0 0x7f39c4160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38214/0 (socket says 172.21.15.179:38214) 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c2d9d700 1 -- 172.21.15.179:0/1507980836 learned_addr learned my addr 172.21.15.179:0/1507980836 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c259c700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40adc40 0x7f39c415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 -- 172.21.15.179:0/1507980836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40adc40 msgr2=0x7f39c415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40adc40 0x7f39c415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 -- 172.21.15.179:0/1507980836 >> v1:172.21.15.179:6789/0 conn(0x7f39c406e6d0 legacy=0x7f39c4160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c159a700 1 -- 172.21.15.179:0/1507980836 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704520056 0 0) 0x7f39c40b3010 con 0x7f39c406e6d0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 -- 172.21.15.179:0/1507980836 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c415e740 con 0x7f39c40b34a0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c1d9b700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c40b34a0 0x7f39c406e190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3b2188ff412d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c159a700 1 -- 172.21.15.179:0/1507980836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400e070 con 0x7f39c40b34a0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c159a700 1 -- 172.21.15.179:0/1507980836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b400bbd0 con 0x7f39c40b34a0 2024-05-08T22:34:39.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c159a700 1 -- 172.21.15.179:0/1507980836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4012430 con 0x7f39c40b34a0 2024-05-08T22:34:39.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 -- 172.21.15.179:0/1507980836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c40b34a0 msgr2=0x7f39c406e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.025+0000 7f39c359e700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c40b34a0 0x7f39c406e190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:39.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- 172.21.15.179:0/1507980836 shutdown_connections 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c40b34a0 0x7f39c406e190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 --2- 172.21.15.179:0/1507980836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40adc40 0x7f39c415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- 172.21.15.179:0/1507980836 >> 172.21.15.179:0/1507980836 conn(0x7f39c40ce9a0 msgr2=0x7f39c40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- 172.21.15.179:0/1507980836 shutdown_connections 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- 172.21.15.179:0/1507980836 wait complete. 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 Processor -- start 2024-05-08T22:34:39.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- start start 2024-05-08T22:34:39.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c40adc40 0x7f39c41451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40b34a0 0x7f39c4142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4146390 con 0x7f39c406e6d0 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c41471d0 con 0x7f39c40adc40 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c41465c0 con 0x7f39c40b34a0 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40974/0 (socket says 172.21.15.179:40974) 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c259c700 1 -- 172.21.15.179:0/1967274621 learned_addr learned my addr 172.21.15.179:0/1967274621 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.029+0000 7f39c2d9d700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40b34a0 0x7f39c4142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c1d9b700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c40adc40 0x7f39c41451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 -- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40b34a0 msgr2=0x7f39c4142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40b34a0 0x7f39c4142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 -- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c40adc40 msgr2=0x7f39c41451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c40adc40 0x7f39c41451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 -- 172.21.15.179:0/1967274621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009a70 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c259c700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e810 tx=0x7f39ac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac013610 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac013770 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c41468a0 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac013b20 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c4146e60 con 0x7f39c406e6d0 2024-05-08T22:34:39.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c404e770 con 0x7f39c406e6d0 2024-05-08T22:34:39.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39ac013c80 con 0x7f39c406e6d0 2024-05-08T22:34:39.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.033+0000 7f39baffd700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 0x7f39a4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:39.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.037+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f39ac083c80 con 0x7f39c406e6d0 2024-05-08T22:34:39.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.037+0000 7f39c1d9b700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 0x7f39a4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:39.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.037+0000 7f39c1d9b700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 0x7f39a4068c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f39c40b3880 tx=0x7f39b4011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:39.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.037+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f39ac051210 con 0x7f39c406e6d0 2024-05-08T22:34:39.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.205+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f39c406afe0 con 0x7f39c406e6d0 2024-05-08T22:34:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.989+0000 7f39baffd700 1 -- 172.21.15.179:0/1967274621 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) v1 ==== 138+0+0 (secure 0 0 0) 0x7f39ac054830 con 0x7f39c406e6d0 2024-05-08T22:34:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 msgr2=0x7f39a4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 0x7f39a4068c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f39c40b3880 tx=0x7f39b4011000 comp rx=0 tx=0).stop 2024-05-08T22:34:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 msgr2=0x7f39c4144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e810 tx=0x7f39ac00ebd0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 shutdown_connections 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39a4066760 0x7f39a4068c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c40adc40 0x7f39c41451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39c40b34a0 0x7f39c4142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 --2- 172.21.15.179:0/1967274621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c406e6d0 0x7f39c4144890 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 >> 172.21.15.179:0/1967274621 conn(0x7f39c40ce9a0 msgr2=0x7f39c40ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 shutdown_connections 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:39.993+0000 7f39c359e700 1 -- 172.21.15.179:0/1967274621 wait complete. 2024-05-08T22:34:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:34:40.009 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:34:40.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 Processor -- start 2024-05-08T22:34:40.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 -- start start 2024-05-08T22:34:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd16006abb0 0x7fd1600b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1600b60b0 con 0x7fd16015d360 2024-05-08T22:34:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1600b54a0 con 0x7fd16006abb0 2024-05-08T22:34:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd1664f9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16015e750 con 0x7fd1600b5010 2024-05-08T22:34:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd15f7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd16006abb0 0x7fd1600b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd15ffff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd16015d360 0x7fd1600b8a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:33164/0 (socket says 172.21.15.179:33164) 2024-05-08T22:34:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd164a96700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd15ffff700 1 -- 172.21.15.179:0/2366032856 learned_addr learned my addr 172.21.15.179:0/2366032856 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd164a96700 1 -- 172.21.15.179:0/2366032856 >> v2:172.21.15.195:3300/0 conn(0x7fd16006abb0 msgr2=0x7fd1600b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd164a96700 1 --2- 172.21.15.179:0/2366032856 >> v2:172.21.15.195:3300/0 conn(0x7fd16006abb0 0x7fd1600b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd164a96700 1 -- 172.21.15.179:0/2366032856 >> v1:172.21.15.195:6789/0 conn(0x7fd16015d360 legacy=0x7fd1600b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.129+0000 7fd164a96700 1 -- 172.21.15.179:0/2366032856 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16006b130 con 0x7fd1600b5010 2024-05-08T22:34:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd164a96700 1 --2- 172.21.15.179:0/2366032856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600d15e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd158009d90 tx=0x7fd15800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c996306da035676b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd15effd700 1 -- 172.21.15.179:0/2366032856 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800e070 con 0x7fd1600b5010 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd15effd700 1 -- 172.21.15.179:0/2366032856 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15800bb20 con 0x7fd1600b5010 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd15effd700 1 -- 172.21.15.179:0/2366032856 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158012430 con 0x7fd1600b5010 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 -- 172.21.15.179:0/2366032856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 msgr2=0x7fd1600d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 --2- 172.21.15.179:0/2366032856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600d15e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd158009d90 tx=0x7fd15800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 -- 172.21.15.179:0/2366032856 shutdown_connections 2024-05-08T22:34:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 --2- 172.21.15.179:0/2366032856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600d15e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 --2- 172.21.15.179:0/2366032856 >> v2:172.21.15.195:3300/0 conn(0x7fd16006abb0 0x7fd1600b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 -- 172.21.15.179:0/2366032856 >> 172.21.15.179:0/2366032856 conn(0x7fd1600d3050 msgr2=0x7fd1600d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 -- 172.21.15.179:0/2366032856 shutdown_connections 2024-05-08T22:34:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.133+0000 7fd1664f9700 1 -- 172.21.15.179:0/2366032856 wait complete. 2024-05-08T22:34:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 Processor -- start 2024-05-08T22:34:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 -- start start 2024-05-08T22:34:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd16006abb0 0x7fd160151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600caaa0 con 0x7fd16015d360 2024-05-08T22:34:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600cb900 con 0x7fd16006abb0 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600cacf0 con 0x7fd1600b5010 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd15f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40990/0 (socket says 172.21.15.179:40990) 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 -- 172.21.15.179:0/4087606588 learned_addr learned my addr 172.21.15.179:0/4087606588 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd15ffff700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd16006abb0 0x7fd160151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 -- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 msgr2=0x7fd1600c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 -- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd16006abb0 msgr2=0x7fd160151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd16006abb0 0x7fd160151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 -- 172.21.15.179:0/4087606588 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009a70 con 0x7fd16015d360 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd164a96700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd158014040 tx=0x7fd158004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158012430 con 0x7fd16015d360 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd158020070 con 0x7fd16015d360 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1600cafd0 con 0x7fd16015d360 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.137+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580098f0 con 0x7fd16015d360 2024-05-08T22:34:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.141+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1600cb590 con 0x7fd16015d360 2024-05-08T22:34:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.141+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd160073290 con 0x7fd16015d360 2024-05-08T22:34:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.141+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd158004040 con 0x7fd16015d360 2024-05-08T22:34:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.145+0000 7fd15cff9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 0x7fd140068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.145+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fd158022070 con 0x7fd16015d360 2024-05-08T22:34:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.145+0000 7fd15ffff700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 0x7fd140068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.145+0000 7fd15ffff700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 0x7fd140068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd150005930 tx=0x7fd150009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:40.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.149+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd1580530f0 con 0x7fd16015d360 2024-05-08T22:34:40.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.317+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd160072ec0 con 0x7fd16015d360 2024-05-08T22:34:41.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:40.997+0000 7fd15cff9700 1 -- 172.21.15.179:0/4087606588 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd15802c0d0 con 0x7fd16015d360 2024-05-08T22:34:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.009+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 msgr2=0x7fd140068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.009+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 0x7fd140068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd150005930 tx=0x7fd150009040 comp rx=0 tx=0).stop 2024-05-08T22:34:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.009+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 msgr2=0x7fd1600cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.009+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd158014040 tx=0x7fd158004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 shutdown_connections 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd140066730 0x7fd140068bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16015d360 0x7fd1600cbc60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1600b5010 0x7fd1600c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 --2- 172.21.15.179:0/4087606588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd16006abb0 0x7fd160151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 >> 172.21.15.179:0/4087606588 conn(0x7fd1600d3050 msgr2=0x7fd1600b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 shutdown_connections 2024-05-08T22:34:41.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.013+0000 7fd1664f9700 1 -- 172.21.15.179:0/4087606588 wait complete. 2024-05-08T22:34:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:34:41.033 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:34:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 Processor -- start 2024-05-08T22:34:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 -- start start 2024-05-08T22:34:41.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0600aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0600b4170 0x7fd0600b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600b5340 con 0x7fd0600ae720 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600b4730 con 0x7fd0600b4170 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd0663ce700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600b4a10 con 0x7fd0600b0210 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd05ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd0600ae720 0x7fd0600afb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38218/0 (socket says 172.21.15.179:38218) 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd05ffff700 1 -- 172.21.15.179:0/400081490 learned_addr learned my addr 172.21.15.179:0/400081490 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd06496b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0600b4170 0x7fd0600b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd05f7fe700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0600aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd05effd700 1 -- 172.21.15.179:0/400081490 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161116214 0 0) 0x7fd0600b5340 con 0x7fd0600ae720 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd05effd700 1 -- 172.21.15.179:0/400081490 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd04c0035d0 con 0x7fd0600ae720 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd06496b700 1 -- 172.21.15.179:0/400081490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 msgr2=0x7fd0600aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd06496b700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0600aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd06496b700 1 -- 172.21.15.179:0/400081490 >> v1:172.21.15.179:6789/0 conn(0x7fd0600ae720 legacy=0x7fd0600afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.149+0000 7fd06496b700 1 -- 172.21.15.179:0/400081490 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600b4cf0 con 0x7fd0600b4170 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd06496b700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600b4170 0x7fd0600b1b30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd058009850 tx=0x7fd058009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f486ba31a053019 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd05effd700 1 -- 172.21.15.179:0/400081490 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011830 con 0x7fd0600b4170 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd05effd700 1 -- 172.21.15.179:0/400081490 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011990 con 0x7fd0600b4170 2024-05-08T22:34:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd05effd700 1 -- 172.21.15.179:0/400081490 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011d40 con 0x7fd0600b4170 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- 172.21.15.179:0/400081490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600b4170 msgr2=0x7fd0600b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600b4170 0x7fd0600b1b30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd058009850 tx=0x7fd058009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- 172.21.15.179:0/400081490 shutdown_connections 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600b4170 0x7fd0600b1b30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 --2- 172.21.15.179:0/400081490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0600aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- 172.21.15.179:0/400081490 >> 172.21.15.179:0/400081490 conn(0x7fd060098a70 msgr2=0x7fd0600b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- 172.21.15.179:0/400081490 shutdown_connections 2024-05-08T22:34:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- 172.21.15.179:0/400081490 wait complete. 2024-05-08T22:34:41.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 Processor -- start 2024-05-08T22:34:41.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.153+0000 7fd0663ce700 1 -- start start 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0601f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0600b4170 0x7fd0600b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600cc420 con 0x7fd0600ae720 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600cbaf0 con 0x7fd0600b4170 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600cbdd0 con 0x7fd0600b0210 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41010/0 (socket says 172.21.15.179:41010) 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 -- 172.21.15.179:0/2862273103 learned_addr learned my addr 172.21.15.179:0/2862273103 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:41.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05f7fe700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0601f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd06496b700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0600b4170 0x7fd0600b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 -- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 msgr2=0x7fd0601f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0601f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 -- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0600b4170 msgr2=0x7fd0600b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0600b4170 0x7fd0600b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 -- 172.21.15.179:0/2862273103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580094b0 con 0x7fd0600ae720 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05ffff700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd05000cbc0 tx=0x7fd05000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000a7b0 con 0x7fd0600ae720 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05000a910 con 0x7fd0600ae720 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600cc0b0 con 0x7fd0600ae720 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000aca0 con 0x7fd0600ae720 2024-05-08T22:34:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.157+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0601f3f50 con 0x7fd0600ae720 2024-05-08T22:34:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.161+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd060073290 con 0x7fd0600ae720 2024-05-08T22:34:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.161+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd05001b3b0 con 0x7fd0600ae720 2024-05-08T22:34:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.161+0000 7fd05cff9700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 0x7fd040068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.161+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fd050083c20 con 0x7fd0600ae720 2024-05-08T22:34:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.161+0000 7fd05f7fe700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 0x7fd040068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.165+0000 7fd05f7fe700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 0x7fd040068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c61b0 tx=0x7fd048009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.169+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd050050eb0 con 0x7fd0600ae720 2024-05-08T22:34:41.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.377+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd060072410 con 0x7fd0600ae720 2024-05-08T22:34:41.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.385+0000 7fd05cff9700 1 -- 172.21.15.179:0/2862273103 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd0500170c0 con 0x7fd0600ae720 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 msgr2=0x7fd040068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 0x7fd040068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c61b0 tx=0x7fd048009240 comp rx=0 tx=0).stop 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 msgr2=0x7fd0600cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd05000cbc0 tx=0x7fd05000cbf0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 shutdown_connections 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd040066730 0x7fd040068bf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0600b4170 0x7fd0600b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0600b0210 0x7fd0601f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 --2- 172.21.15.179:0/2862273103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0600ae720 0x7fd0600cc8b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.393+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 >> 172.21.15.179:0/2862273103 conn(0x7fd060098a70 msgr2=0x7fd0600ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.401+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 shutdown_connections 2024-05-08T22:34:41.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.401+0000 7fd0663ce700 1 -- 172.21.15.179:0/2862273103 wait complete. 2024-05-08T22:34:41.407 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 5 and data pool 6 2024-05-08T22:34:41.421 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:41.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 Processor -- start 2024-05-08T22:34:41.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 -- start start 2024-05-08T22:34:41.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94d41539c0 0x7f94d406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d4159b60 con 0x7f94d406bc50 2024-05-08T22:34:41.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d4159e40 con 0x7f94d41539c0 2024-05-08T22:34:41.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.521+0000 7f94da70b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415a120 con 0x7f94d40b3720 2024-05-08T22:34:41.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d8f08700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f94d406bc50 0x7f94d40b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:33172/0 (socket says 172.21.15.179:33172) 2024-05-08T22:34:41.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9f0a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94d41539c0 0x7f94d406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d8f08700 1 -- 172.21.15.179:0/3771008646 learned_addr learned my addr 172.21.15.179:0/3771008646 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 -- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 msgr2=0x7f94d415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 --2- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 -- 172.21.15.179:0/3771008646 >> v1:172.21.15.195:6789/0 conn(0x7f94d406bc50 legacy=0x7f94d40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 -- 172.21.15.179:0/3771008646 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d415a400 con 0x7f94d41539c0 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94c7fff700 1 -- 172.21.15.179:0/3771008646 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142145166 0 0) 0x7f94d4159b60 con 0x7f94d406bc50 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94d9709700 1 --2- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d41539c0 0x7f94d406b710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f94d000a9f0 tx=0x7f94d000ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607f828ca4ab4d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94c7fff700 1 -- 172.21.15.179:0/3771008646 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d000cb70 con 0x7f94d41539c0 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94c7fff700 1 -- 172.21.15.179:0/3771008646 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d000ae90 con 0x7f94d41539c0 2024-05-08T22:34:41.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94c7fff700 1 -- 172.21.15.179:0/3771008646 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0018400 con 0x7f94d41539c0 2024-05-08T22:34:41.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 -- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d41539c0 msgr2=0x7f94d406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 --2- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d41539c0 0x7f94d406b710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f94d000a9f0 tx=0x7f94d000ad00 comp rx=0 tx=0).stop 2024-05-08T22:34:41.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 -- 172.21.15.179:0/3771008646 shutdown_connections 2024-05-08T22:34:41.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 --2- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 --2- 172.21.15.179:0/3771008646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d41539c0 0x7f94d406b710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 -- 172.21.15.179:0/3771008646 >> 172.21.15.179:0/3771008646 conn(0x7f94d40ce9d0 msgr2=0x7f94d40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 -- 172.21.15.179:0/3771008646 shutdown_connections 2024-05-08T22:34:41.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.525+0000 7f94da70b700 1 -- 172.21.15.179:0/3771008646 wait complete. 2024-05-08T22:34:41.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.529+0000 7f94da70b700 1 Processor -- start 2024-05-08T22:34:41.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.529+0000 7f94da70b700 1 -- start start 2024-05-08T22:34:41.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d406bc50 0x7f94d414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d4147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41f4e80 con 0x7f94d41539c0 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41e3eb0 con 0x7f94d406bc50 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41f4ba0 con 0x7f94d40b3720 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41018/0 (socket says 172.21.15.179:41018) 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 -- 172.21.15.179:0/4066952065 learned_addr learned my addr 172.21.15.179:0/4066952065 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9709700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d406bc50 0x7f94d414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d8f08700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d4147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 -- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 msgr2=0x7f94d4147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d4147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 -- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d406bc50 msgr2=0x7f94d414c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d406bc50 0x7f94d414c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 -- 172.21.15.179:0/4066952065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d000a650 con 0x7f94d41539c0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d8f08700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d4147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94d9f0a700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00ef90 tx=0x7f94cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00cd40 con 0x7f94d41539c0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc00f040 con 0x7f94d41539c0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d41e5590 con 0x7f94d41539c0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00cea0 con 0x7f94d41539c0 2024-05-08T22:34:41.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d41e5b50 con 0x7f94d41539c0 2024-05-08T22:34:41.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.533+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d41e53d0 con 0x7f94d41539c0 2024-05-08T22:34:41.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.537+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94cc0146c0 con 0x7f94d41539c0 2024-05-08T22:34:41.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.537+0000 7f94c5ffb700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.537+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f94cc0852e0 con 0x7f94d41539c0 2024-05-08T22:34:41.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.537+0000 7f94d9709700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.537+0000 7f94d9709700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 0x7f94b8068bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f94d000a1c0 tx=0x7f94d001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.541+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f94cc052680 con 0x7f94d41539c0 2024-05-08T22:34:41.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.713+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94d4067740 con 0x7f94d41539c0 2024-05-08T22:34:41.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94c5ffb700 1 -- 172.21.15.179:0/4066952065 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f94cc009ca0 con 0x7f94d41539c0 2024-05-08T22:34:41.718 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:41.719 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":40,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:41.380523+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:41.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 msgr2=0x7f94b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 0x7f94b8068bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f94d000a1c0 tx=0x7f94d001b040 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 msgr2=0x7f94d415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00ef90 tx=0x7f94cc00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 shutdown_connections 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94b8066730 0x7f94b8068bf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94d41539c0 0x7f94d415aa50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94d40b3720 0x7f94d4147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 --2- 172.21.15.179:0/4066952065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94d406bc50 0x7f94d414c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 >> 172.21.15.179:0/4066952065 conn(0x7f94d40ce9d0 msgr2=0x7f94d406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 shutdown_connections 2024-05-08T22:34:41.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.717+0000 7f94da70b700 1 -- 172.21.15.179:0/4066952065 wait complete. 2024-05-08T22:34:41.734 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:41.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 Processor -- start 2024-05-08T22:34:41.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 -- start start 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 0x7f7a3006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a3006dc20 0x7f7a3006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a300cd200 con 0x7f7a300ce2a0 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300cd4e0 con 0x7f7a3006dc20 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a36bae700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300cd7c0 con 0x7f7a301434d0 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a353ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a3006dc20 0x7f7a3006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a363ad700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7a300ce2a0 0x7f7a300b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38226/0 (socket says 172.21.15.179:38226) 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a363ad700 1 -- 172.21.15.179:0/617943730 learned_addr learned my addr 172.21.15.179:0/617943730 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:41.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.849+0000 7f7a35bac700 1 --2- 172.21.15.179:0/617943730 >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 0x7f7a3006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a353ab700 1 -- 172.21.15.179:0/617943730 >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 msgr2=0x7f7a3006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:41.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a353ab700 1 --2- 172.21.15.179:0/617943730 >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 0x7f7a3006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a353ab700 1 -- 172.21.15.179:0/617943730 >> v1:172.21.15.179:6789/0 conn(0x7f7a300ce2a0 legacy=0x7f7a300b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a353ab700 1 -- 172.21.15.179:0/617943730 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300cdaa0 con 0x7f7a3006dc20 2024-05-08T22:34:41.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a35bac700 1 --2- 172.21.15.179:0/617943730 >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 0x7f7a3006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a34baa700 1 -- 172.21.15.179:0/617943730 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718860469 0 0) 0x7f7a300cd200 con 0x7f7a300ce2a0 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a353ab700 1 --2- 172.21.15.179:0/617943730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3006dc20 0x7f7a3006e640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7a30070800 tx=0x7f7a20009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ea13cfe7d2210 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a34baa700 1 -- 172.21.15.179:0/617943730 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20011830 con 0x7f7a3006dc20 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a34baa700 1 -- 172.21.15.179:0/617943730 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20011990 con 0x7f7a3006dc20 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a34baa700 1 -- 172.21.15.179:0/617943730 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20011d40 con 0x7f7a3006dc20 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 -- 172.21.15.179:0/617943730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3006dc20 msgr2=0x7f7a3006e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 --2- 172.21.15.179:0/617943730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3006dc20 0x7f7a3006e640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7a30070800 tx=0x7f7a20009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 -- 172.21.15.179:0/617943730 shutdown_connections 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 --2- 172.21.15.179:0/617943730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a3006dc20 0x7f7a3006e640 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 --2- 172.21.15.179:0/617943730 >> v2:172.21.15.195:3300/0 conn(0x7f7a301434d0 0x7f7a3006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 -- 172.21.15.179:0/617943730 >> 172.21.15.179:0/617943730 conn(0x7f7a300ce9d0 msgr2=0x7f7a300cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 -- 172.21.15.179:0/617943730 shutdown_connections 2024-05-08T22:34:41.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.853+0000 7f7a36bae700 1 -- 172.21.15.179:0/617943730 wait complete. 2024-05-08T22:34:41.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 Processor -- start 2024-05-08T22:34:41.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- start start 2024-05-08T22:34:41.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 0x7f7a3013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a300c3950 0x7f7a300c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30159a30 con 0x7f7a301434d0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300c9c80 con 0x7f7a300c3950 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3014cd10 con 0x7f7a300ce2a0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41032/0 (socket says 172.21.15.179:41032) 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 -- 172.21.15.179:0/1966694234 learned_addr learned my addr 172.21.15.179:0/1966694234 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a363ad700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a300c3950 0x7f7a300c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a35bac700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 0x7f7a3013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 -- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 msgr2=0x7f7a3013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 0x7f7a3013fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 -- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a300c3950 msgr2=0x7f7a300c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a300c3950 0x7f7a300c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 -- 172.21.15.179:0/1966694234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200094b0 con 0x7f7a301434d0 2024-05-08T22:34:41.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a35bac700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 0x7f7a3013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a353ab700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2001aac0 con 0x7f7a301434d0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2001f070 con 0x7f7a301434d0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3014cff0 con 0x7f7a301434d0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2001ae80 con 0x7f7a301434d0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.857+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a300c9910 con 0x7f7a301434d0 2024-05-08T22:34:41.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a300c9490 con 0x7f7a301434d0 2024-05-08T22:34:41.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a2001b430 con 0x7f7a301434d0 2024-05-08T22:34:41.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a267fc700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 0x7f7a14068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:41.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7a200853e0 con 0x7f7a301434d0 2024-05-08T22:34:41.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a35bac700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 0x7f7a14068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:41.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.861+0000 7f7a35bac700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 0x7f7a14068c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009980 tx=0x7f7a2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:41.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:41.869+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7a300c9490 con 0x7f7a301434d0 2024-05-08T22:34:42.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.041+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a300c9490 con 0x7f7a301434d0 2024-05-08T22:34:42.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.045+0000 7f7a267fc700 1 -- 172.21.15.179:0/1966694234 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f7a300c9490 con 0x7f7a301434d0 2024-05-08T22:34:42.047 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:42.048 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":40,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:41.380523+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:42.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 msgr2=0x7f7a14068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 0x7f7a14068c20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009980 tx=0x7f7a2c008040 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 msgr2=0x7f7a30140120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20004770 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 shutdown_connections 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a14066760 0x7f7a14068c20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a300c3950 0x7f7a300c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a301434d0 0x7f7a30140120 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 --2- 172.21.15.179:0/1966694234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a300ce2a0 0x7f7a3013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 >> 172.21.15.179:0/1966694234 conn(0x7f7a300ce9d0 msgr2=0x7f7a301426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 shutdown_connections 2024-05-08T22:34:42.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.049+0000 7f7a36bae700 1 -- 172.21.15.179:0/1966694234 wait complete. 2024-05-08T22:34:42.068 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:34:42.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.181+0000 7f21be64b700 1 Processor -- start 2024-05-08T22:34:42.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.181+0000 7f21be64b700 1 -- start start 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f21b806d4c0 0x7f21b80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806dfb0 0x7f21b80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b806d0f0 con 0x7f21b80cdf90 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b80cca70 con 0x7f21b806d4c0 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b80cb240 con 0x7f21b806dfb0 2024-05-08T22:34:42.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f21b806d4c0 0x7f21b80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bce48700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f21b80cdf90 0x7f21b806d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38236/0 (socket says 172.21.15.179:38236) 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bce48700 1 -- 172.21.15.179:0/855223635 learned_addr learned my addr 172.21.15.179:0/855223635 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bde4a700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806dfb0 0x7f21b80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21affff700 1 -- 172.21.15.179:0/855223635 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602984175 0 0) 0x7f21b806d0f0 con 0x7f21b80cdf90 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21affff700 1 -- 172.21.15.179:0/855223635 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21a40035d0 con 0x7f21b80cdf90 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 -- 172.21.15.179:0/855223635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806dfb0 msgr2=0x7f21b80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806dfb0 0x7f21b80ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 -- 172.21.15.179:0/855223635 >> v1:172.21.15.179:6789/0 conn(0x7f21b80cdf90 legacy=0x7f21b806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 -- 172.21.15.179:0/855223635 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8143e30 con 0x7f21b806d4c0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21bd649700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806d4c0 0x7f21b80cc530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009850 tx=0x7f21a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791dd8e9c8a2361f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21affff700 1 -- 172.21.15.179:0/855223635 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011830 con 0x7f21b806d4c0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21affff700 1 -- 172.21.15.179:0/855223635 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8011990 con 0x7f21b806d4c0 2024-05-08T22:34:42.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21affff700 1 -- 172.21.15.179:0/855223635 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011d00 con 0x7f21b806d4c0 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- 172.21.15.179:0/855223635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806d4c0 msgr2=0x7f21b80cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806d4c0 0x7f21b80cc530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009850 tx=0x7f21a8009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- 172.21.15.179:0/855223635 shutdown_connections 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806dfb0 0x7f21b80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 --2- 172.21.15.179:0/855223635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806d4c0 0x7f21b80cc530 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- 172.21.15.179:0/855223635 >> 172.21.15.179:0/855223635 conn(0x7f21b80cea30 msgr2=0x7f21b80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- 172.21.15.179:0/855223635 shutdown_connections 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- 172.21.15.179:0/855223635 wait complete. 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 Processor -- start 2024-05-08T22:34:42.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.185+0000 7f21be64b700 1 -- start start 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806d4c0 0x7f21b81f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21b80cdf90 0x7f21b80b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81f35d0 con 0x7f21b806dfb0 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b80b1e10 con 0x7f21b80cdf90 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b80b20f0 con 0x7f21b806d4c0 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41042/0 (socket says 172.21.15.179:41042) 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 -- 172.21.15.179:0/3687077545 learned_addr learned my addr 172.21.15.179:0/3687077545 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bd649700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806d4c0 0x7f21b81f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bde4a700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21b80cdf90 0x7f21b80b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 -- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806d4c0 msgr2=0x7f21b81f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806d4c0 0x7f21b81f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 -- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21b80cdf90 msgr2=0x7f21b80b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21b80cdf90 0x7f21b80b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 -- 172.21.15.179:0/3687077545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80094b0 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21bce48700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f21b400cb40 tx=0x7f21b400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400a7b0 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b400a910 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b80b23d0 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400acc0 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.189+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b80b2990 con 0x7f21b806dfb0 2024-05-08T22:34:42.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b80b2bb0 con 0x7f21b806dfb0 2024-05-08T22:34:42.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21b401b3b0 con 0x7f21b806dfb0 2024-05-08T22:34:42.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21adffb700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 0x7f219c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f21b4084250 con 0x7f21b806dfb0 2024-05-08T22:34:42.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21bd649700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 0x7f219c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.193+0000 7f21bd649700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 0x7f219c068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:42.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.197+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f21b40513d0 con 0x7f21b806dfb0 2024-05-08T22:34:42.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:42.401+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f21b804e700 con 0x7f21b806dfb0 2024-05-08T22:34:43.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.393+0000 7f21adffb700 1 -- 172.21.15.179:0/3687077545 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f21b40170b0 con 0x7f21b806dfb0 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.397+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 msgr2=0x7f219c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.397+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 0x7f219c068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a801c040 comp rx=0 tx=0).stop 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.397+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 msgr2=0x7f21b815bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.397+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f21b400cb40 tx=0x7f21b400cb70 comp rx=0 tx=0).stop 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 shutdown_connections 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f219c066730 0x7f219c068bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21b80cdf90 0x7f21b80b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b806dfb0 0x7f21b815bda0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 --2- 172.21.15.179:0/3687077545 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21b806d4c0 0x7f21b81f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 >> 172.21.15.179:0/3687077545 conn(0x7f21b80cea30 msgr2=0x7f21b806d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:43.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 shutdown_connections 2024-05-08T22:34:43.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.401+0000 7f21be64b700 1 -- 172.21.15.179:0/3687077545 wait complete. 2024-05-08T22:34:43.418 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:34:43.419 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:34:43.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.541+0000 7fa3e0f21700 1 Processor -- start 2024-05-08T22:34:43.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.541+0000 7fa3e0f21700 1 -- start start 2024-05-08T22:34:43.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3dc0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0b6f90 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0d1910 con 0x7fa3dc15d160 2024-05-08T22:34:43.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d1bf0 con 0x7fa3dc0ce230 2024-05-08T22:34:43.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3daffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa3dc0d29b0 0x7fa3dc0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38238/0 (socket says 172.21.15.179:38238) 2024-05-08T22:34:43.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3daffd700 1 -- 172.21.15.179:0/1792659510 learned_addr learned my addr 172.21.15.179:0/1792659510 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:43.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3dc0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117771324 0 0) 0x7fa3c4001110 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c80035d0 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809525637 0 0) 0x7fa3dc0d1910 con 0x7fa3dc15d160 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c4001110 con 0x7fa3dc15d160 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 115258239 0 0) 0x7fa3c80035d0 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 msgr2=0x7fa3dc0b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3dc0b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 >> v1:172.21.15.195:6789/0 conn(0x7fa3dc15d160 legacy=0x7fa3dc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0d1ed0 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fa3c4001800 con 0x7fa3dc0d29b0 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 >> v1:172.21.15.179:6789/0 conn(0x7fa3dc0d29b0 legacy=0x7fa3dc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c8003be0 0x7fa3c8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3c8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c8004db0 0x7fa3c8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d1bf0 con 0x7fa3c8004db0 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c800bae0 con 0x7fa3c8003be0 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c800bca0 con 0x7fa3dc0ce230 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3daffd700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3c8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3db7fe700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c8003be0 0x7fa3c8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c8004db0 0x7fa3c8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 -- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 msgr2=0x7fa3c8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0ce230 0x7fa3c8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 -- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c8003be0 msgr2=0x7fa3c8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c8003be0 0x7fa3c8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 -- 172.21.15.179:0/1792659510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80035d0 con 0x7fa3c8004db0 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c8004db0 0x7fa3c8005230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0b5d50 tx=0x7fa3c80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c7f31f1d378924 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400ebd0 con 0x7fa3c8004db0 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400f040 con 0x7fa3c8004db0 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3da7fc700 1 -- 172.21.15.179:0/1792659510 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4014480 con 0x7fa3c8004db0 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- 172.21.15.179:0/1792659510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c8004db0 msgr2=0x7fa3c8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/1792659510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c8004db0 0x7fa3c8005230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0b5d50 tx=0x7fa3c80058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3dbfff700 1 -- 172.21.15.179:0/1792659510 reap_dead start 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- 172.21.15.179:0/1792659510 shutdown_connections 2024-05-08T22:34:43.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- 172.21.15.179:0/1792659510 >> 172.21.15.179:0/1792659510 conn(0x7fa3dc0d3270 msgr2=0x7fa3dc0d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:43.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.545+0000 7fa3e0f21700 1 -- 172.21.15.179:0/1792659510 shutdown_connections 2024-05-08T22:34:43.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 -- 172.21.15.179:0/1792659510 wait complete. 2024-05-08T22:34:43.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 Processor -- start 2024-05-08T22:34:43.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 -- start start 2024-05-08T22:34:43.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0cc630 0x7fa3dc0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3dc0c73e0 0x7fa3dc0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0ccab0 con 0x7fa3dc151bb0 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc152ba0 con 0x7fa3dc0c73e0 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3e0f21700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0c7860 con 0x7fa3dc0cc630 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41060/0 (socket says 172.21.15.179:41060) 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3db7fe700 1 -- 172.21.15.179:0/275586031 learned_addr learned my addr 172.21.15.179:0/275586031 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.549+0000 7fa3daffd700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0cc630 0x7fa3dc0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3dbfff700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3dc0c73e0 0x7fa3dc0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 -- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0cc630 msgr2=0x7fa3dc0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0cc630 0x7fa3dc0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 -- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3dc0c73e0 msgr2=0x7fa3dc0b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3dc0c73e0 0x7fa3dc0b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 -- 172.21.15.179:0/275586031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc15e320 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3db7fe700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00f840 tx=0x7fa3cc00fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0052e0 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc005440 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc15e600 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0038e0 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc15ebc0 con 0x7fa3dc151bb0 2024-05-08T22:34:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.553+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc04e700 con 0x7fa3dc151bb0 2024-05-08T22:34:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.557+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3cc003a40 con 0x7fa3dc151bb0 2024-05-08T22:34:43.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.557+0000 7fa3d3fff700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 0x7fa3bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.557+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa3cc084010 con 0x7fa3dc151bb0 2024-05-08T22:34:43.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.557+0000 7fa3daffd700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 0x7fa3bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.557+0000 7fa3daffd700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 0x7fa3bc068c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0b2e80 tx=0x7fa3c4007aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.565+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa3cc0512d0 con 0x7fa3dc151bb0 2024-05-08T22:34:43.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:43.777+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa3dc0b2bd0 con 0x7fa3dc151bb0 2024-05-08T22:34:44.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.405+0000 7fa3d3fff700 1 -- 172.21.15.179:0/275586031 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa3cc0548f0 con 0x7fa3dc151bb0 2024-05-08T22:34:44.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 msgr2=0x7fa3bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 0x7fa3bc068c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0b2e80 tx=0x7fa3c4007aa0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 msgr2=0x7fa3dc0cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00f840 tx=0x7fa3cc00fb50 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 shutdown_connections 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3bc066760 0x7fa3bc068c20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3dc0c73e0 0x7fa3dc0b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3dc0cc630 0x7fa3dc0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 --2- 172.21.15.179:0/275586031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3dc151bb0 0x7fa3dc0cc0f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 >> 172.21.15.179:0/275586031 conn(0x7fa3dc0d3270 msgr2=0x7fa3dc0b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 shutdown_connections 2024-05-08T22:34:44.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.409+0000 7fa3e0f21700 1 -- 172.21.15.179:0/275586031 wait complete. 2024-05-08T22:34:44.428 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:34:44.428 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:44.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 Processor -- start 2024-05-08T22:34:44.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 -- start start 2024-05-08T22:34:44.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 0x7f90d006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d00cc310 0x7f90d00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d0161ac0 con 0x7f90d00add40 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d00ae550 con 0x7f90d00cc310 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d6d3e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d006abb0 con 0x7f90d00ce060 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d00cc310 0x7f90d00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d00cc310 0x7f90d00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41070/0 (socket says 172.21.15.179:41070) 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 -- 172.21.15.179:0/1995224450 learned_addr learned my addr 172.21.15.179:0/1995224450 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d4ada700 1 --2- 172.21.15.179:0/1995224450 >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 0x7f90d006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 -- 172.21.15.179:0/1995224450 >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 msgr2=0x7f90d006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 --2- 172.21.15.179:0/1995224450 >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 0x7f90d006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 -- 172.21.15.179:0/1995224450 >> v1:172.21.15.195:6789/0 conn(0x7f90d00add40 legacy=0x7f90d006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 -- 172.21.15.179:0/1995224450 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d00ce980 con 0x7f90d00cc310 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cf7fe700 1 -- 172.21.15.179:0/1995224450 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075978279 0 0) 0x7f90d0161ac0 con 0x7f90d00add40 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90d4ada700 1 --2- 172.21.15.179:0/1995224450 >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 0x7f90d006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cffff700 1 --2- 172.21.15.179:0/1995224450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d00ce440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f90d0074410 tx=0x7f90c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df15eeb6b59bc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cf7fe700 1 -- 172.21.15.179:0/1995224450 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800e070 con 0x7f90d00cc310 2024-05-08T22:34:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cf7fe700 1 -- 172.21.15.179:0/1995224450 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800b910 con 0x7f90d00cc310 2024-05-08T22:34:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.545+0000 7f90cf7fe700 1 -- 172.21.15.179:0/1995224450 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800bcc0 con 0x7f90d00cc310 2024-05-08T22:34:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- 172.21.15.179:0/1995224450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 msgr2=0x7f90d00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/1995224450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d00ce440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f90d0074410 tx=0x7f90c800b330 comp rx=0 tx=0).stop 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- 172.21.15.179:0/1995224450 shutdown_connections 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/1995224450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d00ce440 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/1995224450 >> v2:172.21.15.195:3300/0 conn(0x7f90d00ce060 0x7f90d006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- 172.21.15.179:0/1995224450 >> 172.21.15.179:0/1995224450 conn(0x7f90d00cb660 msgr2=0x7f90d00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- 172.21.15.179:0/1995224450 shutdown_connections 2024-05-08T22:34:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- 172.21.15.179:0/1995224450 wait complete. 2024-05-08T22:34:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 Processor -- start 2024-05-08T22:34:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- start start 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 0x7f90d00c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d00ce060 0x7f90d00c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d015b390 con 0x7f90d00cc310 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d015de30 con 0x7f90d00ce060 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.549+0000 7f90d6d3e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00b32e0 con 0x7f90d00add40 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41082/0 (socket says 172.21.15.179:41082) 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 -- 172.21.15.179:0/2323576070 learned_addr learned my addr 172.21.15.179:0/2323576070 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d4ada700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 0x7f90d00c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d52db700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d00ce060 0x7f90d00c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 -- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 msgr2=0x7f90d00c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 0x7f90d00c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 -- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d00ce060 msgr2=0x7f90d00c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d00ce060 0x7f90d00c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 -- 172.21.15.179:0/2323576070 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80094b0 con 0x7f90d00cc310 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d4ada700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 0x7f90d00c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cffff700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f90c800b370 tx=0x7f90c8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80092e0 con 0x7f90d00cc310 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c801c440 con 0x7f90d00cc310 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c8009b30 con 0x7f90d00cc310 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c801c810 con 0x7f90d00cc310 2024-05-08T22:34:44.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d00b3580 con 0x7f90d00cc310 2024-05-08T22:34:44.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d0067770 con 0x7f90d00cc310 2024-05-08T22:34:44.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90c801c970 con 0x7f90d00cc310 2024-05-08T22:34:44.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.553+0000 7f90cd7fa700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 0x7f90b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.557+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f90c8085cb0 con 0x7f90d00cc310 2024-05-08T22:34:44.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.557+0000 7f90d4ada700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 0x7f90b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.557+0000 7f90d4ada700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 0x7f90b0068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009dd0 tx=0x7f90b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.565+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90c8052f70 con 0x7f90d00cc310 2024-05-08T22:34:44.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.773+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90d0073290 con 0x7f90d00cc310 2024-05-08T22:34:44.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.773+0000 7f90cd7fa700 1 -- 172.21.15.179:0/2323576070 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f90c8056590 con 0x7f90d00cc310 2024-05-08T22:34:44.775 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:44.775 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 msgr2=0x7f90b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 0x7f90b0068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009dd0 tx=0x7f90b8008040 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 msgr2=0x7f90d015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f90c800b370 tx=0x7f90c8004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 shutdown_connections 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f90b0066730 0x7f90b0068bf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90d00ce060 0x7f90d00c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d00cc310 0x7f90d015b500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 --2- 172.21.15.179:0/2323576070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90d00add40 0x7f90d00c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 >> 172.21.15.179:0/2323576070 conn(0x7f90d00cb660 msgr2=0x7f90d00af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 shutdown_connections 2024-05-08T22:34:44.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.777+0000 7f90d6d3e700 1 -- 172.21.15.179:0/2323576070 wait complete. 2024-05-08T22:34:44.781 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:34:44.794 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:44.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 Processor -- start 2024-05-08T22:34:44.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 -- start start 2024-05-08T22:34:44.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9ea00b4250 0x7f9ea00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00b1b50 con 0x7f9ea00b5110 2024-05-08T22:34:44.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00c5e60 con 0x7f9ea00b4250 2024-05-08T22:34:44.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea64de700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00c6390 con 0x7f9ea00c5590 2024-05-08T22:34:44.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9e9ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9ea00b4250 0x7f9ea00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9e977fe700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f9ea00b5110 0x7f9ea015dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:33210/0 (socket says 172.21.15.179:33210) 2024-05-08T22:34:44.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9e977fe700 1 -- 172.21.15.179:0/1768418169 learned_addr learned my addr 172.21.15.179:0/1768418169 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:44.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.905+0000 7f9ea4a7b700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 msgr2=0x7f9ea015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1768418169 >> v1:172.21.15.195:6789/0 conn(0x7f9ea00b5110 legacy=0x7f9ea015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9ffff700 1 -- 172.21.15.179:0/1768418169 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00b4b70 con 0x7f9ea00b4250 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea00b4630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009850 tx=0x7f9e90009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7536d767f2163c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9f7fe700 1 -- 172.21.15.179:0/1768418169 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000f8a0 con 0x7f9ea00b4250 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9f7fe700 1 -- 172.21.15.179:0/1768418169 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9000fa00 con 0x7f9ea00b4250 2024-05-08T22:34:44.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9e9f7fe700 1 -- 172.21.15.179:0/1768418169 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000fdb0 con 0x7f9ea00b4250 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 -- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 msgr2=0x7f9ea00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea00b4630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009850 tx=0x7f9e90009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 -- 172.21.15.179:0/1768418169 shutdown_connections 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1768418169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea00b4630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 -- 172.21.15.179:0/1768418169 >> 172.21.15.179:0/1768418169 conn(0x7f9ea0098a70 msgr2=0x7f9ea00b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 -- 172.21.15.179:0/1768418169 shutdown_connections 2024-05-08T22:34:44.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.909+0000 7f9ea64de700 1 -- 172.21.15.179:0/1768418169 wait complete. 2024-05-08T22:34:44.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 Processor -- start 2024-05-08T22:34:44.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- start start 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea01513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea006b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01e41e0 con 0x7f9ea00b5110 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01e44c0 con 0x7f9ea00b4250 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01e47a0 con 0x7f9ea00c5590 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41088/0 (socket says 172.21.15.179:41088) 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 -- 172.21.15.179:0/1628320744 learned_addr learned my addr 172.21.15.179:0/1628320744 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea4a7b700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea01513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 -- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 msgr2=0x7f9ea006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea006b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 -- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 msgr2=0x7f9ea01513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea01513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 -- 172.21.15.179:0/1628320744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e900094b0 con 0x7f9ea00b5110 2024-05-08T22:34:44.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e977fe700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e88001eb0 tx=0x7f9e8800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88005d90 con 0x7f9ea00b5110 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e880070d0 con 0x7f9ea00b5110 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea01e4a80 con 0x7f9ea00b5110 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88007440 con 0x7f9ea00b5110 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.913+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01518e0 con 0x7f9ea00b5110 2024-05-08T22:34:44.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0073290 con 0x7f9ea00b5110 2024-05-08T22:34:44.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e880075a0 con 0x7f9ea00b5110 2024-05-08T22:34:44.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9e9d7fa700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9e88083540 con 0x7f9ea00b5110 2024-05-08T22:34:44.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.917+0000 7f9e9ffff700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e900090b0 tx=0x7f9e9001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:44.925+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e88050780 con 0x7f9ea00b5110 2024-05-08T22:34:44.963 INFO:tasks.ceph.mds.c.smithi179.stderr:2024-05-08T22:34:44.961+0000 7f5a5e205700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:34:45.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.101+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea00b4ef0 con 0x7f9ea00b5110 2024-05-08T22:34:45.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.101+0000 7f9e9d7fa700 1 -- 172.21.15.179:0/1628320744 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16354 (secure 0 0 0) 0x7f9e88053da0 con 0x7f9ea00b5110 2024-05-08T22:34:45.105 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:45.106 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":41,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:42.384705+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 msgr2=0x7f9e80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e900090b0 tx=0x7f9e9001b040 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 msgr2=0x7f9ea01e37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e88001eb0 tx=0x7f9e8800c7b0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 shutdown_connections 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e80066730 0x7f9e80068bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea00c5590 0x7f9ea006b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ea00b5110 0x7f9ea01e37e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 --2- 172.21.15.179:0/1628320744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea00b4250 0x7f9ea01513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 >> 172.21.15.179:0/1628320744 conn(0x7f9ea0098a70 msgr2=0x7f9ea00af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 shutdown_connections 2024-05-08T22:34:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.105+0000 7f9ea64de700 1 -- 172.21.15.179:0/1628320744 wait complete. 2024-05-08T22:34:45.124 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:34:45.124 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:34:45.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 Processor -- start 2024-05-08T22:34:45.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 -- start start 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 0x7fe0b406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe0b40add40 0x7fe0b406b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b4161ac0 con 0x7fe0b40ce060 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b40ae550 con 0x7fe0b40add40 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0bbbaf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b406abb0 con 0x7fe0b40cc310 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b994b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe0b40ce060 0x7fe0b40ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38248/0 (socket says 172.21.15.179:38248) 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b994b700 1 -- 172.21.15.179:0/3522989670 learned_addr learned my addr 172.21.15.179:0/3522989670 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/3522989670 >> v2:172.21.15.195:3300/0 conn(0x7fe0b40add40 0x7fe0b406b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b914a700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 0x7fe0b406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b8949700 1 -- 172.21.15.179:0/3522989670 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195500568 0 0) 0x7fe0b4161ac0 con 0x7fe0b40ce060 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b8949700 1 -- 172.21.15.179:0/3522989670 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a00035d0 con 0x7fe0b40ce060 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 -- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 msgr2=0x7fe0b406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 0x7fe0b406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 -- 172.21.15.179:0/3522989670 >> v1:172.21.15.179:6789/0 conn(0x7fe0b40ce060 legacy=0x7fe0b40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 -- 172.21.15.179:0/3522989670 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40cdaa0 con 0x7fe0b40add40 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b914a700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 0x7fe0b406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40add40 0x7fe0b406b8a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009850 tx=0x7fe0b0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c9c7ed21a3b3cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b8949700 1 -- 172.21.15.179:0/3522989670 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0011830 con 0x7fe0b40add40 2024-05-08T22:34:45.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b8949700 1 -- 172.21.15.179:0/3522989670 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0011990 con 0x7fe0b40add40 2024-05-08T22:34:45.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.233+0000 7fe0b8949700 1 -- 172.21.15.179:0/3522989670 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0011d40 con 0x7fe0b40add40 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40add40 msgr2=0x7fe0b406b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40add40 0x7fe0b406b8a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009850 tx=0x7fe0b0009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/3522989670 shutdown_connections 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40add40 0x7fe0b406b8a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/3522989670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40cc310 0x7fe0b406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/3522989670 >> 172.21.15.179:0/3522989670 conn(0x7fe0b40cb660 msgr2=0x7fe0b40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/3522989670 shutdown_connections 2024-05-08T22:34:45.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.237+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/3522989670 wait complete. 2024-05-08T22:34:45.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 Processor -- start 2024-05-08T22:34:45.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 -- start start 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40add40 0x7fe0b41f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40cc310 0x7fe0b40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40c3a30 con 0x7fe0b40ce060 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40c3d10 con 0x7fe0b40cc310 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0bbbaf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40c3ff0 con 0x7fe0b40add40 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0b994b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40add40 0x7fe0b41f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0b914a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40cc310 0x7fe0b40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41092/0 (socket says 172.21.15.179:41092) 2024-05-08T22:34:45.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 -- 172.21.15.179:0/1433880206 learned_addr learned my addr 172.21.15.179:0/1433880206 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 -- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40add40 msgr2=0x7fe0b41f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40add40 0x7fe0b41f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 -- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40cc310 msgr2=0x7fe0b40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40cc310 0x7fe0b40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 -- 172.21.15.179:0/1433880206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a80094b0 con 0x7fe0b40ce060 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0ba14c700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00090b0 tx=0x7fe0b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.241+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004510 con 0x7fe0b40ce060 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b001e070 con 0x7fe0b40ce060 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b00094b0 con 0x7fe0b40ce060 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00038e0 con 0x7fe0b40ce060 2024-05-08T22:34:45.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b415e290 con 0x7fe0b40ce060 2024-05-08T22:34:45.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b4067770 con 0x7fe0b40ce060 2024-05-08T22:34:45.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0b0003a40 con 0x7fe0b40ce060 2024-05-08T22:34:45.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0a67fc700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 0x7fe098068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe0b0020070 con 0x7fe0b40ce060 2024-05-08T22:34:45.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0b994b700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 0x7fe098068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.245+0000 7fe0b994b700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 0x7fe098068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac001560 tx=0x7fe0ac001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.249+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe0b0052820 con 0x7fe0b40ce060 2024-05-08T22:34:45.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.417+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0b406b380 con 0x7fe098066730 2024-05-08T22:34:45.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.437+0000 7fe0a67fc700 1 -- 172.21.15.179:0/1433880206 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287987 (secure 0 0 0) 0x7fe0ac0077e0 con 0x7fe098066730 2024-05-08T22:34:45.442 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 msgr2=0x7fe098068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 0x7fe098068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac001560 tx=0x7fe0ac001a90 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 msgr2=0x7fe0b40b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00090b0 tx=0x7fe0b0004770 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 shutdown_connections 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe098066730 0x7fe098068bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b40cc310 0x7fe0b40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0b40ce060 0x7fe0b40b2a60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 --2- 172.21.15.179:0/1433880206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0b40add40 0x7fe0b41f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 >> 172.21.15.179:0/1433880206 conn(0x7fe0b40cb660 msgr2=0x7fe0b40af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 shutdown_connections 2024-05-08T22:34:45.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.441+0000 7fe0bbbaf700 1 -- 172.21.15.179:0/1433880206 wait complete. 2024-05-08T22:34:45.446 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:34:45.458 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-05-08T22:34:45.409818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237412,"kb_used_data":4824,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737308,"statfs":{"total":773094113280,"available":772851003392,"internally_reserved":0,"allocated":4939776,"data_stored":2769900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.427707"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397715+0000","last_change":"2024-05-08T22:34:41.400635+0000","last_active":"2024-05-08T22:34:42.397715+0000","last_peered":"2024-05-08T22:34:42.397715+0000","last_clean":"2024-05-08T22:34:42.397715+0000","last_became_active":"2024-05-08T22:34:41.400383+0000","last_became_peered":"2024-05-08T22:34:41.400383+0000","last_unstale":"2024-05-08T22:34:42.397715+0000","last_undegraded":"2024-05-08T22:34:42.397715+0000","last_fullsized":"2024-05-08T22:34:42.397715+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:41:00.914039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397049+0000","last_change":"2024-05-08T22:34:41.406468+0000","last_active":"2024-05-08T22:34:42.397049+0000","last_peered":"2024-05-08T22:34:42.397049+0000","last_clean":"2024-05-08T22:34:42.397049+0000","last_became_active":"2024-05-08T22:34:41.406201+0000","last_became_peered":"2024-05-08T22:34:41.406201+0000","last_unstale":"2024-05-08T22:34:42.397049+0000","last_undegraded":"2024-05-08T22:34:42.397049+0000","last_fullsized":"2024-05-08T22:34:42.397049+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:35.110806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408715+0000","last_change":"2024-05-08T22:34:41.405938+0000","last_active":"2024-05-08T22:34:41.408715+0000","last_peered":"2024-05-08T22:34:41.408715+0000","last_clean":"2024-05-08T22:34:41.408715+0000","last_became_active":"2024-05-08T22:34:41.405687+0000","last_became_peered":"2024-05-08T22:34:41.405687+0000","last_unstale":"2024-05-08T22:34:41.408715+0000","last_undegraded":"2024-05-08T22:34:41.408715+0000","last_fullsized":"2024-05-08T22:34:41.408715+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728167+0000","last_change":"2024-05-08T22:34:41.403568+0000","last_active":"2024-05-08T22:34:42.728167+0000","last_peered":"2024-05-08T22:34:42.728167+0000","last_clean":"2024-05-08T22:34:42.728167+0000","last_became_active":"2024-05-08T22:34:41.403317+0000","last_became_peered":"2024-05-08T22:34:41.403317+0000","last_unstale":"2024-05-08T22:34:42.728167+0000","last_undegraded":"2024-05-08T22:34:42.728167+0000","last_fullsized":"2024-05-08T22:34:42.728167+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:20.585668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728278+0000","last_change":"2024-05-08T22:34:41.398864+0000","last_active":"2024-05-08T22:34:42.728278+0000","last_peered":"2024-05-08T22:34:42.728278+0000","last_clean":"2024-05-08T22:34:42.728278+0000","last_became_active":"2024-05-08T22:34:41.398644+0000","last_became_peered":"2024-05-08T22:34:41.398644+0000","last_unstale":"2024-05-08T22:34:42.728278+0000","last_undegraded":"2024-05-08T22:34:42.728278+0000","last_fullsized":"2024-05-08T22:34:42.728278+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:45.722225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951999+0000","last_change":"2024-05-08T22:34:41.408773+0000","last_active":"2024-05-08T22:34:43.951999+0000","last_peered":"2024-05-08T22:34:43.951999+0000","last_clean":"2024-05-08T22:34:43.951999+0000","last_became_active":"2024-05-08T22:34:41.408522+0000","last_became_peered":"2024-05-08T22:34:41.408522+0000","last_unstale":"2024-05-08T22:34:43.951999+0000","last_undegraded":"2024-05-08T22:34:43.951999+0000","last_fullsized":"2024-05-08T22:34:43.951999+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:43.790818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408731+0000","last_change":"2024-05-08T22:34:41.405010+0000","last_active":"2024-05-08T22:34:41.408731+0000","last_peered":"2024-05-08T22:34:41.408731+0000","last_clean":"2024-05-08T22:34:41.408731+0000","last_became_active":"2024-05-08T22:34:41.404745+0000","last_became_peered":"2024-05-08T22:34:41.404745+0000","last_unstale":"2024-05-08T22:34:41.408731+0000","last_undegraded":"2024-05-08T22:34:41.408731+0000","last_fullsized":"2024-05-08T22:34:41.408731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396050+0000","last_change":"2024-05-08T22:34:41.406886+0000","last_active":"2024-05-08T22:34:42.396050+0000","last_peered":"2024-05-08T22:34:42.396050+0000","last_clean":"2024-05-08T22:34:42.396050+0000","last_became_active":"2024-05-08T22:34:41.406193+0000","last_became_peered":"2024-05-08T22:34:41.406193+0000","last_unstale":"2024-05-08T22:34:42.396050+0000","last_undegraded":"2024-05-08T22:34:42.396050+0000","last_fullsized":"2024-05-08T22:34:42.396050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:41.552480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408589+0000","last_change":"2024-05-08T22:34:41.405675+0000","last_active":"2024-05-08T22:34:41.408589+0000","last_peered":"2024-05-08T22:34:41.408589+0000","last_clean":"2024-05-08T22:34:41.408589+0000","last_became_active":"2024-05-08T22:34:41.405172+0000","last_became_peered":"2024-05-08T22:34:41.405172+0000","last_unstale":"2024-05-08T22:34:41.408589+0000","last_undegraded":"2024-05-08T22:34:41.408589+0000","last_fullsized":"2024-05-08T22:34:41.408589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418451+0000","last_change":"2024-05-08T22:34:41.417348+0000","last_active":"2024-05-08T22:34:41.418451+0000","last_peered":"2024-05-08T22:34:41.418451+0000","last_clean":"2024-05-08T22:34:41.418451+0000","last_became_active":"2024-05-08T22:34:41.417050+0000","last_became_peered":"2024-05-08T22:34:41.417050+0000","last_unstale":"2024-05-08T22:34:41.418451+0000","last_undegraded":"2024-05-08T22:34:41.418451+0000","last_fullsized":"2024-05-08T22:34:41.418451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:32.559809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416462+0000","last_change":"2024-05-08T22:34:41.409198+0000","last_active":"2024-05-08T22:34:41.416462+0000","last_peered":"2024-05-08T22:34:41.416462+0000","last_clean":"2024-05-08T22:34:41.416462+0000","last_became_active":"2024-05-08T22:34:41.408932+0000","last_became_peered":"2024-05-08T22:34:41.408932+0000","last_unstale":"2024-05-08T22:34:41.416462+0000","last_undegraded":"2024-05-08T22:34:41.416462+0000","last_fullsized":"2024-05-08T22:34:41.416462+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408517+0000","last_change":"2024-05-08T22:34:41.404348+0000","last_active":"2024-05-08T22:34:41.408517+0000","last_peered":"2024-05-08T22:34:41.408517+0000","last_clean":"2024-05-08T22:34:41.408517+0000","last_became_active":"2024-05-08T22:34:41.404094+0000","last_became_peered":"2024-05-08T22:34:41.404094+0000","last_unstale":"2024-05-08T22:34:41.408517+0000","last_undegraded":"2024-05-08T22:34:41.408517+0000","last_fullsized":"2024-05-08T22:34:41.408517+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:41.792260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418932+0000","last_change":"2024-05-08T22:34:41.417278+0000","last_active":"2024-05-08T22:34:41.418932+0000","last_peered":"2024-05-08T22:34:41.418932+0000","last_clean":"2024-05-08T22:34:41.418932+0000","last_became_active":"2024-05-08T22:34:41.416991+0000","last_became_peered":"2024-05-08T22:34:41.416991+0000","last_unstale":"2024-05-08T22:34:41.418932+0000","last_undegraded":"2024-05-08T22:34:41.418932+0000","last_fullsized":"2024-05-08T22:34:41.418932+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:39.156787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396799+0000","last_change":"2024-05-08T22:34:41.396314+0000","last_active":"2024-05-08T22:34:42.396799+0000","last_peered":"2024-05-08T22:34:42.396799+0000","last_clean":"2024-05-08T22:34:42.396799+0000","last_became_active":"2024-05-08T22:34:41.396069+0000","last_became_peered":"2024-05-08T22:34:41.396069+0000","last_unstale":"2024-05-08T22:34:42.396799+0000","last_undegraded":"2024-05-08T22:34:42.396799+0000","last_fullsized":"2024-05-08T22:34:42.396799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:09.894172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408749+0000","last_change":"2024-05-08T22:34:41.406739+0000","last_active":"2024-05-08T22:34:41.408749+0000","last_peered":"2024-05-08T22:34:41.408749+0000","last_clean":"2024-05-08T22:34:41.408749+0000","last_became_active":"2024-05-08T22:34:41.406463+0000","last_became_peered":"2024-05-08T22:34:41.406463+0000","last_unstale":"2024-05-08T22:34:41.408749+0000","last_undegraded":"2024-05-08T22:34:41.408749+0000","last_fullsized":"2024-05-08T22:34:41.408749+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418502+0000","last_change":"2024-05-08T22:34:41.412305+0000","last_active":"2024-05-08T22:34:41.418502+0000","last_peered":"2024-05-08T22:34:41.418502+0000","last_clean":"2024-05-08T22:34:41.418502+0000","last_became_active":"2024-05-08T22:34:41.410310+0000","last_became_peered":"2024-05-08T22:34:41.410310+0000","last_unstale":"2024-05-08T22:34:41.418502+0000","last_undegraded":"2024-05-08T22:34:41.418502+0000","last_fullsized":"2024-05-08T22:34:41.418502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:06.200268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416481+0000","last_change":"2024-05-08T22:34:41.409690+0000","last_active":"2024-05-08T22:34:41.416481+0000","last_peered":"2024-05-08T22:34:41.416481+0000","last_clean":"2024-05-08T22:34:41.416481+0000","last_became_active":"2024-05-08T22:34:41.409428+0000","last_became_peered":"2024-05-08T22:34:41.409428+0000","last_unstale":"2024-05-08T22:34:41.416481+0000","last_undegraded":"2024-05-08T22:34:41.416481+0000","last_fullsized":"2024-05-08T22:34:41.416481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:19.779793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397118+0000","last_change":"2024-05-08T22:34:41.401758+0000","last_active":"2024-05-08T22:34:42.397118+0000","last_peered":"2024-05-08T22:34:42.397118+0000","last_clean":"2024-05-08T22:34:42.397118+0000","last_became_active":"2024-05-08T22:34:41.398599+0000","last_became_peered":"2024-05-08T22:34:41.398599+0000","last_unstale":"2024-05-08T22:34:42.397118+0000","last_undegraded":"2024-05-08T22:34:42.397118+0000","last_fullsized":"2024-05-08T22:34:42.397118+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:32.657624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728446+0000","last_change":"2024-05-08T22:34:41.415542+0000","last_active":"2024-05-08T22:34:42.728446+0000","last_peered":"2024-05-08T22:34:42.728446+0000","last_clean":"2024-05-08T22:34:42.728446+0000","last_became_active":"2024-05-08T22:34:41.415261+0000","last_became_peered":"2024-05-08T22:34:41.415261+0000","last_unstale":"2024-05-08T22:34:42.728446+0000","last_undegraded":"2024-05-08T22:34:42.728446+0000","last_fullsized":"2024-05-08T22:34:42.728446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:11.833382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396826+0000","last_change":"2024-05-08T22:34:41.398134+0000","last_active":"2024-05-08T22:34:42.396826+0000","last_peered":"2024-05-08T22:34:42.396826+0000","last_clean":"2024-05-08T22:34:42.396826+0000","last_became_active":"2024-05-08T22:34:41.397829+0000","last_became_peered":"2024-05-08T22:34:41.397829+0000","last_unstale":"2024-05-08T22:34:42.396826+0000","last_undegraded":"2024-05-08T22:34:42.396826+0000","last_fullsized":"2024-05-08T22:34:42.396826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:26.913962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416777+0000","last_change":"2024-05-08T22:34:41.413309+0000","last_active":"2024-05-08T22:34:41.416777+0000","last_peered":"2024-05-08T22:34:41.416777+0000","last_clean":"2024-05-08T22:34:41.416777+0000","last_became_active":"2024-05-08T22:34:41.413035+0000","last_became_peered":"2024-05-08T22:34:41.413035+0000","last_unstale":"2024-05-08T22:34:41.416777+0000","last_undegraded":"2024-05-08T22:34:41.416777+0000","last_fullsized":"2024-05-08T22:34:41.416777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:46.957082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396325+0000","last_change":"2024-05-08T22:34:41.407385+0000","last_active":"2024-05-08T22:34:42.396325+0000","last_peered":"2024-05-08T22:34:42.396325+0000","last_clean":"2024-05-08T22:34:42.396325+0000","last_became_active":"2024-05-08T22:34:41.407088+0000","last_became_peered":"2024-05-08T22:34:41.407088+0000","last_unstale":"2024-05-08T22:34:42.396325+0000","last_undegraded":"2024-05-08T22:34:42.396325+0000","last_fullsized":"2024-05-08T22:34:42.396325+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:32.109467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416612+0000","last_change":"2024-05-08T22:34:41.410175+0000","last_active":"2024-05-08T22:34:41.416612+0000","last_peered":"2024-05-08T22:34:41.416612+0000","last_clean":"2024-05-08T22:34:41.416612+0000","last_became_active":"2024-05-08T22:34:41.409635+0000","last_became_peered":"2024-05-08T22:34:41.409635+0000","last_unstale":"2024-05-08T22:34:41.416612+0000","last_undegraded":"2024-05-08T22:34:41.416612+0000","last_fullsized":"2024-05-08T22:34:41.416612+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.727869+0000","last_change":"2024-05-08T22:34:41.400291+0000","last_active":"2024-05-08T22:34:42.727869+0000","last_peered":"2024-05-08T22:34:42.727869+0000","last_clean":"2024-05-08T22:34:42.727869+0000","last_became_active":"2024-05-08T22:34:41.400051+0000","last_became_peered":"2024-05-08T22:34:41.400051+0000","last_unstale":"2024-05-08T22:34:42.727869+0000","last_undegraded":"2024-05-08T22:34:42.727869+0000","last_fullsized":"2024-05-08T22:34:42.727869+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:31.084389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397124+0000","last_change":"2024-05-08T22:34:41.398686+0000","last_active":"2024-05-08T22:34:42.397124+0000","last_peered":"2024-05-08T22:34:42.397124+0000","last_clean":"2024-05-08T22:34:42.397124+0000","last_became_active":"2024-05-08T22:34:41.398407+0000","last_became_peered":"2024-05-08T22:34:41.398407+0000","last_unstale":"2024-05-08T22:34:42.397124+0000","last_undegraded":"2024-05-08T22:34:42.397124+0000","last_fullsized":"2024-05-08T22:34:42.397124+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:20.661226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952453+0000","last_change":"2024-05-08T22:34:41.408211+0000","last_active":"2024-05-08T22:34:43.952453+0000","last_peered":"2024-05-08T22:34:43.952453+0000","last_clean":"2024-05-08T22:34:43.952453+0000","last_became_active":"2024-05-08T22:34:41.407935+0000","last_became_peered":"2024-05-08T22:34:41.407935+0000","last_unstale":"2024-05-08T22:34:43.952453+0000","last_undegraded":"2024-05-08T22:34:43.952453+0000","last_fullsized":"2024-05-08T22:34:43.952453+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:03.265817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396906+0000","last_change":"2024-05-08T22:34:41.396020+0000","last_active":"2024-05-08T22:34:42.396906+0000","last_peered":"2024-05-08T22:34:42.396906+0000","last_clean":"2024-05-08T22:34:42.396906+0000","last_became_active":"2024-05-08T22:34:41.395704+0000","last_became_peered":"2024-05-08T22:34:41.395704+0000","last_unstale":"2024-05-08T22:34:42.396906+0000","last_undegraded":"2024-05-08T22:34:42.396906+0000","last_fullsized":"2024-05-08T22:34:42.396906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:43:51.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416840+0000","last_change":"2024-05-08T22:34:41.413564+0000","last_active":"2024-05-08T22:34:41.416840+0000","last_peered":"2024-05-08T22:34:41.416840+0000","last_clean":"2024-05-08T22:34:41.416840+0000","last_became_active":"2024-05-08T22:34:41.413319+0000","last_became_peered":"2024-05-08T22:34:41.413319+0000","last_unstale":"2024-05-08T22:34:41.416840+0000","last_undegraded":"2024-05-08T22:34:41.416840+0000","last_fullsized":"2024-05-08T22:34:41.416840+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397225+0000","last_change":"2024-05-08T22:34:41.411078+0000","last_active":"2024-05-08T22:34:42.397225+0000","last_peered":"2024-05-08T22:34:42.397225+0000","last_clean":"2024-05-08T22:34:42.397225+0000","last_became_active":"2024-05-08T22:34:41.410804+0000","last_became_peered":"2024-05-08T22:34:41.410804+0000","last_unstale":"2024-05-08T22:34:42.397225+0000","last_undegraded":"2024-05-08T22:34:42.397225+0000","last_fullsized":"2024-05-08T22:34:42.397225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:52.399767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396889+0000","last_change":"2024-05-08T22:34:41.402575+0000","last_active":"2024-05-08T22:34:42.396889+0000","last_peered":"2024-05-08T22:34:42.396889+0000","last_clean":"2024-05-08T22:34:42.396889+0000","last_became_active":"2024-05-08T22:34:41.402246+0000","last_became_peered":"2024-05-08T22:34:41.402246+0000","last_unstale":"2024-05-08T22:34:42.396889+0000","last_undegraded":"2024-05-08T22:34:42.396889+0000","last_fullsized":"2024-05-08T22:34:42.396889+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:35.232767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416634+0000","last_change":"2024-05-08T22:34:41.410214+0000","last_active":"2024-05-08T22:34:41.416634+0000","last_peered":"2024-05-08T22:34:41.416634+0000","last_clean":"2024-05-08T22:34:41.416634+0000","last_became_active":"2024-05-08T22:34:41.409950+0000","last_became_peered":"2024-05-08T22:34:41.409950+0000","last_unstale":"2024-05-08T22:34:41.416634+0000","last_undegraded":"2024-05-08T22:34:41.416634+0000","last_fullsized":"2024-05-08T22:34:41.416634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951884+0000","last_change":"2024-05-08T22:34:41.410973+0000","last_active":"2024-05-08T22:34:43.951884+0000","last_peered":"2024-05-08T22:34:43.951884+0000","last_clean":"2024-05-08T22:34:43.951884+0000","last_became_active":"2024-05-08T22:34:41.410730+0000","last_became_peered":"2024-05-08T22:34:41.410730+0000","last_unstale":"2024-05-08T22:34:43.951884+0000","last_undegraded":"2024-05-08T22:34:43.951884+0000","last_fullsized":"2024-05-08T22:34:43.951884+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:57.440176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416635+0000","last_change":"2024-05-08T22:34:41.410066+0000","last_active":"2024-05-08T22:34:41.416635+0000","last_peered":"2024-05-08T22:34:41.416635+0000","last_clean":"2024-05-08T22:34:41.416635+0000","last_became_active":"2024-05-08T22:34:41.409778+0000","last_became_peered":"2024-05-08T22:34:41.409778+0000","last_unstale":"2024-05-08T22:34:41.416635+0000","last_undegraded":"2024-05-08T22:34:41.416635+0000","last_fullsized":"2024-05-08T22:34:41.416635+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397232+0000","last_change":"2024-05-08T22:34:41.014993+0000","last_active":"2024-05-08T22:34:42.397232+0000","last_peered":"2024-05-08T22:34:42.397232+0000","last_clean":"2024-05-08T22:34:42.397232+0000","last_became_active":"2024-05-08T22:34:41.014759+0000","last_became_peered":"2024-05-08T22:34:41.014759+0000","last_unstale":"2024-05-08T22:34:42.397232+0000","last_undegraded":"2024-05-08T22:34:42.397232+0000","last_fullsized":"2024-05-08T22:34:42.397232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:30.375135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952309+0000","last_change":"2024-05-08T22:34:41.409031+0000","last_active":"2024-05-08T22:34:43.952309+0000","last_peered":"2024-05-08T22:34:43.952309+0000","last_clean":"2024-05-08T22:34:43.952309+0000","last_became_active":"2024-05-08T22:34:41.408750+0000","last_became_peered":"2024-05-08T22:34:41.408750+0000","last_unstale":"2024-05-08T22:34:43.952309+0000","last_undegraded":"2024-05-08T22:34:43.952309+0000","last_fullsized":"2024-05-08T22:34:43.952309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:09.088402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418082+0000","last_change":"2024-05-08T22:34:41.017178+0000","last_active":"2024-05-08T22:34:41.418082+0000","last_peered":"2024-05-08T22:34:41.418082+0000","last_clean":"2024-05-08T22:34:41.418082+0000","last_became_active":"2024-05-08T22:34:41.016873+0000","last_became_peered":"2024-05-08T22:34:41.016873+0000","last_unstale":"2024-05-08T22:34:41.418082+0000","last_undegraded":"2024-05-08T22:34:41.418082+0000","last_fullsized":"2024-05-08T22:34:41.418082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397220+0000","last_change":"2024-05-08T22:34:41.396198+0000","last_active":"2024-05-08T22:34:42.397220+0000","last_peered":"2024-05-08T22:34:42.397220+0000","last_clean":"2024-05-08T22:34:42.397220+0000","last_became_active":"2024-05-08T22:34:41.395930+0000","last_became_peered":"2024-05-08T22:34:41.395930+0000","last_unstale":"2024-05-08T22:34:42.397220+0000","last_undegraded":"2024-05-08T22:34:42.397220+0000","last_fullsized":"2024-05-08T22:34:42.397220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:48.608036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952428+0000","last_change":"2024-05-08T22:34:41.022801+0000","last_active":"2024-05-08T22:34:43.952428+0000","last_peered":"2024-05-08T22:34:43.952428+0000","last_clean":"2024-05-08T22:34:43.952428+0000","last_became_active":"2024-05-08T22:34:41.022466+0000","last_became_peered":"2024-05-08T22:34:41.022466+0000","last_unstale":"2024-05-08T22:34:43.952428+0000","last_undegraded":"2024-05-08T22:34:43.952428+0000","last_fullsized":"2024-05-08T22:34:43.952428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:39.592583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397682+0000","last_change":"2024-05-08T22:34:41.400001+0000","last_active":"2024-05-08T22:34:42.397682+0000","last_peered":"2024-05-08T22:34:42.397682+0000","last_clean":"2024-05-08T22:34:42.397682+0000","last_became_active":"2024-05-08T22:34:41.399744+0000","last_became_peered":"2024-05-08T22:34:41.399744+0000","last_unstale":"2024-05-08T22:34:42.397682+0000","last_undegraded":"2024-05-08T22:34:42.397682+0000","last_fullsized":"2024-05-08T22:34:42.397682+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:15.302436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396423+0000","last_change":"2024-05-08T22:34:41.017194+0000","last_active":"2024-05-08T22:34:42.396423+0000","last_peered":"2024-05-08T22:34:42.396423+0000","last_clean":"2024-05-08T22:34:42.396423+0000","last_became_active":"2024-05-08T22:34:41.016905+0000","last_became_peered":"2024-05-08T22:34:41.016905+0000","last_unstale":"2024-05-08T22:34:42.396423+0000","last_undegraded":"2024-05-08T22:34:42.396423+0000","last_fullsized":"2024-05-08T22:34:42.396423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:05.189490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.413614+0000","last_change":"2024-05-08T22:34:41.411887+0000","last_active":"2024-05-08T22:34:41.413614+0000","last_peered":"2024-05-08T22:34:41.413614+0000","last_clean":"2024-05-08T22:34:41.413614+0000","last_became_active":"2024-05-08T22:34:41.411598+0000","last_became_peered":"2024-05-08T22:34:41.411598+0000","last_unstale":"2024-05-08T22:34:41.413614+0000","last_undegraded":"2024-05-08T22:34:41.413614+0000","last_fullsized":"2024-05-08T22:34:41.413614+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.409372+0000","last_change":"2024-05-08T22:34:41.016887+0000","last_active":"2024-05-08T22:34:41.409372+0000","last_peered":"2024-05-08T22:34:41.409372+0000","last_clean":"2024-05-08T22:34:41.409372+0000","last_became_active":"2024-05-08T22:34:41.016652+0000","last_became_peered":"2024-05-08T22:34:41.016652+0000","last_unstale":"2024-05-08T22:34:41.409372+0000","last_undegraded":"2024-05-08T22:34:41.409372+0000","last_fullsized":"2024-05-08T22:34:41.409372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:30.073611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.417060+0000","last_change":"2024-05-08T22:34:41.414428+0000","last_active":"2024-05-08T22:34:41.417060+0000","last_peered":"2024-05-08T22:34:41.417060+0000","last_clean":"2024-05-08T22:34:41.417060+0000","last_became_active":"2024-05-08T22:34:41.414049+0000","last_became_peered":"2024-05-08T22:34:41.414049+0000","last_unstale":"2024-05-08T22:34:41.417060+0000","last_undegraded":"2024-05-08T22:34:41.417060+0000","last_fullsized":"2024-05-08T22:34:41.417060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:06.200268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416557+0000","last_change":"2024-05-08T22:34:41.013385+0000","last_active":"2024-05-08T22:34:41.416557+0000","last_peered":"2024-05-08T22:34:41.416557+0000","last_clean":"2024-05-08T22:34:41.416557+0000","last_became_active":"2024-05-08T22:34:41.013124+0000","last_became_peered":"2024-05-08T22:34:41.013124+0000","last_unstale":"2024-05-08T22:34:41.416557+0000","last_undegraded":"2024-05-08T22:34:41.416557+0000","last_fullsized":"2024-05-08T22:34:41.416557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:15.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396493+0000","last_change":"2024-05-08T22:34:41.410713+0000","last_active":"2024-05-08T22:34:42.396493+0000","last_peered":"2024-05-08T22:34:42.396493+0000","last_clean":"2024-05-08T22:34:42.396493+0000","last_became_active":"2024-05-08T22:34:41.410400+0000","last_became_peered":"2024-05-08T22:34:41.410400+0000","last_unstale":"2024-05-08T22:34:42.396493+0000","last_undegraded":"2024-05-08T22:34:42.396493+0000","last_fullsized":"2024-05-08T22:34:42.396493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:58.033758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951894+0000","last_change":"2024-05-08T22:34:41.022120+0000","last_active":"2024-05-08T22:34:43.951894+0000","last_peered":"2024-05-08T22:34:43.951894+0000","last_clean":"2024-05-08T22:34:43.951894+0000","last_became_active":"2024-05-08T22:34:41.021821+0000","last_became_peered":"2024-05-08T22:34:41.021821+0000","last_unstale":"2024-05-08T22:34:43.951894+0000","last_undegraded":"2024-05-08T22:34:43.951894+0000","last_fullsized":"2024-05-08T22:34:43.951894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:54.382606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396164+0000","last_change":"2024-05-08T22:34:41.407696+0000","last_active":"2024-05-08T22:34:42.396164+0000","last_peered":"2024-05-08T22:34:42.396164+0000","last_clean":"2024-05-08T22:34:42.396164+0000","last_became_active":"2024-05-08T22:34:41.403758+0000","last_became_peered":"2024-05-08T22:34:41.403758+0000","last_unstale":"2024-05-08T22:34:42.396164+0000","last_undegraded":"2024-05-08T22:34:42.396164+0000","last_fullsized":"2024-05-08T22:34:42.396164+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:40.662497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.395887+0000","last_change":"2024-05-08T22:34:41.017190+0000","last_active":"2024-05-08T22:34:42.395887+0000","last_peered":"2024-05-08T22:34:42.395887+0000","last_clean":"2024-05-08T22:34:42.395887+0000","last_became_active":"2024-05-08T22:34:41.016911+0000","last_became_peered":"2024-05-08T22:34:41.016911+0000","last_unstale":"2024-05-08T22:34:42.395887+0000","last_undegraded":"2024-05-08T22:34:42.395887+0000","last_fullsized":"2024-05-08T22:34:42.395887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:31.549031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728361+0000","last_change":"2024-05-08T22:34:41.398502+0000","last_active":"2024-05-08T22:34:42.728361+0000","last_peered":"2024-05-08T22:34:42.728361+0000","last_clean":"2024-05-08T22:34:42.728361+0000","last_became_active":"2024-05-08T22:34:41.398206+0000","last_became_peered":"2024-05-08T22:34:41.398206+0000","last_unstale":"2024-05-08T22:34:42.728361+0000","last_undegraded":"2024-05-08T22:34:42.728361+0000","last_fullsized":"2024-05-08T22:34:42.728361+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:03.426505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397816+0000","last_change":"2024-05-08T22:34:41.021433+0000","last_active":"2024-05-08T22:34:42.397816+0000","last_peered":"2024-05-08T22:34:42.397816+0000","last_clean":"2024-05-08T22:34:42.397816+0000","last_became_active":"2024-05-08T22:34:41.020501+0000","last_became_peered":"2024-05-08T22:34:41.020501+0000","last_unstale":"2024-05-08T22:34:42.397816+0000","last_undegraded":"2024-05-08T22:34:42.397816+0000","last_fullsized":"2024-05-08T22:34:42.397816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:51.388989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418498+0000","last_change":"2024-05-08T22:34:41.416192+0000","last_active":"2024-05-08T22:34:41.418498+0000","last_peered":"2024-05-08T22:34:41.418498+0000","last_clean":"2024-05-08T22:34:41.418498+0000","last_became_active":"2024-05-08T22:34:41.415912+0000","last_became_peered":"2024-05-08T22:34:41.415912+0000","last_unstale":"2024-05-08T22:34:41.418498+0000","last_undegraded":"2024-05-08T22:34:41.418498+0000","last_fullsized":"2024-05-08T22:34:41.418498+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:46.957082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.402885+0000","last_change":"2024-05-08T22:34:41.015442+0000","last_active":"2024-05-08T22:34:41.402885+0000","last_peered":"2024-05-08T22:34:41.402885+0000","last_clean":"2024-05-08T22:34:41.402885+0000","last_became_active":"2024-05-08T22:34:41.015204+0000","last_became_peered":"2024-05-08T22:34:41.015204+0000","last_unstale":"2024-05-08T22:34:41.402885+0000","last_undegraded":"2024-05-08T22:34:41.402885+0000","last_fullsized":"2024-05-08T22:34:41.402885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:19.574890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408647+0000","last_change":"2024-05-08T22:34:41.402521+0000","last_active":"2024-05-08T22:34:41.408647+0000","last_peered":"2024-05-08T22:34:41.408647+0000","last_clean":"2024-05-08T22:34:41.408647+0000","last_became_active":"2024-05-08T22:34:41.402292+0000","last_became_peered":"2024-05-08T22:34:41.402292+0000","last_unstale":"2024-05-08T22:34:41.408647+0000","last_undegraded":"2024-05-08T22:34:41.408647+0000","last_fullsized":"2024-05-08T22:34:41.408647+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.414825+0000","last_change":"2024-05-08T22:34:41.021481+0000","last_active":"2024-05-08T22:34:41.414825+0000","last_peered":"2024-05-08T22:34:41.414825+0000","last_clean":"2024-05-08T22:34:41.414825+0000","last_became_active":"2024-05-08T22:34:41.021198+0000","last_became_peered":"2024-05-08T22:34:41.021198+0000","last_unstale":"2024-05-08T22:34:41.414825+0000","last_undegraded":"2024-05-08T22:34:41.414825+0000","last_fullsized":"2024-05-08T22:34:41.414825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397683+0000","last_change":"2024-05-08T22:34:41.407921+0000","last_active":"2024-05-08T22:34:42.397683+0000","last_peered":"2024-05-08T22:34:42.397683+0000","last_clean":"2024-05-08T22:34:42.397683+0000","last_became_active":"2024-05-08T22:34:41.407688+0000","last_became_peered":"2024-05-08T22:34:41.407688+0000","last_unstale":"2024-05-08T22:34:42.397683+0000","last_undegraded":"2024-05-08T22:34:42.397683+0000","last_fullsized":"2024-05-08T22:34:42.397683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:35.979652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396787+0000","last_change":"2024-05-08T22:34:41.016284+0000","last_active":"2024-05-08T22:34:42.396787+0000","last_peered":"2024-05-08T22:34:42.396787+0000","last_clean":"2024-05-08T22:34:42.396787+0000","last_became_active":"2024-05-08T22:34:41.016012+0000","last_became_peered":"2024-05-08T22:34:41.016012+0000","last_unstale":"2024-05-08T22:34:42.396787+0000","last_undegraded":"2024-05-08T22:34:42.396787+0000","last_fullsized":"2024-05-08T22:34:42.396787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:43:50.549574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408847+0000","last_change":"2024-05-08T22:34:41.405522+0000","last_active":"2024-05-08T22:34:41.408847+0000","last_peered":"2024-05-08T22:34:41.408847+0000","last_clean":"2024-05-08T22:34:41.408847+0000","last_became_active":"2024-05-08T22:34:41.405206+0000","last_became_peered":"2024-05-08T22:34:41.405206+0000","last_unstale":"2024-05-08T22:34:41.408847+0000","last_undegraded":"2024-05-08T22:34:41.408847+0000","last_fullsized":"2024-05-08T22:34:41.408847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.971323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396886+0000","last_change":"2024-05-08T22:34:41.018266+0000","last_active":"2024-05-08T22:34:42.396886+0000","last_peered":"2024-05-08T22:34:42.396886+0000","last_clean":"2024-05-08T22:34:42.396886+0000","last_became_active":"2024-05-08T22:34:41.017999+0000","last_became_peered":"2024-05-08T22:34:41.017999+0000","last_unstale":"2024-05-08T22:34:42.396886+0000","last_undegraded":"2024-05-08T22:34:42.396886+0000","last_fullsized":"2024-05-08T22:34:42.396886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:31.646846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396413+0000","last_change":"2024-05-08T22:34:41.401627+0000","last_active":"2024-05-08T22:34:42.396413+0000","last_peered":"2024-05-08T22:34:42.396413+0000","last_clean":"2024-05-08T22:34:42.396413+0000","last_became_active":"2024-05-08T22:34:41.401391+0000","last_became_peered":"2024-05-08T22:34:41.401391+0000","last_unstale":"2024-05-08T22:34:42.396413+0000","last_undegraded":"2024-05-08T22:34:42.396413+0000","last_fullsized":"2024-05-08T22:34:42.396413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:36.607449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952079+0000","last_change":"2024-05-08T22:34:41.022402+0000","last_active":"2024-05-08T22:34:43.952079+0000","last_peered":"2024-05-08T22:34:43.952079+0000","last_clean":"2024-05-08T22:34:43.952079+0000","last_became_active":"2024-05-08T22:34:41.022063+0000","last_became_peered":"2024-05-08T22:34:41.022063+0000","last_unstale":"2024-05-08T22:34:43.952079+0000","last_undegraded":"2024-05-08T22:34:43.952079+0000","last_fullsized":"2024-05-08T22:34:43.952079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:30.881468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416650+0000","last_change":"2024-05-08T22:34:41.409995+0000","last_active":"2024-05-08T22:34:41.416650+0000","last_peered":"2024-05-08T22:34:41.416650+0000","last_clean":"2024-05-08T22:34:41.416650+0000","last_became_active":"2024-05-08T22:34:41.409496+0000","last_became_peered":"2024-05-08T22:34:41.409496+0000","last_unstale":"2024-05-08T22:34:41.416650+0000","last_undegraded":"2024-05-08T22:34:41.416650+0000","last_fullsized":"2024-05-08T22:34:41.416650+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396400+0000","last_change":"2024-05-08T22:34:41.014340+0000","last_active":"2024-05-08T22:34:42.396400+0000","last_peered":"2024-05-08T22:34:42.396400+0000","last_clean":"2024-05-08T22:34:42.396400+0000","last_became_active":"2024-05-08T22:34:41.013877+0000","last_became_peered":"2024-05-08T22:34:41.013877+0000","last_unstale":"2024-05-08T22:34:42.396400+0000","last_undegraded":"2024-05-08T22:34:42.396400+0000","last_fullsized":"2024-05-08T22:34:42.396400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:20.512002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396429+0000","last_change":"2024-05-08T22:34:41.401697+0000","last_active":"2024-05-08T22:34:42.396429+0000","last_peered":"2024-05-08T22:34:42.396429+0000","last_clean":"2024-05-08T22:34:42.396429+0000","last_became_active":"2024-05-08T22:34:41.401467+0000","last_became_peered":"2024-05-08T22:34:41.401467+0000","last_unstale":"2024-05-08T22:34:42.396429+0000","last_undegraded":"2024-05-08T22:34:42.396429+0000","last_fullsized":"2024-05-08T22:34:42.396429+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:28.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418860+0000","last_change":"2024-05-08T22:34:41.009539+0000","last_active":"2024-05-08T22:34:41.418860+0000","last_peered":"2024-05-08T22:34:41.418860+0000","last_clean":"2024-05-08T22:34:41.418860+0000","last_became_active":"2024-05-08T22:34:41.009266+0000","last_became_peered":"2024-05-08T22:34:41.009266+0000","last_unstale":"2024-05-08T22:34:41.418860+0000","last_undegraded":"2024-05-08T22:34:41.418860+0000","last_fullsized":"2024-05-08T22:34:41.418860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:32.591343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396718+0000","last_change":"2024-05-08T22:34:41.399621+0000","last_active":"2024-05-08T22:34:42.396718+0000","last_peered":"2024-05-08T22:34:42.396718+0000","last_clean":"2024-05-08T22:34:42.396718+0000","last_became_active":"2024-05-08T22:34:41.399337+0000","last_became_peered":"2024-05-08T22:34:41.399337+0000","last_unstale":"2024-05-08T22:34:42.396718+0000","last_undegraded":"2024-05-08T22:34:42.396718+0000","last_fullsized":"2024-05-08T22:34:42.396718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:25.421299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.415405+0000","last_change":"2024-05-08T22:34:41.014480+0000","last_active":"2024-05-08T22:34:41.415405+0000","last_peered":"2024-05-08T22:34:41.415405+0000","last_clean":"2024-05-08T22:34:41.415405+0000","last_became_active":"2024-05-08T22:34:41.014198+0000","last_became_peered":"2024-05-08T22:34:41.014198+0000","last_unstale":"2024-05-08T22:34:41.415405+0000","last_undegraded":"2024-05-08T22:34:41.415405+0000","last_fullsized":"2024-05-08T22:34:41.415405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:10.822604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416784+0000","last_change":"2024-05-08T22:34:41.413349+0000","last_active":"2024-05-08T22:34:41.416784+0000","last_peered":"2024-05-08T22:34:41.416784+0000","last_clean":"2024-05-08T22:34:41.416784+0000","last_became_active":"2024-05-08T22:34:41.413063+0000","last_became_peered":"2024-05-08T22:34:41.413063+0000","last_unstale":"2024-05-08T22:34:41.416784+0000","last_undegraded":"2024-05-08T22:34:41.416784+0000","last_fullsized":"2024-05-08T22:34:41.416784+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.777474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397340+0000","last_change":"2024-05-08T22:34:41.012694+0000","last_active":"2024-05-08T22:34:42.397340+0000","last_peered":"2024-05-08T22:34:42.397340+0000","last_clean":"2024-05-08T22:34:42.397340+0000","last_became_active":"2024-05-08T22:34:41.012391+0000","last_became_peered":"2024-05-08T22:34:41.012391+0000","last_unstale":"2024-05-08T22:34:42.397340+0000","last_undegraded":"2024-05-08T22:34:42.397340+0000","last_fullsized":"2024-05-08T22:34:42.397340+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:45.946304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396980+0000","last_change":"2024-05-08T22:34:41.412163+0000","last_active":"2024-05-08T22:34:42.396980+0000","last_peered":"2024-05-08T22:34:42.396980+0000","last_clean":"2024-05-08T22:34:42.396980+0000","last_became_active":"2024-05-08T22:34:41.411913+0000","last_became_peered":"2024-05-08T22:34:41.411913+0000","last_unstale":"2024-05-08T22:34:42.396980+0000","last_undegraded":"2024-05-08T22:34:42.396980+0000","last_fullsized":"2024-05-08T22:34:42.396980+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:02.623228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728503+0000","last_change":"2024-05-08T22:34:41.016094+0000","last_active":"2024-05-08T22:34:42.728503+0000","last_peered":"2024-05-08T22:34:42.728503+0000","last_clean":"2024-05-08T22:34:42.728503+0000","last_became_active":"2024-05-08T22:34:41.015707+0000","last_became_peered":"2024-05-08T22:34:41.015707+0000","last_unstale":"2024-05-08T22:34:42.728503+0000","last_undegraded":"2024-05-08T22:34:42.728503+0000","last_fullsized":"2024-05-08T22:34:42.728503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:47.410448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728650+0000","last_change":"2024-05-08T22:34:41.400771+0000","last_active":"2024-05-08T22:34:42.728650+0000","last_peered":"2024-05-08T22:34:42.728650+0000","last_clean":"2024-05-08T22:34:42.728650+0000","last_became_active":"2024-05-08T22:34:41.400419+0000","last_became_peered":"2024-05-08T22:34:41.400419+0000","last_unstale":"2024-05-08T22:34:42.728650+0000","last_undegraded":"2024-05-08T22:34:42.728650+0000","last_fullsized":"2024-05-08T22:34:42.728650+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:02.603090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952391+0000","last_change":"2024-05-08T22:34:41.017655+0000","last_active":"2024-05-08T22:34:43.952391+0000","last_peered":"2024-05-08T22:34:43.952391+0000","last_clean":"2024-05-08T22:34:43.952391+0000","last_became_active":"2024-05-08T22:34:41.017410+0000","last_became_peered":"2024-05-08T22:34:41.017410+0000","last_unstale":"2024-05-08T22:34:43.952391+0000","last_undegraded":"2024-05-08T22:34:43.952391+0000","last_fullsized":"2024-05-08T22:34:43.952391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:46.866206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418522+0000","last_change":"2024-05-08T22:34:41.412376+0000","last_active":"2024-05-08T22:34:41.418522+0000","last_peered":"2024-05-08T22:34:41.418522+0000","last_clean":"2024-05-08T22:34:41.418522+0000","last_became_active":"2024-05-08T22:34:41.412081+0000","last_became_peered":"2024-05-08T22:34:41.412081+0000","last_unstale":"2024-05-08T22:34:41.418522+0000","last_undegraded":"2024-05-08T22:34:41.418522+0000","last_fullsized":"2024-05-08T22:34:41.418522+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728177+0000","last_change":"2024-05-08T22:34:41.017795+0000","last_active":"2024-05-08T22:34:42.728177+0000","last_peered":"2024-05-08T22:34:42.728177+0000","last_clean":"2024-05-08T22:34:42.728177+0000","last_became_active":"2024-05-08T22:34:41.017571+0000","last_became_peered":"2024-05-08T22:34:41.017571+0000","last_unstale":"2024-05-08T22:34:42.728177+0000","last_undegraded":"2024-05-08T22:34:42.728177+0000","last_fullsized":"2024-05-08T22:34:42.728177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:19.496227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.403579+0000","last_change":"2024-05-08T22:34:41.401568+0000","last_active":"2024-05-08T22:34:41.403579+0000","last_peered":"2024-05-08T22:34:41.403579+0000","last_clean":"2024-05-08T22:34:41.403579+0000","last_became_active":"2024-05-08T22:34:41.401304+0000","last_became_peered":"2024-05-08T22:34:41.401304+0000","last_unstale":"2024-05-08T22:34:41.403579+0000","last_undegraded":"2024-05-08T22:34:41.403579+0000","last_fullsized":"2024-05-08T22:34:41.403579+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728103+0000","last_change":"2024-05-08T22:34:41.016620+0000","last_active":"2024-05-08T22:34:42.728103+0000","last_peered":"2024-05-08T22:34:42.728103+0000","last_clean":"2024-05-08T22:34:42.728103+0000","last_became_active":"2024-05-08T22:34:41.016302+0000","last_became_peered":"2024-05-08T22:34:41.016302+0000","last_unstale":"2024-05-08T22:34:42.728103+0000","last_undegraded":"2024-05-08T22:34:42.728103+0000","last_fullsized":"2024-05-08T22:34:42.728103+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:35.709885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408671+0000","last_change":"2024-05-08T22:34:41.405608+0000","last_active":"2024-05-08T22:34:41.408671+0000","last_peered":"2024-05-08T22:34:41.408671+0000","last_clean":"2024-05-08T22:34:41.408671+0000","last_became_active":"2024-05-08T22:34:41.405353+0000","last_became_peered":"2024-05-08T22:34:41.405353+0000","last_unstale":"2024-05-08T22:34:41.408671+0000","last_undegraded":"2024-05-08T22:34:41.408671+0000","last_fullsized":"2024-05-08T22:34:41.408671+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:19.779793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.396630+0000","last_change":"2024-05-08T22:34:41.013592+0000","last_active":"2024-05-08T22:34:41.396630+0000","last_peered":"2024-05-08T22:34:41.396630+0000","last_clean":"2024-05-08T22:34:41.396630+0000","last_became_active":"2024-05-08T22:34:41.013332+0000","last_became_peered":"2024-05-08T22:34:41.013332+0000","last_unstale":"2024-05-08T22:34:41.396630+0000","last_undegraded":"2024-05-08T22:34:41.396630+0000","last_fullsized":"2024-05-08T22:34:41.396630+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396646+0000","last_change":"2024-05-08T22:34:41.408259+0000","last_active":"2024-05-08T22:34:42.396646+0000","last_peered":"2024-05-08T22:34:42.396646+0000","last_clean":"2024-05-08T22:34:42.396646+0000","last_became_active":"2024-05-08T22:34:41.403558+0000","last_became_peered":"2024-05-08T22:34:41.403558+0000","last_unstale":"2024-05-08T22:34:42.396646+0000","last_undegraded":"2024-05-08T22:34:42.396646+0000","last_fullsized":"2024-05-08T22:34:42.396646+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:13.765495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.727935+0000","last_change":"2024-05-08T22:34:41.018830+0000","last_active":"2024-05-08T22:34:42.727935+0000","last_peered":"2024-05-08T22:34:42.727935+0000","last_clean":"2024-05-08T22:34:42.727935+0000","last_became_active":"2024-05-08T22:34:41.018555+0000","last_became_peered":"2024-05-08T22:34:41.018555+0000","last_unstale":"2024-05-08T22:34:42.727935+0000","last_undegraded":"2024-05-08T22:34:42.727935+0000","last_fullsized":"2024-05-08T22:34:42.727935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:10.145261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952146+0000","last_change":"2024-05-08T22:34:41.410872+0000","last_active":"2024-05-08T22:34:43.952146+0000","last_peered":"2024-05-08T22:34:43.952146+0000","last_clean":"2024-05-08T22:34:43.952146+0000","last_became_active":"2024-05-08T22:34:41.410641+0000","last_became_peered":"2024-05-08T22:34:41.410641+0000","last_unstale":"2024-05-08T22:34:43.952146+0000","last_undegraded":"2024-05-08T22:34:43.952146+0000","last_fullsized":"2024-05-08T22:34:43.952146+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:38.385130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.414090+0000","last_change":"2024-05-08T22:34:41.021277+0000","last_active":"2024-05-08T22:34:41.414090+0000","last_peered":"2024-05-08T22:34:41.414090+0000","last_clean":"2024-05-08T22:34:41.414090+0000","last_became_active":"2024-05-08T22:34:41.020944+0000","last_became_peered":"2024-05-08T22:34:41.020944+0000","last_unstale":"2024-05-08T22:34:41.414090+0000","last_undegraded":"2024-05-08T22:34:41.414090+0000","last_fullsized":"2024-05-08T22:34:41.414090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418514+0000","last_change":"2024-05-08T22:34:41.416858+0000","last_active":"2024-05-08T22:34:41.418514+0000","last_peered":"2024-05-08T22:34:41.418514+0000","last_clean":"2024-05-08T22:34:41.418514+0000","last_became_active":"2024-05-08T22:34:41.416479+0000","last_became_peered":"2024-05-08T22:34:41.416479+0000","last_unstale":"2024-05-08T22:34:41.418514+0000","last_undegraded":"2024-05-08T22:34:41.418514+0000","last_fullsized":"2024-05-08T22:34:41.418514+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:39.156787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728568+0000","last_change":"2024-05-08T22:34:41.015168+0000","last_active":"2024-05-08T22:34:42.728568+0000","last_peered":"2024-05-08T22:34:42.728568+0000","last_clean":"2024-05-08T22:34:42.728568+0000","last_became_active":"2024-05-08T22:34:41.012650+0000","last_became_peered":"2024-05-08T22:34:41.012650+0000","last_unstale":"2024-05-08T22:34:42.728568+0000","last_undegraded":"2024-05-08T22:34:42.728568+0000","last_fullsized":"2024-05-08T22:34:42.728568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:34.163447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397069+0000","last_change":"2024-05-08T22:34:41.400559+0000","last_active":"2024-05-08T22:34:42.397069+0000","last_peered":"2024-05-08T22:34:42.397069+0000","last_clean":"2024-05-08T22:34:42.397069+0000","last_became_active":"2024-05-08T22:34:41.399367+0000","last_became_peered":"2024-05-08T22:34:41.399367+0000","last_unstale":"2024-05-08T22:34:42.397069+0000","last_undegraded":"2024-05-08T22:34:42.397069+0000","last_fullsized":"2024-05-08T22:34:42.397069+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:56.067072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.397037+0000","last_change":"2024-05-08T22:34:41.023930+0000","last_active":"2024-05-08T22:34:41.397037+0000","last_peered":"2024-05-08T22:34:41.397037+0000","last_clean":"2024-05-08T22:34:41.397037+0000","last_became_active":"2024-05-08T22:34:41.023541+0000","last_became_peered":"2024-05-08T22:34:41.023541+0000","last_unstale":"2024-05-08T22:34:41.397037+0000","last_undegraded":"2024-05-08T22:34:41.397037+0000","last_fullsized":"2024-05-08T22:34:41.397037+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:15.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.403522+0000","last_change":"2024-05-08T22:34:41.401548+0000","last_active":"2024-05-08T22:34:41.403522+0000","last_peered":"2024-05-08T22:34:41.403522+0000","last_clean":"2024-05-08T22:34:41.403522+0000","last_became_active":"2024-05-08T22:34:41.401289+0000","last_became_peered":"2024-05-08T22:34:41.401289+0000","last_unstale":"2024-05-08T22:34:41.403522+0000","last_undegraded":"2024-05-08T22:34:41.403522+0000","last_fullsized":"2024-05-08T22:34:41.403522+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:03.828673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.393867+0000","last_change":"2024-05-08T22:34:41.017645+0000","last_active":"2024-05-08T22:34:41.393867+0000","last_peered":"2024-05-08T22:34:41.393867+0000","last_clean":"2024-05-08T22:34:41.393867+0000","last_became_active":"2024-05-08T22:34:41.017397+0000","last_became_peered":"2024-05-08T22:34:41.017397+0000","last_unstale":"2024-05-08T22:34:41.393867+0000","last_undegraded":"2024-05-08T22:34:41.393867+0000","last_fullsized":"2024-05-08T22:34:41.393867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952012+0000","last_change":"2024-05-08T22:34:41.411252+0000","last_active":"2024-05-08T22:34:43.952012+0000","last_peered":"2024-05-08T22:34:43.952012+0000","last_clean":"2024-05-08T22:34:43.952012+0000","last_became_active":"2024-05-08T22:34:41.411000+0000","last_became_peered":"2024-05-08T22:34:41.411000+0000","last_unstale":"2024-05-08T22:34:43.952012+0000","last_undegraded":"2024-05-08T22:34:43.952012+0000","last_fullsized":"2024-05-08T22:34:43.952012+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:12.975318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396998+0000","last_change":"2024-05-08T22:34:41.015548+0000","last_active":"2024-05-08T22:34:42.396998+0000","last_peered":"2024-05-08T22:34:42.396998+0000","last_clean":"2024-05-08T22:34:42.396998+0000","last_became_active":"2024-05-08T22:34:41.015286+0000","last_became_peered":"2024-05-08T22:34:41.015286+0000","last_unstale":"2024-05-08T22:34:42.396998+0000","last_undegraded":"2024-05-08T22:34:42.396998+0000","last_fullsized":"2024-05-08T22:34:42.396998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:20.163235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396150+0000","last_change":"2024-05-08T22:34:41.400313+0000","last_active":"2024-05-08T22:34:42.396150+0000","last_peered":"2024-05-08T22:34:42.396150+0000","last_clean":"2024-05-08T22:34:42.396150+0000","last_became_active":"2024-05-08T22:34:41.400026+0000","last_became_peered":"2024-05-08T22:34:41.400026+0000","last_unstale":"2024-05-08T22:34:42.396150+0000","last_undegraded":"2024-05-08T22:34:42.396150+0000","last_fullsized":"2024-05-08T22:34:42.396150+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:21.851459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.395820+0000","last_change":"2024-05-08T22:34:41.017145+0000","last_active":"2024-05-08T22:34:42.395820+0000","last_peered":"2024-05-08T22:34:42.395820+0000","last_clean":"2024-05-08T22:34:42.395820+0000","last_became_active":"2024-05-08T22:34:41.016866+0000","last_became_peered":"2024-05-08T22:34:41.016866+0000","last_unstale":"2024-05-08T22:34:42.395820+0000","last_undegraded":"2024-05-08T22:34:42.395820+0000","last_fullsized":"2024-05-08T22:34:42.395820+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:38.146009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416605+0000","last_change":"2024-05-08T22:34:41.411827+0000","last_active":"2024-05-08T22:34:41.416605+0000","last_peered":"2024-05-08T22:34:41.416605+0000","last_clean":"2024-05-08T22:34:41.416605+0000","last_became_active":"2024-05-08T22:34:41.410758+0000","last_became_peered":"2024-05-08T22:34:41.410758+0000","last_unstale":"2024-05-08T22:34:41.416605+0000","last_undegraded":"2024-05-08T22:34:41.416605+0000","last_fullsized":"2024-05-08T22:34:41.416605+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:32.559809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396070+0000","last_change":"2024-05-08T22:34:41.020741+0000","last_active":"2024-05-08T22:34:42.396070+0000","last_peered":"2024-05-08T22:34:42.396070+0000","last_clean":"2024-05-08T22:34:42.396070+0000","last_became_active":"2024-05-08T22:34:41.020423+0000","last_became_peered":"2024-05-08T22:34:41.020423+0000","last_unstale":"2024-05-08T22:34:42.396070+0000","last_undegraded":"2024-05-08T22:34:42.396070+0000","last_fullsized":"2024-05-08T22:34:42.396070+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:19.650448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416356+0000","last_change":"2024-05-08T22:34:41.406698+0000","last_active":"2024-05-08T22:34:41.416356+0000","last_peered":"2024-05-08T22:34:41.416356+0000","last_clean":"2024-05-08T22:34:41.416356+0000","last_became_active":"2024-05-08T22:34:41.405784+0000","last_became_peered":"2024-05-08T22:34:41.405784+0000","last_unstale":"2024-05-08T22:34:41.416356+0000","last_undegraded":"2024-05-08T22:34:41.416356+0000","last_fullsized":"2024-05-08T22:34:41.416356+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.397941+0000","last_change":"2024-05-08T22:34:41.011845+0000","last_active":"2024-05-08T22:34:41.397941+0000","last_peered":"2024-05-08T22:34:41.397941+0000","last_clean":"2024-05-08T22:34:41.397941+0000","last_became_active":"2024-05-08T22:34:41.011585+0000","last_became_peered":"2024-05-08T22:34:41.011585+0000","last_unstale":"2024-05-08T22:34:41.397941+0000","last_undegraded":"2024-05-08T22:34:41.397941+0000","last_fullsized":"2024-05-08T22:34:41.397941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-08T22:34:40.999648+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:34:40.999648+0000","last_peered":"2024-05-08T22:34:40.999648+0000","last_clean":"2024-05-08T22:34:40.999648+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:34:40.999648+0000","last_undegraded":"2024-05-08T22:34:40.999648+0000","last_fullsized":"2024-05-08T22:34:40.999648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":202963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.306},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.276},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.476}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.446},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.387}]}]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":201455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89600000000000002}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.98799999999999999}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.95599999999999996}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.111},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.03}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.008}]}]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1064960,"data_stored":786747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.069},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed May 8 22:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed May 8 22:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.01},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.127},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0980000000000001}]},{"osd":6,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.1719999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":471040,"data_stored":201947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.92300000000000004}]},{"osd":1,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.137}]},{"osd":2,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.83299999999999996}]},{"osd":3,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0740000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.026}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.79100000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":462848,"data_stored":194857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.03},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90700000000000003}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0029999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.96199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":462848,"data_stored":195349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.92800000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.335},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0169999999999999}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.3939999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":201435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.363},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.95999999999999996}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.876}]},{"osd":6,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.012},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.23}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.1890000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":785147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 22:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.036},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.94099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:34:45.463 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:34:45.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 Processor -- start 2024-05-08T22:34:45.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 -- start start 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c0c9230 0x7fa82c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa82c06af90 0x7fa82c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0c9bd0 con 0x7fa82c15d130 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ce650 con 0x7fa82c06af90 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa83187d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cd340 con 0x7fa82c0c9230 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82affd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa82c15d130 0x7fa82c0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39028/0 (socket says 172.21.15.179:39028) 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82affd700 1 -- 172.21.15.179:0/3363762679 learned_addr learned my addr 172.21.15.179:0/3363762679 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 --2- 172.21.15.179:0/3363762679 >> v2:172.21.15.195:3300/0 conn(0x7fa82c06af90 0x7fa82c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82a7fc700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c0c9230 0x7fa82c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754826431 0 0) 0x7fa82c0c9bd0 con 0x7fa82c15d130 2024-05-08T22:34:45.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8140035d0 con 0x7fa82c15d130 2024-05-08T22:34:45.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 -- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c0c9230 msgr2=0x7fa82c0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:45.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c0c9230 0x7fa82c0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078388862 0 0) 0x7fa8140035d0 con 0x7fa82c15d130 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 -- 172.21.15.179:0/3363762679 >> v1:172.21.15.179:6789/0 conn(0x7fa82c15d130 legacy=0x7fa82c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 -- 172.21.15.179:0/3363762679 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0cd620 con 0x7fa82c06af90 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa82b7fe700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c06b420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa824009850 tx=0x7fa824009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5061ca29c6b94731 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820001670 con 0x7fa82c06af90 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa824004030 con 0x7fa82c06af90 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.573+0000 7fa829ffb700 1 -- 172.21.15.179:0/3363762679 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240043e0 con 0x7fa82c06af90 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 msgr2=0x7fa82c06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c06b420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa824009850 tx=0x7fa824009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- 172.21.15.179:0/3363762679 shutdown_connections 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c06b420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 --2- 172.21.15.179:0/3363762679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c0c9230 0x7fa82c0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- 172.21.15.179:0/3363762679 >> 172.21.15.179:0/3363762679 conn(0x7fa82c0cb660 msgr2=0x7fa82c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- 172.21.15.179:0/3363762679 shutdown_connections 2024-05-08T22:34:45.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- 172.21.15.179:0/3363762679 wait complete. 2024-05-08T22:34:45.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 Processor -- start 2024-05-08T22:34:45.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.577+0000 7fa83187d700 1 -- start start 2024-05-08T22:34:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c15d130 0x7fa82c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c159f90 con 0x7fa82c0c9230 2024-05-08T22:34:45.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c158620 con 0x7fa82c06af90 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c159380 con 0x7fa82c15d130 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35308/0 (socket says 172.21.15.179:35308) 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 -- 172.21.15.179:0/1211579943 learned_addr learned my addr 172.21.15.179:0/1211579943 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82affd700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82b7fe700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c15d130 0x7fa82c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 -- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c15d130 msgr2=0x7fa82c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c15d130 0x7fa82c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 -- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 msgr2=0x7fa82c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 -- 172.21.15.179:0/1211579943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240094b0 con 0x7fa82c0c9230 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa82a7fc700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00efc0 tx=0x7fa81c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00cdc0 con 0x7fa82c0c9230 2024-05-08T22:34:45.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c00f040 con 0x7fa82c0c9230 2024-05-08T22:34:45.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0146c0 con 0x7fa82c0c9230 2024-05-08T22:34:45.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c159660 con 0x7fa82c0c9230 2024-05-08T22:34:45.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c159c20 con 0x7fa82c0c9230 2024-05-08T22:34:45.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c067770 con 0x7fa82c0c9230 2024-05-08T22:34:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.581+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa81c014820 con 0x7fa82c0c9230 2024-05-08T22:34:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.585+0000 7fa813fff700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.585+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa81c084f70 con 0x7fa82c0c9230 2024-05-08T22:34:45.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.585+0000 7fa82affd700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.585+0000 7fa82affd700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 0x7fa818068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa820001eb0 tx=0x7fa820001c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.593+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa81c0b6cf0 con 0x7fa82c0c9230 2024-05-08T22:34:45.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.761+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa82c15d510 con 0x7fa818066730 2024-05-08T22:34:45.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.781+0000 7fa813fff700 1 -- 172.21.15.179:0/1211579943 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287987 (secure 0 0 0) 0x7fa820005e10 con 0x7fa818066730 2024-05-08T22:34:45.788 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 msgr2=0x7fa818068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 0x7fa818068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa820001eb0 tx=0x7fa820001c40 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 msgr2=0x7fa82c1e9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00efc0 tx=0x7fa81c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 shutdown_connections 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa818066730 0x7fa818068bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa82c15d130 0x7fa82c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa82c0c9230 0x7fa82c1e9290 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 --2- 172.21.15.179:0/1211579943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa82c06af90 0x7fa82c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 >> 172.21.15.179:0/1211579943 conn(0x7fa82c0cb660 msgr2=0x7fa82c09f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 shutdown_connections 2024-05-08T22:34:45.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.789+0000 7fa83187d700 1 -- 172.21.15.179:0/1211579943 wait complete. 2024-05-08T22:34:45.792 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:34:45.804 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-05-08T22:34:45.409818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237412,"kb_used_data":4824,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737308,"statfs":{"total":773094113280,"available":772851003392,"internally_reserved":0,"allocated":4939776,"data_stored":2769900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.427707"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397715+0000","last_change":"2024-05-08T22:34:41.400635+0000","last_active":"2024-05-08T22:34:42.397715+0000","last_peered":"2024-05-08T22:34:42.397715+0000","last_clean":"2024-05-08T22:34:42.397715+0000","last_became_active":"2024-05-08T22:34:41.400383+0000","last_became_peered":"2024-05-08T22:34:41.400383+0000","last_unstale":"2024-05-08T22:34:42.397715+0000","last_undegraded":"2024-05-08T22:34:42.397715+0000","last_fullsized":"2024-05-08T22:34:42.397715+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:41:00.914039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397049+0000","last_change":"2024-05-08T22:34:41.406468+0000","last_active":"2024-05-08T22:34:42.397049+0000","last_peered":"2024-05-08T22:34:42.397049+0000","last_clean":"2024-05-08T22:34:42.397049+0000","last_became_active":"2024-05-08T22:34:41.406201+0000","last_became_peered":"2024-05-08T22:34:41.406201+0000","last_unstale":"2024-05-08T22:34:42.397049+0000","last_undegraded":"2024-05-08T22:34:42.397049+0000","last_fullsized":"2024-05-08T22:34:42.397049+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:35.110806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408715+0000","last_change":"2024-05-08T22:34:41.405938+0000","last_active":"2024-05-08T22:34:41.408715+0000","last_peered":"2024-05-08T22:34:41.408715+0000","last_clean":"2024-05-08T22:34:41.408715+0000","last_became_active":"2024-05-08T22:34:41.405687+0000","last_became_peered":"2024-05-08T22:34:41.405687+0000","last_unstale":"2024-05-08T22:34:41.408715+0000","last_undegraded":"2024-05-08T22:34:41.408715+0000","last_fullsized":"2024-05-08T22:34:41.408715+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728167+0000","last_change":"2024-05-08T22:34:41.403568+0000","last_active":"2024-05-08T22:34:42.728167+0000","last_peered":"2024-05-08T22:34:42.728167+0000","last_clean":"2024-05-08T22:34:42.728167+0000","last_became_active":"2024-05-08T22:34:41.403317+0000","last_became_peered":"2024-05-08T22:34:41.403317+0000","last_unstale":"2024-05-08T22:34:42.728167+0000","last_undegraded":"2024-05-08T22:34:42.728167+0000","last_fullsized":"2024-05-08T22:34:42.728167+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:20.585668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728278+0000","last_change":"2024-05-08T22:34:41.398864+0000","last_active":"2024-05-08T22:34:42.728278+0000","last_peered":"2024-05-08T22:34:42.728278+0000","last_clean":"2024-05-08T22:34:42.728278+0000","last_became_active":"2024-05-08T22:34:41.398644+0000","last_became_peered":"2024-05-08T22:34:41.398644+0000","last_unstale":"2024-05-08T22:34:42.728278+0000","last_undegraded":"2024-05-08T22:34:42.728278+0000","last_fullsized":"2024-05-08T22:34:42.728278+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:45.722225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951999+0000","last_change":"2024-05-08T22:34:41.408773+0000","last_active":"2024-05-08T22:34:43.951999+0000","last_peered":"2024-05-08T22:34:43.951999+0000","last_clean":"2024-05-08T22:34:43.951999+0000","last_became_active":"2024-05-08T22:34:41.408522+0000","last_became_peered":"2024-05-08T22:34:41.408522+0000","last_unstale":"2024-05-08T22:34:43.951999+0000","last_undegraded":"2024-05-08T22:34:43.951999+0000","last_fullsized":"2024-05-08T22:34:43.951999+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:43.790818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408731+0000","last_change":"2024-05-08T22:34:41.405010+0000","last_active":"2024-05-08T22:34:41.408731+0000","last_peered":"2024-05-08T22:34:41.408731+0000","last_clean":"2024-05-08T22:34:41.408731+0000","last_became_active":"2024-05-08T22:34:41.404745+0000","last_became_peered":"2024-05-08T22:34:41.404745+0000","last_unstale":"2024-05-08T22:34:41.408731+0000","last_undegraded":"2024-05-08T22:34:41.408731+0000","last_fullsized":"2024-05-08T22:34:41.408731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396050+0000","last_change":"2024-05-08T22:34:41.406886+0000","last_active":"2024-05-08T22:34:42.396050+0000","last_peered":"2024-05-08T22:34:42.396050+0000","last_clean":"2024-05-08T22:34:42.396050+0000","last_became_active":"2024-05-08T22:34:41.406193+0000","last_became_peered":"2024-05-08T22:34:41.406193+0000","last_unstale":"2024-05-08T22:34:42.396050+0000","last_undegraded":"2024-05-08T22:34:42.396050+0000","last_fullsized":"2024-05-08T22:34:42.396050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:41.552480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408589+0000","last_change":"2024-05-08T22:34:41.405675+0000","last_active":"2024-05-08T22:34:41.408589+0000","last_peered":"2024-05-08T22:34:41.408589+0000","last_clean":"2024-05-08T22:34:41.408589+0000","last_became_active":"2024-05-08T22:34:41.405172+0000","last_became_peered":"2024-05-08T22:34:41.405172+0000","last_unstale":"2024-05-08T22:34:41.408589+0000","last_undegraded":"2024-05-08T22:34:41.408589+0000","last_fullsized":"2024-05-08T22:34:41.408589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418451+0000","last_change":"2024-05-08T22:34:41.417348+0000","last_active":"2024-05-08T22:34:41.418451+0000","last_peered":"2024-05-08T22:34:41.418451+0000","last_clean":"2024-05-08T22:34:41.418451+0000","last_became_active":"2024-05-08T22:34:41.417050+0000","last_became_peered":"2024-05-08T22:34:41.417050+0000","last_unstale":"2024-05-08T22:34:41.418451+0000","last_undegraded":"2024-05-08T22:34:41.418451+0000","last_fullsized":"2024-05-08T22:34:41.418451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:32.559809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416462+0000","last_change":"2024-05-08T22:34:41.409198+0000","last_active":"2024-05-08T22:34:41.416462+0000","last_peered":"2024-05-08T22:34:41.416462+0000","last_clean":"2024-05-08T22:34:41.416462+0000","last_became_active":"2024-05-08T22:34:41.408932+0000","last_became_peered":"2024-05-08T22:34:41.408932+0000","last_unstale":"2024-05-08T22:34:41.416462+0000","last_undegraded":"2024-05-08T22:34:41.416462+0000","last_fullsized":"2024-05-08T22:34:41.416462+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408517+0000","last_change":"2024-05-08T22:34:41.404348+0000","last_active":"2024-05-08T22:34:41.408517+0000","last_peered":"2024-05-08T22:34:41.408517+0000","last_clean":"2024-05-08T22:34:41.408517+0000","last_became_active":"2024-05-08T22:34:41.404094+0000","last_became_peered":"2024-05-08T22:34:41.404094+0000","last_unstale":"2024-05-08T22:34:41.408517+0000","last_undegraded":"2024-05-08T22:34:41.408517+0000","last_fullsized":"2024-05-08T22:34:41.408517+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:41.792260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418932+0000","last_change":"2024-05-08T22:34:41.417278+0000","last_active":"2024-05-08T22:34:41.418932+0000","last_peered":"2024-05-08T22:34:41.418932+0000","last_clean":"2024-05-08T22:34:41.418932+0000","last_became_active":"2024-05-08T22:34:41.416991+0000","last_became_peered":"2024-05-08T22:34:41.416991+0000","last_unstale":"2024-05-08T22:34:41.418932+0000","last_undegraded":"2024-05-08T22:34:41.418932+0000","last_fullsized":"2024-05-08T22:34:41.418932+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:39.156787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396799+0000","last_change":"2024-05-08T22:34:41.396314+0000","last_active":"2024-05-08T22:34:42.396799+0000","last_peered":"2024-05-08T22:34:42.396799+0000","last_clean":"2024-05-08T22:34:42.396799+0000","last_became_active":"2024-05-08T22:34:41.396069+0000","last_became_peered":"2024-05-08T22:34:41.396069+0000","last_unstale":"2024-05-08T22:34:42.396799+0000","last_undegraded":"2024-05-08T22:34:42.396799+0000","last_fullsized":"2024-05-08T22:34:42.396799+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:09.894172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408749+0000","last_change":"2024-05-08T22:34:41.406739+0000","last_active":"2024-05-08T22:34:41.408749+0000","last_peered":"2024-05-08T22:34:41.408749+0000","last_clean":"2024-05-08T22:34:41.408749+0000","last_became_active":"2024-05-08T22:34:41.406463+0000","last_became_peered":"2024-05-08T22:34:41.406463+0000","last_unstale":"2024-05-08T22:34:41.408749+0000","last_undegraded":"2024-05-08T22:34:41.408749+0000","last_fullsized":"2024-05-08T22:34:41.408749+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418502+0000","last_change":"2024-05-08T22:34:41.412305+0000","last_active":"2024-05-08T22:34:41.418502+0000","last_peered":"2024-05-08T22:34:41.418502+0000","last_clean":"2024-05-08T22:34:41.418502+0000","last_became_active":"2024-05-08T22:34:41.410310+0000","last_became_peered":"2024-05-08T22:34:41.410310+0000","last_unstale":"2024-05-08T22:34:41.418502+0000","last_undegraded":"2024-05-08T22:34:41.418502+0000","last_fullsized":"2024-05-08T22:34:41.418502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:06.200268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416481+0000","last_change":"2024-05-08T22:34:41.409690+0000","last_active":"2024-05-08T22:34:41.416481+0000","last_peered":"2024-05-08T22:34:41.416481+0000","last_clean":"2024-05-08T22:34:41.416481+0000","last_became_active":"2024-05-08T22:34:41.409428+0000","last_became_peered":"2024-05-08T22:34:41.409428+0000","last_unstale":"2024-05-08T22:34:41.416481+0000","last_undegraded":"2024-05-08T22:34:41.416481+0000","last_fullsized":"2024-05-08T22:34:41.416481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:19.779793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397118+0000","last_change":"2024-05-08T22:34:41.401758+0000","last_active":"2024-05-08T22:34:42.397118+0000","last_peered":"2024-05-08T22:34:42.397118+0000","last_clean":"2024-05-08T22:34:42.397118+0000","last_became_active":"2024-05-08T22:34:41.398599+0000","last_became_peered":"2024-05-08T22:34:41.398599+0000","last_unstale":"2024-05-08T22:34:42.397118+0000","last_undegraded":"2024-05-08T22:34:42.397118+0000","last_fullsized":"2024-05-08T22:34:42.397118+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:32.657624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728446+0000","last_change":"2024-05-08T22:34:41.415542+0000","last_active":"2024-05-08T22:34:42.728446+0000","last_peered":"2024-05-08T22:34:42.728446+0000","last_clean":"2024-05-08T22:34:42.728446+0000","last_became_active":"2024-05-08T22:34:41.415261+0000","last_became_peered":"2024-05-08T22:34:41.415261+0000","last_unstale":"2024-05-08T22:34:42.728446+0000","last_undegraded":"2024-05-08T22:34:42.728446+0000","last_fullsized":"2024-05-08T22:34:42.728446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:11.833382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396826+0000","last_change":"2024-05-08T22:34:41.398134+0000","last_active":"2024-05-08T22:34:42.396826+0000","last_peered":"2024-05-08T22:34:42.396826+0000","last_clean":"2024-05-08T22:34:42.396826+0000","last_became_active":"2024-05-08T22:34:41.397829+0000","last_became_peered":"2024-05-08T22:34:41.397829+0000","last_unstale":"2024-05-08T22:34:42.396826+0000","last_undegraded":"2024-05-08T22:34:42.396826+0000","last_fullsized":"2024-05-08T22:34:42.396826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:26.913962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416777+0000","last_change":"2024-05-08T22:34:41.413309+0000","last_active":"2024-05-08T22:34:41.416777+0000","last_peered":"2024-05-08T22:34:41.416777+0000","last_clean":"2024-05-08T22:34:41.416777+0000","last_became_active":"2024-05-08T22:34:41.413035+0000","last_became_peered":"2024-05-08T22:34:41.413035+0000","last_unstale":"2024-05-08T22:34:41.416777+0000","last_undegraded":"2024-05-08T22:34:41.416777+0000","last_fullsized":"2024-05-08T22:34:41.416777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:46.957082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396325+0000","last_change":"2024-05-08T22:34:41.407385+0000","last_active":"2024-05-08T22:34:42.396325+0000","last_peered":"2024-05-08T22:34:42.396325+0000","last_clean":"2024-05-08T22:34:42.396325+0000","last_became_active":"2024-05-08T22:34:41.407088+0000","last_became_peered":"2024-05-08T22:34:41.407088+0000","last_unstale":"2024-05-08T22:34:42.396325+0000","last_undegraded":"2024-05-08T22:34:42.396325+0000","last_fullsized":"2024-05-08T22:34:42.396325+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:32.109467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416612+0000","last_change":"2024-05-08T22:34:41.410175+0000","last_active":"2024-05-08T22:34:41.416612+0000","last_peered":"2024-05-08T22:34:41.416612+0000","last_clean":"2024-05-08T22:34:41.416612+0000","last_became_active":"2024-05-08T22:34:41.409635+0000","last_became_peered":"2024-05-08T22:34:41.409635+0000","last_unstale":"2024-05-08T22:34:41.416612+0000","last_undegraded":"2024-05-08T22:34:41.416612+0000","last_fullsized":"2024-05-08T22:34:41.416612+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.727869+0000","last_change":"2024-05-08T22:34:41.400291+0000","last_active":"2024-05-08T22:34:42.727869+0000","last_peered":"2024-05-08T22:34:42.727869+0000","last_clean":"2024-05-08T22:34:42.727869+0000","last_became_active":"2024-05-08T22:34:41.400051+0000","last_became_peered":"2024-05-08T22:34:41.400051+0000","last_unstale":"2024-05-08T22:34:42.727869+0000","last_undegraded":"2024-05-08T22:34:42.727869+0000","last_fullsized":"2024-05-08T22:34:42.727869+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:31.084389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397124+0000","last_change":"2024-05-08T22:34:41.398686+0000","last_active":"2024-05-08T22:34:42.397124+0000","last_peered":"2024-05-08T22:34:42.397124+0000","last_clean":"2024-05-08T22:34:42.397124+0000","last_became_active":"2024-05-08T22:34:41.398407+0000","last_became_peered":"2024-05-08T22:34:41.398407+0000","last_unstale":"2024-05-08T22:34:42.397124+0000","last_undegraded":"2024-05-08T22:34:42.397124+0000","last_fullsized":"2024-05-08T22:34:42.397124+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:20.661226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952453+0000","last_change":"2024-05-08T22:34:41.408211+0000","last_active":"2024-05-08T22:34:43.952453+0000","last_peered":"2024-05-08T22:34:43.952453+0000","last_clean":"2024-05-08T22:34:43.952453+0000","last_became_active":"2024-05-08T22:34:41.407935+0000","last_became_peered":"2024-05-08T22:34:41.407935+0000","last_unstale":"2024-05-08T22:34:43.952453+0000","last_undegraded":"2024-05-08T22:34:43.952453+0000","last_fullsized":"2024-05-08T22:34:43.952453+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:03.265817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396906+0000","last_change":"2024-05-08T22:34:41.396020+0000","last_active":"2024-05-08T22:34:42.396906+0000","last_peered":"2024-05-08T22:34:42.396906+0000","last_clean":"2024-05-08T22:34:42.396906+0000","last_became_active":"2024-05-08T22:34:41.395704+0000","last_became_peered":"2024-05-08T22:34:41.395704+0000","last_unstale":"2024-05-08T22:34:42.396906+0000","last_undegraded":"2024-05-08T22:34:42.396906+0000","last_fullsized":"2024-05-08T22:34:42.396906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:43:51.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416840+0000","last_change":"2024-05-08T22:34:41.413564+0000","last_active":"2024-05-08T22:34:41.416840+0000","last_peered":"2024-05-08T22:34:41.416840+0000","last_clean":"2024-05-08T22:34:41.416840+0000","last_became_active":"2024-05-08T22:34:41.413319+0000","last_became_peered":"2024-05-08T22:34:41.413319+0000","last_unstale":"2024-05-08T22:34:41.416840+0000","last_undegraded":"2024-05-08T22:34:41.416840+0000","last_fullsized":"2024-05-08T22:34:41.416840+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397225+0000","last_change":"2024-05-08T22:34:41.411078+0000","last_active":"2024-05-08T22:34:42.397225+0000","last_peered":"2024-05-08T22:34:42.397225+0000","last_clean":"2024-05-08T22:34:42.397225+0000","last_became_active":"2024-05-08T22:34:41.410804+0000","last_became_peered":"2024-05-08T22:34:41.410804+0000","last_unstale":"2024-05-08T22:34:42.397225+0000","last_undegraded":"2024-05-08T22:34:42.397225+0000","last_fullsized":"2024-05-08T22:34:42.397225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:52.399767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396889+0000","last_change":"2024-05-08T22:34:41.402575+0000","last_active":"2024-05-08T22:34:42.396889+0000","last_peered":"2024-05-08T22:34:42.396889+0000","last_clean":"2024-05-08T22:34:42.396889+0000","last_became_active":"2024-05-08T22:34:41.402246+0000","last_became_peered":"2024-05-08T22:34:41.402246+0000","last_unstale":"2024-05-08T22:34:42.396889+0000","last_undegraded":"2024-05-08T22:34:42.396889+0000","last_fullsized":"2024-05-08T22:34:42.396889+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:35.232767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416634+0000","last_change":"2024-05-08T22:34:41.410214+0000","last_active":"2024-05-08T22:34:41.416634+0000","last_peered":"2024-05-08T22:34:41.416634+0000","last_clean":"2024-05-08T22:34:41.416634+0000","last_became_active":"2024-05-08T22:34:41.409950+0000","last_became_peered":"2024-05-08T22:34:41.409950+0000","last_unstale":"2024-05-08T22:34:41.416634+0000","last_undegraded":"2024-05-08T22:34:41.416634+0000","last_fullsized":"2024-05-08T22:34:41.416634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:03.410146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951884+0000","last_change":"2024-05-08T22:34:41.410973+0000","last_active":"2024-05-08T22:34:43.951884+0000","last_peered":"2024-05-08T22:34:43.951884+0000","last_clean":"2024-05-08T22:34:43.951884+0000","last_became_active":"2024-05-08T22:34:41.410730+0000","last_became_peered":"2024-05-08T22:34:41.410730+0000","last_unstale":"2024-05-08T22:34:43.951884+0000","last_undegraded":"2024-05-08T22:34:43.951884+0000","last_fullsized":"2024-05-08T22:34:43.951884+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:57.440176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416635+0000","last_change":"2024-05-08T22:34:41.410066+0000","last_active":"2024-05-08T22:34:41.416635+0000","last_peered":"2024-05-08T22:34:41.416635+0000","last_clean":"2024-05-08T22:34:41.416635+0000","last_became_active":"2024-05-08T22:34:41.409778+0000","last_became_peered":"2024-05-08T22:34:41.409778+0000","last_unstale":"2024-05-08T22:34:41.416635+0000","last_undegraded":"2024-05-08T22:34:41.416635+0000","last_fullsized":"2024-05-08T22:34:41.416635+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397232+0000","last_change":"2024-05-08T22:34:41.014993+0000","last_active":"2024-05-08T22:34:42.397232+0000","last_peered":"2024-05-08T22:34:42.397232+0000","last_clean":"2024-05-08T22:34:42.397232+0000","last_became_active":"2024-05-08T22:34:41.014759+0000","last_became_peered":"2024-05-08T22:34:41.014759+0000","last_unstale":"2024-05-08T22:34:42.397232+0000","last_undegraded":"2024-05-08T22:34:42.397232+0000","last_fullsized":"2024-05-08T22:34:42.397232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:30.375135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952309+0000","last_change":"2024-05-08T22:34:41.409031+0000","last_active":"2024-05-08T22:34:43.952309+0000","last_peered":"2024-05-08T22:34:43.952309+0000","last_clean":"2024-05-08T22:34:43.952309+0000","last_became_active":"2024-05-08T22:34:41.408750+0000","last_became_peered":"2024-05-08T22:34:41.408750+0000","last_unstale":"2024-05-08T22:34:43.952309+0000","last_undegraded":"2024-05-08T22:34:43.952309+0000","last_fullsized":"2024-05-08T22:34:43.952309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:09.088402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418082+0000","last_change":"2024-05-08T22:34:41.017178+0000","last_active":"2024-05-08T22:34:41.418082+0000","last_peered":"2024-05-08T22:34:41.418082+0000","last_clean":"2024-05-08T22:34:41.418082+0000","last_became_active":"2024-05-08T22:34:41.016873+0000","last_became_peered":"2024-05-08T22:34:41.016873+0000","last_unstale":"2024-05-08T22:34:41.418082+0000","last_undegraded":"2024-05-08T22:34:41.418082+0000","last_fullsized":"2024-05-08T22:34:41.418082+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397220+0000","last_change":"2024-05-08T22:34:41.396198+0000","last_active":"2024-05-08T22:34:42.397220+0000","last_peered":"2024-05-08T22:34:42.397220+0000","last_clean":"2024-05-08T22:34:42.397220+0000","last_became_active":"2024-05-08T22:34:41.395930+0000","last_became_peered":"2024-05-08T22:34:41.395930+0000","last_unstale":"2024-05-08T22:34:42.397220+0000","last_undegraded":"2024-05-08T22:34:42.397220+0000","last_fullsized":"2024-05-08T22:34:42.397220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:48.608036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952428+0000","last_change":"2024-05-08T22:34:41.022801+0000","last_active":"2024-05-08T22:34:43.952428+0000","last_peered":"2024-05-08T22:34:43.952428+0000","last_clean":"2024-05-08T22:34:43.952428+0000","last_became_active":"2024-05-08T22:34:41.022466+0000","last_became_peered":"2024-05-08T22:34:41.022466+0000","last_unstale":"2024-05-08T22:34:43.952428+0000","last_undegraded":"2024-05-08T22:34:43.952428+0000","last_fullsized":"2024-05-08T22:34:43.952428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:39.592583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397682+0000","last_change":"2024-05-08T22:34:41.400001+0000","last_active":"2024-05-08T22:34:42.397682+0000","last_peered":"2024-05-08T22:34:42.397682+0000","last_clean":"2024-05-08T22:34:42.397682+0000","last_became_active":"2024-05-08T22:34:41.399744+0000","last_became_peered":"2024-05-08T22:34:41.399744+0000","last_unstale":"2024-05-08T22:34:42.397682+0000","last_undegraded":"2024-05-08T22:34:42.397682+0000","last_fullsized":"2024-05-08T22:34:42.397682+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:15.302436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396423+0000","last_change":"2024-05-08T22:34:41.017194+0000","last_active":"2024-05-08T22:34:42.396423+0000","last_peered":"2024-05-08T22:34:42.396423+0000","last_clean":"2024-05-08T22:34:42.396423+0000","last_became_active":"2024-05-08T22:34:41.016905+0000","last_became_peered":"2024-05-08T22:34:41.016905+0000","last_unstale":"2024-05-08T22:34:42.396423+0000","last_undegraded":"2024-05-08T22:34:42.396423+0000","last_fullsized":"2024-05-08T22:34:42.396423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:05.189490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.413614+0000","last_change":"2024-05-08T22:34:41.411887+0000","last_active":"2024-05-08T22:34:41.413614+0000","last_peered":"2024-05-08T22:34:41.413614+0000","last_clean":"2024-05-08T22:34:41.413614+0000","last_became_active":"2024-05-08T22:34:41.411598+0000","last_became_peered":"2024-05-08T22:34:41.411598+0000","last_unstale":"2024-05-08T22:34:41.413614+0000","last_undegraded":"2024-05-08T22:34:41.413614+0000","last_fullsized":"2024-05-08T22:34:41.413614+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:25.411706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.409372+0000","last_change":"2024-05-08T22:34:41.016887+0000","last_active":"2024-05-08T22:34:41.409372+0000","last_peered":"2024-05-08T22:34:41.409372+0000","last_clean":"2024-05-08T22:34:41.409372+0000","last_became_active":"2024-05-08T22:34:41.016652+0000","last_became_peered":"2024-05-08T22:34:41.016652+0000","last_unstale":"2024-05-08T22:34:41.409372+0000","last_undegraded":"2024-05-08T22:34:41.409372+0000","last_fullsized":"2024-05-08T22:34:41.409372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:30.073611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.417060+0000","last_change":"2024-05-08T22:34:41.414428+0000","last_active":"2024-05-08T22:34:41.417060+0000","last_peered":"2024-05-08T22:34:41.417060+0000","last_clean":"2024-05-08T22:34:41.417060+0000","last_became_active":"2024-05-08T22:34:41.414049+0000","last_became_peered":"2024-05-08T22:34:41.414049+0000","last_unstale":"2024-05-08T22:34:41.417060+0000","last_undegraded":"2024-05-08T22:34:41.417060+0000","last_fullsized":"2024-05-08T22:34:41.417060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:06.200268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416557+0000","last_change":"2024-05-08T22:34:41.013385+0000","last_active":"2024-05-08T22:34:41.416557+0000","last_peered":"2024-05-08T22:34:41.416557+0000","last_clean":"2024-05-08T22:34:41.416557+0000","last_became_active":"2024-05-08T22:34:41.013124+0000","last_became_peered":"2024-05-08T22:34:41.013124+0000","last_unstale":"2024-05-08T22:34:41.416557+0000","last_undegraded":"2024-05-08T22:34:41.416557+0000","last_fullsized":"2024-05-08T22:34:41.416557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:15.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396493+0000","last_change":"2024-05-08T22:34:41.410713+0000","last_active":"2024-05-08T22:34:42.396493+0000","last_peered":"2024-05-08T22:34:42.396493+0000","last_clean":"2024-05-08T22:34:42.396493+0000","last_became_active":"2024-05-08T22:34:41.410400+0000","last_became_peered":"2024-05-08T22:34:41.410400+0000","last_unstale":"2024-05-08T22:34:42.396493+0000","last_undegraded":"2024-05-08T22:34:42.396493+0000","last_fullsized":"2024-05-08T22:34:42.396493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:58.033758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.951894+0000","last_change":"2024-05-08T22:34:41.022120+0000","last_active":"2024-05-08T22:34:43.951894+0000","last_peered":"2024-05-08T22:34:43.951894+0000","last_clean":"2024-05-08T22:34:43.951894+0000","last_became_active":"2024-05-08T22:34:41.021821+0000","last_became_peered":"2024-05-08T22:34:41.021821+0000","last_unstale":"2024-05-08T22:34:43.951894+0000","last_undegraded":"2024-05-08T22:34:43.951894+0000","last_fullsized":"2024-05-08T22:34:43.951894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:54.382606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396164+0000","last_change":"2024-05-08T22:34:41.407696+0000","last_active":"2024-05-08T22:34:42.396164+0000","last_peered":"2024-05-08T22:34:42.396164+0000","last_clean":"2024-05-08T22:34:42.396164+0000","last_became_active":"2024-05-08T22:34:41.403758+0000","last_became_peered":"2024-05-08T22:34:41.403758+0000","last_unstale":"2024-05-08T22:34:42.396164+0000","last_undegraded":"2024-05-08T22:34:42.396164+0000","last_fullsized":"2024-05-08T22:34:42.396164+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:40.662497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.395887+0000","last_change":"2024-05-08T22:34:41.017190+0000","last_active":"2024-05-08T22:34:42.395887+0000","last_peered":"2024-05-08T22:34:42.395887+0000","last_clean":"2024-05-08T22:34:42.395887+0000","last_became_active":"2024-05-08T22:34:41.016911+0000","last_became_peered":"2024-05-08T22:34:41.016911+0000","last_unstale":"2024-05-08T22:34:42.395887+0000","last_undegraded":"2024-05-08T22:34:42.395887+0000","last_fullsized":"2024-05-08T22:34:42.395887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:31.549031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728361+0000","last_change":"2024-05-08T22:34:41.398502+0000","last_active":"2024-05-08T22:34:42.728361+0000","last_peered":"2024-05-08T22:34:42.728361+0000","last_clean":"2024-05-08T22:34:42.728361+0000","last_became_active":"2024-05-08T22:34:41.398206+0000","last_became_peered":"2024-05-08T22:34:41.398206+0000","last_unstale":"2024-05-08T22:34:42.728361+0000","last_undegraded":"2024-05-08T22:34:42.728361+0000","last_fullsized":"2024-05-08T22:34:42.728361+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:03.426505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397816+0000","last_change":"2024-05-08T22:34:41.021433+0000","last_active":"2024-05-08T22:34:42.397816+0000","last_peered":"2024-05-08T22:34:42.397816+0000","last_clean":"2024-05-08T22:34:42.397816+0000","last_became_active":"2024-05-08T22:34:41.020501+0000","last_became_peered":"2024-05-08T22:34:41.020501+0000","last_unstale":"2024-05-08T22:34:42.397816+0000","last_undegraded":"2024-05-08T22:34:42.397816+0000","last_fullsized":"2024-05-08T22:34:42.397816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:51.388989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418498+0000","last_change":"2024-05-08T22:34:41.416192+0000","last_active":"2024-05-08T22:34:41.418498+0000","last_peered":"2024-05-08T22:34:41.418498+0000","last_clean":"2024-05-08T22:34:41.418498+0000","last_became_active":"2024-05-08T22:34:41.415912+0000","last_became_peered":"2024-05-08T22:34:41.415912+0000","last_unstale":"2024-05-08T22:34:41.418498+0000","last_undegraded":"2024-05-08T22:34:41.418498+0000","last_fullsized":"2024-05-08T22:34:41.418498+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:46.957082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.402885+0000","last_change":"2024-05-08T22:34:41.015442+0000","last_active":"2024-05-08T22:34:41.402885+0000","last_peered":"2024-05-08T22:34:41.402885+0000","last_clean":"2024-05-08T22:34:41.402885+0000","last_became_active":"2024-05-08T22:34:41.015204+0000","last_became_peered":"2024-05-08T22:34:41.015204+0000","last_unstale":"2024-05-08T22:34:41.402885+0000","last_undegraded":"2024-05-08T22:34:41.402885+0000","last_fullsized":"2024-05-08T22:34:41.402885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:19.574890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408647+0000","last_change":"2024-05-08T22:34:41.402521+0000","last_active":"2024-05-08T22:34:41.408647+0000","last_peered":"2024-05-08T22:34:41.408647+0000","last_clean":"2024-05-08T22:34:41.408647+0000","last_became_active":"2024-05-08T22:34:41.402292+0000","last_became_peered":"2024-05-08T22:34:41.402292+0000","last_unstale":"2024-05-08T22:34:41.408647+0000","last_undegraded":"2024-05-08T22:34:41.408647+0000","last_fullsized":"2024-05-08T22:34:41.408647+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.414825+0000","last_change":"2024-05-08T22:34:41.021481+0000","last_active":"2024-05-08T22:34:41.414825+0000","last_peered":"2024-05-08T22:34:41.414825+0000","last_clean":"2024-05-08T22:34:41.414825+0000","last_became_active":"2024-05-08T22:34:41.021198+0000","last_became_peered":"2024-05-08T22:34:41.021198+0000","last_unstale":"2024-05-08T22:34:41.414825+0000","last_undegraded":"2024-05-08T22:34:41.414825+0000","last_fullsized":"2024-05-08T22:34:41.414825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397683+0000","last_change":"2024-05-08T22:34:41.407921+0000","last_active":"2024-05-08T22:34:42.397683+0000","last_peered":"2024-05-08T22:34:42.397683+0000","last_clean":"2024-05-08T22:34:42.397683+0000","last_became_active":"2024-05-08T22:34:41.407688+0000","last_became_peered":"2024-05-08T22:34:41.407688+0000","last_unstale":"2024-05-08T22:34:42.397683+0000","last_undegraded":"2024-05-08T22:34:42.397683+0000","last_fullsized":"2024-05-08T22:34:42.397683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:35.979652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396787+0000","last_change":"2024-05-08T22:34:41.016284+0000","last_active":"2024-05-08T22:34:42.396787+0000","last_peered":"2024-05-08T22:34:42.396787+0000","last_clean":"2024-05-08T22:34:42.396787+0000","last_became_active":"2024-05-08T22:34:41.016012+0000","last_became_peered":"2024-05-08T22:34:41.016012+0000","last_unstale":"2024-05-08T22:34:42.396787+0000","last_undegraded":"2024-05-08T22:34:42.396787+0000","last_fullsized":"2024-05-08T22:34:42.396787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:43:50.549574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408847+0000","last_change":"2024-05-08T22:34:41.405522+0000","last_active":"2024-05-08T22:34:41.408847+0000","last_peered":"2024-05-08T22:34:41.408847+0000","last_clean":"2024-05-08T22:34:41.408847+0000","last_became_active":"2024-05-08T22:34:41.405206+0000","last_became_peered":"2024-05-08T22:34:41.405206+0000","last_unstale":"2024-05-08T22:34:41.408847+0000","last_undegraded":"2024-05-08T22:34:41.408847+0000","last_fullsized":"2024-05-08T22:34:41.408847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:55.971323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396886+0000","last_change":"2024-05-08T22:34:41.018266+0000","last_active":"2024-05-08T22:34:42.396886+0000","last_peered":"2024-05-08T22:34:42.396886+0000","last_clean":"2024-05-08T22:34:42.396886+0000","last_became_active":"2024-05-08T22:34:41.017999+0000","last_became_peered":"2024-05-08T22:34:41.017999+0000","last_unstale":"2024-05-08T22:34:42.396886+0000","last_undegraded":"2024-05-08T22:34:42.396886+0000","last_fullsized":"2024-05-08T22:34:42.396886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:31.646846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396413+0000","last_change":"2024-05-08T22:34:41.401627+0000","last_active":"2024-05-08T22:34:42.396413+0000","last_peered":"2024-05-08T22:34:42.396413+0000","last_clean":"2024-05-08T22:34:42.396413+0000","last_became_active":"2024-05-08T22:34:41.401391+0000","last_became_peered":"2024-05-08T22:34:41.401391+0000","last_unstale":"2024-05-08T22:34:42.396413+0000","last_undegraded":"2024-05-08T22:34:42.396413+0000","last_fullsized":"2024-05-08T22:34:42.396413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:36.607449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952079+0000","last_change":"2024-05-08T22:34:41.022402+0000","last_active":"2024-05-08T22:34:43.952079+0000","last_peered":"2024-05-08T22:34:43.952079+0000","last_clean":"2024-05-08T22:34:43.952079+0000","last_became_active":"2024-05-08T22:34:41.022063+0000","last_became_peered":"2024-05-08T22:34:41.022063+0000","last_unstale":"2024-05-08T22:34:43.952079+0000","last_undegraded":"2024-05-08T22:34:43.952079+0000","last_fullsized":"2024-05-08T22:34:43.952079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:30.881468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416650+0000","last_change":"2024-05-08T22:34:41.409995+0000","last_active":"2024-05-08T22:34:41.416650+0000","last_peered":"2024-05-08T22:34:41.416650+0000","last_clean":"2024-05-08T22:34:41.416650+0000","last_became_active":"2024-05-08T22:34:41.409496+0000","last_became_peered":"2024-05-08T22:34:41.409496+0000","last_unstale":"2024-05-08T22:34:41.416650+0000","last_undegraded":"2024-05-08T22:34:41.416650+0000","last_fullsized":"2024-05-08T22:34:41.416650+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:37.545136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396400+0000","last_change":"2024-05-08T22:34:41.014340+0000","last_active":"2024-05-08T22:34:42.396400+0000","last_peered":"2024-05-08T22:34:42.396400+0000","last_clean":"2024-05-08T22:34:42.396400+0000","last_became_active":"2024-05-08T22:34:41.013877+0000","last_became_peered":"2024-05-08T22:34:41.013877+0000","last_unstale":"2024-05-08T22:34:42.396400+0000","last_undegraded":"2024-05-08T22:34:42.396400+0000","last_fullsized":"2024-05-08T22:34:42.396400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:20.512002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396429+0000","last_change":"2024-05-08T22:34:41.401697+0000","last_active":"2024-05-08T22:34:42.396429+0000","last_peered":"2024-05-08T22:34:42.396429+0000","last_clean":"2024-05-08T22:34:42.396429+0000","last_became_active":"2024-05-08T22:34:41.401467+0000","last_became_peered":"2024-05-08T22:34:41.401467+0000","last_unstale":"2024-05-08T22:34:42.396429+0000","last_undegraded":"2024-05-08T22:34:42.396429+0000","last_fullsized":"2024-05-08T22:34:42.396429+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:28.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418860+0000","last_change":"2024-05-08T22:34:41.009539+0000","last_active":"2024-05-08T22:34:41.418860+0000","last_peered":"2024-05-08T22:34:41.418860+0000","last_clean":"2024-05-08T22:34:41.418860+0000","last_became_active":"2024-05-08T22:34:41.009266+0000","last_became_peered":"2024-05-08T22:34:41.009266+0000","last_unstale":"2024-05-08T22:34:41.418860+0000","last_undegraded":"2024-05-08T22:34:41.418860+0000","last_fullsized":"2024-05-08T22:34:41.418860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:32.591343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396718+0000","last_change":"2024-05-08T22:34:41.399621+0000","last_active":"2024-05-08T22:34:42.396718+0000","last_peered":"2024-05-08T22:34:42.396718+0000","last_clean":"2024-05-08T22:34:42.396718+0000","last_became_active":"2024-05-08T22:34:41.399337+0000","last_became_peered":"2024-05-08T22:34:41.399337+0000","last_unstale":"2024-05-08T22:34:42.396718+0000","last_undegraded":"2024-05-08T22:34:42.396718+0000","last_fullsized":"2024-05-08T22:34:42.396718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:25.421299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.415405+0000","last_change":"2024-05-08T22:34:41.014480+0000","last_active":"2024-05-08T22:34:41.415405+0000","last_peered":"2024-05-08T22:34:41.415405+0000","last_clean":"2024-05-08T22:34:41.415405+0000","last_became_active":"2024-05-08T22:34:41.014198+0000","last_became_peered":"2024-05-08T22:34:41.014198+0000","last_unstale":"2024-05-08T22:34:41.415405+0000","last_undegraded":"2024-05-08T22:34:41.415405+0000","last_fullsized":"2024-05-08T22:34:41.415405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:10.822604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416784+0000","last_change":"2024-05-08T22:34:41.413349+0000","last_active":"2024-05-08T22:34:41.416784+0000","last_peered":"2024-05-08T22:34:41.416784+0000","last_clean":"2024-05-08T22:34:41.416784+0000","last_became_active":"2024-05-08T22:34:41.413063+0000","last_became_peered":"2024-05-08T22:34:41.413063+0000","last_unstale":"2024-05-08T22:34:41.416784+0000","last_undegraded":"2024-05-08T22:34:41.416784+0000","last_fullsized":"2024-05-08T22:34:41.416784+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.777474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397340+0000","last_change":"2024-05-08T22:34:41.012694+0000","last_active":"2024-05-08T22:34:42.397340+0000","last_peered":"2024-05-08T22:34:42.397340+0000","last_clean":"2024-05-08T22:34:42.397340+0000","last_became_active":"2024-05-08T22:34:41.012391+0000","last_became_peered":"2024-05-08T22:34:41.012391+0000","last_unstale":"2024-05-08T22:34:42.397340+0000","last_undegraded":"2024-05-08T22:34:42.397340+0000","last_fullsized":"2024-05-08T22:34:42.397340+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:45.946304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396980+0000","last_change":"2024-05-08T22:34:41.412163+0000","last_active":"2024-05-08T22:34:42.396980+0000","last_peered":"2024-05-08T22:34:42.396980+0000","last_clean":"2024-05-08T22:34:42.396980+0000","last_became_active":"2024-05-08T22:34:41.411913+0000","last_became_peered":"2024-05-08T22:34:41.411913+0000","last_unstale":"2024-05-08T22:34:42.396980+0000","last_undegraded":"2024-05-08T22:34:42.396980+0000","last_fullsized":"2024-05-08T22:34:42.396980+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:02.623228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728503+0000","last_change":"2024-05-08T22:34:41.016094+0000","last_active":"2024-05-08T22:34:42.728503+0000","last_peered":"2024-05-08T22:34:42.728503+0000","last_clean":"2024-05-08T22:34:42.728503+0000","last_became_active":"2024-05-08T22:34:41.015707+0000","last_became_peered":"2024-05-08T22:34:41.015707+0000","last_unstale":"2024-05-08T22:34:42.728503+0000","last_undegraded":"2024-05-08T22:34:42.728503+0000","last_fullsized":"2024-05-08T22:34:42.728503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:47.410448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728650+0000","last_change":"2024-05-08T22:34:41.400771+0000","last_active":"2024-05-08T22:34:42.728650+0000","last_peered":"2024-05-08T22:34:42.728650+0000","last_clean":"2024-05-08T22:34:42.728650+0000","last_became_active":"2024-05-08T22:34:41.400419+0000","last_became_peered":"2024-05-08T22:34:41.400419+0000","last_unstale":"2024-05-08T22:34:42.728650+0000","last_undegraded":"2024-05-08T22:34:42.728650+0000","last_fullsized":"2024-05-08T22:34:42.728650+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:02.603090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952391+0000","last_change":"2024-05-08T22:34:41.017655+0000","last_active":"2024-05-08T22:34:43.952391+0000","last_peered":"2024-05-08T22:34:43.952391+0000","last_clean":"2024-05-08T22:34:43.952391+0000","last_became_active":"2024-05-08T22:34:41.017410+0000","last_became_peered":"2024-05-08T22:34:41.017410+0000","last_unstale":"2024-05-08T22:34:43.952391+0000","last_undegraded":"2024-05-08T22:34:43.952391+0000","last_fullsized":"2024-05-08T22:34:43.952391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:46.866206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418522+0000","last_change":"2024-05-08T22:34:41.412376+0000","last_active":"2024-05-08T22:34:41.418522+0000","last_peered":"2024-05-08T22:34:41.418522+0000","last_clean":"2024-05-08T22:34:41.418522+0000","last_became_active":"2024-05-08T22:34:41.412081+0000","last_became_peered":"2024-05-08T22:34:41.412081+0000","last_unstale":"2024-05-08T22:34:41.418522+0000","last_undegraded":"2024-05-08T22:34:41.418522+0000","last_fullsized":"2024-05-08T22:34:41.418522+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728177+0000","last_change":"2024-05-08T22:34:41.017795+0000","last_active":"2024-05-08T22:34:42.728177+0000","last_peered":"2024-05-08T22:34:42.728177+0000","last_clean":"2024-05-08T22:34:42.728177+0000","last_became_active":"2024-05-08T22:34:41.017571+0000","last_became_peered":"2024-05-08T22:34:41.017571+0000","last_unstale":"2024-05-08T22:34:42.728177+0000","last_undegraded":"2024-05-08T22:34:42.728177+0000","last_fullsized":"2024-05-08T22:34:42.728177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:19.496227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.403579+0000","last_change":"2024-05-08T22:34:41.401568+0000","last_active":"2024-05-08T22:34:41.403579+0000","last_peered":"2024-05-08T22:34:41.403579+0000","last_clean":"2024-05-08T22:34:41.403579+0000","last_became_active":"2024-05-08T22:34:41.401304+0000","last_became_peered":"2024-05-08T22:34:41.401304+0000","last_unstale":"2024-05-08T22:34:41.403579+0000","last_undegraded":"2024-05-08T22:34:41.403579+0000","last_fullsized":"2024-05-08T22:34:41.403579+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:11.965935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728103+0000","last_change":"2024-05-08T22:34:41.016620+0000","last_active":"2024-05-08T22:34:42.728103+0000","last_peered":"2024-05-08T22:34:42.728103+0000","last_clean":"2024-05-08T22:34:42.728103+0000","last_became_active":"2024-05-08T22:34:41.016302+0000","last_became_peered":"2024-05-08T22:34:41.016302+0000","last_unstale":"2024-05-08T22:34:42.728103+0000","last_undegraded":"2024-05-08T22:34:42.728103+0000","last_fullsized":"2024-05-08T22:34:42.728103+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:35.709885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.408671+0000","last_change":"2024-05-08T22:34:41.405608+0000","last_active":"2024-05-08T22:34:41.408671+0000","last_peered":"2024-05-08T22:34:41.408671+0000","last_clean":"2024-05-08T22:34:41.408671+0000","last_became_active":"2024-05-08T22:34:41.405353+0000","last_became_peered":"2024-05-08T22:34:41.405353+0000","last_unstale":"2024-05-08T22:34:41.408671+0000","last_undegraded":"2024-05-08T22:34:41.408671+0000","last_fullsized":"2024-05-08T22:34:41.408671+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:19.779793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.396630+0000","last_change":"2024-05-08T22:34:41.013592+0000","last_active":"2024-05-08T22:34:41.396630+0000","last_peered":"2024-05-08T22:34:41.396630+0000","last_clean":"2024-05-08T22:34:41.396630+0000","last_became_active":"2024-05-08T22:34:41.013332+0000","last_became_peered":"2024-05-08T22:34:41.013332+0000","last_unstale":"2024-05-08T22:34:41.396630+0000","last_undegraded":"2024-05-08T22:34:41.396630+0000","last_fullsized":"2024-05-08T22:34:41.396630+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:42.950762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396646+0000","last_change":"2024-05-08T22:34:41.408259+0000","last_active":"2024-05-08T22:34:42.396646+0000","last_peered":"2024-05-08T22:34:42.396646+0000","last_clean":"2024-05-08T22:34:42.396646+0000","last_became_active":"2024-05-08T22:34:41.403558+0000","last_became_peered":"2024-05-08T22:34:41.403558+0000","last_unstale":"2024-05-08T22:34:42.396646+0000","last_undegraded":"2024-05-08T22:34:42.396646+0000","last_fullsized":"2024-05-08T22:34:42.396646+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:13.765495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.727935+0000","last_change":"2024-05-08T22:34:41.018830+0000","last_active":"2024-05-08T22:34:42.727935+0000","last_peered":"2024-05-08T22:34:42.727935+0000","last_clean":"2024-05-08T22:34:42.727935+0000","last_became_active":"2024-05-08T22:34:41.018555+0000","last_became_peered":"2024-05-08T22:34:41.018555+0000","last_unstale":"2024-05-08T22:34:42.727935+0000","last_undegraded":"2024-05-08T22:34:42.727935+0000","last_fullsized":"2024-05-08T22:34:42.727935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:10.145261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952146+0000","last_change":"2024-05-08T22:34:41.410872+0000","last_active":"2024-05-08T22:34:43.952146+0000","last_peered":"2024-05-08T22:34:43.952146+0000","last_clean":"2024-05-08T22:34:43.952146+0000","last_became_active":"2024-05-08T22:34:41.410641+0000","last_became_peered":"2024-05-08T22:34:41.410641+0000","last_unstale":"2024-05-08T22:34:43.952146+0000","last_undegraded":"2024-05-08T22:34:43.952146+0000","last_fullsized":"2024-05-08T22:34:43.952146+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:38.385130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.414090+0000","last_change":"2024-05-08T22:34:41.021277+0000","last_active":"2024-05-08T22:34:41.414090+0000","last_peered":"2024-05-08T22:34:41.414090+0000","last_clean":"2024-05-08T22:34:41.414090+0000","last_became_active":"2024-05-08T22:34:41.020944+0000","last_became_peered":"2024-05-08T22:34:41.020944+0000","last_unstale":"2024-05-08T22:34:41.414090+0000","last_undegraded":"2024-05-08T22:34:41.414090+0000","last_fullsized":"2024-05-08T22:34:41.414090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.418514+0000","last_change":"2024-05-08T22:34:41.416858+0000","last_active":"2024-05-08T22:34:41.418514+0000","last_peered":"2024-05-08T22:34:41.418514+0000","last_clean":"2024-05-08T22:34:41.418514+0000","last_became_active":"2024-05-08T22:34:41.416479+0000","last_became_peered":"2024-05-08T22:34:41.416479+0000","last_unstale":"2024-05-08T22:34:41.418514+0000","last_undegraded":"2024-05-08T22:34:41.418514+0000","last_fullsized":"2024-05-08T22:34:41.418514+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:39.156787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.728568+0000","last_change":"2024-05-08T22:34:41.015168+0000","last_active":"2024-05-08T22:34:42.728568+0000","last_peered":"2024-05-08T22:34:42.728568+0000","last_clean":"2024-05-08T22:34:42.728568+0000","last_became_active":"2024-05-08T22:34:41.012650+0000","last_became_peered":"2024-05-08T22:34:41.012650+0000","last_unstale":"2024-05-08T22:34:42.728568+0000","last_undegraded":"2024-05-08T22:34:42.728568+0000","last_fullsized":"2024-05-08T22:34:42.728568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:34.163447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.397069+0000","last_change":"2024-05-08T22:34:41.400559+0000","last_active":"2024-05-08T22:34:42.397069+0000","last_peered":"2024-05-08T22:34:42.397069+0000","last_clean":"2024-05-08T22:34:42.397069+0000","last_became_active":"2024-05-08T22:34:41.399367+0000","last_became_peered":"2024-05-08T22:34:41.399367+0000","last_unstale":"2024-05-08T22:34:42.397069+0000","last_undegraded":"2024-05-08T22:34:42.397069+0000","last_fullsized":"2024-05-08T22:34:42.397069+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:56.067072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.397037+0000","last_change":"2024-05-08T22:34:41.023930+0000","last_active":"2024-05-08T22:34:41.397037+0000","last_peered":"2024-05-08T22:34:41.397037+0000","last_clean":"2024-05-08T22:34:41.397037+0000","last_became_active":"2024-05-08T22:34:41.023541+0000","last_became_peered":"2024-05-08T22:34:41.023541+0000","last_unstale":"2024-05-08T22:34:41.397037+0000","last_undegraded":"2024-05-08T22:34:41.397037+0000","last_fullsized":"2024-05-08T22:34:41.397037+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:15.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.403522+0000","last_change":"2024-05-08T22:34:41.401548+0000","last_active":"2024-05-08T22:34:41.403522+0000","last_peered":"2024-05-08T22:34:41.403522+0000","last_clean":"2024-05-08T22:34:41.403522+0000","last_became_active":"2024-05-08T22:34:41.401289+0000","last_became_peered":"2024-05-08T22:34:41.401289+0000","last_unstale":"2024-05-08T22:34:41.403522+0000","last_undegraded":"2024-05-08T22:34:41.403522+0000","last_fullsized":"2024-05-08T22:34:41.403522+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:03.828673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.393867+0000","last_change":"2024-05-08T22:34:41.017645+0000","last_active":"2024-05-08T22:34:41.393867+0000","last_peered":"2024-05-08T22:34:41.393867+0000","last_clean":"2024-05-08T22:34:41.393867+0000","last_became_active":"2024-05-08T22:34:41.017397+0000","last_became_peered":"2024-05-08T22:34:41.017397+0000","last_unstale":"2024-05-08T22:34:41.393867+0000","last_undegraded":"2024-05-08T22:34:41.393867+0000","last_fullsized":"2024-05-08T22:34:41.393867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:43.952012+0000","last_change":"2024-05-08T22:34:41.411252+0000","last_active":"2024-05-08T22:34:43.952012+0000","last_peered":"2024-05-08T22:34:43.952012+0000","last_clean":"2024-05-08T22:34:43.952012+0000","last_became_active":"2024-05-08T22:34:41.411000+0000","last_became_peered":"2024-05-08T22:34:41.411000+0000","last_unstale":"2024-05-08T22:34:43.952012+0000","last_undegraded":"2024-05-08T22:34:43.952012+0000","last_fullsized":"2024-05-08T22:34:43.952012+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:12.975318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396998+0000","last_change":"2024-05-08T22:34:41.015548+0000","last_active":"2024-05-08T22:34:42.396998+0000","last_peered":"2024-05-08T22:34:42.396998+0000","last_clean":"2024-05-08T22:34:42.396998+0000","last_became_active":"2024-05-08T22:34:41.015286+0000","last_became_peered":"2024-05-08T22:34:41.015286+0000","last_unstale":"2024-05-08T22:34:42.396998+0000","last_undegraded":"2024-05-08T22:34:42.396998+0000","last_fullsized":"2024-05-08T22:34:42.396998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:20.163235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396150+0000","last_change":"2024-05-08T22:34:41.400313+0000","last_active":"2024-05-08T22:34:42.396150+0000","last_peered":"2024-05-08T22:34:42.396150+0000","last_clean":"2024-05-08T22:34:42.396150+0000","last_became_active":"2024-05-08T22:34:41.400026+0000","last_became_peered":"2024-05-08T22:34:41.400026+0000","last_unstale":"2024-05-08T22:34:42.396150+0000","last_undegraded":"2024-05-08T22:34:42.396150+0000","last_fullsized":"2024-05-08T22:34:42.396150+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:21.851459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.395820+0000","last_change":"2024-05-08T22:34:41.017145+0000","last_active":"2024-05-08T22:34:42.395820+0000","last_peered":"2024-05-08T22:34:42.395820+0000","last_clean":"2024-05-08T22:34:42.395820+0000","last_became_active":"2024-05-08T22:34:41.016866+0000","last_became_peered":"2024-05-08T22:34:41.016866+0000","last_unstale":"2024-05-08T22:34:42.395820+0000","last_undegraded":"2024-05-08T22:34:42.395820+0000","last_fullsized":"2024-05-08T22:34:42.395820+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:38.146009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416605+0000","last_change":"2024-05-08T22:34:41.411827+0000","last_active":"2024-05-08T22:34:41.416605+0000","last_peered":"2024-05-08T22:34:41.416605+0000","last_clean":"2024-05-08T22:34:41.416605+0000","last_became_active":"2024-05-08T22:34:41.410758+0000","last_became_peered":"2024-05-08T22:34:41.410758+0000","last_unstale":"2024-05-08T22:34:41.416605+0000","last_undegraded":"2024-05-08T22:34:41.416605+0000","last_fullsized":"2024-05-08T22:34:41.416605+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:32.559809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T22:34:42.396070+0000","last_change":"2024-05-08T22:34:41.020741+0000","last_active":"2024-05-08T22:34:42.396070+0000","last_peered":"2024-05-08T22:34:42.396070+0000","last_clean":"2024-05-08T22:34:42.396070+0000","last_became_active":"2024-05-08T22:34:41.020423+0000","last_became_peered":"2024-05-08T22:34:41.020423+0000","last_unstale":"2024-05-08T22:34:42.396070+0000","last_undegraded":"2024-05-08T22:34:42.396070+0000","last_fullsized":"2024-05-08T22:34:42.396070+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:30:19.650448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.416356+0000","last_change":"2024-05-08T22:34:41.406698+0000","last_active":"2024-05-08T22:34:41.416356+0000","last_peered":"2024-05-08T22:34:41.416356+0000","last_clean":"2024-05-08T22:34:41.416356+0000","last_became_active":"2024-05-08T22:34:41.405784+0000","last_became_peered":"2024-05-08T22:34:41.405784+0000","last_unstale":"2024-05-08T22:34:41.416356+0000","last_undegraded":"2024-05-08T22:34:41.416356+0000","last_fullsized":"2024-05-08T22:34:41.416356+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:40.982384+0000","last_clean_scrub_stamp":"2024-05-08T22:34:40.982384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:24.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T22:34:41.397941+0000","last_change":"2024-05-08T22:34:41.011845+0000","last_active":"2024-05-08T22:34:41.397941+0000","last_peered":"2024-05-08T22:34:41.397941+0000","last_clean":"2024-05-08T22:34:41.397941+0000","last_became_active":"2024-05-08T22:34:41.011585+0000","last_became_peered":"2024-05-08T22:34:41.011585+0000","last_unstale":"2024-05-08T22:34:41.397941+0000","last_undegraded":"2024-05-08T22:34:41.397941+0000","last_fullsized":"2024-05-08T22:34:41.397941+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:34:39.971606+0000","last_clean_scrub_stamp":"2024-05-08T22:34:39.971606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:20.007700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-08T22:34:40.999648+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:34:40.999648+0000","last_peered":"2024-05-08T22:34:40.999648+0000","last_clean":"2024-05-08T22:34:40.999648+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:34:40.999648+0000","last_undegraded":"2024-05-08T22:34:40.999648+0000","last_fullsized":"2024-05-08T22:34:40.999648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":202963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.306},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.276},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.476}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.446},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.387}]}]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":201455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89600000000000002}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.98799999999999999}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.95599999999999996}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.111},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.03}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.008}]}]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1064960,"data_stored":786747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.069},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed May 8 22:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed May 8 22:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.01},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.127},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0980000000000001}]},{"osd":6,"last update":"Wed May 8 22:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.1719999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":471040,"data_stored":201947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.92300000000000004}]},{"osd":1,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.137}]},{"osd":2,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.83299999999999996}]},{"osd":3,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0740000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.026}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.79100000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":462848,"data_stored":194857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.03},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90700000000000003}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0029999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.96199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":462848,"data_stored":195349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.92800000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.335},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0169999999999999}]},{"osd":7,"last update":"Wed May 8 22:34:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.3939999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":201435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.363},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.95999999999999996}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.876}]},{"osd":6,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.012},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.23}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.1890000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":785147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 22:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Wed May 8 22:34:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.036},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.94099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:34:45.809 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:34:45.809 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:34:45.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.973+0000 7fb6e3fff700 1 Processor -- start 2024-05-08T22:34:45.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.973+0000 7fb6e3fff700 1 -- start start 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 0x7fb6e4167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40b8ee0 con 0x7fb6e415d0c0 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e415d4a0 con 0x7fb6e40b8a80 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e406abb0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e2ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb6e415d0c0 0x7fb6e40d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39042/0 (socket says 172.21.15.179:39042) 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e2ffd700 1 -- 172.21.15.179:0/4250058232 learned_addr learned my addr 172.21.15.179:0/4250058232 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:45.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e37fe700 1 --2- 172.21.15.179:0/4250058232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 0x7fb6e4167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772330330 0 0) 0x7fb6e40b8ee0 con 0x7fb6e415d0c0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d00035d0 con 0x7fb6e415d0c0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624654032 0 0) 0x7fb6e415d4a0 con 0x7fb6e40b8a80 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e40b8ee0 con 0x7fb6e40b8a80 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e37fe700 1 -- 172.21.15.179:0/4250058232 >> v1:172.21.15.195:6789/0 conn(0x7fb6e40b8a80 legacy=0x7fb6e40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e37fe700 1 -- 172.21.15.179:0/4250058232 >> v1:172.21.15.179:6789/0 conn(0x7fb6e415d0c0 legacy=0x7fb6e40d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e37fe700 1 -- 172.21.15.179:0/4250058232 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e40d2500 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e37fe700 1 --2- 172.21.15.179:0/4250058232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 0x7fb6e4167ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009880 tx=0x7fb6dc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552cc788890de4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc011830 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc011990 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e1ffb700 1 -- 172.21.15.179:0/4250058232 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc011d20 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- 172.21.15.179:0/4250058232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 msgr2=0x7fb6e4167ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/4250058232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 0x7fb6e4167ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009880 tx=0x7fb6dc009b90 comp rx=0 tx=0).stop 2024-05-08T22:34:45.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- 172.21.15.179:0/4250058232 shutdown_connections 2024-05-08T22:34:45.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/4250058232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e40ce2b0 0x7fb6e4167ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- 172.21.15.179:0/4250058232 >> 172.21.15.179:0/4250058232 conn(0x7fb6e40d2ed0 msgr2=0x7fb6e40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:45.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- 172.21.15.179:0/4250058232 shutdown_connections 2024-05-08T22:34:45.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.977+0000 7fb6e3fff700 1 -- 172.21.15.179:0/4250058232 wait complete. 2024-05-08T22:34:45.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 Processor -- start 2024-05-08T22:34:45.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 -- start start 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 0x7fb6e40c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e415d0c0 0x7fb6e41f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c84e0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c6f00 con 0x7fb6e40b8a80 2024-05-08T22:34:45.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c71e0 con 0x7fb6e415d0c0 2024-05-08T22:34:45.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35320/0 (socket says 172.21.15.179:35320) 2024-05-08T22:34:45.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e27fc700 1 -- 172.21.15.179:0/3111222830 learned_addr learned my addr 172.21.15.179:0/3111222830 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:45.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e2ffd700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 0x7fb6e40c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.981+0000 7fb6e37fe700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e415d0c0 0x7fb6e41f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 -- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e415d0c0 msgr2=0x7fb6e41f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e415d0c0 0x7fb6e41f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 -- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 msgr2=0x7fb6e40c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:45.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 0x7fb6e40c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 -- 172.21.15.179:0/3111222830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0094b0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e2ffd700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 0x7fb6e40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e27fc700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00c730 tx=0x7fb6cc00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0052d0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc005450 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0038e0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e40c74c0 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e40c7a80 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.985+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e404e700 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.989+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6cc003a40 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.989+0000 7fb6db7fe700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 0x7fb6c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:45.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.989+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb6cc083b60 con 0x7fb6e40ce2b0 2024-05-08T22:34:45.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.989+0000 7fb6e2ffd700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 0x7fb6c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:45.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.989+0000 7fb6e2ffd700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 0x7fb6c4068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40012e0 tx=0x7fb6d4001a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:45.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:45.997+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6cc050d90 con 0x7fb6e40ce2b0 2024-05-08T22:34:46.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.229+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb6e40676e0 con 0x7fb6e40ce2b0 2024-05-08T22:34:46.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.237+0000 7fb6db7fe700 1 -- 172.21.15.179:0/3111222830 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb6cc0170e0 con 0x7fb6e40ce2b0 2024-05-08T22:34:46.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 msgr2=0x7fb6c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 0x7fb6c4068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40012e0 tx=0x7fb6d4001a30 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 msgr2=0x7fb6e40ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00c730 tx=0x7fb6cc00ca40 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 shutdown_connections 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6c4066730 0x7fb6c4068bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6e415d0c0 0x7fb6e41f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6e40ce2b0 0x7fb6e40ca800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 --2- 172.21.15.179:0/3111222830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6e40b8a80 0x7fb6e40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 >> 172.21.15.179:0/3111222830 conn(0x7fb6e40d2ed0 msgr2=0x7fb6e406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 shutdown_connections 2024-05-08T22:34:46.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.241+0000 7fb6e3fff700 1 -- 172.21.15.179:0/3111222830 wait complete. 2024-05-08T22:34:46.245 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:34:46.258 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:46.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 Processor -- start 2024-05-08T22:34:46.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 -- start start 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0d200ce380 0x7f0d200cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0d200cb510 0x7f0d2006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2006abb0 con 0x7f0d200cc700 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d200cb980 con 0x7f0d200cb510 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d27084700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2006ae90 con 0x7f0d200ce380 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d26883700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0d200cb510 0x7f0d2006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d26082700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0d200cc700 0x7f0d2015b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39056/0 (socket says 172.21.15.179:39056) 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d26082700 1 -- 172.21.15.179:0/1951555180 learned_addr learned my addr 172.21.15.179:0/1951555180 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:46.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.369+0000 7f0d25881700 1 --2- 172.21.15.179:0/1951555180 >> v2:172.21.15.195:3300/0 conn(0x7f0d200ce380 0x7f0d200cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d26883700 1 -- 172.21.15.179:0/1951555180 >> v2:172.21.15.195:3300/0 conn(0x7f0d200ce380 msgr2=0x7f0d200cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d26883700 1 --2- 172.21.15.179:0/1951555180 >> v2:172.21.15.195:3300/0 conn(0x7f0d200ce380 0x7f0d200cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d26883700 1 -- 172.21.15.179:0/1951555180 >> v1:172.21.15.179:6789/0 conn(0x7f0d200cc700 legacy=0x7f0d2015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d25080700 1 -- 172.21.15.179:0/1951555180 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236516680 0 0) 0x7f0d2006abb0 con 0x7f0d200cc700 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d26883700 1 -- 172.21.15.179:0/1951555180 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200ce800 con 0x7f0d200cb510 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d26883700 1 --2- 172.21.15.179:0/1951555180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200cb510 0x7f0d2006b320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0d200b3e50 tx=0x7f0d18009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77429a370e2c7b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d25080700 1 -- 172.21.15.179:0/1951555180 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011830 con 0x7f0d200cb510 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d25080700 1 -- 172.21.15.179:0/1951555180 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18011990 con 0x7f0d200cb510 2024-05-08T22:34:46.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d25080700 1 -- 172.21.15.179:0/1951555180 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011d20 con 0x7f0d200cb510 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 -- 172.21.15.179:0/1951555180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200cb510 msgr2=0x7f0d2006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 --2- 172.21.15.179:0/1951555180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200cb510 0x7f0d2006b320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0d200b3e50 tx=0x7f0d18009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 -- 172.21.15.179:0/1951555180 shutdown_connections 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 --2- 172.21.15.179:0/1951555180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200cb510 0x7f0d2006b320 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 --2- 172.21.15.179:0/1951555180 >> v2:172.21.15.195:3300/0 conn(0x7f0d200ce380 0x7f0d200cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 -- 172.21.15.179:0/1951555180 >> 172.21.15.179:0/1951555180 conn(0x7f0d200ce9d0 msgr2=0x7f0d200cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 -- 172.21.15.179:0/1951555180 shutdown_connections 2024-05-08T22:34:46.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.373+0000 7f0d27084700 1 -- 172.21.15.179:0/1951555180 wait complete. 2024-05-08T22:34:46.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 Processor -- start 2024-05-08T22:34:46.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 -- start start 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d200cb510 0x7f0d2014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d200cc700 0x7f0d2014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2014e260 con 0x7f0d200ce380 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201f4ee0 con 0x7f0d200cc700 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201f4360 con 0x7f0d200cb510 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35330/0 (socket says 172.21.15.179:35330) 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 -- 172.21.15.179:0/2215460391 learned_addr learned my addr 172.21.15.179:0/2215460391 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26082700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d200cb510 0x7f0d2014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d25881700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d200cc700 0x7f0d2014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 -- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d200cb510 msgr2=0x7f0d2014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d200cb510 0x7f0d2014dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 -- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d200cc700 msgr2=0x7f0d2014f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d200cc700 0x7f0d2014f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 -- 172.21.15.179:0/2215460391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180094b0 con 0x7f0d200ce380 2024-05-08T22:34:46.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d26883700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0d200b3e80 tx=0x7f0d18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:46.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004380 con 0x7f0d200ce380 2024-05-08T22:34:46.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1801f070 con 0x7f0d200ce380 2024-05-08T22:34:46.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180036c0 con 0x7f0d200ce380 2024-05-08T22:34:46.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.377+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d201f4640 con 0x7f0d200ce380 2024-05-08T22:34:46.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d201f4c00 con 0x7f0d200ce380 2024-05-08T22:34:46.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d20067740 con 0x7f0d200ce380 2024-05-08T22:34:46.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d18021030 con 0x7f0d200ce380 2024-05-08T22:34:46.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d1effd700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 0x7f0d0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0d18085760 con 0x7f0d200ce380 2024-05-08T22:34:46.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.381+0000 7f0d26082700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 0x7f0d0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.385+0000 7f0d26082700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 0x7f0d0c068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000f60 tx=0x7f0d14007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:46.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.393+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0d18052a20 con 0x7f0d200ce380 2024-05-08T22:34:46.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.601+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d2004e700 con 0x7f0d200ce380 2024-05-08T22:34:46.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d1effd700 1 -- 172.21.15.179:0/2215460391 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f0d18056040 con 0x7f0d200ce380 2024-05-08T22:34:46.606 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:46.606 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 msgr2=0x7f0d0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 0x7f0d0c068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000f60 tx=0x7f0d14007e00 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 msgr2=0x7f0d20150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0d200b3e80 tx=0x7f0d18004770 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 shutdown_connections 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d0c066730 0x7f0d0c068bf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d200cc700 0x7f0d2014f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d200ce380 0x7f0d20150420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 --2- 172.21.15.179:0/2215460391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d200cb510 0x7f0d2014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 >> 172.21.15.179:0/2215460391 conn(0x7f0d200ce9d0 msgr2=0x7f0d2006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:46.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 shutdown_connections 2024-05-08T22:34:46.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:46.605+0000 7f0d27084700 1 -- 172.21.15.179:0/2215460391 wait complete. 2024-05-08T22:34:46.610 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:34:47.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:34:41.381266+0000', 'modified': '2024-05-08T22:34:44.395407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5562': {'gid': 5562, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/2344889573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 2344889573}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 2344889573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:34:47.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:34:47.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T22:34:47.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.149+0000 7f7f14c28700 1 Processor -- start 2024-05-08T22:34:47.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.149+0000 7f7f14c28700 1 -- start start 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f080e0250 0x7f7f0804f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f08006150 0x7f7f0804f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080540b0 con 0x7f7f080e3b40 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080532f0 con 0x7f7f08006150 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080535d0 con 0x7f7f080e0250 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f08006150 0x7f7f0804f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f08006150 0x7f7f0804f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35332/0 (socket says 172.21.15.179:35332) 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 -- 172.21.15.179:0/2977680175 learned_addr learned my addr 172.21.15.179:0/2977680175 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0f7fe700 1 --1- 172.21.15.179:0/2977680175 >> v1:172.21.15.179:6789/0 conn(0x7f7f080e3b40 0x7f7f080e3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39066/0 (socket says 172.21.15.179:39066) 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0effd700 1 --2- 172.21.15.179:0/2977680175 >> v2:172.21.15.195:3300/0 conn(0x7f7f080e0250 0x7f7f0804f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 -- 172.21.15.179:0/2977680175 >> v2:172.21.15.195:3300/0 conn(0x7f7f080e0250 msgr2=0x7f7f0804f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/2977680175 >> v2:172.21.15.195:3300/0 conn(0x7f7f080e0250 0x7f7f0804f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 -- 172.21.15.179:0/2977680175 >> v1:172.21.15.179:6789/0 conn(0x7f7f080e3b40 legacy=0x7f7f080e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 -- 172.21.15.179:0/2977680175 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080538b0 con 0x7f7f08006150 2024-05-08T22:34:47.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/2977680175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0804f700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f10068290 tx=0x7f7f1004f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f180031452f351 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0e7fc700 1 -- 172.21.15.179:0/2977680175 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10067250 con 0x7f7f08006150 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0e7fc700 1 -- 172.21.15.179:0/2977680175 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100673b0 con 0x7f7f08006150 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f0e7fc700 1 -- 172.21.15.179:0/2977680175 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10067960 con 0x7f7f08006150 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- 172.21.15.179:0/2977680175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 msgr2=0x7f7f0804f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 --2- 172.21.15.179:0/2977680175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0804f700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f10068290 tx=0x7f7f1004f740 comp rx=0 tx=0).stop 2024-05-08T22:34:47.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- 172.21.15.179:0/2977680175 shutdown_connections 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 --2- 172.21.15.179:0/2977680175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0804f700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 --2- 172.21.15.179:0/2977680175 >> v2:172.21.15.195:3300/0 conn(0x7f7f080e0250 0x7f7f0804f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- 172.21.15.179:0/2977680175 >> 172.21.15.179:0/2977680175 conn(0x7f7f08054b70 msgr2=0x7f7f08054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- 172.21.15.179:0/2977680175 shutdown_connections 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.153+0000 7f7f14c28700 1 -- 172.21.15.179:0/2977680175 wait complete. 2024-05-08T22:34:47.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 Processor -- start 2024-05-08T22:34:47.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 -- start start 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 0x7f7f0804e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f080d1f70 0x7f7f0804a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0804ed30 con 0x7f7f08006150 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080d6d90 con 0x7f7f080e3b40 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f14c28700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080e1900 con 0x7f7f080d1f70 2024-05-08T22:34:47.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35340/0 (socket says 172.21.15.179:35340) 2024-05-08T22:34:47.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/958128641 learned_addr learned my addr 172.21.15.179:0/958128641 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f080d1f70 0x7f7f0804a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0effd700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 0x7f7f0804e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f080d1f70 msgr2=0x7f7f0804a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f080d1f70 0x7f7f0804a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 msgr2=0x7f7f0804e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 0x7f7f0804e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/958128641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0817cb50 con 0x7f7f08006150 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0effd700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 0x7f7f0804e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c4d0 tx=0x7f7f0000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.157+0000 7f7ef7fff700 1 -- 172.21.15.179:0/958128641 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00005cd0 con 0x7f7f08006150 2024-05-08T22:34:47.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0817ce30 con 0x7f7f08006150 2024-05-08T22:34:47.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7ef7fff700 1 -- 172.21.15.179:0/958128641 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00005e30 con 0x7f7f08006150 2024-05-08T22:34:47.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0817d3f0 con 0x7f7f08006150 2024-05-08T22:34:47.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7ef7fff700 1 -- 172.21.15.179:0/958128641 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000072d0 con 0x7f7f08006150 2024-05-08T22:34:47.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7ef7fff700 1 -- 172.21.15.179:0/958128641 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f00007430 con 0x7f7f08006150 2024-05-08T22:34:47.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.161+0000 7f7ef7fff700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 0x7f7efc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7ef7fff700 1 -- 172.21.15.179:0/958128641 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f00083250 con 0x7f7f08006150 2024-05-08T22:34:47.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0effd700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 0x7f7efc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0effd700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 0x7f7efc068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f08038580 tx=0x7f7f04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 Processor -- start 2024-05-08T22:34:47.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 -- start start 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1e7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ddbd80 con 0x1eb4b00 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd9510 con 0x1eb5420 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd4e00 con 0x1eabbe0 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35354/0 (socket says 172.21.15.179:35354) 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/2571853443 learned_addr learned my addr 172.21.15.179:0/2571853443 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0effd700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1e7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 msgr2=0x1e7c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.165+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e7c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 msgr2=0x1e7c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1e7c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/2571853443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c47960 con 0x1eb4b00 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1e7c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f000070a0 tx=0x7f7f00053b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58167ffefa01ddaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7ef57fa700 1 -- 172.21.15.179:0/2571853443 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00057ee0 con 0x1eb4b00 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7ef57fa700 1 -- 172.21.15.179:0/2571853443 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0005c4a0 con 0x1eb4b00 2024-05-08T22:34:47.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7ef57fa700 1 -- 172.21.15.179:0/2571853443 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000522a0 con 0x1eb4b00 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- 172.21.15.179:0/2571853443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 msgr2=0x1eb4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f000070a0 tx=0x7f7f00053b90 comp rx=0 tx=0).stop 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- 172.21.15.179:0/2571853443 shutdown_connections 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1e7c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e7c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 --2- 172.21.15.179:0/2571853443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1eb4ee0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- 172.21.15.179:0/2571853443 >> 172.21.15.179:0/2571853443 conn(0x1bfc4d0 msgr2=0x1e96250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- 172.21.15.179:0/2571853443 shutdown_connections 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- 172.21.15.179:0/2571853443 wait complete. 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 -- start start 2024-05-08T22:34:47.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.169+0000 7f7f19139740 1 Processor -- start 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eabbe0 0x1eab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1e6c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d92730 con 0x1eb5420 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d985b0 con 0x1eabbe0 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c80aa0 con 0x1eb4b00 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 10 client.0 _ll_register_callbacks cb 0x1dce610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35362/0 (socket says 172.21.15.179:35362) 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 -- 172.21.15.179:0/4285617613 learned_addr learned my addr 172.21.15.179:0/4285617613 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eabbe0 0x1eab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0effd700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1e6c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 -- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 msgr2=0x1e6c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1e6c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 -- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eabbe0 msgr2=0x1eab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eabbe0 0x1eab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 -- 172.21.15.179:0/4285617613 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0817d3f0 con 0x1eb5420 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0effd700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1e6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000b60 tx=0x7f7f1006a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e73ee8560930d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10099040 con 0x1eb5420 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f10067d70 con 0x1eb5420 2024-05-08T22:34:47.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c091e0 con 0x1eb5420 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f100669e0 con 0x1eb5420 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 10 client.5742 fetch_fsmap learned FSMap version 17 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1d9b220 con 0x1eb5420 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c091e0 con 0x1eb5420 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7f10066b40 con 0x1eb5420 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 10 client.5742 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 10 client.5742 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 20 client.5742 populate_metadata read hostname 'smithi179' 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1ead2e0 0x1ec83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 4 client.5742 mds_command: new command op to 5562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 --> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c53460 con 0x1ead2e0 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1ead2e0 0x1ec83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1ead2e0 0x1ec83f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e47e6508bae677 server_cookie=441e66ecbf761255 in_seq=0 out_seq=0 2024-05-08T22:34:47.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.173+0000 7f7eee7fc700 10 client.5742 ms_handle_connect on v2:172.21.15.179:6834/2344889573 2024-05-08T22:34:47.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7eee7fc700 1 -- 172.21.15.179:0/4285617613 <== mds.0 v2:172.21.15.179:6834/2344889573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1c53460 con 0x1ead2e0 2024-05-08T22:34:47.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7eee7fc700 10 client.5742 handle_command_reply: tid=0 2024-05-08T22:34:47.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 client.5742 shutdown 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 msgr2=0x1e6ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000b60 tx=0x7f7f1006a050 comp rx=0 tx=0).stop 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 shutdown_connections 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1ead2e0 0x1ec83f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5420 0x1e6ceb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1e6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7eee7fc700 0 client.5742 ms_handle_reset on v2:172.21.15.179:6834/2344889573 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 --2- 172.21.15.179:0/4285617613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eabbe0 0x1eab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 >> 172.21.15.179:0/4285617613 conn(0x1bfc4d0 msgr2=0x1e95a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 shutdown_connections 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 1 -- 172.21.15.179:0/4285617613 wait complete. 2024-05-08T22:34:47.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.177+0000 7f7f19139740 20 client.5742 trim_cache size 0 max 16384 2024-05-08T22:34:47.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.193+0000 7f7f19139740 1 Processor -- start 2024-05-08T22:34:47.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.193+0000 7f7f19139740 1 -- start start 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e6ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1ec4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e03ec0 con 0x1eb4b00 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d73470 con 0x1eb5420 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d96e90 con 0x1eabbe0 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35364/0 (socket says 172.21.15.179:35364) 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 -- 172.21.15.179:0/2021843805 learned_addr learned my addr 172.21.15.179:0/2021843805 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1ec4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e6ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 -- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 msgr2=0x1e6ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e6ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 -- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 msgr2=0x1ec4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1ec4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 -- 172.21.15.179:0/2021843805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d81450 con 0x1eb4b00 2024-05-08T22:34:47.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f0effd700 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f040107b0 tx=0x7f7f04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d4354c7c9edad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7ef5ffb700 1 -- 172.21.15.179:0/2021843805 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04015880 con 0x1eb4b00 2024-05-08T22:34:47.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7ef5ffb700 1 -- 172.21.15.179:0/2021843805 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f040159e0 con 0x1eb4b00 2024-05-08T22:34:47.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7ef5ffb700 1 -- 172.21.15.179:0/2021843805 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04015d70 con 0x1eb4b00 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- 172.21.15.179:0/2021843805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 msgr2=0x1ec41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f040107b0 tx=0x7f7f04016000 comp rx=0 tx=0).stop 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- 172.21.15.179:0/2021843805 shutdown_connections 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1ec4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb4b00 0x1ec41e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 --2- 172.21.15.179:0/2021843805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eabbe0 0x1e6ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- 172.21.15.179:0/2021843805 >> 172.21.15.179:0/2021843805 conn(0x1bfc4d0 msgr2=0x1eaa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- 172.21.15.179:0/2021843805 shutdown_connections 2024-05-08T22:34:47.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.197+0000 7f7f19139740 1 -- 172.21.15.179:0/2021843805 wait complete. 2024-05-08T22:34:47.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 -- start start 2024-05-08T22:34:47.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 Processor -- start 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1eb9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1eba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1edc690 con 0x1eabbe0 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eccf40 con 0x1eb5420 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eba700 con 0x1eb4b00 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f19139740 10 client.0 _ll_register_callbacks cb 0x7f7f000504a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35372/0 (socket says 172.21.15.179:35372) 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/368132391 learned_addr learned my addr 172.21.15.179:0/368132391 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0effd700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1eb9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0ffff700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1eba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 msgr2=0x1eb9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1eb9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 msgr2=0x1eba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1eba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.201+0000 7f7f0f7fe700 1 -- 172.21.15.179:0/368132391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0400e050 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f0f7fe700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f00053e60 tx=0x7f7f00009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b5fa8831cf4f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00057f60 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f00081070 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f0005c4a0 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f000720d0 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 10 client.5748 fetch_fsmap learned FSMap version 17 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1d75720 con 0x1eabbe0 2024-05-08T22:34:47.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0005c4a0 con 0x1eabbe0 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7f0005a070 con 0x1eabbe0 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 10 client.5748 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 10 client.5748 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 20 client.5748 populate_metadata read hostname 'smithi179' 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1e69f10 0x1e6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 4 client.5748 mds_command: new command op to 5562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 --> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1c781e0 con 0x1e69f10 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f0effd700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1e69f10 0x1e6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 10 client.5748 ms_handle_connect on v2:172.21.15.179:6834/2344889573 2024-05-08T22:34:47.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f0effd700 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1e69f10 0x1e6a2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60179991da69835e server_cookie=22e38ad005cf3736 in_seq=0 out_seq=0 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 1 -- 172.21.15.179:0/368132391 <== mds.0 v2:172.21.15.179:6834/2344889573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1c781e0 con 0x1e69f10 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 10 client.5748 handle_command_reply: tid=0 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 client.5748 shutdown 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 msgr2=0x1f1e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f00053e60 tx=0x7f7f00009040 comp rx=0 tx=0).stop 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 shutdown_connections 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb5420 0x1eba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:6834/2344889573,v1:172.21.15.179:6835/2344889573] conn(0x1e69f10 0x1e6a2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb4b00 0x1eb9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 --2- 172.21.15.179:0/368132391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eabbe0 0x1f1e1b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7eee7fc700 0 client.5748 ms_handle_reset on v2:172.21.15.179:6834/2344889573 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 >> 172.21.15.179:0/368132391 conn(0x1bfc4d0 msgr2=0x1eaa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 shutdown_connections 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 1 -- 172.21.15.179:0/368132391 wait complete. 2024-05-08T22:34:47.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.205+0000 7f7f19139740 20 client.5748 trim_cache size 0 max 16384 2024-05-08T22:34:47.213 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:34:47.213 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:34:47.213 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:34:47.213 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 5562, 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 5.8258015329999999, 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 17, 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 41, 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 41, 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 13.133361903000001 2024-05-08T22:34:47.214 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:34:47.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 msgr2=0x7f7efc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 0x7f7efc068bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f08038580 tx=0x7f7f04008040 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 msgr2=0x7f7f0817c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c4d0 tx=0x7f7f0000c7e0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 shutdown_connections 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7efc066730 0x7f7efc068bf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f080e3b40 0x7f7f0804e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f080d1f70 0x7f7f0804a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 --2- 172.21.15.179:0/958128641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f08006150 0x7f7f0817c520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 >> 172.21.15.179:0/958128641 conn(0x7f7f08054b70 msgr2=0x7f7f080371e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 shutdown_connections 2024-05-08T22:34:47.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.209+0000 7f7f14c28700 1 -- 172.21.15.179:0/958128641 wait complete. 2024-05-08T22:34:47.225 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:34:47.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.345+0000 7f1207fff700 1 Processor -- start 2024-05-08T22:34:47.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.345+0000 7f1207fff700 1 -- start start 2024-05-08T22:34:47.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f120815b000 0x7f12080cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080cc040 0x7f12080b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080cf630 con 0x7f12080d06d0 2024-05-08T22:34:47.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120815b3e0 con 0x7f120815b000 2024-05-08T22:34:47.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080cf910 con 0x7f12080cc040 2024-05-08T22:34:47.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f11fe7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f12080d06d0 0x7f12080cb930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39070/0 (socket says 172.21.15.179:39070) 2024-05-08T22:34:47.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f11fe7fc700 1 -- 172.21.15.179:0/992691900 learned_addr learned my addr 172.21.15.179:0/992691900 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:47.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 --2- 172.21.15.179:0/992691900 >> v2:172.21.15.179:3300/0 conn(0x7f120815b000 0x7f12080cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f12077fe700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080cc040 0x7f12080b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 -- 172.21.15.179:0/992691900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080cc040 msgr2=0x7f12080b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080cc040 0x7f12080b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 -- 172.21.15.179:0/992691900 >> v1:172.21.15.179:6789/0 conn(0x7f12080d06d0 legacy=0x7f12080cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f12067fc700 1 -- 172.21.15.179:0/992691900 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577568557 0 0) 0x7f12080cf630 con 0x7f12080d06d0 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 -- 172.21.15.179:0/992691900 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080cfbf0 con 0x7f120815b000 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1206ffd700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f12080cefc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009850 tx=0x7f11f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18af608ee158380 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f12067fc700 1 -- 172.21.15.179:0/992691900 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8011830 con 0x7f120815b000 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f12067fc700 1 -- 172.21.15.179:0/992691900 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8011990 con 0x7f120815b000 2024-05-08T22:34:47.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f12067fc700 1 -- 172.21.15.179:0/992691900 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8011d40 con 0x7f120815b000 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- 172.21.15.179:0/992691900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 msgr2=0x7f12080cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f12080cefc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009850 tx=0x7f11f8009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- 172.21.15.179:0/992691900 shutdown_connections 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080cc040 0x7f12080b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 --2- 172.21.15.179:0/992691900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f12080cefc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- 172.21.15.179:0/992691900 >> 172.21.15.179:0/992691900 conn(0x7f12080b2780 msgr2=0x7f12080b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- 172.21.15.179:0/992691900 shutdown_connections 2024-05-08T22:34:47.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.349+0000 7f1207fff700 1 -- 172.21.15.179:0/992691900 wait complete. 2024-05-08T22:34:47.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 Processor -- start 2024-05-08T22:34:47.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- start start 2024-05-08T22:34:47.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080cc040 0x7f12081536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080d06d0 0x7f120815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208150550 con 0x7f120815b000 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120815bc30 con 0x7f12080cc040 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120815bf10 con 0x7f12080d06d0 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35394/0 (socket says 172.21.15.179:35394) 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 -- 172.21.15.179:0/2517841567 learned_addr learned my addr 172.21.15.179:0/2517841567 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1206ffd700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080cc040 0x7f12081536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f11fe7fc700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080d06d0 0x7f120815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 -- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080d06d0 msgr2=0x7f120815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080d06d0 0x7f120815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 -- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080cc040 msgr2=0x7f12081536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080cc040 0x7f12081536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 -- 172.21.15.179:0/2517841567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80094b0 con 0x7f120815b000 2024-05-08T22:34:47.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f12077fe700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f120000efc0 tx=0x7f120000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200009ea0 con 0x7f120815b000 2024-05-08T22:34:47.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000f040 con 0x7f120815b000 2024-05-08T22:34:47.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000ced0 con 0x7f120815b000 2024-05-08T22:34:47.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120815c1f0 con 0x7f120815b000 2024-05-08T22:34:47.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.353+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1208150c20 con 0x7f120815b000 2024-05-08T22:34:47.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1200014480 con 0x7f120815b000 2024-05-08T22:34:47.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1208150e40 con 0x7f120815b000 2024-05-08T22:34:47.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f11fffff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 0x7f11e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:47.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1200084a20 con 0x7f120815b000 2024-05-08T22:34:47.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f1206ffd700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 0x7f11e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:47.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.357+0000 7f1206ffd700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 0x7f11e8068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f11f80090b0 tx=0x7f11f801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:47.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.365+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1200051cf0 con 0x7f120815b000 2024-05-08T22:34:47.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.577+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f120804e700 con 0x7f120815b000 2024-05-08T22:34:47.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.577+0000 7f11fffff700 1 -- 172.21.15.179:0/2517841567 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1200018020 con 0x7f120815b000 2024-05-08T22:34:47.580 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 msgr2=0x7f11e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 0x7f11e8068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f11f80090b0 tx=0x7f11f801c040 comp rx=0 tx=0).stop 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 msgr2=0x7f1208150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f120000efc0 tx=0x7f120000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 shutdown_connections 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f11e8066730 0x7f11e8068bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f120815b000 0x7f1208150010 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12080d06d0 0x7f120815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 --2- 172.21.15.179:0/2517841567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080cc040 0x7f12081536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:47.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 >> 172.21.15.179:0/2517841567 conn(0x7f12080b2780 msgr2=0x7f12080b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:47.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 shutdown_connections 2024-05-08T22:34:47.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:47.581+0000 7f1207fff700 1 -- 172.21.15.179:0/2517841567 wait complete. 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:34:47.595 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:34:47.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.596 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:34:47.600 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:34:47.600 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-05-08T22:34:47.601 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:34:47.602 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:34:47.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.603 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:34:47.617 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:34:47.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.621 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:34:47.629 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:34:47.630 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:34:47.630 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:34:47.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:34:47.630 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:34:47.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.665 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:34:47.716 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:34:47.716 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:34:47.716 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:34:47.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:34:47.733 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:34:47.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:34:47.733 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:47.739 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:34:47.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:34:47.739 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:47.788 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:34:47.803 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:34:47.803 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:34:47.814 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:34:47.815 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:34:47.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.816 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:34:47.821 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:34:47.821 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:34:47.863 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:34:47.882 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:34:47.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:34:47.898 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:34:47.900 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:34:47.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:47.900 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:34:47.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:34:47.949+0000 7efe524c6640 -1 init, newargv = 0x564d77699a90 newargc=15 2024-05-08T22:34:47.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[37163]: starting ceph client 2024-05-08T22:34:47.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[37163]: starting fuse 2024-05-08T22:34:49.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:34:49.025 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:34:49.025 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:34:49.035 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:34:49.036 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:34:49.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.037 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:34:49.088 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:34:49.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:34:49.088 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:49.088 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:34:49.088 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:34:49.088 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:49.088 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:34:49.089 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:34:49.090 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:49.170 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.37163.asok 2024-05-08T22:34:49.356 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37163.asok 2024-05-08T22:34:49.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.356 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37163.asok status 2024-05-08T22:34:49.457 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "37163", 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 5760, 2024-05-08T22:34:49.458 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 5760 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 754831204 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 754831204 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:34:49.459 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.5760 192.168.0.1:0/754831204", 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/754831204", 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 17, 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 41, 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:34:49.460 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:34:49.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.467 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:34:49.467 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:49.475 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:34:49.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:49.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.475 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:49.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.535 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-08T22:34:49.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.552 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-08T22:34:49.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.564 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-08T22:34:49.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.617 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-08T22:34:49.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.636 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-08T22:34:49.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:49.647 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T22:34:49.673 INFO:teuthology.orchestra.run.smithi179.stdout:./ 2024-05-08T22:34:49.673 INFO:teuthology.orchestra.run.smithi179.stdout:./rfile 2024-05-08T22:34:49.673 INFO:teuthology.orchestra.run.smithi179.stdout:./parentdir 2024-05-08T22:34:49.673 INFO:teuthology.orchestra.run.smithi179.stdout:./parentdir/pfile 2024-05-08T22:34:49.673 INFO:teuthology.orchestra.run.smithi179.stdout:./parentdir/childdir 2024-05-08T22:34:49.674 INFO:teuthology.orchestra.run.smithi179.stdout:./parentdir/childdir/cfile 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T22:34:49.675 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:49.742 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:34:49.932 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:49.932 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:49.932 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:49.932 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:49.932 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-08T22:34:49.933 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:49.996 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627778 2024-05-08T22:34:50.187 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:50.187 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:50.187 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:50.187 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:50.188 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-08T22:34:50.188 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:50.298 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:34:50.486 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:50.486 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:50.487 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:50.487 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:50.487 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-08T22:34:50.487 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:50.557 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628279 2024-05-08T22:34:50.743 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:50.744 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:50.744 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:50.744 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:50.744 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-08T22:34:50.744 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:50.807 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:34:50.995 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:34:50.995 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:34:50.995 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:34:50.995 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:34:50.995 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-08T22:34:50.996 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:34:51.067 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628280 2024-05-08T22:34:51.254 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:51.255 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:51.262 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:51.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 Processor -- start 2024-05-08T22:34:51.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 -- start start 2024-05-08T22:34:51.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd38c15d5b0 0x7fd38c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd38c0ce580 0x7fd38c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c06afc0 con 0x7fd38c0b2120 2024-05-08T22:34:51.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c15d990 con 0x7fd38c0ce580 2024-05-08T22:34:51.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd391017700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0d26f0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd390816700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd38c0ce580 0x7fd38c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd38b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd38c0b2120 0x7fd38c0cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39072/0 (socket says 172.21.15.179:39072) 2024-05-08T22:34:51.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd38b7fe700 1 -- 172.21.15.179:0/3737243851 learned_addr learned my addr 172.21.15.179:0/3737243851 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:51.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd38bfff700 1 --2- 172.21.15.179:0/3737243851 >> v2:172.21.15.195:3300/0 conn(0x7fd38c15d5b0 0x7fd38c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd390816700 1 -- 172.21.15.179:0/3737243851 >> v2:172.21.15.195:3300/0 conn(0x7fd38c15d5b0 msgr2=0x7fd38c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd390816700 1 --2- 172.21.15.179:0/3737243851 >> v2:172.21.15.195:3300/0 conn(0x7fd38c15d5b0 0x7fd38c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd390816700 1 -- 172.21.15.179:0/3737243851 >> v1:172.21.15.179:6789/0 conn(0x7fd38c0b2120 legacy=0x7fd38c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd38affd700 1 -- 172.21.15.179:0/3737243851 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117884313 0 0) 0x7fd38c06afc0 con 0x7fd38c0b2120 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.425+0000 7fd390816700 1 -- 172.21.15.179:0/3737243851 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0d2130 con 0x7fd38c0ce580 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd390816700 1 --2- 172.21.15.179:0/3737243851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c0ce580 0x7fd38c06b450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd38c08a630 tx=0x7fd384009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b1223933b7aa0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd38affd700 1 -- 172.21.15.179:0/3737243851 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384011830 con 0x7fd38c0ce580 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd38affd700 1 -- 172.21.15.179:0/3737243851 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384011990 con 0x7fd38c0ce580 2024-05-08T22:34:51.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd38affd700 1 -- 172.21.15.179:0/3737243851 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384011d20 con 0x7fd38c0ce580 2024-05-08T22:34:51.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 -- 172.21.15.179:0/3737243851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c0ce580 msgr2=0x7fd38c06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 --2- 172.21.15.179:0/3737243851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c0ce580 0x7fd38c06b450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd38c08a630 tx=0x7fd384009b30 comp rx=0 tx=0).stop 2024-05-08T22:34:51.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 -- 172.21.15.179:0/3737243851 shutdown_connections 2024-05-08T22:34:51.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 --2- 172.21.15.179:0/3737243851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c0ce580 0x7fd38c06b450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 --2- 172.21.15.179:0/3737243851 >> v2:172.21.15.195:3300/0 conn(0x7fd38c15d5b0 0x7fd38c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 -- 172.21.15.179:0/3737243851 >> 172.21.15.179:0/3737243851 conn(0x7fd38c0b4940 msgr2=0x7fd38c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 -- 172.21.15.179:0/3737243851 shutdown_connections 2024-05-08T22:34:51.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 -- 172.21.15.179:0/3737243851 wait complete. 2024-05-08T22:34:51.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.429+0000 7fd391017700 1 Processor -- start 2024-05-08T22:34:51.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- start start 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 0x7fd38c14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 0x7fd38c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0ccf10 con 0x7fd38c15d5b0 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c14f150 con 0x7fd38c0ce580 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cc060 con 0x7fd38c0b2120 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35420/0 (socket says 172.21.15.179:35420) 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 -- 172.21.15.179:0/239270859 learned_addr learned my addr 172.21.15.179:0/239270859 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd38bfff700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 0x7fd38c14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd38b7fe700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 0x7fd38c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 -- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 msgr2=0x7fd38c14ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 0x7fd38c14ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 -- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 msgr2=0x7fd38c1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 0x7fd38c1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 -- 172.21.15.179:0/239270859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840094b0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd38bfff700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 0x7fd38c14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd38b7fe700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 0x7fd38c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd390816700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fd0 tx=0x7fd384004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840045b0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38401f070 con 0x7fd38c15d5b0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c160920 con 0x7fd38c15d5b0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840038e0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.433+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c0cc5e0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c0cd080 con 0x7fd38c15d5b0 2024-05-08T22:34:51.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd384021030 con 0x7fd38c15d5b0 2024-05-08T22:34:51.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd388ff9700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 0x7fd36c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd384085620 con 0x7fd38c15d5b0 2024-05-08T22:34:51.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd38bfff700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 0x7fd36c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.437+0000 7fd38bfff700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 0x7fd36c068c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ba0 tx=0x7fd374008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.445+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd3840528e0 con 0x7fd38c15d5b0 2024-05-08T22:34:51.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38c04e700 con 0x7fd38c15d5b0 2024-05-08T22:34:51.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd388ff9700 1 -- 172.21.15.179:0/239270859 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16354 (secure 0 0 0) 0x7fd384055f00 con 0x7fd38c15d5b0 2024-05-08T22:34:51.615 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:51.615 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":41,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:42.384705+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:51.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 msgr2=0x7fd36c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 0x7fd36c068c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ba0 tx=0x7fd374008040 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 msgr2=0x7fd38c15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fd0 tx=0x7fd384004770 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 shutdown_connections 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd36c066760 0x7fd36c068c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0ce580 0x7fd38c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c15d5b0 0x7fd38c15f150 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 --2- 172.21.15.179:0/239270859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b2120 0x7fd38c14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.613+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 >> 172.21.15.179:0/239270859 conn(0x7fd38c0b4940 msgr2=0x7fd38c15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.617+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 shutdown_connections 2024-05-08T22:34:51.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.617+0000 7fd391017700 1 -- 172.21.15.179:0/239270859 wait complete. 2024-05-08T22:34:51.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:34:39.249198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:34:40.358534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:34:51.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:51.680 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:34:51.682 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:34:51.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 Processor -- start 2024-05-08T22:34:51.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 -- start start 2024-05-08T22:34:51.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142fd0 0x7f59bc06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59bc143520 0x7f59bc06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0b22d0 con 0x7f59bc142890 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0b19e0 con 0x7f59bc143520 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c1060700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc15e640 con 0x7f59bc142fd0 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.745+0000 7f59c085f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59bc143520 0x7f59bc06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59bc143520 0x7f59bc06d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35424/0 (socket says 172.21.15.179:35424) 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 -- 172.21.15.179:0/3909709850 learned_addr learned my addr 172.21.15.179:0/3909709850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59b37fe700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142fd0 0x7f59bc06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 -- 172.21.15.179:0/3909709850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142fd0 msgr2=0x7f59bc06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142fd0 0x7f59bc06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 -- 172.21.15.179:0/3909709850 >> v1:172.21.15.195:6789/0 conn(0x7f59bc142890 legacy=0x7f59bc0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 -- 172.21.15.179:0/3909709850 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0b1c80 con 0x7f59bc143520 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c085f700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc06d2e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76293fbdc38c685f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59bb7fe700 1 -- 172.21.15.179:0/3909709850 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400e070 con 0x7f59bc143520 2024-05-08T22:34:51.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59bb7fe700 1 -- 172.21.15.179:0/3909709850 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b400b9c0 con 0x7f59bc143520 2024-05-08T22:34:51.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59bb7fe700 1 -- 172.21.15.179:0/3909709850 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400bd70 con 0x7f59bc143520 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 -- 172.21.15.179:0/3909709850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 msgr2=0x7f59bc06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc06d2e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 -- 172.21.15.179:0/3909709850 shutdown_connections 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142fd0 0x7f59bc06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 --2- 172.21.15.179:0/3909709850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc06d2e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 -- 172.21.15.179:0/3909709850 >> 172.21.15.179:0/3909709850 conn(0x7f59bc0cea30 msgr2=0x7f59bc0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 -- 172.21.15.179:0/3909709850 shutdown_connections 2024-05-08T22:34:51.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.749+0000 7f59c1060700 1 -- 172.21.15.179:0/3909709850 wait complete. 2024-05-08T22:34:51.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 Processor -- start 2024-05-08T22:34:51.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 -- start start 2024-05-08T22:34:51.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142890 0x7f59bc0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 0x7f59bc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cd5d0 con 0x7f59bc143520 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cd8b0 con 0x7f59bc142fd0 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c1060700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1f3d20 con 0x7f59bc142890 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c085f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c085f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35432/0 (socket says 172.21.15.179:35432) 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59c085f700 1 -- 172.21.15.179:0/1736583858 learned_addr learned my addr 172.21.15.179:0/1736583858 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59b37fe700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 0x7f59bc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.753+0000 7f59bbfff700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142890 0x7f59bc0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 -- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142890 msgr2=0x7f59bc0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142890 0x7f59bc0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 -- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 msgr2=0x7f59bc1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 0x7f59bc1575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 -- 172.21.15.179:0/1736583858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009b30 con 0x7f59bc143520 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b37fe700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 0x7f59bc1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c085f700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f59bc06c6c0 tx=0x7f59b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40042d0 con 0x7f59bc143520 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4004430 con 0x7f59bc143520 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc1f03d0 con 0x7f59bc143520 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40047e0 con 0x7f59bc143520 2024-05-08T22:34:51.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc14a6b0 con 0x7f59bc143520 2024-05-08T22:34:51.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc0cd710 con 0x7f59bc143520 2024-05-08T22:34:51.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59b401c440 con 0x7f59bc143520 2024-05-08T22:34:51.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.757+0000 7f59b97fa700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 0x7f599c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.761+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59b4085be0 con 0x7f59bc143520 2024-05-08T22:34:51.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.761+0000 7f59bbfff700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 0x7f599c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.761+0000 7f59bbfff700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 0x7f599c068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.765+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59bc0cd710 con 0x7f59bc143520 2024-05-08T22:34:51.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.933+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59bc0cd710 con 0x7f59bc143520 2024-05-08T22:34:51.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59b97fa700 1 -- 172.21.15.179:0/1736583858 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59bc0cd710 con 0x7f59bc143520 2024-05-08T22:34:51.938 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 msgr2=0x7f599c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 0x7f599c068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac009040 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 msgr2=0x7f59bc1efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f59bc06c6c0 tx=0x7f59b4004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 shutdown_connections 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f599c066730 0x7f599c068bf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59bc142fd0 0x7f59bc1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc143520 0x7f59bc1efe90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 --2- 172.21.15.179:0/1736583858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59bc142890 0x7f59bc0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 >> 172.21.15.179:0/1736583858 conn(0x7f59bc0cea30 msgr2=0x7f59bc0cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 shutdown_connections 2024-05-08T22:34:51.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:51.937+0000 7f59c1060700 1 -- 172.21.15.179:0/1736583858 wait complete. 2024-05-08T22:34:51.953 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:52.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 Processor -- start 2024-05-08T22:34:52.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 -- start start 2024-05-08T22:34:52.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6a000d1aa0 0x7f6a00167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a000d1fe0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a000d22c0 con 0x7f6a0015d2f0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f6a049ca700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0015d6d0 con 0x7f6a000d1aa0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f69fe59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6a000b4ff0 0x7f6a000d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39076/0 (socket says 172.21.15.179:39076) 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.065+0000 7f69fe59c700 1 -- 172.21.15.179:0/3124090194 learned_addr learned my addr 172.21.15.179:0/3124090194 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69ff59e700 1 --2- 172.21.15.179:0/3124090194 >> v2:172.21.15.195:3300/0 conn(0x7f6a000d1aa0 0x7f6a00167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916848635 0 0) 0x7f6a000d1fe0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e80035d0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004780863 0 0) 0x7f6a000d22c0 con 0x7f6a0015d2f0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a000d1fe0 con 0x7f6a0015d2f0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2099533312 0 0) 0x7f69e80035d0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 >> v2:172.21.15.195:3300/0 conn(0x7f6a000d1aa0 msgr2=0x7f6a00167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 --2- 172.21.15.179:0/3124090194 >> v2:172.21.15.195:3300/0 conn(0x7f6a000d1aa0 0x7f6a00167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 >> v1:172.21.15.195:6789/0 conn(0x7f6a0015d2f0 legacy=0x7f6a000b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000d25a0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f69f00017f0 con 0x7f6a000b4ff0 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 >> v1:172.21.15.179:6789/0 conn(0x7f6a000b4ff0 legacy=0x7f6a000d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 0x7f69e8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69e8004870 0x7f69e8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8005280 0x7f69e800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0015d6d0 con 0x7f69e8003c30 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e800c1d0 con 0x7f69e8004870 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e800c390 con 0x7f69e8005280 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 0x7f69e8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fe59c700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69e8004870 0x7f69e8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69ff59e700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8005280 0x7f69e800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 -- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8005280 msgr2=0x7f69e800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8005280 0x7f69e800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 -- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69e8004870 msgr2=0x7f69e8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:52.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69e8004870 0x7f69e8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69ff59e700 1 -- 172.21.15.179:0/3124090194 reap_dead start 2024-05-08T22:34:52.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 -- 172.21.15.179:0/3124090194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80035d0 con 0x7f69e8003c30 2024-05-08T22:34:52.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fed9d700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 0x7f69e8004030 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f69f4001960 tx=0x7f69e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253df9ca7abd7d89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4001450 con 0x7f69e8003c30 2024-05-08T22:34:52.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f40052f0 con 0x7f69e8003c30 2024-05-08T22:34:52.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f69fdd9b700 1 -- 172.21.15.179:0/3124090194 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f40036a0 con 0x7f69e8003c30 2024-05-08T22:34:52.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 -- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 msgr2=0x7f69e8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 0x7f69e8004030 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f69f4001960 tx=0x7f69e80058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 -- 172.21.15.179:0/3124090194 shutdown_connections 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 --2- 172.21.15.179:0/3124090194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69e8003c30 0x7f69e8004030 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 -- 172.21.15.179:0/3124090194 >> 172.21.15.179:0/3124090194 conn(0x7f6a000cfac0 msgr2=0x7f6a000cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 -- 172.21.15.179:0/3124090194 shutdown_connections 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.069+0000 7f6a049ca700 1 -- 172.21.15.179:0/3124090194 wait complete. 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 Processor -- start 2024-05-08T22:34:52.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 -- start start 2024-05-08T22:34:52.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8003c30 0x7f6a00141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a001452f0 0x7f6a00145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00142340 con 0x7f6a001f2490 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00143140 con 0x7f6a001452f0 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f6a049ca700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00142530 con 0x7f69e8003c30 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35444/0 (socket says 172.21.15.179:35444) 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 -- 172.21.15.179:0/2627206136 learned_addr learned my addr 172.21.15.179:0/2627206136 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fed9d700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8003c30 0x7f6a00141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69ff59e700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a001452f0 0x7f6a00145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 -- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8003c30 msgr2=0x7f6a00141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8003c30 0x7f6a00141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 -- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a001452f0 msgr2=0x7f6a00145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a001452f0 0x7f6a00145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 -- 172.21.15.179:0/2627206136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00142810 con 0x7f6a001f2490 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.073+0000 7f69fe59c700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f69f000f7d0 tx=0x7f69f000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0001120 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00072e0 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a00142af0 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0007670 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a00145cb0 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a000d1310 con 0x7f6a001f2490 2024-05-08T22:34:52.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69f0005850 con 0x7f6a001f2490 2024-05-08T22:34:52.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 0x7f69ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f69f0084030 con 0x7f6a001f2490 2024-05-08T22:34:52.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.077+0000 7f69fed9d700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 0x7f69ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.081+0000 7f69fed9d700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 0x7f69ec068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005040 tx=0x7f69f400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.085+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f69f0051260 con 0x7f6a001f2490 2024-05-08T22:34:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.257+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a0004e700 con 0x7f6a001f2490 2024-05-08T22:34:52.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.257+0000 7f69e77fe700 1 -- 172.21.15.179:0/2627206136 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69f0054880 con 0x7f6a001f2490 2024-05-08T22:34:52.259 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:52.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 msgr2=0x7f69ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 0x7f69ec068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005040 tx=0x7f69f400a000 comp rx=0 tx=0).stop 2024-05-08T22:34:52.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 msgr2=0x7f6a001435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f69f000f7d0 tx=0x7f69f000fae0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 shutdown_connections 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f69ec066730 0x7f69ec068bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a001452f0 0x7f6a00145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a001f2490 0x7f6a001435d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 --2- 172.21.15.179:0/2627206136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69e8003c30 0x7f6a00141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 >> 172.21.15.179:0/2627206136 conn(0x7f6a000cfac0 msgr2=0x7f6a00154550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 shutdown_connections 2024-05-08T22:34:52.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.261+0000 7f6a049ca700 1 -- 172.21.15.179:0/2627206136 wait complete. 2024-05-08T22:34:52.274 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 Processor -- start 2024-05-08T22:34:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 -- start start 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fae3c0cffc0 0x7fae3c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0cd070 con 0x7fae3c15d000 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0b3b00 con 0x7fae3c0cffc0 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41f86700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0b3d10 con 0x7fae3c0b47f0 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae40f84700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fae3c15d000 0x7fae3c0cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39086/0 (socket says 172.21.15.179:39086) 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae40f84700 1 -- 172.21.15.179:0/3097377773 learned_addr learned my addr 172.21.15.179:0/3097377773 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae41785700 1 --2- 172.21.15.179:0/3097377773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae3bfff700 1 --2- 172.21.15.179:0/3097377773 >> v2:172.21.15.195:3300/0 conn(0x7fae3c0cffc0 0x7fae3c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae3b7fe700 1 -- 172.21.15.179:0/3097377773 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029792072 0 0) 0x7fae3c0cd070 con 0x7fae3c15d000 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.393+0000 7fae3b7fe700 1 -- 172.21.15.179:0/3097377773 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae240035d0 con 0x7fae3c15d000 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 -- 172.21.15.179:0/3097377773 >> v2:172.21.15.195:3300/0 conn(0x7fae3c0cffc0 msgr2=0x7fae3c0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 --2- 172.21.15.179:0/3097377773 >> v2:172.21.15.195:3300/0 conn(0x7fae3c0cffc0 0x7fae3c0cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 -- 172.21.15.179:0/3097377773 >> v1:172.21.15.179:6789/0 conn(0x7fae3c15d000 legacy=0x7fae3c0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 -- 172.21.15.179:0/3097377773 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0b3ff0 con 0x7fae3c0b47f0 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 --2- 172.21.15.179:0/3097377773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c0cb860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318824cc7c2ffeb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3b7fe700 1 -- 172.21.15.179:0/3097377773 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400e070 con 0x7fae3c0b47f0 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3b7fe700 1 -- 172.21.15.179:0/3097377773 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400bb20 con 0x7fae3c0b47f0 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3b7fe700 1 -- 172.21.15.179:0/3097377773 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34012430 con 0x7fae3c0b47f0 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- 172.21.15.179:0/3097377773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 msgr2=0x7fae3c0cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- 172.21.15.179:0/3097377773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c0cb860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- 172.21.15.179:0/3097377773 shutdown_connections 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- 172.21.15.179:0/3097377773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c0cb860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- 172.21.15.179:0/3097377773 >> v2:172.21.15.195:3300/0 conn(0x7fae3c0cffc0 0x7fae3c0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- 172.21.15.179:0/3097377773 >> 172.21.15.179:0/3097377773 conn(0x7fae3c0b27c0 msgr2=0x7fae3c0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- 172.21.15.179:0/3097377773 shutdown_connections 2024-05-08T22:34:52.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- 172.21.15.179:0/3097377773 wait complete. 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 Processor -- start 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- start start 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae3c15d000 0x7fae3c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0c52b0 con 0x7fae3c0cffc0 2024-05-08T22:34:52.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0c7100 con 0x7fae3c15d000 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41f86700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0c4910 con 0x7fae3c0b47f0 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35448/0 (socket says 172.21.15.179:35448) 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae3bfff700 1 -- 172.21.15.179:0/3746854892 learned_addr learned my addr 172.21.15.179:0/3746854892 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae40f84700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.397+0000 7fae41785700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae3c15d000 0x7fae3c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 -- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 msgr2=0x7fae3c151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 -- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae3c15d000 msgr2=0x7fae3c0b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae3c15d000 0x7fae3c0b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 -- 172.21.15.179:0/3746854892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009a70 con 0x7fae3c0cffc0 2024-05-08T22:34:52.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae3bfff700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e9d0 tx=0x7fae2c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c009720 con 0x7fae3c0cffc0 2024-05-08T22:34:52.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0c4bf0 con 0x7fae3c0cffc0 2024-05-08T22:34:52.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00ee70 con 0x7fae3c0cffc0 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00ca50 con 0x7fae3c0cffc0 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1516b0 con 0x7fae3c0cffc0 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c067740 con 0x7fae3c0cffc0 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae2c00cbb0 con 0x7fae3c0cffc0 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.401+0000 7fae397fa700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 0x7fae28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.405+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae2c084240 con 0x7fae3c0cffc0 2024-05-08T22:34:52.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.405+0000 7fae40f84700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 0x7fae28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.405+0000 7fae40f84700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 0x7fae28068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae30007fd0 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.409+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae2c051480 con 0x7fae3c0cffc0 2024-05-08T22:34:52.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.581+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae3c04e700 con 0x7fae3c0cffc0 2024-05-08T22:34:52.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.581+0000 7fae397fa700 1 -- 172.21.15.179:0/3746854892 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae2c00ce60 con 0x7fae3c0cffc0 2024-05-08T22:34:52.584 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:52.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 msgr2=0x7fae28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 0x7fae28068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae30007fd0 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-05-08T22:34:52.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 msgr2=0x7fae3c0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e9d0 tx=0x7fae2c00ece0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 shutdown_connections 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae28066730 0x7fae28068bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae3c15d000 0x7fae3c0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae3c0cffc0 0x7fae3c0b0730 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 --2- 172.21.15.179:0/3746854892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae3c0b47f0 0x7fae3c151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 >> 172.21.15.179:0/3746854892 conn(0x7fae3c0b27c0 msgr2=0x7fae3c0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 shutdown_connections 2024-05-08T22:34:52.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.585+0000 7fae41f86700 1 -- 172.21.15.179:0/3746854892 wait complete. 2024-05-08T22:34:52.599 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:52.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 Processor -- start 2024-05-08T22:34:52.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- start start 2024-05-08T22:34:52.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff33c0b25d0 0x7ff33c0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c0cc340 con 0x7ff33c06e180 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c06b540 con 0x7ff33c06dda0 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c06ac50 con 0x7ff33c0b25d0 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff3327fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff33c06e180 0x7ff33c06e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39088/0 (socket says 172.21.15.179:39088) 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff3327fc700 1 -- 172.21.15.179:0/1379709342 learned_addr learned my addr 172.21.15.179:0/1379709342 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33b7fe700 1 --2- 172.21.15.179:0/1379709342 >> v2:172.21.15.195:3300/0 conn(0x7ff33c0b25d0 0x7ff33c0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33a7fc700 1 -- 172.21.15.179:0/1379709342 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127279754 0 0) 0x7ff324001110 con 0x7ff33c06e180 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33a7fc700 1 -- 172.21.15.179:0/1379709342 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3280035d0 con 0x7ff33c06e180 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33b7fe700 1 -- 172.21.15.179:0/1379709342 >> v1:172.21.15.195:6789/0 conn(0x7ff33c06dda0 legacy=0x7ff33c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33b7fe700 1 -- 172.21.15.179:0/1379709342 >> v1:172.21.15.179:6789/0 conn(0x7ff33c06e180 legacy=0x7ff33c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33b7fe700 1 -- 172.21.15.179:0/1379709342 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c06aef0 con 0x7ff33c0b25d0 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33b7fe700 1 --2- 172.21.15.179:0/1379709342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c0b25d0 0x7ff33c0b2a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d90 tx=0x7ff33400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1149526b045ee750 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33a7fc700 1 -- 172.21.15.179:0/1379709342 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400e070 con 0x7ff33c0b25d0 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33a7fc700 1 -- 172.21.15.179:0/1379709342 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33400bb20 con 0x7ff33c0b25d0 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33a7fc700 1 -- 172.21.15.179:0/1379709342 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400be90 con 0x7ff33c0b25d0 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- 172.21.15.179:0/1379709342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c0b25d0 msgr2=0x7ff33c0b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 --2- 172.21.15.179:0/1379709342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c0b25d0 0x7ff33c0b2a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d90 tx=0x7ff33400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- 172.21.15.179:0/1379709342 shutdown_connections 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 --2- 172.21.15.179:0/1379709342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c0b25d0 0x7ff33c0b2a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- 172.21.15.179:0/1379709342 >> 172.21.15.179:0/1379709342 conn(0x7ff33c0ce960 msgr2=0x7ff33c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- 172.21.15.179:0/1379709342 shutdown_connections 2024-05-08T22:34:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.717+0000 7ff33bfff700 1 -- 172.21.15.179:0/1379709342 wait complete. 2024-05-08T22:34:52.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 Processor -- start 2024-05-08T22:34:52.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 -- start start 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c06e180 0x7ff33c1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff33c0b25d0 0x7ff33c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c15e2c0 con 0x7ff33c06dda0 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c15e5a0 con 0x7ff33c06e180 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c15e880 con 0x7ff33c0b25d0 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35450/0 (socket says 172.21.15.179:35450) 2024-05-08T22:34:52.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 -- 172.21.15.179:0/2976834641 learned_addr learned my addr 172.21.15.179:0/2976834641 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff3327fc700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c06e180 0x7ff33c1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33b7fe700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff33c0b25d0 0x7ff33c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 -- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff33c0b25d0 msgr2=0x7ff33c1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff33c0b25d0 0x7ff33c1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 -- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c06e180 msgr2=0x7ff33c1633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c06e180 0x7ff33c1633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 -- 172.21.15.179:0/2976834641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334009a70 con 0x7ff33c06dda0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.721+0000 7ff33affd700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c410 tx=0x7ff32c00c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0052d0 con 0x7ff33c06dda0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c005430 con 0x7ff33c06dda0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c15eb60 con 0x7ff33c06dda0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0055a0 con 0x7ff33c06dda0 2024-05-08T22:34:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c156d60 con 0x7ff33c06dda0 2024-05-08T22:34:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c15f080 con 0x7ff33c06dda0 2024-05-08T22:34:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff32c0036a0 con 0x7ff33c06dda0 2024-05-08T22:34:52.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 0x7ff31c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:52.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff32c083340 con 0x7ff33c06dda0 2024-05-08T22:34:52.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.725+0000 7ff3327fc700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 0x7ff31c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:52.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.729+0000 7ff3327fc700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 0x7ff31c068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff324007fd0 tx=0x7ff324007c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:52.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.733+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff32c050600 con 0x7ff33c06dda0 2024-05-08T22:34:52.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff33c04e700 con 0x7ff33c06dda0 2024-05-08T22:34:52.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff333fff700 1 -- 172.21.15.179:0/2976834641 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff32c053c20 con 0x7ff33c06dda0 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 msgr2=0x7ff31c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 0x7ff31c068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff324007fd0 tx=0x7ff324007c60 comp rx=0 tx=0).stop 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 msgr2=0x7ff33c156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.917+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c410 tx=0x7ff32c00c720 comp rx=0 tx=0).stop 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 shutdown_connections 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff31c066730 0x7ff31c068bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff33c06e180 0x7ff33c1633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff33c0b25d0 0x7ff33c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 --2- 172.21.15.179:0/2976834641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff33c06dda0 0x7ff33c156820 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 >> 172.21.15.179:0/2976834641 conn(0x7ff33c0ce960 msgr2=0x7ff33c0cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 shutdown_connections 2024-05-08T22:34:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:52.921+0000 7ff33bfff700 1 -- 172.21.15.179:0/2976834641 wait complete. 2024-05-08T22:34:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:52.934 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:34:52.934 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:34:52.939 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:53.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 Processor -- start 2024-05-08T22:34:53.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- start start 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa9400cc310 0x7fa94006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa9400add40 0x7fa94006b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa940161ac0 con 0x7fa9400ce060 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400ae550 con 0x7fa9400add40 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94006abb0 con 0x7fa9400cc310 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa9400add40 0x7fa94006b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa9400add40 0x7fa94006b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35464/0 (socket says 172.21.15.179:35464) 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 -- 172.21.15.179:0/3264478490 learned_addr learned my addr 172.21.15.179:0/3264478490 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa93ffff700 1 --2- 172.21.15.179:0/3264478490 >> v2:172.21.15.195:3300/0 conn(0x7fa9400cc310 0x7fa94006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 -- 172.21.15.179:0/3264478490 >> v2:172.21.15.195:3300/0 conn(0x7fa9400cc310 msgr2=0x7fa94006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 --2- 172.21.15.179:0/3264478490 >> v2:172.21.15.195:3300/0 conn(0x7fa9400cc310 0x7fa94006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 -- 172.21.15.179:0/3264478490 >> v1:172.21.15.195:6789/0 conn(0x7fa9400ce060 legacy=0x7fa9400ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 -- 172.21.15.179:0/3264478490 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400cdaa0 con 0x7fa9400add40 2024-05-08T22:34:53.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa93f7fe700 1 -- 172.21.15.179:0/3264478490 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498653086 0 0) 0x7fa940161ac0 con 0x7fa9400ce060 2024-05-08T22:34:53.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa9453ab700 1 --2- 172.21.15.179:0/3264478490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400add40 0x7fa94006b8a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa940073c10 tx=0x7fa930009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbf874e5b9e1e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa93f7fe700 1 -- 172.21.15.179:0/3264478490 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930011830 con 0x7fa9400add40 2024-05-08T22:34:53.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa93f7fe700 1 -- 172.21.15.179:0/3264478490 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930011990 con 0x7fa9400add40 2024-05-08T22:34:53.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa93f7fe700 1 -- 172.21.15.179:0/3264478490 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930011d20 con 0x7fa9400add40 2024-05-08T22:34:53.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- 172.21.15.179:0/3264478490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400add40 msgr2=0x7fa94006b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 --2- 172.21.15.179:0/3264478490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400add40 0x7fa94006b8a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa940073c10 tx=0x7fa930009bf0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- 172.21.15.179:0/3264478490 shutdown_connections 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 --2- 172.21.15.179:0/3264478490 >> v2:172.21.15.195:3300/0 conn(0x7fa9400cc310 0x7fa94006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 --2- 172.21.15.179:0/3264478490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400add40 0x7fa94006b8a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- 172.21.15.179:0/3264478490 >> 172.21.15.179:0/3264478490 conn(0x7fa9400cb660 msgr2=0x7fa9400cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- 172.21.15.179:0/3264478490 shutdown_connections 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.045+0000 7fa946e0e700 1 -- 172.21.15.179:0/3264478490 wait complete. 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 Processor -- start 2024-05-08T22:34:53.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- start start 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9400ce060 0x7fa9400c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b3530 con 0x7fa9400cc310 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b30a0 con 0x7fa9400add40 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b2490 con 0x7fa9400ce060 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa944baa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa944baa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52188/0 (socket says 172.21.15.179:52188) 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35470/0 (socket says 172.21.15.179:35470) 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa944baa700 1 -- 172.21.15.179:0/1935152762 learned_addr learned my addr 172.21.15.179:0/1935152762 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa9453ab700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9400ce060 0x7fa9400c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 -- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9400ce060 msgr2=0x7fa9400c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9400ce060 0x7fa9400c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 -- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 msgr2=0x7fa94015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 -- 172.21.15.179:0/1935152762 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009570 con 0x7fa9400cc310 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa944baa700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93ffff700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa93400e900 tx=0x7fa93400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934013680 con 0x7fa9400cc310 2024-05-08T22:34:53.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340137e0 con 0x7fa9400cc310 2024-05-08T22:34:53.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9400b2770 con 0x7fa9400cc310 2024-05-08T22:34:53.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934013b50 con 0x7fa9400cc310 2024-05-08T22:34:53.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.049+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9400b2d30 con 0x7fa9400cc310 2024-05-08T22:34:53.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.053+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940067770 con 0x7fa9400cc310 2024-05-08T22:34:53.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.053+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa934013cb0 con 0x7fa9400cc310 2024-05-08T22:34:53.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.053+0000 7fa93d7fa700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 0x7fa92c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.053+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa9340841e0 con 0x7fa9400cc310 2024-05-08T22:34:53.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.053+0000 7fa944baa700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 0x7fa92c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.057+0000 7fa944baa700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 0x7fa92c068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa938002230 tx=0x7fa938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.065+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa934051420 con 0x7fa9400cc310 2024-05-08T22:34:53.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.237+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9400cd7a0 con 0x7fa9400cc310 2024-05-08T22:34:53.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa93d7fa700 1 -- 172.21.15.179:0/1935152762 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16354 (secure 0 0 0) 0x7fa9400cd7a0 con 0x7fa9400cc310 2024-05-08T22:34:53.242 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:34:53.242 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":41,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:34:42.384705+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 msgr2=0x7fa92c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 0x7fa92c068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa938002230 tx=0x7fa938009040 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 msgr2=0x7fa9400c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa93400e900 tx=0x7fa93400ec10 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 shutdown_connections 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa92c066730 0x7fa92c068bf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9400ce060 0x7fa9400c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9400cc310 0x7fa9400c6d00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 --2- 172.21.15.179:0/1935152762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9400add40 0x7fa94015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 >> 172.21.15.179:0/1935152762 conn(0x7fa9400cb660 msgr2=0x7fa9400af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 shutdown_connections 2024-05-08T22:34:53.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.241+0000 7fa946e0e700 1 -- 172.21.15.179:0/1935152762 wait complete. 2024-05-08T22:34:53.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:34:39.249198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:34:40.358534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:34:53.261 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.377+0000 7f14c3fff700 1 Processor -- start 2024-05-08T22:34:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.377+0000 7f14c3fff700 1 -- start start 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 0x7f14c40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c40b8ee0 con 0x7f14c415d0c0 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40ce570 con 0x7f14c40b8a80 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c415d4a0 con 0x7f14c40ce0e0 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c2ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f14c415d0c0 0x7f14c40d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39098/0 (socket says 172.21.15.179:39098) 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c2ffd700 1 -- 172.21.15.179:0/3292076346 learned_addr learned my addr 172.21.15.179:0/3292076346 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3292076346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c27fc700 1 --2- 172.21.15.179:0/3292076346 >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 0x7f14c40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c1ffb700 1 -- 172.21.15.179:0/3292076346 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438717054 0 0) 0x7f14c40b8ee0 con 0x7f14c415d0c0 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c1ffb700 1 -- 172.21.15.179:0/3292076346 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b00035d0 con 0x7f14c415d0c0 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 -- 172.21.15.179:0/3292076346 >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 msgr2=0x7f14c40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3292076346 >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 0x7f14c40cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 -- 172.21.15.179:0/3292076346 >> v1:172.21.15.179:6789/0 conn(0x7f14c415d0c0 legacy=0x7f14c40d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 -- 172.21.15.179:0/3292076346 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c406abb0 con 0x7f14c40ce0e0 2024-05-08T22:34:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c27fc700 1 --2- 172.21.15.179:0/3292076346 >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 0x7f14c40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3292076346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c406b370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009d90 tx=0x7f14bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff4a8d3ff1d86c80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c1ffb700 1 -- 172.21.15.179:0/3292076346 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00e070 con 0x7f14c40ce0e0 2024-05-08T22:34:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c1ffb700 1 -- 172.21.15.179:0/3292076346 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc00bb20 con 0x7f14c40ce0e0 2024-05-08T22:34:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.381+0000 7f14c1ffb700 1 -- 172.21.15.179:0/3292076346 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc012430 con 0x7f14c40ce0e0 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- 172.21.15.179:0/3292076346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 msgr2=0x7f14c406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3292076346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c406b370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009d90 tx=0x7f14bc00b330 comp rx=0 tx=0).stop 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- 172.21.15.179:0/3292076346 shutdown_connections 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3292076346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c406b370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3292076346 >> v2:172.21.15.195:3300/0 conn(0x7f14c40b8a80 0x7f14c40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- 172.21.15.179:0/3292076346 >> 172.21.15.179:0/3292076346 conn(0x7f14c40d2ed0 msgr2=0x7f14c40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- 172.21.15.179:0/3292076346 shutdown_connections 2024-05-08T22:34:53.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- 172.21.15.179:0/3292076346 wait complete. 2024-05-08T22:34:53.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 Processor -- start 2024-05-08T22:34:53.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.385+0000 7f14c3fff700 1 -- start start 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c40b8a80 0x7f14c4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40c8b90 con 0x7f14c415d0c0 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40c88b0 con 0x7f14c40b8a80 2024-05-08T22:34:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40ca040 con 0x7f14c40ce0e0 2024-05-08T22:34:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35482/0 (socket says 172.21.15.179:35482) 2024-05-08T22:34:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 -- 172.21.15.179:0/3385176734 learned_addr learned my addr 172.21.15.179:0/3385176734 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c27fc700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c2ffd700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c40b8a80 0x7f14c4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 -- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 msgr2=0x7f14c415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 -- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c40b8a80 msgr2=0x7f14c4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c40b8a80 0x7f14c4152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 -- 172.21.15.179:0/3385176734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0094b0 con 0x7f14c415d0c0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c37fe700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14bc014040 tx=0x7f14bc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc012430 con 0x7f14c415d0c0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc020070 con 0x7f14c415d0c0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc009a70 con 0x7f14c415d0c0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc003e10 con 0x7f14c415d0c0 2024-05-08T22:34:53.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.389+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c40ca600 con 0x7f14c415d0c0 2024-05-08T22:34:53.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c404e700 con 0x7f14c415d0c0 2024-05-08T22:34:53.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14bc004040 con 0x7f14c415d0c0 2024-05-08T22:34:53.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14bb7fe700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 0x7f14a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f14bc022070 con 0x7f14c415d0c0 2024-05-08T22:34:53.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14c2ffd700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 0x7f14a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.393+0000 7f14c2ffd700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 0x7f14a4068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14b4007fd0 tx=0x7f14b4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.397+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f14bc0537d0 con 0x7f14c415d0c0 2024-05-08T22:34:53.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14c40676e0 con 0x7f14c415d0c0 2024-05-08T22:34:53.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14bb7fe700 1 -- 172.21.15.179:0/3385176734 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14bc056df0 con 0x7f14c415d0c0 2024-05-08T22:34:53.571 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 msgr2=0x7f14a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 0x7f14a4068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14b4007fd0 tx=0x7f14b4007de0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 msgr2=0x7f14c414e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.569+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14bc014040 tx=0x7f14bc004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 shutdown_connections 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f14a4066730 0x7f14a4068bf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c415d0c0 0x7f14c414e360 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14c40ce0e0 0x7f14c415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 --2- 172.21.15.179:0/3385176734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c40b8a80 0x7f14c4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 >> 172.21.15.179:0/3385176734 conn(0x7f14c40d2ed0 msgr2=0x7f14c40b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 shutdown_connections 2024-05-08T22:34:53.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.573+0000 7f14c3fff700 1 -- 172.21.15.179:0/3385176734 wait complete. 2024-05-08T22:34:53.586 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:34:53.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 Processor -- start 2024-05-08T22:34:53.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 -- start start 2024-05-08T22:34:53.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f943c06d4b0 0x7f943c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0b22d0 con 0x7f943c142fd0 2024-05-08T22:34:53.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0b19e0 con 0x7f943c142890 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9441633700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c15e640 con 0x7f943c06d4b0 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9433fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f943c142fd0 0x7f943c06cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39110/0 (socket says 172.21.15.179:39110) 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9433fff700 1 -- 172.21.15.179:0/1488203328 learned_addr learned my addr 172.21.15.179:0/1488203328 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f9440e32700 1 --2- 172.21.15.179:0/1488203328 >> v2:172.21.15.195:3300/0 conn(0x7f943c06d4b0 0x7f943c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866831799 0 0) 0x7f943c0b22d0 con 0x7f943c142fd0 2024-05-08T22:34:53.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94280035d0 con 0x7f943c142fd0 2024-05-08T22:34:53.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 657243418 0 0) 0x7f94280035d0 con 0x7f943c142fd0 2024-05-08T22:34:53.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 >> v2:172.21.15.195:3300/0 conn(0x7f943c06d4b0 msgr2=0x7f943c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 --2- 172.21.15.179:0/1488203328 >> v2:172.21.15.195:3300/0 conn(0x7f943c06d4b0 0x7f943c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 >> v1:172.21.15.195:6789/0 conn(0x7f943c142890 legacy=0x7f943c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0b1c80 con 0x7f943c142fd0 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9424001670 con 0x7f943c142fd0 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 >> v1:172.21.15.179:6789/0 conn(0x7f943c142fd0 legacy=0x7f943c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428003a20 0x7f9428003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9428004870 0x7f9428004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 0x7f942800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c15e640 con 0x7f9428003a20 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942800c220 con 0x7f9428005350 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.701+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942800c3e0 con 0x7f9428004870 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f943bfff700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428003a20 0x7f9428003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 0x7f942800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9433fff700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9428004870 0x7f9428004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 -- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9428004870 msgr2=0x7f9428004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9428004870 0x7f9428004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 -- 172.21.15.179:0/1488203328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428003a20 msgr2=0x7f9428003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428003a20 0x7f9428003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 -- 172.21.15.179:0/1488203328 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280035d0 con 0x7f9428005350 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 -- 172.21.15.179:0/1488203328 reap_dead start 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9440e32700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 0x7f942800ba30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9434009f90 tx=0x7f94280058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6bb1360ed00f541 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434003a40 con 0x7f9428005350 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400f040 con 0x7f9428005350 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f943b7fe700 1 -- 172.21.15.179:0/1488203328 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434003de0 con 0x7f9428005350 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 -- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 msgr2=0x7f942800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 0x7f942800ba30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9434009f90 tx=0x7f94280058a0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 -- 172.21.15.179:0/1488203328 shutdown_connections 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 --2- 172.21.15.179:0/1488203328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9428005350 0x7f942800ba30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 -- 172.21.15.179:0/1488203328 >> 172.21.15.179:0/1488203328 conn(0x7f943c0cea30 msgr2=0x7f943c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 -- 172.21.15.179:0/1488203328 shutdown_connections 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.705+0000 7f9441633700 1 -- 172.21.15.179:0/1488203328 wait complete. 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 Processor -- start 2024-05-08T22:34:53.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- start start 2024-05-08T22:34:53.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 0x7f943c130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 0x7f943c131000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c12da60 con 0x7f9428005350 2024-05-08T22:34:53.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c12dd40 con 0x7f943c14e120 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c1439c0 con 0x7f943c130b80 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35492/0 (socket says 172.21.15.179:35492) 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 -- 172.21.15.179:0/3985565288 learned_addr learned my addr 172.21.15.179:0/3985565288 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9440e32700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 0x7f943c131000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 -- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 msgr2=0x7f943c131000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 0x7f943c131000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9433fff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 0x7f943c130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 -- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 msgr2=0x7f943c130640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 0x7f943c130640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 -- 172.21.15.179:0/3985565288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c143ca0 con 0x7f9428005350 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9440e32700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 0x7f943c131000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9433fff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 0x7f943c130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f943bfff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f942c00f7f0 tx=0x7f942c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c005320 con 0x7f9428005350 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c005480 con 0x7f9428005350 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c006aa0 con 0x7f9428005350 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c143f80 con 0x7f9428005350 2024-05-08T22:34:53.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.709+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c12f510 con 0x7f9428005350 2024-05-08T22:34:53.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c144110 con 0x7f9428005350 2024-05-08T22:34:53.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f942c006c00 con 0x7f9428005350 2024-05-08T22:34:53.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f94397fa700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 0x7f941c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f942c083600 con 0x7f9428005350 2024-05-08T22:34:53.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f9433fff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 0x7f941c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.713+0000 7f9433fff700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 0x7f941c068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f943c12d470 tx=0x7f9424009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.717+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f942c050970 con 0x7f9428005350 2024-05-08T22:34:53.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.885+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f943c04e700 con 0x7f9428005350 2024-05-08T22:34:53.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.885+0000 7f94397fa700 1 -- 172.21.15.179:0/3985565288 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f942c053f90 con 0x7f9428005350 2024-05-08T22:34:53.889 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:34:53.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 msgr2=0x7f941c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 0x7f941c068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f943c12d470 tx=0x7f9424009040 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 msgr2=0x7f943c159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f942c00f7f0 tx=0x7f942c00fbb0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 shutdown_connections 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f941c066730 0x7f941c068bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f943c14e120 0x7f943c130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f943c130b80 0x7f943c131000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 --2- 172.21.15.179:0/3985565288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9428005350 0x7f943c159890 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 >> 172.21.15.179:0/3985565288 conn(0x7f943c0cea30 msgr2=0x7f943c0aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 shutdown_connections 2024-05-08T22:34:53.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:53.889+0000 7f9441633700 1 -- 172.21.15.179:0/3985565288 wait complete. 2024-05-08T22:34:53.908 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:34:54.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 Processor -- start 2024-05-08T22:34:54.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 -- start start 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb28014ae30 0x7fb280165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb2800b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb280166090 con 0x7fb2800b2c10 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28006e9d0 con 0x7fb28014ae30 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb2854a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800b30c0 con 0x7fb2800b1a50 2024-05-08T22:34:54.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb284c9f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb2800b2c10 0x7fb280161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39114/0 (socket says 172.21.15.179:39114) 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb284c9f700 1 -- 172.21.15.179:0/1574176877 learned_addr learned my addr 172.21.15.179:0/1574176877 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 --2- 172.21.15.179:0/1574176877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb2800b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27ffff700 1 --2- 172.21.15.179:0/1574176877 >> v2:172.21.15.195:3300/0 conn(0x7fb28014ae30 0x7fb280165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27effd700 1 -- 172.21.15.179:0/1574176877 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380985989 0 0) 0x7fb280166090 con 0x7fb2800b2c10 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27effd700 1 -- 172.21.15.179:0/1574176877 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c0035d0 con 0x7fb2800b2c10 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 -- 172.21.15.179:0/1574176877 >> v2:172.21.15.195:3300/0 conn(0x7fb28014ae30 msgr2=0x7fb280165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 --2- 172.21.15.179:0/1574176877 >> v2:172.21.15.195:3300/0 conn(0x7fb28014ae30 0x7fb280165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 -- 172.21.15.179:0/1574176877 >> v1:172.21.15.179:6789/0 conn(0x7fb2800b2c10 legacy=0x7fb280161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 -- 172.21.15.179:0/1574176877 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28006dd50 con 0x7fb2800b1a50 2024-05-08T22:34:54.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27f7fe700 1 --2- 172.21.15.179:0/1574176877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb2800b26d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb268009850 tx=0x7fb268009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4740eeccd8826 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27effd700 1 -- 172.21.15.179:0/1574176877 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268011830 con 0x7fb2800b1a50 2024-05-08T22:34:54.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27effd700 1 -- 172.21.15.179:0/1574176877 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268011990 con 0x7fb2800b1a50 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.021+0000 7fb27effd700 1 -- 172.21.15.179:0/1574176877 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268011d00 con 0x7fb2800b1a50 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- 172.21.15.179:0/1574176877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 msgr2=0x7fb2800b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 --2- 172.21.15.179:0/1574176877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb2800b26d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb268009850 tx=0x7fb268009b60 comp rx=0 tx=0).stop 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- 172.21.15.179:0/1574176877 shutdown_connections 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 --2- 172.21.15.179:0/1574176877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb2800b26d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 --2- 172.21.15.179:0/1574176877 >> v2:172.21.15.195:3300/0 conn(0x7fb28014ae30 0x7fb280165b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- 172.21.15.179:0/1574176877 >> 172.21.15.179:0/1574176877 conn(0x7fb2800cea30 msgr2=0x7fb2800cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- 172.21.15.179:0/1574176877 shutdown_connections 2024-05-08T22:34:54.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- 172.21.15.179:0/1574176877 wait complete. 2024-05-08T22:34:54.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 Processor -- start 2024-05-08T22:34:54.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.025+0000 7fb2854a0700 1 -- start start 2024-05-08T22:34:54.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb280141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2800b2c10 0x7fb280160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280145c90 con 0x7fb28014ae30 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280144990 con 0x7fb2800b2c10 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb280144c70 con 0x7fb2800b1a50 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb280141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35494/0 (socket says 172.21.15.179:35494) 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 -- 172.21.15.179:0/489702400 learned_addr learned my addr 172.21.15.179:0/489702400 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27f7fe700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2800b2c10 0x7fb280160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 -- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 msgr2=0x7fb280141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb280141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 -- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2800b2c10 msgr2=0x7fb280160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2800b2c10 0x7fb280160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 -- 172.21.15.179:0/489702400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2680094b0 con 0x7fb28014ae30 2024-05-08T22:34:54.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27ffff700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb280141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb284c9f700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb27800ca20 tx=0x7fb27800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278007480 con 0x7fb28014ae30 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2780075e0 con 0x7fb28014ae30 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb280144f50 con 0x7fb28014ae30 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278007970 con 0x7fb28014ae30 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.029+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb280141a00 con 0x7fb28014ae30 2024-05-08T22:34:54.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb280145470 con 0x7fb28014ae30 2024-05-08T22:34:54.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb27801b3b0 con 0x7fb28014ae30 2024-05-08T22:34:54.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb27cff9700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 0x7fb260068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb278083a80 con 0x7fb28014ae30 2024-05-08T22:34:54.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb27ffff700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 0x7fb260068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.033+0000 7fb27ffff700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 0x7fb260068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb2800ce300 tx=0x7fb270009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.041+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb280145470 con 0x7fb28014ae30 2024-05-08T22:34:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.205+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb280145470 con 0x7fb28014ae30 2024-05-08T22:34:54.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.205+0000 7fb27cff9700 1 -- 172.21.15.179:0/489702400 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb280145470 con 0x7fb28014ae30 2024-05-08T22:34:54.207 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:34:54.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 msgr2=0x7fb260068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 0x7fb260068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb2800ce300 tx=0x7fb270009240 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 msgr2=0x7fb280145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb27800ca20 tx=0x7fb27800ca50 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 shutdown_connections 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb260066730 0x7fb260068bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2800b2c10 0x7fb280160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb28014ae30 0x7fb280145750 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 --2- 172.21.15.179:0/489702400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2800b1a50 0x7fb280141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 >> 172.21.15.179:0/489702400 conn(0x7fb2800cea30 msgr2=0x7fb2801479b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 shutdown_connections 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.209+0000 7fb2854a0700 1 -- 172.21.15.179:0/489702400 wait complete. 2024-05-08T22:34:54.228 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:34:54.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.337+0000 7f7aa213f700 1 Processor -- start 2024-05-08T22:34:54.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.337+0000 7f7aa213f700 1 -- start start 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a9c15d340 0x7f7a9c160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c0cd6d0 con 0x7f7a9c15cf60 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c161b30 con 0x7f7a9c15d340 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0cd970 con 0x7f7a9c0b4f70 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a9c15d340 0x7f7a9c160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a9c15d340 0x7f7a9c160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35508/0 (socket says 172.21.15.179:35508) 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 -- 172.21.15.179:0/2208160261 learned_addr learned my addr 172.21.15.179:0/2208160261 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:54.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa093c700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 -- 172.21.15.179:0/2208160261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b4f70 msgr2=0x7f7a9c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 -- 172.21.15.179:0/2208160261 >> v1:172.21.15.195:6789/0 conn(0x7f7a9c15cf60 legacy=0x7f7a9c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 -- 172.21.15.179:0/2208160261 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0cdc50 con 0x7f7a9c15d340 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa193e700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15d340 0x7f7a9c160e00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba74c9dba9274b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7a93fff700 1 -- 172.21.15.179:0/2208160261 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400e070 con 0x7f7a9c15d340 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7a93fff700 1 -- 172.21.15.179:0/2208160261 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9400b910 con 0x7f7a9c15d340 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7a93fff700 1 -- 172.21.15.179:0/2208160261 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400bcc0 con 0x7f7a9c15d340 2024-05-08T22:34:54.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- 172.21.15.179:0/2208160261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15d340 msgr2=0x7f7a9c160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15d340 0x7f7a9c160e00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9400b330 comp rx=0 tx=0).stop 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- 172.21.15.179:0/2208160261 shutdown_connections 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 --2- 172.21.15.179:0/2208160261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15d340 0x7f7a9c160e00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- 172.21.15.179:0/2208160261 >> 172.21.15.179:0/2208160261 conn(0x7f7a9c0cf860 msgr2=0x7f7a9c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- 172.21.15.179:0/2208160261 shutdown_connections 2024-05-08T22:34:54.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.341+0000 7f7aa213f700 1 -- 172.21.15.179:0/2208160261 wait complete. 2024-05-08T22:34:54.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 Processor -- start 2024-05-08T22:34:54.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 -- start start 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 0x7f7a9c15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0aa690 con 0x7f7a9c15cf60 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0cd4a0 con 0x7f7a9c0b4f70 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa213f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0ca450 con 0x7f7a9c15d340 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa093c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa113d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa093c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35510/0 (socket says 172.21.15.179:35510) 2024-05-08T22:34:54.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 0x7f7a9c15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa113d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:52292/0 (socket says 172.21.15.179:52292) 2024-05-08T22:34:54.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa093c700 1 -- 172.21.15.179:0/4064540496 learned_addr learned my addr 172.21.15.179:0/4064540496 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:34:54.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 -- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 msgr2=0x7f7a9c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 -- 172.21.15.179:0/4064540496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 msgr2=0x7f7a9c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 -- 172.21.15.179:0/4064540496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009b30 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa113d700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa093c700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7aa193e700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 0x7f7a9c15edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a94015000 tx=0x7f7a94004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94004800 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.345+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940092e0 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c0ca730 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9401b650 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c0cacf0 con 0x7f7a9c15d340 2024-05-08T22:34:54.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c067250 con 0x7f7a9c15d340 2024-05-08T22:34:54.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a9401b7b0 con 0x7f7a9c15d340 2024-05-08T22:34:54.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.349+0000 7f7a91ffb700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 0x7f7a80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:54.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.353+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7a94085a50 con 0x7f7a9c15d340 2024-05-08T22:34:54.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.353+0000 7f7aa113d700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 0x7f7a80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:54.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.353+0000 7f7aa113d700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 0x7f7a80068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000f60 tx=0x7f7a98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:54.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.361+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7a94052f10 con 0x7f7a9c15d340 2024-05-08T22:34:54.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.549+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a9c04e700 con 0x7f7a9c15d340 2024-05-08T22:34:54.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.549+0000 7f7a91ffb700 1 -- 172.21.15.179:0/4064540496 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7a94056530 con 0x7f7a9c15d340 2024-05-08T22:34:54.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 msgr2=0x7f7a80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 0x7f7a80068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000f60 tx=0x7f7a98009040 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 msgr2=0x7f7a9c15edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 0x7f7a9c15edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a94015000 tx=0x7f7a94004af0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 shutdown_connections 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7a80066730 0x7f7a80068bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c15d340 0x7f7a9c15edf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9c15cf60 0x7f7a9c15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 --2- 172.21.15.179:0/4064540496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a9c0b4f70 0x7f7a9c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 >> 172.21.15.179:0/4064540496 conn(0x7f7a9c0cf860 msgr2=0x7f7a9c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 shutdown_connections 2024-05-08T22:34:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:34:54.553+0000 7f7aa213f700 1 -- 172.21.15.179:0/4064540496 wait complete. 2024-05-08T22:34:54.558 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:34:54.571 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:34:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:34:54.571 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:34:54.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[37163]: fuse finished with error 0 and tester_r 0 2024-05-08T22:34:54.584 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:35:00.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:35:00.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:00.686 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:35:00.695 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:00.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.813+0000 7f0c4359e700 1 Processor -- start 2024-05-08T22:35:00.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.813+0000 7f0c4359e700 1 -- start start 2024-05-08T22:35:00.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c441447c0 0x7f0c440ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c440cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 0x7f0c4415e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440b5500 con 0x7f0c441447c0 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440b4620 con 0x7f0c440d0480 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b48f0 con 0x7f0c440b41c0 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c441447c0 0x7f0c440ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c441447c0 0x7f0c440ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37380/0 (socket says 172.21.15.179:37380) 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 -- 172.21.15.179:0/137299905 learned_addr learned my addr 172.21.15.179:0/137299905 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c41d9b700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c440cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c42d9d700 1 --2- 172.21.15.179:0/137299905 >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 0x7f0c4415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 -- 172.21.15.179:0/137299905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 msgr2=0x7f0c440cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c440cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 -- 172.21.15.179:0/137299905 >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 msgr2=0x7f0c4415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 --2- 172.21.15.179:0/137299905 >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 0x7f0c4415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 -- 172.21.15.179:0/137299905 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440b4bd0 con 0x7f0c441447c0 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c42d9d700 1 --2- 172.21.15.179:0/137299905 >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 0x7f0c4415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4259c700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c441447c0 0x7f0c440ce700 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0c4415b370 tx=0x7f0c34009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecdac93c6c558af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4159a700 1 -- 172.21.15.179:0/137299905 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400b8a0 con 0x7f0c441447c0 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4159a700 1 -- 172.21.15.179:0/137299905 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400ba00 con 0x7f0c441447c0 2024-05-08T22:35:00.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4159a700 1 -- 172.21.15.179:0/137299905 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400bd90 con 0x7f0c441447c0 2024-05-08T22:35:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- 172.21.15.179:0/137299905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c441447c0 msgr2=0x7f0c440ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c441447c0 0x7f0c440ce700 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0c4415b370 tx=0x7f0c34009bf0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- 172.21.15.179:0/137299905 shutdown_connections 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- 172.21.15.179:0/137299905 >> v2:172.21.15.195:3300/0 conn(0x7f0c440d0480 0x7f0c4415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c440cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 --2- 172.21.15.179:0/137299905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c441447c0 0x7f0c440ce700 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- 172.21.15.179:0/137299905 >> 172.21.15.179:0/137299905 conn(0x7f0c440b2840 msgr2=0x7f0c440b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- 172.21.15.179:0/137299905 shutdown_connections 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.817+0000 7f0c4359e700 1 -- 172.21.15.179:0/137299905 wait complete. 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 Processor -- start 2024-05-08T22:35:00.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- start start 2024-05-08T22:35:00.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c4406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 0x7f0c44159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4413ffb0 con 0x7f0c440d0480 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4406fe10 con 0x7f0c441447c0 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440700f0 con 0x7f0c440b41c0 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37394/0 (socket says 172.21.15.179:37394) 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 -- 172.21.15.179:0/427922916 learned_addr learned my addr 172.21.15.179:0/427922916 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4259c700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c4406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c42d9d700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 0x7f0c44159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 -- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 msgr2=0x7f0c4406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c4406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 -- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 msgr2=0x7f0c44159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 0x7f0c44159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 -- 172.21.15.179:0/427922916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34009570 con 0x7f0c440d0480 2024-05-08T22:35:00.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c42d9d700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 0x7f0c44159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c41d9b700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d800 tx=0x7f0c2c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c012630 con 0x7f0c440d0480 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0127b0 con 0x7f0c440d0480 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440703d0 con 0x7f0c440d0480 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c012b60 con 0x7f0c440d0480 2024-05-08T22:35:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.821+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4406d7d0 con 0x7f0c440d0480 2024-05-08T22:35:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4404e700 con 0x7f0c440d0480 2024-05-08T22:35:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c2c012cc0 con 0x7f0c440d0480 2024-05-08T22:35:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c3affd700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 0x7f0c28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0c2c084600 con 0x7f0c440d0480 2024-05-08T22:35:00.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c4259c700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 0x7f0c28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:00.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.825+0000 7f0c4259c700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 0x7f0c28068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000c00 tx=0x7f0c34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:00.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:00.833+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0c2c051830 con 0x7f0c440d0480 2024-05-08T22:35:01.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.041+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c44070560 con 0x7f0c440d0480 2024-05-08T22:35:01.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.041+0000 7f0c3affd700 1 -- 172.21.15.179:0/427922916 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f0c2c054e50 con 0x7f0c440d0480 2024-05-08T22:35:01.044 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:01.044 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:01.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 msgr2=0x7f0c28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 0x7f0c28068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000c00 tx=0x7f0c34008030 comp rx=0 tx=0).stop 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 msgr2=0x7f0c440cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d800 tx=0x7f0c2c00dbc0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 shutdown_connections 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c28066730 0x7f0c28068bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c441447c0 0x7f0c44159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c440d0480 0x7f0c440cc180 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 --2- 172.21.15.179:0/427922916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c440b41c0 0x7f0c4406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 >> 172.21.15.179:0/427922916 conn(0x7f0c440b2840 msgr2=0x7f0c4415d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 shutdown_connections 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.045+0000 7f0c4359e700 1 -- 172.21.15.179:0/427922916 wait complete. 2024-05-08T22:35:01.050 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:01.065 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-08T22:35:01.190 INFO:teuthology.orchestra.run.smithi179.stdout:{"message":"","return_code":0} 2024-05-08T22:35:01.191 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:35:01.191 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:01.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 Processor -- start 2024-05-08T22:35:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 -- start start 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b4ca0 0x7f503c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f503c0b45b0 0x7f503c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0b5180 con 0x7f503c0b41d0 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c0cf0d0 con 0x7f503c0b45b0 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.293+0000 7f503bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0ceb10 con 0x7f503c0b4ca0 2024-05-08T22:35:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f503c0b45b0 0x7f503c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f503c0b45b0 0x7f503c15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37404/0 (socket says 172.21.15.179:37404) 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 -- 172.21.15.179:0/1923796134 learned_addr learned my addr 172.21.15.179:0/1923796134 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503a7fc700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b4ca0 0x7f503c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 -- 172.21.15.179:0/1923796134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b4ca0 msgr2=0x7f503c15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b4ca0 0x7f503c15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 -- 172.21.15.179:0/1923796134 >> v1:172.21.15.195:6789/0 conn(0x7f503c0b41d0 legacy=0x7f503c0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 -- 172.21.15.179:0/1923796134 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0cedf0 con 0x7f503c0b45b0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503b7fe700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b45b0 0x7f503c15e790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5034009e50 tx=0x7f503400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b266c851aa3a603e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f5039ffb700 1 -- 172.21.15.179:0/1923796134 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400e070 con 0x7f503c0b45b0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f5039ffb700 1 -- 172.21.15.179:0/1923796134 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400b9c0 con 0x7f503c0b45b0 2024-05-08T22:35:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f5039ffb700 1 -- 172.21.15.179:0/1923796134 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400bd70 con 0x7f503c0b45b0 2024-05-08T22:35:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- 172.21.15.179:0/1923796134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b45b0 msgr2=0x7f503c15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b45b0 0x7f503c15e790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5034009e50 tx=0x7f503400b330 comp rx=0 tx=0).stop 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- 172.21.15.179:0/1923796134 shutdown_connections 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b4ca0 0x7f503c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 --2- 172.21.15.179:0/1923796134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b45b0 0x7f503c15e790 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- 172.21.15.179:0/1923796134 >> 172.21.15.179:0/1923796134 conn(0x7f503c098a70 msgr2=0x7f503c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- 172.21.15.179:0/1923796134 shutdown_connections 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- 172.21.15.179:0/1923796134 wait complete. 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 Processor -- start 2024-05-08T22:35:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.297+0000 7f503bfff700 1 -- start start 2024-05-08T22:35:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503c0b41d0 0x7f503c06b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b45b0 0x7f503c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c1f3640 con 0x7f503c0b4ca0 2024-05-08T22:35:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0ae350 con 0x7f503c0b41d0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0ae630 con 0x7f503c0b45b0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37410/0 (socket says 172.21.15.179:37410) 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 -- 172.21.15.179:0/2341089633 learned_addr learned my addr 172.21.15.179:0/2341089633 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503affd700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503c0b41d0 0x7f503c06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503a7fc700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b45b0 0x7f503c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 -- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b45b0 msgr2=0x7f503c1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b45b0 0x7f503c1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 -- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503c0b41d0 msgr2=0x7f503c06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503c0b41d0 0x7f503c06b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 -- 172.21.15.179:0/2341089633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034009b30 con 0x7f503c0b4ca0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503b7fe700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f503c06fea0 tx=0x7f5034004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034004800 con 0x7f503c0b4ca0 2024-05-08T22:35:01.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50340099b0 con 0x7f503c0b4ca0 2024-05-08T22:35:01.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503401c680 con 0x7f503c0b4ca0 2024-05-08T22:35:01.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c0ae910 con 0x7f503c0b4ca0 2024-05-08T22:35:01.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.301+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c0aeed0 con 0x7f503c0b4ca0 2024-05-08T22:35:01.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c04e700 con 0x7f503c0b4ca0 2024-05-08T22:35:01.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f503401c7e0 con 0x7f503c0b4ca0 2024-05-08T22:35:01.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f5023fff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 0x7f5028068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5034085dc0 con 0x7f503c0b4ca0 2024-05-08T22:35:01.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f503affd700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 0x7f5028068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.305+0000 7f503affd700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 0x7f5028068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:01.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.313+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5034053080 con 0x7f503c0b4ca0 2024-05-08T22:35:01.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f503c06d120 con 0x7f503c0b4ca0 2024-05-08T22:35:01.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f5023fff700 1 -- 172.21.15.179:0/2341089633 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f50340566a0 con 0x7f503c0b4ca0 2024-05-08T22:35:01.527 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:01.527 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 msgr2=0x7f5028068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 0x7f5028068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030009040 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 msgr2=0x7f503c1f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f503c06fea0 tx=0x7f5034004af0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 shutdown_connections 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5028066730 0x7f5028068bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f503c0b4ca0 0x7f503c1f3100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f503c0b45b0 0x7f503c1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 --2- 172.21.15.179:0/2341089633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503c0b41d0 0x7f503c06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 >> 172.21.15.179:0/2341089633 conn(0x7f503c098a70 msgr2=0x7f503c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:01.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 shutdown_connections 2024-05-08T22:35:01.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:01.525+0000 7f503bfff700 1 -- 172.21.15.179:0/2341089633 wait complete. 2024-05-08T22:35:01.531 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:01.541 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-05-08T22:35:01.651 INFO:teuthology.orchestra.run.smithi179.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-08T22:35:01.652 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-08T22:35:11.653 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-08T22:35:11.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 Processor -- start 2024-05-08T22:35:11.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 -- start start 2024-05-08T22:35:11.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 0x55ee3c9d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 0x55ee3ca818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c93c2f0 con 0x55ee3ca7e620 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c8f88e0 con 0x55ee3c9ab750 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9c88c140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c8f8580 con 0x55ee3c9ad780 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9abc2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 0x55ee3ca818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9abc2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53794/0 (socket says 172.21.15.179:53794) 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9abc2700 1 -- 172.21.15.179:0/2019593233 learned_addr learned my addr 172.21.15.179:0/2019593233 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9a3c1700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 0x55ee3c9d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 -- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 msgr2=0x55ee3c9d9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 0x55ee3c9d9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 -- 172.21.15.179:0/2019593233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 msgr2=0x55ee3c9abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef99bc0700 1 -- 172.21.15.179:0/2019593233 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c005d20 con 0x55ee3c9ad780 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9a3c1700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 0x55ee3c9d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.693+0000 7fef9abc2700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 0x55ee3ca818b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef84001c10 tx=0x7fef8400b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2251d9364dd0121c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef98bbe700 1 -- 172.21.15.179:0/2019593233 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400bd30 con 0x55ee3c9ad780 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef98bbe700 1 -- 172.21.15.179:0/2019593233 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8400be90 con 0x55ee3c9ad780 2024-05-08T22:35:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef98bbe700 1 -- 172.21.15.179:0/2019593233 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400e6b0 con 0x55ee3c9ad780 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 -- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 msgr2=0x55ee3ca818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 0x55ee3ca818b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef84001c10 tx=0x7fef8400b5d0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 -- 172.21.15.179:0/2019593233 shutdown_connections 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3ca7e620 0x55ee3c9abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3c9ad780 0x55ee3ca818b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2019593233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ab750 0x55ee3c9d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 -- 172.21.15.179:0/2019593233 >> 172.21.15.179:0/2019593233 conn(0x55ee3c9d6110 msgr2=0x55ee3c9f29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 -- 172.21.15.179:0/2019593233 shutdown_connections 2024-05-08T22:35:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.697+0000 7fef9c88c140 1 -- 172.21.15.179:0/2019593233 wait complete. 2024-05-08T22:35:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 Processor -- start 2024-05-08T22:35:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- start start 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 0x55ee3ca78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ad780 0x55ee3c9abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3ca7e620 0x55ee3c9ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c8ce8d0 con 0x55ee3c9ab750 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ca81260 con 0x55ee3c9ad780 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c9f0f10 con 0x55ee3ca7e620 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 0x55ee3ca78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 0x55ee3ca78ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53808/0 (socket says 172.21.15.179:53808) 2024-05-08T22:35:11.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 -- 172.21.15.179:0/2603212467 learned_addr learned my addr 172.21.15.179:0/2603212467 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ad780 0x55ee3c9abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9abc2700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3ca7e620 0x55ee3c9ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 -- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3ca7e620 msgr2=0x55ee3c9ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ee3ca7e620 0x55ee3c9ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 -- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ad780 msgr2=0x55ee3c9abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee3c9ad780 0x55ee3c9abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 -- 172.21.15.179:0/2603212467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3c900ca0 con 0x55ee3c9ab750 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9a3c1700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 0x55ee3ca78ed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fef8c011000 tx=0x7fef8c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef937fe700 1 -- 172.21.15.179:0/2603212467 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c008690 con 0x55ee3c9ab750 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef937fe700 1 -- 172.21.15.179:0/2603212467 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c012820 con 0x55ee3c9ab750 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef94002530 con 0x55ee3c9ab750 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef937fe700 1 -- 172.21.15.179:0/2603212467 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c012bb0 con 0x55ee3c9ab750 2024-05-08T22:35:11.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.701+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee3c8d9bb0 con 0x55ee3c9ab750 2024-05-08T22:35:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef937fe700 1 -- 172.21.15.179:0/2603212467 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef8c012d10 con 0x55ee3c9ab750 2024-05-08T22:35:11.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef937fe700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef80065060 0x7fef80067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef937fe700 1 -- 172.21.15.179:0/2603212467 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fef8c084700 con 0x55ee3c9ab750 2024-05-08T22:35:11.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef80065060 0x7fef80067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ee3ca81e70 0x55ee3c9ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.705+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55ee3c9f29e0 con 0x55ee3ca81e70 2024-05-08T22:35:11.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9abc2700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ee3ca81e70 0x55ee3c9ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef99bc0700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef80065060 0x7fef80067520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef84011040 tx=0x7fef84009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9abc2700 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ee3ca81e70 0x55ee3c9ef380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9abc2700 1 -- 172.21.15.179:0/2603212467 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fef94002be0 con 0x55ee3ca81e70 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ee3ca81e70 msgr2=0x55ee3c9ef380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ee3ca81e70 0x55ee3c9ef380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef80065060 msgr2=0x7fef80067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef80065060 0x7fef80067520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef84011040 tx=0x7fef84009000 comp rx=0 tx=0).stop 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 msgr2=0x55ee3ca78ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 --2- 172.21.15.179:0/2603212467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3c9ab750 0x55ee3ca78ed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fef8c011000 tx=0x7fef8c002fb0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9abc2700 1 -- 172.21.15.179:0/2603212467 reap_dead start 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 shutdown_connections 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 >> 172.21.15.179:0/2603212467 conn(0x55ee3c9d6110 msgr2=0x55ee3c9d64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 shutdown_connections 2024-05-08T22:35:11.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.709+0000 7fef9c88c140 1 -- 172.21.15.179:0/2603212467 wait complete. 2024-05-08T22:35:11.715 INFO:teuthology.orchestra.run.smithi179.stdout:mytag 2024-05-08T22:35:11.715 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-08T22:35:11.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.753+0000 7fce68356140 1 Processor -- start 2024-05-08T22:35:11.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.753+0000 7fce68356140 1 -- start start 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.753+0000 7fce68356140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f0e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.753+0000 7fce68356140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.753+0000 7fce68356140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190f030a00 con 0x56190f173ab0 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190f02c2f0 con 0x56190f098ac0 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190efe88e0 con 0x56190f170850 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53818/0 (socket says 172.21.15.179:53818) 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 -- 172.21.15.179:0/171122840 learned_addr learned my addr 172.21.15.179:0/171122840 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce65e8b700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f0e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6568a700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 -- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 msgr2=0x56190f0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 -- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 msgr2=0x56190f0e35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f0e35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 -- 172.21.15.179:0/171122840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50005d20 con 0x56190f173ab0 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6568a700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce6668c700 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fce60006ea0 tx=0x7fce60001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6077cda7332469 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce5ffff700 1 -- 172.21.15.179:0/171122840 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce600115f0 con 0x56190f173ab0 2024-05-08T22:35:11.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce5ffff700 1 -- 172.21.15.179:0/171122840 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60011750 con 0x56190f173ab0 2024-05-08T22:35:11.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce5ffff700 1 -- 172.21.15.179:0/171122840 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60011ae0 con 0x56190f173ab0 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- 172.21.15.179:0/171122840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 msgr2=0x56190f0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fce60006ea0 tx=0x7fce60001890 comp rx=0 tx=0).stop 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- 172.21.15.179:0/171122840 shutdown_connections 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c9ba0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 --2- 172.21.15.179:0/171122840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f0e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- 172.21.15.179:0/171122840 >> 172.21.15.179:0/171122840 conn(0x56190f0c6110 msgr2=0x56190f09cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- 172.21.15.179:0/171122840 shutdown_connections 2024-05-08T22:35:11.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.757+0000 7fce68356140 1 -- 172.21.15.179:0/171122840 wait complete. 2024-05-08T22:35:11.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 Processor -- start 2024-05-08T22:35:11.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- start start 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f1fe240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190efe8580 con 0x56190f173ab0 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190efbe8d0 con 0x56190f098ac0 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190f1595f0 con 0x56190f170850 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53824/0 (socket says 172.21.15.179:53824) 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 -- 172.21.15.179:0/2307803211 learned_addr learned my addr 172.21.15.179:0/2307803211 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6568a700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce65e8b700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f1fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 -- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 msgr2=0x56190f156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 -- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 msgr2=0x56190f1fe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f1fe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 -- 172.21.15.179:0/2307803211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190efc9bb0 con 0x56190f173ab0 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6568a700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56190f170850 0x56190f156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce65e8b700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56190f098ac0 0x56190f1fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce6668c700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c8c10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fce60013040 tx=0x7fce60001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce5effd700 1 -- 172.21.15.179:0/2307803211 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6001d860 con 0x56190f173ab0 2024-05-08T22:35:11.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce5effd700 1 -- 172.21.15.179:0/2307803211 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6001d9c0 con 0x56190f173ab0 2024-05-08T22:35:11.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56190efc9f40 con 0x56190f173ab0 2024-05-08T22:35:11.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce5effd700 1 -- 172.21.15.179:0/2307803211 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6001dd50 con 0x56190f173ab0 2024-05-08T22:35:11.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.761+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56190efc99a0 con 0x56190f173ab0 2024-05-08T22:35:11.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce5effd700 1 -- 172.21.15.179:0/2307803211 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce6001c3b0 con 0x56190f173ab0 2024-05-08T22:35:11.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce5effd700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce48065090 0x7fce48067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce5effd700 1 -- 172.21.15.179:0/2307803211 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce60084ab0 con 0x56190f173ab0 2024-05-08T22:35:11.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce68356140 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x56190f159a20 0x56190f159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce65e8b700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce48065090 0x7fce48067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x56190f15a370 con 0x56190f159a20 2024-05-08T22:35:11.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.765+0000 7fce6568a700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x56190f159a20 0x56190f159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce6568a700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x56190f159a20 0x56190f159e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce65e8b700 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce48065090 0x7fce48067550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce58001ed0 tx=0x7fce58001770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce6568a700 1 -- 172.21.15.179:0/2307803211 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fce5000a040 con 0x56190f159a20 2024-05-08T22:35:11.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x56190f159a20 msgr2=0x56190f159e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x56190f159a20 0x56190f159e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce48065090 msgr2=0x7fce48067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce48065090 0x7fce48067550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce58001ed0 tx=0x7fce58001770 comp rx=0 tx=0).stop 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 msgr2=0x56190f0c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 --2- 172.21.15.179:0/2307803211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56190f173ab0 0x56190f0c8c10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fce60013040 tx=0x7fce60001250 comp rx=0 tx=0).stop 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce6668c700 1 -- 172.21.15.179:0/2307803211 reap_dead start 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 shutdown_connections 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 >> 172.21.15.179:0/2307803211 conn(0x56190f0c6110 msgr2=0x56190f0ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 shutdown_connections 2024-05-08T22:35:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.769+0000 7fce68356140 1 -- 172.21.15.179:0/2307803211 wait complete. 2024-05-08T22:35:11.776 INFO:teuthology.orchestra.run.smithi179.stdout:mytag 2024-05-08T22:35:11.776 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.813+0000 7fa704389140 1 Processor -- start 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.813+0000 7fa704389140 1 -- start start 2024-05-08T22:35:11.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 0x563cff08f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfef4e2f0 con 0x563cfefee8b0 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfef0a8e0 con 0x563cfefbb1e0 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cfef0a580 con 0x563cff092700 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa7016bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 0x563cff08f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa7016bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53838/0 (socket says 172.21.15.179:53838) 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa7016bd700 1 -- 172.21.15.179:0/360736301 learned_addr learned my addr 172.21.15.179:0/360736301 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa7026bf700 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 -- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cff092700 msgr2=0x563cfefe8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 -- 172.21.15.179:0/360736301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 msgr2=0x563cff091740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 -- 172.21.15.179:0/360736301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec005d20 con 0x563cfefbb1e0 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa7016bd700 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa701ebe700 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 0x563cff08f400 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4012000 tx=0x7fa6f400e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df70a770bc42af59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa6fbfff700 1 -- 172.21.15.179:0/360736301 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400e710 con 0x563cfefbb1e0 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa6fbfff700 1 -- 172.21.15.179:0/360736301 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400e9e0 con 0x563cfefbb1e0 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa6fbfff700 1 -- 172.21.15.179:0/360736301 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400ed90 con 0x563cfefbb1e0 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 msgr2=0x563cff08f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 0x563cff08f400 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4012000 tx=0x7fa6f400e000 comp rx=0 tx=0).stop 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- 172.21.15.179:0/360736301 shutdown_connections 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cfefee8b0 0x563cff091740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 --2- 172.21.15.179:0/360736301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefbb1e0 0x563cff08f400 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- 172.21.15.179:0/360736301 >> 172.21.15.179:0/360736301 conn(0x563cfefc1740 msgr2=0x563cff004c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- 172.21.15.179:0/360736301 shutdown_connections 2024-05-08T22:35:11.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.817+0000 7fa704389140 1 -- 172.21.15.179:0/360736301 wait complete. 2024-05-08T22:35:11.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 Processor -- start 2024-05-08T22:35:11.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- start start 2024-05-08T22:35:11.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cfefbb1e0 0x563cff082ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefee8b0 0x563cff003b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeee08d0 con 0x563cff092700 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cff083000 con 0x563cfefee8b0 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cfefff210 con 0x563cfefbb1e0 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa701ebe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cfefbb1e0 0x563cff082ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 0x563cfefe8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53846/0 (socket says 172.21.15.179:53846) 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 -- 172.21.15.179:0/3554755959 learned_addr learned my addr 172.21.15.179:0/3554755959 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7016bd700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefee8b0 0x563cff003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 -- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cfefbb1e0 msgr2=0x563cff082ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cfefbb1e0 0x563cff082ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 -- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefee8b0 msgr2=0x563cff003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfefee8b0 0x563cff003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 -- 172.21.15.179:0/3554755959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009600 con 0x563cff092700 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa7026bf700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 0x563cfefe8f80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00ef60 tx=0x7fa6fc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa6faffd700 1 -- 172.21.15.179:0/3554755959 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00c9c0 con 0x563cff092700 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa6faffd700 1 -- 172.21.15.179:0/3554755959 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc008070 con 0x563cff092700 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa6faffd700 1 -- 172.21.15.179:0/3554755959 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00cd60 con 0x563cff092700 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563cfef12ca0 con 0x563cff092700 2024-05-08T22:35:11.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.821+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f400a1c0 con 0x563cff092700 2024-05-08T22:35:11.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa6faffd700 1 -- 172.21.15.179:0/3554755959 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6fc01b460 con 0x563cff092700 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa6faffd700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa6e4065090 0x7fa6e4067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa6faffd700 1 -- 172.21.15.179:0/3554755959 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6fc082b40 con 0x563cff092700 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa704389140 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563cfeffec90 0x563cff000640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa701ebe700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa6e4065090 0x7fa6e4067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x563cff000b80 con 0x563cfeffec90 2024-05-08T22:35:11.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa7016bd700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563cfeffec90 0x563cff000640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa701ebe700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa6e4065090 0x7fa6e4067550 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401f000 tx=0x7fa6f4028040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.825+0000 7fa7016bd700 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563cfeffec90 0x563cff000640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa7016bd700 1 -- 172.21.15.179:0/3554755959 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa6ec008400 con 0x563cfeffec90 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563cfeffec90 msgr2=0x563cff000640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563cfeffec90 0x563cff000640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa6e4065090 msgr2=0x7fa6e4067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa6e4065090 0x7fa6e4067550 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401f000 tx=0x7fa6f4028040 comp rx=0 tx=0).stop 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 msgr2=0x563cfefe8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 --2- 172.21.15.179:0/3554755959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cff092700 0x563cfefe8f80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00ef60 tx=0x7fa6fc00ef90 comp rx=0 tx=0).stop 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa7026bf700 1 -- 172.21.15.179:0/3554755959 reap_dead start 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 shutdown_connections 2024-05-08T22:35:11.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 >> 172.21.15.179:0/3554755959 conn(0x563cfefc1740 msgr2=0x563cff008570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 shutdown_connections 2024-05-08T22:35:11.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.829+0000 7fa704389140 1 -- 172.21.15.179:0/3554755959 wait complete. 2024-05-08T22:35:11.835 INFO:teuthology.orchestra.run.smithi179.stdout:mytag 2024-05-08T22:35:11.836 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-08T22:35:11.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.917+0000 7fc2b5e6c140 1 Processor -- start 2024-05-08T22:35:11.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.917+0000 7fc2b5e6c140 1 -- start start 2024-05-08T22:35:11.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 0x559bcb110b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bcb05a2f0 con 0x559bcb0cb780 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bcb0168e0 con 0x559bcb19c620 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bcb016580 con 0x559bcb0c9580 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53856/0 (socket says 172.21.15.179:53856) 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 -- 172.21.15.179:0/342806474 learned_addr learned my addr 172.21.15.179:0/342806474 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b39a1700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 0x559bcb110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b41a2700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 -- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 msgr2=0x559bcb110b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 0x559bcb110b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 -- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 msgr2=0x559bcb199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 -- 172.21.15.179:0/342806474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac005de0 con 0x559bcb0cb780 2024-05-08T22:35:11.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b39a1700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 0x559bcb110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a810 tx=0x7fc29c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf7cb774cc4463 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b219e700 1 -- 172.21.15.179:0/342806474 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c012660 con 0x559bcb0cb780 2024-05-08T22:35:11.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b219e700 1 -- 172.21.15.179:0/342806474 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c0127c0 con 0x559bcb0cb780 2024-05-08T22:35:11.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b219e700 1 -- 172.21.15.179:0/342806474 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c012b70 con 0x559bcb0cb780 2024-05-08T22:35:11.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/342806474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 msgr2=0x559bcb0f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a810 tx=0x7fc29c005d80 comp rx=0 tx=0).stop 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/342806474 shutdown_connections 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0cb780 0x559bcb0f8b00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/342806474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0c9580 0x559bcb110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/342806474 >> 172.21.15.179:0/342806474 conn(0x559bcb0f4110 msgr2=0x559bcb19f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/342806474 shutdown_connections 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.921+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/342806474 wait complete. 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 Processor -- start 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- start start 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 0x559bcb0f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0cb780 0x559bcb237230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb10e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bcafec8d0 con 0x559bcb0c9580 2024-05-08T22:35:11.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bcb237bb0 con 0x559bcb19c620 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bcb237e50 con 0x559bcb0cb780 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 0x559bcb0f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 0x559bcb0f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53858/0 (socket says 172.21.15.179:53858) 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 -- 172.21.15.179:0/3610478498 learned_addr learned my addr 172.21.15.179:0/3610478498 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b41a2700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb10e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0cb780 0x559bcb237230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 -- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0cb780 msgr2=0x559bcb237230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0cb780 0x559bcb237230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 -- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 msgr2=0x559bcb10e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bcb19c620 0x559bcb10e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 -- 172.21.15.179:0/3610478498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c00a410 con 0x559bcb0c9580 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bcb0cb780 0x559bcb237230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b39a1700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 0x559bcb0f1ed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c010 tx=0x7fc2ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b119c700 1 -- 172.21.15.179:0/3610478498 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac008690 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b119c700 1 -- 172.21.15.179:0/3610478498 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac01d690 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559bcb004b00 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b119c700 1 -- 172.21.15.179:0/3610478498 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01da60 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.925+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bcaff7f40 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b119c700 1 -- 172.21.15.179:0/3610478498 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2ac019040 con 0x559bcb0c9580 2024-05-08T22:35:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b119c700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc294065090 0x7fc294067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b119c700 1 -- 172.21.15.179:0/3610478498 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2ac084790 con 0x559bcb0c9580 2024-05-08T22:35:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc294065090 0x7fc294067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x559bcb10ee20 0x559bcb10c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x559bcb10cb60 con 0x559bcb10ee20 2024-05-08T22:35:11.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b41a2700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x559bcb10ee20 0x559bcb10c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.929+0000 7fc2b31a0700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc294065090 0x7fc294067550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005dc0 tx=0x7fc29c01e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b41a2700 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x559bcb10ee20 0x559bcb10c620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b41a2700 1 -- 172.21.15.179:0/3610478498 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fc2a400b020 con 0x559bcb10ee20 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x559bcb10ee20 msgr2=0x559bcb10c620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x559bcb10ee20 0x559bcb10c620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc294065090 msgr2=0x7fc294067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc294065090 0x7fc294067550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005dc0 tx=0x7fc29c01e040 comp rx=0 tx=0).stop 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 msgr2=0x559bcb0f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 --2- 172.21.15.179:0/3610478498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bcb0c9580 0x559bcb0f1ed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c010 tx=0x7fc2ac002fb0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b41a2700 1 -- 172.21.15.179:0/3610478498 reap_dead start 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 shutdown_connections 2024-05-08T22:35:11.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 >> 172.21.15.179:0/3610478498 conn(0x559bcb0f4110 msgr2=0x559bcb0f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 shutdown_connections 2024-05-08T22:35:11.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.933+0000 7fc2b5e6c140 1 -- 172.21.15.179:0/3610478498 wait complete. 2024-05-08T22:35:11.938 INFO:teuthology.orchestra.run.smithi179.stdout:mytag 2024-05-08T22:35:11.938 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-08T22:35:11.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 Processor -- start 2024-05-08T22:35:11.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 -- start start 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef66cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef6f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef5b8a00 con 0x5632ef6211e0 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef5b42f0 con 0x5632ef6f8700 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98737ad140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632ef5708e0 con 0x5632ef654a80 2024-05-08T22:35:11.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98712e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98712e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53864/0 (socket says 172.21.15.179:53864) 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98712e2700 1 -- 172.21.15.179:0/3486000788 learned_addr learned my addr 172.21.15.179:0/3486000788 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f9871ae3700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef6f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f9870ae1700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef66cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98712e2700 1 -- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 msgr2=0x5632ef66cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.977+0000 7f98712e2700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef66cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98712e2700 1 -- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 msgr2=0x5632ef6f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98712e2700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef6f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98712e2700 1 -- 172.21.15.179:0/3486000788 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c00a100 con 0x5632ef6211e0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f9870ae1700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef66cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98712e2700 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a7c0 tx=0x7f986c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e059abe43ce5142b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f986b7fe700 1 -- 172.21.15.179:0/3486000788 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00f790 con 0x5632ef6211e0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f986b7fe700 1 -- 172.21.15.179:0/3486000788 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c00fa60 con 0x5632ef6211e0 2024-05-08T22:35:11.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f986b7fe700 1 -- 172.21.15.179:0/3486000788 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00fe10 con 0x5632ef6211e0 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- 172.21.15.179:0/3486000788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 msgr2=0x5632ef6678b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a7c0 tx=0x7f986c00f000 comp rx=0 tx=0).stop 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- 172.21.15.179:0/3486000788 shutdown_connections 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef6f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef66cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 --2- 172.21.15.179:0/3486000788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6678b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- 172.21.15.179:0/3486000788 >> 172.21.15.179:0/3486000788 conn(0x5632ef627740 msgr2=0x5632ef66aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- 172.21.15.179:0/3486000788 shutdown_connections 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- 172.21.15.179:0/3486000788 wait complete. 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 Processor -- start 2024-05-08T22:35:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.981+0000 7f98737ad140 1 -- start start 2024-05-08T22:35:11.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef647400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef64ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef570580 con 0x5632ef6211e0 2024-05-08T22:35:11.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef5468d0 con 0x5632ef6f8700 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632ef6f4680 con 0x5632ef654a80 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53868/0 (socket says 172.21.15.179:53868) 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 -- 172.21.15.179:0/961850245 learned_addr learned my addr 172.21.15.179:0/961850245 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f9870ae1700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef647400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f9871ae3700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef64ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 -- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 msgr2=0x5632ef647400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5632ef654a80 0x5632ef647400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 -- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 msgr2=0x5632ef64ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef64ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 -- 172.21.15.179:0/961850245 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ef55eb00 con 0x5632ef6211e0 2024-05-08T22:35:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f9871ae3700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ef6f8700 0x5632ef64ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:11.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98712e2700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6f4140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5632ef628e50 tx=0x7f986c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f986a7fc700 1 -- 172.21.15.179:0/961850245 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c010630 con 0x5632ef6211e0 2024-05-08T22:35:11.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f986a7fc700 1 -- 172.21.15.179:0/961850245 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c009030 con 0x5632ef6211e0 2024-05-08T22:35:11.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632ef551f40 con 0x5632ef6211e0 2024-05-08T22:35:11.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f986a7fc700 1 -- 172.21.15.179:0/961850245 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0109b0 con 0x5632ef6211e0 2024-05-08T22:35:11.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.985+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632ef5519a0 con 0x5632ef6211e0 2024-05-08T22:35:11.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f986a7fc700 1 -- 172.21.15.179:0/961850245 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f986c017050 con 0x5632ef6211e0 2024-05-08T22:35:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f986a7fc700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9854065090 0x7f9854067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f986a7fc700 1 -- 172.21.15.179:0/961850245 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f986c084b50 con 0x5632ef6211e0 2024-05-08T22:35:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f9870ae1700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9854065090 0x7f9854067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f98737ad140 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5632ef78e3a0 0x5632ef64df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x5632ef64e490 con 0x5632ef78e3a0 2024-05-08T22:35:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f9871ae3700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5632ef78e3a0 0x5632ef64df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f9870ae1700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9854065090 0x7f9854067550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f985c003f10 tx=0x7f985c003870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.989+0000 7f9871ae3700 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5632ef78e3a0 0x5632ef64df50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f9871ae3700 1 -- 172.21.15.179:0/961850245 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f986400e440 con 0x5632ef78e3a0 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5632ef78e3a0 msgr2=0x5632ef64df50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5632ef78e3a0 0x5632ef64df50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9854065090 msgr2=0x7f9854067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9854065090 0x7f9854067550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f985c003f10 tx=0x7f985c003870 comp rx=0 tx=0).stop 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 msgr2=0x5632ef6f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 --2- 172.21.15.179:0/961850245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5632ef6211e0 0x5632ef6f4140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5632ef628e50 tx=0x7f986c0033a0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f9871ae3700 1 -- 172.21.15.179:0/961850245 reap_dead start 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 shutdown_connections 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 >> 172.21.15.179:0/961850245 conn(0x5632ef627740 msgr2=0x5632ef66cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 shutdown_connections 2024-05-08T22:35:11.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:11.993+0000 7f98737ad140 1 -- 172.21.15.179:0/961850245 wait complete. 2024-05-08T22:35:11.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:35:11.998 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:35:11.998 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:35:12.014 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:35:12.033 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:35:12.034 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:35:12.034 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:12.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:35:12.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:35:12.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:12.041 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:35:12.047 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T22:35:12.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:35:12.049 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:35:12.049 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:12.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:35:12.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:35:12.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:12.057 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:35:12.063 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:35:12.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:35:12.064 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:35:12.064 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:12.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:35:12.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:35:12.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:12.100 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:35:12.106 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:35:12.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:35:12.108 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:35:12.108 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:12.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:35:12.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:35:12.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:12.115 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:35:12.122 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:35:12.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:35:12.123 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:12.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.241+0000 7f29b086f700 1 Processor -- start 2024-05-08T22:35:12.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.241+0000 7f29b086f700 1 -- start start 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4033fb0 0x7f29a40eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29a4047cc0 0x7f29a4052f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a404fe20 con 0x7f29a4048210 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a404fb80 con 0x7f29a4047cc0 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40502b0 con 0x7f29a4033fb0 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29a4047cc0 0x7f29a4052f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29a4047cc0 0x7f29a4052f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53880/0 (socket says 172.21.15.179:53880) 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 -- 172.21.15.179:0/2053742301 learned_addr learned my addr 172.21.15.179:0/2053742301 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29aad9d700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4033fb0 0x7f29a40eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 -- 172.21.15.179:0/2053742301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4033fb0 msgr2=0x7f29a40eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4033fb0 0x7f29a40eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 -- 172.21.15.179:0/2053742301 >> v1:172.21.15.195:6789/0 conn(0x7f29a4048210 legacy=0x7f29a40eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 -- 172.21.15.179:0/2053742301 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40ebe70 con 0x7f29a4047cc0 2024-05-08T22:35:12.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29ab59e700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4052f70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0641b0 tx=0x7f29ac04f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424c00740e34c822 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29a9d9b700 1 -- 172.21.15.179:0/2053742301 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac067250 con 0x7f29a4047cc0 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29a9d9b700 1 -- 172.21.15.179:0/2053742301 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0673b0 con 0x7f29a4047cc0 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29a9d9b700 1 -- 172.21.15.179:0/2053742301 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac067960 con 0x7f29a4047cc0 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- 172.21.15.179:0/2053742301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 msgr2=0x7f29a4052f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4052f70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0641b0 tx=0x7f29ac04f740 comp rx=0 tx=0).stop 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- 172.21.15.179:0/2053742301 shutdown_connections 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4052f70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 --2- 172.21.15.179:0/2053742301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4033fb0 0x7f29a40eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- 172.21.15.179:0/2053742301 >> 172.21.15.179:0/2053742301 conn(0x7f29a4054b70 msgr2=0x7f29a4054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- 172.21.15.179:0/2053742301 shutdown_connections 2024-05-08T22:35:12.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.245+0000 7f29b086f700 1 -- 172.21.15.179:0/2053742301 wait complete. 2024-05-08T22:35:12.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.249+0000 7f29b086f700 1 Processor -- start 2024-05-08T22:35:12.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.249+0000 7f29b086f700 1 -- start start 2024-05-08T22:35:12.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.249+0000 7f29b086f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 0x7f29a404c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.249+0000 7f29b086f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4048210 0x7f29a404d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a404e230 con 0x7f29a4047cc0 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a404df50 con 0x7f29a4033fb0 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a404bde0 con 0x7f29a4048210 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53890/0 (socket says 172.21.15.179:53890) 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 -- 172.21.15.179:0/1114634168 learned_addr learned my addr 172.21.15.179:0/1114634168 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aad9d700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 0x7f29a404c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29ab59e700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4048210 0x7f29a404d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 -- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4048210 msgr2=0x7f29a404d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4048210 0x7f29a404d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 -- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 msgr2=0x7f29a404c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 0x7f29a404c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 -- 172.21.15.179:0/1114634168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a404c0c0 con 0x7f29a4047cc0 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aad9d700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 0x7f29a404c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:12.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29aa59c700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c4d0 tx=0x7f299c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0052d0 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c005430 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a404c3a0 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0055a0 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a40396d0 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.253+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a404bc20 con 0x7f29a4047cc0 2024-05-08T22:35:12.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.257+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f299c0038c0 con 0x7f29a4047cc0 2024-05-08T22:35:12.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.257+0000 7f29937fe700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 0x7f2998068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.257+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f299c083600 con 0x7f29a4047cc0 2024-05-08T22:35:12.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.257+0000 7f29aad9d700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 0x7f2998068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.257+0000 7f29aad9d700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 0x7f2998068bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f29a00112c0 tx=0x7f29a0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.265+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f299c050800 con 0x7f29a4047cc0 2024-05-08T22:35:12.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.469+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a4005440 con 0x7f29a4047cc0 2024-05-08T22:35:12.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.469+0000 7f29937fe700 1 -- 172.21.15.179:0/1114634168 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f299c053e20 con 0x7f29a4047cc0 2024-05-08T22:35:12.473 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:12.473 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:12.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 msgr2=0x7f2998068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 0x7f2998068bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f29a00112c0 tx=0x7f29a0009240 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 msgr2=0x7f29a4039190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c4d0 tx=0x7f299c00c7e0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 shutdown_connections 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2998066730 0x7f2998068bf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a4048210 0x7f29a404d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a4047cc0 0x7f29a4039190 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 --2- 172.21.15.179:0/1114634168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a4033fb0 0x7f29a404c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 >> 172.21.15.179:0/1114634168 conn(0x7f29a4054b70 msgr2=0x7f29a40e5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 shutdown_connections 2024-05-08T22:35:12.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.473+0000 7f29b086f700 1 -- 172.21.15.179:0/1114634168 wait complete. 2024-05-08T22:35:12.477 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:12.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:12.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 Processor -- start 2024-05-08T22:35:12.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 -- start start 2024-05-08T22:35:12.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8d00b19e0 0x7fd8d00b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00b3980 con 0x7fd8d00b3550 2024-05-08T22:35:12.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d0159b60 con 0x7fd8d01539c0 2024-05-08T22:35:12.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d6386700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d0159e40 con 0x7fd8d00b19e0 2024-05-08T22:35:12.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d4b83700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8d00b19e0 0x7fd8d00b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d5b85700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd8d00b3550 0x7fd8d015d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36786/0 (socket says 172.21.15.179:36786) 2024-05-08T22:35:12.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d5b85700 1 -- 172.21.15.179:0/3068369755 learned_addr learned my addr 172.21.15.179:0/3068369755 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:12.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d4b83700 1 -- 172.21.15.179:0/3068369755 >> v1:172.21.15.195:6789/0 conn(0x7fd8d01539c0 legacy=0x7fd8d006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d4b83700 1 -- 172.21.15.179:0/3068369755 >> v1:172.21.15.179:6789/0 conn(0x7fd8d00b3550 legacy=0x7fd8d015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8c3fff700 1 -- 172.21.15.179:0/3068369755 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272351198 0 0) 0x7fd8d00b3980 con 0x7fd8d00b3550 2024-05-08T22:35:12.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d4b83700 1 -- 172.21.15.179:0/3068369755 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d015a120 con 0x7fd8d00b19e0 2024-05-08T22:35:12.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.601+0000 7fd8d4b83700 1 --2- 172.21.15.179:0/3068369755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b19e0 0x7fd8d00b3010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009850 tx=0x7fd8cc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a65b5958d7fadd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8c3fff700 1 -- 172.21.15.179:0/3068369755 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc011830 con 0x7fd8d00b19e0 2024-05-08T22:35:12.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8c3fff700 1 -- 172.21.15.179:0/3068369755 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc011990 con 0x7fd8d00b19e0 2024-05-08T22:35:12.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8c3fff700 1 -- 172.21.15.179:0/3068369755 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc011d20 con 0x7fd8d00b19e0 2024-05-08T22:35:12.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- 172.21.15.179:0/3068369755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b19e0 msgr2=0x7fd8d00b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 --2- 172.21.15.179:0/3068369755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b19e0 0x7fd8d00b3010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009850 tx=0x7fd8cc009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- 172.21.15.179:0/3068369755 shutdown_connections 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 --2- 172.21.15.179:0/3068369755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b19e0 0x7fd8d00b3010 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- 172.21.15.179:0/3068369755 >> 172.21.15.179:0/3068369755 conn(0x7fd8d00ce9d0 msgr2=0x7fd8d00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- 172.21.15.179:0/3068369755 shutdown_connections 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- 172.21.15.179:0/3068369755 wait complete. 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 Processor -- start 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- start start 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8d00b19e0 0x7fd8d014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8d01539c0 0x7fd8d00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01ef530 con 0x7fd8d00b3550 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d014c9f0 con 0x7fd8d00b19e0 2024-05-08T22:35:12.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d6386700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d014ce70 con 0x7fd8d01539c0 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53906/0 (socket says 172.21.15.179:53906) 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d5b85700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8d01539c0 0x7fd8d00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d5384700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8d00b19e0 0x7fd8d014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 -- 172.21.15.179:0/586276572 learned_addr learned my addr 172.21.15.179:0/586276572 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 -- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8d01539c0 msgr2=0x7fd8d00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8d01539c0 0x7fd8d00cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 -- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8d00b19e0 msgr2=0x7fd8d014c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8d00b19e0 0x7fd8d014c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.605+0000 7fd8d4b83700 1 -- 172.21.15.179:0/586276572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc0094b0 con 0x7fd8d00b3550 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8d4b83700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc000c00 tx=0x7fd8cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0045d0 con 0x7fd8d00b3550 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc01e070 con 0x7fd8d00b3550 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d014d150 con 0x7fd8d00b3550 2024-05-08T22:35:12.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc003920 con 0x7fd8d00b3550 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d014d430 con 0x7fd8d00b3550 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d01f1e20 con 0x7fd8d00b3550 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8cc003a80 con 0x7fd8d00b3550 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 0x7fd8b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.609+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8cc0853a0 con 0x7fd8d00b3550 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.613+0000 7fd8d5384700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 0x7fd8b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.613+0000 7fd8d5384700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 0x7fd8b4068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00cab00 tx=0x7fd8c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.617+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8d01f1e20 con 0x7fd8d00b3550 2024-05-08T22:35:12.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.789+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8d01f1e20 con 0x7fd8d00b3550 2024-05-08T22:35:12.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.793+0000 7fd8c1ffb700 1 -- 172.21.15.179:0/586276572 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fd8d01f1e20 con 0x7fd8d00b3550 2024-05-08T22:35:12.794 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:12.795 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":43,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:05.486153+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:12.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 msgr2=0x7fd8b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 0x7fd8b4068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00cab00 tx=0x7fd8c4009040 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 msgr2=0x7fd8d01eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc000c00 tx=0x7fd8cc004770 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 shutdown_connections 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8b4066730 0x7fd8b4068bf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8d01539c0 0x7fd8d00cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d00b3550 0x7fd8d01eeff0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 --2- 172.21.15.179:0/586276572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8d00b19e0 0x7fd8d014c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 >> 172.21.15.179:0/586276572 conn(0x7fd8d00ce9d0 msgr2=0x7fd8d00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 shutdown_connections 2024-05-08T22:35:12.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.797+0000 7fd8d6386700 1 -- 172.21.15.179:0/586276572 wait complete. 2024-05-08T22:35:12.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:34:39.249198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:34:40.358534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:12.816 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:12.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.937+0000 7fca5df11700 1 Processor -- start 2024-05-08T22:35:12.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.937+0000 7fca5df11700 1 -- start start 2024-05-08T22:35:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fca58142890 0x7fca580b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fca580b2ca0 0x7fca5806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca58142fd0 0x7fca5806d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580b22d0 con 0x7fca58142890 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580b19e0 con 0x7fca580b2ca0 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5815e640 con 0x7fca58142fd0 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fca58142890 0x7fca580b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fca58142890 0x7fca580b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53920/0 (socket says 172.21.15.179:53920) 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 -- 172.21.15.179:0/1993377293 learned_addr learned my addr 172.21.15.179:0/1993377293 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5d710700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca58142fd0 0x7fca5806d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca57fff700 1 --2- 172.21.15.179:0/1993377293 >> v2:172.21.15.195:3300/0 conn(0x7fca580b2ca0 0x7fca5806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 -- 172.21.15.179:0/1993377293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca58142fd0 msgr2=0x7fca5806d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca58142fd0 0x7fca5806d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 -- 172.21.15.179:0/1993377293 >> v2:172.21.15.195:3300/0 conn(0x7fca580b2ca0 msgr2=0x7fca5806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/1993377293 >> v2:172.21.15.195:3300/0 conn(0x7fca580b2ca0 0x7fca5806cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 -- 172.21.15.179:0/1993377293 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580b1c80 con 0x7fca58142890 2024-05-08T22:35:12.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca580b2760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fca5806b3f0 tx=0x7fca48009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a1f622e1fef91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca577fe700 1 -- 172.21.15.179:0/1993377293 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48011830 con 0x7fca58142890 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca577fe700 1 -- 172.21.15.179:0/1993377293 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca48011990 con 0x7fca58142890 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca577fe700 1 -- 172.21.15.179:0/1993377293 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48011d40 con 0x7fca58142890 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- 172.21.15.179:0/1993377293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 msgr2=0x7fca580b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca580b2760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fca5806b3f0 tx=0x7fca48009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- 172.21.15.179:0/1993377293 shutdown_connections 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca58142fd0 0x7fca5806d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- 172.21.15.179:0/1993377293 >> v2:172.21.15.195:3300/0 conn(0x7fca580b2ca0 0x7fca5806cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 --2- 172.21.15.179:0/1993377293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca580b2760 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- 172.21.15.179:0/1993377293 >> 172.21.15.179:0/1993377293 conn(0x7fca580cea30 msgr2=0x7fca580cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- 172.21.15.179:0/1993377293 shutdown_connections 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.941+0000 7fca5df11700 1 -- 172.21.15.179:0/1993377293 wait complete. 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 Processor -- start 2024-05-08T22:35:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- start start 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 0x7fca581533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 0x7fca58130430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5812e950 con 0x7fca58142890 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5812ec30 con 0x7fca58142fd0 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5812ef10 con 0x7fca580b2ca0 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53926/0 (socket says 172.21.15.179:53926) 2024-05-08T22:35:12.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 -- 172.21.15.179:0/2824273403 learned_addr learned my addr 172.21.15.179:0/2824273403 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 0x7fca581533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 -- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 msgr2=0x7fca581533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5d710700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 0x7fca58130430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 0x7fca581533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 -- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 msgr2=0x7fca58130430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 0x7fca58130430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 -- 172.21.15.179:0/2824273403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48009570 con 0x7fca58142890 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 0x7fca581533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5d710700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 0x7fca58130430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca57fff700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e9d0 tx=0x7fca4000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40009720 con 0x7fca58142890 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4000c850 con 0x7fca58142890 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5812f1f0 con 0x7fca58142890 2024-05-08T22:35:12.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000cbc0 con 0x7fca58142890 2024-05-08T22:35:12.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58153900 con 0x7fca58142890 2024-05-08T22:35:12.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.945+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5804e700 con 0x7fca58142890 2024-05-08T22:35:12.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.949+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca4000cd20 con 0x7fca58142890 2024-05-08T22:35:12.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.949+0000 7fca557fa700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 0x7fca38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.949+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca400843f0 con 0x7fca58142890 2024-05-08T22:35:12.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.949+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 0x7fca38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.949+0000 7fca5cf0f700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 0x7fca38068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca581333d0 tx=0x7fca48008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:12.953+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fca40051600 con 0x7fca58142890 2024-05-08T22:35:13.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca5806c340 con 0x7fca58142890 2024-05-08T22:35:13.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca557fa700 1 -- 172.21.15.179:0/2824273403 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca40054c20 con 0x7fca58142890 2024-05-08T22:35:13.130 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 msgr2=0x7fca38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 0x7fca38068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca581333d0 tx=0x7fca48008030 comp rx=0 tx=0).stop 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 msgr2=0x7fca5812df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e9d0 tx=0x7fca4000ed90 comp rx=0 tx=0).stop 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 shutdown_connections 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fca38066730 0x7fca38068bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca58142fd0 0x7fca58130430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca58142890 0x7fca5812df40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 --2- 172.21.15.179:0/2824273403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca580b2ca0 0x7fca581533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 >> 172.21.15.179:0/2824273403 conn(0x7fca580cea30 msgr2=0x7fca58141a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 shutdown_connections 2024-05-08T22:35:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.129+0000 7fca5df11700 1 -- 172.21.15.179:0/2824273403 wait complete. 2024-05-08T22:35:13.145 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:35:13.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.256+0000 7f38c7fff700 1 Processor -- start 2024-05-08T22:35:13.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.256+0000 7f38c7fff700 1 -- start start 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38c8142890 0x7f38c80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c80b22d0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c80b19e0 con 0x7f38c806d4b0 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c815e640 con 0x7f38c8142890 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c67fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f38c80b2ca0 0x7f38c806cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36798/0 (socket says 172.21.15.179:36798) 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c67fc700 1 -- 172.21.15.179:0/401266129 learned_addr learned my addr 172.21.15.179:0/401266129 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:13.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/401266129 >> v2:172.21.15.195:3300/0 conn(0x7f38c8142890 0x7f38c80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111941090 0 0) 0x7f38c80b22d0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b40035d0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c6ffd700 1 -- 172.21.15.179:0/401266129 >> v1:172.21.15.195:6789/0 conn(0x7f38c806d4b0 legacy=0x7f38c815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2584262226 0 0) 0x7f38b40035d0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c6ffd700 1 -- 172.21.15.179:0/401266129 >> v1:172.21.15.179:6789/0 conn(0x7f38c80b2ca0 legacy=0x7f38c806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c6ffd700 1 -- 172.21.15.179:0/401266129 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80b1c80 con 0x7f38c8142890 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/401266129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c8142890 0x7f38c80b2760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009850 tx=0x7f38b0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a4f3a73e5d100a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8001670 con 0x7f38c8142890 2024-05-08T22:35:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b0004030 con 0x7f38c8142890 2024-05-08T22:35:13.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c5ffb700 1 -- 172.21.15.179:0/401266129 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00043a0 con 0x7f38c8142890 2024-05-08T22:35:13.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- 172.21.15.179:0/401266129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c8142890 msgr2=0x7f38c80b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 --2- 172.21.15.179:0/401266129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c8142890 0x7f38c80b2760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009850 tx=0x7f38b0009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:13.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- 172.21.15.179:0/401266129 shutdown_connections 2024-05-08T22:35:13.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 --2- 172.21.15.179:0/401266129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c8142890 0x7f38c80b2760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- 172.21.15.179:0/401266129 >> 172.21.15.179:0/401266129 conn(0x7f38c80cea30 msgr2=0x7f38c80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- 172.21.15.179:0/401266129 shutdown_connections 2024-05-08T22:35:13.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.260+0000 7f38c7fff700 1 -- 172.21.15.179:0/401266129 wait complete. 2024-05-08T22:35:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 Processor -- start 2024-05-08T22:35:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 -- start start 2024-05-08T22:35:13.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c806d4b0 0x7f38c81339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38c8142890 0x7f38c8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c81312e0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8131560 con 0x7f38c806d4b0 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8131840 con 0x7f38c8142890 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53930/0 (socket says 172.21.15.179:53930) 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 -- 172.21.15.179:0/471372627 learned_addr learned my addr 172.21.15.179:0/471372627 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c806d4b0 0x7f38c81339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c77fe700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38c8142890 0x7f38c8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 -- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38c8142890 msgr2=0x7f38c8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38c8142890 0x7f38c8151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 -- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c806d4b0 msgr2=0x7f38c81339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c806d4b0 0x7f38c81339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.264+0000 7f38c67fc700 1 -- 172.21.15.179:0/471372627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00094b0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38c67fc700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c760 tx=0x7f38b800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8007560 con 0x7f38c80b2ca0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c8131b20 con 0x7f38c80b2ca0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c8133f00 con 0x7f38c80b2ca0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80076c0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004ad0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac005310 con 0x7f38c80b2ca0 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38b8004c30 con 0x7f38c80b2ca0 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 0x7f38a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38b80845a0 con 0x7f38c80b2ca0 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.268+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 0x7f38a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.272+0000 7f38c6ffd700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 0x7f38a8068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.276+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f38b8051840 con 0x7f38c80b2ca0 2024-05-08T22:35:13.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.444+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38ac005b40 con 0x7f38c80b2ca0 2024-05-08T22:35:13.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.444+0000 7f38bffff700 1 -- 172.21.15.179:0/471372627 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38b8054e60 con 0x7f38c80b2ca0 2024-05-08T22:35:13.450 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 msgr2=0x7f38a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 0x7f38a8068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0008030 comp rx=0 tx=0).stop 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 msgr2=0x7f38c8151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c760 tx=0x7f38b800ca70 comp rx=0 tx=0).stop 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 shutdown_connections 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38a8066730 0x7f38a8068bf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38c8142890 0x7f38c8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38c80b2ca0 0x7f38c8151600 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 --2- 172.21.15.179:0/471372627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c806d4b0 0x7f38c81339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 >> 172.21.15.179:0/471372627 conn(0x7f38c80cea30 msgr2=0x7f38c806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 shutdown_connections 2024-05-08T22:35:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.448+0000 7f38c7fff700 1 -- 172.21.15.179:0/471372627 wait complete. 2024-05-08T22:35:13.464 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:35:13.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 Processor -- start 2024-05-08T22:35:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 -- start start 2024-05-08T22:35:13.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 0x7f13d40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40b8ee0 con 0x7f13d40ce0e0 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d415d4a0 con 0x7f13d415d0c0 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d9415700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40ce570 con 0x7f13d40b8a80 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d8c14700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f13d40ce0e0 0x7f13d406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36804/0 (socket says 172.21.15.179:36804) 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.580+0000 7f13d8c14700 1 -- 172.21.15.179:0/3294987812 learned_addr learned my addr 172.21.15.179:0/3294987812 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d37fe700 1 --2- 172.21.15.179:0/3294987812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 0x7f13d40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d2ffd700 1 -- 172.21.15.179:0/3294987812 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407979930 0 0) 0x7f13d40b8ee0 con 0x7f13d40ce0e0 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d2ffd700 1 -- 172.21.15.179:0/3294987812 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc0035d0 con 0x7f13d40ce0e0 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d37fe700 1 -- 172.21.15.179:0/3294987812 >> v1:172.21.15.195:6789/0 conn(0x7f13d415d0c0 legacy=0x7f13d40d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d37fe700 1 -- 172.21.15.179:0/3294987812 >> v1:172.21.15.179:6789/0 conn(0x7f13d40ce0e0 legacy=0x7f13d406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d37fe700 1 -- 172.21.15.179:0/3294987812 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d406abb0 con 0x7f13d40b8a80 2024-05-08T22:35:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d37fe700 1 --2- 172.21.15.179:0/3294987812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 0x7f13d40cdba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13c40098b0 tx=0x7f13c4009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580421228e279c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d2ffd700 1 -- 172.21.15.179:0/3294987812 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4011830 con 0x7f13d40b8a80 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d2ffd700 1 -- 172.21.15.179:0/3294987812 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4011990 con 0x7f13d40b8a80 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d2ffd700 1 -- 172.21.15.179:0/3294987812 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4011d20 con 0x7f13d40b8a80 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 -- 172.21.15.179:0/3294987812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 msgr2=0x7f13d40cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 --2- 172.21.15.179:0/3294987812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 0x7f13d40cdba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13c40098b0 tx=0x7f13c4009bc0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 -- 172.21.15.179:0/3294987812 shutdown_connections 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 --2- 172.21.15.179:0/3294987812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d40b8a80 0x7f13d40cdba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 -- 172.21.15.179:0/3294987812 >> 172.21.15.179:0/3294987812 conn(0x7f13d40d2ed0 msgr2=0x7f13d40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 -- 172.21.15.179:0/3294987812 shutdown_connections 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.584+0000 7f13d9415700 1 -- 172.21.15.179:0/3294987812 wait complete. 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 Processor -- start 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- start start 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d40ce0e0 0x7f13d406b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d415d0c0 0x7f13d41502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4153bb0 con 0x7f13d4150820 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40cbd00 con 0x7f13d40ce0e0 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4156e80 con 0x7f13d415d0c0 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53932/0 (socket says 172.21.15.179:53932) 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 -- 172.21.15.179:0/628238358 learned_addr learned my addr 172.21.15.179:0/628238358 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d37fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d415d0c0 0x7f13d41502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d3fff700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d40ce0e0 0x7f13d406b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 -- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d415d0c0 msgr2=0x7f13d41502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d415d0c0 0x7f13d41502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 -- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d40ce0e0 msgr2=0x7f13d406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d40ce0e0 0x7f13d406b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 -- 172.21.15.179:0/628238358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40094b0 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d8c14700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00cb40 tx=0x7f13cc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc007500 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc007660 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d4150e50 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0079f0 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.588+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40cb6b0 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d40cbe90 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f13cc01b3b0 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d0ff9700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 0x7f13c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13cc084160 con 0x7f13d4150820 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d3fff700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 0x7f13c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.592+0000 7f13d3fff700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 0x7f13c0068c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000f60 tx=0x7f13c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.600+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f13cc051350 con 0x7f13d4150820 2024-05-08T22:35:13.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13d404e700 con 0x7f13d4150820 2024-05-08T22:35:13.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d0ff9700 1 -- 172.21.15.179:0/628238358 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13cc054970 con 0x7f13d4150820 2024-05-08T22:35:13.774 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:35:13.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 msgr2=0x7f13c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 0x7f13c0068c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000f60 tx=0x7f13c8009040 comp rx=0 tx=0).stop 2024-05-08T22:35:13.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 msgr2=0x7f13d414ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00cb40 tx=0x7f13cc00cb70 comp rx=0 tx=0).stop 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 shutdown_connections 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f13c0066760 0x7f13c0068c20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13d4150820 0x7f13d414ee20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13d415d0c0 0x7f13d41502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 --2- 172.21.15.179:0/628238358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13d40ce0e0 0x7f13d406b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 >> 172.21.15.179:0/628238358 conn(0x7f13d40d2ed0 msgr2=0x7f13d40d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 shutdown_connections 2024-05-08T22:35:13.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.772+0000 7f13d9415700 1 -- 172.21.15.179:0/628238358 wait complete. 2024-05-08T22:35:13.791 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:13.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 Processor -- start 2024-05-08T22:35:13.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- start start 2024-05-08T22:35:13.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fc0143640 0x7f7fc006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc015de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc006ce80 con 0x7f7fc00cea40 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc015e340 con 0x7f7fc0143640 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0143a20 con 0x7f7fc015d970 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbf7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fc0143640 0x7f7fc006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbeffd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f7fc00cea40 0x7f7fc00b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:51178/0 (socket says 172.21.15.179:51178) 2024-05-08T22:35:13.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbeffd700 1 -- 172.21.15.179:0/2974196638 learned_addr learned my addr 172.21.15.179:0/2974196638 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:13.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 -- 172.21.15.179:0/2974196638 >> v2:172.21.15.195:3300/0 conn(0x7f7fc0143640 msgr2=0x7f7fc006eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 --2- 172.21.15.179:0/2974196638 >> v2:172.21.15.195:3300/0 conn(0x7f7fc0143640 0x7f7fc006eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 -- 172.21.15.179:0/2974196638 >> v1:172.21.15.195:6789/0 conn(0x7f7fc00cea40 legacy=0x7f7fc00b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 -- 172.21.15.179:0/2974196638 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00cf220 con 0x7f7fc015d970 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbe7fc700 1 -- 172.21.15.179:0/2974196638 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956779222 0 0) 0x7f7fc006ce80 con 0x7f7fc00cea40 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbffff700 1 --2- 172.21.15.179:0/2974196638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc015de00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009dc0 tx=0x7f7fb800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f79905eee31280 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbe7fc700 1 -- 172.21.15.179:0/2974196638 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800bce0 con 0x7f7fc015d970 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbe7fc700 1 -- 172.21.15.179:0/2974196638 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb800be40 con 0x7f7fc015d970 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fbe7fc700 1 -- 172.21.15.179:0/2974196638 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80116a0 con 0x7f7fc015d970 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- 172.21.15.179:0/2974196638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 msgr2=0x7f7fc015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/2974196638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc015de00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009dc0 tx=0x7f7fb800b330 comp rx=0 tx=0).stop 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- 172.21.15.179:0/2974196638 shutdown_connections 2024-05-08T22:35:13.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/2974196638 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc015de00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/2974196638 >> v2:172.21.15.195:3300/0 conn(0x7f7fc0143640 0x7f7fc006eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- 172.21.15.179:0/2974196638 >> 172.21.15.179:0/2974196638 conn(0x7f7fc00cfae0 msgr2=0x7f7fc00cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.904+0000 7f7fc4a95700 1 -- 172.21.15.179:0/2974196638 shutdown_connections 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 -- 172.21.15.179:0/2974196638 wait complete. 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 Processor -- start 2024-05-08T22:35:13.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 -- start start 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc0143640 0x7f7fc015c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc0157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00ad470 con 0x7f7fc00cea40 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00ac7f0 con 0x7f7fc0143640 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fc4a95700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00acad0 con 0x7f7fc015d970 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53938/0 (socket says 172.21.15.179:53938) 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 -- 172.21.15.179:0/1418538879 learned_addr learned my addr 172.21.15.179:0/1418538879 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbeffd700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc0143640 0x7f7fc015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbffff700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 -- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 msgr2=0x7f7fc0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc0157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 -- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc0143640 msgr2=0x7f7fc015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc0143640 0x7f7fc015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.908+0000 7f7fbf7fe700 1 -- 172.21.15.179:0/1418538879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009a70 con 0x7f7fc00cea40 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fbf7fe700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9d0 tx=0x7f7fa800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8009720 con 0x7f7fc00cea40 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800ee70 con 0x7f7fc00cea40 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800ca50 con 0x7f7fc00cea40 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc00acdb0 con 0x7f7fc00cea40 2024-05-08T22:35:13.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc015ce90 con 0x7f7fc00cea40 2024-05-08T22:35:13.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc015d080 con 0x7f7fc00cea40 2024-05-08T22:35:13.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.912+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7fa800cbb0 con 0x7f7fc00cea40 2024-05-08T22:35:13.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.916+0000 7f7fb7fff700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 0x7f7fa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.916+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7fa80842b0 con 0x7f7fc00cea40 2024-05-08T22:35:13.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.916+0000 7f7fbeffd700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 0x7f7fa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.916+0000 7f7fbeffd700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 0x7f7fa0068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fc006ed50 tx=0x7f7fb0007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:13.920+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7fa80514d0 con 0x7f7fc00cea40 2024-05-08T22:35:14.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc004e700 con 0x7f7fc00cea40 2024-05-08T22:35:14.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fb7fff700 1 -- 172.21.15.179:0/1418538879 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f7fa800ce60 con 0x7f7fc00cea40 2024-05-08T22:35:14.139 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:14.139 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:14.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 msgr2=0x7f7fa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 0x7f7fa0068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fc006ed50 tx=0x7f7fb0007e00 comp rx=0 tx=0).stop 2024-05-08T22:35:14.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 msgr2=0x7f7fc013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9d0 tx=0x7f7fa800ece0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 shutdown_connections 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7fa0066730 0x7f7fa0068bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc0143640 0x7f7fc015c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc015d970 0x7f7fc0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 --2- 172.21.15.179:0/1418538879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc00cea40 0x7f7fc013f690 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 >> 172.21.15.179:0/1418538879 conn(0x7f7fc00cfae0 msgr2=0x7f7fc00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 shutdown_connections 2024-05-08T22:35:14.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.136+0000 7f7fc4a95700 1 -- 172.21.15.179:0/1418538879 wait complete. 2024-05-08T22:35:14.143 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:14.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:14.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 Processor -- start 2024-05-08T22:35:14.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 -- start start 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f047c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f047c0d1a90 0x7f047c0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c06ad50 con 0x7f047c0d16b0 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047c06b030 con 0x7f047c0d1a90 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f048096f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c06b310 con 0x7f047c0d1120 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047a59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f047c0d16b0 0x7f047c0cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36818/0 (socket says 172.21.15.179:36818) 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047a59c700 1 -- 172.21.15.179:0/287129759 learned_addr learned my addr 172.21.15.179:0/287129759 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f047c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047ad9d700 1 --2- 172.21.15.179:0/287129759 >> v2:172.21.15.195:3300/0 conn(0x7f047c0d1a90 0x7f047c0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985321854 0 0) 0x7f047c06ad50 con 0x7f047c0d16b0 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f045c0035d0 con 0x7f047c0d16b0 2024-05-08T22:35:14.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235867631 0 0) 0x7f045c0035d0 con 0x7f047c0d16b0 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 msgr2=0x7f047c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f047c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 >> v2:172.21.15.195:3300/0 conn(0x7f047c0d1a90 msgr2=0x7f047c0d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 --2- 172.21.15.179:0/287129759 >> v2:172.21.15.195:3300/0 conn(0x7f047c0d1a90 0x7f047c0d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c06b5f0 con 0x7f047c0d16b0 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f04640019f0 con 0x7f047c0d16b0 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 >> v1:172.21.15.179:6789/0 conn(0x7f047c0d16b0 legacy=0x7f047c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f045c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f045c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f045c004db0 0x7f045c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f045c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c06b030 con 0x7f045c003f60 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c06b310 con 0x7f045c004db0 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c00ba30 con 0x7f047c0d1120 2024-05-08T22:35:14.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047a59c700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f045c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f047ad9d700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f045c004db0 0x7f045c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 -- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 msgr2=0x7f045c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0d1120 0x7f045c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 -- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f045c004db0 msgr2=0x7f045c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f045c004db0 0x7f045c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.264+0000 7f0479d9b700 1 -- 172.21.15.179:0/287129759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0035d0 con 0x7f045c003f60 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f0479d9b700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f045c004870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f046c009b70 tx=0x7f045c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4636abddc5c15525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00e6c0 con 0x7f045c003f60 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00e820 con 0x7f045c003f60 2024-05-08T22:35:14.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f047959a700 1 -- 172.21.15.179:0/287129759 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00eb90 con 0x7f045c003f60 2024-05-08T22:35:14.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- 172.21.15.179:0/287129759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 msgr2=0x7f045c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 --2- 172.21.15.179:0/287129759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f045c004870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f046c009b70 tx=0x7f045c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f047ad9d700 1 -- 172.21.15.179:0/287129759 reap_dead start 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- 172.21.15.179:0/287129759 shutdown_connections 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- 172.21.15.179:0/287129759 >> 172.21.15.179:0/287129759 conn(0x7f047c0cfae0 msgr2=0x7f047c0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- 172.21.15.179:0/287129759 shutdown_connections 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- 172.21.15.179:0/287129759 wait complete. 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 Processor -- start 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.268+0000 7f048096f700 1 -- start start 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 0x7f047c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0a99f0 0x7f047c0a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0aa3b0 con 0x7f045c003f60 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0cc700 con 0x7f047c0cbb90 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0cc9e0 con 0x7f047c0a99f0 2024-05-08T22:35:14.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53954/0 (socket says 172.21.15.179:53954) 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 -- 172.21.15.179:0/655781210 learned_addr learned my addr 172.21.15.179:0/655781210 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047ad9d700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0a99f0 0x7f047c0a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0479d9b700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 0x7f047c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 -- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0a99f0 msgr2=0x7f047c0a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0a99f0 0x7f047c0a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 -- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 msgr2=0x7f047c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 0x7f047c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 -- 172.21.15.179:0/655781210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c14a1f0 con 0x7f045c003f60 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0479d9b700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 0x7f047c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f047a59c700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0464008ba0 tx=0x7f0464006460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464006cf0 con 0x7f045c003f60 2024-05-08T22:35:14.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0464006e50 con 0x7f045c003f60 2024-05-08T22:35:14.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c14a4d0 con 0x7f045c003f60 2024-05-08T22:35:14.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04640156c0 con 0x7f045c003f60 2024-05-08T22:35:14.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c159560 con 0x7f045c003f60 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c0ccb20 con 0x7f045c003f60 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.272+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f046401e3b0 con 0x7f045c003f60 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.276+0000 7f0472ffd700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 0x7f0460068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.276+0000 7f0479d9b700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 0x7f0460068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.276+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0464084790 con 0x7f045c003f60 2024-05-08T22:35:14.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.276+0000 7f0479d9b700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 0x7f0460068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f047c150900 tx=0x7f046c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.284+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0464051a30 con 0x7f045c003f60 2024-05-08T22:35:14.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.456+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f047c073290 con 0x7f045c003f60 2024-05-08T22:35:14.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.456+0000 7f0472ffd700 1 -- 172.21.15.179:0/655781210 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f046401a090 con 0x7f045c003f60 2024-05-08T22:35:14.462 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:14.462 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":43,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:05.486153+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:14.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 msgr2=0x7f0460068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 0x7f0460068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f047c150900 tx=0x7f046c008030 comp rx=0 tx=0).stop 2024-05-08T22:35:14.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 msgr2=0x7f047c0b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0464008ba0 tx=0x7f0464006460 comp rx=0 tx=0).stop 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 shutdown_connections 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0460066730 0x7f0460068bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f047c0cbb90 0x7f047c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f047c0a99f0 0x7f047c0a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 --2- 172.21.15.179:0/655781210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c003f60 0x7f047c0b1520 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 >> 172.21.15.179:0/655781210 conn(0x7f047c0cfae0 msgr2=0x7f047c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 shutdown_connections 2024-05-08T22:35:14.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.460+0000 7f048096f700 1 -- 172.21.15.179:0/655781210 wait complete. 2024-05-08T22:35:14.477 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:35:14.477 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:14.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 Processor -- start 2024-05-08T22:35:14.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 -- start start 2024-05-08T22:35:14.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f91f006de70 0x7f91f0161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f0165d50 con 0x7f91f015ef80 2024-05-08T22:35:14.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f006ad70 con 0x7f91f006de70 2024-05-08T22:35:14.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.584+0000 7f91f6e19700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0165260 con 0x7f91f0164db0 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f5616700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f91f006de70 0x7f91f0161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f5e17700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f91f015ef80 0x7f91f006d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:51190/0 (socket says 172.21.15.179:51190) 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f5e17700 1 -- 172.21.15.179:0/4031329283 learned_addr learned my addr 172.21.15.179:0/4031329283 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f4e15700 1 -- 172.21.15.179:0/4031329283 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393756324 0 0) 0x7f91f0165d50 con 0x7f91f015ef80 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f4e15700 1 -- 172.21.15.179:0/4031329283 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91dc0035d0 con 0x7f91f015ef80 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 -- 172.21.15.179:0/4031329283 >> v2:172.21.15.195:3300/0 conn(0x7f91f006de70 msgr2=0x7f91f0161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 --2- 172.21.15.179:0/4031329283 >> v2:172.21.15.195:3300/0 conn(0x7f91f006de70 0x7f91f0161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 -- 172.21.15.179:0/4031329283 >> v1:172.21.15.195:6789/0 conn(0x7f91f015ef80 legacy=0x7f91f006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 -- 172.21.15.179:0/4031329283 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f006e4b0 con 0x7f91f0164db0 2024-05-08T22:35:14.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6618700 1 --2- 172.21.15.179:0/4031329283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f006b200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009850 tx=0x7f91ec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5c1c8c133f49a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f4e15700 1 -- 172.21.15.179:0/4031329283 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec011830 con 0x7f91f0164db0 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f4e15700 1 -- 172.21.15.179:0/4031329283 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec011990 con 0x7f91f0164db0 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f4e15700 1 -- 172.21.15.179:0/4031329283 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec011d40 con 0x7f91f0164db0 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 -- 172.21.15.179:0/4031329283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 msgr2=0x7f91f006b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 --2- 172.21.15.179:0/4031329283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f006b200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009850 tx=0x7f91ec009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 -- 172.21.15.179:0/4031329283 shutdown_connections 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 --2- 172.21.15.179:0/4031329283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f006b200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 --2- 172.21.15.179:0/4031329283 >> v2:172.21.15.195:3300/0 conn(0x7f91f006de70 0x7f91f0161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 -- 172.21.15.179:0/4031329283 >> 172.21.15.179:0/4031329283 conn(0x7f91f00ce9d0 msgr2=0x7f91f00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 -- 172.21.15.179:0/4031329283 shutdown_connections 2024-05-08T22:35:14.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.588+0000 7f91f6e19700 1 -- 172.21.15.179:0/4031329283 wait complete. 2024-05-08T22:35:14.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 Processor -- start 2024-05-08T22:35:14.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- start start 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 0x7f91f0161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f0162210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f014d840 con 0x7f91f006de70 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f014d560 con 0x7f91f015ef80 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01fa2e0 con 0x7f91f0164db0 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53970/0 (socket says 172.21.15.179:53970) 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 -- 172.21.15.179:0/2019822243 learned_addr learned my addr 172.21.15.179:0/2019822243 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5616700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 0x7f91f0161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6618700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f0162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 -- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 msgr2=0x7f91f0162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f0162210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 -- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 msgr2=0x7f91f0161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 0x7f91f0161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 -- 172.21.15.179:0/2019822243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0094b0 con 0x7f91f006de70 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5616700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 0x7f91f0161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6618700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f0162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f5e17700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cbc0 tx=0x7f91e800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800a9e0 con 0x7f91f006de70 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800ab40 con 0x7f91f006de70 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f01fa5c0 con 0x7f91f006de70 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8012400 con 0x7f91f006de70 2024-05-08T22:35:14.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.592+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f01f68c0 con 0x7f91f006de70 2024-05-08T22:35:14.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0067740 con 0x7f91f006de70 2024-05-08T22:35:14.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91e801b3b0 con 0x7f91f006de70 2024-05-08T22:35:14.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91e67fc700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 0x7f91d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91e8083e10 con 0x7f91f006de70 2024-05-08T22:35:14.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91f5616700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 0x7f91d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.596+0000 7f91f5616700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 0x7f91d4068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91f015b6c0 tx=0x7f91e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.604+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f91e8087030 con 0x7f91f006de70 2024-05-08T22:35:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91f004e700 con 0x7f91f006de70 2024-05-08T22:35:14.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91e67fc700 1 -- 172.21.15.179:0/2019822243 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f91e8017090 con 0x7f91f006de70 2024-05-08T22:35:14.818 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:14.818 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 msgr2=0x7f91d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 0x7f91d4068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91f015b6c0 tx=0x7f91e0008040 comp rx=0 tx=0).stop 2024-05-08T22:35:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 msgr2=0x7f91f01f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cbc0 tx=0x7f91e800cca0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 shutdown_connections 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91d4066730 0x7f91d4068bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91f015ef80 0x7f91f0161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91f0164db0 0x7f91f0162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 --2- 172.21.15.179:0/2019822243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f006de70 0x7f91f01f6380 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 >> 172.21.15.179:0/2019822243 conn(0x7f91f00ce9d0 msgr2=0x7f91f0164370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 shutdown_connections 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.812+0000 7f91f6e19700 1 -- 172.21.15.179:0/2019822243 wait complete. 2024-05-08T22:35:14.820 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:14.830 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:14.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.940+0000 7f6e76b82700 1 Processor -- start 2024-05-08T22:35:14.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.940+0000 7f6e76b82700 1 -- start start 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.940+0000 7f6e76b82700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e7006ed50 0x7f6e700b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.940+0000 7f6e76b82700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7006ddf0 con 0x7f6e700ce2a0 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e700cd200 con 0x7f6e701434d0 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e700cd4e0 con 0x7f6e7006ed50 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e7537f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6e700ce2a0 0x7f6e7006e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36820/0 (socket says 172.21.15.179:36820) 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e7537f700 1 -- 172.21.15.179:0/3675762765 learned_addr learned my addr 172.21.15.179:0/3675762765 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:14.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76381700 1 --2- 172.21.15.179:0/3675762765 >> v2:172.21.15.195:3300/0 conn(0x7f6e7006ed50 0x7f6e700b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e74b7e700 1 -- 172.21.15.179:0/3675762765 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327376242 0 0) 0x7f6e7006ddf0 con 0x7f6e700ce2a0 2024-05-08T22:35:14.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e74b7e700 1 -- 172.21.15.179:0/3675762765 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e5c0035d0 con 0x7f6e700ce2a0 2024-05-08T22:35:14.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76381700 1 -- 172.21.15.179:0/3675762765 >> v1:172.21.15.195:6789/0 conn(0x7f6e701434d0 legacy=0x7f6e7006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76381700 1 -- 172.21.15.179:0/3675762765 >> v1:172.21.15.179:6789/0 conn(0x7f6e700ce2a0 legacy=0x7f6e7006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76381700 1 -- 172.21.15.179:0/3675762765 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e700cd7c0 con 0x7f6e7006ed50 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76381700 1 --2- 172.21.15.179:0/3675762765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e7006ed50 0x7f6e700b19e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d002cde81030a2eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e74b7e700 1 -- 172.21.15.179:0/3675762765 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800e070 con 0x7f6e7006ed50 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e74b7e700 1 -- 172.21.15.179:0/3675762765 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800bb20 con 0x7f6e7006ed50 2024-05-08T22:35:14.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e74b7e700 1 -- 172.21.15.179:0/3675762765 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68012430 con 0x7f6e7006ed50 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- 172.21.15.179:0/3675762765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e7006ed50 msgr2=0x7f6e700b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 --2- 172.21.15.179:0/3675762765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e7006ed50 0x7f6e700b19e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6800b330 comp rx=0 tx=0).stop 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- 172.21.15.179:0/3675762765 shutdown_connections 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 --2- 172.21.15.179:0/3675762765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e7006ed50 0x7f6e700b19e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- 172.21.15.179:0/3675762765 >> 172.21.15.179:0/3675762765 conn(0x7f6e700ce9d0 msgr2=0x7f6e700cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- 172.21.15.179:0/3675762765 shutdown_connections 2024-05-08T22:35:14.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.944+0000 7f6e76b82700 1 -- 172.21.15.179:0/3675762765 wait complete. 2024-05-08T22:35:14.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 Processor -- start 2024-05-08T22:35:14.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 -- start start 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e7006ed50 0x7f6e701595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e700ce2a0 0x7f6e7013b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c97c0 con 0x7f6e701434d0 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c99f0 con 0x7f6e700ce2a0 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76b82700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c9c90 con 0x7f6e7006ed50 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53986/0 (socket says 172.21.15.179:53986) 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 -- 172.21.15.179:0/677506680 learned_addr learned my addr 172.21.15.179:0/677506680 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e7537f700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e700ce2a0 0x7f6e7013b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e75b80700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e7006ed50 0x7f6e701595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 -- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e7006ed50 msgr2=0x7f6e701595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e7006ed50 0x7f6e701595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 -- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e700ce2a0 msgr2=0x7f6e7013b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e700ce2a0 0x7f6e7013b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 -- 172.21.15.179:0/677506680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009a70 con 0x7f6e701434d0 2024-05-08T22:35:14.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e76381700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e68013000 tx=0x7f6e68004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.948+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68012430 con 0x7f6e701434d0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68020070 con 0x7f6e701434d0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e700c9f70 con 0x7f6e701434d0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68004280 con 0x7f6e701434d0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700ca530 con 0x7f6e701434d0 2024-05-08T22:35:14.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70067740 con 0x7f6e701434d0 2024-05-08T22:35:14.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e680043e0 con 0x7f6e701434d0 2024-05-08T22:35:14.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e667fc700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 0x7f6e54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:14.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e68022070 con 0x7f6e701434d0 2024-05-08T22:35:14.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.952+0000 7f6e75b80700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 0x7f6e54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:14.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.956+0000 7f6e75b80700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 0x7f6e54068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6e7013fbd0 tx=0x7f6e6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:14.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:14.956+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6e68053570 con 0x7f6e701434d0 2024-05-08T22:35:15.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e7004e700 con 0x7f6e701434d0 2024-05-08T22:35:15.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e667fc700 1 -- 172.21.15.179:0/677506680 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f6e68056b90 con 0x7f6e701434d0 2024-05-08T22:35:15.171 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:15.171 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:34:41.381266+0000","modified":"2024-05-08T22:34:44.395407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/2344889573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2344889573},{"type":"v1","addr":"172.21.15.179:6835","nonce":2344889573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T22:35:15.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 msgr2=0x7f6e54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 0x7f6e54068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6e7013fbd0 tx=0x7f6e6c009040 comp rx=0 tx=0).stop 2024-05-08T22:35:15.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 msgr2=0x7f6e700caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e68013000 tx=0x7f6e68004af0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 shutdown_connections 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e54066730 0x7f6e54068bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e700ce2a0 0x7f6e7013b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e701434d0 0x7f6e700caa50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 --2- 172.21.15.179:0/677506680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e7006ed50 0x7f6e701595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 >> 172.21.15.179:0/677506680 conn(0x7f6e700ce9d0 msgr2=0x7f6e7015b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 shutdown_connections 2024-05-08T22:35:15.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.168+0000 7f6e76b82700 1 -- 172.21.15.179:0/677506680 wait complete. 2024-05-08T22:35:15.175 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 17 2024-05-08T22:35:15.189 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:15.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 Processor -- start 2024-05-08T22:35:15.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 -- start start 2024-05-08T22:35:15.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99700b4ad0 0x7f997006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f997015d590 0x7f997015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997015d9c0 con 0x7f99700d07d0 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997006d370 con 0x7f99700b4ad0 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f996ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997006d650 con 0x7f997015d590 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.292+0000 7f99667fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99700b4ad0 0x7f997006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99700b4ad0 0x7f997006cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44398/0 (socket says 172.21.15.179:44398) 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996effd700 1 --1- 172.21.15.179:0/2269062525 >> v1:172.21.15.195:6789/0 conn(0x7f99700d07d0 0x7f99700b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:49004/0 (socket says 172.21.15.179:49004) 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 -- 172.21.15.179:0/2269062525 learned_addr learned my addr 172.21.15.179:0/2269062525 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:15.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996f7fe700 1 --2- 172.21.15.179:0/2269062525 >> v2:172.21.15.195:3300/0 conn(0x7f997015d590 0x7f997015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 -- 172.21.15.179:0/2269062525 >> v2:172.21.15.195:3300/0 conn(0x7f997015d590 msgr2=0x7f997015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 --2- 172.21.15.179:0/2269062525 >> v2:172.21.15.195:3300/0 conn(0x7f997015d590 0x7f997015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996e7fc700 1 -- 172.21.15.179:0/2269062525 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571954618 0 0) 0x7f9960001110 con 0x7f99700d07d0 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 -- 172.21.15.179:0/2269062525 >> v1:172.21.15.195:6789/0 conn(0x7f99700d07d0 legacy=0x7f99700b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 -- 172.21.15.179:0/2269062525 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997006d930 con 0x7f99700b4ad0 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f99667fc700 1 --2- 172.21.15.179:0/2269062525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700b4ad0 0x7f997006cd00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f997006e710 tx=0x7f9958009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e37b31c3cf5062 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996e7fc700 1 -- 172.21.15.179:0/2269062525 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958011830 con 0x7f99700b4ad0 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996e7fc700 1 -- 172.21.15.179:0/2269062525 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9958011990 con 0x7f99700b4ad0 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996e7fc700 1 -- 172.21.15.179:0/2269062525 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958011d20 con 0x7f99700b4ad0 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 -- 172.21.15.179:0/2269062525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700b4ad0 msgr2=0x7f997006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 --2- 172.21.15.179:0/2269062525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700b4ad0 0x7f997006cd00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f997006e710 tx=0x7f9958009bf0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 -- 172.21.15.179:0/2269062525 shutdown_connections 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 --2- 172.21.15.179:0/2269062525 >> v2:172.21.15.195:3300/0 conn(0x7f997015d590 0x7f997015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 --2- 172.21.15.179:0/2269062525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700b4ad0 0x7f997006cd00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 -- 172.21.15.179:0/2269062525 >> 172.21.15.179:0/2269062525 conn(0x7f99700b2840 msgr2=0x7f99700b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 -- 172.21.15.179:0/2269062525 shutdown_connections 2024-05-08T22:35:15.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.296+0000 7f996ffff700 1 -- 172.21.15.179:0/2269062525 wait complete. 2024-05-08T22:35:15.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 Processor -- start 2024-05-08T22:35:15.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 -- start start 2024-05-08T22:35:15.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99700b4ad0 0x7f99700cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997015d590 0x7f99700a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700cbad0 con 0x7f99700d07d0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700ccd20 con 0x7f99700b4ad0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99700a9110 con 0x7f997015d590 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f99667fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f99667fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44414/0 (socket says 172.21.15.179:44414) 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f99667fc700 1 -- 172.21.15.179:0/3757384941 learned_addr learned my addr 172.21.15.179:0/3757384941 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996effd700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99700b4ad0 0x7f99700cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.300+0000 7f996f7fe700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997015d590 0x7f99700a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 -- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997015d590 msgr2=0x7f99700a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997015d590 0x7f99700a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 -- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99700b4ad0 msgr2=0x7f99700cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99700b4ad0 0x7f99700cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 -- 172.21.15.179:0/3757384941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958009570 con 0x7f99700d07d0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f99667fc700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9970067a00 tx=0x7f9958004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580044d0 con 0x7f99700d07d0 2024-05-08T22:35:15.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995801f070 con 0x7f99700d07d0 2024-05-08T22:35:15.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99700caab0 con 0x7f99700d07d0 2024-05-08T22:35:15.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958003900 con 0x7f99700d07d0 2024-05-08T22:35:15.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99700cb070 con 0x7f99700d07d0 2024-05-08T22:35:15.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99700ca910 con 0x7f99700d07d0 2024-05-08T22:35:15.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9958003a60 con 0x7f99700d07d0 2024-05-08T22:35:15.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 0x7f9950068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9958021070 con 0x7f99700d07d0 2024-05-08T22:35:15.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.304+0000 7f996effd700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 0x7f9950068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.308+0000 7f996effd700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 0x7f9950068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9960000f60 tx=0x7f9960007c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.312+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9958052e90 con 0x7f99700d07d0 2024-05-08T22:35:15.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.484+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99700ca910 con 0x7f99700d07d0 2024-05-08T22:35:15.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.488+0000 7f9967fff700 1 -- 172.21.15.179:0/3757384941 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f99700ca910 con 0x7f99700d07d0 2024-05-08T22:35:15.492 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:15.492 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":43,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:05.486153+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:34:39.249198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T22:34:40.358534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:15.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 msgr2=0x7f9950068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 0x7f9950068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9960000f60 tx=0x7f9960007c40 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 msgr2=0x7f99700cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9970067a00 tx=0x7f9958004770 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 shutdown_connections 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9950066730 0x7f9950068bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997015d590 0x7f99700a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99700d07d0 0x7f99700cc7e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 --2- 172.21.15.179:0/3757384941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99700b4ad0 0x7f99700cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 >> 172.21.15.179:0/3757384941 conn(0x7f99700b2840 msgr2=0x7f997006f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 shutdown_connections 2024-05-08T22:35:15.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.492+0000 7f996ffff700 1 -- 172.21.15.179:0/3757384941 wait complete. 2024-05-08T22:35:15.508 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:35:15.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 Processor -- start 2024-05-08T22:35:15.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 -- start start 2024-05-08T22:35:15.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e1c0b4f70 0x7f0e1c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c0cd6d0 con 0x7f0e1c15cf60 2024-05-08T22:35:15.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c161b30 con 0x7f0e1c15d340 2024-05-08T22:35:15.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e22d67700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cd970 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e21d65700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0e1c15cf60 0x7f0e1c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50700/0 (socket says 172.21.15.179:50700) 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e21d65700 1 -- 172.21.15.179:0/4023116342 learned_addr learned my addr 172.21.15.179:0/4023116342 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e21564700 1 --2- 172.21.15.179:0/4023116342 >> v2:172.21.15.195:3300/0 conn(0x7f0e1c0b4f70 0x7f0e1c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322520587 0 0) 0x7f0e1c0cd6d0 con 0x7f0e1c15cf60 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.616+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e080035d0 con 0x7f0e1c15cf60 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e21564700 1 -- 172.21.15.179:0/4023116342 >> v1:172.21.15.195:6789/0 conn(0x7f0e1c15d340 legacy=0x7f0e1c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e21564700 1 -- 172.21.15.179:0/4023116342 >> v1:172.21.15.179:6789/0 conn(0x7f0e1c15cf60 legacy=0x7f0e1c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576242541 0 0) 0x7f0e080035d0 con 0x7f0e1c15cf60 2024-05-08T22:35:15.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e21564700 1 -- 172.21.15.179:0/4023116342 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0cdc50 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e21564700 1 --2- 172.21.15.179:0/4023116342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c0ce450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009850 tx=0x7f0e18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86beec30a6397ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10001670 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18004030 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e20d63700 1 -- 172.21.15.179:0/4023116342 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180043a0 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- 172.21.15.179:0/4023116342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c0b4f70 msgr2=0x7f0e1c0ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 --2- 172.21.15.179:0/4023116342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c0ce450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009850 tx=0x7f0e18009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- 172.21.15.179:0/4023116342 shutdown_connections 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 --2- 172.21.15.179:0/4023116342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c0ce450 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- 172.21.15.179:0/4023116342 >> 172.21.15.179:0/4023116342 conn(0x7f0e1c0cf860 msgr2=0x7f0e1c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- 172.21.15.179:0/4023116342 shutdown_connections 2024-05-08T22:35:15.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- 172.21.15.179:0/4023116342 wait complete. 2024-05-08T22:35:15.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 Processor -- start 2024-05-08T22:35:15.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.620+0000 7f0e22d67700 1 -- start start 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1c15cf60 0x7f0e1c1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 0x7f0e1c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0c6dd0 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0c70b0 con 0x7f0e1c15d340 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0c7390 con 0x7f0e1c15cf60 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44420/0 (socket says 172.21.15.179:44420) 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 -- 172.21.15.179:0/1723553637 learned_addr learned my addr 172.21.15.179:0/1723553637 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22566700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 0x7f0e1c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21564700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1c15cf60 0x7f0e1c1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 -- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1c15cf60 msgr2=0x7f0e1c1f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1c15cf60 0x7f0e1c1f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 -- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 msgr2=0x7f0e1c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 0x7f0e1c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 -- 172.21.15.179:0/1723553637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180094b0 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22566700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 0x7f0e1c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e21d65700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0e100011e0 tx=0x7f0e1000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004ca0 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e10004e00 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c0c7670 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100072f0 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1f5d30 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.624+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c0c7b90 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.628+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e10007450 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.628+0000 7f0e0e7fc700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 0x7f0e00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.628+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e10084550 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.628+0000 7f0e21564700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 0x7f0e00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.628+0000 7f0e21564700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 0x7f0e00068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e180090b0 tx=0x7f0e18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.636+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e10051740 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.844+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e1c0c7b90 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.848 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:35:15.844+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:35:15.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.856+0000 7f0e0e7fc700 1 -- 172.21.15.179:0/1723553637 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e10054d60 con 0x7f0e1c0b4f70 2024-05-08T22:35:15.863 INFO:tasks.ceph.mds.c.smithi179.stderr: -5748> 2024-05-08T22:34:44.961+0000 7f5a5e205700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 msgr2=0x7f0e00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 0x7f0e00068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e180090b0 tx=0x7f0e18008030 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 msgr2=0x7f0e1c1f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0e100011e0 tx=0x7f0e1000ca50 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 shutdown_connections 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e00066730 0x7f0e00068bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1c15d340 0x7f0e1c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e1c15cf60 0x7f0e1c1f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 --2- 172.21.15.179:0/1723553637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e1c0b4f70 0x7f0e1c1f2b40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.860+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 >> 172.21.15.179:0/1723553637 conn(0x7f0e1c0cf860 msgr2=0x7f0e1c0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.864+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 shutdown_connections 2024-05-08T22:35:15.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.864+0000 7f0e22d67700 1 -- 172.21.15.179:0/1723553637 wait complete. 2024-05-08T22:35:15.867 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:35:15.881 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:35:15.901 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:35:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 Processor -- start 2024-05-08T22:35:15.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 -- start start 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fccc81648b0 0x7fccc806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc8165b30 con 0x7fccc80b30a0 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc80b3640 con 0x7fccc81648b0 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.984+0000 7fcccc980700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8165fc0 con 0x7fccc80b37a0 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:32902/0 (socket says 172.21.15.179:32902) 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 -- 172.21.15.179:0/1154592732 learned_addr learned my addr 172.21.15.179:0/1154592732 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:15.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc77fe700 1 --2- 172.21.15.179:0/1154592732 >> v2:172.21.15.195:3300/0 conn(0x7fccc81648b0 0x7fccc806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 -- 172.21.15.179:0/1154592732 >> v2:172.21.15.195:3300/0 conn(0x7fccc81648b0 msgr2=0x7fccc806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 --2- 172.21.15.179:0/1154592732 >> v2:172.21.15.195:3300/0 conn(0x7fccc81648b0 0x7fccc806bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 -- 172.21.15.179:0/1154592732 >> v1:172.21.15.195:6789/0 conn(0x7fccc80b30a0 legacy=0x7fccc80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 -- 172.21.15.179:0/1154592732 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8165200 con 0x7fccc80b37a0 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc7fff700 1 --2- 172.21.15.179:0/1154592732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009910 tx=0x7fccb8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46805fab12117758 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc67fc700 1 -- 172.21.15.179:0/1154592732 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8011830 con 0x7fccc80b37a0 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc67fc700 1 -- 172.21.15.179:0/1154592732 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb8011990 con 0x7fccc80b37a0 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fccc67fc700 1 -- 172.21.15.179:0/1154592732 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8011d40 con 0x7fccc80b37a0 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 -- 172.21.15.179:0/1154592732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 msgr2=0x7fccc806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 --2- 172.21.15.179:0/1154592732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009910 tx=0x7fccb8009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 -- 172.21.15.179:0/1154592732 shutdown_connections 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 --2- 172.21.15.179:0/1154592732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b37a0 0x7fccc806c100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 --2- 172.21.15.179:0/1154592732 >> v2:172.21.15.195:3300/0 conn(0x7fccc81648b0 0x7fccc806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 -- 172.21.15.179:0/1154592732 >> 172.21.15.179:0/1154592732 conn(0x7fccc8070d80 msgr2=0x7fccc8071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 -- 172.21.15.179:0/1154592732 shutdown_connections 2024-05-08T22:35:15.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.988+0000 7fcccc980700 1 -- 172.21.15.179:0/1154592732 wait complete. 2024-05-08T22:35:15.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 Processor -- start 2024-05-08T22:35:15.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- start start 2024-05-08T22:35:15.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b30a0 0x7fccc8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc80b37a0 0x7fccc8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815ab80 con 0x7fccc81648b0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8159f70 con 0x7fccc80b37a0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc815a250 con 0x7fccc80b30a0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b30a0 0x7fccc8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44450/0 (socket says 172.21.15.179:44450) 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 -- 172.21.15.179:0/1526539685 learned_addr learned my addr 172.21.15.179:0/1526539685 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc6ffd700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc80b37a0 0x7fccc8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 -- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b30a0 msgr2=0x7fccc8159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b30a0 0x7fccc8159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 -- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc80b37a0 msgr2=0x7fccc8156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc80b37a0 0x7fccc8156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 -- 172.21.15.179:0/1526539685 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8009570 con 0x7fccc81648b0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccc7fff700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fccc8166100 tx=0x7fccb8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8004300 con 0x7fccc81648b0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb801f070 con 0x7fccc81648b0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc815a530 con 0x7fccc81648b0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb80036c0 con 0x7fccc81648b0 2024-05-08T22:35:15.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.992+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc815c460 con 0x7fccc81648b0 2024-05-08T22:35:15.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc815c5f0 con 0x7fccc81648b0 2024-05-08T22:35:15.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccb8021030 con 0x7fccc81648b0 2024-05-08T22:35:16.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fccaffff700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 0x7fccb4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fccb80855f0 con 0x7fccc81648b0 2024-05-08T22:35:16.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fccc77fe700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 0x7fccb4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:15.996+0000 7fccc77fe700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 0x7fccb4068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011340 tx=0x7fccc000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:16.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.000+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fccb80527c0 con 0x7fccc81648b0 2024-05-08T22:35:16.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.208+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fccc804e700 con 0x7fccc81648b0 2024-05-08T22:35:16.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.860+0000 7fccaffff700 1 -- 172.21.15.179:0/1526539685 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fccb8006da0 con 0x7fccc81648b0 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 msgr2=0x7fccb4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 0x7fccb4068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011340 tx=0x7fccc000b040 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 msgr2=0x7fccc8156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fccc8166100 tx=0x7fccb8004770 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 shutdown_connections 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccb4066730 0x7fccb4068bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc80b37a0 0x7fccc8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc81648b0 0x7fccc8156760 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 --2- 172.21.15.179:0/1526539685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80b30a0 0x7fccc8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 >> 172.21.15.179:0/1526539685 conn(0x7fccc8070d80 msgr2=0x7fccc80cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 shutdown_connections 2024-05-08T22:35:16.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:16.864+0000 7fcccc980700 1 -- 172.21.15.179:0/1526539685 wait complete. 2024-05-08T22:35:16.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:35:17.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.056+0000 7f1d46213700 1 Processor -- start 2024-05-08T22:35:17.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.056+0000 7f1d46213700 1 -- start start 2024-05-08T22:35:17.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 0x7f1d4006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400b17c0 con 0x7f1d4015d390 2024-05-08T22:35:17.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4015d770 con 0x7f1d400ac990 2024-05-08T22:35:17.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cd380 con 0x7f1d400cd7a0 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d45a12700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 0x7f1d4006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d45211700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1d4015d390 0x7f1d400ac280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:49026/0 (socket says 172.21.15.179:49026) 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:32926/0 (socket says 172.21.15.179:32926) 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d45211700 1 -- 172.21.15.179:0/2833326964 learned_addr learned my addr 172.21.15.179:0/2833326964 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:17.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 -- 172.21.15.179:0/2833326964 >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 msgr2=0x7f1d4006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:17.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 --2- 172.21.15.179:0/2833326964 >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 0x7f1d4006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 -- 172.21.15.179:0/2833326964 >> v1:172.21.15.195:6789/0 conn(0x7f1d4015d390 legacy=0x7f1d400ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 -- 172.21.15.179:0/2833326964 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400b1aa0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d45a12700 1 --2- 172.21.15.179:0/2833326964 >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 0x7f1d4006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:17.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d44a10700 1 --2- 172.21.15.179:0/2833326964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009850 tx=0x7f1d30009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120a3fd1f59f3cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d37fff700 1 -- 172.21.15.179:0/2833326964 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30011830 con 0x7f1d400cd7a0 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d37fff700 1 -- 172.21.15.179:0/2833326964 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30011990 con 0x7f1d400cd7a0 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d37fff700 1 -- 172.21.15.179:0/2833326964 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30011d40 con 0x7f1d400cd7a0 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- 172.21.15.179:0/2833326964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 msgr2=0x7f1d400ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 --2- 172.21.15.179:0/2833326964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009850 tx=0x7f1d30009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- 172.21.15.179:0/2833326964 shutdown_connections 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 --2- 172.21.15.179:0/2833326964 >> v2:172.21.15.195:3300/0 conn(0x7f1d400ac990 0x7f1d4006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 --2- 172.21.15.179:0/2833326964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d400cd7a0 0x7f1d400ce5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- 172.21.15.179:0/2833326964 >> 172.21.15.179:0/2833326964 conn(0x7f1d40098a70 msgr2=0x7f1d400b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- 172.21.15.179:0/2833326964 shutdown_connections 2024-05-08T22:35:17.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.060+0000 7f1d46213700 1 -- 172.21.15.179:0/2833326964 wait complete. 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 Processor -- start 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- start start 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d400ac990 0x7f1d401590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4015d390 0x7f1d4009baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40152b60 con 0x7f1d400cd7a0 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4009c3b0 con 0x7f1d400ac990 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4009c690 con 0x7f1d4015d390 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44464/0 (socket says 172.21.15.179:44464) 2024-05-08T22:35:17.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 -- 172.21.15.179:0/2111971528 learned_addr learned my addr 172.21.15.179:0/2111971528 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:17.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d45a12700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4015d390 0x7f1d4009baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d45211700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d400ac990 0x7f1d401590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 -- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4015d390 msgr2=0x7f1d4009baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4015d390 0x7f1d4009baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 -- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d400ac990 msgr2=0x7f1d401590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d400ac990 0x7f1d401590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 -- 172.21.15.179:0/2111971528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38009a70 con 0x7f1d400cd7a0 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d44a10700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1d300090b0 tx=0x7f1d30004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300045d0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3001f070 con 0x7f1d400cd7a0 2024-05-08T22:35:17.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300038e0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d300094b0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.064+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4009cc50 con 0x7f1d400cd7a0 2024-05-08T22:35:17.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4004e770 con 0x7f1d400cd7a0 2024-05-08T22:35:17.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d30003a40 con 0x7f1d400cd7a0 2024-05-08T22:35:17.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d35ffb700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1d30021070 con 0x7f1d400cd7a0 2024-05-08T22:35:17.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d45211700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.068+0000 7f1d45211700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 0x7f1d24068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c000f60 tx=0x7f1d3c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.076+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1d30052d00 con 0x7f1d400cd7a0 2024-05-08T22:35:17.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.244+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d400ae7f0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.868+0000 7f1d35ffb700 1 -- 172.21.15.179:0/2111971528 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1d3002b0d0 con 0x7f1d400cd7a0 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 msgr2=0x7f1d24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 0x7f1d24068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c000f60 tx=0x7f1d3c007f30 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 msgr2=0x7f1d40152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1d300090b0 tx=0x7f1d30004770 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 shutdown_connections 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d4015d390 0x7f1d4009baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d400cd7a0 0x7f1d40152620 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 --2- 172.21.15.179:0/2111971528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d400ac990 0x7f1d401590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 >> 172.21.15.179:0/2111971528 conn(0x7f1d40098a70 msgr2=0x7f1d400b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 shutdown_connections 2024-05-08T22:35:17.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:17.872+0000 7f1d46213700 1 -- 172.21.15.179:0/2111971528 wait complete. 2024-05-08T22:35:17.876 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:35:17.892 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:35:18.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 Processor -- start 2024-05-08T22:35:18.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 -- start start 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f57040d00a0 0x7f57040d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040d1a10 con 0x7f57040d2ab0 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040d1cf0 con 0x7f570415d3c0 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570c684700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040d1fd0 con 0x7f57040d00a0 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570ac21700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f57040d2ab0 0x7f57040b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50710/0 (socket says 172.21.15.179:50710) 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570ac21700 1 -- 172.21.15.179:0/598403507 learned_addr learned my addr 172.21.15.179:0/598403507 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:18.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f5709c1f700 1 --2- 172.21.15.179:0/598403507 >> v2:172.21.15.195:3300/0 conn(0x7f57040d00a0 0x7f57040d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817167714 0 0) 0x7f57040d1a10 con 0x7f57040d2ab0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f00035d0 con 0x7f57040d2ab0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070946929 0 0) 0x7f57040d1cf0 con 0x7f570415d3c0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57040d1a10 con 0x7f570415d3c0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501003553 0 0) 0x7f56f00035d0 con 0x7f57040d2ab0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 >> v2:172.21.15.195:3300/0 conn(0x7f57040d00a0 msgr2=0x7f57040d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 --2- 172.21.15.179:0/598403507 >> v2:172.21.15.195:3300/0 conn(0x7f57040d00a0 0x7f57040d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 >> v1:172.21.15.195:6789/0 conn(0x7f570415d3c0 legacy=0x7f57040cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040d22b0 con 0x7f57040d2ab0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f56fc0019f0 con 0x7f57040d2ab0 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 >> v1:172.21.15.179:6789/0 conn(0x7f57040d2ab0 legacy=0x7f57040b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f56f0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56f0004870 0x7f56f0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0005230 0x7f56f000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040d1fd0 con 0x7f56f0003bb0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f000c0b0 con 0x7f56f0005230 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f000c270 con 0x7f56f0004870 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570a420700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f56f0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.012+0000 7f570ac21700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0005230 0x7f56f000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f5709c1f700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56f0004870 0x7f56f0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 -- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56f0004870 msgr2=0x7f56f0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56f0004870 0x7f56f0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 -- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0005230 msgr2=0x7f56f000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0005230 0x7f56f000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570ac21700 1 -- 172.21.15.179:0/598403507 reap_dead start 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 -- 172.21.15.179:0/598403507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00035d0 con 0x7f56f0003bb0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f56f0003fb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5700001960 tx=0x7f56f00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea38180b68889836 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:18.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700001450 con 0x7f56f0003bb0 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57000052f0 con 0x7f56f0003bb0 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570941e700 1 -- 172.21.15.179:0/598403507 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57000036a0 con 0x7f56f0003bb0 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 msgr2=0x7f56f0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f56f0003fb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5700001960 tx=0x7f56f00058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- 172.21.15.179:0/598403507 shutdown_connections 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 --2- 172.21.15.179:0/598403507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f56f0003fb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- 172.21.15.179:0/598403507 >> 172.21.15.179:0/598403507 conn(0x7f57040b4ca0 msgr2=0x7f57040b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- 172.21.15.179:0/598403507 shutdown_connections 2024-05-08T22:35:18.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- 172.21.15.179:0/598403507 wait complete. 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 Processor -- start 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- start start 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5704153460 0x7f5704154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5704155530 0x7f570415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041538e0 con 0x7f56f0003bb0 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041f6340 con 0x7f5704153460 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570c684700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5704156300 con 0x7f5704155530 2024-05-08T22:35:18.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44482/0 (socket says 172.21.15.179:44482) 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570a420700 1 -- 172.21.15.179:0/2524884332 learned_addr learned my addr 172.21.15.179:0/2524884332 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f5709c1f700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5704153460 0x7f5704154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.016+0000 7f570ac21700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5704155530 0x7f570415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 -- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5704155530 msgr2=0x7f570415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5704155530 0x7f570415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 -- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5704153460 msgr2=0x7f5704154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5704153460 0x7f5704154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 -- 172.21.15.179:0/2524884332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570415ea10 con 0x7f56f0003bb0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570a420700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f570000c3c0 tx=0x7f57000012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700006a90 con 0x7f56f0003bb0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700006c10 con 0x7f56f0003bb0 2024-05-08T22:35:18.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5704155d40 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007040 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570415ecb0 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57000071a0 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570415e870 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 0x7f56e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f5700084c50 con 0x7f56f0003bb0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f5709c1f700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 0x7f56e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.020+0000 7f5709c1f700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 0x7f56e8068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009b40 tx=0x7f56f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:18.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.024+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f570415e870 con 0x7f56f0003bb0 2024-05-08T22:35:18.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.192+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f570415e870 con 0x7f56f0003bb0 2024-05-08T22:35:18.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.872+0000 7f56faffd700 1 -- 172.21.15.179:0/2524884332 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f570415e870 con 0x7f56f0003bb0 2024-05-08T22:35:18.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 msgr2=0x7f56e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 0x7f56e8068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009b40 tx=0x7f56f4008040 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 msgr2=0x7f5704152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f570000c3c0 tx=0x7f57000012d0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 shutdown_connections 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56e8066730 0x7f56e8068bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5704153460 0x7f5704154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5704155530 0x7f570415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 --2- 172.21.15.179:0/2524884332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56f0003bb0 0x7f5704152f20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 >> 172.21.15.179:0/2524884332 conn(0x7f57040b4ca0 msgr2=0x7f57040cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:18.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 shutdown_connections 2024-05-08T22:35:18.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:18.876+0000 7f570c684700 1 -- 172.21.15.179:0/2524884332 wait complete. 2024-05-08T22:35:18.883 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:35:18.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:35:19.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 Processor -- start 2024-05-08T22:35:19.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 -- start start 2024-05-08T22:35:19.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 0x7f2cac0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2cac0b30a0 0x7f2cac0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac165b30 con 0x7f2cac1648b0 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0b3640 con 0x7f2cac0b30a0 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb17f8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac165fc0 con 0x7f2cac06c2d0 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2cac0b30a0 0x7f2cac0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2cac0b30a0 0x7f2cac0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44488/0 (socket says 172.21.15.179:44488) 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/152726385 learned_addr learned my addr 172.21.15.179:0/152726385 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cabfff700 1 --1- 172.21.15.179:0/152726385 >> v1:172.21.15.179:6789/0 conn(0x7f2cac1648b0 0x7f2cac06bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50720/0 (socket says 172.21.15.179:50720) 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cab7fe700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 0x7f2cac0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/152726385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 msgr2=0x7f2cac0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 0x7f2cac0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/152726385 >> v1:172.21.15.179:6789/0 conn(0x7f2cac1648b0 legacy=0x7f2cac06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/152726385 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac165200 con 0x7f2cac0b30a0 2024-05-08T22:35:19.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cab7fe700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 0x7f2cac0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2cb0ff7700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac0b30a0 0x7f2cac0cc420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009910 tx=0x7f2ca4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09b9a81ad0db047 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2caaffd700 1 -- 172.21.15.179:0/152726385 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400ba60 con 0x7f2cac0b30a0 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2caaffd700 1 -- 172.21.15.179:0/152726385 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400bbc0 con 0x7f2cac0b30a0 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.004+0000 7f2caaffd700 1 -- 172.21.15.179:0/152726385 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4009e60 con 0x7f2cac0b30a0 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 -- 172.21.15.179:0/152726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac0b30a0 msgr2=0x7f2cac0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac0b30a0 0x7f2cac0cc420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009910 tx=0x7f2ca4009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 -- 172.21.15.179:0/152726385 shutdown_connections 2024-05-08T22:35:19.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac0b30a0 0x7f2cac0cc420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/152726385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac06c2d0 0x7f2cac0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 -- 172.21.15.179:0/152726385 >> 172.21.15.179:0/152726385 conn(0x7f2cac070d80 msgr2=0x7f2cac071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:19.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 -- 172.21.15.179:0/152726385 shutdown_connections 2024-05-08T22:35:19.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.008+0000 7f2cb17f8700 1 -- 172.21.15.179:0/152726385 wait complete. 2024-05-08T22:35:19.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 Processor -- start 2024-05-08T22:35:19.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 -- start start 2024-05-08T22:35:19.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 0x7f2cac1fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac0b30a0 0x7f2cac159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac15a520 con 0x7f2cac1648b0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1599b0 con 0x7f2cac06c2d0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac159c80 con 0x7f2cac0b30a0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44496/0 (socket says 172.21.15.179:44496) 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/169203942 learned_addr learned my addr 172.21.15.179:0/169203942 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cab7fe700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac0b30a0 0x7f2cac159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cabfff700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 0x7f2cac1fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac0b30a0 msgr2=0x7f2cac159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac0b30a0 0x7f2cac159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 msgr2=0x7f2cac1fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 0x7f2cac1fae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 -- 172.21.15.179:0/169203942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca4009570 con 0x7f2cac1648b0 2024-05-08T22:35:19.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cabfff700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 0x7f2cac1fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb0ff7700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009c60 tx=0x7f2ca4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4004510 con 0x7f2cac1648b0 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca401f070 con 0x7f2cac1648b0 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.012+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac159f60 con 0x7f2cac1648b0 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4003900 con 0x7f2cac1648b0 2024-05-08T22:35:19.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac148630 con 0x7f2cac1648b0 2024-05-08T22:35:19.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac148490 con 0x7f2cac1648b0 2024-05-08T22:35:19.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ca4003a60 con 0x7f2cac1648b0 2024-05-08T22:35:19.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2ca8ff9700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 0x7f2c98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2ca4021070 con 0x7f2cac1648b0 2024-05-08T22:35:19.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.016+0000 7f2cabfff700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 0x7f2c98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.020+0000 7f2cabfff700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 0x7f2c98068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000f60 tx=0x7f2ca0007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.024+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2cac148490 con 0x7f2cac1648b0 2024-05-08T22:35:19.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.192+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2cac148490 con 0x7f2cac1648b0 2024-05-08T22:35:19.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.204+0000 7f2ca8ff9700 1 -- 172.21.15.179:0/169203942 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2cac148490 con 0x7f2cac1648b0 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 msgr2=0x7f2c98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 0x7f2c98068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000f60 tx=0x7f2ca0007df0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 msgr2=0x7f2cac15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009c60 tx=0x7f2ca4004770 comp rx=0 tx=0).stop 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 shutdown_connections 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2c98066730 0x7f2c98068bf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac1648b0 0x7f2cac15f630 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cac0b30a0 0x7f2cac159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 --2- 172.21.15.179:0/169203942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cac06c2d0 0x7f2cac1fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 >> 172.21.15.179:0/169203942 conn(0x7f2cac070d80 msgr2=0x7f2cac0cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:19.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 shutdown_connections 2024-05-08T22:35:19.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.208+0000 7f2cb17f8700 1 -- 172.21.15.179:0/169203942 wait complete. 2024-05-08T22:35:19.227 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-08T22:35:19.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 Processor -- start 2024-05-08T22:35:19.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 -- start start 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99e01434d0 0x7f99e006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e006dc20 0x7f99e006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00cd200 con 0x7f99e00ce2a0 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00cd4e0 con 0x7f99e01434d0 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99df59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00cd7c0 con 0x7f99e006dc20 2024-05-08T22:35:19.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99e01434d0 0x7f99e006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99e01434d0 0x7f99e006d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:42520/0 (socket says 172.21.15.179:42520) 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 -- 172.21.15.179:0/2521347189 learned_addr learned my addr 172.21.15.179:0/2521347189 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99ded9d700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f99e00ce2a0 0x7f99e00b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:49050/0 (socket says 172.21.15.179:49050) 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99ddd9b700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e006dc20 0x7f99e006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 -- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e006dc20 msgr2=0x7f99e006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e006dc20 0x7f99e006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 -- 172.21.15.179:0/2521347189 >> v1:172.21.15.195:6789/0 conn(0x7f99e00ce2a0 legacy=0x7f99e00b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 -- 172.21.15.179:0/2521347189 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00cdaa0 con 0x7f99e01434d0 2024-05-08T22:35:19.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99de59c700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e01434d0 0x7f99e006d6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009e50 tx=0x7f99d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b6a0474ecf4525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99dd59a700 1 -- 172.21.15.179:0/2521347189 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800bbc0 con 0x7f99e01434d0 2024-05-08T22:35:19.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99dd59a700 1 -- 172.21.15.179:0/2521347189 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d800bd20 con 0x7f99e01434d0 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.336+0000 7f99dd59a700 1 -- 172.21.15.179:0/2521347189 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80116a0 con 0x7f99e01434d0 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e01434d0 msgr2=0x7f99e006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e01434d0 0x7f99e006d6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009e50 tx=0x7f99d800b330 comp rx=0 tx=0).stop 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- 172.21.15.179:0/2521347189 shutdown_connections 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e006dc20 0x7f99e006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- 172.21.15.179:0/2521347189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e01434d0 0x7f99e006d6e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- 172.21.15.179:0/2521347189 >> 172.21.15.179:0/2521347189 conn(0x7f99e00ce9d0 msgr2=0x7f99e00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- 172.21.15.179:0/2521347189 shutdown_connections 2024-05-08T22:35:19.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- 172.21.15.179:0/2521347189 wait complete. 2024-05-08T22:35:19.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 Processor -- start 2024-05-08T22:35:19.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- start start 2024-05-08T22:35:19.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e00ce2a0 0x7f99e013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e0141c40 0x7f99e013e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e013c8e0 con 0x7f99e01434d0 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e013cb80 con 0x7f99e0141c40 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99df59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e013ce60 con 0x7f99e00ce2a0 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99ddd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99de59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e00ce2a0 0x7f99e013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99ddd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44510/0 (socket says 172.21.15.179:44510) 2024-05-08T22:35:19.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99ddd9b700 1 -- 172.21.15.179:0/2671179921 learned_addr learned my addr 172.21.15.179:0/2671179921 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.340+0000 7f99ded9d700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e0141c40 0x7f99e013e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 -- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e00ce2a0 msgr2=0x7f99e013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e00ce2a0 0x7f99e013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 -- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e0141c40 msgr2=0x7f99e013e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e0141c40 0x7f99e013e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 -- 172.21.15.179:0/2671179921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8009b30 con 0x7f99e01434d0 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ddd9b700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f99d400e730 tx=0x7f99d400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d400c070 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d4013610 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e013d140 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40139c0 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e013e680 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99d4013b20 con 0x7f99e01434d0 2024-05-08T22:35:19.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0067740 con 0x7f99e01434d0 2024-05-08T22:35:19.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f99d4083ca0 con 0x7f99e01434d0 2024-05-08T22:35:19.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.344+0000 7f99de59c700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.348+0000 7f99de59c700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b370 tx=0x7f99d801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.352+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f99d40511d0 con 0x7f99e01434d0 2024-05-08T22:35:19.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.584+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f99e004e700 con 0x7f99e01434d0 2024-05-08T22:35:19.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.884+0000 7f99ceffd700 1 -- 172.21.15.179:0/2671179921 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v124) v1 ==== 142+0+0 (secure 0 0 0) 0x7f99d40547f0 con 0x7f99e01434d0 2024-05-08T22:35:19.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 msgr2=0x7f99c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b370 tx=0x7f99d801c040 comp rx=0 tx=0).stop 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 msgr2=0x7f99e01591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f99d400e730 tx=0x7f99d400ea40 comp rx=0 tx=0).stop 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 shutdown_connections 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f99c0066730 0x7f99c0068bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e0141c40 0x7f99e013e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99e01434d0 0x7f99e01591b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 --2- 172.21.15.179:0/2671179921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99e00ce2a0 0x7f99e013d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 >> 172.21.15.179:0/2671179921 conn(0x7f99e00ce9d0 msgr2=0x7f99e00b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 shutdown_connections 2024-05-08T22:35:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:19.888+0000 7f99df59e700 1 -- 172.21.15.179:0/2671179921 wait complete. 2024-05-08T22:35:19.906 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T22:35:19.907 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-08T22:35:19.907 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:35:19.907 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:20.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 Processor -- start 2024-05-08T22:35:20.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 -- start start 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbb200d07d0 0x7fbb2006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb2015d590 0x7fbb200b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2006d370 con 0x7fbb200b4900 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2015da10 con 0x7fbb200d07d0 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25875700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200b4d90 con 0x7fbb2015d590 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbb200d07d0 0x7fbb2006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25074700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbb200b4900 0x7fbb20160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50732/0 (socket says 172.21.15.179:50732) 2024-05-08T22:35:20.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb25074700 1 -- 172.21.15.179:0/1898574283 learned_addr learned my addr 172.21.15.179:0/1898574283 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb1ffff700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb2015d590 0x7fbb200b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 -- 172.21.15.179:0/1898574283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb2015d590 msgr2=0x7fbb200b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb2015d590 0x7fbb200b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 -- 172.21.15.179:0/1898574283 >> v1:172.21.15.179:6789/0 conn(0x7fbb200b4900 legacy=0x7fbb20160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 -- 172.21.15.179:0/1898574283 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2006d650 con 0x7fbb200d07d0 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb1f7fe700 1 -- 172.21.15.179:0/1898574283 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283949747 0 0) 0x7fbb2006d370 con 0x7fbb200b4900 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb24873700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200d07d0 0x7fbb2006cd00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009850 tx=0x7fbb10009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df57c03c76d4c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb1f7fe700 1 -- 172.21.15.179:0/1898574283 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10011830 con 0x7fbb200d07d0 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb1f7fe700 1 -- 172.21.15.179:0/1898574283 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10011990 con 0x7fbb200d07d0 2024-05-08T22:35:20.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.024+0000 7fbb1f7fe700 1 -- 172.21.15.179:0/1898574283 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10011d40 con 0x7fbb200d07d0 2024-05-08T22:35:20.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- 172.21.15.179:0/1898574283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200d07d0 msgr2=0x7fbb2006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200d07d0 0x7fbb2006cd00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009850 tx=0x7fbb10009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:20.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- 172.21.15.179:0/1898574283 shutdown_connections 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb2015d590 0x7fbb200b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 --2- 172.21.15.179:0/1898574283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200d07d0 0x7fbb2006cd00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- 172.21.15.179:0/1898574283 >> 172.21.15.179:0/1898574283 conn(0x7fbb200b2840 msgr2=0x7fbb200b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- 172.21.15.179:0/1898574283 shutdown_connections 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- 172.21.15.179:0/1898574283 wait complete. 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 Processor -- start 2024-05-08T22:35:20.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.028+0000 7fbb25875700 1 -- start start 2024-05-08T22:35:20.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb200b4900 0x7fbb201546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb200d07d0 0x7fbb201f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20154c30 con 0x7fbb2015d590 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20156f20 con 0x7fbb200d07d0 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20151e70 con 0x7fbb200b4900 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44532/0 (socket says 172.21.15.179:44532) 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 -- 172.21.15.179:0/561279164 learned_addr learned my addr 172.21.15.179:0/561279164 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb24873700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb200b4900 0x7fbb201546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb1ffff700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb200d07d0 0x7fbb201f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 -- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb200b4900 msgr2=0x7fbb201546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb200b4900 0x7fbb201546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 -- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb200d07d0 msgr2=0x7fbb201f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb200d07d0 0x7fbb201f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 -- 172.21.15.179:0/561279164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100094b0 con 0x7fbb2015d590 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25074700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c490 tx=0x7fbb1800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004aa0 con 0x7fbb2015d590 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18004c20 con 0x7fbb2015d590 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb20151840 con 0x7fbb2015d590 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180070d0 con 0x7fbb2015d590 2024-05-08T22:35:20.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb20156c40 con 0x7fbb2015d590 2024-05-08T22:35:20.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.032+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2004e700 con 0x7fbb2015d590 2024-05-08T22:35:20.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.036+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb18007230 con 0x7fbb2015d590 2024-05-08T22:35:20.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.036+0000 7fbb1d7fa700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 0x7fbb00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.036+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fbb18083ad0 con 0x7fbb2015d590 2024-05-08T22:35:20.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.036+0000 7fbb24873700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 0x7fbb00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.036+0000 7fbb24873700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 0x7fbb00068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb100090b0 tx=0x7fbb10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.040+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb18050f70 con 0x7fbb2015d590 2024-05-08T22:35:20.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.216+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb20151fb0 con 0x7fbb2015d590 2024-05-08T22:35:20.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb1d7fa700 1 -- 172.21.15.179:0/561279164 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12773 (secure 0 0 0) 0x7fbb18054590 con 0x7fbb2015d590 2024-05-08T22:35:20.223 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:20.223 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":46,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:18.869334+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/2344889573":"2024-05-09T22:35:15.848887+0000","172.21.15.179:6834/2344889573":"2024-05-09T22:35:15.848887+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 msgr2=0x7fbb00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 0x7fbb00068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb100090b0 tx=0x7fbb10008030 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 msgr2=0x7fbb201503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c490 tx=0x7fbb1800c7a0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 shutdown_connections 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb00066730 0x7fbb00068bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb200d07d0 0x7fbb201f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb2015d590 0x7fbb201503c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.220+0000 7fbb25875700 1 --2- 172.21.15.179:0/561279164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb200b4900 0x7fbb201546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.224+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 >> 172.21.15.179:0/561279164 conn(0x7fbb200b2840 msgr2=0x7fbb200b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.224+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 shutdown_connections 2024-05-08T22:35:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.224+0000 7fbb25875700 1 -- 172.21.15.179:0/561279164 wait complete. 2024-05-08T22:35:20.238 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:20.239 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:20.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.356+0000 7f4dd7fff700 1 Processor -- start 2024-05-08T22:35:20.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.356+0000 7f4dd7fff700 1 -- start start 2024-05-08T22:35:20.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 0x7f4dd80b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80d1910 con 0x7f4dd80d29b0 2024-05-08T22:35:20.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80ce510 con 0x7f4dd80cdb20 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80b6f90 con 0x7f4dd815d160 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44546/0 (socket says 172.21.15.179:44546) 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 -- 172.21.15.179:0/1243062177 learned_addr learned my addr 172.21.15.179:0/1243062177 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 0x7f4dd80b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 -- 172.21.15.179:0/1243062177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 msgr2=0x7f4dd80b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 0x7f4dd80b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 -- 172.21.15.179:0/1243062177 >> v1:172.21.15.195:6789/0 conn(0x7f4dd80d29b0 legacy=0x7f4dd80d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd5ffb700 1 -- 172.21.15.179:0/1243062177 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012411843 0 0) 0x7f4dd80d1910 con 0x7f4dd80d29b0 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 -- 172.21.15.179:0/1243062177 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80d1bf0 con 0x7f4dd80cdb20 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 0x7f4dd80b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:20.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd77fe700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009e50 tx=0x7f4dd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fda83bb06946f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd5ffb700 1 -- 172.21.15.179:0/1243062177 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000e070 con 0x7f4dd80cdb20 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd5ffb700 1 -- 172.21.15.179:0/1243062177 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd000b910 con 0x7f4dd80cdb20 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd5ffb700 1 -- 172.21.15.179:0/1243062177 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000bcc0 con 0x7f4dd80cdb20 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- 172.21.15.179:0/1243062177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd80cdb20 msgr2=0x7f4dd80cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009e50 tx=0x7f4dd000b330 comp rx=0 tx=0).stop 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- 172.21.15.179:0/1243062177 shutdown_connections 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cdfd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/1243062177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd815d160 0x7f4dd80b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- 172.21.15.179:0/1243062177 >> 172.21.15.179:0/1243062177 conn(0x7f4dd80d3270 msgr2=0x7f4dd80d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- 172.21.15.179:0/1243062177 shutdown_connections 2024-05-08T22:35:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.360+0000 7f4dd7fff700 1 -- 172.21.15.179:0/1243062177 wait complete. 2024-05-08T22:35:20.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 Processor -- start 2024-05-08T22:35:20.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 -- start start 2024-05-08T22:35:20.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd80d29b0 0x7f4dd81f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80caf00 con 0x7f4dd815d160 2024-05-08T22:35:20.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80cc0c0 con 0x7f4dd80cdb20 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.364+0000 7f4dd7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80cc3a0 con 0x7f4dd80d29b0 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44560/0 (socket says 172.21.15.179:44560) 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 -- 172.21.15.179:0/535440223 learned_addr learned my addr 172.21.15.179:0/535440223 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd67fc700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd80d29b0 0x7f4dd81f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 -- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd80d29b0 msgr2=0x7f4dd81f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd80d29b0 0x7f4dd81f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 -- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 msgr2=0x7f4dd80cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 -- 172.21.15.179:0/535440223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80094b0 con 0x7f4dd815d160 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:20.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd77fe700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000b680 tx=0x7f4dd0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00048b0 con 0x7f4dd815d160 2024-05-08T22:35:20.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd00099b0 con 0x7f4dd815d160 2024-05-08T22:35:20.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd0009b30 con 0x7f4dd815d160 2024-05-08T22:35:20.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd001c680 con 0x7f4dd815d160 2024-05-08T22:35:20.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd80cc960 con 0x7f4dd815d160 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4dd001c7e0 con 0x7f4dd815d160 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd80ccb20 con 0x7f4dd815d160 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 0x7f4dc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.368+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4dd0085ae0 con 0x7f4dd815d160 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.372+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 0x7f4dc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.372+0000 7f4dd6ffd700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 0x7f4dc4068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80113d0 tx=0x7f4dc8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.380+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4dd00530a0 con 0x7f4dd815d160 2024-05-08T22:35:20.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.548+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4dd804e700 con 0x7f4dd815d160 2024-05-08T22:35:20.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dbf7fe700 1 -- 172.21.15.179:0/535440223 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4dd00566c0 con 0x7f4dd815d160 2024-05-08T22:35:20.554 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 msgr2=0x7f4dc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 0x7f4dc4068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80113d0 tx=0x7f4dc8009240 comp rx=0 tx=0).stop 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 msgr2=0x7f4dd80ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000b680 tx=0x7f4dd0004af0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 shutdown_connections 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4dc4066730 0x7f4dc4068bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dd815d160 0x7f4dd80ca9c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4dd80d29b0 0x7f4dd81f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 --2- 172.21.15.179:0/535440223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4dd80cdb20 0x7f4dd80cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 >> 172.21.15.179:0/535440223 conn(0x7f4dd80d3270 msgr2=0x7f4dd80d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 shutdown_connections 2024-05-08T22:35:20.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.552+0000 7f4dd7fff700 1 -- 172.21.15.179:0/535440223 wait complete. 2024-05-08T22:35:20.569 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-08T22:35:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 Processor -- start 2024-05-08T22:35:20.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 -- start start 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e74161790 0x7f7e74164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e74165d50 con 0x7f7e7415ef80 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7406ad70 con 0x7f7e7406de70 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e78c8e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7406e4b0 con 0x7f7e74161790 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e737fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7e7415ef80 0x7f7e7406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50738/0 (socket says 172.21.15.179:50738) 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e737fe700 1 -- 172.21.15.179:0/576992073 learned_addr learned my addr 172.21.15.179:0/576992073 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.672+0000 7f7e73fff700 1 --2- 172.21.15.179:0/576992073 >> v2:172.21.15.179:3300/0 conn(0x7f7e74161790 0x7f7e74164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347035411 0 0) 0x7f7e74165d50 con 0x7f7e7415ef80 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e600035d0 con 0x7f7e7415ef80 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926221144 0 0) 0x7f7e7406ad70 con 0x7f7e7406de70 2024-05-08T22:35:20.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e73fff700 1 -- 172.21.15.179:0/576992073 >> v1:172.21.15.195:6789/0 conn(0x7f7e7406de70 legacy=0x7f7e7406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e73fff700 1 -- 172.21.15.179:0/576992073 >> v1:172.21.15.179:6789/0 conn(0x7f7e7415ef80 legacy=0x7f7e7406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e73fff700 1 -- 172.21.15.179:0/576992073 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e741654b0 con 0x7f7e74161790 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e73fff700 1 --2- 172.21.15.179:0/576992073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74161790 0x7f7e74164db0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009850 tx=0x7f7e6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b6bbeb30fc9313 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c011830 con 0x7f7e74161790 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c011990 con 0x7f7e74161790 2024-05-08T22:35:20.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e727fc700 1 -- 172.21.15.179:0/576992073 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c011d40 con 0x7f7e74161790 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 -- 172.21.15.179:0/576992073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74161790 msgr2=0x7f7e74164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/576992073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74161790 0x7f7e74164db0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009850 tx=0x7f7e6c009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 -- 172.21.15.179:0/576992073 shutdown_connections 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/576992073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74161790 0x7f7e74164db0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 -- 172.21.15.179:0/576992073 >> 172.21.15.179:0/576992073 conn(0x7f7e740ce9d0 msgr2=0x7f7e740cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 -- 172.21.15.179:0/576992073 shutdown_connections 2024-05-08T22:35:20.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.676+0000 7f7e78c8e700 1 -- 172.21.15.179:0/576992073 wait complete. 2024-05-08T22:35:20.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 Processor -- start 2024-05-08T22:35:20.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 -- start start 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e7415ef80 0x7f7e741f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 0x7f7e74159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741f7bf0 con 0x7f7e7406de70 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741f7f80 con 0x7f7e74161790 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e78c8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741f70f0 con 0x7f7e7415ef80 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44582/0 (socket says 172.21.15.179:44582) 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.680+0000 7f7e737fe700 1 -- 172.21.15.179:0/1384419619 learned_addr learned my addr 172.21.15.179:0/1384419619 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e73fff700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 0x7f7e74159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e72ffd700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e7415ef80 0x7f7e741f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 -- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e7415ef80 msgr2=0x7f7e741f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e7415ef80 0x7f7e741f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 -- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 msgr2=0x7f7e74159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 0x7f7e74159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 -- 172.21.15.179:0/1384419619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0094b0 con 0x7f7e7406de70 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e73fff700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 0x7f7e74159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e737fe700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c520 tx=0x7f7e5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c005d90 con 0x7f7e7406de70 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0070d0 con 0x7f7e7406de70 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c007440 con 0x7f7e7406de70 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e741f7350 con 0x7f7e7406de70 2024-05-08T22:35:20.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741f7910 con 0x7f7e7406de70 2024-05-08T22:35:20.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.684+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e74067740 con 0x7f7e7406de70 2024-05-08T22:35:20.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.688+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e5c0075a0 con 0x7f7e7406de70 2024-05-08T22:35:20.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.688+0000 7f7e6bfff700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 0x7f7e54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:20.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.688+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7e5c083270 con 0x7f7e7406de70 2024-05-08T22:35:20.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.688+0000 7f7e72ffd700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 0x7f7e54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:20.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.688+0000 7f7e72ffd700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 0x7f7e54068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e64002230 tx=0x7f7e64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:20.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.696+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e5c0508e0 con 0x7f7e7406de70 2024-05-08T22:35:20.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:20.924+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f7e741f7d70 con 0x7f7e7406de70 2024-05-08T22:35:21.769 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:35:21.772 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:35:21.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.884+0000 7f7e6bfff700 1 -- 172.21.15.179:0/1384419619 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v126) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7e5c053f00 con 0x7f7e7406de70 2024-05-08T22:35:21.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 msgr2=0x7f7e54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:21.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 0x7f7e54068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e64002230 tx=0x7f7e64009040 comp rx=0 tx=0).stop 2024-05-08T22:35:21.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 msgr2=0x7f7e74158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c520 tx=0x7f7e5c00c830 comp rx=0 tx=0).stop 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 shutdown_connections 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e54066730 0x7f7e54068bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e74161790 0x7f7e74159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e7415ef80 0x7f7e741f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 --2- 172.21.15.179:0/1384419619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e7406de70 0x7f7e74158050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 >> 172.21.15.179:0/1384419619 conn(0x7f7e740ce9d0 msgr2=0x7f7e74161ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 shutdown_connections 2024-05-08T22:35:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:21.892+0000 7f7e78c8e700 1 -- 172.21.15.179:0/1384419619 wait complete. 2024-05-08T22:35:21.912 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:35:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 Processor -- start 2024-05-08T22:35:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 -- start start 2024-05-08T22:35:22.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd43c0b8a80 0x7fd43c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd43c0ce0e0 0x7fd43c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c0b8ee0 con 0x7fd43c15d0c0 2024-05-08T22:35:22.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c0ce570 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44113e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c15d4a0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd44093d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd43c0ce0e0 0x7fd43c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd43bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd43c15d0c0 0x7fd43c0d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50748/0 (socket says 172.21.15.179:50748) 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.028+0000 7fd43bfff700 1 -- 172.21.15.179:0/2655365575 learned_addr learned my addr 172.21.15.179:0/2655365575 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd43b7fe700 1 --2- 172.21.15.179:0/2655365575 >> v2:172.21.15.195:3300/0 conn(0x7fd43c0b8a80 0x7fd43c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44093d700 1 -- 172.21.15.179:0/2655365575 >> v2:172.21.15.195:3300/0 conn(0x7fd43c0b8a80 msgr2=0x7fd43c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44093d700 1 --2- 172.21.15.179:0/2655365575 >> v2:172.21.15.195:3300/0 conn(0x7fd43c0b8a80 0x7fd43c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44093d700 1 -- 172.21.15.179:0/2655365575 >> v1:172.21.15.179:6789/0 conn(0x7fd43c15d0c0 legacy=0x7fd43c0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44093d700 1 -- 172.21.15.179:0/2655365575 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c06abb0 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44093d700 1 --2- 172.21.15.179:0/2655365575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c06b370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd43c160b00 tx=0x7fd43400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86919f1fea87b4d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd43affd700 1 -- 172.21.15.179:0/2655365575 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400e070 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd43affd700 1 -- 172.21.15.179:0/2655365575 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400bb20 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd43affd700 1 -- 172.21.15.179:0/2655365575 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434012430 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 -- 172.21.15.179:0/2655365575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0ce0e0 msgr2=0x7fd43c06b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 --2- 172.21.15.179:0/2655365575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c06b370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd43c160b00 tx=0x7fd43400b330 comp rx=0 tx=0).stop 2024-05-08T22:35:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 -- 172.21.15.179:0/2655365575 shutdown_connections 2024-05-08T22:35:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 --2- 172.21.15.179:0/2655365575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c06b370 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 --2- 172.21.15.179:0/2655365575 >> v2:172.21.15.195:3300/0 conn(0x7fd43c0b8a80 0x7fd43c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 -- 172.21.15.179:0/2655365575 >> 172.21.15.179:0/2655365575 conn(0x7fd43c0d2ed0 msgr2=0x7fd43c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 -- 172.21.15.179:0/2655365575 shutdown_connections 2024-05-08T22:35:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.032+0000 7fd44113e700 1 -- 172.21.15.179:0/2655365575 wait complete. 2024-05-08T22:35:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 Processor -- start 2024-05-08T22:35:22.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 -- start start 2024-05-08T22:35:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd43c15d0c0 0x7fd43c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c15b7b0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c15a750 con 0x7fd43c0ce0e0 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd44113e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c15a9f0 con 0x7fd43c15d0c0 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44588/0 (socket says 172.21.15.179:44588) 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.036+0000 7fd43bfff700 1 -- 172.21.15.179:0/3385848792 learned_addr learned my addr 172.21.15.179:0/3385848792 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43b7fe700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd44093d700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd43c15d0c0 0x7fd43c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 -- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd43c15d0c0 msgr2=0x7fd43c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd43c15d0c0 0x7fd43c1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 -- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 msgr2=0x7fd43c157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 -- 172.21.15.179:0/3385848792 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009a70 con 0x7fd43c0b8a80 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43b7fe700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43bfff700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c410 tx=0x7fd43000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430005d90 con 0x7fd43c0b8a80 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4300070d0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430007460 con 0x7fd43c0b8a80 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c15acd0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c15b290 con 0x7fd43c0b8a80 2024-05-08T22:35:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c04e700 con 0x7fd43c0b8a80 2024-05-08T22:35:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4300075c0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 0x7fd428068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd430083350 con 0x7fd43c0b8a80 2024-05-08T22:35:22.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.040+0000 7fd43b7fe700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 0x7fd428068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.044+0000 7fd43b7fe700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 0x7fd428068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0d3820 tx=0x7fd42c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.048+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd430050910 con 0x7fd43c0b8a80 2024-05-08T22:35:22.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.220+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd43c0676e0 con 0x7fd43c0b8a80 2024-05-08T22:35:22.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.228+0000 7fd438ff9700 1 -- 172.21.15.179:0/3385848792 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd430081e60 con 0x7fd43c0b8a80 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 msgr2=0x7fd428068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 0x7fd428068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0d3820 tx=0x7fd42c008040 comp rx=0 tx=0).stop 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 msgr2=0x7fd43c14fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c410 tx=0x7fd43000c7d0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 shutdown_connections 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd428066730 0x7fd428068bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd43c0ce0e0 0x7fd43c157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd43c15d0c0 0x7fd43c1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 --2- 172.21.15.179:0/3385848792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd43c0b8a80 0x7fd43c14fa50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 >> 172.21.15.179:0/3385848792 conn(0x7fd43c0d2ed0 msgr2=0x7fd43c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 shutdown_connections 2024-05-08T22:35:22.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.232+0000 7fd44113e700 1 -- 172.21.15.179:0/3385848792 wait complete. 2024-05-08T22:35:22.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:35:22.248 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:35:22.267 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:35:22.288 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:35:22.289 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:35:22.289 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:22.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:35:22.297 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:35:22.297 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:22.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:35:22.303 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:35:22.303 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:22.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:35:22.310 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:35:22.310 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:35:22.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:35:22.316 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:35:22.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:22.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 Processor -- start 2024-05-08T22:35:22.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 -- start start 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fde7406de70 0x7fde74161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde74165d50 con 0x7fde7415ef80 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7406ad70 con 0x7fde74164db0 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde73fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde74165220 con 0x7fde7406de70 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde72ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fde7415ef80 0x7fde7406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50756/0 (socket says 172.21.15.179:50756) 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde72ffd700 1 -- 172.21.15.179:0/1139631512 learned_addr learned my addr 172.21.15.179:0/1139631512 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:22.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.432+0000 7fde727fc700 1 --2- 172.21.15.179:0/1139631512 >> v2:172.21.15.195:3300/0 conn(0x7fde7406de70 0x7fde74161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde71ffb700 1 -- 172.21.15.179:0/1139631512 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745018836 0 0) 0x7fde74165d50 con 0x7fde7415ef80 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde71ffb700 1 -- 172.21.15.179:0/1139631512 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde600035d0 con 0x7fde7415ef80 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde727fc700 1 -- 172.21.15.179:0/1139631512 >> v1:172.21.15.195:6789/0 conn(0x7fde74164db0 legacy=0x7fde7406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde727fc700 1 -- 172.21.15.179:0/1139631512 >> v1:172.21.15.179:6789/0 conn(0x7fde7415ef80 legacy=0x7fde7406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde727fc700 1 -- 172.21.15.179:0/1139631512 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7406e4b0 con 0x7fde7406de70 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde727fc700 1 --2- 172.21.15.179:0/1139631512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde7406de70 0x7fde74161790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009850 tx=0x7fde5c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526009e174d29ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde71ffb700 1 -- 172.21.15.179:0/1139631512 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c011830 con 0x7fde7406de70 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde71ffb700 1 -- 172.21.15.179:0/1139631512 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c011990 con 0x7fde7406de70 2024-05-08T22:35:22.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde71ffb700 1 -- 172.21.15.179:0/1139631512 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c011d00 con 0x7fde7406de70 2024-05-08T22:35:22.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 -- 172.21.15.179:0/1139631512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde7406de70 msgr2=0x7fde74161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 --2- 172.21.15.179:0/1139631512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde7406de70 0x7fde74161790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009850 tx=0x7fde5c009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:22.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 -- 172.21.15.179:0/1139631512 shutdown_connections 2024-05-08T22:35:22.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 --2- 172.21.15.179:0/1139631512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde7406de70 0x7fde74161790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 -- 172.21.15.179:0/1139631512 >> 172.21.15.179:0/1139631512 conn(0x7fde740ce9d0 msgr2=0x7fde740cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 -- 172.21.15.179:0/1139631512 shutdown_connections 2024-05-08T22:35:22.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.436+0000 7fde73fff700 1 -- 172.21.15.179:0/1139631512 wait complete. 2024-05-08T22:35:22.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 Processor -- start 2024-05-08T22:35:22.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- start start 2024-05-08T22:35:22.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde7406de70 0x7fde7414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde74164db0 0x7fde74162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde740c4a10 con 0x7fde7415ef80 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde740ca830 con 0x7fde74164db0 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde740ca270 con 0x7fde7406de70 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44590/0 (socket says 172.21.15.179:44590) 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 -- 172.21.15.179:0/2689335984 learned_addr learned my addr 172.21.15.179:0/2689335984 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde72ffd700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde7406de70 0x7fde7414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde737fe700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde74164db0 0x7fde74162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 -- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde7406de70 msgr2=0x7fde7414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde7406de70 0x7fde7414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 -- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde74164db0 msgr2=0x7fde74162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde74164db0 0x7fde74162af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 -- 172.21.15.179:0/2689335984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0094b0 con 0x7fde7415ef80 2024-05-08T22:35:22.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde727fc700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009ba0 tx=0x7fde5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004550 con 0x7fde7415ef80 2024-05-08T22:35:22.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c01f070 con 0x7fde7415ef80 2024-05-08T22:35:22.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c003900 con 0x7fde7415ef80 2024-05-08T22:35:22.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde740ca550 con 0x7fde7415ef80 2024-05-08T22:35:22.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.440+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde740c9e80 con 0x7fde7415ef80 2024-05-08T22:35:22.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.444+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde740c4490 con 0x7fde7415ef80 2024-05-08T22:35:22.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.444+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde5c003a60 con 0x7fde7415ef80 2024-05-08T22:35:22.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.444+0000 7fde6b7fe700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 0x7fde54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.444+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fde5c021070 con 0x7fde7415ef80 2024-05-08T22:35:22.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.444+0000 7fde72ffd700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 0x7fde54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.448+0000 7fde72ffd700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 0x7fde54068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fde6c001560 tx=0x7fde6c001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.452+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fde740c4490 con 0x7fde7415ef80 2024-05-08T22:35:22.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.664+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde740c4490 con 0x7fde7415ef80 2024-05-08T22:35:22.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde6b7fe700 1 -- 172.21.15.179:0/2689335984 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fde5c02adf0 con 0x7fde7415ef80 2024-05-08T22:35:22.670 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:22.670 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3445711709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3445711709},{"type":"v1","addr":"172.21.15.195:6836","nonce":3445711709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2667554685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":2667554685},{"type":"v1","addr":"172.21.15.195:6837","nonce":2667554685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1034952406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1034952406},{"type":"v1","addr":"172.21.15.179:6833","nonce":1034952406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4283354259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4283354259},{"type":"v1","addr":"172.21.15.179:6835","nonce":4283354259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-05-08T22:35:22.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 msgr2=0x7fde54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 0x7fde54068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fde6c001560 tx=0x7fde6c001a90 comp rx=0 tx=0).stop 2024-05-08T22:35:22.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 msgr2=0x7fde740c9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009ba0 tx=0x7fde5c004770 comp rx=0 tx=0).stop 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 shutdown_connections 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde54066730 0x7fde54068bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde74164db0 0x7fde74162af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7415ef80 0x7fde740c9940 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 --2- 172.21.15.179:0/2689335984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde7406de70 0x7fde7414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 >> 172.21.15.179:0/2689335984 conn(0x7fde740ce9d0 msgr2=0x7fde74160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 shutdown_connections 2024-05-08T22:35:22.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.668+0000 7fde73fff700 1 -- 172.21.15.179:0/2689335984 wait complete. 2024-05-08T22:35:22.674 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 19 2024-05-08T22:35:22.684 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:35:22.685 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:35:22.685 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:35:22.685 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:35:22.685 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:35:22.685 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:35:22.685 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:35:22.686 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:35:22.686 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:35:22.686 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:35:22.686 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:35:22.686 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:35:22.690 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:35:22.690 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:35:22.690 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:35:22.691 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:35:22.698 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:35:22.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:35:22.700 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:35:22.700 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:22.702 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:35:22.702 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:35:22.703 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:35:22.703 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:35:22.705 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:35:22.705 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:22.706 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:35:22.706 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:22.707 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:35:22.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:35:22.709 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:35:22.709 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:22.710 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:35:22.769 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:35:22.775 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:35:22.802 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:35:22.805 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:35:22.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.808+0000 7f3db36c3700 1 Processor -- start 2024-05-08T22:35:22.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.808+0000 7f3db36c3700 1 -- start start 2024-05-08T22:35:22.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac159930 0x7f3dac06bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3dac06c100 0x7f3dac0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dac0b4610 0x7f3dac06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac06c560 con 0x7f3dac0b4610 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0ca620 con 0x7f3dac06c100 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0ca900 con 0x7f3dac159930 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dac0b4610 0x7f3dac06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dac0b4610 0x7f3dac06ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44654/0 (socket says 172.21.15.179:44654) 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 -- 172.21.15.179:0/3792426954 learned_addr learned my addr 172.21.15.179:0/3792426954 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db145f700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac159930 0x7f3dac06bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db0c5e700 1 --2- 172.21.15.179:0/3792426954 >> v2:172.21.15.195:3300/0 conn(0x7f3dac06c100 0x7f3dac0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 -- 172.21.15.179:0/3792426954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac159930 msgr2=0x7f3dac06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac159930 0x7f3dac06bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 -- 172.21.15.179:0/3792426954 >> v2:172.21.15.195:3300/0 conn(0x7f3dac06c100 msgr2=0x7f3dac0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 --2- 172.21.15.179:0/3792426954 >> v2:172.21.15.195:3300/0 conn(0x7f3dac06c100 0x7f3dac0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 -- 172.21.15.179:0/3792426954 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac0cabe0 con 0x7f3dac0b4610 2024-05-08T22:35:22.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db1c60700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac0b4610 0x7f3dac06ceb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dac072200 tx=0x7f3da800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1634153c11ec2fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3da3fff700 1 -- 172.21.15.179:0/3792426954 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800dd30 con 0x7f3dac0b4610 2024-05-08T22:35:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3da3fff700 1 -- 172.21.15.179:0/3792426954 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800de90 con 0x7f3dac0b4610 2024-05-08T22:35:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3da3fff700 1 -- 172.21.15.179:0/3792426954 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8010690 con 0x7f3dac0b4610 2024-05-08T22:35:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- 172.21.15.179:0/3792426954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac0b4610 msgr2=0x7f3dac06ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac0b4610 0x7f3dac06ceb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dac072200 tx=0x7f3da800be90 comp rx=0 tx=0).stop 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- 172.21.15.179:0/3792426954 shutdown_connections 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac0b4610 0x7f3dac06ceb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3792426954 >> v2:172.21.15.195:3300/0 conn(0x7f3dac06c100 0x7f3dac0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3792426954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac159930 0x7f3dac06bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.812+0000 7f3db36c3700 1 -- 172.21.15.179:0/3792426954 >> 172.21.15.179:0/3792426954 conn(0x7f3dac070e50 msgr2=0x7f3dac071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- 172.21.15.179:0/3792426954 shutdown_connections 2024-05-08T22:35:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- 172.21.15.179:0/3792426954 wait complete. 2024-05-08T22:35:22.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 Processor -- start 2024-05-08T22:35:22.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- start start 2024-05-08T22:35:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 0x7f3dac150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dac159930 0x7f3dac157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac14db50 con 0x7f3dac06c100 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac15a280 con 0x7f3dac159930 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db36c3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac15a560 con 0x7f3dac0b4610 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44666/0 (socket says 172.21.15.179:44666) 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 -- 172.21.15.179:0/3078323072 learned_addr learned my addr 172.21.15.179:0/3078323072 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db0c5e700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 0x7f3dac150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db1c60700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dac159930 0x7f3dac157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 -- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 msgr2=0x7f3dac150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 0x7f3dac150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 -- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dac159930 msgr2=0x7f3dac157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dac159930 0x7f3dac157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 -- 172.21.15.179:0/3078323072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da800b810 con 0x7f3dac06c100 2024-05-08T22:35:22.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db0c5e700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 0x7f3dac150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:22.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.816+0000 7f3db145f700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3da400efc0 tx=0x7f3da400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4007a70 con 0x7f3dac06c100 2024-05-08T22:35:22.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da400f040 con 0x7f3dac06c100 2024-05-08T22:35:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac15a840 con 0x7f3dac06c100 2024-05-08T22:35:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4006950 con 0x7f3dac06c100 2024-05-08T22:35:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac1590a0 con 0x7f3dac06c100 2024-05-08T22:35:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3da4006ab0 con 0x7f3dac06c100 2024-05-08T22:35:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3da1ffb700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:22.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.820+0000 7f3db0c5e700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:22.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.824+0000 7f3db0c5e700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 0x7f3d90068bf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0cc750 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:22.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.824+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3da4055c60 con 0x7f3dac06c100 2024-05-08T22:35:22.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.824+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac067770 con 0x7f3dac06c100 2024-05-08T22:35:22.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:22.828+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3da40518a0 con 0x7f3dac06c100 2024-05-08T22:35:23.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.008+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3dac073290 con 0x7f3dac06c100 2024-05-08T22:35:23.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.900+0000 7f3da1ffb700 1 -- 172.21.15.179:0/3078323072 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3da4054ec0 con 0x7f3dac06c100 2024-05-08T22:35:23.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 msgr2=0x7f3d90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 0x7f3d90068bf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0cc750 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 msgr2=0x7f3dac158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3da400efc0 tx=0x7f3da400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 shutdown_connections 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dac159930 0x7f3dac157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3dac0b4610 0x7f3dac150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 --2- 172.21.15.179:0/3078323072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dac06c100 0x7f3dac158b60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 >> 172.21.15.179:0/3078323072 conn(0x7f3dac070e50 msgr2=0x7f3dac0cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 shutdown_connections 2024-05-08T22:35:23.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:23.904+0000 7f3db36c3700 1 -- 172.21.15.179:0/3078323072 wait complete. 2024-05-08T22:35:23.910 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:35:23.926 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:35:24.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 Processor -- start 2024-05-08T22:35:24.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 -- start start 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27080add40 0x7f270806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2708161ac0 con 0x7f27080cc310 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27080cdaa0 con 0x7f27080ce060 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270ea3f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27080ae550 con 0x7f27080add40 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f27077fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f27080cc310 0x7f270806b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50776/0 (socket says 172.21.15.179:50776) 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f27077fe700 1 -- 172.21.15.179:0/2980614252 learned_addr learned my addr 172.21.15.179:0/2980614252 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270cfdc700 1 --2- 172.21.15.179:0/2980614252 >> v2:172.21.15.179:3300/0 conn(0x7f27080add40 0x7f270806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109900034 0 0) 0x7f2708161ac0 con 0x7f27080cc310 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f40035d0 con 0x7f27080cc310 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270cfdc700 1 -- 172.21.15.179:0/2980614252 >> v1:172.21.15.195:6789/0 conn(0x7f27080ce060 legacy=0x7f27080ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904186870 0 0) 0x7f27080cdaa0 con 0x7f27080ce060 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270cfdc700 1 -- 172.21.15.179:0/2980614252 >> v1:172.21.15.179:6789/0 conn(0x7f27080cc310 legacy=0x7f270806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270cfdc700 1 -- 172.21.15.179:0/2980614252 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270806adc0 con 0x7f27080add40 2024-05-08T22:35:24.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f270cfdc700 1 --2- 172.21.15.179:0/2980614252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f270806ce50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009850 tx=0x7f26f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10a4e8a332eb8cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8004230 con 0x7f27080add40 2024-05-08T22:35:24.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f8004390 con 0x7f27080add40 2024-05-08T22:35:24.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.044+0000 7f2706ffd700 1 -- 172.21.15.179:0/2980614252 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80036c0 con 0x7f27080add40 2024-05-08T22:35:24.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- 172.21.15.179:0/2980614252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 msgr2=0x7f270806ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 --2- 172.21.15.179:0/2980614252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f270806ce50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009850 tx=0x7f26f8009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- 172.21.15.179:0/2980614252 shutdown_connections 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 --2- 172.21.15.179:0/2980614252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f270806ce50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- 172.21.15.179:0/2980614252 >> 172.21.15.179:0/2980614252 conn(0x7f27080cb660 msgr2=0x7f27080cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- 172.21.15.179:0/2980614252 shutdown_connections 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- 172.21.15.179:0/2980614252 wait complete. 2024-05-08T22:35:24.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 Processor -- start 2024-05-08T22:35:24.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- start start 2024-05-08T22:35:24.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27080cc310 0x7f270806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27080ce060 0x7f270806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080b1840 con 0x7f27080add40 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080b1a90 con 0x7f27080cc310 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270ea3f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27080b1d70 con 0x7f27080ce060 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f2707fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f2707fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44680/0 (socket says 172.21.15.179:44680) 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f2707fff700 1 -- 172.21.15.179:0/1429182029 learned_addr learned my addr 172.21.15.179:0/1429182029 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f27077fe700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27080cc310 0x7f270806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.048+0000 7f270cfdc700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27080ce060 0x7f270806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 -- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27080ce060 msgr2=0x7f270806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27080ce060 0x7f270806db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 -- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27080cc310 msgr2=0x7f270806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27080cc310 0x7f270806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 -- 172.21.15.179:0/1429182029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80094b0 con 0x7f27080add40 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2707fff700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c520 tx=0x7f26f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00054e0 con 0x7f27080add40 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f00036a0 con 0x7f27080add40 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27080b2050 con 0x7f27080add40 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003a50 con 0x7f27080add40 2024-05-08T22:35:24.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27080b2850 con 0x7f27080add40 2024-05-08T22:35:24.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2708067770 con 0x7f27080add40 2024-05-08T22:35:24.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.052+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26f0003bb0 con 0x7f27080add40 2024-05-08T22:35:24.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.056+0000 7f2704ff9700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 0x7f26e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.056+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f26f0083390 con 0x7f27080add40 2024-05-08T22:35:24.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.056+0000 7f27077fe700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 0x7f26e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.056+0000 7f27077fe700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 0x7f26e8068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2708161360 tx=0x7f2700009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.060+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f26f0050870 con 0x7f27080add40 2024-05-08T22:35:24.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2708073290 con 0x7f27080add40 2024-05-08T22:35:24.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f2704ff9700 1 -- 172.21.15.179:0/1429182029 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f26f0003e60 con 0x7f27080add40 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:35:24.295 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.296 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.297 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.298 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.299 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.300 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:35:24.302 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:35:24.303 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:35:24.304 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:35:24.305 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.306 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:35:24.307 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:35:24.308 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:35:24.308 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.308 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:24.308 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:35:24.308 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 msgr2=0x7f26e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 0x7f26e8068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2708161360 tx=0x7f2700009040 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 msgr2=0x7f27081588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c520 tx=0x7f26f000c830 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 shutdown_connections 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.292+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f26e8066730 0x7f26e8068bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27080cc310 0x7f270806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27080ce060 0x7f270806db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 --2- 172.21.15.179:0/1429182029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27080add40 0x7f27081588e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 >> 172.21.15.179:0/1429182029 conn(0x7f27080cb660 msgr2=0x7f27080ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 shutdown_connections 2024-05-08T22:35:24.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.296+0000 7f270ea3f700 1 -- 172.21.15.179:0/1429182029 wait complete. 2024-05-08T22:35:24.311 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 Processor -- start 2024-05-08T22:35:24.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- start start 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff328159440 0x7ff3280b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3280cbd50 con 0x7ff32815ddb0 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3280cc030 con 0x7ff32815d900 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3280cbb40 con 0x7ff328159440 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff327fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff328159440 0x7ff3280b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff3277fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff32815ddb0 0x7ff32815d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50782/0 (socket says 172.21.15.179:50782) 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff3277fe700 1 -- 172.21.15.179:0/1266837414 learned_addr learned my addr 172.21.15.179:0/1266837414 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff327fff700 1 -- 172.21.15.179:0/1266837414 >> v1:172.21.15.195:6789/0 conn(0x7ff32815d900 legacy=0x7ff32806c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff327fff700 1 -- 172.21.15.179:0/1266837414 >> v1:172.21.15.179:6789/0 conn(0x7ff32815ddb0 legacy=0x7ff32815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff327fff700 1 -- 172.21.15.179:0/1266837414 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3280cc310 con 0x7ff328159440 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff326ffd700 1 -- 172.21.15.179:0/1266837414 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248747076 0 0) 0x7ff3280cbd50 con 0x7ff32815ddb0 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff327fff700 1 --2- 172.21.15.179:0/1266837414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff328159440 0x7ff3280b1510 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff31800baf0 tx=0x7ff31800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27be52fbe1b3845 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff326ffd700 1 -- 172.21.15.179:0/1266837414 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800dc20 con 0x7ff328159440 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff326ffd700 1 -- 172.21.15.179:0/1266837414 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800dda0 con 0x7ff328159440 2024-05-08T22:35:24.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff326ffd700 1 -- 172.21.15.179:0/1266837414 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318017650 con 0x7ff328159440 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- 172.21.15.179:0/1266837414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff328159440 msgr2=0x7ff3280b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 --2- 172.21.15.179:0/1266837414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff328159440 0x7ff3280b1510 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff31800baf0 tx=0x7ff31800be00 comp rx=0 tx=0).stop 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- 172.21.15.179:0/1266837414 shutdown_connections 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 --2- 172.21.15.179:0/1266837414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff328159440 0x7ff3280b1510 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- 172.21.15.179:0/1266837414 >> 172.21.15.179:0/1266837414 conn(0x7ff328098a70 msgr2=0x7ff3280b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- 172.21.15.179:0/1266837414 shutdown_connections 2024-05-08T22:35:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.424+0000 7ff32d650700 1 -- 172.21.15.179:0/1266837414 wait complete. 2024-05-08T22:35:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.428+0000 7ff32d650700 1 Processor -- start 2024-05-08T22:35:24.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.428+0000 7ff32d650700 1 -- start start 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328159440 0x7ff328147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff32815ddb0 0x7ff32806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280c4ab0 con 0x7ff32815d900 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280c4d90 con 0x7ff32815ddb0 2024-05-08T22:35:24.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280c5070 con 0x7ff328159440 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44684/0 (socket says 172.21.15.179:44684) 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 -- 172.21.15.179:0/237410947 learned_addr learned my addr 172.21.15.179:0/237410947 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32ce4f700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff32815ddb0 0x7ff32806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff327fff700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328159440 0x7ff328147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 -- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328159440 msgr2=0x7ff328147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328159440 0x7ff328147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 -- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff32815ddb0 msgr2=0x7ff32806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff32815ddb0 0x7ff32806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 -- 172.21.15.179:0/237410947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31800b750 con 0x7ff32815d900 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff3277fe700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c4a0 tx=0x7ff31c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c004ca0 con 0x7ff32815d900 2024-05-08T22:35:24.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c004e00 con 0x7ff32815d900 2024-05-08T22:35:24.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3280c5350 con 0x7ff32815d900 2024-05-08T22:35:24.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c007310 con 0x7ff32815d900 2024-05-08T22:35:24.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.432+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff328147b90 con 0x7ff32815d900 2024-05-08T22:35:24.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3280c5870 con 0x7ff32815d900 2024-05-08T22:35:24.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff31c007470 con 0x7ff32815d900 2024-05-08T22:35:24.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff324ff9700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 0x7ff314068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff31c083b60 con 0x7ff32815d900 2024-05-08T22:35:24.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff327fff700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 0x7ff314068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.436+0000 7ff327fff700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 0x7ff314068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff31800baf0 tx=0x7ff318014000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.440+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff3280c5870 con 0x7ff32815d900 2024-05-08T22:35:24.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.612+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3280c5870 con 0x7ff32815d900 2024-05-08T22:35:24.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.612+0000 7ff324ff9700 1 -- 172.21.15.179:0/237410947 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12642 (secure 0 0 0) 0x7ff3280c5870 con 0x7ff32815d900 2024-05-08T22:35:24.617 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:24.617 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":51,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:23.888809+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 msgr2=0x7ff314068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 0x7ff314068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff31800baf0 tx=0x7ff318014000 comp rx=0 tx=0).stop 2024-05-08T22:35:24.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 msgr2=0x7ff3280c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c4a0 tx=0x7ff31c00c7b0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 shutdown_connections 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff314066730 0x7ff314068bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff32815ddb0 0x7ff32806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff32815d900 0x7ff3280c2b20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 --2- 172.21.15.179:0/237410947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328159440 0x7ff328147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 >> 172.21.15.179:0/237410947 conn(0x7ff328098a70 msgr2=0x7ff32806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 shutdown_connections 2024-05-08T22:35:24.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.616+0000 7ff32d650700 1 -- 172.21.15.179:0/237410947 wait complete. 2024-05-08T22:35:24.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:24.635 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:24.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 Processor -- start 2024-05-08T22:35:24.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 -- start start 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f74800cd7d0 0x7f748006b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7480162960 con 0x7f748006ccb0 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74801623a0 con 0x7f74800cdbb0 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485822700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7480162680 con 0x7f74800cd7d0 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485021700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f748006ccb0 0x7f74800d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50790/0 (socket says 172.21.15.179:50790) 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7485021700 1 -- 172.21.15.179:0/4240298972 learned_addr learned my addr 172.21.15.179:0/4240298972 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:24.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7484820700 1 --2- 172.21.15.179:0/4240298972 >> v2:172.21.15.195:3300/0 conn(0x7f74800cd7d0 0x7f748006b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f747f7fe700 1 -- 172.21.15.179:0/4240298972 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132958814 0 0) 0x7f7480162960 con 0x7f748006ccb0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f747f7fe700 1 -- 172.21.15.179:0/4240298972 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c0035d0 con 0x7f748006ccb0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7484820700 1 -- 172.21.15.179:0/4240298972 >> v1:172.21.15.195:6789/0 conn(0x7f74800cdbb0 legacy=0x7f7480161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7484820700 1 -- 172.21.15.179:0/4240298972 >> v1:172.21.15.179:6789/0 conn(0x7f748006ccb0 legacy=0x7f74800d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7484820700 1 -- 172.21.15.179:0/4240298972 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748006acc0 con 0x7f74800cd7d0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f7484820700 1 --2- 172.21.15.179:0/4240298972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cd7d0 0x7f748006b4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7470009850 tx=0x7f7470009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5c9860bec0d088 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.752+0000 7f747f7fe700 1 -- 172.21.15.179:0/4240298972 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470011830 con 0x7f74800cd7d0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f747f7fe700 1 -- 172.21.15.179:0/4240298972 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470011990 con 0x7f74800cd7d0 2024-05-08T22:35:24.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f747f7fe700 1 -- 172.21.15.179:0/4240298972 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470011d40 con 0x7f74800cd7d0 2024-05-08T22:35:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 -- 172.21.15.179:0/4240298972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cd7d0 msgr2=0x7f748006b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 --2- 172.21.15.179:0/4240298972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cd7d0 0x7f748006b4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7470009850 tx=0x7f7470009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:24.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 -- 172.21.15.179:0/4240298972 shutdown_connections 2024-05-08T22:35:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 --2- 172.21.15.179:0/4240298972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cd7d0 0x7f748006b4c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 -- 172.21.15.179:0/4240298972 >> 172.21.15.179:0/4240298972 conn(0x7f74800d3270 msgr2=0x7f74800d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 -- 172.21.15.179:0/4240298972 shutdown_connections 2024-05-08T22:35:24.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 -- 172.21.15.179:0/4240298972 wait complete. 2024-05-08T22:35:24.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.756+0000 7f7485822700 1 Processor -- start 2024-05-08T22:35:24.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- start start 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800cd7d0 0x7f74801eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cdbb0 0x7f74800cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801515f0 con 0x7f748006ccb0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801ecb30 con 0x7f74800cdbb0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74801ebc80 con 0x7f74800cd7d0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44686/0 (socket says 172.21.15.179:44686) 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 -- 172.21.15.179:0/3499688892 learned_addr learned my addr 172.21.15.179:0/3499688892 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f747ffff700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800cd7d0 0x7f74801eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485021700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cdbb0 0x7f74800cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 -- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800cd7d0 msgr2=0x7f74801eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800cd7d0 0x7f74801eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 -- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cdbb0 msgr2=0x7f74800cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cdbb0 0x7f74800cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 -- 172.21.15.179:0/3499688892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700094b0 con 0x7f748006ccb0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7484820700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74700090b0 tx=0x7f7470004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747001aa10 con 0x7f748006ccb0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747001f070 con 0x7f748006ccb0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748006bb80 con 0x7f748006ccb0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747001adb0 con 0x7f748006ccb0 2024-05-08T22:35:24.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.760+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74801eb6c0 con 0x7f748006ccb0 2024-05-08T22:35:24.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74801513c0 con 0x7f748006ccb0 2024-05-08T22:35:24.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f747001b430 con 0x7f748006ccb0 2024-05-08T22:35:24.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f747d7fa700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 0x7f7460068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:24.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7470084fe0 con 0x7f748006ccb0 2024-05-08T22:35:24.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f747ffff700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 0x7f7460068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:24.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.764+0000 7f747ffff700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 0x7f7460068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74800cd370 tx=0x7f7468009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:24.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.772+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f74801513c0 con 0x7f748006ccb0 2024-05-08T22:35:24.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.940+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f748004e700 con 0x7f748006ccb0 2024-05-08T22:35:24.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.940+0000 7f747d7fa700 1 -- 172.21.15.179:0/3499688892 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7470052570 con 0x7f748006ccb0 2024-05-08T22:35:24.945 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:35:24.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 msgr2=0x7f7460068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 0x7f7460068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74800cd370 tx=0x7f7468009040 comp rx=0 tx=0).stop 2024-05-08T22:35:24.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 msgr2=0x7f74801505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:24.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74700090b0 tx=0x7f7470004770 comp rx=0 tx=0).stop 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 shutdown_connections 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7460066730 0x7f7460068bf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74800cdbb0 0x7f74800cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800cd7d0 0x7f74801eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 --2- 172.21.15.179:0/3499688892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f748006ccb0 0x7f74801505e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 >> 172.21.15.179:0/3499688892 conn(0x7f74800d3270 msgr2=0x7f7480162b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 shutdown_connections 2024-05-08T22:35:24.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:24.944+0000 7f7485822700 1 -- 172.21.15.179:0/3499688892 wait complete. 2024-05-08T22:35:24.960 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:35:25.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 Processor -- start 2024-05-08T22:35:25.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 -- start start 2024-05-08T22:35:25.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4f7c15d5b0 0x7f4f7c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f7c0b2120 0x7f4f7c0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c06afc0 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c0ce820 con 0x7f4f7c0b2120 2024-05-08T22:35:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c15d990 con 0x7f4f7c15d5b0 2024-05-08T22:35:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4f7c0ce3b0 0x7f4f7c161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50798/0 (socket says 172.21.15.179:50798) 2024-05-08T22:35:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 -- 172.21.15.179:0/3560555910 learned_addr learned my addr 172.21.15.179:0/3560555910 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f79d9b700 1 --2- 172.21.15.179:0/3560555910 >> v2:172.21.15.179:3300/0 conn(0x7f4f7c0b2120 0x7f4f7c0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7a59c700 1 --2- 172.21.15.179:0/3560555910 >> v2:172.21.15.195:3300/0 conn(0x7f4f7c15d5b0 0x7f4f7c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280133848 0 0) 0x7f4f7c06afc0 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f5c0035d0 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2838517875 0 0) 0x7f4f5c0035d0 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 >> v2:172.21.15.195:3300/0 conn(0x7f4f7c15d5b0 msgr2=0x7f4f7c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 --2- 172.21.15.179:0/3560555910 >> v2:172.21.15.195:3300/0 conn(0x7f4f7c15d5b0 0x7f4f7c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 >> v2:172.21.15.179:3300/0 conn(0x7f4f7c0b2120 msgr2=0x7f4f7c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 --2- 172.21.15.179:0/3560555910 >> v2:172.21.15.179:3300/0 conn(0x7f4f7c0b2120 0x7f4f7c0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c06c460 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f4f74001670 con 0x7f4f7c0ce3b0 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 >> v1:172.21.15.179:6789/0 conn(0x7f4f7c0ce3b0 legacy=0x7f4f7c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f5c003a20 0x7f4f5c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c004870 0x7f4f5c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 0x7f4f5c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0ce820 con 0x7f4f5c005230 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c15d990 con 0x7f4f5c004870 2024-05-08T22:35:25.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c00bfc0 con 0x7f4f5c003a20 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 0x7f4f5c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7a59c700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f5c003a20 0x7f4f5c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f79d9b700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c004870 0x7f4f5c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 -- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f5c003a20 msgr2=0x7f4f5c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f5c003a20 0x7f4f5c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 -- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c004870 msgr2=0x7f4f5c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c004870 0x7f4f5c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 -- 172.21.15.179:0/3560555910 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0035d0 con 0x7f4f5c005230 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f79d9b700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c004870 0x7f4f5c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 -- 172.21.15.179:0/3560555910 reap_dead start 2024-05-08T22:35:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.064+0000 7f4f7ad9d700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 0x7f4f5c00ba30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400c2f0 tx=0x7f4f5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca508d8133b54a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74007c80 con 0x7f4f5c005230 2024-05-08T22:35:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f74005980 con 0x7f4f5c005230 2024-05-08T22:35:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7959a700 1 -- 172.21.15.179:0/3560555910 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74005d30 con 0x7f4f5c005230 2024-05-08T22:35:25.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 msgr2=0x7f4f5c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 0x7f4f5c00ba30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400c2f0 tx=0x7f4f5c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- 172.21.15.179:0/3560555910 shutdown_connections 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/3560555910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f5c005230 0x7f4f5c00ba30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- 172.21.15.179:0/3560555910 >> 172.21.15.179:0/3560555910 conn(0x7f4f7c0b4940 msgr2=0x7f4f7c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- 172.21.15.179:0/3560555910 shutdown_connections 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- 172.21.15.179:0/3560555910 wait complete. 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 Processor -- start 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.068+0000 7f4f7b59e700 1 -- start start 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f7c0b30a0 0x7f4f7c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f7c156950 0x7f4f7c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0b3a60 con 0x7f4f7c0b30a0 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c155c50 con 0x7f4f5c005230 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c154bb0 con 0x7f4f7c156950 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f79d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f7c0b30a0 0x7f4f7c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:42720/0 (socket says 172.21.15.179:42720) 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 -- 172.21.15.179:0/4223795600 learned_addr learned my addr 172.21.15.179:0/4223795600 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7ad9d700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f7c156950 0x7f4f7c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 -- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f7c156950 msgr2=0x7f4f7c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f7c156950 0x7f4f7c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 -- 172.21.15.179:0/4223795600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f7c0b30a0 msgr2=0x7f4f7c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f7c0b30a0 0x7f4f7c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 -- 172.21.15.179:0/4223795600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c154e90 con 0x7f4f5c005230 2024-05-08T22:35:25.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7a59c700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f640108f0 tx=0x7f4f64010c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64016670 con 0x7f4f5c005230 2024-05-08T22:35:25.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f640167d0 con 0x7f4f5c005230 2024-05-08T22:35:25.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c155170 con 0x7f4f5c005230 2024-05-08T22:35:25.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64016b60 con 0x7f4f5c005230 2024-05-08T22:35:25.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.072+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c155730 con 0x7f4f5c005230 2024-05-08T22:35:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7c04e700 con 0x7f4f5c005230 2024-05-08T22:35:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f64016cc0 con 0x7f4f5c005230 2024-05-08T22:35:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f72ffd700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 0x7f4f60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4f64084160 con 0x7f4f5c005230 2024-05-08T22:35:25.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f79d9b700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 0x7f4f60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.076+0000 7f4f79d9b700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 0x7f4f60068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0c8e20 tx=0x7f4f6c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.084+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4f640516e0 con 0x7f4f5c005230 2024-05-08T22:35:25.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.256+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f7c0d0e50 con 0x7f4f5c005230 2024-05-08T22:35:25.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.256+0000 7f4f72ffd700 1 -- 172.21.15.179:0/4223795600 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4f64054d00 con 0x7f4f5c005230 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:25.261 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:35:25.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 msgr2=0x7f4f60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 0x7f4f60068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0c8e20 tx=0x7f4f6c00b040 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 msgr2=0x7f4f7c153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f640108f0 tx=0x7f4f64010c00 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 shutdown_connections 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4f60066730 0x7f4f60068bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f7c156950 0x7f4f7c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f7c0b30a0 0x7f4f7c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 --2- 172.21.15.179:0/4223795600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f5c005230 0x7f4f7c153c60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 >> 172.21.15.179:0/4223795600 conn(0x7f4f7c0b4940 msgr2=0x7f4f7c0b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 shutdown_connections 2024-05-08T22:35:25.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.260+0000 7f4f7b59e700 1 -- 172.21.15.179:0/4223795600 wait complete. 2024-05-08T22:35:25.275 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:35:25.275 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:35:25.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.388+0000 7f0ebee68700 1 Processor -- start 2024-05-08T22:35:25.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.388+0000 7f0ebee68700 1 -- start start 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebee68700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0eb806bc20 0x7f0eb80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebee68700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80b5360 con 0x7f0eb8159a30 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebee68700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80b4070 con 0x7f0eb80b0310 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebee68700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb806ce20 con 0x7f0eb806bc20 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebe667700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0eb806bc20 0x7f0eb80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebd665700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0eb8159a30 0x7f0eb80cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57836/0 (socket says 172.21.15.179:57836) 2024-05-08T22:35:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebd665700 1 -- 172.21.15.179:0/1670012982 learned_addr learned my addr 172.21.15.179:0/1670012982 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:25.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebe667700 1 -- 172.21.15.179:0/1670012982 >> v1:172.21.15.195:6789/0 conn(0x7f0eb80b0310 legacy=0x7f0eb80b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebe667700 1 -- 172.21.15.179:0/1670012982 >> v1:172.21.15.179:6789/0 conn(0x7f0eb8159a30 legacy=0x7f0eb80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebce64700 1 -- 172.21.15.179:0/1670012982 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686780286 0 0) 0x7f0ea8001110 con 0x7f0eb8159a30 2024-05-08T22:35:25.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebe667700 1 -- 172.21.15.179:0/1670012982 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80b1df0 con 0x7f0eb806bc20 2024-05-08T22:35:25.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebe667700 1 --2- 172.21.15.179:0/1670012982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb806bc20 0x7f0eb80cd8d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009d90 tx=0x7f0eb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a831d9b2463f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebce64700 1 -- 172.21.15.179:0/1670012982 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000e070 con 0x7f0eb806bc20 2024-05-08T22:35:25.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebce64700 1 -- 172.21.15.179:0/1670012982 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb000bbd0 con 0x7f0eb806bc20 2024-05-08T22:35:25.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.392+0000 7f0ebce64700 1 -- 172.21.15.179:0/1670012982 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0012430 con 0x7f0eb806bc20 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- 172.21.15.179:0/1670012982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb806bc20 msgr2=0x7f0eb80cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 --2- 172.21.15.179:0/1670012982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb806bc20 0x7f0eb80cd8d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009d90 tx=0x7f0eb000b330 comp rx=0 tx=0).stop 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- 172.21.15.179:0/1670012982 shutdown_connections 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 --2- 172.21.15.179:0/1670012982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb806bc20 0x7f0eb80cd8d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- 172.21.15.179:0/1670012982 >> 172.21.15.179:0/1670012982 conn(0x7f0eb8098a70 msgr2=0x7f0eb80b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- 172.21.15.179:0/1670012982 shutdown_connections 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- 172.21.15.179:0/1670012982 wait complete. 2024-05-08T22:35:25.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 Processor -- start 2024-05-08T22:35:25.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- start start 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eb806bc20 0x7f0eb814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb80b0310 0x7f0eb81f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815be40 con 0x7f0eb8159a30 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81f0da0 con 0x7f0eb80b0310 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.396+0000 7f0ebee68700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81efc30 con 0x7f0eb806bc20 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60936/0 (socket says 172.21.15.179:60936) 2024-05-08T22:35:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 -- 172.21.15.179:0/3769958471 learned_addr learned my addr 172.21.15.179:0/3769958471 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebde66700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eb806bc20 0x7f0eb814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebd665700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb80b0310 0x7f0eb81f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 -- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eb806bc20 msgr2=0x7f0eb814a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eb806bc20 0x7f0eb814a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 -- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb80b0310 msgr2=0x7f0eb81f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb80b0310 0x7f0eb81f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 -- 172.21.15.179:0/3769958471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0009a70 con 0x7f0eb8159a30 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebe667700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8066560 tx=0x7f0eb0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0012430 con 0x7f0eb8159a30 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0020070 con 0x7f0eb8159a30 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00098f0 con 0x7f0eb8159a30 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb814ae30 con 0x7f0eb8159a30 2024-05-08T22:35:25.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb814b3f0 con 0x7f0eb8159a30 2024-05-08T22:35:25.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.400+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0eb0004040 con 0x7f0eb8159a30 2024-05-08T22:35:25.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.404+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb806ba40 con 0x7f0eb8159a30 2024-05-08T22:35:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.404+0000 7f0eae7fc700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 0x7f0e9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.404+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0eb0022070 con 0x7f0eb8159a30 2024-05-08T22:35:25.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.404+0000 7f0ebde66700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 0x7f0e9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.404+0000 7f0ebde66700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 0x7f0e9c068c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000f60 tx=0x7f0eb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.412+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0eb00532c0 con 0x7f0eb8159a30 2024-05-08T22:35:25.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.580+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0eb806c380 con 0x7f0eb8159a30 2024-05-08T22:35:25.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.912+0000 7f0eae7fc700 1 -- 172.21.15.179:0/3769958471 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0eb00568e0 con 0x7f0eb8159a30 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 msgr2=0x7f0e9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 0x7f0e9c068c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000f60 tx=0x7f0eb4009040 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 msgr2=0x7f0eb81ef6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8066560 tx=0x7f0eb0004af0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 shutdown_connections 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e9c066760 0x7f0e9c068c20 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000f60 tx=0x7f0eb4009040 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb80b0310 0x7f0eb81f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb8159a30 0x7f0eb81ef6f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 --2- 172.21.15.179:0/3769958471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eb806bc20 0x7f0eb814a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 >> 172.21.15.179:0/3769958471 conn(0x7f0eb8098a70 msgr2=0x7f0eb806ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 shutdown_connections 2024-05-08T22:35:25.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:25.920+0000 7f0ebee68700 1 -- 172.21.15.179:0/3769958471 wait complete. 2024-05-08T22:35:25.927 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:35:25.938 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:35:26.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 Processor -- start 2024-05-08T22:35:26.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 -- start start 2024-05-08T22:35:26.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 0x7f828c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0d1910 con 0x7f828c0d29b0 2024-05-08T22:35:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0ce510 con 0x7f828c0cdb20 2024-05-08T22:35:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b6f90 con 0x7f828c15d160 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828b7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 0x7f828c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:49634/0 (socket says 172.21.15.179:49634) 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828b7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 0x7f828c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:36752/0 (socket says 172.21.15.179:36752) 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 -- 172.21.15.179:0/2490996747 learned_addr learned my addr 172.21.15.179:0/2490996747 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828a7fc700 1 --1- 172.21.15.179:0/2490996747 >> v1:172.21.15.195:6789/0 conn(0x7f828c0d29b0 0x7f828c0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:47550/0 (socket says 172.21.15.179:47550) 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 -- 172.21.15.179:0/2490996747 >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 msgr2=0x7f828c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 --2- 172.21.15.179:0/2490996747 >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 0x7f828c0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 -- 172.21.15.179:0/2490996747 >> v1:172.21.15.195:6789/0 conn(0x7f828c0d29b0 legacy=0x7f828c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 -- 172.21.15.179:0/2490996747 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0d1bf0 con 0x7f828c15d160 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.060+0000 7f828affd700 1 --2- 172.21.15.179:0/2490996747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f827c009910 tx=0x7f827c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9aa12f512f49c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f8289ffb700 1 -- 172.21.15.179:0/2490996747 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c011830 con 0x7f828c15d160 2024-05-08T22:35:26.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f8289ffb700 1 -- 172.21.15.179:0/2490996747 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c011990 con 0x7f828c15d160 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f8289ffb700 1 -- 172.21.15.179:0/2490996747 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c011d40 con 0x7f828c15d160 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- 172.21.15.179:0/2490996747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 msgr2=0x7f828c0b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 --2- 172.21.15.179:0/2490996747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f827c009910 tx=0x7f827c009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- 172.21.15.179:0/2490996747 shutdown_connections 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 --2- 172.21.15.179:0/2490996747 >> v2:172.21.15.195:3300/0 conn(0x7f828c0cdb20 0x7f828c0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 --2- 172.21.15.179:0/2490996747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c15d160 0x7f828c0b8a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- 172.21.15.179:0/2490996747 >> 172.21.15.179:0/2490996747 conn(0x7f828c0d3270 msgr2=0x7f828c0d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- 172.21.15.179:0/2490996747 shutdown_connections 2024-05-08T22:35:26.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- 172.21.15.179:0/2490996747 wait complete. 2024-05-08T22:35:26.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 Processor -- start 2024-05-08T22:35:26.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.064+0000 7f828bfff700 1 -- start start 2024-05-08T22:35:26.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c0cdb20 0x7f828c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c15d160 0x7f828c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1592b0 con 0x7f828c0d29b0 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c157720 con 0x7f828c15d160 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1597a0 con 0x7f828c0cdb20 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60944/0 (socket says 172.21.15.179:60944) 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 -- 172.21.15.179:0/579036402 learned_addr learned my addr 172.21.15.179:0/579036402 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828affd700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c0cdb20 0x7f828c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828b7fe700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c15d160 0x7f828c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 -- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c0cdb20 msgr2=0x7f828c1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c0cdb20 0x7f828c1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 -- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c15d160 msgr2=0x7f828c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c15d160 0x7f828c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 -- 172.21.15.179:0/579036402 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009570 con 0x7f828c0d29b0 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828a7fc700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f827400c4d0 tx=0x7f827400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:26.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274005cd0 con 0x7f828c0d29b0 2024-05-08T22:35:26.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274005e30 con 0x7f828c0d29b0 2024-05-08T22:35:26.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c159a80 con 0x7f828c0d29b0 2024-05-08T22:35:26.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740072d0 con 0x7f828c0d29b0 2024-05-08T22:35:26.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c15a040 con 0x7f828c0d29b0 2024-05-08T22:35:26.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.068+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c04e700 con 0x7f828c0d29b0 2024-05-08T22:35:26.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.072+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8274007430 con 0x7f828c0d29b0 2024-05-08T22:35:26.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.072+0000 7f82837fe700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 0x7f826c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:26.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.072+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f82740837c0 con 0x7f828c0d29b0 2024-05-08T22:35:26.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.072+0000 7f828affd700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 0x7f826c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:26.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.072+0000 7f828affd700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 0x7f826c068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f827c000c00 tx=0x7f827c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:26.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.076+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8274050d10 con 0x7f828c0d29b0 2024-05-08T22:35:26.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.248+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f828c0d3860 con 0x7f828c0d29b0 2024-05-08T22:35:26.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.920+0000 7f82837fe700 1 -- 172.21.15.179:0/579036402 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8274017020 con 0x7f828c0d29b0 2024-05-08T22:35:26.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.940+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 msgr2=0x7f826c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.940+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 0x7f826c068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f827c000c00 tx=0x7f827c008030 comp rx=0 tx=0).stop 2024-05-08T22:35:26.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.940+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 msgr2=0x7f828c15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:26.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.940+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f827400c4d0 tx=0x7f827400c7e0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 shutdown_connections 2024-05-08T22:35:26.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f826c066730 0x7f826c068bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c15d160 0x7f828c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0d29b0 0x7f828c15a560 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 --2- 172.21.15.179:0/579036402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c0cdb20 0x7f828c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 >> 172.21.15.179:0/579036402 conn(0x7f828c0d3270 msgr2=0x7f828c0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 shutdown_connections 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:26.944+0000 7f828bfff700 1 -- 172.21.15.179:0/579036402 wait complete. 2024-05-08T22:35:26.947 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:35:26.961 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:35:27.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 Processor -- start 2024-05-08T22:35:27.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 -- start start 2024-05-08T22:35:27.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f816c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f806b9a0 con 0x7fb0f8161ed0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f806bc80 con 0x7fb0f80acfc0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fea09700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f806def0 con 0x7fb0f81629c0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb0f8161ed0 0x7fb0f806cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57844/0 (socket says 172.21.15.179:57844) 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f7fff700 1 -- 172.21.15.179:0/2878100611 learned_addr learned my addr 172.21.15.179:0/2878100611 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fcfa6700 1 --2- 172.21.15.179:0/2878100611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f816c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f6ffd700 1 -- 172.21.15.179:0/2878100611 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168884245 0 0) 0x7fb0f806b9a0 con 0x7fb0f8161ed0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f6ffd700 1 -- 172.21.15.179:0/2878100611 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e40035d0 con 0x7fb0f8161ed0 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fcfa6700 1 -- 172.21.15.179:0/2878100611 >> v1:172.21.15.195:6789/0 conn(0x7fb0f80acfc0 legacy=0x7fb0f81622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fcfa6700 1 -- 172.21.15.179:0/2878100611 >> v1:172.21.15.179:6789/0 conn(0x7fb0f8161ed0 legacy=0x7fb0f806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fcfa6700 1 -- 172.21.15.179:0/2878100611 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f806bf60 con 0x7fb0f81629c0 2024-05-08T22:35:27.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0fcfa6700 1 --2- 172.21.15.179:0/2878100611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f816c480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009dc0 tx=0x7fb0f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8073f2ea770beab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f6ffd700 1 -- 172.21.15.179:0/2878100611 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000e070 con 0x7fb0f81629c0 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f6ffd700 1 -- 172.21.15.179:0/2878100611 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000bb20 con 0x7fb0f81629c0 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.068+0000 7fb0f6ffd700 1 -- 172.21.15.179:0/2878100611 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000be90 con 0x7fb0f81629c0 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- 172.21.15.179:0/2878100611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 msgr2=0x7fb0f816c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 --2- 172.21.15.179:0/2878100611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f816c480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009dc0 tx=0x7fb0f000b330 comp rx=0 tx=0).stop 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- 172.21.15.179:0/2878100611 shutdown_connections 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 --2- 172.21.15.179:0/2878100611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f816c480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- 172.21.15.179:0/2878100611 >> 172.21.15.179:0/2878100611 conn(0x7fb0f80ceca0 msgr2=0x7fb0f80cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- 172.21.15.179:0/2878100611 shutdown_connections 2024-05-08T22:35:27.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- 172.21.15.179:0/2878100611 wait complete. 2024-05-08T22:35:27.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 Processor -- start 2024-05-08T22:35:27.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.072+0000 7fb0fea09700 1 -- start start 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0f8161ed0 0x7fb0f80ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f80aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81fb0e0 con 0x7fb0f80acfc0 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81fa560 con 0x7fb0f8161ed0 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81fa840 con 0x7fb0f81629c0 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60960/0 (socket says 172.21.15.179:60960) 2024-05-08T22:35:27.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 -- 172.21.15.179:0/1409407868 learned_addr learned my addr 172.21.15.179:0/1409407868 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fcfa6700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f80aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f77fe700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0f8161ed0 0x7fb0f80ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 -- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 msgr2=0x7fb0f80aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f80aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 -- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0f8161ed0 msgr2=0x7fb0f80ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0f8161ed0 0x7fb0f80ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 -- 172.21.15.179:0/1409407868 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0009a70 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f7fff700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c700 tx=0x7fb0e800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8005d90 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e80070d0 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f81fab20 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8007480 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.076+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81fa1f0 con 0x7fb0f80acfc0 2024-05-08T22:35:27.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f806ee80 con 0x7fb0f80acfc0 2024-05-08T22:35:27.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0e80075e0 con 0x7fb0f80acfc0 2024-05-08T22:35:27.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0f4ff9700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 0x7fb0d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0f77fe700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 0x7fb0d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fb0e80836a0 con 0x7fb0f80acfc0 2024-05-08T22:35:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.080+0000 7fb0f77fe700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 0x7fb0d8068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb0f815b3f0 tx=0x7fb0e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.092+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb0e8050920 con 0x7fb0f80acfc0 2024-05-08T22:35:27.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.288+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb0f8073290 con 0x7fb0f80acfc0 2024-05-08T22:35:27.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.296+0000 7fb0f4ff9700 1 -- 172.21.15.179:0/1409407868 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb0e8053f40 con 0x7fb0f80acfc0 2024-05-08T22:35:27.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.304+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 msgr2=0x7fb0d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.304+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 0x7fb0d8068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb0f815b3f0 tx=0x7fb0e0009040 comp rx=0 tx=0).stop 2024-05-08T22:35:27.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.304+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 msgr2=0x7fb0f81646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.304+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c700 tx=0x7fb0e800ca10 comp rx=0 tx=0).stop 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 shutdown_connections 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb0d8066730 0x7fb0d8068bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0f8161ed0 0x7fb0f80ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0f81629c0 0x7fb0f80aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 --2- 172.21.15.179:0/1409407868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f80acfc0 0x7fb0f81646f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 >> 172.21.15.179:0/1409407868 conn(0x7fb0f80ceca0 msgr2=0x7fb0f81622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 shutdown_connections 2024-05-08T22:35:27.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.308+0000 7fb0fea09700 1 -- 172.21.15.179:0/1409407868 wait complete. 2024-05-08T22:35:27.317 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 7 and data pool 8 2024-05-08T22:35:27.332 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:27.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 Processor -- start 2024-05-08T22:35:27.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 -- start start 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54cc0b2120 0x7f54cc0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc06afc0 con 0x7f54cc0ce3b0 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0ce840 con 0x7f54cc0b2120 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54d0a00700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc15d990 con 0x7f54cc15d5b0 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54cb7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54cc0b2120 0x7f54cc0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54cb7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:49672/0 (socket says 172.21.15.179:49672) 2024-05-08T22:35:27.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54cb7fe700 1 -- 172.21.15.179:0/2468870927 learned_addr learned my addr 172.21.15.179:0/2468870927 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:27.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 -- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 msgr2=0x7f54cc0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54ca7fc700 1 -- 172.21.15.179:0/2468870927 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543185659 0 0) 0x7f54cc06afc0 con 0x7f54cc0ce3b0 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 -- 172.21.15.179:0/2468870927 >> v1:172.21.15.195:6789/0 conn(0x7f54cc0ce3b0 legacy=0x7f54cc161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 -- 172.21.15.179:0/2468870927 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc06c460 con 0x7f54cc0b2120 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54cb7fe700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54caffd700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0cde70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009850 tx=0x7f54b4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e771ec3d32d5402c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54ca7fc700 1 -- 172.21.15.179:0/2468870927 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4011830 con 0x7f54cc0b2120 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54ca7fc700 1 -- 172.21.15.179:0/2468870927 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4011990 con 0x7f54cc0b2120 2024-05-08T22:35:27.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.428+0000 7f54ca7fc700 1 -- 172.21.15.179:0/2468870927 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4011d40 con 0x7f54cc0b2120 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 msgr2=0x7f54cc0cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0cde70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009850 tx=0x7f54b4009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- 172.21.15.179:0/2468870927 shutdown_connections 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0cde70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- 172.21.15.179:0/2468870927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15d5b0 0x7f54cc0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- 172.21.15.179:0/2468870927 >> 172.21.15.179:0/2468870927 conn(0x7f54cc0b4940 msgr2=0x7f54cc0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- 172.21.15.179:0/2468870927 shutdown_connections 2024-05-08T22:35:27.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- 172.21.15.179:0/2468870927 wait complete. 2024-05-08T22:35:27.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 Processor -- start 2024-05-08T22:35:27.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- start start 2024-05-08T22:35:27.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc0ce3b0 0x7f54cc151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc15aba0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0a9d00 con 0x7f54cc0b2120 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54d0a00700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0a9fe0 con 0x7f54cc0ce3b0 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54cbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54cbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60974/0 (socket says 172.21.15.179:60974) 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.432+0000 7f54cbfff700 1 -- 172.21.15.179:0/954710455 learned_addr learned my addr 172.21.15.179:0/954710455 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cb7fe700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54caffd700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc0ce3b0 0x7f54cc151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 -- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc0ce3b0 msgr2=0x7f54cc151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc0ce3b0 0x7f54cc151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 -- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 msgr2=0x7f54cc0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 -- 172.21.15.179:0/954710455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4009b30 con 0x7f54cc15d5b0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cb7fe700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cbfff700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c3f0 tx=0x7f54bc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0054e0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc0036a0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b40094b0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc003a50 con 0x7f54cc15d5b0 2024-05-08T22:35:27.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc0aa5a0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc04e700 con 0x7f54cc15d5b0 2024-05-08T22:35:27.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54bc003bb0 con 0x7f54cc15d5b0 2024-05-08T22:35:27.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 0x7f54ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f54bc083770 con 0x7f54cc15d5b0 2024-05-08T22:35:27.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.436+0000 7f54cb7fe700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 0x7f54ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.440+0000 7f54cb7fe700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 0x7f54ac068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54c40092c0 tx=0x7f54c400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.448+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f54bc050910 con 0x7f54cc15d5b0 2024-05-08T22:35:27.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.620+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54cc0d0e50 con 0x7f54cc15d5b0 2024-05-08T22:35:27.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.620+0000 7f54c3fff700 1 -- 172.21.15.179:0/954710455 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f54bc017090 con 0x7f54cc15d5b0 2024-05-08T22:35:27.625 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:27.625 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":54,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:27.294756+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 msgr2=0x7f54ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 0x7f54ac068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54c40092c0 tx=0x7f54c400b040 comp rx=0 tx=0).stop 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 msgr2=0x7f54cc0aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c3f0 tx=0x7f54bc00c7b0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 shutdown_connections 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f54ac066730 0x7f54ac068bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc15d5b0 0x7f54cc0aada0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc0ce3b0 0x7f54cc151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 --2- 172.21.15.179:0/954710455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc0b2120 0x7f54cc0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 >> 172.21.15.179:0/954710455 conn(0x7f54cc0b4940 msgr2=0x7f54cc15fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 shutdown_connections 2024-05-08T22:35:27.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.624+0000 7f54d0a00700 1 -- 172.21.15.179:0/954710455 wait complete. 2024-05-08T22:35:27.640 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:27.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 Processor -- start 2024-05-08T22:35:27.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 -- start start 2024-05-08T22:35:27.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12cc06de70 0x7f12cc161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc165d50 con 0x7f12cc15ef80 2024-05-08T22:35:27.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc06ad70 con 0x7f12cc06de70 2024-05-08T22:35:27.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d34ea700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc165260 con 0x7f12cc164db0 2024-05-08T22:35:27.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d1ce7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12cc06de70 0x7f12cc161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d1ce7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12cc06de70 0x7f12cc161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:36802/0 (socket says 172.21.15.179:36802) 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d1ce7700 1 -- 172.21.15.179:0/2593888786 learned_addr learned my addr 172.21.15.179:0/2593888786 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d2ce9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.748+0000 7f12d24e8700 1 --1- 172.21.15.179:0/2593888786 >> v1:172.21.15.195:6789/0 conn(0x7f12cc15ef80 0x7f12cc06d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:47580/0 (socket says 172.21.15.179:47580) 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d1ce7700 1 -- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 msgr2=0x7f12cc06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d1ce7700 1 --2- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc06b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d1ce7700 1 -- 172.21.15.179:0/2593888786 >> v1:172.21.15.195:6789/0 conn(0x7f12cc15ef80 legacy=0x7f12cc06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d1ce7700 1 -- 172.21.15.179:0/2593888786 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc06e4b0 con 0x7f12cc06de70 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d1ce7700 1 --2- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc161790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009e50 tx=0x7f12c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edab1e0d43def57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d14e6700 1 -- 172.21.15.179:0/2593888786 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400e070 con 0x7f12cc06de70 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d14e6700 1 -- 172.21.15.179:0/2593888786 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400b910 con 0x7f12cc06de70 2024-05-08T22:35:27.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d14e6700 1 -- 172.21.15.179:0/2593888786 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400bcc0 con 0x7f12cc06de70 2024-05-08T22:35:27.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 -- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 msgr2=0x7f12cc161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 --2- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc161790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009e50 tx=0x7f12c400b330 comp rx=0 tx=0).stop 2024-05-08T22:35:27.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 -- 172.21.15.179:0/2593888786 shutdown_connections 2024-05-08T22:35:27.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 --2- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc06b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 --2- 172.21.15.179:0/2593888786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc161790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 -- 172.21.15.179:0/2593888786 >> 172.21.15.179:0/2593888786 conn(0x7f12cc0ce9d0 msgr2=0x7f12cc0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 -- 172.21.15.179:0/2593888786 shutdown_connections 2024-05-08T22:35:27.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.752+0000 7f12d34ea700 1 -- 172.21.15.179:0/2593888786 wait complete. 2024-05-08T22:35:27.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 Processor -- start 2024-05-08T22:35:27.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 -- start start 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15d4d0 con 0x7f12cc15ef80 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1474f0 con 0x7f12cc06de70 2024-05-08T22:35:27.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.756+0000 7f12d34ea700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc147930 con 0x7f12cc164db0 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60988/0 (socket says 172.21.15.179:60988) 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d2ce9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 -- 172.21.15.179:0/3608023449 learned_addr learned my addr 172.21.15.179:0/3608023449 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d24e8700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 -- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 msgr2=0x7f12cc0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc0c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 -- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 msgr2=0x7f12cc146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 -- 172.21.15.179:0/3608023449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009b30 con 0x7f12cc15ef80 2024-05-08T22:35:27.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d1ce7700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f12c400b370 tx=0x7f12c4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4004800 con 0x7f12cc15ef80 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c40092e0 con 0x7f12cc15ef80 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc147c10 con 0x7f12cc15ef80 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c401c680 con 0x7f12cc15ef80 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc1481d0 con 0x7f12cc15ef80 2024-05-08T22:35:27.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc067740 con 0x7f12cc15ef80 2024-05-08T22:35:27.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.760+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12c401c7e0 con 0x7f12cc15ef80 2024-05-08T22:35:27.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.764+0000 7f12c2ffd700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 0x7f12b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:27.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.764+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f12c4085e30 con 0x7f12cc15ef80 2024-05-08T22:35:27.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.764+0000 7f12d24e8700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 0x7f12b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:27.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.764+0000 7f12d24e8700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 0x7f12b8068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12c8000f60 tx=0x7f12c8007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:27.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.772+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f12c40530f0 con 0x7f12cc15ef80 2024-05-08T22:35:27.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.948+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12cc04e700 con 0x7f12cc15ef80 2024-05-08T22:35:27.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.948+0000 7f12c2ffd700 1 -- 172.21.15.179:0/3608023449 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f12c4056710 con 0x7f12cc15ef80 2024-05-08T22:35:27.952 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:27.953 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":54,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:27.294756+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:27.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 msgr2=0x7f12b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 0x7f12b8068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12c8000f60 tx=0x7f12c8007ef0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 msgr2=0x7f12cc0c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f12c400b370 tx=0x7f12c4004af0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 shutdown_connections 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f12b8066730 0x7f12b8068bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12cc164db0 0x7f12cc0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12cc15ef80 0x7f12cc0c3950 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 --2- 172.21.15.179:0/3608023449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12cc06de70 0x7f12cc146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 >> 172.21.15.179:0/3608023449 conn(0x7f12cc0ce9d0 msgr2=0x7f12cc0cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 shutdown_connections 2024-05-08T22:35:27.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:27.952+0000 7f12d34ea700 1 -- 172.21.15.179:0/3608023449 wait complete. 2024-05-08T22:35:27.967 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:35:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 Processor -- start 2024-05-08T22:35:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 -- start start 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe6880b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880b6f90 con 0x7fe6880d29b0 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880d1910 con 0x7fe68815d160 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe68d0fc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880d1bf0 con 0x7fe6880ce230 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe6877fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe6880d29b0 0x7fe6880cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57860/0 (socket says 172.21.15.179:57860) 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.080+0000 7fe6877fe700 1 -- 172.21.15.179:0/1439295922 learned_addr learned my addr 172.21.15.179:0/1439295922 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68c8fb700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe6880b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199125628 0 0) 0x7fe670001110 con 0x7fe6880d29b0 2024-05-08T22:35:28.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6740035d0 con 0x7fe6880d29b0 2024-05-08T22:35:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729402104 0 0) 0x7fe6740035d0 con 0x7fe6880d29b0 2024-05-08T22:35:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 msgr2=0x7fe6880b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe6880b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 >> v1:172.21.15.195:6789/0 conn(0x7fe68815d160 legacy=0x7fe6880d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880d1ed0 con 0x7fe6880d29b0 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe670001800 con 0x7fe6880d29b0 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 >> v1:172.21.15.179:6789/0 conn(0x7fe6880d29b0 legacy=0x7fe6880cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe674003a50 0x7fe674003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe674004400 0x7fe6740048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe674005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880d1bf0 con 0x7fe674003a50 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400ba60 con 0x7fe674004400 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bc20 con 0x7fe6880ce230 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe674003a50 0x7fe674003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe6877fe700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe674004400 0x7fe6740048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68c8fb700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe674005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 -- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 msgr2=0x7fe674005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6880ce230 0x7fe674005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 -- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe674004400 msgr2=0x7fe6740048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe674004400 0x7fe6740048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 -- 172.21.15.179:0/1439295922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740035d0 con 0x7fe674003a50 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe687fff700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe674003a50 0x7fe674003e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe68000c290 tx=0x7fe6740058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb4d19553bc1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680001e70 con 0x7fe674003a50 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000ce30 con 0x7fe674003a50 2024-05-08T22:35:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe686ffd700 1 -- 172.21.15.179:0/1439295922 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680012660 con 0x7fe674003a50 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1439295922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe674003a50 msgr2=0x7fe674003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1439295922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe674003a50 0x7fe674003e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe68000c290 tx=0x7fe6740058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68c8fb700 1 -- 172.21.15.179:0/1439295922 reap_dead start 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1439295922 shutdown_connections 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1439295922 >> 172.21.15.179:0/1439295922 conn(0x7fe6880d3270 msgr2=0x7fe6880d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1439295922 shutdown_connections 2024-05-08T22:35:28.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.084+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1439295922 wait complete. 2024-05-08T22:35:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 Processor -- start 2024-05-08T22:35:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 -- start start 2024-05-08T22:35:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6880ccd60 0x7fe6880c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 0x7fe68815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880cd140 con 0x7fe6880c83c0 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688152dc0 con 0x7fe6880ccd60 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68d0fc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68815fed0 con 0x7fe68815f510 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe6877fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe68c8fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 0x7fe68815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe687fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6880ccd60 0x7fe6880c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.088+0000 7fe6877fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32776/0 (socket says 172.21.15.179:32776) 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe68c8fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 0x7fe68815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:49740/0 (socket says 172.21.15.179:49740) 2024-05-08T22:35:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 -- 172.21.15.179:0/1835816384 learned_addr learned my addr 172.21.15.179:0/1835816384 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 -- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 msgr2=0x7fe68815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 0x7fe68815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 -- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6880ccd60 msgr2=0x7fe6880c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6880ccd60 0x7fe6880c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 -- 172.21.15.179:0/1835816384 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688156790 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe6877fe700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe670001930 tx=0x7fe670007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670012560 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700126c0 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe688156a70 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670012a50 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe688157030 con 0x7fe6880c83c0 2024-05-08T22:35:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68804e700 con 0x7fe6880c83c0 2024-05-08T22:35:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.092+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe670012bb0 con 0x7fe6880c83c0 2024-05-08T22:35:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.096+0000 7fe684ff9700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 0x7fe668068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:28.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.096+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe670083fe0 con 0x7fe6880c83c0 2024-05-08T22:35:28.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.096+0000 7fe687fff700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 0x7fe668068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:28.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.096+0000 7fe687fff700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 0x7fe668068c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6880c6760 tx=0x7fe680005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:28.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.100+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe670051210 con 0x7fe6880c83c0 2024-05-08T22:35:28.242 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:35:28.239+0000 7f5d381a1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:35:28.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:28.308+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe6880b6f90 con 0x7fe6880c83c0 2024-05-08T22:35:29.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.308+0000 7fe684ff9700 1 -- 172.21.15.179:0/1835816384 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe6880b6f90 con 0x7fe6880c83c0 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 msgr2=0x7fe668068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 0x7fe668068c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6880c6760 tx=0x7fe680005040 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 msgr2=0x7fe6880c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe670001930 tx=0x7fe670007570 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 shutdown_connections 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe668066760 0x7fe668068c20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe68815f510 0x7fe68815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880c83c0 0x7fe6880c71a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 --2- 172.21.15.179:0/1835816384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6880ccd60 0x7fe6880c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 >> 172.21.15.179:0/1835816384 conn(0x7fe6880d3270 msgr2=0x7fe68806b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:29.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 shutdown_connections 2024-05-08T22:35:29.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.312+0000 7fe68d0fc700 1 -- 172.21.15.179:0/1835816384 wait complete. 2024-05-08T22:35:29.331 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:35:29.331 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:35:29.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 Processor -- start 2024-05-08T22:35:29.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 -- start start 2024-05-08T22:35:29.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc780ce2b0 0x7fdc78167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:29.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc780b8ee0 con 0x7fdc7815d0c0 2024-05-08T22:35:29.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7815d4a0 con 0x7fdc780b8a80 2024-05-08T22:35:29.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.448+0000 7fdc7e1bf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7806abb0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc780ce2b0 0x7fdc78167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc780ce2b0 0x7fdc78167ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32790/0 (socket says 172.21.15.179:32790) 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 -- 172.21.15.179:0/1599344500 learned_addr learned my addr 172.21.15.179:0/1599344500 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d1bd700 1 --1- 172.21.15.179:0/1599344500 >> v1:172.21.15.179:6789/0 conn(0x7fdc7815d0c0 0x7fdc780d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57876/0 (socket says 172.21.15.179:57876) 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 -- 172.21.15.179:0/1599344500 >> v1:172.21.15.195:6789/0 conn(0x7fdc780b8a80 legacy=0x7fdc780cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 -- 172.21.15.179:0/1599344500 >> v1:172.21.15.179:6789/0 conn(0x7fdc7815d0c0 legacy=0x7fdc780d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 -- 172.21.15.179:0/1599344500 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780d2500 con 0x7fdc780ce2b0 2024-05-08T22:35:29.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7d9be700 1 --2- 172.21.15.179:0/1599344500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78167ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009e50 tx=0x7fdc7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad39ac7895f04335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc6ffff700 1 -- 172.21.15.179:0/1599344500 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000e070 con 0x7fdc780ce2b0 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc6ffff700 1 -- 172.21.15.179:0/1599344500 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7000b9c0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc6ffff700 1 -- 172.21.15.179:0/1599344500 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000bd70 con 0x7fdc780ce2b0 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1599344500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 msgr2=0x7fdc78167ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1599344500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78167ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009e50 tx=0x7fdc7000b330 comp rx=0 tx=0).stop 2024-05-08T22:35:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1599344500 shutdown_connections 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1599344500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78167ed0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1599344500 >> 172.21.15.179:0/1599344500 conn(0x7fdc780d2ed0 msgr2=0x7fdc780d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1599344500 shutdown_connections 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.452+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1599344500 wait complete. 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 Processor -- start 2024-05-08T22:35:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 -- start start 2024-05-08T22:35:29.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc780b8a80 0x7fdc780b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:29.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:29.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7815d0c0 0x7fdc78153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781e6970 con 0x7fdc780ce2b0 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781e6c50 con 0x7fdc7815d0c0 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7e1bf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781e6f30 con 0x7fdc780b8a80 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32794/0 (socket says 172.21.15.179:32794) 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 -- 172.21.15.179:0/1304971499 learned_addr learned my addr 172.21.15.179:0/1304971499 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7d9be700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7815d0c0 0x7fdc78153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:29.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7d1bd700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc780b8a80 0x7fdc780b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:29.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 -- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc780b8a80 msgr2=0x7fdc780b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc780b8a80 0x7fdc780b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 -- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7815d0c0 msgr2=0x7fdc78153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:29.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7815d0c0 0x7fdc78153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:29.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 -- 172.21.15.179:0/1304971499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70009b30 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc7c9bc700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c500 tx=0x7fdc7400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740052d0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74005430 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.456+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740055a0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.460+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc781e7210 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.460+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc781e77d0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.460+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7804e700 con 0x7fdc780ce2b0 2024-05-08T22:35:29.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.460+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc740036a0 con 0x7fdc780ce2b0 2024-05-08T22:35:29.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.460+0000 7fdc6dffb700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 0x7fdc5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:29.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.464+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc74083500 con 0x7fdc780ce2b0 2024-05-08T22:35:29.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.464+0000 7fdc7d1bd700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 0x7fdc5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:29.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.464+0000 7fdc7d1bd700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 0x7fdc5c068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000f60 tx=0x7fdc68007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:29.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.468+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc74050690 con 0x7fdc780ce2b0 2024-05-08T22:35:29.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:29.680+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc780676e0 con 0x7fdc780ce2b0 2024-05-08T22:35:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.316+0000 7fdc6dffb700 1 -- 172.21.15.179:0/1304971499 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc74053cb0 con 0x7fdc780ce2b0 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 msgr2=0x7fdc5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 0x7fdc5c068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000f60 tx=0x7fdc68007ef0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 msgr2=0x7fdc78155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c500 tx=0x7fdc7400c810 comp rx=0 tx=0).stop 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 shutdown_connections 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdc5c066730 0x7fdc5c068bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7815d0c0 0x7fdc78153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc780ce2b0 0x7fdc78155920 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 --2- 172.21.15.179:0/1304971499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc780b8a80 0x7fdc780b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 >> 172.21.15.179:0/1304971499 conn(0x7fdc780d2ed0 msgr2=0x7fdc780b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.320+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 shutdown_connections 2024-05-08T22:35:30.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.324+0000 7fdc7e1bf700 1 -- 172.21.15.179:0/1304971499 wait complete. 2024-05-08T22:35:30.343 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:35:30.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:30.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 Processor -- start 2024-05-08T22:35:30.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 -- start start 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe4406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efe440b4ad0 0x7efe4415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4406d370 con 0x7efe4415d590 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4415da10 con 0x7efe440b4ad0 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49c32700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4406d650 con 0x7efe440d07d0 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe49431700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efe440b4ad0 0x7efe4415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe43fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efe4415d590 0x7efe440b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57890/0 (socket says 172.21.15.179:57890) 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe43fff700 1 -- 172.21.15.179:0/2344692273 learned_addr learned my addr 172.21.15.179:0/2344692273 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:30.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.460+0000 7efe48c30700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe4406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49431700 1 -- 172.21.15.179:0/2344692273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 msgr2=0x7efe4406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49431700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe4406cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49431700 1 -- 172.21.15.179:0/2344692273 >> v1:172.21.15.179:6789/0 conn(0x7efe4415d590 legacy=0x7efe440b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe437fe700 1 -- 172.21.15.179:0/2344692273 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858093563 0 0) 0x7efe4406d370 con 0x7efe4415d590 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49431700 1 -- 172.21.15.179:0/2344692273 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4406d930 con 0x7efe440b4ad0 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49431700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe4415ea40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009d90 tx=0x7efe3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4844e13c26c22e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe437fe700 1 -- 172.21.15.179:0/2344692273 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00e070 con 0x7efe440b4ad0 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe437fe700 1 -- 172.21.15.179:0/2344692273 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c00bb20 con 0x7efe440b4ad0 2024-05-08T22:35:30.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe437fe700 1 -- 172.21.15.179:0/2344692273 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c012430 con 0x7efe440b4ad0 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- 172.21.15.179:0/2344692273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 msgr2=0x7efe4415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe4415ea40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009d90 tx=0x7efe3c00b330 comp rx=0 tx=0).stop 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- 172.21.15.179:0/2344692273 shutdown_connections 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe4415ea40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 --2- 172.21.15.179:0/2344692273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe4406cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- 172.21.15.179:0/2344692273 >> 172.21.15.179:0/2344692273 conn(0x7efe440b2840 msgr2=0x7efe440b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- 172.21.15.179:0/2344692273 shutdown_connections 2024-05-08T22:35:30.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- 172.21.15.179:0/2344692273 wait complete. 2024-05-08T22:35:30.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 Processor -- start 2024-05-08T22:35:30.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.464+0000 7efe49c32700 1 -- start start 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe440caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe4415d590 0x7efe440aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44149870 con 0x7efe440b4ad0 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440cb910 con 0x7efe4415d590 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440cbbf0 con 0x7efe440d07d0 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32822/0 (socket says 172.21.15.179:32822) 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 -- 172.21.15.179:0/1037629440 learned_addr learned my addr 172.21.15.179:0/1037629440 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe43fff700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe440caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49431700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe4415d590 0x7efe440aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 -- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 msgr2=0x7efe440caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe440caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 -- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe4415d590 msgr2=0x7efe440aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe4415d590 0x7efe440aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 -- 172.21.15.179:0/1037629440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009a70 con 0x7efe440b4ad0 2024-05-08T22:35:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe48c30700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efe3400e9d0 tx=0x7efe3400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34009720 con 0x7efe440b4ad0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3400ee70 con 0x7efe440b4ad0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe440cbed0 con 0x7efe440b4ad0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400ca50 con 0x7efe440b4ad0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.468+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe440cc490 con 0x7efe440b4ad0 2024-05-08T22:35:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4415ef30 con 0x7efe440b4ad0 2024-05-08T22:35:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe3400cbb0 con 0x7efe440b4ad0 2024-05-08T22:35:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe417fa700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 0x7efe30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efe34084260 con 0x7efe440b4ad0 2024-05-08T22:35:30.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe43fff700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 0x7efe30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.472+0000 7efe43fff700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 0x7efe30068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe440b2290 tx=0x7efe38007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.480+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe34051520 con 0x7efe440b4ad0 2024-05-08T22:35:30.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.684+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe4404e700 con 0x7efe440b4ad0 2024-05-08T22:35:30.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe417fa700 1 -- 172.21.15.179:0/1037629440 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7efe3400ce60 con 0x7efe440b4ad0 2024-05-08T22:35:30.690 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:30.690 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 msgr2=0x7efe30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 0x7efe30068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe440b2290 tx=0x7efe38007e90 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 msgr2=0x7efe44149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efe3400e9d0 tx=0x7efe3400ece0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 shutdown_connections 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe30066730 0x7efe30068bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe4415d590 0x7efe440aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe440d07d0 0x7efe440caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 --2- 172.21.15.179:0/1037629440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe440b4ad0 0x7efe44149330 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 >> 172.21.15.179:0/1037629440 conn(0x7efe440b2840 msgr2=0x7efe4415db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 shutdown_connections 2024-05-08T22:35:30.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.688+0000 7efe49c32700 1 -- 172.21.15.179:0/1037629440 wait complete. 2024-05-08T22:35:30.694 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:35:30.705 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.812+0000 7f7984960700 1 Processor -- start 2024-05-08T22:35:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.812+0000 7f7984960700 1 -- start start 2024-05-08T22:35:30.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7980161580 0x7f79800cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79800d1d50 0x7f79800b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7980165ed0 con 0x7f798006baa0 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798006c5e0 con 0x7f7980161580 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798006b610 con 0x7f79800d1d50 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7980161580 0x7f79800cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797f59e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f798006baa0 0x7f798006bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57900/0 (socket says 172.21.15.179:57900) 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7980161580 0x7f79800cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32838/0 (socket says 172.21.15.179:32838) 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797f59e700 1 -- 172.21.15.179:0/446974841 learned_addr learned my addr 172.21.15.179:0/446974841 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797659c700 1 --2- 172.21.15.179:0/446974841 >> v2:172.21.15.195:3300/0 conn(0x7f79800d1d50 0x7f79800b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 -- 172.21.15.179:0/446974841 >> v2:172.21.15.195:3300/0 conn(0x7f79800d1d50 msgr2=0x7f79800b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 --2- 172.21.15.179:0/446974841 >> v2:172.21.15.195:3300/0 conn(0x7f79800d1d50 0x7f79800b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 -- 172.21.15.179:0/446974841 >> v1:172.21.15.179:6789/0 conn(0x7f798006baa0 legacy=0x7f798006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797e59c700 1 -- 172.21.15.179:0/446974841 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327850219 0 0) 0x7f7980165ed0 con 0x7f798006baa0 2024-05-08T22:35:30.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 -- 172.21.15.179:0/446974841 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006ace0 con 0x7f7980161580 2024-05-08T22:35:30.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797ed9d700 1 --2- 172.21.15.179:0/446974841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980161580 0x7f79800cf2a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f798006cc20 tx=0x7f7970009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831887ce7297fd8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797e59c700 1 -- 172.21.15.179:0/446974841 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011830 con 0x7f7980161580 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797e59c700 1 -- 172.21.15.179:0/446974841 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970011990 con 0x7f7980161580 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f797e59c700 1 -- 172.21.15.179:0/446974841 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011d20 con 0x7f7980161580 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 -- 172.21.15.179:0/446974841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980161580 msgr2=0x7f79800cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 --2- 172.21.15.179:0/446974841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980161580 0x7f79800cf2a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f798006cc20 tx=0x7f7970009b30 comp rx=0 tx=0).stop 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 -- 172.21.15.179:0/446974841 shutdown_connections 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 --2- 172.21.15.179:0/446974841 >> v2:172.21.15.195:3300/0 conn(0x7f79800d1d50 0x7f79800b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.816+0000 7f7984960700 1 --2- 172.21.15.179:0/446974841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980161580 0x7f79800cf2a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.820+0000 7f7984960700 1 -- 172.21.15.179:0/446974841 >> 172.21.15.179:0/446974841 conn(0x7f79800d2e70 msgr2=0x7f79800d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.820+0000 7f7984960700 1 -- 172.21.15.179:0/446974841 shutdown_connections 2024-05-08T22:35:30.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.820+0000 7f7984960700 1 -- 172.21.15.179:0/446974841 wait complete. 2024-05-08T22:35:30.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.820+0000 7f7984960700 1 Processor -- start 2024-05-08T22:35:30.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.820+0000 7f7984960700 1 -- start start 2024-05-08T22:35:30.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79800d1d50 0x7f798015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7980161580 0x7f7980154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cb9c0 con 0x7f798006baa0 2024-05-08T22:35:30.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cbca0 con 0x7f7980161580 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cbf80 con 0x7f79800d1d50 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32846/0 (socket says 172.21.15.179:32846) 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 -- 172.21.15.179:0/4179928420 learned_addr learned my addr 172.21.15.179:0/4179928420 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797f59e700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7980161580 0x7f7980154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797659c700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79800d1d50 0x7f798015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 -- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79800d1d50 msgr2=0x7f798015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79800d1d50 0x7f798015f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 -- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7980161580 msgr2=0x7f7980154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7980161580 0x7f7980154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 -- 172.21.15.179:0/4179928420 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700094b0 con 0x7f798006baa0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f797ed9d700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7970009fd0 tx=0x7f7970004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970004480 con 0x7f798006baa0 2024-05-08T22:35:30.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797001f070 con 0x7f798006baa0 2024-05-08T22:35:30.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79800cc260 con 0x7f798006baa0 2024-05-08T22:35:30.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970003900 con 0x7f798006baa0 2024-05-08T22:35:30.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.824+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800cca60 con 0x7f798006baa0 2024-05-08T22:35:30.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.828+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79800cc780 con 0x7f798006baa0 2024-05-08T22:35:30.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.828+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7970021030 con 0x7f798006baa0 2024-05-08T22:35:30.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.828+0000 7f7977fff700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 0x7f7964068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.828+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7970085bb0 con 0x7f798006baa0 2024-05-08T22:35:30.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.828+0000 7f797659c700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 0x7f7964068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.832+0000 7f797659c700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 0x7f7964068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79800cb000 tx=0x7f7968009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:30.832+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7970052f70 con 0x7f798006baa0 2024-05-08T22:35:31.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.004+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79800cc780 con 0x7f798006baa0 2024-05-08T22:35:31.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.004+0000 7f7977fff700 1 -- 172.21.15.179:0/4179928420 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16354 (secure 0 0 0) 0x7f79800cc780 con 0x7f798006baa0 2024-05-08T22:35:31.009 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:31.009 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":55,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:35:28.298349+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:31.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 msgr2=0x7f7964068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 0x7f7964068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79800cb000 tx=0x7f7968009240 comp rx=0 tx=0).stop 2024-05-08T22:35:31.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 msgr2=0x7f79800cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7970009fd0 tx=0x7f7970004770 comp rx=0 tx=0).stop 2024-05-08T22:35:31.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 shutdown_connections 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7964066730 0x7f7964068bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7980161580 0x7f7980154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79800d1d50 0x7f798015f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 --2- 172.21.15.179:0/4179928420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f798006baa0 0x7f79800cb070 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 >> 172.21.15.179:0/4179928420 conn(0x7f79800d2e70 msgr2=0x7f79800b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 shutdown_connections 2024-05-08T22:35:31.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.008+0000 7f7984960700 1 -- 172.21.15.179:0/4179928420 wait complete. 2024-05-08T22:35:31.024 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:35:31.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:35:31.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 Processor -- start 2024-05-08T22:35:31.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 -- start start 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 0x7f6c980caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6c980cff50 0x7f6c980cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c980c7d30 con 0x7f6c980c3b80 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c980b45c0 con 0x7f6c980cff50 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9ebae700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c980b4b40 con 0x7f6c980b0320 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9d3ab700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6c980c3b80 0x7f6c980cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57916/0 (socket says 172.21.15.179:57916) 2024-05-08T22:35:31.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9d3ab700 1 -- 172.21.15.179:0/1861012868 learned_addr learned my addr 172.21.15.179:0/1861012868 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/1861012868 >> v2:172.21.15.179:3300/0 conn(0x7f6c980cff50 0x7f6c980cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9dbac700 1 --2- 172.21.15.179:0/1861012868 >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 0x7f6c980caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9cbaa700 1 -- 172.21.15.179:0/1861012868 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502877623 0 0) 0x7f6c980c7d30 con 0x7f6c980c3b80 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9cbaa700 1 -- 172.21.15.179:0/1861012868 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c800035d0 con 0x7f6c980c3b80 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/1861012868 >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 msgr2=0x7f6c980caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/1861012868 >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 0x7f6c980caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/1861012868 >> v1:172.21.15.179:6789/0 conn(0x7f6c980c3b80 legacy=0x7f6c980cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/1861012868 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980b3a90 con 0x7f6c980cff50 2024-05-08T22:35:31.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9dbac700 1 --2- 172.21.15.179:0/1861012868 >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 0x7f6c980caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/1861012868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980cc350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6c980668d0 tx=0x7f6c90009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca96fecf540cf521 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9cbaa700 1 -- 172.21.15.179:0/1861012868 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011830 con 0x7f6c980cff50 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9cbaa700 1 -- 172.21.15.179:0/1861012868 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c90011990 con 0x7f6c980cff50 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.132+0000 7f6c9cbaa700 1 -- 172.21.15.179:0/1861012868 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011d20 con 0x7f6c980cff50 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- 172.21.15.179:0/1861012868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 msgr2=0x7f6c980cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/1861012868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980cc350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6c980668d0 tx=0x7f6c90009b30 comp rx=0 tx=0).stop 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- 172.21.15.179:0/1861012868 shutdown_connections 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/1861012868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980cc350 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/1861012868 >> v2:172.21.15.195:3300/0 conn(0x7f6c980b0320 0x7f6c980caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- 172.21.15.179:0/1861012868 >> 172.21.15.179:0/1861012868 conn(0x7f6c980cd6e0 msgr2=0x7f6c980cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- 172.21.15.179:0/1861012868 shutdown_connections 2024-05-08T22:35:31.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- 172.21.15.179:0/1861012868 wait complete. 2024-05-08T22:35:31.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 Processor -- start 2024-05-08T22:35:31.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.136+0000 7f6c9ebae700 1 -- start start 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c980b0320 0x7f6c9806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c980c3b80 0x7f6c981f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980c0740 con 0x7f6c980cff50 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981f7db0 con 0x7f6c980c3b80 2024-05-08T22:35:31.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980c4480 con 0x7f6c980b0320 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32870/0 (socket says 172.21.15.179:32870) 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/3006827167 learned_addr learned my addr 172.21.15.179:0/3006827167 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9dbac700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c980b0320 0x7f6c9806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9d3ab700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c980c3b80 0x7f6c981f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c980b0320 msgr2=0x7f6c9806fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c980b0320 0x7f6c9806fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c980c3b80 msgr2=0x7f6c981f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c980c3b80 0x7f6c981f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 -- 172.21.15.179:0/3006827167 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940094b0 con 0x7f6c980cff50 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9e3ad700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6c900090b0 tx=0x7f6c90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90004450 con 0x7f6c980cff50 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9001e070 con 0x7f6c980cff50 2024-05-08T22:35:31.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c900094b0 con 0x7f6c980cff50 2024-05-08T22:35:31.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90003900 con 0x7f6c980cff50 2024-05-08T22:35:31.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c980bfac0 con 0x7f6c980cff50 2024-05-08T22:35:31.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.140+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c980cc840 con 0x7f6c980cff50 2024-05-08T22:35:31.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.144+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c90003a60 con 0x7f6c980cff50 2024-05-08T22:35:31.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.144+0000 7f6c8e7fc700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 0x7f6c84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.144+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6c90084f70 con 0x7f6c980cff50 2024-05-08T22:35:31.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.144+0000 7f6c9dbac700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 0x7f6c84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.144+0000 7f6c9dbac700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 0x7f6c84068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009d80 tx=0x7f6c94009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.152+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6c90052110 con 0x7f6c980cff50 2024-05-08T22:35:31.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.320+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c980673e0 con 0x7f6c84066760 2024-05-08T22:35:31.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.336+0000 7f6c8e7fc700 1 -- 172.21.15.179:0/3006827167 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283627 (secure 0 0 0) 0x7f6c980673e0 con 0x7f6c84066760 2024-05-08T22:35:31.342 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 msgr2=0x7f6c84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 0x7f6c84068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009d80 tx=0x7f6c94009240 comp rx=0 tx=0).stop 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 msgr2=0x7f6c980c0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6c900090b0 tx=0x7f6c90004770 comp rx=0 tx=0).stop 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 shutdown_connections 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6c84066760 0x7f6c84068c20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c980c3b80 0x7f6c981f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c980cff50 0x7f6c980c0d00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 --2- 172.21.15.179:0/3006827167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c980b0320 0x7f6c9806fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 >> 172.21.15.179:0/3006827167 conn(0x7f6c980cd6e0 msgr2=0x7f6c980ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 shutdown_connections 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.340+0000 7f6c9ebae700 1 -- 172.21.15.179:0/3006827167 wait complete. 2024-05-08T22:35:31.346 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:35:31.358 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-05-08T22:35:29.424699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":238644,"kb_used_data":6048,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736076,"statfs":{"total":773094113280,"available":772849741824,"internally_reserved":0,"allocated":6193152,"data_stored":3363313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001117"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309175+0000","last_change":"2024-05-08T22:35:27.325823+0000","last_active":"2024-05-08T22:35:28.309175+0000","last_peered":"2024-05-08T22:35:28.309175+0000","last_clean":"2024-05-08T22:35:28.309175+0000","last_became_active":"2024-05-08T22:35:27.325376+0000","last_became_peered":"2024-05-08T22:35:27.325376+0000","last_unstale":"2024-05-08T22:35:28.309175+0000","last_undegraded":"2024-05-08T22:35:28.309175+0000","last_fullsized":"2024-05-08T22:35:28.309175+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:48.752736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389739+0000","last_change":"2024-05-08T22:35:27.322868+0000","last_active":"2024-05-08T22:35:28.389739+0000","last_peered":"2024-05-08T22:35:28.389739+0000","last_clean":"2024-05-08T22:35:28.389739+0000","last_became_active":"2024-05-08T22:35:27.322593+0000","last_became_peered":"2024-05-08T22:35:27.322593+0000","last_unstale":"2024-05-08T22:35:28.389739+0000","last_undegraded":"2024-05-08T22:35:28.389739+0000","last_fullsized":"2024-05-08T22:35:28.389739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:42.412562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308680+0000","last_change":"2024-05-08T22:35:27.314234+0000","last_active":"2024-05-08T22:35:28.308680+0000","last_peered":"2024-05-08T22:35:28.308680+0000","last_clean":"2024-05-08T22:35:28.308680+0000","last_became_active":"2024-05-08T22:35:27.313955+0000","last_became_peered":"2024-05-08T22:35:27.313955+0000","last_unstale":"2024-05-08T22:35:28.308680+0000","last_undegraded":"2024-05-08T22:35:28.308680+0000","last_fullsized":"2024-05-08T22:35:28.308680+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:49.793646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.314048+0000","last_change":"2024-05-08T22:35:27.311599+0000","last_active":"2024-05-08T22:35:27.314048+0000","last_peered":"2024-05-08T22:35:27.314048+0000","last_clean":"2024-05-08T22:35:27.314048+0000","last_became_active":"2024-05-08T22:35:27.310937+0000","last_became_peered":"2024-05-08T22:35:27.310937+0000","last_unstale":"2024-05-08T22:35:27.314048+0000","last_undegraded":"2024-05-08T22:35:27.314048+0000","last_fullsized":"2024-05-08T22:35:27.314048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:15.146319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.388984+0000","last_change":"2024-05-08T22:35:27.320616+0000","last_active":"2024-05-08T22:35:28.388984+0000","last_peered":"2024-05-08T22:35:28.388984+0000","last_clean":"2024-05-08T22:35:28.388984+0000","last_became_active":"2024-05-08T22:35:27.320099+0000","last_became_peered":"2024-05-08T22:35:27.320099+0000","last_unstale":"2024-05-08T22:35:28.388984+0000","last_undegraded":"2024-05-08T22:35:28.388984+0000","last_fullsized":"2024-05-08T22:35:28.388984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:31.157377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.319171+0000","last_change":"2024-05-08T22:35:27.313253+0000","last_active":"2024-05-08T22:35:27.319171+0000","last_peered":"2024-05-08T22:35:27.319171+0000","last_clean":"2024-05-08T22:35:27.319171+0000","last_became_active":"2024-05-08T22:35:27.312920+0000","last_became_peered":"2024-05-08T22:35:27.312920+0000","last_unstale":"2024-05-08T22:35:27.319171+0000","last_undegraded":"2024-05-08T22:35:27.319171+0000","last_fullsized":"2024-05-08T22:35:27.319171+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.285997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389282+0000","last_change":"2024-05-08T22:35:27.317585+0000","last_active":"2024-05-08T22:35:28.389282+0000","last_peered":"2024-05-08T22:35:28.389282+0000","last_clean":"2024-05-08T22:35:28.389282+0000","last_became_active":"2024-05-08T22:35:27.316985+0000","last_became_peered":"2024-05-08T22:35:27.316985+0000","last_unstale":"2024-05-08T22:35:28.389282+0000","last_undegraded":"2024-05-08T22:35:28.389282+0000","last_fullsized":"2024-05-08T22:35:28.389282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:20.226511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308397+0000","last_change":"2024-05-08T22:35:27.310416+0000","last_active":"2024-05-08T22:35:28.308397+0000","last_peered":"2024-05-08T22:35:28.308397+0000","last_clean":"2024-05-08T22:35:28.308397+0000","last_became_active":"2024-05-08T22:35:27.310116+0000","last_became_peered":"2024-05-08T22:35:27.310116+0000","last_unstale":"2024-05-08T22:35:28.308397+0000","last_undegraded":"2024-05-08T22:35:28.308397+0000","last_fullsized":"2024-05-08T22:35:28.308397+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:05.411796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390218+0000","last_change":"2024-05-08T22:35:27.328166+0000","last_active":"2024-05-08T22:35:28.390218+0000","last_peered":"2024-05-08T22:35:28.390218+0000","last_clean":"2024-05-08T22:35:28.390218+0000","last_became_active":"2024-05-08T22:35:27.327214+0000","last_became_peered":"2024-05-08T22:35:27.327214+0000","last_unstale":"2024-05-08T22:35:28.390218+0000","last_undegraded":"2024-05-08T22:35:28.390218+0000","last_fullsized":"2024-05-08T22:35:28.390218+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:15.146319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309380+0000","last_change":"2024-05-08T22:35:27.328835+0000","last_active":"2024-05-08T22:35:28.309380+0000","last_peered":"2024-05-08T22:35:28.309380+0000","last_clean":"2024-05-08T22:35:28.309380+0000","last_became_active":"2024-05-08T22:35:27.327786+0000","last_became_peered":"2024-05-08T22:35:27.327786+0000","last_unstale":"2024-05-08T22:35:28.309380+0000","last_undegraded":"2024-05-08T22:35:28.309380+0000","last_fullsized":"2024-05-08T22:35:28.309380+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:39.573030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389830+0000","last_change":"2024-05-08T22:35:27.323674+0000","last_active":"2024-05-08T22:35:28.389830+0000","last_peered":"2024-05-08T22:35:28.389830+0000","last_clean":"2024-05-08T22:35:28.389830+0000","last_became_active":"2024-05-08T22:35:27.323407+0000","last_became_peered":"2024-05-08T22:35:27.323407+0000","last_unstale":"2024-05-08T22:35:28.389830+0000","last_undegraded":"2024-05-08T22:35:28.389830+0000","last_fullsized":"2024-05-08T22:35:28.389830+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:58:54.724332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310376+0000","last_change":"2024-05-08T22:35:27.316013+0000","last_active":"2024-05-08T22:35:28.310376+0000","last_peered":"2024-05-08T22:35:28.310376+0000","last_clean":"2024-05-08T22:35:28.310376+0000","last_became_active":"2024-05-08T22:35:27.315670+0000","last_became_peered":"2024-05-08T22:35:27.315670+0000","last_unstale":"2024-05-08T22:35:28.310376+0000","last_undegraded":"2024-05-08T22:35:28.310376+0000","last_fullsized":"2024-05-08T22:35:28.310376+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:35.891712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310505+0000","last_change":"2024-05-08T22:35:27.312588+0000","last_active":"2024-05-08T22:35:28.310505+0000","last_peered":"2024-05-08T22:35:28.310505+0000","last_clean":"2024-05-08T22:35:28.310505+0000","last_became_active":"2024-05-08T22:35:27.312321+0000","last_became_peered":"2024-05-08T22:35:27.312321+0000","last_unstale":"2024-05-08T22:35:28.310505+0000","last_undegraded":"2024-05-08T22:35:28.310505+0000","last_fullsized":"2024-05-08T22:35:28.310505+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.405123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.322019+0000","last_change":"2024-05-08T22:35:27.314922+0000","last_active":"2024-05-08T22:35:27.322019+0000","last_peered":"2024-05-08T22:35:27.322019+0000","last_clean":"2024-05-08T22:35:27.322019+0000","last_became_active":"2024-05-08T22:35:27.314611+0000","last_became_peered":"2024-05-08T22:35:27.314611+0000","last_unstale":"2024-05-08T22:35:27.322019+0000","last_undegraded":"2024-05-08T22:35:27.322019+0000","last_fullsized":"2024-05-08T22:35:27.322019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:25.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.388777+0000","last_change":"2024-05-08T22:35:27.320409+0000","last_active":"2024-05-08T22:35:28.388777+0000","last_peered":"2024-05-08T22:35:28.388777+0000","last_clean":"2024-05-08T22:35:28.388777+0000","last_became_active":"2024-05-08T22:35:27.320162+0000","last_became_peered":"2024-05-08T22:35:27.320162+0000","last_unstale":"2024-05-08T22:35:28.388777+0000","last_undegraded":"2024-05-08T22:35:28.388777+0000","last_fullsized":"2024-05-08T22:35:28.388777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:06.580248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390304+0000","last_change":"2024-05-08T22:35:27.322796+0000","last_active":"2024-05-08T22:35:28.390304+0000","last_peered":"2024-05-08T22:35:28.390304+0000","last_clean":"2024-05-08T22:35:28.390304+0000","last_became_active":"2024-05-08T22:35:27.316878+0000","last_became_peered":"2024-05-08T22:35:27.316878+0000","last_unstale":"2024-05-08T22:35:28.390304+0000","last_undegraded":"2024-05-08T22:35:28.390304+0000","last_fullsized":"2024-05-08T22:35:28.390304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:23.999057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310463+0000","last_change":"2024-05-08T22:35:27.311737+0000","last_active":"2024-05-08T22:35:28.310463+0000","last_peered":"2024-05-08T22:35:28.310463+0000","last_clean":"2024-05-08T22:35:28.310463+0000","last_became_active":"2024-05-08T22:35:27.311480+0000","last_became_peered":"2024-05-08T22:35:27.311480+0000","last_unstale":"2024-05-08T22:35:28.310463+0000","last_undegraded":"2024-05-08T22:35:28.310463+0000","last_fullsized":"2024-05-08T22:35:28.310463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:32.494317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389252+0000","last_change":"2024-05-08T22:35:27.317925+0000","last_active":"2024-05-08T22:35:28.389252+0000","last_peered":"2024-05-08T22:35:28.389252+0000","last_clean":"2024-05-08T22:35:28.389252+0000","last_became_active":"2024-05-08T22:35:27.317495+0000","last_became_peered":"2024-05-08T22:35:27.317495+0000","last_unstale":"2024-05-08T22:35:28.389252+0000","last_undegraded":"2024-05-08T22:35:28.389252+0000","last_fullsized":"2024-05-08T22:35:28.389252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:05.060058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308685+0000","last_change":"2024-05-08T22:35:27.317354+0000","last_active":"2024-05-08T22:35:28.308685+0000","last_peered":"2024-05-08T22:35:28.308685+0000","last_clean":"2024-05-08T22:35:28.308685+0000","last_became_active":"2024-05-08T22:35:27.317065+0000","last_became_peered":"2024-05-08T22:35:27.317065+0000","last_unstale":"2024-05-08T22:35:28.308685+0000","last_undegraded":"2024-05-08T22:35:28.308685+0000","last_fullsized":"2024-05-08T22:35:28.308685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:36.025168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389148+0000","last_change":"2024-05-08T22:35:27.320949+0000","last_active":"2024-05-08T22:35:28.389148+0000","last_peered":"2024-05-08T22:35:28.389148+0000","last_clean":"2024-05-08T22:35:28.389148+0000","last_became_active":"2024-05-08T22:35:27.320717+0000","last_became_peered":"2024-05-08T22:35:27.320717+0000","last_unstale":"2024-05-08T22:35:28.389148+0000","last_undegraded":"2024-05-08T22:35:28.389148+0000","last_fullsized":"2024-05-08T22:35:28.389148+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:04.120516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330922+0000","last_change":"2024-05-08T22:35:27.330044+0000","last_active":"2024-05-08T22:35:27.330922+0000","last_peered":"2024-05-08T22:35:27.330922+0000","last_clean":"2024-05-08T22:35:27.330922+0000","last_became_active":"2024-05-08T22:35:27.329721+0000","last_became_peered":"2024-05-08T22:35:27.329721+0000","last_unstale":"2024-05-08T22:35:27.330922+0000","last_undegraded":"2024-05-08T22:35:27.330922+0000","last_fullsized":"2024-05-08T22:35:27.330922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:18.405587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308145+0000","last_change":"2024-05-08T22:35:27.326013+0000","last_active":"2024-05-08T22:35:28.308145+0000","last_peered":"2024-05-08T22:35:28.308145+0000","last_clean":"2024-05-08T22:35:28.308145+0000","last_became_active":"2024-05-08T22:35:27.325678+0000","last_became_peered":"2024-05-08T22:35:27.325678+0000","last_unstale":"2024-05-08T22:35:28.308145+0000","last_undegraded":"2024-05-08T22:35:28.308145+0000","last_fullsized":"2024-05-08T22:35:28.308145+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:18.405587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310763+0000","last_change":"2024-05-08T22:35:26.937256+0000","last_active":"2024-05-08T22:35:28.310763+0000","last_peered":"2024-05-08T22:35:28.310763+0000","last_clean":"2024-05-08T22:35:28.310763+0000","last_became_active":"2024-05-08T22:35:26.936950+0000","last_became_peered":"2024-05-08T22:35:26.936950+0000","last_unstale":"2024-05-08T22:35:28.310763+0000","last_undegraded":"2024-05-08T22:35:28.310763+0000","last_fullsized":"2024-05-08T22:35:28.310763+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:30.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.326166+0000","last_change":"2024-05-08T22:35:27.324975+0000","last_active":"2024-05-08T22:35:27.326166+0000","last_peered":"2024-05-08T22:35:27.326166+0000","last_clean":"2024-05-08T22:35:27.326166+0000","last_became_active":"2024-05-08T22:35:27.324646+0000","last_became_peered":"2024-05-08T22:35:27.324646+0000","last_unstale":"2024-05-08T22:35:27.326166+0000","last_undegraded":"2024-05-08T22:35:27.326166+0000","last_fullsized":"2024-05-08T22:35:27.326166+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:17.288386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310443+0000","last_change":"2024-05-08T22:35:27.310545+0000","last_active":"2024-05-08T22:35:28.310443+0000","last_peered":"2024-05-08T22:35:28.310443+0000","last_clean":"2024-05-08T22:35:28.310443+0000","last_became_active":"2024-05-08T22:35:27.310281+0000","last_became_peered":"2024-05-08T22:35:27.310281+0000","last_unstale":"2024-05-08T22:35:28.310443+0000","last_undegraded":"2024-05-08T22:35:28.310443+0000","last_fullsized":"2024-05-08T22:35:28.310443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:34.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.335221+0000","last_change":"2024-05-08T22:35:26.950336+0000","last_active":"2024-05-08T22:35:27.335221+0000","last_peered":"2024-05-08T22:35:27.335221+0000","last_clean":"2024-05-08T22:35:27.335221+0000","last_became_active":"2024-05-08T22:35:26.949995+0000","last_became_peered":"2024-05-08T22:35:26.949995+0000","last_unstale":"2024-05-08T22:35:27.335221+0000","last_undegraded":"2024-05-08T22:35:27.335221+0000","last_fullsized":"2024-05-08T22:35:27.335221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:20.556169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.318204+0000","last_change":"2024-05-08T22:35:27.317311+0000","last_active":"2024-05-08T22:35:27.318204+0000","last_peered":"2024-05-08T22:35:27.318204+0000","last_clean":"2024-05-08T22:35:27.318204+0000","last_became_active":"2024-05-08T22:35:27.317045+0000","last_became_peered":"2024-05-08T22:35:27.317045+0000","last_unstale":"2024-05-08T22:35:27.318204+0000","last_undegraded":"2024-05-08T22:35:27.318204+0000","last_fullsized":"2024-05-08T22:35:27.318204+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:06.683189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389333+0000","last_change":"2024-05-08T22:35:26.941527+0000","last_active":"2024-05-08T22:35:28.389333+0000","last_peered":"2024-05-08T22:35:28.389333+0000","last_clean":"2024-05-08T22:35:28.389333+0000","last_became_active":"2024-05-08T22:35:26.941048+0000","last_became_peered":"2024-05-08T22:35:26.941048+0000","last_unstale":"2024-05-08T22:35:28.389333+0000","last_undegraded":"2024-05-08T22:35:28.389333+0000","last_fullsized":"2024-05-08T22:35:28.389333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:23.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308875+0000","last_change":"2024-05-08T22:35:27.317366+0000","last_active":"2024-05-08T22:35:28.308875+0000","last_peered":"2024-05-08T22:35:28.308875+0000","last_clean":"2024-05-08T22:35:28.308875+0000","last_became_active":"2024-05-08T22:35:27.317043+0000","last_became_peered":"2024-05-08T22:35:27.317043+0000","last_unstale":"2024-05-08T22:35:28.308875+0000","last_undegraded":"2024-05-08T22:35:28.308875+0000","last_fullsized":"2024-05-08T22:35:28.308875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:03.886275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.334604+0000","last_change":"2024-05-08T22:35:26.934242+0000","last_active":"2024-05-08T22:35:27.334604+0000","last_peered":"2024-05-08T22:35:27.334604+0000","last_clean":"2024-05-08T22:35:27.334604+0000","last_became_active":"2024-05-08T22:35:26.932077+0000","last_became_peered":"2024-05-08T22:35:26.932077+0000","last_unstale":"2024-05-08T22:35:27.334604+0000","last_undegraded":"2024-05-08T22:35:27.334604+0000","last_fullsized":"2024-05-08T22:35:27.334604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:52.669092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310315+0000","last_change":"2024-05-08T22:35:26.933939+0000","last_active":"2024-05-08T22:35:28.310315+0000","last_peered":"2024-05-08T22:35:28.310315+0000","last_clean":"2024-05-08T22:35:28.310315+0000","last_became_active":"2024-05-08T22:35:26.933674+0000","last_became_peered":"2024-05-08T22:35:26.933674+0000","last_unstale":"2024-05-08T22:35:28.310315+0000","last_undegraded":"2024-05-08T22:35:28.310315+0000","last_fullsized":"2024-05-08T22:35:28.310315+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:17.519768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308861+0000","last_change":"2024-05-08T22:35:27.320148+0000","last_active":"2024-05-08T22:35:28.308861+0000","last_peered":"2024-05-08T22:35:28.308861+0000","last_clean":"2024-05-08T22:35:28.308861+0000","last_became_active":"2024-05-08T22:35:27.319860+0000","last_became_peered":"2024-05-08T22:35:27.319860+0000","last_unstale":"2024-05-08T22:35:28.308861+0000","last_undegraded":"2024-05-08T22:35:28.308861+0000","last_fullsized":"2024-05-08T22:35:28.308861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:50.819918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389017+0000","last_change":"2024-05-08T22:35:26.945863+0000","last_active":"2024-05-08T22:35:28.389017+0000","last_peered":"2024-05-08T22:35:28.389017+0000","last_clean":"2024-05-08T22:35:28.389017+0000","last_became_active":"2024-05-08T22:35:26.945607+0000","last_became_peered":"2024-05-08T22:35:26.945607+0000","last_unstale":"2024-05-08T22:35:28.389017+0000","last_undegraded":"2024-05-08T22:35:28.389017+0000","last_fullsized":"2024-05-08T22:35:28.389017+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:37.159065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.317691+0000","last_change":"2024-05-08T22:35:27.315255+0000","last_active":"2024-05-08T22:35:27.317691+0000","last_peered":"2024-05-08T22:35:27.317691+0000","last_clean":"2024-05-08T22:35:27.317691+0000","last_became_active":"2024-05-08T22:35:27.314765+0000","last_became_peered":"2024-05-08T22:35:27.314765+0000","last_unstale":"2024-05-08T22:35:27.317691+0000","last_undegraded":"2024-05-08T22:35:27.317691+0000","last_fullsized":"2024-05-08T22:35:27.317691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:05.899140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308996+0000","last_change":"2024-05-08T22:35:26.933923+0000","last_active":"2024-05-08T22:35:28.308996+0000","last_peered":"2024-05-08T22:35:28.308996+0000","last_clean":"2024-05-08T22:35:28.308996+0000","last_became_active":"2024-05-08T22:35:26.932465+0000","last_became_peered":"2024-05-08T22:35:26.932465+0000","last_unstale":"2024-05-08T22:35:28.308996+0000","last_undegraded":"2024-05-08T22:35:28.308996+0000","last_fullsized":"2024-05-08T22:35:28.308996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:19.028838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.326186+0000","last_change":"2024-05-08T22:35:27.325064+0000","last_active":"2024-05-08T22:35:27.326186+0000","last_peered":"2024-05-08T22:35:27.326186+0000","last_clean":"2024-05-08T22:35:27.326186+0000","last_became_active":"2024-05-08T22:35:27.324784+0000","last_became_peered":"2024-05-08T22:35:27.324784+0000","last_unstale":"2024-05-08T22:35:27.326186+0000","last_undegraded":"2024-05-08T22:35:27.326186+0000","last_fullsized":"2024-05-08T22:35:27.326186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:32.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.304147+0000","last_change":"2024-05-08T22:35:26.942188+0000","last_active":"2024-05-08T22:35:27.304147+0000","last_peered":"2024-05-08T22:35:27.304147+0000","last_clean":"2024-05-08T22:35:27.304147+0000","last_became_active":"2024-05-08T22:35:26.941953+0000","last_became_peered":"2024-05-08T22:35:26.941953+0000","last_unstale":"2024-05-08T22:35:27.304147+0000","last_undegraded":"2024-05-08T22:35:27.304147+0000","last_fullsized":"2024-05-08T22:35:27.304147+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:47.743474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389365+0000","last_change":"2024-05-08T22:35:27.317757+0000","last_active":"2024-05-08T22:35:28.389365+0000","last_peered":"2024-05-08T22:35:28.389365+0000","last_clean":"2024-05-08T22:35:28.389365+0000","last_became_active":"2024-05-08T22:35:27.317271+0000","last_became_peered":"2024-05-08T22:35:27.317271+0000","last_unstale":"2024-05-08T22:35:28.389365+0000","last_undegraded":"2024-05-08T22:35:28.389365+0000","last_fullsized":"2024-05-08T22:35:28.389365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:58.388506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309033+0000","last_change":"2024-05-08T22:35:26.936242+0000","last_active":"2024-05-08T22:35:28.309033+0000","last_peered":"2024-05-08T22:35:28.309033+0000","last_clean":"2024-05-08T22:35:28.309033+0000","last_became_active":"2024-05-08T22:35:26.935959+0000","last_became_peered":"2024-05-08T22:35:26.935959+0000","last_unstale":"2024-05-08T22:35:28.309033+0000","last_undegraded":"2024-05-08T22:35:28.309033+0000","last_fullsized":"2024-05-08T22:35:28.309033+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.731401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.314060+0000","last_change":"2024-05-08T22:35:27.311868+0000","last_active":"2024-05-08T22:35:27.314060+0000","last_peered":"2024-05-08T22:35:27.314060+0000","last_clean":"2024-05-08T22:35:27.314060+0000","last_became_active":"2024-05-08T22:35:27.311275+0000","last_became_peered":"2024-05-08T22:35:27.311275+0000","last_unstale":"2024-05-08T22:35:27.314060+0000","last_undegraded":"2024-05-08T22:35:27.314060+0000","last_fullsized":"2024-05-08T22:35:27.314060+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:04.120516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.327484+0000","last_change":"2024-05-08T22:35:27.325790+0000","last_active":"2024-05-08T22:35:27.327484+0000","last_peered":"2024-05-08T22:35:27.327484+0000","last_clean":"2024-05-08T22:35:27.327484+0000","last_became_active":"2024-05-08T22:35:27.325237+0000","last_became_peered":"2024-05-08T22:35:27.325237+0000","last_unstale":"2024-05-08T22:35:27.327484+0000","last_undegraded":"2024-05-08T22:35:27.327484+0000","last_fullsized":"2024-05-08T22:35:27.327484+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.678354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389461+0000","last_change":"2024-05-08T22:35:26.945770+0000","last_active":"2024-05-08T22:35:28.389461+0000","last_peered":"2024-05-08T22:35:28.389461+0000","last_clean":"2024-05-08T22:35:28.389461+0000","last_became_active":"2024-05-08T22:35:26.945490+0000","last_became_peered":"2024-05-08T22:35:26.945490+0000","last_unstale":"2024-05-08T22:35:28.389461+0000","last_undegraded":"2024-05-08T22:35:28.389461+0000","last_fullsized":"2024-05-08T22:35:28.389461+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.731401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389968+0000","last_change":"2024-05-08T22:35:27.320671+0000","last_active":"2024-05-08T22:35:28.389968+0000","last_peered":"2024-05-08T22:35:28.389968+0000","last_clean":"2024-05-08T22:35:28.389968+0000","last_became_active":"2024-05-08T22:35:27.318357+0000","last_became_peered":"2024-05-08T22:35:27.318357+0000","last_unstale":"2024-05-08T22:35:28.389968+0000","last_undegraded":"2024-05-08T22:35:28.389968+0000","last_fullsized":"2024-05-08T22:35:28.389968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:47.849291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310532+0000","last_change":"2024-05-08T22:35:26.942632+0000","last_active":"2024-05-08T22:35:28.310532+0000","last_peered":"2024-05-08T22:35:28.310532+0000","last_clean":"2024-05-08T22:35:28.310532+0000","last_became_active":"2024-05-08T22:35:26.942381+0000","last_became_peered":"2024-05-08T22:35:26.942381+0000","last_unstale":"2024-05-08T22:35:28.310532+0000","last_undegraded":"2024-05-08T22:35:28.310532+0000","last_fullsized":"2024-05-08T22:35:28.310532+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:17.212132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308397+0000","last_change":"2024-05-08T22:35:26.935401+0000","last_active":"2024-05-08T22:35:28.308397+0000","last_peered":"2024-05-08T22:35:28.308397+0000","last_clean":"2024-05-08T22:35:28.308397+0000","last_became_active":"2024-05-08T22:35:26.935106+0000","last_became_peered":"2024-05-08T22:35:26.935106+0000","last_unstale":"2024-05-08T22:35:28.308397+0000","last_undegraded":"2024-05-08T22:35:28.308397+0000","last_fullsized":"2024-05-08T22:35:28.308397+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:20.556169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390194+0000","last_change":"2024-05-08T22:35:27.327396+0000","last_active":"2024-05-08T22:35:28.390194+0000","last_peered":"2024-05-08T22:35:28.390194+0000","last_clean":"2024-05-08T22:35:28.390194+0000","last_became_active":"2024-05-08T22:35:27.326988+0000","last_became_peered":"2024-05-08T22:35:27.326988+0000","last_unstale":"2024-05-08T22:35:28.390194+0000","last_undegraded":"2024-05-08T22:35:28.390194+0000","last_fullsized":"2024-05-08T22:35:28.390194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:10.388711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390334+0000","last_change":"2024-05-08T22:35:26.953911+0000","last_active":"2024-05-08T22:35:28.390334+0000","last_peered":"2024-05-08T22:35:28.390334+0000","last_clean":"2024-05-08T22:35:28.390334+0000","last_became_active":"2024-05-08T22:35:26.953515+0000","last_became_peered":"2024-05-08T22:35:26.953515+0000","last_unstale":"2024-05-08T22:35:28.390334+0000","last_undegraded":"2024-05-08T22:35:28.390334+0000","last_fullsized":"2024-05-08T22:35:28.390334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:51.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.335280+0000","last_change":"2024-05-08T22:35:26.935048+0000","last_active":"2024-05-08T22:35:27.335280+0000","last_peered":"2024-05-08T22:35:27.335280+0000","last_clean":"2024-05-08T22:35:27.335280+0000","last_became_active":"2024-05-08T22:35:26.934674+0000","last_became_peered":"2024-05-08T22:35:26.934674+0000","last_unstale":"2024-05-08T22:35:27.335280+0000","last_undegraded":"2024-05-08T22:35:27.335280+0000","last_fullsized":"2024-05-08T22:35:27.335280+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:16.644630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.318243+0000","last_change":"2024-05-08T22:35:27.317319+0000","last_active":"2024-05-08T22:35:27.318243+0000","last_peered":"2024-05-08T22:35:27.318243+0000","last_clean":"2024-05-08T22:35:27.318243+0000","last_became_active":"2024-05-08T22:35:27.317033+0000","last_became_peered":"2024-05-08T22:35:27.317033+0000","last_unstale":"2024-05-08T22:35:27.318243+0000","last_undegraded":"2024-05-08T22:35:27.318243+0000","last_fullsized":"2024-05-08T22:35:27.318243+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:10.388711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310834+0000","last_change":"2024-05-08T22:35:26.939712+0000","last_active":"2024-05-08T22:35:28.310834+0000","last_peered":"2024-05-08T22:35:28.310834+0000","last_clean":"2024-05-08T22:35:28.310834+0000","last_became_active":"2024-05-08T22:35:26.939466+0000","last_became_peered":"2024-05-08T22:35:26.939466+0000","last_unstale":"2024-05-08T22:35:28.310834+0000","last_undegraded":"2024-05-08T22:35:28.310834+0000","last_fullsized":"2024-05-08T22:35:28.310834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:11.813254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310588+0000","last_change":"2024-05-08T22:35:26.950270+0000","last_active":"2024-05-08T22:35:28.310588+0000","last_peered":"2024-05-08T22:35:28.310588+0000","last_clean":"2024-05-08T22:35:28.310588+0000","last_became_active":"2024-05-08T22:35:26.949989+0000","last_became_peered":"2024-05-08T22:35:26.949989+0000","last_unstale":"2024-05-08T22:35:28.310588+0000","last_undegraded":"2024-05-08T22:35:28.310588+0000","last_fullsized":"2024-05-08T22:35:28.310588+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:54.732370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.327502+0000","last_change":"2024-05-08T22:35:27.325954+0000","last_active":"2024-05-08T22:35:27.327502+0000","last_peered":"2024-05-08T22:35:27.327502+0000","last_clean":"2024-05-08T22:35:27.327502+0000","last_became_active":"2024-05-08T22:35:27.325547+0000","last_became_peered":"2024-05-08T22:35:27.325547+0000","last_unstale":"2024-05-08T22:35:27.327502+0000","last_undegraded":"2024-05-08T22:35:27.327502+0000","last_fullsized":"2024-05-08T22:35:27.327502+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:28.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310654+0000","last_change":"2024-05-08T22:35:27.313024+0000","last_active":"2024-05-08T22:35:28.310654+0000","last_peered":"2024-05-08T22:35:28.310654+0000","last_clean":"2024-05-08T22:35:28.310654+0000","last_became_active":"2024-05-08T22:35:27.312702+0000","last_became_peered":"2024-05-08T22:35:27.312702+0000","last_unstale":"2024-05-08T22:35:28.310654+0000","last_undegraded":"2024-05-08T22:35:28.310654+0000","last_fullsized":"2024-05-08T22:35:28.310654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:46.408378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390107+0000","last_change":"2024-05-08T22:35:27.317319+0000","last_active":"2024-05-08T22:35:28.390107+0000","last_peered":"2024-05-08T22:35:28.390107+0000","last_clean":"2024-05-08T22:35:28.390107+0000","last_became_active":"2024-05-08T22:35:27.316970+0000","last_became_peered":"2024-05-08T22:35:27.316970+0000","last_unstale":"2024-05-08T22:35:28.390107+0000","last_undegraded":"2024-05-08T22:35:28.390107+0000","last_fullsized":"2024-05-08T22:35:28.390107+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:59.628309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389215+0000","last_change":"2024-05-08T22:35:26.944044+0000","last_active":"2024-05-08T22:35:28.389215+0000","last_peered":"2024-05-08T22:35:28.389215+0000","last_clean":"2024-05-08T22:35:28.389215+0000","last_became_active":"2024-05-08T22:35:26.943342+0000","last_became_peered":"2024-05-08T22:35:26.943342+0000","last_unstale":"2024-05-08T22:35:28.389215+0000","last_undegraded":"2024-05-08T22:35:28.389215+0000","last_fullsized":"2024-05-08T22:35:28.389215+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:20.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.303707+0000","last_change":"2024-05-08T22:35:26.937951+0000","last_active":"2024-05-08T22:35:27.303707+0000","last_peered":"2024-05-08T22:35:27.303707+0000","last_clean":"2024-05-08T22:35:27.303707+0000","last_became_active":"2024-05-08T22:35:26.937674+0000","last_became_peered":"2024-05-08T22:35:26.937674+0000","last_unstale":"2024-05-08T22:35:27.303707+0000","last_undegraded":"2024-05-08T22:35:27.303707+0000","last_fullsized":"2024-05-08T22:35:27.303707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:11.049425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330740+0000","last_change":"2024-05-08T22:35:27.329009+0000","last_active":"2024-05-08T22:35:27.330740+0000","last_peered":"2024-05-08T22:35:27.330740+0000","last_clean":"2024-05-08T22:35:27.330740+0000","last_became_active":"2024-05-08T22:35:27.328775+0000","last_became_peered":"2024-05-08T22:35:27.328775+0000","last_unstale":"2024-05-08T22:35:27.330740+0000","last_undegraded":"2024-05-08T22:35:27.330740+0000","last_fullsized":"2024-05-08T22:35:27.330740+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:48.752736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310771+0000","last_change":"2024-05-08T22:35:26.948548+0000","last_active":"2024-05-08T22:35:28.310771+0000","last_peered":"2024-05-08T22:35:28.310771+0000","last_clean":"2024-05-08T22:35:28.310771+0000","last_became_active":"2024-05-08T22:35:26.948278+0000","last_became_peered":"2024-05-08T22:35:26.948278+0000","last_unstale":"2024-05-08T22:35:28.310771+0000","last_undegraded":"2024-05-08T22:35:28.310771+0000","last_fullsized":"2024-05-08T22:35:28.310771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:18.517835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309185+0000","last_change":"2024-05-08T22:35:26.936561+0000","last_active":"2024-05-08T22:35:28.309185+0000","last_peered":"2024-05-08T22:35:28.309185+0000","last_clean":"2024-05-08T22:35:28.309185+0000","last_became_active":"2024-05-08T22:35:26.936294+0000","last_became_peered":"2024-05-08T22:35:26.936294+0000","last_unstale":"2024-05-08T22:35:28.309185+0000","last_undegraded":"2024-05-08T22:35:28.309185+0000","last_fullsized":"2024-05-08T22:35:28.309185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:09.211946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389611+0000","last_change":"2024-05-08T22:35:26.933098+0000","last_active":"2024-05-08T22:35:28.389611+0000","last_peered":"2024-05-08T22:35:28.389611+0000","last_clean":"2024-05-08T22:35:28.389611+0000","last_became_active":"2024-05-08T22:35:26.932764+0000","last_became_peered":"2024-05-08T22:35:26.932764+0000","last_unstale":"2024-05-08T22:35:28.389611+0000","last_undegraded":"2024-05-08T22:35:28.389611+0000","last_fullsized":"2024-05-08T22:35:28.389611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:19.028838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389632+0000","last_change":"2024-05-08T22:35:27.317900+0000","last_active":"2024-05-08T22:35:28.389632+0000","last_peered":"2024-05-08T22:35:28.389632+0000","last_clean":"2024-05-08T22:35:28.389632+0000","last_became_active":"2024-05-08T22:35:27.317163+0000","last_became_peered":"2024-05-08T22:35:27.317163+0000","last_unstale":"2024-05-08T22:35:28.389632+0000","last_undegraded":"2024-05-08T22:35:28.389632+0000","last_fullsized":"2024-05-08T22:35:28.389632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:39.414972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330015+0000","last_change":"2024-05-08T22:35:27.326411+0000","last_active":"2024-05-08T22:35:27.330015+0000","last_peered":"2024-05-08T22:35:27.330015+0000","last_clean":"2024-05-08T22:35:27.330015+0000","last_became_active":"2024-05-08T22:35:27.326049+0000","last_became_peered":"2024-05-08T22:35:27.326049+0000","last_unstale":"2024-05-08T22:35:27.330015+0000","last_undegraded":"2024-05-08T22:35:27.330015+0000","last_fullsized":"2024-05-08T22:35:27.330015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:39.573030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:25.903880+0000","last_change":"2024-05-08T22:35:25.903880+0000","last_active":"2024-05-08T22:35:25.903880+0000","last_peered":"2024-05-08T22:35:25.903880+0000","last_clean":"2024-05-08T22:35:25.903880+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:25.903880+0000","last_undegraded":"2024-05-08T22:35:25.903880+0000","last_fullsized":"2024-05-08T22:35:25.903880+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389802+0000","last_change":"2024-05-08T22:35:27.318014+0000","last_active":"2024-05-08T22:35:28.389802+0000","last_peered":"2024-05-08T22:35:28.389802+0000","last_clean":"2024-05-08T22:35:28.389802+0000","last_became_active":"2024-05-08T22:35:27.317499+0000","last_became_peered":"2024-05-08T22:35:27.317499+0000","last_unstale":"2024-05-08T22:35:28.389802+0000","last_undegraded":"2024-05-08T22:35:28.389802+0000","last_fullsized":"2024-05-08T22:35:28.389802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:03.886275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.306013+0000","last_change":"2024-05-08T22:35:26.947286+0000","last_active":"2024-05-08T22:35:27.306013+0000","last_peered":"2024-05-08T22:35:27.306013+0000","last_clean":"2024-05-08T22:35:27.306013+0000","last_became_active":"2024-05-08T22:35:26.947018+0000","last_became_peered":"2024-05-08T22:35:26.947018+0000","last_unstale":"2024-05-08T22:35:27.306013+0000","last_undegraded":"2024-05-08T22:35:27.306013+0000","last_fullsized":"2024-05-08T22:35:27.306013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:38.563768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:35:26.913142+0000","last_change":"2024-05-08T22:35:26.913142+0000","last_active":"2024-05-08T22:35:26.913142+0000","last_peered":"2024-05-08T22:35:26.913142+0000","last_clean":"2024-05-08T22:35:26.913142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:35:26.913142+0000","last_undegraded":"2024-05-08T22:35:26.913142+0000","last_fullsized":"2024-05-08T22:35:26.913142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.323979+0000","last_change":"2024-05-08T22:35:26.939371+0000","last_active":"2024-05-08T22:35:27.323979+0000","last_peered":"2024-05-08T22:35:27.323979+0000","last_clean":"2024-05-08T22:35:27.323979+0000","last_became_active":"2024-05-08T22:35:26.939120+0000","last_became_peered":"2024-05-08T22:35:26.939120+0000","last_unstale":"2024-05-08T22:35:27.323979+0000","last_undegraded":"2024-05-08T22:35:27.323979+0000","last_fullsized":"2024-05-08T22:35:27.323979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:23.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310181+0000","last_change":"2024-05-08T22:35:27.324273+0000","last_active":"2024-05-08T22:35:28.310181+0000","last_peered":"2024-05-08T22:35:28.310181+0000","last_clean":"2024-05-08T22:35:28.310181+0000","last_became_active":"2024-05-08T22:35:27.323940+0000","last_became_peered":"2024-05-08T22:35:27.323940+0000","last_unstale":"2024-05-08T22:35:28.310181+0000","last_undegraded":"2024-05-08T22:35:28.310181+0000","last_fullsized":"2024-05-08T22:35:28.310181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:09.216183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390035+0000","last_change":"2024-05-08T22:35:26.945692+0000","last_active":"2024-05-08T22:35:28.390035+0000","last_peered":"2024-05-08T22:35:28.390035+0000","last_clean":"2024-05-08T22:35:28.390035+0000","last_became_active":"2024-05-08T22:35:26.945440+0000","last_became_peered":"2024-05-08T22:35:26.945440+0000","last_unstale":"2024-05-08T22:35:28.390035+0000","last_undegraded":"2024-05-08T22:35:28.390035+0000","last_fullsized":"2024-05-08T22:35:28.390035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:09.211946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.301950+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:35:27.301950+0000","last_peered":"2024-05-08T22:35:27.301950+0000","last_clean":"2024-05-08T22:35:27.301950+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:35:27.301950+0000","last_undegraded":"2024-05-08T22:35:27.301950+0000","last_fullsized":"2024-05-08T22:35:27.301950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":48,"acting":48,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":254454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95699999999999996,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.40999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.89900000000000002,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88300000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.98699999999999999}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.751,"15min":0.75700000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95699999999999996,"5min":1.21,"15min":1.21},"last":0.80300000000000005}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.627,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.68300000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.234}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.276,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.02,"15min":1.03},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.056}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.96999999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.0649999999999999,"15min":1.0629999999999999},"min":{"1min":0.83399999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3480000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.097,"15min":1.107},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.6779999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.034,"15min":1.0329999999999999},"min":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.462,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.3029999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":254454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82599999999999996,"5min":0.995,"15min":0.995},"last":1.006}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98799999999999999,"5min":1.04,"15min":1.04},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1499999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.018},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1.583}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.96399999999999997,"15min":0.98199999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.1839999999999999,"5min":1.431,"15min":1.431},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96199999999999997,"15min":0.96799999999999997},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3180000000000001,"5min":1.508,"15min":1.508},"last":0.91800000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.163,"5min":1.327,"15min":1.327},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.79600000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.605,"5min":1.726,"15min":1.726},"last":0.85399999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.996},"min":{"1min":0.72499999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.244,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.427},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.0409999999999999}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.266,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0509999999999999,"15min":1.0609999999999999},"min":{"1min":0.81200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.5049999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1236992,"data_stored":870220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.83799999999999997,"15min":0.85899999999999999},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91600000000000004,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.73199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.071,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85299999999999998},"min":{"1min":0.499,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.167,"15min":1.167},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.139,"5min":1.419,"15min":1.419},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95399999999999996,"15min":0.96299999999999997},"min":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.091,"5min":1.337,"15min":1.337},"last":1.0369999999999999}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1899999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.83499999999999996},"min":{"1min":0.627,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.125,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.1619999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.96699999999999997,"15min":0.97499999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2370000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0629999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.98399999999999999},"min":{"1min":0.65500000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.042,"15min":1.0509999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.331,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.099}]},{"osd":6,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94499999999999995,"15min":0.94899999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.133,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.075,"15min":1.091},"min":{"1min":0.78200000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.359,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.4870000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":643072,"data_stored":280380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.077,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.86699999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0529999999999999,"5min":1.141,"15min":1.141},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.878,"15min":0.88900000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.05,"5min":1.758,"15min":1.758},"last":0.747},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.02}]},{"osd":2,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":1},"min":{"1min":0.58999999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1759999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.93000000000000005,"15min":0.93799999999999994},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.363,"5min":1.419,"15min":1.419},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.091,"15min":1.115},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3220000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":1.004},"min":{"1min":0.61499999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.274,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.002,"15min":1.0049999999999999},"min":{"1min":0.69699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.054,"15min":1.0680000000000001},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4430000000000001,"5min":1.458,"15min":1.458},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.139,"15min":1.157},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4079999999999999,"5min":1.681,"15min":1.681},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.139,"15min":1.159},"min":{"1min":0.79300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.466,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.96699999999999997}]},{"osd":7,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.087,"15min":1.109},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4330000000000001,"5min":1.835,"15min":1.835},"last":0.997},{"interface":"front","average":{"1min":1.119,"5min":1.234,"15min":1.2529999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6970000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":634880,"data_stored":279728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.64200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.86099999999999999,"15min":0.85699999999999998},"min":{"1min":0.60799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.80800000000000005,"15min":0.79700000000000004},"min":{"1min":0.57299999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.876,"15min":0.87},"min":{"1min":0.51500000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.879,"15min":0.86699999999999999},"min":{"1min":0.78600000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.96999999999999997,"15min":0.96399999999999997},"min":{"1min":0.63400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.90600000000000003,"15min":0.89200000000000002},"min":{"1min":0.70899999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.087},{"interface":"front","average":{"1min":1.032,"5min":0.98299999999999998,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.92100000000000004,"15min":0.90100000000000002},"min":{"1min":0.73599999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0469999999999999,"15min":1.0369999999999999},"min":{"1min":0.88300000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.083,"15min":1.081},"min":{"1min":0.81599999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.117},{"interface":"front","average":{"1min":1.159,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.95999999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2769999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":622592,"data_stored":272633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1339999999999999,"5min":1.28,"15min":1.28},"last":1.165},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.85499999999999998,"15min":0.85799999999999998},"min":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.878},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1499999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":2,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.878,"15min":0.89400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98699999999999999,"5min":1.29,"15min":1.29},"last":1.214},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0960000000000001,"5min":1.637,"15min":1.637},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0289999999999999,"15min":1.0529999999999999},"min":{"1min":0.68300000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.131,"5min":2.601,"15min":2.601},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0980000000000001,"15min":1.127},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1559999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.1140000000000001}]},{"osd":5,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2609999999999999,"5min":1.792,"15min":1.792},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0269999999999999,"15min":1.036},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.357,"5min":1.575,"15min":1.575},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0980000000000001,"15min":1.1060000000000001},"min":{"1min":0.75900000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.335,"5min":1.831,"15min":1.831},"last":1.081},{"interface":"front","average":{"1min":1.077,"5min":1.101,"15min":1.105},"min":{"1min":0.89200000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3340000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.024}]},{"osd":7,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3009999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.1830000000000001},"min":{"1min":0.95699999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.429,"5min":1.944,"15min":1.944},"last":1.1439999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":647168,"data_stored":281292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.90400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86699999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88100000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90700000000000003,"15min":0.90600000000000003},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0269999999999999,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2230000000000001,"5min":1.282,"15min":1.282},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0389999999999999,"15min":1.0429999999999999},"min":{"1min":0.74399999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.302,"5min":1.484,"15min":1.484},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0209999999999999},"min":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0609999999999999,"15min":1.0640000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.996,"5min":1.0720000000000001,"15min":1.085},"min":{"1min":0.71599999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3580000000000001,"5min":1.673,"15min":1.673},"last":0.96499999999999997}]},{"osd":7,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0740000000000001,"15min":1.0820000000000001},"min":{"1min":0.81699999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3620000000000001,"5min":1.409,"15min":1.409},"last":1.05}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":870152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.67100000000000004,"15min":0.68799999999999994},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.749,"5min":1.617,"15min":1.617},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81499999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93600000000000005,"5min":1.446,"15min":1.446},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85599999999999998,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81499999999999995}]},{"osd":3,"last update":"Wed May 8 22:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.80100000000000005,"15min":0.81799999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90100000000000002,"5min":1.714,"15min":1.714},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.327,"15min":1.327},"last":0.68000000000000005}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.70599999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2729999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.996,"15min":1.01},"min":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.149,"5min":1.669,"15min":1.669},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0600000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1,"15min":1.006},"min":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3580000000000001,"5min":1.474,"15min":1.474},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.96099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70299999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0509999999999999,"15min":1.0580000000000001},"min":{"1min":0.77000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3149999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.085,"15min":1.0960000000000001},"min":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.23,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.74299999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3260000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.03}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:31.363 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:35:31.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 Processor -- start 2024-05-08T22:35:31.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 -- start start 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efbf8161790 0x7efbf8164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf8165d50 con 0x7efbf815ef80 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf806ad70 con 0x7efbf806de70 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfd4c6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf806e4b0 con 0x7efbf8161790 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efbf815ef80 0x7efbf806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57918/0 (socket says 172.21.15.179:57918) 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf7fff700 1 -- 172.21.15.179:0/3781836930 learned_addr learned my addr 172.21.15.179:0/3781836930 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfccc5700 1 --2- 172.21.15.179:0/3781836930 >> v2:172.21.15.195:3300/0 conn(0x7efbf8161790 0x7efbf8164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf6ffd700 1 -- 172.21.15.179:0/3781836930 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312322767 0 0) 0x7efbf8165d50 con 0x7efbf815ef80 2024-05-08T22:35:31.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf6ffd700 1 -- 172.21.15.179:0/3781836930 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbd80035d0 con 0x7efbf815ef80 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfccc5700 1 -- 172.21.15.179:0/3781836930 >> v1:172.21.15.195:6789/0 conn(0x7efbf806de70 legacy=0x7efbf806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfccc5700 1 -- 172.21.15.179:0/3781836930 >> v1:172.21.15.179:6789/0 conn(0x7efbf815ef80 legacy=0x7efbf806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfccc5700 1 -- 172.21.15.179:0/3781836930 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf81654b0 con 0x7efbf8161790 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbfccc5700 1 --2- 172.21.15.179:0/3781836930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf8161790 0x7efbf8164db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009850 tx=0x7efbf0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd8fc8a35f44e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf6ffd700 1 -- 172.21.15.179:0/3781836930 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0011830 con 0x7efbf8161790 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf6ffd700 1 -- 172.21.15.179:0/3781836930 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0011990 con 0x7efbf8161790 2024-05-08T22:35:31.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.476+0000 7efbf6ffd700 1 -- 172.21.15.179:0/3781836930 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0011d00 con 0x7efbf8161790 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3781836930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf8161790 msgr2=0x7efbf8164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3781836930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf8161790 0x7efbf8164db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009850 tx=0x7efbf0009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3781836930 shutdown_connections 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3781836930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf8161790 0x7efbf8164db0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3781836930 >> 172.21.15.179:0/3781836930 conn(0x7efbf80ce9d0 msgr2=0x7efbf80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3781836930 shutdown_connections 2024-05-08T22:35:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3781836930 wait complete. 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 Processor -- start 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- start start 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 0x7efbf8143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbf815ef80 0x7efbf8147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8158660 con 0x7efbf8161790 2024-05-08T22:35:31.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8158940 con 0x7efbf806de70 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8158c20 con 0x7efbf815ef80 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32878/0 (socket says 172.21.15.179:32878) 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 -- 172.21.15.179:0/3634590451 learned_addr learned my addr 172.21.15.179:0/3634590451 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf77fe700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbf815ef80 0x7efbf8147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf7fff700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 0x7efbf8143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 -- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbf815ef80 msgr2=0x7efbf8147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbf815ef80 0x7efbf8147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 -- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 msgr2=0x7efbf8143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 0x7efbf8143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 -- 172.21.15.179:0/3634590451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00094b0 con 0x7efbf8161790 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf7fff700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 0x7efbf8143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:31.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfccc5700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00036c0 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0003840 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8158f00 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0003bf0 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.480+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8143fe0 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf8159420 con 0x7efbf8161790 2024-05-08T22:35:31.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efbf0003d50 con 0x7efbf8161790 2024-05-08T22:35:31.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbf4ff9700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 0x7efbdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efbf0085680 con 0x7efbf8161790 2024-05-08T22:35:31.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbf7fff700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 0x7efbdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.484+0000 7efbf7fff700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 0x7efbdc068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efbe0001560 tx=0x7efbe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.492+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efbf8159420 con 0x7efbf8161790 2024-05-08T22:35:31.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.664+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efbf80cb2e0 con 0x7efbdc066730 2024-05-08T22:35:31.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.684+0000 7efbf4ff9700 1 -- 172.21.15.179:0/3634590451 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287862 (secure 0 0 0) 0x7efbf80cb2e0 con 0x7efbdc066730 2024-05-08T22:35:31.686 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.684+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 msgr2=0x7efbdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.684+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 0x7efbdc068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efbe0001560 tx=0x7efbe0009040 comp rx=0 tx=0).stop 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 msgr2=0x7efbf8159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004770 comp rx=0 tx=0).stop 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 shutdown_connections 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efbdc066730 0x7efbdc068bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efbf8161790 0x7efbf8159700 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efbf815ef80 0x7efbf8147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 --2- 172.21.15.179:0/3634590451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efbf806de70 0x7efbf8143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 >> 172.21.15.179:0/3634590451 conn(0x7efbf80ce9d0 msgr2=0x7efbf80cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 shutdown_connections 2024-05-08T22:35:31.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.688+0000 7efbfd4c6700 1 -- 172.21.15.179:0/3634590451 wait complete. 2024-05-08T22:35:31.692 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:35:31.704 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-05-08T22:35:31.425489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238788,"kb_used_data":6180,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735932,"statfs":{"total":773094113280,"available":772849594368,"internally_reserved":0,"allocated":6328320,"data_stored":3422227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.511984"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960617+0000","last_change":"2024-05-08T22:35:27.329461+0000","last_active":"2024-05-08T22:35:28.960617+0000","last_peered":"2024-05-08T22:35:28.960617+0000","last_clean":"2024-05-08T22:35:28.960617+0000","last_became_active":"2024-05-08T22:35:27.328930+0000","last_became_peered":"2024-05-08T22:35:27.328930+0000","last_unstale":"2024-05-08T22:35:28.960617+0000","last_undegraded":"2024-05-08T22:35:28.960617+0000","last_fullsized":"2024-05-08T22:35:28.960617+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:34.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309175+0000","last_change":"2024-05-08T22:35:27.325823+0000","last_active":"2024-05-08T22:35:28.309175+0000","last_peered":"2024-05-08T22:35:28.309175+0000","last_clean":"2024-05-08T22:35:28.309175+0000","last_became_active":"2024-05-08T22:35:27.325376+0000","last_became_peered":"2024-05-08T22:35:27.325376+0000","last_unstale":"2024-05-08T22:35:28.309175+0000","last_undegraded":"2024-05-08T22:35:28.309175+0000","last_fullsized":"2024-05-08T22:35:28.309175+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:48.752736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389739+0000","last_change":"2024-05-08T22:35:27.322868+0000","last_active":"2024-05-08T22:35:28.389739+0000","last_peered":"2024-05-08T22:35:28.389739+0000","last_clean":"2024-05-08T22:35:28.389739+0000","last_became_active":"2024-05-08T22:35:27.322593+0000","last_became_peered":"2024-05-08T22:35:27.322593+0000","last_unstale":"2024-05-08T22:35:28.389739+0000","last_undegraded":"2024-05-08T22:35:28.389739+0000","last_fullsized":"2024-05-08T22:35:28.389739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:42.412562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308680+0000","last_change":"2024-05-08T22:35:27.314234+0000","last_active":"2024-05-08T22:35:28.308680+0000","last_peered":"2024-05-08T22:35:28.308680+0000","last_clean":"2024-05-08T22:35:28.308680+0000","last_became_active":"2024-05-08T22:35:27.313955+0000","last_became_peered":"2024-05-08T22:35:27.313955+0000","last_unstale":"2024-05-08T22:35:28.308680+0000","last_undegraded":"2024-05-08T22:35:28.308680+0000","last_fullsized":"2024-05-08T22:35:28.308680+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:49.793646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.314048+0000","last_change":"2024-05-08T22:35:27.311599+0000","last_active":"2024-05-08T22:35:27.314048+0000","last_peered":"2024-05-08T22:35:27.314048+0000","last_clean":"2024-05-08T22:35:27.314048+0000","last_became_active":"2024-05-08T22:35:27.310937+0000","last_became_peered":"2024-05-08T22:35:27.310937+0000","last_unstale":"2024-05-08T22:35:27.314048+0000","last_undegraded":"2024-05-08T22:35:27.314048+0000","last_fullsized":"2024-05-08T22:35:27.314048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:15.146319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960131+0000","last_change":"2024-05-08T22:35:27.317419+0000","last_active":"2024-05-08T22:35:28.960131+0000","last_peered":"2024-05-08T22:35:28.960131+0000","last_clean":"2024-05-08T22:35:28.960131+0000","last_became_active":"2024-05-08T22:35:27.317176+0000","last_became_peered":"2024-05-08T22:35:27.317176+0000","last_unstale":"2024-05-08T22:35:28.960131+0000","last_undegraded":"2024-05-08T22:35:28.960131+0000","last_fullsized":"2024-05-08T22:35:28.960131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:23.660887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.388984+0000","last_change":"2024-05-08T22:35:27.320616+0000","last_active":"2024-05-08T22:35:28.388984+0000","last_peered":"2024-05-08T22:35:28.388984+0000","last_clean":"2024-05-08T22:35:28.388984+0000","last_became_active":"2024-05-08T22:35:27.320099+0000","last_became_peered":"2024-05-08T22:35:27.320099+0000","last_unstale":"2024-05-08T22:35:28.388984+0000","last_undegraded":"2024-05-08T22:35:28.388984+0000","last_fullsized":"2024-05-08T22:35:28.388984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:31.157377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330605+0000","last_change":"2024-05-08T22:35:27.326178+0000","last_active":"2024-05-08T22:35:27.330605+0000","last_peered":"2024-05-08T22:35:27.330605+0000","last_clean":"2024-05-08T22:35:27.330605+0000","last_became_active":"2024-05-08T22:35:27.325852+0000","last_became_peered":"2024-05-08T22:35:27.325852+0000","last_unstale":"2024-05-08T22:35:27.330605+0000","last_undegraded":"2024-05-08T22:35:27.330605+0000","last_fullsized":"2024-05-08T22:35:27.330605+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:20.354573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330671+0000","last_change":"2024-05-08T22:35:27.326215+0000","last_active":"2024-05-08T22:35:27.330671+0000","last_peered":"2024-05-08T22:35:27.330671+0000","last_clean":"2024-05-08T22:35:27.330671+0000","last_became_active":"2024-05-08T22:35:27.325877+0000","last_became_peered":"2024-05-08T22:35:27.325877+0000","last_unstale":"2024-05-08T22:35:27.330671+0000","last_undegraded":"2024-05-08T22:35:27.330671+0000","last_fullsized":"2024-05-08T22:35:27.330671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:05.047629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960410+0000","last_change":"2024-05-08T22:35:27.315536+0000","last_active":"2024-05-08T22:35:28.960410+0000","last_peered":"2024-05-08T22:35:28.960410+0000","last_clean":"2024-05-08T22:35:28.960410+0000","last_became_active":"2024-05-08T22:35:27.315308+0000","last_became_peered":"2024-05-08T22:35:27.315308+0000","last_unstale":"2024-05-08T22:35:28.960410+0000","last_undegraded":"2024-05-08T22:35:28.960410+0000","last_fullsized":"2024-05-08T22:35:28.960410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:09.216183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.319171+0000","last_change":"2024-05-08T22:35:27.313253+0000","last_active":"2024-05-08T22:35:27.319171+0000","last_peered":"2024-05-08T22:35:27.319171+0000","last_clean":"2024-05-08T22:35:27.319171+0000","last_became_active":"2024-05-08T22:35:27.312920+0000","last_became_peered":"2024-05-08T22:35:27.312920+0000","last_unstale":"2024-05-08T22:35:27.319171+0000","last_undegraded":"2024-05-08T22:35:27.319171+0000","last_fullsized":"2024-05-08T22:35:27.319171+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.285997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.315596+0000","last_change":"2024-05-08T22:35:27.314567+0000","last_active":"2024-05-08T22:35:27.315596+0000","last_peered":"2024-05-08T22:35:27.315596+0000","last_clean":"2024-05-08T22:35:27.315596+0000","last_became_active":"2024-05-08T22:35:27.314292+0000","last_became_peered":"2024-05-08T22:35:27.314292+0000","last_unstale":"2024-05-08T22:35:27.315596+0000","last_undegraded":"2024-05-08T22:35:27.315596+0000","last_fullsized":"2024-05-08T22:35:27.315596+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:32.851617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960051+0000","last_change":"2024-05-08T22:35:27.316303+0000","last_active":"2024-05-08T22:35:28.960051+0000","last_peered":"2024-05-08T22:35:28.960051+0000","last_clean":"2024-05-08T22:35:28.960051+0000","last_became_active":"2024-05-08T22:35:27.315993+0000","last_became_peered":"2024-05-08T22:35:27.315993+0000","last_unstale":"2024-05-08T22:35:28.960051+0000","last_undegraded":"2024-05-08T22:35:28.960051+0000","last_fullsized":"2024-05-08T22:35:28.960051+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:50.597540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389282+0000","last_change":"2024-05-08T22:35:27.317585+0000","last_active":"2024-05-08T22:35:28.389282+0000","last_peered":"2024-05-08T22:35:28.389282+0000","last_clean":"2024-05-08T22:35:28.389282+0000","last_became_active":"2024-05-08T22:35:27.316985+0000","last_became_peered":"2024-05-08T22:35:27.316985+0000","last_unstale":"2024-05-08T22:35:28.389282+0000","last_undegraded":"2024-05-08T22:35:28.389282+0000","last_fullsized":"2024-05-08T22:35:28.389282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:20.226511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308397+0000","last_change":"2024-05-08T22:35:27.310416+0000","last_active":"2024-05-08T22:35:28.308397+0000","last_peered":"2024-05-08T22:35:28.308397+0000","last_clean":"2024-05-08T22:35:28.308397+0000","last_became_active":"2024-05-08T22:35:27.310116+0000","last_became_peered":"2024-05-08T22:35:27.310116+0000","last_unstale":"2024-05-08T22:35:28.308397+0000","last_undegraded":"2024-05-08T22:35:28.308397+0000","last_fullsized":"2024-05-08T22:35:28.308397+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:05.411796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.959777+0000","last_change":"2024-05-08T22:35:27.314616+0000","last_active":"2024-05-08T22:35:28.959777+0000","last_peered":"2024-05-08T22:35:28.959777+0000","last_clean":"2024-05-08T22:35:28.959777+0000","last_became_active":"2024-05-08T22:35:27.314354+0000","last_became_peered":"2024-05-08T22:35:27.314354+0000","last_unstale":"2024-05-08T22:35:28.959777+0000","last_undegraded":"2024-05-08T22:35:28.959777+0000","last_fullsized":"2024-05-08T22:35:28.959777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:32.494317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390218+0000","last_change":"2024-05-08T22:35:27.328166+0000","last_active":"2024-05-08T22:35:28.390218+0000","last_peered":"2024-05-08T22:35:28.390218+0000","last_clean":"2024-05-08T22:35:28.390218+0000","last_became_active":"2024-05-08T22:35:27.327214+0000","last_became_peered":"2024-05-08T22:35:27.327214+0000","last_unstale":"2024-05-08T22:35:28.390218+0000","last_undegraded":"2024-05-08T22:35:28.390218+0000","last_fullsized":"2024-05-08T22:35:28.390218+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:15.146319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309380+0000","last_change":"2024-05-08T22:35:27.328835+0000","last_active":"2024-05-08T22:35:28.309380+0000","last_peered":"2024-05-08T22:35:28.309380+0000","last_clean":"2024-05-08T22:35:28.309380+0000","last_became_active":"2024-05-08T22:35:27.327786+0000","last_became_peered":"2024-05-08T22:35:27.327786+0000","last_unstale":"2024-05-08T22:35:28.309380+0000","last_undegraded":"2024-05-08T22:35:28.309380+0000","last_fullsized":"2024-05-08T22:35:28.309380+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:39.573030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389830+0000","last_change":"2024-05-08T22:35:27.323674+0000","last_active":"2024-05-08T22:35:28.389830+0000","last_peered":"2024-05-08T22:35:28.389830+0000","last_clean":"2024-05-08T22:35:28.389830+0000","last_became_active":"2024-05-08T22:35:27.323407+0000","last_became_peered":"2024-05-08T22:35:27.323407+0000","last_unstale":"2024-05-08T22:35:28.389830+0000","last_undegraded":"2024-05-08T22:35:28.389830+0000","last_fullsized":"2024-05-08T22:35:28.389830+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:58:54.724332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310376+0000","last_change":"2024-05-08T22:35:27.316013+0000","last_active":"2024-05-08T22:35:28.310376+0000","last_peered":"2024-05-08T22:35:28.310376+0000","last_clean":"2024-05-08T22:35:28.310376+0000","last_became_active":"2024-05-08T22:35:27.315670+0000","last_became_peered":"2024-05-08T22:35:27.315670+0000","last_unstale":"2024-05-08T22:35:28.310376+0000","last_undegraded":"2024-05-08T22:35:28.310376+0000","last_fullsized":"2024-05-08T22:35:28.310376+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:35.891712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310505+0000","last_change":"2024-05-08T22:35:27.312588+0000","last_active":"2024-05-08T22:35:28.310505+0000","last_peered":"2024-05-08T22:35:28.310505+0000","last_clean":"2024-05-08T22:35:28.310505+0000","last_became_active":"2024-05-08T22:35:27.312321+0000","last_became_peered":"2024-05-08T22:35:27.312321+0000","last_unstale":"2024-05-08T22:35:28.310505+0000","last_undegraded":"2024-05-08T22:35:28.310505+0000","last_fullsized":"2024-05-08T22:35:28.310505+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.405123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330644+0000","last_change":"2024-05-08T22:35:27.326183+0000","last_active":"2024-05-08T22:35:27.330644+0000","last_peered":"2024-05-08T22:35:27.330644+0000","last_clean":"2024-05-08T22:35:27.330644+0000","last_became_active":"2024-05-08T22:35:27.325813+0000","last_became_peered":"2024-05-08T22:35:27.325813+0000","last_unstale":"2024-05-08T22:35:27.330644+0000","last_undegraded":"2024-05-08T22:35:27.330644+0000","last_fullsized":"2024-05-08T22:35:27.330644+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:15.097766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.322019+0000","last_change":"2024-05-08T22:35:27.314922+0000","last_active":"2024-05-08T22:35:27.322019+0000","last_peered":"2024-05-08T22:35:27.322019+0000","last_clean":"2024-05-08T22:35:27.322019+0000","last_became_active":"2024-05-08T22:35:27.314611+0000","last_became_peered":"2024-05-08T22:35:27.314611+0000","last_unstale":"2024-05-08T22:35:27.322019+0000","last_undegraded":"2024-05-08T22:35:27.322019+0000","last_fullsized":"2024-05-08T22:35:27.322019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:25.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.388777+0000","last_change":"2024-05-08T22:35:27.320409+0000","last_active":"2024-05-08T22:35:28.388777+0000","last_peered":"2024-05-08T22:35:28.388777+0000","last_clean":"2024-05-08T22:35:28.388777+0000","last_became_active":"2024-05-08T22:35:27.320162+0000","last_became_peered":"2024-05-08T22:35:27.320162+0000","last_unstale":"2024-05-08T22:35:28.388777+0000","last_undegraded":"2024-05-08T22:35:28.388777+0000","last_fullsized":"2024-05-08T22:35:28.388777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:06.580248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390304+0000","last_change":"2024-05-08T22:35:27.322796+0000","last_active":"2024-05-08T22:35:28.390304+0000","last_peered":"2024-05-08T22:35:28.390304+0000","last_clean":"2024-05-08T22:35:28.390304+0000","last_became_active":"2024-05-08T22:35:27.316878+0000","last_became_peered":"2024-05-08T22:35:27.316878+0000","last_unstale":"2024-05-08T22:35:28.390304+0000","last_undegraded":"2024-05-08T22:35:28.390304+0000","last_fullsized":"2024-05-08T22:35:28.390304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:23.999057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.323931+0000","last_change":"2024-05-08T22:35:27.319021+0000","last_active":"2024-05-08T22:35:27.323931+0000","last_peered":"2024-05-08T22:35:27.323931+0000","last_clean":"2024-05-08T22:35:27.323931+0000","last_became_active":"2024-05-08T22:35:27.316818+0000","last_became_peered":"2024-05-08T22:35:27.316818+0000","last_unstale":"2024-05-08T22:35:27.323931+0000","last_undegraded":"2024-05-08T22:35:27.323931+0000","last_fullsized":"2024-05-08T22:35:27.323931+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:55.602510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960443+0000","last_change":"2024-05-08T22:35:27.324505+0000","last_active":"2024-05-08T22:35:28.960443+0000","last_peered":"2024-05-08T22:35:28.960443+0000","last_clean":"2024-05-08T22:35:28.960443+0000","last_became_active":"2024-05-08T22:35:27.324218+0000","last_became_peered":"2024-05-08T22:35:27.324218+0000","last_unstale":"2024-05-08T22:35:28.960443+0000","last_undegraded":"2024-05-08T22:35:28.960443+0000","last_fullsized":"2024-05-08T22:35:28.960443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:51.553137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310463+0000","last_change":"2024-05-08T22:35:27.311737+0000","last_active":"2024-05-08T22:35:28.310463+0000","last_peered":"2024-05-08T22:35:28.310463+0000","last_clean":"2024-05-08T22:35:28.310463+0000","last_became_active":"2024-05-08T22:35:27.311480+0000","last_became_peered":"2024-05-08T22:35:27.311480+0000","last_unstale":"2024-05-08T22:35:28.310463+0000","last_undegraded":"2024-05-08T22:35:28.310463+0000","last_fullsized":"2024-05-08T22:35:28.310463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:32.494317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389252+0000","last_change":"2024-05-08T22:35:27.317925+0000","last_active":"2024-05-08T22:35:28.389252+0000","last_peered":"2024-05-08T22:35:28.389252+0000","last_clean":"2024-05-08T22:35:28.389252+0000","last_became_active":"2024-05-08T22:35:27.317495+0000","last_became_peered":"2024-05-08T22:35:27.317495+0000","last_unstale":"2024-05-08T22:35:28.389252+0000","last_undegraded":"2024-05-08T22:35:28.389252+0000","last_fullsized":"2024-05-08T22:35:28.389252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:05.060058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308685+0000","last_change":"2024-05-08T22:35:27.317354+0000","last_active":"2024-05-08T22:35:28.308685+0000","last_peered":"2024-05-08T22:35:28.308685+0000","last_clean":"2024-05-08T22:35:28.308685+0000","last_became_active":"2024-05-08T22:35:27.317065+0000","last_became_peered":"2024-05-08T22:35:27.317065+0000","last_unstale":"2024-05-08T22:35:28.308685+0000","last_undegraded":"2024-05-08T22:35:28.308685+0000","last_fullsized":"2024-05-08T22:35:28.308685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:36.025168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389148+0000","last_change":"2024-05-08T22:35:27.320949+0000","last_active":"2024-05-08T22:35:28.389148+0000","last_peered":"2024-05-08T22:35:28.389148+0000","last_clean":"2024-05-08T22:35:28.389148+0000","last_became_active":"2024-05-08T22:35:27.320717+0000","last_became_peered":"2024-05-08T22:35:27.320717+0000","last_unstale":"2024-05-08T22:35:28.389148+0000","last_undegraded":"2024-05-08T22:35:28.389148+0000","last_fullsized":"2024-05-08T22:35:28.389148+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:04.120516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330922+0000","last_change":"2024-05-08T22:35:27.330044+0000","last_active":"2024-05-08T22:35:27.330922+0000","last_peered":"2024-05-08T22:35:27.330922+0000","last_clean":"2024-05-08T22:35:27.330922+0000","last_became_active":"2024-05-08T22:35:27.329721+0000","last_became_peered":"2024-05-08T22:35:27.329721+0000","last_unstale":"2024-05-08T22:35:27.330922+0000","last_undegraded":"2024-05-08T22:35:27.330922+0000","last_fullsized":"2024-05-08T22:35:27.330922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:18.405587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308145+0000","last_change":"2024-05-08T22:35:27.326013+0000","last_active":"2024-05-08T22:35:28.308145+0000","last_peered":"2024-05-08T22:35:28.308145+0000","last_clean":"2024-05-08T22:35:28.308145+0000","last_became_active":"2024-05-08T22:35:27.325678+0000","last_became_peered":"2024-05-08T22:35:27.325678+0000","last_unstale":"2024-05-08T22:35:28.308145+0000","last_undegraded":"2024-05-08T22:35:28.308145+0000","last_fullsized":"2024-05-08T22:35:28.308145+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:18.405587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310763+0000","last_change":"2024-05-08T22:35:26.937256+0000","last_active":"2024-05-08T22:35:28.310763+0000","last_peered":"2024-05-08T22:35:28.310763+0000","last_clean":"2024-05-08T22:35:28.310763+0000","last_became_active":"2024-05-08T22:35:26.936950+0000","last_became_peered":"2024-05-08T22:35:26.936950+0000","last_unstale":"2024-05-08T22:35:28.310763+0000","last_undegraded":"2024-05-08T22:35:28.310763+0000","last_fullsized":"2024-05-08T22:35:28.310763+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:30.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.326166+0000","last_change":"2024-05-08T22:35:27.324975+0000","last_active":"2024-05-08T22:35:27.326166+0000","last_peered":"2024-05-08T22:35:27.326166+0000","last_clean":"2024-05-08T22:35:27.326166+0000","last_became_active":"2024-05-08T22:35:27.324646+0000","last_became_peered":"2024-05-08T22:35:27.324646+0000","last_unstale":"2024-05-08T22:35:27.326166+0000","last_undegraded":"2024-05-08T22:35:27.326166+0000","last_fullsized":"2024-05-08T22:35:27.326166+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:53:17.288386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960497+0000","last_change":"2024-05-08T22:35:26.951813+0000","last_active":"2024-05-08T22:35:28.960497+0000","last_peered":"2024-05-08T22:35:28.960497+0000","last_clean":"2024-05-08T22:35:28.960497+0000","last_became_active":"2024-05-08T22:35:26.951453+0000","last_became_peered":"2024-05-08T22:35:26.951453+0000","last_unstale":"2024-05-08T22:35:28.960497+0000","last_undegraded":"2024-05-08T22:35:28.960497+0000","last_fullsized":"2024-05-08T22:35:28.960497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:17.519768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310443+0000","last_change":"2024-05-08T22:35:27.310545+0000","last_active":"2024-05-08T22:35:28.310443+0000","last_peered":"2024-05-08T22:35:28.310443+0000","last_clean":"2024-05-08T22:35:28.310443+0000","last_became_active":"2024-05-08T22:35:27.310281+0000","last_became_peered":"2024-05-08T22:35:27.310281+0000","last_unstale":"2024-05-08T22:35:28.310443+0000","last_undegraded":"2024-05-08T22:35:28.310443+0000","last_fullsized":"2024-05-08T22:35:28.310443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:34.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960658+0000","last_change":"2024-05-08T22:35:26.940293+0000","last_active":"2024-05-08T22:35:28.960658+0000","last_peered":"2024-05-08T22:35:28.960658+0000","last_clean":"2024-05-08T22:35:28.960658+0000","last_became_active":"2024-05-08T22:35:26.939934+0000","last_became_peered":"2024-05-08T22:35:26.939934+0000","last_unstale":"2024-05-08T22:35:28.960658+0000","last_undegraded":"2024-05-08T22:35:28.960658+0000","last_fullsized":"2024-05-08T22:35:28.960658+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:30.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.323819+0000","last_change":"2024-05-08T22:35:27.317380+0000","last_active":"2024-05-08T22:35:27.323819+0000","last_peered":"2024-05-08T22:35:27.323819+0000","last_clean":"2024-05-08T22:35:27.323819+0000","last_became_active":"2024-05-08T22:35:27.317103+0000","last_became_peered":"2024-05-08T22:35:27.317103+0000","last_unstale":"2024-05-08T22:35:27.323819+0000","last_undegraded":"2024-05-08T22:35:27.323819+0000","last_fullsized":"2024-05-08T22:35:27.323819+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:56.069079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.335221+0000","last_change":"2024-05-08T22:35:26.950336+0000","last_active":"2024-05-08T22:35:27.335221+0000","last_peered":"2024-05-08T22:35:27.335221+0000","last_clean":"2024-05-08T22:35:27.335221+0000","last_became_active":"2024-05-08T22:35:26.949995+0000","last_became_peered":"2024-05-08T22:35:26.949995+0000","last_unstale":"2024-05-08T22:35:27.335221+0000","last_undegraded":"2024-05-08T22:35:27.335221+0000","last_fullsized":"2024-05-08T22:35:27.335221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:20.556169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.318204+0000","last_change":"2024-05-08T22:35:27.317311+0000","last_active":"2024-05-08T22:35:27.318204+0000","last_peered":"2024-05-08T22:35:27.318204+0000","last_clean":"2024-05-08T22:35:27.318204+0000","last_became_active":"2024-05-08T22:35:27.317045+0000","last_became_peered":"2024-05-08T22:35:27.317045+0000","last_unstale":"2024-05-08T22:35:27.318204+0000","last_undegraded":"2024-05-08T22:35:27.318204+0000","last_fullsized":"2024-05-08T22:35:27.318204+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:06.683189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389333+0000","last_change":"2024-05-08T22:35:26.941527+0000","last_active":"2024-05-08T22:35:28.389333+0000","last_peered":"2024-05-08T22:35:28.389333+0000","last_clean":"2024-05-08T22:35:28.389333+0000","last_became_active":"2024-05-08T22:35:26.941048+0000","last_became_peered":"2024-05-08T22:35:26.941048+0000","last_unstale":"2024-05-08T22:35:28.389333+0000","last_undegraded":"2024-05-08T22:35:28.389333+0000","last_fullsized":"2024-05-08T22:35:28.389333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:23.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308875+0000","last_change":"2024-05-08T22:35:27.317366+0000","last_active":"2024-05-08T22:35:28.308875+0000","last_peered":"2024-05-08T22:35:28.308875+0000","last_clean":"2024-05-08T22:35:28.308875+0000","last_became_active":"2024-05-08T22:35:27.317043+0000","last_became_peered":"2024-05-08T22:35:27.317043+0000","last_unstale":"2024-05-08T22:35:28.308875+0000","last_undegraded":"2024-05-08T22:35:28.308875+0000","last_fullsized":"2024-05-08T22:35:28.308875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:03.886275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.334604+0000","last_change":"2024-05-08T22:35:26.934242+0000","last_active":"2024-05-08T22:35:27.334604+0000","last_peered":"2024-05-08T22:35:27.334604+0000","last_clean":"2024-05-08T22:35:27.334604+0000","last_became_active":"2024-05-08T22:35:26.932077+0000","last_became_peered":"2024-05-08T22:35:26.932077+0000","last_unstale":"2024-05-08T22:35:27.334604+0000","last_undegraded":"2024-05-08T22:35:27.334604+0000","last_fullsized":"2024-05-08T22:35:27.334604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:52.669092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.323913+0000","last_change":"2024-05-08T22:35:27.318320+0000","last_active":"2024-05-08T22:35:27.323913+0000","last_peered":"2024-05-08T22:35:27.323913+0000","last_clean":"2024-05-08T22:35:27.323913+0000","last_became_active":"2024-05-08T22:35:27.317135+0000","last_became_peered":"2024-05-08T22:35:27.317135+0000","last_unstale":"2024-05-08T22:35:27.323913+0000","last_undegraded":"2024-05-08T22:35:27.323913+0000","last_fullsized":"2024-05-08T22:35:27.323913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:58.529327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310315+0000","last_change":"2024-05-08T22:35:26.933939+0000","last_active":"2024-05-08T22:35:28.310315+0000","last_peered":"2024-05-08T22:35:28.310315+0000","last_clean":"2024-05-08T22:35:28.310315+0000","last_became_active":"2024-05-08T22:35:26.933674+0000","last_became_peered":"2024-05-08T22:35:26.933674+0000","last_unstale":"2024-05-08T22:35:28.310315+0000","last_undegraded":"2024-05-08T22:35:28.310315+0000","last_fullsized":"2024-05-08T22:35:28.310315+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:17.519768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308861+0000","last_change":"2024-05-08T22:35:27.320148+0000","last_active":"2024-05-08T22:35:28.308861+0000","last_peered":"2024-05-08T22:35:28.308861+0000","last_clean":"2024-05-08T22:35:28.308861+0000","last_became_active":"2024-05-08T22:35:27.319860+0000","last_became_peered":"2024-05-08T22:35:27.319860+0000","last_unstale":"2024-05-08T22:35:28.308861+0000","last_undegraded":"2024-05-08T22:35:28.308861+0000","last_fullsized":"2024-05-08T22:35:28.308861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:50.819918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389017+0000","last_change":"2024-05-08T22:35:26.945863+0000","last_active":"2024-05-08T22:35:28.389017+0000","last_peered":"2024-05-08T22:35:28.389017+0000","last_clean":"2024-05-08T22:35:28.389017+0000","last_became_active":"2024-05-08T22:35:26.945607+0000","last_became_peered":"2024-05-08T22:35:26.945607+0000","last_unstale":"2024-05-08T22:35:28.389017+0000","last_undegraded":"2024-05-08T22:35:28.389017+0000","last_fullsized":"2024-05-08T22:35:28.389017+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:37.159065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.317691+0000","last_change":"2024-05-08T22:35:27.315255+0000","last_active":"2024-05-08T22:35:27.317691+0000","last_peered":"2024-05-08T22:35:27.317691+0000","last_clean":"2024-05-08T22:35:27.317691+0000","last_became_active":"2024-05-08T22:35:27.314765+0000","last_became_peered":"2024-05-08T22:35:27.314765+0000","last_unstale":"2024-05-08T22:35:27.317691+0000","last_undegraded":"2024-05-08T22:35:27.317691+0000","last_fullsized":"2024-05-08T22:35:27.317691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:05.899140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308996+0000","last_change":"2024-05-08T22:35:26.933923+0000","last_active":"2024-05-08T22:35:28.308996+0000","last_peered":"2024-05-08T22:35:28.308996+0000","last_clean":"2024-05-08T22:35:28.308996+0000","last_became_active":"2024-05-08T22:35:26.932465+0000","last_became_peered":"2024-05-08T22:35:26.932465+0000","last_unstale":"2024-05-08T22:35:28.308996+0000","last_undegraded":"2024-05-08T22:35:28.308996+0000","last_fullsized":"2024-05-08T22:35:28.308996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:19.028838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.326186+0000","last_change":"2024-05-08T22:35:27.325064+0000","last_active":"2024-05-08T22:35:27.326186+0000","last_peered":"2024-05-08T22:35:27.326186+0000","last_clean":"2024-05-08T22:35:27.326186+0000","last_became_active":"2024-05-08T22:35:27.324784+0000","last_became_peered":"2024-05-08T22:35:27.324784+0000","last_unstale":"2024-05-08T22:35:27.326186+0000","last_undegraded":"2024-05-08T22:35:27.326186+0000","last_fullsized":"2024-05-08T22:35:27.326186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:32.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.304147+0000","last_change":"2024-05-08T22:35:26.942188+0000","last_active":"2024-05-08T22:35:27.304147+0000","last_peered":"2024-05-08T22:35:27.304147+0000","last_clean":"2024-05-08T22:35:27.304147+0000","last_became_active":"2024-05-08T22:35:26.941953+0000","last_became_peered":"2024-05-08T22:35:26.941953+0000","last_unstale":"2024-05-08T22:35:27.304147+0000","last_undegraded":"2024-05-08T22:35:27.304147+0000","last_fullsized":"2024-05-08T22:35:27.304147+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:47.743474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389365+0000","last_change":"2024-05-08T22:35:27.317757+0000","last_active":"2024-05-08T22:35:28.389365+0000","last_peered":"2024-05-08T22:35:28.389365+0000","last_clean":"2024-05-08T22:35:28.389365+0000","last_became_active":"2024-05-08T22:35:27.317271+0000","last_became_peered":"2024-05-08T22:35:27.317271+0000","last_unstale":"2024-05-08T22:35:28.389365+0000","last_undegraded":"2024-05-08T22:35:28.389365+0000","last_fullsized":"2024-05-08T22:35:28.389365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:58.388506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309033+0000","last_change":"2024-05-08T22:35:26.936242+0000","last_active":"2024-05-08T22:35:28.309033+0000","last_peered":"2024-05-08T22:35:28.309033+0000","last_clean":"2024-05-08T22:35:28.309033+0000","last_became_active":"2024-05-08T22:35:26.935959+0000","last_became_peered":"2024-05-08T22:35:26.935959+0000","last_unstale":"2024-05-08T22:35:28.309033+0000","last_undegraded":"2024-05-08T22:35:28.309033+0000","last_fullsized":"2024-05-08T22:35:28.309033+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.731401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.314060+0000","last_change":"2024-05-08T22:35:27.311868+0000","last_active":"2024-05-08T22:35:27.314060+0000","last_peered":"2024-05-08T22:35:27.314060+0000","last_clean":"2024-05-08T22:35:27.314060+0000","last_became_active":"2024-05-08T22:35:27.311275+0000","last_became_peered":"2024-05-08T22:35:27.311275+0000","last_unstale":"2024-05-08T22:35:27.314060+0000","last_undegraded":"2024-05-08T22:35:27.314060+0000","last_fullsized":"2024-05-08T22:35:27.314060+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:04.120516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.339385+0000","last_change":"2024-05-08T22:35:26.951437+0000","last_active":"2024-05-08T22:35:27.339385+0000","last_peered":"2024-05-08T22:35:27.339385+0000","last_clean":"2024-05-08T22:35:27.339385+0000","last_became_active":"2024-05-08T22:35:26.951179+0000","last_became_peered":"2024-05-08T22:35:26.951179+0000","last_unstale":"2024-05-08T22:35:27.339385+0000","last_undegraded":"2024-05-08T22:35:27.339385+0000","last_fullsized":"2024-05-08T22:35:27.339385+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.206921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.327484+0000","last_change":"2024-05-08T22:35:27.325790+0000","last_active":"2024-05-08T22:35:27.327484+0000","last_peered":"2024-05-08T22:35:27.327484+0000","last_clean":"2024-05-08T22:35:27.327484+0000","last_became_active":"2024-05-08T22:35:27.325237+0000","last_became_peered":"2024-05-08T22:35:27.325237+0000","last_unstale":"2024-05-08T22:35:27.327484+0000","last_undegraded":"2024-05-08T22:35:27.327484+0000","last_fullsized":"2024-05-08T22:35:27.327484+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.678354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389461+0000","last_change":"2024-05-08T22:35:26.945770+0000","last_active":"2024-05-08T22:35:28.389461+0000","last_peered":"2024-05-08T22:35:28.389461+0000","last_clean":"2024-05-08T22:35:28.389461+0000","last_became_active":"2024-05-08T22:35:26.945490+0000","last_became_peered":"2024-05-08T22:35:26.945490+0000","last_unstale":"2024-05-08T22:35:28.389461+0000","last_undegraded":"2024-05-08T22:35:28.389461+0000","last_fullsized":"2024-05-08T22:35:28.389461+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:29.731401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389968+0000","last_change":"2024-05-08T22:35:27.320671+0000","last_active":"2024-05-08T22:35:28.389968+0000","last_peered":"2024-05-08T22:35:28.389968+0000","last_clean":"2024-05-08T22:35:28.389968+0000","last_became_active":"2024-05-08T22:35:27.318357+0000","last_became_peered":"2024-05-08T22:35:27.318357+0000","last_unstale":"2024-05-08T22:35:28.389968+0000","last_undegraded":"2024-05-08T22:35:28.389968+0000","last_fullsized":"2024-05-08T22:35:28.389968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:47.849291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310532+0000","last_change":"2024-05-08T22:35:26.942632+0000","last_active":"2024-05-08T22:35:28.310532+0000","last_peered":"2024-05-08T22:35:28.310532+0000","last_clean":"2024-05-08T22:35:28.310532+0000","last_became_active":"2024-05-08T22:35:26.942381+0000","last_became_peered":"2024-05-08T22:35:26.942381+0000","last_unstale":"2024-05-08T22:35:28.310532+0000","last_undegraded":"2024-05-08T22:35:28.310532+0000","last_fullsized":"2024-05-08T22:35:28.310532+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:17.212132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.321295+0000","last_change":"2024-05-08T22:35:27.316121+0000","last_active":"2024-05-08T22:35:27.321295+0000","last_peered":"2024-05-08T22:35:27.321295+0000","last_clean":"2024-05-08T22:35:27.321295+0000","last_became_active":"2024-05-08T22:35:27.315828+0000","last_became_peered":"2024-05-08T22:35:27.315828+0000","last_unstale":"2024-05-08T22:35:27.321295+0000","last_undegraded":"2024-05-08T22:35:27.321295+0000","last_fullsized":"2024-05-08T22:35:27.321295+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:12.624435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.308397+0000","last_change":"2024-05-08T22:35:26.935401+0000","last_active":"2024-05-08T22:35:28.308397+0000","last_peered":"2024-05-08T22:35:28.308397+0000","last_clean":"2024-05-08T22:35:28.308397+0000","last_became_active":"2024-05-08T22:35:26.935106+0000","last_became_peered":"2024-05-08T22:35:26.935106+0000","last_unstale":"2024-05-08T22:35:28.308397+0000","last_undegraded":"2024-05-08T22:35:28.308397+0000","last_fullsized":"2024-05-08T22:35:28.308397+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:20.556169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390194+0000","last_change":"2024-05-08T22:35:27.327396+0000","last_active":"2024-05-08T22:35:28.390194+0000","last_peered":"2024-05-08T22:35:28.390194+0000","last_clean":"2024-05-08T22:35:28.390194+0000","last_became_active":"2024-05-08T22:35:27.326988+0000","last_became_peered":"2024-05-08T22:35:27.326988+0000","last_unstale":"2024-05-08T22:35:28.390194+0000","last_undegraded":"2024-05-08T22:35:28.390194+0000","last_fullsized":"2024-05-08T22:35:28.390194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:10.388711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.339417+0000","last_change":"2024-05-08T22:35:26.947340+0000","last_active":"2024-05-08T22:35:27.339417+0000","last_peered":"2024-05-08T22:35:27.339417+0000","last_clean":"2024-05-08T22:35:27.339417+0000","last_became_active":"2024-05-08T22:35:26.942820+0000","last_became_peered":"2024-05-08T22:35:26.942820+0000","last_unstale":"2024-05-08T22:35:27.339417+0000","last_undegraded":"2024-05-08T22:35:27.339417+0000","last_fullsized":"2024-05-08T22:35:27.339417+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.811882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330704+0000","last_change":"2024-05-08T22:35:27.327028+0000","last_active":"2024-05-08T22:35:27.330704+0000","last_peered":"2024-05-08T22:35:27.330704+0000","last_clean":"2024-05-08T22:35:27.330704+0000","last_became_active":"2024-05-08T22:35:27.326731+0000","last_became_peered":"2024-05-08T22:35:27.326731+0000","last_unstale":"2024-05-08T22:35:27.330704+0000","last_undegraded":"2024-05-08T22:35:27.330704+0000","last_fullsized":"2024-05-08T22:35:27.330704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:57:53.387412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390334+0000","last_change":"2024-05-08T22:35:26.953911+0000","last_active":"2024-05-08T22:35:28.390334+0000","last_peered":"2024-05-08T22:35:28.390334+0000","last_clean":"2024-05-08T22:35:28.390334+0000","last_became_active":"2024-05-08T22:35:26.953515+0000","last_became_peered":"2024-05-08T22:35:26.953515+0000","last_unstale":"2024-05-08T22:35:28.390334+0000","last_undegraded":"2024-05-08T22:35:28.390334+0000","last_fullsized":"2024-05-08T22:35:28.390334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:51.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.959954+0000","last_change":"2024-05-08T22:35:27.323859+0000","last_active":"2024-05-08T22:35:28.959954+0000","last_peered":"2024-05-08T22:35:28.959954+0000","last_clean":"2024-05-08T22:35:28.959954+0000","last_became_active":"2024-05-08T22:35:27.323557+0000","last_became_peered":"2024-05-08T22:35:27.323557+0000","last_unstale":"2024-05-08T22:35:28.959954+0000","last_undegraded":"2024-05-08T22:35:28.959954+0000","last_fullsized":"2024-05-08T22:35:28.959954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:09.821143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.335280+0000","last_change":"2024-05-08T22:35:26.935048+0000","last_active":"2024-05-08T22:35:27.335280+0000","last_peered":"2024-05-08T22:35:27.335280+0000","last_clean":"2024-05-08T22:35:27.335280+0000","last_became_active":"2024-05-08T22:35:26.934674+0000","last_became_peered":"2024-05-08T22:35:26.934674+0000","last_unstale":"2024-05-08T22:35:27.335280+0000","last_undegraded":"2024-05-08T22:35:27.335280+0000","last_fullsized":"2024-05-08T22:35:27.335280+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:16.644630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.318243+0000","last_change":"2024-05-08T22:35:27.317319+0000","last_active":"2024-05-08T22:35:27.318243+0000","last_peered":"2024-05-08T22:35:27.318243+0000","last_clean":"2024-05-08T22:35:27.318243+0000","last_became_active":"2024-05-08T22:35:27.317033+0000","last_became_peered":"2024-05-08T22:35:27.317033+0000","last_unstale":"2024-05-08T22:35:27.318243+0000","last_undegraded":"2024-05-08T22:35:27.318243+0000","last_fullsized":"2024-05-08T22:35:27.318243+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:10.388711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310834+0000","last_change":"2024-05-08T22:35:26.939712+0000","last_active":"2024-05-08T22:35:28.310834+0000","last_peered":"2024-05-08T22:35:28.310834+0000","last_clean":"2024-05-08T22:35:28.310834+0000","last_became_active":"2024-05-08T22:35:26.939466+0000","last_became_peered":"2024-05-08T22:35:26.939466+0000","last_unstale":"2024-05-08T22:35:28.310834+0000","last_undegraded":"2024-05-08T22:35:28.310834+0000","last_fullsized":"2024-05-08T22:35:28.310834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:11.813254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960171+0000","last_change":"2024-05-08T22:35:27.325442+0000","last_active":"2024-05-08T22:35:28.960171+0000","last_peered":"2024-05-08T22:35:28.960171+0000","last_clean":"2024-05-08T22:35:28.960171+0000","last_became_active":"2024-05-08T22:35:27.325193+0000","last_became_peered":"2024-05-08T22:35:27.325193+0000","last_unstale":"2024-05-08T22:35:28.960171+0000","last_undegraded":"2024-05-08T22:35:28.960171+0000","last_fullsized":"2024-05-08T22:35:28.960171+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.405123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310588+0000","last_change":"2024-05-08T22:35:26.950270+0000","last_active":"2024-05-08T22:35:28.310588+0000","last_peered":"2024-05-08T22:35:28.310588+0000","last_clean":"2024-05-08T22:35:28.310588+0000","last_became_active":"2024-05-08T22:35:26.949989+0000","last_became_peered":"2024-05-08T22:35:26.949989+0000","last_unstale":"2024-05-08T22:35:28.310588+0000","last_undegraded":"2024-05-08T22:35:28.310588+0000","last_fullsized":"2024-05-08T22:35:28.310588+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:31:54.732370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.327502+0000","last_change":"2024-05-08T22:35:27.325954+0000","last_active":"2024-05-08T22:35:27.327502+0000","last_peered":"2024-05-08T22:35:27.327502+0000","last_clean":"2024-05-08T22:35:27.327502+0000","last_became_active":"2024-05-08T22:35:27.325547+0000","last_became_peered":"2024-05-08T22:35:27.325547+0000","last_unstale":"2024-05-08T22:35:27.327502+0000","last_undegraded":"2024-05-08T22:35:27.327502+0000","last_fullsized":"2024-05-08T22:35:27.327502+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:28.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960266+0000","last_change":"2024-05-08T22:35:26.939660+0000","last_active":"2024-05-08T22:35:28.960266+0000","last_peered":"2024-05-08T22:35:28.960266+0000","last_clean":"2024-05-08T22:35:28.960266+0000","last_became_active":"2024-05-08T22:35:26.937040+0000","last_became_peered":"2024-05-08T22:35:26.937040+0000","last_unstale":"2024-05-08T22:35:28.960266+0000","last_undegraded":"2024-05-08T22:35:28.960266+0000","last_fullsized":"2024-05-08T22:35:28.960266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:17.212132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310654+0000","last_change":"2024-05-08T22:35:27.313024+0000","last_active":"2024-05-08T22:35:28.310654+0000","last_peered":"2024-05-08T22:35:28.310654+0000","last_clean":"2024-05-08T22:35:28.310654+0000","last_became_active":"2024-05-08T22:35:27.312702+0000","last_became_peered":"2024-05-08T22:35:27.312702+0000","last_unstale":"2024-05-08T22:35:28.310654+0000","last_undegraded":"2024-05-08T22:35:28.310654+0000","last_fullsized":"2024-05-08T22:35:28.310654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:46.408378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.307864+0000","last_change":"2024-05-08T22:35:26.951319+0000","last_active":"2024-05-08T22:35:27.307864+0000","last_peered":"2024-05-08T22:35:27.307864+0000","last_clean":"2024-05-08T22:35:27.307864+0000","last_became_active":"2024-05-08T22:35:26.951062+0000","last_became_peered":"2024-05-08T22:35:26.951062+0000","last_unstale":"2024-05-08T22:35:27.307864+0000","last_undegraded":"2024-05-08T22:35:27.307864+0000","last_fullsized":"2024-05-08T22:35:27.307864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:33.179541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390107+0000","last_change":"2024-05-08T22:35:27.317319+0000","last_active":"2024-05-08T22:35:28.390107+0000","last_peered":"2024-05-08T22:35:28.390107+0000","last_clean":"2024-05-08T22:35:28.390107+0000","last_became_active":"2024-05-08T22:35:27.316970+0000","last_became_peered":"2024-05-08T22:35:27.316970+0000","last_unstale":"2024-05-08T22:35:28.390107+0000","last_undegraded":"2024-05-08T22:35:28.390107+0000","last_fullsized":"2024-05-08T22:35:28.390107+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:59.628309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389215+0000","last_change":"2024-05-08T22:35:26.944044+0000","last_active":"2024-05-08T22:35:28.389215+0000","last_peered":"2024-05-08T22:35:28.389215+0000","last_clean":"2024-05-08T22:35:28.389215+0000","last_became_active":"2024-05-08T22:35:26.943342+0000","last_became_peered":"2024-05-08T22:35:26.943342+0000","last_unstale":"2024-05-08T22:35:28.389215+0000","last_undegraded":"2024-05-08T22:35:28.389215+0000","last_fullsized":"2024-05-08T22:35:28.389215+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:20.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960535+0000","last_change":"2024-05-08T22:35:27.317567+0000","last_active":"2024-05-08T22:35:28.960535+0000","last_peered":"2024-05-08T22:35:28.960535+0000","last_clean":"2024-05-08T22:35:28.960535+0000","last_became_active":"2024-05-08T22:35:27.317358+0000","last_became_peered":"2024-05-08T22:35:27.317358+0000","last_unstale":"2024-05-08T22:35:28.960535+0000","last_undegraded":"2024-05-08T22:35:28.960535+0000","last_fullsized":"2024-05-08T22:35:28.960535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:34.188803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.303707+0000","last_change":"2024-05-08T22:35:26.937951+0000","last_active":"2024-05-08T22:35:27.303707+0000","last_peered":"2024-05-08T22:35:27.303707+0000","last_clean":"2024-05-08T22:35:27.303707+0000","last_became_active":"2024-05-08T22:35:26.937674+0000","last_became_peered":"2024-05-08T22:35:26.937674+0000","last_unstale":"2024-05-08T22:35:27.303707+0000","last_undegraded":"2024-05-08T22:35:27.303707+0000","last_fullsized":"2024-05-08T22:35:27.303707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:11.049425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330740+0000","last_change":"2024-05-08T22:35:27.329009+0000","last_active":"2024-05-08T22:35:27.330740+0000","last_peered":"2024-05-08T22:35:27.330740+0000","last_clean":"2024-05-08T22:35:27.330740+0000","last_became_active":"2024-05-08T22:35:27.328775+0000","last_became_peered":"2024-05-08T22:35:27.328775+0000","last_unstale":"2024-05-08T22:35:27.330740+0000","last_undegraded":"2024-05-08T22:35:27.330740+0000","last_fullsized":"2024-05-08T22:35:27.330740+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:48.752736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310771+0000","last_change":"2024-05-08T22:35:26.948548+0000","last_active":"2024-05-08T22:35:28.310771+0000","last_peered":"2024-05-08T22:35:28.310771+0000","last_clean":"2024-05-08T22:35:28.310771+0000","last_became_active":"2024-05-08T22:35:26.948278+0000","last_became_peered":"2024-05-08T22:35:26.948278+0000","last_unstale":"2024-05-08T22:35:28.310771+0000","last_undegraded":"2024-05-08T22:35:28.310771+0000","last_fullsized":"2024-05-08T22:35:28.310771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:18.517835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.960178+0000","last_change":"2024-05-08T22:35:27.317903+0000","last_active":"2024-05-08T22:35:28.960178+0000","last_peered":"2024-05-08T22:35:28.960178+0000","last_clean":"2024-05-08T22:35:28.960178+0000","last_became_active":"2024-05-08T22:35:27.317639+0000","last_became_peered":"2024-05-08T22:35:27.317639+0000","last_unstale":"2024-05-08T22:35:28.960178+0000","last_undegraded":"2024-05-08T22:35:28.960178+0000","last_fullsized":"2024-05-08T22:35:28.960178+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:35.891712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.309185+0000","last_change":"2024-05-08T22:35:26.936561+0000","last_active":"2024-05-08T22:35:28.309185+0000","last_peered":"2024-05-08T22:35:28.309185+0000","last_clean":"2024-05-08T22:35:28.309185+0000","last_became_active":"2024-05-08T22:35:26.936294+0000","last_became_peered":"2024-05-08T22:35:26.936294+0000","last_unstale":"2024-05-08T22:35:28.309185+0000","last_undegraded":"2024-05-08T22:35:28.309185+0000","last_fullsized":"2024-05-08T22:35:28.309185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:09.211946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.321239+0000","last_change":"2024-05-08T22:35:27.316025+0000","last_active":"2024-05-08T22:35:27.321239+0000","last_peered":"2024-05-08T22:35:27.321239+0000","last_clean":"2024-05-08T22:35:27.321239+0000","last_became_active":"2024-05-08T22:35:27.315590+0000","last_became_peered":"2024-05-08T22:35:27.315590+0000","last_unstale":"2024-05-08T22:35:27.321239+0000","last_undegraded":"2024-05-08T22:35:27.321239+0000","last_fullsized":"2024-05-08T22:35:27.321239+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:59.366779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389611+0000","last_change":"2024-05-08T22:35:26.933098+0000","last_active":"2024-05-08T22:35:28.389611+0000","last_peered":"2024-05-08T22:35:28.389611+0000","last_clean":"2024-05-08T22:35:28.389611+0000","last_became_active":"2024-05-08T22:35:26.932764+0000","last_became_peered":"2024-05-08T22:35:26.932764+0000","last_unstale":"2024-05-08T22:35:28.389611+0000","last_undegraded":"2024-05-08T22:35:28.389611+0000","last_fullsized":"2024-05-08T22:35:28.389611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:19.028838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389632+0000","last_change":"2024-05-08T22:35:27.317900+0000","last_active":"2024-05-08T22:35:28.389632+0000","last_peered":"2024-05-08T22:35:28.389632+0000","last_clean":"2024-05-08T22:35:28.389632+0000","last_became_active":"2024-05-08T22:35:27.317163+0000","last_became_peered":"2024-05-08T22:35:27.317163+0000","last_unstale":"2024-05-08T22:35:28.389632+0000","last_undegraded":"2024-05-08T22:35:28.389632+0000","last_fullsized":"2024-05-08T22:35:28.389632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:39.414972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.309043+0000","last_change":"2024-05-08T22:35:26.951250+0000","last_active":"2024-05-08T22:35:27.309043+0000","last_peered":"2024-05-08T22:35:27.309043+0000","last_clean":"2024-05-08T22:35:27.309043+0000","last_became_active":"2024-05-08T22:35:26.950864+0000","last_became_peered":"2024-05-08T22:35:26.950864+0000","last_unstale":"2024-05-08T22:35:27.309043+0000","last_undegraded":"2024-05-08T22:35:27.309043+0000","last_fullsized":"2024-05-08T22:35:27.309043+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:49.588278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.330015+0000","last_change":"2024-05-08T22:35:27.326411+0000","last_active":"2024-05-08T22:35:27.330015+0000","last_peered":"2024-05-08T22:35:27.330015+0000","last_clean":"2024-05-08T22:35:27.330015+0000","last_became_active":"2024-05-08T22:35:27.326049+0000","last_became_peered":"2024-05-08T22:35:27.326049+0000","last_unstale":"2024-05-08T22:35:27.330015+0000","last_undegraded":"2024-05-08T22:35:27.330015+0000","last_fullsized":"2024-05-08T22:35:27.330015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:39.573030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.307966+0000","last_change":"2024-05-08T22:35:26.947088+0000","last_active":"2024-05-08T22:35:27.307966+0000","last_peered":"2024-05-08T22:35:27.307966+0000","last_clean":"2024-05-08T22:35:27.307966+0000","last_became_active":"2024-05-08T22:35:26.942836+0000","last_became_peered":"2024-05-08T22:35:26.942836+0000","last_unstale":"2024-05-08T22:35:27.307966+0000","last_undegraded":"2024-05-08T22:35:27.307966+0000","last_fullsized":"2024-05-08T22:35:27.307966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:22.651625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.389802+0000","last_change":"2024-05-08T22:35:27.318014+0000","last_active":"2024-05-08T22:35:28.389802+0000","last_peered":"2024-05-08T22:35:28.389802+0000","last_clean":"2024-05-08T22:35:28.389802+0000","last_became_active":"2024-05-08T22:35:27.317499+0000","last_became_peered":"2024-05-08T22:35:27.317499+0000","last_unstale":"2024-05-08T22:35:28.389802+0000","last_undegraded":"2024-05-08T22:35:28.389802+0000","last_fullsized":"2024-05-08T22:35:28.389802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:03.886275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.306013+0000","last_change":"2024-05-08T22:35:26.947286+0000","last_active":"2024-05-08T22:35:27.306013+0000","last_peered":"2024-05-08T22:35:27.306013+0000","last_clean":"2024-05-08T22:35:27.306013+0000","last_became_active":"2024-05-08T22:35:26.947018+0000","last_became_peered":"2024-05-08T22:35:26.947018+0000","last_unstale":"2024-05-08T22:35:27.306013+0000","last_undegraded":"2024-05-08T22:35:27.306013+0000","last_fullsized":"2024-05-08T22:35:27.306013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:38.563768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.322849+0000","last_change":"2024-05-08T22:35:27.317100+0000","last_active":"2024-05-08T22:35:27.322849+0000","last_peered":"2024-05-08T22:35:27.322849+0000","last_clean":"2024-05-08T22:35:27.322849+0000","last_became_active":"2024-05-08T22:35:27.316854+0000","last_became_peered":"2024-05-08T22:35:27.316854+0000","last_unstale":"2024-05-08T22:35:27.322849+0000","last_undegraded":"2024-05-08T22:35:27.322849+0000","last_fullsized":"2024-05-08T22:35:27.322849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:24.198064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.323979+0000","last_change":"2024-05-08T22:35:26.939371+0000","last_active":"2024-05-08T22:35:27.323979+0000","last_peered":"2024-05-08T22:35:27.323979+0000","last_clean":"2024-05-08T22:35:27.323979+0000","last_became_active":"2024-05-08T22:35:26.939120+0000","last_became_peered":"2024-05-08T22:35:26.939120+0000","last_unstale":"2024-05-08T22:35:27.323979+0000","last_undegraded":"2024-05-08T22:35:27.323979+0000","last_fullsized":"2024-05-08T22:35:27.323979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:23.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.310181+0000","last_change":"2024-05-08T22:35:27.324273+0000","last_active":"2024-05-08T22:35:28.310181+0000","last_peered":"2024-05-08T22:35:28.310181+0000","last_clean":"2024-05-08T22:35:28.310181+0000","last_became_active":"2024-05-08T22:35:27.323940+0000","last_became_peered":"2024-05-08T22:35:27.323940+0000","last_unstale":"2024-05-08T22:35:28.310181+0000","last_undegraded":"2024-05-08T22:35:28.310181+0000","last_fullsized":"2024-05-08T22:35:28.310181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:26.913142+0000","last_clean_scrub_stamp":"2024-05-08T22:35:26.913142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:09.216183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T22:35:28.390035+0000","last_change":"2024-05-08T22:35:26.945692+0000","last_active":"2024-05-08T22:35:28.390035+0000","last_peered":"2024-05-08T22:35:28.390035+0000","last_clean":"2024-05-08T22:35:28.390035+0000","last_became_active":"2024-05-08T22:35:26.945440+0000","last_became_peered":"2024-05-08T22:35:26.945440+0000","last_unstale":"2024-05-08T22:35:28.390035+0000","last_undegraded":"2024-05-08T22:35:28.390035+0000","last_fullsized":"2024-05-08T22:35:28.390035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:25.903880+0000","last_clean_scrub_stamp":"2024-05-08T22:35:25.903880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:09.211946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T22:35:27.301950+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:35:27.301950+0000","last_peered":"2024-05-08T22:35:27.301950+0000","last_clean":"2024-05-08T22:35:27.301950+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:35:27.301950+0000","last_undegraded":"2024-05-08T22:35:27.301950+0000","last_fullsized":"2024-05-08T22:35:27.301950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":281314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95699999999999996,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.499},{"interface":"front","average":{"1min":0.625,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.40999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.89900000000000002,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88300000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.751,"15min":0.75700000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95699999999999996,"5min":1.21,"15min":1.21},"last":0.85799999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.627,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.68300000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.276,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.02,"15min":1.03},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.96999999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.0649999999999999,"15min":1.0629999999999999},"min":{"1min":0.83399999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3480000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.097,"15min":1.107},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.6779999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.034,"15min":1.0329999999999999},"min":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.462,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":286508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82599999999999996,"5min":0.995,"15min":0.995},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98799999999999999,"5min":1.04,"15min":1.04},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1499999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.046}]},{"osd":3,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.018},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.87}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.96399999999999997,"15min":0.98199999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.1839999999999999,"5min":1.431,"15min":1.431},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96199999999999997,"15min":0.96799999999999997},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3180000000000001,"5min":1.508,"15min":1.508},"last":1.226}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94299999999999995,"15min":0.95499999999999996},"min":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.163,"5min":1.327,"15min":1.327},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.79600000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.605,"5min":1.726,"15min":1.726},"last":0.755}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.996},"min":{"1min":0.72499999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.244,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.96899999999999997}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.266,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0509999999999999,"15min":1.0609999999999999},"min":{"1min":0.81200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.097}]}]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1236992,"data_stored":870220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.83799999999999997,"15min":0.85899999999999999},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91600000000000004,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.73199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.071,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85299999999999998},"min":{"1min":0.499,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.167,"15min":1.167},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.139,"5min":1.419,"15min":1.419},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95399999999999996,"15min":0.96299999999999997},"min":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.091,"5min":1.337,"15min":1.337},"last":1.0369999999999999}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1899999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.83499999999999996},"min":{"1min":0.627,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.125,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.1619999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.96699999999999997,"15min":0.97499999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2370000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0629999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.98399999999999999},"min":{"1min":0.65500000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.042,"15min":1.0509999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.331,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.099}]},{"osd":6,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94499999999999995,"15min":0.94899999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.133,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.075,"15min":1.091},"min":{"1min":0.78200000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.359,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.4870000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":643072,"data_stored":280380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.077,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.86699999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0529999999999999,"5min":1.141,"15min":1.141},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.878,"15min":0.88900000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.05,"5min":1.758,"15min":1.758},"last":0.747},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.02}]},{"osd":2,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":1},"min":{"1min":0.58999999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1759999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.93000000000000005,"15min":0.93799999999999994},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.363,"5min":1.419,"15min":1.419},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.091,"15min":1.115},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3220000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":1.004},"min":{"1min":0.61499999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.274,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.002,"15min":1.0049999999999999},"min":{"1min":0.69699999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.054,"15min":1.0680000000000001},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4430000000000001,"5min":1.458,"15min":1.458},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.139,"15min":1.157},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4079999999999999,"5min":1.681,"15min":1.681},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.139,"15min":1.159},"min":{"1min":0.79300000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.466,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.96699999999999997}]},{"osd":7,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.087,"15min":1.109},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4330000000000001,"5min":1.835,"15min":1.835},"last":0.997},{"interface":"front","average":{"1min":1.119,"5min":1.234,"15min":1.2529999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6970000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":634880,"data_stored":279728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.64200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.86099999999999999,"15min":0.85699999999999998},"min":{"1min":0.60799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.80800000000000005,"15min":0.79700000000000004},"min":{"1min":0.57299999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.876,"15min":0.87},"min":{"1min":0.51500000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.879,"15min":0.86699999999999999},"min":{"1min":0.78600000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.96999999999999997,"15min":0.96399999999999997},"min":{"1min":0.63400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.90600000000000003,"15min":0.89200000000000002},"min":{"1min":0.70899999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.087},{"interface":"front","average":{"1min":1.032,"5min":0.98299999999999998,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.92100000000000004,"15min":0.90100000000000002},"min":{"1min":0.73599999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0469999999999999,"15min":1.0369999999999999},"min":{"1min":0.88300000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.083,"15min":1.081},"min":{"1min":0.81599999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.117},{"interface":"front","average":{"1min":1.159,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.95999999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2769999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":622592,"data_stored":272633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1339999999999999,"5min":1.28,"15min":1.28},"last":1.165},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.85499999999999998,"15min":0.85799999999999998},"min":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.878},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1499999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":2,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.878,"15min":0.89400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98699999999999999,"5min":1.29,"15min":1.29},"last":1.214},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0960000000000001,"5min":1.637,"15min":1.637},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0289999999999999,"15min":1.0529999999999999},"min":{"1min":0.68300000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.131,"5min":2.601,"15min":2.601},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0980000000000001,"15min":1.127},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1559999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.1140000000000001}]},{"osd":5,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2609999999999999,"5min":1.792,"15min":1.792},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0269999999999999,"15min":1.036},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.357,"5min":1.575,"15min":1.575},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0980000000000001,"15min":1.1060000000000001},"min":{"1min":0.75900000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.335,"5min":1.831,"15min":1.831},"last":1.081},{"interface":"front","average":{"1min":1.077,"5min":1.101,"15min":1.105},"min":{"1min":0.89200000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3340000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.024}]},{"osd":7,"last update":"Wed May 8 22:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3009999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.1830000000000001},"min":{"1min":0.95699999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.429,"5min":1.944,"15min":1.944},"last":1.1439999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":647168,"data_stored":281292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.90400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86699999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88100000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90700000000000003,"15min":0.90600000000000003},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0269999999999999,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2230000000000001,"5min":1.282,"15min":1.282},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0389999999999999,"15min":1.0429999999999999},"min":{"1min":0.74399999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.302,"5min":1.484,"15min":1.484},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0209999999999999},"min":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0609999999999999,"15min":1.0640000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.996,"5min":1.0720000000000001,"15min":1.085},"min":{"1min":0.71599999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3580000000000001,"5min":1.673,"15min":1.673},"last":0.96499999999999997}]},{"osd":7,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0740000000000001,"15min":1.0820000000000001},"min":{"1min":0.81699999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3620000000000001,"5min":1.409,"15min":1.409},"last":1.05}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":870152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.67100000000000004,"15min":0.68799999999999994},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.749,"5min":1.617,"15min":1.617},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81499999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93600000000000005,"5min":1.446,"15min":1.446},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85599999999999998,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81499999999999995}]},{"osd":3,"last update":"Wed May 8 22:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.80100000000000005,"15min":0.81799999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90100000000000002,"5min":1.714,"15min":1.714},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.327,"15min":1.327},"last":0.68000000000000005}]},{"osd":4,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.70599999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2729999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.996,"15min":1.01},"min":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.149,"5min":1.669,"15min":1.669},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0600000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1,"15min":1.006},"min":{"1min":0.73699999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3580000000000001,"5min":1.474,"15min":1.474},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.96099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70299999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0509999999999999,"15min":1.0580000000000001},"min":{"1min":0.77000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3149999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Wed May 8 22:35:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.085,"15min":1.0960000000000001},"min":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.23,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.74299999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3260000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.03}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:31.709 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:35:31.709 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:35:31.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 Processor -- start 2024-05-08T22:35:31.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 -- start start 2024-05-08T22:35:31.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4c741400d0 0x7f4c741428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c74142e10 0x7f4c74140a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c741423b0 con 0x7f4c74140fb0 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c74143280 con 0x7f4c741400d0 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c78a43700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c741417a0 con 0x7f4c74142e10 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c737fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4c741400d0 0x7f4c741428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c737fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4c741400d0 0x7f4c741428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32882/0 (socket says 172.21.15.179:32882) 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.836+0000 7f4c737fe700 1 -- 172.21.15.179:0/4138242462 learned_addr learned my addr 172.21.15.179:0/4138242462 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:31.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c72ffd700 1 --2- 172.21.15.179:0/4138242462 >> v2:172.21.15.195:3300/0 conn(0x7f4c74142e10 0x7f4c74140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c737fe700 1 -- 172.21.15.179:0/4138242462 >> v2:172.21.15.195:3300/0 conn(0x7f4c74142e10 msgr2=0x7f4c74140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c737fe700 1 --2- 172.21.15.179:0/4138242462 >> v2:172.21.15.195:3300/0 conn(0x7f4c74142e10 0x7f4c74140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c737fe700 1 -- 172.21.15.179:0/4138242462 >> v1:172.21.15.179:6789/0 conn(0x7f4c74140fb0 legacy=0x7f4c74160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c737fe700 1 -- 172.21.15.179:0/4138242462 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74141a80 con 0x7f4c741400d0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c727fc700 1 -- 172.21.15.179:0/4138242462 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980077978 0 0) 0x7f4c741423b0 con 0x7f4c74140fb0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c737fe700 1 --2- 172.21.15.179:0/4138242462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c741428d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4c740719b0 tx=0x7f4c64009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7052b8e0cc864f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c727fc700 1 -- 172.21.15.179:0/4138242462 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011830 con 0x7f4c741400d0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c727fc700 1 -- 172.21.15.179:0/4138242462 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64011990 con 0x7f4c741400d0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c727fc700 1 -- 172.21.15.179:0/4138242462 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011d20 con 0x7f4c741400d0 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 -- 172.21.15.179:0/4138242462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 msgr2=0x7f4c741428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 --2- 172.21.15.179:0/4138242462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c741428d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4c740719b0 tx=0x7f4c64009bf0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 -- 172.21.15.179:0/4138242462 shutdown_connections 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 --2- 172.21.15.179:0/4138242462 >> v2:172.21.15.195:3300/0 conn(0x7f4c74142e10 0x7f4c74140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 --2- 172.21.15.179:0/4138242462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c741428d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 -- 172.21.15.179:0/4138242462 >> 172.21.15.179:0/4138242462 conn(0x7f4c740cfb50 msgr2=0x7f4c740cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 -- 172.21.15.179:0/4138242462 shutdown_connections 2024-05-08T22:35:31.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.840+0000 7f4c78a43700 1 -- 172.21.15.179:0/4138242462 wait complete. 2024-05-08T22:35:31.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 Processor -- start 2024-05-08T22:35:31.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 -- start start 2024-05-08T22:35:31.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c74140fb0 0x7f4c7413eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 0x7f4c740cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cb300 con 0x7f4c741400d0 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7406c900 con 0x7f4c74142e10 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c78a43700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7413f730 con 0x7f4c74140fb0 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32888/0 (socket says 172.21.15.179:32888) 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c737fe700 1 -- 172.21.15.179:0/1093109996 learned_addr learned my addr 172.21.15.179:0/1093109996 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c72ffd700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c74140fb0 0x7f4c7413eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.844+0000 7f4c73fff700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 0x7f4c740cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 -- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c74140fb0 msgr2=0x7f4c7413eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c74140fb0 0x7f4c7413eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 -- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 msgr2=0x7f4c740cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 0x7f4c740cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 -- 172.21.15.179:0/1093109996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009570 con 0x7f4c741400d0 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c73fff700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 0x7f4c740cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:31.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c737fe700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b2840 tx=0x7f4c64004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64004510 con 0x7f4c741400d0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6401e070 con 0x7f4c741400d0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7406cd90 con 0x7f4c741400d0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640038e0 con 0x7f4c741400d0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7406c2b0 con 0x7f4c741400d0 2024-05-08T22:35:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c740cd140 con 0x7f4c741400d0 2024-05-08T22:35:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c64003a40 con 0x7f4c741400d0 2024-05-08T22:35:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.848+0000 7f4c6bfff700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 0x7f4c54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.852+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4c64085370 con 0x7f4c741400d0 2024-05-08T22:35:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.852+0000 7f4c72ffd700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 0x7f4c54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.852+0000 7f4c72ffd700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 0x7f4c54068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c7406f150 tx=0x7f4c5c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:31.856+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4c640526b0 con 0x7f4c741400d0 2024-05-08T22:35:32.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.084+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4c7406cf20 con 0x7f4c741400d0 2024-05-08T22:35:32.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.088+0000 7f4c6bfff700 1 -- 172.21.15.179:0/1093109996 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4c64055cd0 con 0x7f4c741400d0 2024-05-08T22:35:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 msgr2=0x7f4c54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 0x7f4c54068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c7406f150 tx=0x7f4c5c009240 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 msgr2=0x7f4c7406dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b2840 tx=0x7f4c64004770 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 shutdown_connections 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c54066730 0x7f4c54068bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c74142e10 0x7f4c740cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c74140fb0 0x7f4c7413eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 --2- 172.21.15.179:0/1093109996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c741400d0 0x7f4c7406dfe0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 >> 172.21.15.179:0/1093109996 conn(0x7f4c740cfb50 msgr2=0x7f4c7415ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 shutdown_connections 2024-05-08T22:35:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.092+0000 7f4c78a43700 1 -- 172.21.15.179:0/1093109996 wait complete. 2024-05-08T22:35:32.097 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:35:32.107 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:32.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 Processor -- start 2024-05-08T22:35:32.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 -- start start 2024-05-08T22:35:32.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164806de70 0x7f1648161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1648164db0 0x7f164806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1648165d50 con 0x7f164815ef80 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164806ad70 con 0x7f1648164db0 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1648165260 con 0x7f164806de70 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1648164db0 0x7f164806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164cafc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f164815ef80 0x7f164806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57948/0 (socket says 172.21.15.179:57948) 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164cafc700 1 -- 172.21.15.179:0/2978529345 learned_addr learned my addr 172.21.15.179:0/2978529345 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f1647fff700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164806de70 0x7f1648161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 -- 172.21.15.179:0/2978529345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164806de70 msgr2=0x7f1648161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164806de70 0x7f1648161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 -- 172.21.15.179:0/2978529345 >> v1:172.21.15.179:6789/0 conn(0x7f164815ef80 legacy=0x7f164806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f16477fe700 1 -- 172.21.15.179:0/2978529345 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280163293 0 0) 0x7f1648165d50 con 0x7f164815ef80 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 -- 172.21.15.179:0/2978529345 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164806e4b0 con 0x7f1648164db0 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164d2fd700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1648164db0 0x7f164806b200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1638009850 tx=0x7f1638009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5132f4e98c0fa1e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f16477fe700 1 -- 172.21.15.179:0/2978529345 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638011830 con 0x7f1648164db0 2024-05-08T22:35:32.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f16477fe700 1 -- 172.21.15.179:0/2978529345 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638011990 con 0x7f1648164db0 2024-05-08T22:35:32.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f16477fe700 1 -- 172.21.15.179:0/2978529345 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638011d40 con 0x7f1648164db0 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 -- 172.21.15.179:0/2978529345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1648164db0 msgr2=0x7f164806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.216+0000 7f164dafe700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1648164db0 0x7f164806b200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1638009850 tx=0x7f1638009b60 comp rx=0 tx=0).stop 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- 172.21.15.179:0/2978529345 shutdown_connections 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1648164db0 0x7f164806b200 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 --2- 172.21.15.179:0/2978529345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164806de70 0x7f1648161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- 172.21.15.179:0/2978529345 >> 172.21.15.179:0/2978529345 conn(0x7f16480ce9d0 msgr2=0x7f16480cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- 172.21.15.179:0/2978529345 shutdown_connections 2024-05-08T22:35:32.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- 172.21.15.179:0/2978529345 wait complete. 2024-05-08T22:35:32.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 Processor -- start 2024-05-08T22:35:32.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- start start 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 0x7f16480ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 0x7f16480ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c94f0 con 0x7f164806de70 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c97d0 con 0x7f1648164db0 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164dafe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c9ab0 con 0x7f164815ef80 2024-05-08T22:35:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32904/0 (socket says 172.21.15.179:32904) 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 -- 172.21.15.179:0/3903291559 learned_addr learned my addr 172.21.15.179:0/3903291559 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f1647fff700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 0x7f16480ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164d2fd700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 0x7f16480ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 -- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 msgr2=0x7f16480ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 0x7f16480ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 -- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 msgr2=0x7f16480ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 0x7f16480ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 -- 172.21.15.179:0/3903291559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16380094b0 con 0x7f164806de70 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f1647fff700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 0x7f16480ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164d2fd700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 0x7f16480ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.220+0000 7f164cafc700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f164000c580 tx=0x7f164000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640004ca0 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640004e20 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16480c9d90 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640007310 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16480ca7f0 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648067740 con 0x7f164806de70 2024-05-08T22:35:32.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1640007470 con 0x7f164806de70 2024-05-08T22:35:32.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 0x7f162c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.224+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1640084010 con 0x7f164806de70 2024-05-08T22:35:32.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.228+0000 7f1647fff700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 0x7f162c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.228+0000 7f1647fff700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 0x7f162c068c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f16480ce140 tx=0x7f163000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.236+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1640051240 con 0x7f164806de70 2024-05-08T22:35:32.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16480c9630 con 0x7f164806de70 2024-05-08T22:35:32.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f16457fa700 1 -- 172.21.15.179:0/3903291559 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f16480c9630 con 0x7f164806de70 2024-05-08T22:35:32.451 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:32.451 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 msgr2=0x7f162c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 0x7f162c068c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f16480ce140 tx=0x7f163000b040 comp rx=0 tx=0).stop 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 msgr2=0x7f16481510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f164000c580 tx=0x7f164000c940 comp rx=0 tx=0).stop 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 shutdown_connections 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f162c066760 0x7f162c068c20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1648164db0 0x7f16480ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f164815ef80 0x7f16480ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.448+0000 7f164dafe700 1 --2- 172.21.15.179:0/3903291559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f164806de70 0x7f16481510a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.452+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 >> 172.21.15.179:0/3903291559 conn(0x7f16480ce9d0 msgr2=0x7f164806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.452+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 shutdown_connections 2024-05-08T22:35:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.452+0000 7f164dafe700 1 -- 172.21.15.179:0/3903291559 wait complete. 2024-05-08T22:35:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:35:32.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:35:27.295084+0000', 'modified': '2024-05-08T22:35:30.308670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55412': {'gid': 55412, 'name': 'd', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/1172092943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 1172092943}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1172092943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:35:32.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:35:32.869 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:35:32.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.988+0000 7f4188fc7700 1 Processor -- start 2024-05-08T22:35:32.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.988+0000 7f4188fc7700 1 -- start start 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f41840adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840ae0a0 con 0x7f41840ccdd0 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840b1cb0 con 0x7f4184150bc0 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41840b2930 con 0x7f41840ad740 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4183fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f41840ccdd0 0x7f41840cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57958/0 (socket says 172.21.15.179:57958) 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4183fff700 1 -- 172.21.15.179:0/2825303351 learned_addr learned my addr 172.21.15.179:0/2825303351 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:32.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f41840adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844838594 0 0) 0x7f41840ae0a0 con 0x7f41840ccdd0 2024-05-08T22:35:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41700035d0 con 0x7f41840ccdd0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256838071 0 0) 0x7f41840b1cb0 con 0x7f4184150bc0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41840ae0a0 con 0x7f4184150bc0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323628650 0 0) 0x7f41700035d0 con 0x7f41840ccdd0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 msgr2=0x7f41840adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f41840adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 >> v1:172.21.15.195:6789/0 conn(0x7f4184150bc0 legacy=0x7f418415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840b1f90 con 0x7f41840ccdd0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f417c0019f0 con 0x7f41840ccdd0 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 >> v1:172.21.15.179:6789/0 conn(0x7f41840ccdd0 legacy=0x7f41840cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f4170003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4170004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4170004db0 0x7f4170005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41840b2930 con 0x7f4170004140 2024-05-08T22:35:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417000bbc0 con 0x7f4170004db0 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417000df60 con 0x7f41840ad740 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41837fe700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f4170003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4183fff700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4170004db0 0x7f4170005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4170004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 -- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 msgr2=0x7f4170003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41840ad740 0x7f4170003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 -- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4170004db0 msgr2=0x7f4170005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4170004db0 0x7f4170005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 -- 172.21.15.179:0/2825303351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700035d0 con 0x7f4170004140 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4170004870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4174009b70 tx=0x7f41700058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc476b00f89d0ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400e6c0 con 0x7f4170004140 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400e820 con 0x7f4170004140 2024-05-08T22:35:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f41827fc700 1 -- 172.21.15.179:0/2825303351 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400eb90 con 0x7f4170004140 2024-05-08T22:35:32.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 -- 172.21.15.179:0/2825303351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 msgr2=0x7f4170004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4188fc7700 1 --2- 172.21.15.179:0/2825303351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4170004870 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4174009b70 tx=0x7f41700058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.992+0000 7f4183fff700 1 -- 172.21.15.179:0/2825303351 reap_dead start 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- 172.21.15.179:0/2825303351 shutdown_connections 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- 172.21.15.179:0/2825303351 >> 172.21.15.179:0/2825303351 conn(0x7f41840ce960 msgr2=0x7f41840ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- 172.21.15.179:0/2825303351 shutdown_connections 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- 172.21.15.179:0/2825303351 wait complete. 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 Processor -- start 2024-05-08T22:35:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- start start 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4184143cb0 0x7f418415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418415c2e0 0x7f418406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184145b20 con 0x7f4170004140 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184145e00 con 0x7f418415c2e0 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4188fc7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41841460e0 con 0x7f4184143cb0 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f41837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f41837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32922/0 (socket says 172.21.15.179:32922) 2024-05-08T22:35:33.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f41837fe700 1 -- 172.21.15.179:0/3787727764 learned_addr learned my addr 172.21.15.179:0/3787727764 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4182ffd700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4184143cb0 0x7f418415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:32.996+0000 7f4183fff700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418415c2e0 0x7f418406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 -- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4184143cb0 msgr2=0x7f418415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4184143cb0 0x7f418415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 -- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418415c2e0 msgr2=0x7f418406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418415c2e0 0x7f418406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 -- 172.21.15.179:0/3787727764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41841463c0 con 0x7f4170004140 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f41837fe700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f810 tx=0x7f416c00fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f417bfff700 1 -- 172.21.15.179:0/3787727764 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0052e0 con 0x7f4170004140 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f417bfff700 1 -- 172.21.15.179:0/3787727764 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c005440 con 0x7f4170004140 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41841466a0 con 0x7f4170004140 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f417bfff700 1 -- 172.21.15.179:0/3787727764 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0038e0 con 0x7f4170004140 2024-05-08T22:35:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.000+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4184144640 con 0x7f4170004140 2024-05-08T22:35:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f417bfff700 1 -- 172.21.15.179:0/3787727764 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f416c003a40 con 0x7f4170004140 2024-05-08T22:35:33.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f417bfff700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 0x7f4164068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f417bfff700 1 -- 172.21.15.179:0/3787727764 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f416c0840e0 con 0x7f4170004140 2024-05-08T22:35:33.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f4182ffd700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 0x7f4164068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f4182ffd700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 0x7f4164068c20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f41740090b0 tx=0x7f417401f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 Processor -- start 2024-05-08T22:35:33.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 -- start start 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xe21d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xdf2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd51d80 con 0xdf29b0 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4f510 con 0xe21970 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.004+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xd4ae00 con 0xe09a00 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4182ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xdf2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32934/0 (socket says 172.21.15.179:32934) 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f41837fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xe21d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 -- 172.21.15.179:0/4103684301 learned_addr learned my addr 172.21.15.179:0/4103684301 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 -- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 msgr2=0xdf2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xdf2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 -- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 msgr2=0xe21d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xe21d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 -- 172.21.15.179:0/4103684301 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xbbd960 con 0xdf29b0 2024-05-08T22:35:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f4183fff700 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f418406c580 tx=0x7f417c002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788d07d003c57606 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f41797fa700 1 -- 172.21.15.179:0/4103684301 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c007710 con 0xdf29b0 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f41797fa700 1 -- 172.21.15.179:0/4103684301 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c007870 con 0xdf29b0 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f41797fa700 1 -- 172.21.15.179:0/4103684301 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c007a20 con 0xdf29b0 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- 172.21.15.179:0/4103684301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 msgr2=0xdf2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f418406c580 tx=0x7f417c002190 comp rx=0 tx=0).stop 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- 172.21.15.179:0/4103684301 shutdown_connections 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xdf2d90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xdf2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 --2- 172.21.15.179:0/4103684301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xe21d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- 172.21.15.179:0/4103684301 >> 172.21.15.179:0/4103684301 conn(0xb724d0 msgr2=0xe0c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- 172.21.15.179:0/4103684301 shutdown_connections 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- 172.21.15.179:0/4103684301 wait complete. 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 -- start start 2024-05-08T22:35:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.008+0000 7f418d4d8740 1 Processor -- start 2024-05-08T22:35:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe08d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd08730 con 0xe21970 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xd0e5b0 con 0xdf29b0 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xbf6aa0 con 0xe09a00 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 10 client.0 _ll_register_callbacks cb 0xd44610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32942/0 (socket says 172.21.15.179:32942) 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 -- 172.21.15.179:0/2431265398 learned_addr learned my addr 172.21.15.179:0/2431265398 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41837fe700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4182ffd700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe08d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 -- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 msgr2=0xe08d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe08d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 -- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 msgr2=0xe1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 -- 172.21.15.179:0/2431265398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41840b1f90 con 0xe21970 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f4183fff700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f41840cc910 tx=0x7f417c001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62babcc418cbaca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c007710 con 0xe21970 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f417c02ac30 con 0xe21970 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xb7f1e0 con 0xe21970 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f417c02adf0 con 0xe21970 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 10 client.6180 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:33.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0xd11220 con 0xe21970 2024-05-08T22:35:33.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xb7f1e0 con 0xe21970 2024-05-08T22:35:33.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f417c012050 con 0xe21970 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 10 client.6180 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 10 client.6180 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 20 client.6180 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xdf5990 0xe3e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 4 client.6180 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xbc9460 con 0xdf5990 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41837fe700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xdf5990 0xe3e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41627fc700 10 client.6180 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.012+0000 7f41837fe700 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xdf5990 0xe3e460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acade38f65b8f44c server_cookie=e1f7faa5b6f4de43 in_seq=0 out_seq=0 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f41627fc700 1 -- 172.21.15.179:0/2431265398 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xbc9460 con 0xdf5990 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f41627fc700 10 client.6180 handle_command_reply: tid=0 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 client.6180 shutdown 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 msgr2=0xe092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f41840cc910 tx=0x7f417c001d00 comp rx=0 tx=0).stop 2024-05-08T22:35:33.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 shutdown_connections 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xdf5990 0xe3e460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe21970 0xe092d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe08d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 --2- 172.21.15.179:0/2431265398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 >> 172.21.15.179:0/2431265398 conn(0xb724d0 msgr2=0xe20cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f41627fc700 0 client.6180 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 shutdown_connections 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 1 -- 172.21.15.179:0/2431265398 wait complete. 2024-05-08T22:35:33.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.016+0000 7f418d4d8740 20 client.6180 trim_cache size 0 max 16384 2024-05-08T22:35:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 Processor -- start 2024-05-08T22:35:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 -- start start 2024-05-08T22:35:33.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xde0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd154f0 con 0xdf29b0 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4e6e0 con 0xe21970 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xce9470 con 0xe09a00 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32946/0 (socket says 172.21.15.179:32946) 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 -- 172.21.15.179:0/3538663247 learned_addr learned my addr 172.21.15.179:0/3538663247 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f4182ffd700 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f4183fff700 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xde0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 -- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 msgr2=0xe429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 -- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 msgr2=0xde0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xde0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.032+0000 7f41837fe700 1 -- 172.21.15.179:0/3538663247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcf7450 con 0xdf29b0 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f41837fe700 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f416c00fe70 tx=0x7f416c073040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e9af4e484339fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f4179ffb700 1 -- 172.21.15.179:0/3538663247 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c05b0b0 con 0xdf29b0 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f4179ffb700 1 -- 172.21.15.179:0/3538663247 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c054fd0 con 0xdf29b0 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f4179ffb700 1 -- 172.21.15.179:0/3538663247 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c058e00 con 0xdf29b0 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- 172.21.15.179:0/3538663247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 msgr2=0xe42480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f416c00fe70 tx=0x7f416c073040 comp rx=0 tx=0).stop 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- 172.21.15.179:0/3538663247 shutdown_connections 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe21970 0xde0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe09a00 0xe429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- 172.21.15.179:0/3538663247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xdf29b0 0xe42480 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- 172.21.15.179:0/3538663247 >> 172.21.15.179:0/3538663247 conn(0xb724d0 msgr2=0xe3a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- 172.21.15.179:0/3538663247 shutdown_connections 2024-05-08T22:35:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- 172.21.15.179:0/3538663247 wait complete. 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- start start 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 Processor -- start 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe49ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe21970 0xe48600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xe47ee0 con 0xe09a00 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xe0cae0 con 0xdf29b0 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xe8c4a0 con 0xe21970 2024-05-08T22:35:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f418d4d8740 10 client.0 _ll_register_callbacks cb 0xe0c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f4182ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.036+0000 7f4182ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32956/0 (socket says 172.21.15.179:32956) 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4183fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe21970 0xe48600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 -- 172.21.15.179:0/1240169461 learned_addr learned my addr 172.21.15.179:0/1240169461 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41837fe700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe49ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 -- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe21970 msgr2=0xe48600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe21970 0xe48600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 -- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 msgr2=0xe49ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe49ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 -- 172.21.15.179:0/1240169461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4184144640 con 0xe09a00 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41837fe700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe49ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f4182ffd700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4174009a40 tx=0x7f4174004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16723a20f0ffc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174031040 con 0xe09a00 2024-05-08T22:35:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4174003c90 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f417c007870 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4174022040 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 10 client.6186 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0xceb720 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c007870 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4174034070 con 0xe09a00 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 10 client.6186 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 10 client.6186 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 20 client.6186 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xde3b50 0xde3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 4 client.6186 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "status"}) v1 -- 0xbee1e0 con 0xde3b50 2024-05-08T22:35:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41837fe700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xde3b50 0xde3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41837fe700 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xde3b50 0xde3f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259a0ae1b219338d server_cookie=22999ca01b8dcd0a in_seq=0 out_seq=0 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 10 client.6186 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 1 -- 172.21.15.179:0/1240169461 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0xbee1e0 con 0xde3b50 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 10 client.6186 handle_command_reply: tid=0 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 client.6186 shutdown 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 msgr2=0xe4a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4174009a40 tx=0x7f4174004770 comp rx=0 tx=0).stop 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 shutdown_connections 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0xde3b50 0xde3f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe21970 0xe48600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f41627fc700 0 client.6186 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe09a00 0xe4a370 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 --2- 172.21.15.179:0/1240169461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xdf29b0 0xe49ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.040+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 >> 172.21.15.179:0/1240169461 conn(0xb724d0 msgr2=0xe42480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 shutdown_connections 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f418d4d8740 1 -- 172.21.15.179:0/1240169461 wait complete. 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f418d4d8740 20 client.6186 trim_cache size 0 max 16384 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:35:33.049 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 55412, 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 5.7470822659999996, 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 24, 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 55, 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 55, 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 10.237438045999999 2024-05-08T22:35:33.050 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 msgr2=0x7f4164068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 0x7f4164068c20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f41740090b0 tx=0x7f417401f040 comp rx=0 tx=0).stop 2024-05-08T22:35:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 msgr2=0x7f4184146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f810 tx=0x7f416c00fb20 comp rx=0 tx=0).stop 2024-05-08T22:35:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 shutdown_connections 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4164066760 0x7f4164068c20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418415c2e0 0x7f418406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4184143cb0 0x7f418415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 --2- 172.21.15.179:0/3787727764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4170004140 0x7f4184146bc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 >> 172.21.15.179:0/3787727764 conn(0x7f41840ce960 msgr2=0x7f41840cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 shutdown_connections 2024-05-08T22:35:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.044+0000 7f4188fc7700 1 -- 172.21.15.179:0/3787727764 wait complete. 2024-05-08T22:35:33.063 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:35:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 Processor -- start 2024-05-08T22:35:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 -- start start 2024-05-08T22:35:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 0x7f58000cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58000cb510 0x7f580006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580006abb0 con 0x7f58000cc700 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000cb9a0 con 0x7f58000cb510 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f58073e6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580006ae90 con 0x7f58000ce380 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f5806be5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58000cb510 0x7f580006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f5806be5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58000cb510 0x7f580006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32958/0 (socket says 172.21.15.179:32958) 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.176+0000 7f5806be5700 1 -- 172.21.15.179:0/3414947419 learned_addr learned my addr 172.21.15.179:0/3414947419 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5805be3700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 0x7f58000cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5806be5700 1 -- 172.21.15.179:0/3414947419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 msgr2=0x7f58000cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5806be5700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 0x7f58000cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5806be5700 1 -- 172.21.15.179:0/3414947419 >> v1:172.21.15.195:6789/0 conn(0x7f58000cc700 legacy=0x7f580015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5806be5700 1 -- 172.21.15.179:0/3414947419 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000ce800 con 0x7f58000cb510 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5805be3700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 0x7f58000cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f5806be5700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cb510 0x7f580006b320 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009910 tx=0x7f57f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f97a23379ff09b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58053e2700 1 -- 172.21.15.179:0/3414947419 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800ba60 con 0x7f58000cb510 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58053e2700 1 -- 172.21.15.179:0/3414947419 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f800bbc0 con 0x7f58000cb510 2024-05-08T22:35:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58053e2700 1 -- 172.21.15.179:0/3414947419 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8009e60 con 0x7f58000cb510 2024-05-08T22:35:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- 172.21.15.179:0/3414947419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cb510 msgr2=0x7f580006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cb510 0x7f580006b320 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009910 tx=0x7f57f8009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- 172.21.15.179:0/3414947419 shutdown_connections 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000ce380 0x7f58000cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 --2- 172.21.15.179:0/3414947419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cb510 0x7f580006b320 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- 172.21.15.179:0/3414947419 >> 172.21.15.179:0/3414947419 conn(0x7f58000ce9d0 msgr2=0x7f58000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- 172.21.15.179:0/3414947419 shutdown_connections 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- 172.21.15.179:0/3414947419 wait complete. 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 Processor -- start 2024-05-08T22:35:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.180+0000 7f58073e6700 1 -- start start 2024-05-08T22:35:33.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000cb510 0x7f580014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58000ce380 0x7f580015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580014ece0 con 0x7f58000cc700 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580013eed0 con 0x7f58000ce380 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5800142170 con 0x7f58000cb510 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:32962/0 (socket says 172.21.15.179:32962) 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 -- 172.21.15.179:0/3175081003 learned_addr learned my addr 172.21.15.179:0/3175081003 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58063e4700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000cb510 0x7f580014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5806be5700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58000ce380 0x7f580015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 -- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000cb510 msgr2=0x7f580014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000cb510 0x7f580014f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 -- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58000ce380 msgr2=0x7f580015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58000ce380 0x7f580015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 -- 172.21.15.179:0/3175081003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009570 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f5805be3700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009fd0 tx=0x7f57f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f000dde0 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f000f040 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5800156fc0 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0003c80 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5800141690 con 0x7f58000cc700 2024-05-08T22:35:33.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.184+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5800067740 con 0x7f58000cc700 2024-05-08T22:35:33.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.188+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57f0004030 con 0x7f58000cc700 2024-05-08T22:35:33.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.188+0000 7f57feffd700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.188+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f57f0084230 con 0x7f58000cc700 2024-05-08T22:35:33.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.188+0000 7f58063e4700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.188+0000 7f58063e4700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 0x7f57ec068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000f60 tx=0x7f57f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.196+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57f0051470 con 0x7f58000cc700 2024-05-08T22:35:33.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f580004e700 con 0x7f58000cc700 2024-05-08T22:35:33.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f57feffd700 1 -- 172.21.15.179:0/3175081003 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f57f0054a90 con 0x7f58000cc700 2024-05-08T22:35:33.407 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:35:33.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 msgr2=0x7f57ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 0x7f57ec068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000f60 tx=0x7f57f4009040 comp rx=0 tx=0).stop 2024-05-08T22:35:33.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 msgr2=0x7f580015d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009fd0 tx=0x7f57f000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 shutdown_connections 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58000ce380 0x7f580015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58000cc700 0x7f580015d340 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 --2- 172.21.15.179:0/3175081003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58000cb510 0x7f580014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 >> 172.21.15.179:0/3175081003 conn(0x7f58000ce9d0 msgr2=0x7f580006e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 shutdown_connections 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:33.404+0000 7f58073e6700 1 -- 172.21.15.179:0/3175081003 wait complete. 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:35:33.421 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:35:33.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.422 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.427 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42438sec preferred_lft 42438sec 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:35:33.428 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:35:33.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.430 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:35:33.443 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:35:33.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.445 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:35:33.452 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:35:33.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:35:33.453 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:35:33.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.534 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:35:33.540 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:35:33.540 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:35:33.541 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:35:33.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.541 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:35:33.557 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:35:33.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:35:33.557 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:33.563 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:35:33.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:35:33.563 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:33.611 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:35:33.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:35:33.629 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:35:33.640 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:35:33.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:35:33.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.642 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:35:33.647 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:35:33.647 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:35:33.651 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:35:33.671 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:35:33.671 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:35:33.733 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:35:33.734 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:35:33.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:33.735 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:35:33.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:35:33.740+0000 7fb79b62b640 -1 init, newargv = 0x559a7bbe50a0 newargc=15 2024-05-08T22:35:33.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[38952]: starting ceph client 2024-05-08T22:35:33.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[38952]: starting fuse 2024-05-08T22:35:34.840 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:35:34.859 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:35:34.859 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:35:34.875 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:35:34.875 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:35:34.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:34.876 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:35:34.882 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:35:34.882 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:35:34.883 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:35:34.884 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:35:35.009 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.38952.asok 2024-05-08T22:35:35.194 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38952.asok 2024-05-08T22:35:35.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:35.194 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38952.asok status 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:35:35.298 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "38952", 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 6142, 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 6142 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:35:35.299 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "172.21.15.179:0", 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 730907200 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "172.21.15.179:0", 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 730907200 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.6142 172.21.15.179:0/730907200", 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "172.21.15.179:0/730907200", 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 24, 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 55, 2024-05-08T22:35:35.300 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:35:35.301 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:35:35.301 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:35:35.301 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:35.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:35.308 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:35:35.308 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:35.313 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:35:35.314 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:35.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:35.314 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:35:35.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:35.375 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-08T22:35:35.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:35:35.397 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-08T22:35:35.409 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:35:35.410 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:35:35.410 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:35:35.410 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:35:35.410 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-08T22:35:35.410 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:35:35.522 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:35:35.711 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:35.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 Processor -- start 2024-05-08T22:35:35.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 -- start start 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 0x7f44780c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f44780ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f44780cd2a0 0x7f44780cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780ca350 con 0x7f44780cd2a0 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780cf280 con 0x7f4478145ab0 2024-05-08T22:35:35.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4477fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b2a50 con 0x7f44780ce8e0 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f44780cd2a0 0x7f44780cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f44780cd2a0 0x7f44780cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60166/0 (socket says 172.21.15.179:60166) 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 -- 172.21.15.179:0/3111170049 learned_addr learned my addr 172.21.15.179:0/3111170049 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44767fc700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f44780ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4476ffd700 1 --2- 172.21.15.179:0/3111170049 >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 0x7f44780c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 -- 172.21.15.179:0/3111170049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 msgr2=0x7f44780ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f44780ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 -- 172.21.15.179:0/3111170049 >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 msgr2=0x7f44780c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 --2- 172.21.15.179:0/3111170049 >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 0x7f44780c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 -- 172.21.15.179:0/3111170049 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780b32f0 con 0x7f44780cd2a0 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4476ffd700 1 --2- 172.21.15.179:0/3111170049 >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 0x7f44780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:35.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f44777fe700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f44780cd730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4478071a60 tx=0x7f4470009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0172eb8c3a17ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4475ffb700 1 -- 172.21.15.179:0/3111170049 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470011830 con 0x7f44780cd2a0 2024-05-08T22:35:35.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4475ffb700 1 -- 172.21.15.179:0/3111170049 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470011990 con 0x7f44780cd2a0 2024-05-08T22:35:35.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.824+0000 7f4475ffb700 1 -- 172.21.15.179:0/3111170049 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470011d20 con 0x7f44780cd2a0 2024-05-08T22:35:35.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 -- 172.21.15.179:0/3111170049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 msgr2=0x7f44780cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f44780cd730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4478071a60 tx=0x7f4470009bf0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 -- 172.21.15.179:0/3111170049 shutdown_connections 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f44780cd730 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 --2- 172.21.15.179:0/3111170049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f44780ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 --2- 172.21.15.179:0/3111170049 >> v2:172.21.15.195:3300/0 conn(0x7f4478145ab0 0x7f44780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 -- 172.21.15.179:0/3111170049 >> 172.21.15.179:0/3111170049 conn(0x7f44780cfae0 msgr2=0x7f44780cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 -- 172.21.15.179:0/3111170049 shutdown_connections 2024-05-08T22:35:35.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 -- 172.21.15.179:0/3111170049 wait complete. 2024-05-08T22:35:35.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.828+0000 7f4477fff700 1 Processor -- start 2024-05-08T22:35:35.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- start start 2024-05-08T22:35:35.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f447815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4478145ab0 0x7f447813c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447815ed40 con 0x7f44780cd2a0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447806d390 con 0x7f4478145ab0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4478161360 con 0x7f44780ce8e0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60170/0 (socket says 172.21.15.179:60170) 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 -- 172.21.15.179:0/2600427577 learned_addr learned my addr 172.21.15.179:0/2600427577 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f44777fe700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4478145ab0 0x7f447813c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f44767fc700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f447815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 -- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 msgr2=0x7f447815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f447815e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 -- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4478145ab0 msgr2=0x7f447813c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4478145ab0 0x7f447813c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 -- 172.21.15.179:0/2600427577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009570 con 0x7f44780cd2a0 2024-05-08T22:35:35.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f44767fc700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f447815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4476ffd700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f446800e810 tx=0x7f446800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468013610 con 0x7f44780cd2a0 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468013770 con 0x7f44780cd2a0 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4478161640 con 0x7f44780cd2a0 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468013b00 con 0x7f44780cd2a0 2024-05-08T22:35:35.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.832+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447813cca0 con 0x7f44780cd2a0 2024-05-08T22:35:35.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478161b60 con 0x7f44780cd2a0 2024-05-08T22:35:35.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4468013c60 con 0x7f44780cd2a0 2024-05-08T22:35:35.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f446ffff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 0x7f445c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4468083d20 con 0x7f44780cd2a0 2024-05-08T22:35:35.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f44767fc700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 0x7f445c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.836+0000 7f44767fc700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 0x7f445c068c20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f44781f6420 tx=0x7f446000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:35.844+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4468050f60 con 0x7f44780cd2a0 2024-05-08T22:35:36.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.056+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f447806d500 con 0x7f44780cd2a0 2024-05-08T22:35:36.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.056+0000 7f446ffff700 1 -- 172.21.15.179:0/2600427577 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f4468021090 con 0x7f44780cd2a0 2024-05-08T22:35:36.060 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:36.060 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 msgr2=0x7f445c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 0x7f445c068c20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f44781f6420 tx=0x7f446000b040 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 msgr2=0x7f4478160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f446800e810 tx=0x7f446800eb20 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 shutdown_connections 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f445c066760 0x7f445c068c20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4478145ab0 0x7f447813c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44780ce8e0 0x7f447815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 --2- 172.21.15.179:0/2600427577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44780cd2a0 0x7f4478160c60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 >> 172.21.15.179:0/2600427577 conn(0x7f44780cfae0 msgr2=0x7f447806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 shutdown_connections 2024-05-08T22:35:36.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.060+0000 7f4477fff700 1 -- 172.21.15.179:0/2600427577 wait complete. 2024-05-08T22:35:36.064 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:35:36.075 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T22:35:36.217 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:35:36.217 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:35:36.217 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T22:35:36.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 Processor -- start 2024-05-08T22:35:36.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- start start 2024-05-08T22:35:36.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71f9841e0 0x55c71f9c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71fa5b700 0x55c71fa57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71f91ba00 con 0x55c71f9b7a80 2024-05-08T22:35:36.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71f9172f0 con 0x55c71fa5b700 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c71f8d38e0 con 0x55c71f9841e0 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60176/0 (socket says 172.21.15.179:60176) 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 -- 172.21.15.179:0/3409824394 learned_addr learned my addr 172.21.15.179:0/3409824394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04e0a700 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71f9841e0 0x55c71f9c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb0560b700 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71fa5b700 0x55c71fa57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 -- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71f9841e0 msgr2=0x55c71f9c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71f9841e0 0x55c71f9c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 -- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71fa5b700 msgr2=0x55c71fa57650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71fa5b700 0x55c71fa57650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 -- 172.21.15.179:0/3409824394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4005d20 con 0x55c71f9b7a80 2024-05-08T22:35:36.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb04609700 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8001de0 tx=0x7ffaf800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab1eac86293268d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb03607700 1 -- 172.21.15.179:0/3409824394 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8014070 con 0x55c71f9b7a80 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb03607700 1 -- 172.21.15.179:0/3409824394 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8008030 con 0x55c71f9b7a80 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb03607700 1 -- 172.21.15.179:0/3409824394 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800bdb0 con 0x55c71f9b7a80 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- 172.21.15.179:0/3409824394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 msgr2=0x55c71f9b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8001de0 tx=0x7ffaf800b4c0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- 172.21.15.179:0/3409824394 shutdown_connections 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71fa5b700 0x55c71fa57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9b7a80 0x55c71f9b5c10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 --2- 172.21.15.179:0/3409824394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71f9841e0 0x55c71f9c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- 172.21.15.179:0/3409824394 >> 172.21.15.179:0/3409824394 conn(0x55c71f98a740 msgr2=0x55c71f9b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- 172.21.15.179:0/3409824394 shutdown_connections 2024-05-08T22:35:36.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.256+0000 7ffb072d5140 1 -- 172.21.15.179:0/3409824394 wait complete. 2024-05-08T22:35:36.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 Processor -- start 2024-05-08T22:35:36.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 -- start start 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 0x55c71fa47e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71f9b7a80 0x55c71fa47650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71fa5b700 0x55c71f9c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71f8d3580 con 0x55c71f9841e0 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71f8a98d0 con 0x55c71f9b7a80 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb072d5140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c71fa47b90 con 0x55c71fa5b700 2024-05-08T22:35:36.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb04e0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 0x55c71fa47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb04e0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 0x55c71fa47e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60192/0 (socket says 172.21.15.179:60192) 2024-05-08T22:35:36.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb04e0a700 1 -- 172.21.15.179:0/1267096867 learned_addr learned my addr 172.21.15.179:0/1267096867 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb04609700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71f9b7a80 0x55c71fa47650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.260+0000 7ffb0560b700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71fa5b700 0x55c71f9c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 -- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71fa5b700 msgr2=0x55c71f9c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c71fa5b700 0x55c71f9c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 -- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71f9b7a80 msgr2=0x55c71fa47650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c71f9b7a80 0x55c71fa47650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 -- 172.21.15.179:0/1267096867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf800a040 con 0x55c71f9841e0 2024-05-08T22:35:36.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb04e0a700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 0x55c71fa47e30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffafc002230 tx=0x7ffafc00d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb02605700 1 -- 172.21.15.179:0/1267096867 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0062e0 con 0x55c71f9841e0 2024-05-08T22:35:36.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb02605700 1 -- 172.21.15.179:0/1267096867 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00dac0 con 0x55c71f9841e0 2024-05-08T22:35:36.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c71f8b4f40 con 0x55c71f9841e0 2024-05-08T22:35:36.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb02605700 1 -- 172.21.15.179:0/1267096867 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00de70 con 0x55c71f9841e0 2024-05-08T22:35:36.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c71f8b49a0 con 0x55c71f9841e0 2024-05-08T22:35:36.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.264+0000 7ffb02605700 1 -- 172.21.15.179:0/1267096867 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffafc0123e0 con 0x55c71f9841e0 2024-05-08T22:35:36.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb02605700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffaf0065060 0x7ffaf0067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb02605700 1 -- 172.21.15.179:0/1267096867 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffafc0824a0 con 0x55c71f9841e0 2024-05-08T22:35:36.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb04609700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffaf0065060 0x7ffaf0067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c71f9cc080 0x55c71fa49890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55) v8 -- 0x55c71f9b3200 con 0x55c71f9cc080 2024-05-08T22:35:36.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb0560b700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c71f9cc080 0x55c71fa49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb04609700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffaf0065060 0x7ffaf0067520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000fd0 tx=0x7ffaf8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb0560b700 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c71f9cc080 0x55c71fa49890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb0560b700 1 -- 172.21.15.179:0/1267096867 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7ffaf4019040 con 0x55c71f9cc080 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c71f9cc080 msgr2=0x55c71fa49890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c71f9cc080 0x55c71fa49890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffaf0065060 msgr2=0x7ffaf0067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffaf0065060 0x7ffaf0067520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000fd0 tx=0x7ffaf8009000 comp rx=0 tx=0).stop 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 msgr2=0x55c71fa47e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 --2- 172.21.15.179:0/1267096867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c71f9841e0 0x55c71fa47e30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffafc002230 tx=0x7ffafc00d440 comp rx=0 tx=0).stop 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb0560b700 1 -- 172.21.15.179:0/1267096867 reap_dead start 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 shutdown_connections 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 >> 172.21.15.179:0/1267096867 conn(0x55c71f98a740 msgr2=0x55c71f9b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 shutdown_connections 2024-05-08T22:35:36.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.268+0000 7ffb072d5140 1 -- 172.21.15.179:0/1267096867 wait complete. 2024-05-08T22:35:36.275 INFO:teuthology.orchestra.run.smithi179.stdout:Yalphaparent_a 2024-05-08T22:35:36.276 DEBUG:teuthology.orchestra.run.smithi179:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T22:35:36.332 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "ancestors": [ 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1099511627776, 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "alpha", 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 2 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1, 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "parent_a", 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 4 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:36.333 INFO:teuthology.orchestra.run.smithi179.stdout: ], 2024-05-08T22:35:36.334 INFO:teuthology.orchestra.run.smithi179.stdout: "pool": 8, 2024-05-08T22:35:36.334 INFO:teuthology.orchestra.run.smithi179.stdout: "old_pools": [] 2024-05-08T22:35:36.334 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:36.334 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:36.335 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-08T22:35:36.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 Processor -- start 2024-05-08T22:35:36.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 -- start start 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742010c220 0x55742013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 0x5574201597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742009f2f0 con 0x5574201e3740 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742005b8e0 con 0x55742010c220 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aac015140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55742005b580 con 0x55742013fac0 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aaa34b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aa9349700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 0x5574201597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aa9b4a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742010c220 0x55742013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aaa34b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201e02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60202/0 (socket says 172.21.15.179:60202) 2024-05-08T22:35:36.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.368+0000 7f8aaa34b700 1 -- 172.21.15.179:0/2058579492 learned_addr learned my addr 172.21.15.179:0/2058579492 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 -- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742010c220 msgr2=0x55742013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742010c220 0x55742013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 -- 172.21.15.179:0/2058579492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 msgr2=0x5574201e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 -- 172.21.15.179:0/2058579492 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9400a100 con 0x55742013fac0 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa9349700 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 0x5574201597a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b2f0 tx=0x7f8a9c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43d6d26a89f9a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa3fff700 1 -- 172.21.15.179:0/2058579492 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c014070 con 0x55742013fac0 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa3fff700 1 -- 172.21.15.179:0/2058579492 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00bdc0 con 0x55742013fac0 2024-05-08T22:35:36.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aa3fff700 1 -- 172.21.15.179:0/2058579492 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00f5f0 con 0x55742013fac0 2024-05-08T22:35:36.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 msgr2=0x5574201597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 0x5574201597a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b2f0 tx=0x7f8a9c001e70 comp rx=0 tx=0).stop 2024-05-08T22:35:36.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- 172.21.15.179:0/2058579492 shutdown_connections 2024-05-08T22:35:36.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742013fac0 0x5574201597a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 --2- 172.21.15.179:0/2058579492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742010c220 0x55742013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- 172.21.15.179:0/2058579492 >> 172.21.15.179:0/2058579492 conn(0x557420112780 msgr2=0x55742013b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- 172.21.15.179:0/2058579492 shutdown_connections 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- 172.21.15.179:0/2058579492 wait complete. 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 Processor -- start 2024-05-08T22:35:36.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.372+0000 7f8aac015140 1 -- start start 2024-05-08T22:35:36.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742010c220 0x5574201503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742013fac0 0x5574201e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574200318d0 con 0x5574201e3740 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574201e1730 con 0x55742013fac0 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557420133800 con 0x55742010c220 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60218/0 (socket says 172.21.15.179:60218) 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 -- 172.21.15.179:0/3829212231 learned_addr learned my addr 172.21.15.179:0/3829212231 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aa9349700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742013fac0 0x5574201e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aa9b4a700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742010c220 0x5574201503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 -- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742010c220 msgr2=0x5574201503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55742010c220 0x5574201503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 -- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742013fac0 msgr2=0x5574201e11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55742013fac0 0x5574201e11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 -- 172.21.15.179:0/3829212231 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c001b50 con 0x5574201e3740 2024-05-08T22:35:36.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aaa34b700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201332c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40089f0 tx=0x7f8aa4008d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aa2ffd700 1 -- 172.21.15.179:0/3829212231 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400f040 con 0x5574201e3740 2024-05-08T22:35:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aa2ffd700 1 -- 172.21.15.179:0/3829212231 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40107c0 con 0x5574201e3740 2024-05-08T22:35:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aa2ffd700 1 -- 172.21.15.179:0/3829212231 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4010b70 con 0x5574201e3740 2024-05-08T22:35:36.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557420049b00 con 0x5574201e3740 2024-05-08T22:35:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.376+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55742003cf40 con 0x5574201e3740 2024-05-08T22:35:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa2ffd700 1 -- 172.21.15.179:0/3829212231 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8aa4010cd0 con 0x5574201e3740 2024-05-08T22:35:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa2ffd700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8a8c065060 0x7f8a8c067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa2ffd700 1 -- 172.21.15.179:0/3829212231 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8aa4083790 con 0x5574201e3740 2024-05-08T22:35:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa9b4a700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8a8c065060 0x7f8a8c067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aac015140 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55742013b600 0x557420278bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v8 -- 0x557420279100 con 0x55742013b600 2024-05-08T22:35:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa9349700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55742013b600 0x557420278bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa9b4a700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8a8c065060 0x7f8a8c067520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400a3f0 tx=0x7f8a94007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.380+0000 7f8aa9349700 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55742013b600 0x557420278bc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aa9349700 1 -- 172.21.15.179:0/3829212231 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v55'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8a9c016070 con 0x55742013b600 2024-05-08T22:35:36.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55742013b600 msgr2=0x557420278bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55742013b600 0x557420278bc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8a8c065060 msgr2=0x7f8a8c067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8a8c065060 0x7f8a8c067520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400a3f0 tx=0x7f8a94007030 comp rx=0 tx=0).stop 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 msgr2=0x5574201332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 --2- 172.21.15.179:0/3829212231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574201e3740 0x5574201332c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40089f0 tx=0x7f8aa4008d00 comp rx=0 tx=0).stop 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aaa34b700 1 -- 172.21.15.179:0/3829212231 reap_dead start 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 shutdown_connections 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 >> 172.21.15.179:0/3829212231 conn(0x557420112780 msgr2=0x55742013ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 shutdown_connections 2024-05-08T22:35:36.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.384+0000 7f8aac015140 1 -- 172.21.15.179:0/3829212231 wait complete. 2024-05-08T22:35:36.391 DEBUG:teuthology.orchestra.run.smithi179:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T22:35:36.435 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-08T22:35:36.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.564+0000 7f0ad681e700 1 Processor -- start 2024-05-08T22:35:36.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.564+0000 7f0ad681e700 1 -- start start 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0ad006cab0 0x7f0ad006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0ad00ced10 0x7f0ad006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad006bc80 con 0x7f0ad01598a0 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad0159c80 con 0x7f0ad006cab0 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad006b9a0 con 0x7f0ad00ced10 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0ad006cab0 0x7f0ad006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0ad006cab0 0x7f0ad006d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60224/0 (socket says 172.21.15.179:60224) 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 -- 172.21.15.179:0/3083884408 learned_addr learned my addr 172.21.15.179:0/3083884408 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad601d700 1 --2- 172.21.15.179:0/3083884408 >> v2:172.21.15.195:3300/0 conn(0x7f0ad00ced10 0x7f0ad006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 -- 172.21.15.179:0/3083884408 >> v2:172.21.15.195:3300/0 conn(0x7f0ad00ced10 msgr2=0x7f0ad006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3083884408 >> v2:172.21.15.195:3300/0 conn(0x7f0ad00ced10 0x7f0ad006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 -- 172.21.15.179:0/3083884408 >> v1:172.21.15.195:6789/0 conn(0x7f0ad01598a0 legacy=0x7f0ad006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 -- 172.21.15.179:0/3083884408 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00b1850 con 0x7f0ad006cab0 2024-05-08T22:35:36.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3083884408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad006cab0 0x7f0ad006d710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0ad015d240 tx=0x7f0acc00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269fa95dddd27871 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad481a700 1 -- 172.21.15.179:0/3083884408 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc00dd50 con 0x7f0ad006cab0 2024-05-08T22:35:36.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad481a700 1 -- 172.21.15.179:0/3083884408 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc004030 con 0x7f0ad006cab0 2024-05-08T22:35:36.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad481a700 1 -- 172.21.15.179:0/3083884408 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0043c0 con 0x7f0ad006cab0 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- 172.21.15.179:0/3083884408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad006cab0 msgr2=0x7f0ad006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3083884408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad006cab0 0x7f0ad006d710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0ad015d240 tx=0x7f0acc00be90 comp rx=0 tx=0).stop 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- 172.21.15.179:0/3083884408 shutdown_connections 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3083884408 >> v2:172.21.15.195:3300/0 conn(0x7f0ad00ced10 0x7f0ad006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3083884408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad006cab0 0x7f0ad006d710 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- 172.21.15.179:0/3083884408 >> 172.21.15.179:0/3083884408 conn(0x7f0ad0098a70 msgr2=0x7f0ad00b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- 172.21.15.179:0/3083884408 shutdown_connections 2024-05-08T22:35:36.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- 172.21.15.179:0/3083884408 wait complete. 2024-05-08T22:35:36.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 Processor -- start 2024-05-08T22:35:36.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.568+0000 7f0ad681e700 1 -- start start 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad006cab0 0x7f0ad015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad01598a0 0x7f0ad00cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad015cc70 con 0x7f0ad00ced10 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad014cbf0 con 0x7f0ad01598a0 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad014ced0 con 0x7f0ad006cab0 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60226/0 (socket says 172.21.15.179:60226) 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 -- 172.21.15.179:0/3595395996 learned_addr learned my addr 172.21.15.179:0/3595395996 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad006cab0 0x7f0ad015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad601d700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad01598a0 0x7f0ad00cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 -- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad006cab0 msgr2=0x7f0ad015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad006cab0 0x7f0ad015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 -- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad01598a0 msgr2=0x7f0ad00cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad01598a0 0x7f0ad00cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 -- 172.21.15.179:0/3595395996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc00b810 con 0x7f0ad00ced10 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0070fb0 tx=0x7f0acc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 -- 172.21.15.179:0/3595395996 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc020070 con 0x7f0ad00ced10 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 -- 172.21.15.179:0/3595395996 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc003b00 con 0x7f0ad00ced10 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad014d1b0 con 0x7f0ad00ced10 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 -- 172.21.15.179:0/3595395996 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc003e90 con 0x7f0ad00ced10 2024-05-08T22:35:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad014b1f0 con 0x7f0ad00ced10 2024-05-08T22:35:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 -- 172.21.15.179:0/3595395996 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0acc0193f0 con 0x7f0ad00ced10 2024-05-08T22:35:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 0x7f0abc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ac67fc700 1 -- 172.21.15.179:0/3595395996 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0acc087790 con 0x7f0ad00ced10 2024-05-08T22:35:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.572+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 0x7f0abc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.576+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 0x7f0abc068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009890 tx=0x7f0ac0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 Processor -- start 2024-05-08T22:35:36.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 -- start start 2024-05-08T22:35:36.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 Processor -- start 2024-05-08T22:35:36.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 -- start start 2024-05-08T22:35:36.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x169b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15f9500 con 0x16d26f0 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x15f6c90 con 0x16d3010 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x15f2580 con 0x168d670 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60228/0 (socket says 172.21.15.179:60228) 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 -- 172.21.15.179:0/249688783 learned_addr learned my addr 172.21.15.179:0/249688783 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad601d700 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x169b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad501b700 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 -- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 msgr2=0x169b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 -- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 msgr2=0x169b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x169b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 -- 172.21.15.179:0/249688783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14e4ee0 con 0x16d26f0 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ad581c700 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0014690 tx=0x7f0ac0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ceb67d1d240970 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ab77fe700 1 -- 172.21.15.179:0/249688783 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0014430 con 0x16d26f0 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ab77fe700 1 -- 172.21.15.179:0/249688783 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0014e30 con 0x16d26f0 2024-05-08T22:35:36.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7f0ab77fe700 1 -- 172.21.15.179:0/249688783 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0012830 con 0x16d26f0 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdbf0150bc0 0x7fdbf015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbf00ad740 0x7fdbf00adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf00ae0d0 con 0x7fdbf015d840 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf00b2930 con 0x7fdbf00ad740 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.580+0000 7fdbf61e4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf00b1cb0 con 0x7fdbf0150bc0 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbf00ad740 0x7fdbf00adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbf00ad740 0x7fdbf00adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60240/0 (socket says 172.21.15.179:60240) 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 -- 172.21.15.179:0/1635651763 learned_addr learned my addr 172.21.15.179:0/1635651763 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf51e2700 1 --2- 172.21.15.179:0/1635651763 >> v2:172.21.15.195:3300/0 conn(0x7fdbf0150bc0 0x7fdbf015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/249688783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 msgr2=0x16d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0014690 tx=0x7f0ac0010000 comp rx=0 tx=0).stop 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/249688783 shutdown_connections 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x169b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- 172.21.15.179:0/249688783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d2ad0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/249688783 >> 172.21.15.179:0/249688783 conn(0x1419780 msgr2=0x16b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 -- 172.21.15.179:0/1635651763 >> v2:172.21.15.195:3300/0 conn(0x7fdbf0150bc0 msgr2=0x7fdbf015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 --2- 172.21.15.179:0/1635651763 >> v2:172.21.15.195:3300/0 conn(0x7fdbf0150bc0 0x7fdbf015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 -- 172.21.15.179:0/1635651763 >> v1:172.21.15.195:6789/0 conn(0x7fdbf015d840 legacy=0x7fdbf015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 -- 172.21.15.179:0/1635651763 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00b1f90 con 0x7fdbf00ad740 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/249688783 shutdown_connections 2024-05-08T22:35:36.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/249688783 wait complete. 2024-05-08T22:35:36.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf49e1700 1 --2- 172.21.15.179:0/1635651763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf00ad740 0x7fdbf00adb90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009910 tx=0x7fdbe0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b26fa620b664ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbe7fff700 1 -- 172.21.15.179:0/1635651763 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0011830 con 0x7fdbf00ad740 2024-05-08T22:35:36.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbe7fff700 1 -- 172.21.15.179:0/1635651763 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0011990 con 0x7fdbf00ad740 2024-05-08T22:35:36.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbe7fff700 1 -- 172.21.15.179:0/1635651763 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0011d40 con 0x7fdbf00ad740 2024-05-08T22:35:36.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- start start 2024-05-08T22:35:36.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 Processor -- start 2024-05-08T22:35:36.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 -- 172.21.15.179:0/1635651763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf00ad740 msgr2=0x7fdbf00adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 --2- 172.21.15.179:0/1635651763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf00ad740 0x7fdbf00adb90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009910 tx=0x7fdbe0009c20 comp rx=0 tx=0).stop 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 -- 172.21.15.179:0/1635651763 shutdown_connections 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 --2- 172.21.15.179:0/1635651763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf00ad740 0x7fdbf00adb90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 --2- 172.21.15.179:0/1635651763 >> v2:172.21.15.195:3300/0 conn(0x7fdbf0150bc0 0x7fdbf015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 -- 172.21.15.179:0/1635651763 >> 172.21.15.179:0/1635651763 conn(0x7fdbf00ce960 msgr2=0x7fdbf00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 -- 172.21.15.179:0/1635651763 shutdown_connections 2024-05-08T22:35:36.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7fdbf61e4700 1 -- 172.21.15.179:0/1635651763 wait complete. 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x16b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x168cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b5920 con 0x16d26f0 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b55c0 con 0x16d3010 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1587e10 con 0x168d670 2024-05-08T22:35:36.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 10 client.0 _ll_register_callbacks cb 0x15ebbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60254/0 (socket says 172.21.15.179:60254) 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 -- 172.21.15.179:0/1704176040 learned_addr learned my addr 172.21.15.179:0/1704176040 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad601d700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x168cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad581c700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x16b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 -- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 msgr2=0x16b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x16b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 -- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 msgr2=0x168cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x168cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 -- 172.21.15.179:0/1704176040 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac000e040 con 0x16d26f0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ad501b700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0586a0 tx=0x7f0acc058f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98b328e5d1f5acc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc05c320 con 0x16d26f0 2024-05-08T22:35:36.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0acc060b00 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1426490 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0acc057c00 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 10 client.6240 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.584+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x15b8600 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1426490 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0acc060670 con 0x16d26f0 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 10 client.6240 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 10 client.6240 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 20 client.6240 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x168bf40 0x16e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 4 client.6240 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1470710 con 0x168bf40 2024-05-08T22:35:36.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ad581c700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x168bf40 0x16e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 Processor -- start 2024-05-08T22:35:36.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- start start 2024-05-08T22:35:36.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 10 client.6240 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ad581c700 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x168bf40 0x16e6260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f63f7b6faaa0ad0 server_cookie=4dcf274bfefe9661 in_seq=0 out_seq=0 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbf00ad740 0x7fdbf01381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf0150bc0 0x7fdbf015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf015d840 0x7fdbf0148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf015cda0 con 0x7fdbf015d840 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0138730 con 0x7fdbf0150bc0 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf015c120 con 0x7fdbf00ad740 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf015d840 0x7fdbf0148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf015d840 0x7fdbf0148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60268/0 (socket says 172.21.15.179:60268) 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 -- 172.21.15.179:0/195961913 learned_addr learned my addr 172.21.15.179:0/195961913 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf51e2700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbf00ad740 0x7fdbf01381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf49e1700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf0150bc0 0x7fdbf015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 -- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbf00ad740 msgr2=0x7fdbf01381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbf00ad740 0x7fdbf01381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 -- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf0150bc0 msgr2=0x7fdbf015b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf0150bc0 0x7fdbf015b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 -- 172.21.15.179:0/195961913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009570 con 0x7fdbf015d840 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf49e1700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf0150bc0 0x7fdbf015b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf59e3700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbf015d840 0x7fdbf0148bb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c430 tx=0x7fdbec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/1704176040 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1470710 con 0x168bf40 2024-05-08T22:35:36.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 10 client.6240 handle_command_reply: tid=0 2024-05-08T22:35:36.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0052d0 con 0x7fdbf015d840 2024-05-08T22:35:36.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 client.6240 shutdown 2024-05-08T22:35:36.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec005430 con 0x7fdbf015d840 2024-05-08T22:35:36.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- 172.21.15.179:0/195961913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf015c400 con 0x7fdbf015d840 2024-05-08T22:35:36.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0055a0 con 0x7fdbf015d840 2024-05-08T22:35:36.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7fdbf61e4700 1 -- 172.21.15.179:0/195961913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf015c9c0 con 0x7fdbf015d840 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 msgr2=0x1686540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0586a0 tx=0x7f0acc058f40 comp rx=0 tx=0).stop 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 shutdown_connections 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x168bf40 0x16e6260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x168cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x1686540 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 --2- 172.21.15.179:0/1704176040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x16b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 >> 172.21.15.179:0/1704176040 conn(0x1419780 msgr2=0x16b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0ab4ff9700 0 client.6240 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 shutdown_connections 2024-05-08T22:35:36.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 1 -- 172.21.15.179:0/1704176040 wait complete. 2024-05-08T22:35:36.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.588+0000 7f0adad2f740 20 client.6240 trim_cache size 0 max 16384 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbf61e4700 1 -- 172.21.15.179:0/195961913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdbf015cee0 con 0x7fdbf015d840 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbec0038e0 con 0x7fdbf015d840 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbe5ffb700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdbec083850 con 0x7fdbf015d840 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7fdbec0842d0 con 0x7fdbf015d840 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbf51e2700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbf61e4700 1 -- 172.21.15.179:0/195961913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fdbf00ae0d0 con 0x7fdbf015d840 2024-05-08T22:35:36.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.592+0000 7fdbf51e2700 1 --2- 172.21.15.179:0/195961913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80074a0 tx=0x7fdbe800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 Processor -- start 2024-05-08T22:35:36.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- start start 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ac010 con 0x16d26f0 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x15f2e30 con 0x16d3010 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x15b6070 con 0x168d670 2024-05-08T22:35:36.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60284/0 (socket says 172.21.15.179:60284) 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 -- 172.21.15.179:0/3107786809 learned_addr learned my addr 172.21.15.179:0/3107786809 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad601d700 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 -- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 msgr2=0x169d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 -- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 msgr2=0x16e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 -- 172.21.15.179:0/3107786809 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x159e830 con 0x16d26f0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b360 tx=0x7f0acc026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caf6f5710b054df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ab7fff700 1 -- 172.21.15.179:0/3107786809 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0584e0 con 0x16d26f0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ab7fff700 1 -- 172.21.15.179:0/3107786809 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc057ea0 con 0x16d26f0 2024-05-08T22:35:36.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0ab7fff700 1 -- 172.21.15.179:0/3107786809 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc058250 con 0x16d26f0 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- 172.21.15.179:0/3107786809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 msgr2=0x16e2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b360 tx=0x7f0acc026000 comp rx=0 tx=0).stop 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 -- 172.21.15.179:0/3107786809 shutdown_connections 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.608+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16e2510 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3107786809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x169d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- 172.21.15.179:0/3107786809 >> 172.21.15.179:0/3107786809 conn(0x1419780 msgr2=0x16e1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- 172.21.15.179:0/3107786809 shutdown_connections 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- 172.21.15.179:0/3107786809 wait complete. 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- start start 2024-05-08T22:35:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 Processor -- start 2024-05-08T22:35:36.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x1733400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x16eae00 con 0x16d26f0 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b4710 con 0x16d3010 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1733940 con 0x168d670 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 10 client.0 _ll_register_callbacks cb 0x15f1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60300/0 (socket says 172.21.15.179:60300) 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 -- 172.21.15.179:0/3852240689 learned_addr learned my addr 172.21.15.179:0/3852240689 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad601d700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x1733400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 -- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 msgr2=0x1733400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x1733400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 -- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 msgr2=0x16e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 -- 172.21.15.179:0/3852240689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15b8600 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x1733400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad601d700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ad501b700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0586a0 tx=0x7f0acc085ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2bc4389e104414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0584e0 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0acc060b00 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.612+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ac0014e30 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0acc058250 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 10 client.6249 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x1464c10 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0014e30 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0acc07fb10 con 0x16d26f0 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 10 client.6249 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:36.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:35:36.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 20 client.6249 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:36.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x16e27a0 0x1734460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 4 client.6249 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T22:35:36.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x14975f0 con 0x16e27a0 2024-05-08T22:35:36.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x16e27a0 0x1734460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.621 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:35:36.619+0000 7f5d3499a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6142=0-4194304@1} caps={6142=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6142 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560385257700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:35:36.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ad581c700 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x16e27a0 0x1734460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c09e4f8cb091065 server_cookie=7facfd546da54322 in_seq=0 out_seq=0 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 10 client.6249 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 1 -- 172.21.15.179:0/3852240689 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x14975f0 con 0x16e27a0 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 10 client.6249 handle_command_reply: tid=0 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 client.6249 shutdown 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 msgr2=0x16d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0586a0 tx=0x7f0acc085ae0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 shutdown_connections 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x16e27a0 0x1734460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16d3010 0x16e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0ab4ff9700 0 client.6249 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16d26f0 0x16d1150 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 --2- 172.21.15.179:0/3852240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x168d670 0x1733400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 >> 172.21.15.179:0/3852240689 conn(0x1419780 msgr2=0x169be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 shutdown_connections 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 1 -- 172.21.15.179:0/3852240689 wait complete. 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.616+0000 7f0adad2f740 20 client.6249 trim_cache size 0 max 16384 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "f1af6218-3664-46d7-b32e-80a22f3e3aba", 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:35:36.624 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 msgr2=0x7f0abc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 0x7f0abc068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009890 tx=0x7f0ac0008040 comp rx=0 tx=0).stop 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 msgr2=0x7f0ad014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0070fb0 tx=0x7f0acc004770 comp rx=0 tx=0).stop 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 shutdown_connections 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0abc066730 0x7f0abc068bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ad01598a0 0x7f0ad00cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ad00ced10 0x7f0ad014acb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 --2- 172.21.15.179:0/3595395996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ad006cab0 0x7f0ad015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 >> 172.21.15.179:0/3595395996 conn(0x7f0ad0098a70 msgr2=0x7f0ad006adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 shutdown_connections 2024-05-08T22:35:36.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.620+0000 7f0ad681e700 1 -- 172.21.15.179:0/3595395996 wait complete. 2024-05-08T22:35:36.638 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-08T22:35:36.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 Processor -- start 2024-05-08T22:35:36.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 -- start start 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5f4015b490 0x7f5f400afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f4006abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f400b5460 con 0x7f5f4015b0b0 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f400b0510 con 0x7f5f4015b490 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400b4170 con 0x7f5f400b0080 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3effd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5f4015b0b0 0x7f5f400ae870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60524/0 (socket says 172.21.15.179:60524) 2024-05-08T22:35:36.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3effd700 1 -- 172.21.15.179:0/275702981 learned_addr learned my addr 172.21.15.179:0/275702981 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f4006abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/275702981 >> v2:172.21.15.195:3300/0 conn(0x7f5f4015b490 0x7f5f400afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770569209 0 0) 0x7f5f30001490 con 0x7f5f4015b0b0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2c0035d0 con 0x7f5f4015b0b0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698427987 0 0) 0x7f5f2c0035d0 con 0x7f5f4015b0b0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 msgr2=0x7f5f4006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f4006abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 >> v2:172.21.15.195:3300/0 conn(0x7f5f4015b490 msgr2=0x7f5f400afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 --2- 172.21.15.179:0/275702981 >> v2:172.21.15.195:3300/0 conn(0x7f5f4015b490 0x7f5f400afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400b43c0 con 0x7f5f4015b0b0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5f30001b80 con 0x7f5f4015b0b0 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 >> v1:172.21.15.179:6789/0 conn(0x7f5f4015b0b0 legacy=0x7f5f400ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f2c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f2c003fa0 0x7f5f2c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f2c004db0 0x7f5f2c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400b0510 con 0x7f5f2c004db0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400b4170 con 0x7f5f2c003fa0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.752+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c00ba30 con 0x7f5f400b0080 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f2c004db0 0x7f5f2c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3effd700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f2c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f2c003fa0 0x7f5f2c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 msgr2=0x7f5f2c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f2c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f2c003fa0 msgr2=0x7f5f2c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f2c003fa0 0x7f5f2c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/275702981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0035d0 con 0x7f5f2c004db0 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3effd700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f400b0080 0x7f5f2c003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:36.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f2c004db0 0x7f5f2c005230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009b70 tx=0x7f5f2c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3b588cfd798668 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800e660 con 0x7f5f2c004db0 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800e7c0 con 0x7f5f2c004db0 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3dffb700 1 -- 172.21.15.179:0/275702981 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800eb30 con 0x7f5f2c004db0 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- 172.21.15.179:0/275702981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f2c004db0 msgr2=0x7f5f2c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/275702981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f2c004db0 0x7f5f2c005230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009b70 tx=0x7f5f2c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/275702981 reap_dead start 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- 172.21.15.179:0/275702981 shutdown_connections 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- 172.21.15.179:0/275702981 >> 172.21.15.179:0/275702981 conn(0x7f5f40098a70 msgr2=0x7f5f400b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- 172.21.15.179:0/275702981 shutdown_connections 2024-05-08T22:35:36.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- 172.21.15.179:0/275702981 wait complete. 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 Processor -- start 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.756+0000 7f5f3ffff700 1 -- start start 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 0x7f5f40155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f40154280 0x7f5f40156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40156f80 con 0x7f5f40156ad0 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400ac720 con 0x7f5f40143db0 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40155370 con 0x7f5f40154280 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 0x7f5f40155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 0x7f5f40155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:42626/0 (socket says 172.21.15.179:42626) 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60310/0 (socket says 172.21.15.179:60310) 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3effd700 1 -- 172.21.15.179:0/2598802285 learned_addr learned my addr 172.21.15.179:0/2598802285 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f40154280 0x7f5f40156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f40154280 msgr2=0x7f5f40156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f40154280 0x7f5f40156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 msgr2=0x7f5f40155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 0x7f5f40155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/2598802285 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40154760 con 0x7f5f40156ad0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009b40 tx=0x7f5f380047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f377fe700 1 -- 172.21.15.179:0/2598802285 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800e660 con 0x7f5f40156ad0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f40154a40 con 0x7f5f40156ad0 2024-05-08T22:35:36.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f377fe700 1 -- 172.21.15.179:0/2598802285 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380038e0 con 0x7f5f40156ad0 2024-05-08T22:35:36.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f377fe700 1 -- 172.21.15.179:0/2598802285 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38003c90 con 0x7f5f40156ad0 2024-05-08T22:35:36.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.760+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40155000 con 0x7f5f40156ad0 2024-05-08T22:35:36.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.764+0000 7f5f377fe700 1 -- 172.21.15.179:0/2598802285 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f38013470 con 0x7f5f40156ad0 2024-05-08T22:35:36.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.764+0000 7f5f377fe700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 0x7f5f20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.764+0000 7f5f377fe700 1 -- 172.21.15.179:0/2598802285 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f38085290 con 0x7f5f40156ad0 2024-05-08T22:35:36.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.764+0000 7f5f3effd700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 0x7f5f20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.764+0000 7f5f3effd700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 0x7f5f20068c20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f30002230 tx=0x7f5f30001940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 Processor -- start 2024-05-08T22:35:36.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 -- start start 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19aa6f0 0x1980c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x1981810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1913010 con 0x1980840 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a0f40 con 0x19ad5d0 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18f3e30 con 0x19aa6f0 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60314/0 (socket says 172.21.15.179:60314) 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 -- 172.21.15.179:0/680359646 learned_addr learned my addr 172.21.15.179:0/680359646 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19aa6f0 0x1980c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x1981810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 -- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19aa6f0 msgr2=0x1980c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19aa6f0 0x1980c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 -- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 msgr2=0x1981810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x1981810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 -- 172.21.15.179:0/680359646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x175fa40 con 0x1980840 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f3effd700 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5f300175b0 tx=0x7f5f30014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec28668c702b4e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f34ff9700 1 -- 172.21.15.179:0/680359646 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30017e00 con 0x1980840 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f34ff9700 1 -- 172.21.15.179:0/680359646 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3001b070 con 0x1980840 2024-05-08T22:35:36.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.768+0000 7f5f34ff9700 1 -- 172.21.15.179:0/680359646 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30015840 con 0x1980840 2024-05-08T22:35:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/680359646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 msgr2=0x1985d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5f300175b0 tx=0x7f5f30014000 comp rx=0 tx=0).stop 2024-05-08T22:35:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/680359646 shutdown_connections 2024-05-08T22:35:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x1981810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19aa6f0 0x1980c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- 172.21.15.179:0/680359646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1980840 0x1985d90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/680359646 >> 172.21.15.179:0/680359646 conn(0x17145b0 msgr2=0x19acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/680359646 shutdown_connections 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/680359646 wait complete. 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- start start 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 Processor -- start 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1980840 0x19d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x19af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18ef150 con 0x19aa6f0 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18eceb0 con 0x19ad5d0 2024-05-08T22:35:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18aa8e0 con 0x1980840 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 10 client.0 _ll_register_callbacks cb 0x1889670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60330/0 (socket says 172.21.15.179:60330) 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 -- 172.21.15.179:0/3798304651 learned_addr learned my addr 172.21.15.179:0/3798304651 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3effd700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1980840 0x19d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x19af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 -- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1980840 msgr2=0x19d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1980840 0x19d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 -- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 msgr2=0x19af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x19af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 -- 172.21.15.179:0/3798304651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3000f040 con 0x19aa6f0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f28003d10 tx=0x7f5f28009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3c4a474373e409 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:36.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28003eb0 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f28016640 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17212c0 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f28016820 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 10 client.6261 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x18b3290 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17212c0 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5f28016980 con 0x19aa6f0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 10 client.6261 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 20 client.6261 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.772+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19841d0 0x19dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 4 client.6261 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x176b540 con 0x19841d0 2024-05-08T22:35:36.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f3effd700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19841d0 0x19dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f1dffb700 10 client.6261 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f3effd700 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19841d0 0x19dfd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f599a4a41dad5177 server_cookie=cd832675849aa1c2 in_seq=0 out_seq=0 2024-05-08T22:35:36.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f1dffb700 1 -- 172.21.15.179:0/3798304651 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x176b540 con 0x19841d0 2024-05-08T22:35:36.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f1dffb700 10 client.6261 handle_command_reply: tid=0 2024-05-08T22:35:36.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 client.6261 shutdown 2024-05-08T22:35:36.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 msgr2=0x19aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f28003d10 tx=0x7f5f28009960 comp rx=0 tx=0).stop 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 shutdown_connections 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19841d0 0x19dfd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ad5d0 0x19af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f1dffb700 0 client.6261 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19aefe0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 --2- 172.21.15.179:0/3798304651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1980840 0x19d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 >> 172.21.15.179:0/3798304651 conn(0x17145b0 msgr2=0x19accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 shutdown_connections 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 1 -- 172.21.15.179:0/3798304651 wait complete. 2024-05-08T22:35:36.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.776+0000 7f5f4866a740 20 client.6261 trim_cache size 0 max 16384 2024-05-08T22:35:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 Processor -- start 2024-05-08T22:35:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 -- start start 2024-05-08T22:35:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x197ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1798b80 con 0x19aa6f0 2024-05-08T22:35:36.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18bdc00 con 0x1980840 2024-05-08T22:35:36.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x19afc50 con 0x19841d0 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60346/0 (socket says 172.21.15.179:60346) 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/600583337 learned_addr learned my addr 172.21.15.179:0/600583337 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3effd700 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x197ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 msgr2=0x19a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 msgr2=0x197ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.796+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x197ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/600583337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19d5500 con 0x19aa6f0 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f380561a0 tx=0x7f5f3805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb705b448d5d47e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f357fa700 1 -- 172.21.15.179:0/600583337 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38056910 con 0x19aa6f0 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f357fa700 1 -- 172.21.15.179:0/600583337 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38056210 con 0x19aa6f0 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f357fa700 1 -- 172.21.15.179:0/600583337 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3805a040 con 0x19aa6f0 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- 172.21.15.179:0/600583337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 msgr2=0x19a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f380561a0 tx=0x7f5f3805c040 comp rx=0 tx=0).stop 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- 172.21.15.179:0/600583337 shutdown_connections 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19a8c50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- 172.21.15.179:0/600583337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x197ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- 172.21.15.179:0/600583337 >> 172.21.15.179:0/600583337 conn(0x17145b0 msgr2=0x19e3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- 172.21.15.179:0/600583337 shutdown_connections 2024-05-08T22:35:36.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- 172.21.15.179:0/600583337 wait complete. 2024-05-08T22:35:36.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- start start 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 Processor -- start 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x1a34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1914610 con 0x19aa6f0 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x188b620 con 0x1980840 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x19d4a50 con 0x19841d0 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f4866a740 10 client.0 _ll_register_callbacks cb 0x18a6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60356/0 (socket says 172.21.15.179:60356) 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/1126942281 learned_addr learned my addr 172.21.15.179:0/1126942281 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:35:36.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3effd700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x1a34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3e7fc700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 msgr2=0x19d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 msgr2=0x1a34c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x1a34c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.800+0000 7f5f3f7fe700 1 -- 172.21.15.179:0/1126942281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40155000 con 0x19aa6f0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f3f7fe700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009480 tx=0x7f5f38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f0d4ec0095ba1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38056910 con 0x19aa6f0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f3805e600 con 0x19aa6f0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f3001b070 con 0x19aa6f0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f38004580 con 0x19aa6f0 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 10 client.6267 fetch_fsmap learned FSMap version 24 2024-05-08T22:35:36.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x18994c0 con 0x19aa6f0 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3001b070 con 0x19aa6f0 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5f38083120 con 0x19aa6f0 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 10 client.6267 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 10 client.6267 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 20 client.6267 populate_metadata read hostname 'smithi179' 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19ad5d0 0x19ed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 4 client.6267 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x17902c0 con 0x19ad5d0 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f3effd700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19ad5d0 0x19ed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 10 client.6267 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f3effd700 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19ad5d0 0x19ed820 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33f94d9798fac23 server_cookie=9ee0afb466de9630 in_seq=0 out_seq=0 2024-05-08T22:35:36.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 1 -- 172.21.15.179:0/1126942281 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x17902c0 con 0x19ad5d0 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 10 client.6267 handle_command_reply: tid=0 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 client.6267 shutdown 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 msgr2=0x19d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009480 tx=0x7f5f38000bc0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 shutdown_connections 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x19ad5d0 0x19ed820 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19aa6f0 0x19d3e50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19841d0 0x19d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 --2- 172.21.15.179:0/1126942281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1980840 0x1a34c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f1dffb700 0 client.6267 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 >> 172.21.15.179:0/1126942281 conn(0x17145b0 msgr2=0x19acae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 shutdown_connections 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 1 -- 172.21.15.179:0/1126942281 wait complete. 2024-05-08T22:35:36.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.804+0000 7f5f4866a740 20 client.6267 trim_cache size 0 max 16384 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "f1af6218-3664-46d7-b32e-80a22f3e3aba": { 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/", 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "f1af6218-3664-46d7-b32e-80a22f3e3aba", 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:35:36.813 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:35:36.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 msgr2=0x7f5f20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 0x7f5f20068c20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f30002230 tx=0x7f5f30001940 comp rx=0 tx=0).stop 2024-05-08T22:35:36.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 msgr2=0x7f5f400acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:36.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009b40 tx=0x7f5f380047e0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 shutdown_connections 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f20066760 0x7f5f20068c20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f40156ad0 0x7f5f400acf70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f40154280 0x7f5f40156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 --2- 172.21.15.179:0/2598802285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f40143db0 0x7f5f40155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 >> 172.21.15.179:0/2598802285 conn(0x7f5f40098a70 msgr2=0x7f5f4006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 shutdown_connections 2024-05-08T22:35:36.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:36.808+0000 7f5f3ffff700 1 -- 172.21.15.179:0/2598802285 wait complete. 2024-05-08T22:35:36.825 INFO:tasks.cephfs.filesystem:scrub status for tag:f1af6218-3664-46d7-b32e-80a22f3e3aba - {'path': '/', 'tag': 'f1af6218-3664-46d7-b32e-80a22f3e3aba', 'options': 'recursive,repair'} 2024-05-08T22:35:37.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:37.484+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-08T22:35:36.620146+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7fdbec054d30 con 0x7fdbf015d840 2024-05-08T22:35:37.488 INFO:teuthology.orchestra.run.smithi179.stdout: cluster: 2024-05-08T22:35:37.488 INFO:teuthology.orchestra.run.smithi179.stdout: id: 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:35:37.488 INFO:teuthology.orchestra.run.smithi179.stdout: health: HEALTH_OK 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: services: 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: mgr: x(active, since 2m), standbys: y 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: data: 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: volumes: 1/1 healthy 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: pools: 3 pools, 97 pgs 2024-05-08T22:35:37.489 INFO:teuthology.orchestra.run.smithi179.stdout: objects: 24 objects, 579 KiB 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: pgs: 97 active+clean 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: io: 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: client: 1.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.620146+0000 mds.d [INF] scrub queued for path: / 2024-05-08T22:35:37.490 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.620153+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:35:37.491 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.620167+0000 mds.d [INF] scrub summary: active paths [/] 2024-05-08T22:35:37.491 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.621942+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-08T22:35:37.491 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.621958+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-08T22:35:37.491 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:36.622825+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:35:39.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:35:39.500+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 8 ==== log(1 entries from seq 7 at 2024-05-08T22:35:37.813435+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fdbec054630 con 0x7fdbf015d840 2024-05-08T22:35:39.505 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:35:37.813435+0000 mds.d [INF] scrub summary: idle 2024-05-08T22:35:51.825 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:35:51.828 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:36:04.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:04.571+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 9 ==== log(4 entries from seq 111 at 2024-05-08T22:36:04.496302+0000) v1 ==== 1112+0+0 (secure 0 0 0) 0x7fdbec058460 con 0x7fdbf015d840 2024-05-08T22:36:05.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:05.579+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/195961913 <== mon.0 v2:172.21.15.179:3300/0 10 ==== log(2 entries from seq 517 at 2024-05-08T22:36:04.570116+0000) v1 ==== 558+0+0 (secure 0 0 0) 0x7fdbec05cab0 con 0x7fdbf015d840 2024-05-08T22:36:06.827 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-08T22:36:06.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 Processor -- start 2024-05-08T22:36:06.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- start start 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47806c2d0 0x7fc4780b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc47806e6b0 0x7fc47806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc478159b00 con 0x7fc478157040 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47806f040 con 0x7fc47806e6b0 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc478159d80 con 0x7fc47806c2d0 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47d6d9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc478157040 0x7fc47806bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38792/0 (socket says 172.21.15.179:38792) 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47d6d9700 1 -- 172.21.15.179:0/2138514473 learned_addr learned my addr 172.21.15.179:0/2138514473 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 --2- 172.21.15.179:0/2138514473 >> v2:172.21.15.179:3300/0 conn(0x7fc47806e6b0 0x7fc47806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47ced8700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47806c2d0 0x7fc4780b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 -- 172.21.15.179:0/2138514473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47806c2d0 msgr2=0x7fc4780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:06.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47806c2d0 0x7fc4780b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 -- 172.21.15.179:0/2138514473 >> v1:172.21.15.179:6789/0 conn(0x7fc478157040 legacy=0x7fc47806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 -- 172.21.15.179:0/2138514473 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47815a060 con 0x7fc47806e6b0 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc477fff700 1 -- 172.21.15.179:0/2138514473 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265703539 0 0) 0x7fc478159b00 con 0x7fc478157040 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47deda700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47806eb00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc468009850 tx=0x7fc468009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828216c1cb41e9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc477fff700 1 -- 172.21.15.179:0/2138514473 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468011830 con 0x7fc47806e6b0 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc477fff700 1 -- 172.21.15.179:0/2138514473 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468011990 con 0x7fc47806e6b0 2024-05-08T22:36:06.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc477fff700 1 -- 172.21.15.179:0/2138514473 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468011d40 con 0x7fc47806e6b0 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- 172.21.15.179:0/2138514473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 msgr2=0x7fc47806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47806eb00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc468009850 tx=0x7fc468009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- 172.21.15.179:0/2138514473 shutdown_connections 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47806eb00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 --2- 172.21.15.179:0/2138514473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47806c2d0 0x7fc4780b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- 172.21.15.179:0/2138514473 >> 172.21.15.179:0/2138514473 conn(0x7fc478070d80 msgr2=0x7fc478071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:06.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- 172.21.15.179:0/2138514473 shutdown_connections 2024-05-08T22:36:06.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.955+0000 7fc47e6db700 1 -- 172.21.15.179:0/2138514473 wait complete. 2024-05-08T22:36:06.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 Processor -- start 2024-05-08T22:36:06.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- start start 2024-05-08T22:36:06.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47806c2d0 0x7fc47814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc478157040 0x7fc47806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47814abe0 con 0x7fc47806e6b0 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc478147920 con 0x7fc47806c2d0 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc478147c00 con 0x7fc478157040 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34016/0 (socket says 172.21.15.179:34016) 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 -- 172.21.15.179:0/1742537336 learned_addr learned my addr 172.21.15.179:0/1742537336 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47806c2d0 0x7fc47814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47deda700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc478157040 0x7fc47806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 -- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc478157040 msgr2=0x7fc47806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc478157040 0x7fc47806b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 -- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47806c2d0 msgr2=0x7fc47814fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47806c2d0 0x7fc47814fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 -- 172.21.15.179:0/1742537336 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680094b0 con 0x7fc47806e6b0 2024-05-08T22:36:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47ced8700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e9d0 tx=0x7fc46c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc475ffb700 1 -- 172.21.15.179:0/1742537336 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c009720 con 0x7fc47806e6b0 2024-05-08T22:36:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc475ffb700 1 -- 172.21.15.179:0/1742537336 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00ee70 con 0x7fc47806e6b0 2024-05-08T22:36:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc478147ee0 con 0x7fc47806e6b0 2024-05-08T22:36:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc475ffb700 1 -- 172.21.15.179:0/1742537336 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00ca50 con 0x7fc47806e6b0 2024-05-08T22:36:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4781451c0 con 0x7fc47806e6b0 2024-05-08T22:36:06.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc475ffb700 1 -- 172.21.15.179:0/1742537336 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc46c00cbb0 con 0x7fc47806e6b0 2024-05-08T22:36:06.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.963+0000 7fc475ffb700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 0x7fc464068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc475ffb700 1 -- 172.21.15.179:0/1742537336 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc46c084240 con 0x7fc47806e6b0 2024-05-08T22:36:06.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 0x7fc464068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 0x7fc464068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4700012e0 tx=0x7fc470001aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 Processor -- start 2024-05-08T22:36:06.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 -- start start 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e28840 0x1e2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e555d0 0x1e29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dbb010 con 0x1e526f0 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d48f40 con 0x1e28840 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d9be30 con 0x1e555d0 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34024/0 (socket says 172.21.15.179:34024) 2024-05-08T22:36:06.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 -- 172.21.15.179:0/4169455872 learned_addr learned my addr 172.21.15.179:0/4169455872 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47deda700 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e555d0 0x1e29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e28840 0x1e2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 -- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e555d0 msgr2=0x1e29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e555d0 0x1e29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 -- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e28840 msgr2=0x1e2dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e28840 0x1e2dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.967+0000 7fc47ced8700 1 -- 172.21.15.179:0/4169455872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c07a40 con 0x1e526f0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc47ced8700 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc46c082680 tx=0x7fc46c054c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc6aebdbadce0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc45effd700 1 -- 172.21.15.179:0/4169455872 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c058fb0 con 0x1e526f0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc45effd700 1 -- 172.21.15.179:0/4169455872 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c05d570 con 0x1e526f0 2024-05-08T22:36:06.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc45effd700 1 -- 172.21.15.179:0/4169455872 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c05b0c0 con 0x1e526f0 2024-05-08T22:36:06.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- 172.21.15.179:0/4169455872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 msgr2=0x1e28c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc46c082680 tx=0x7fc46c054c10 comp rx=0 tx=0).stop 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- 172.21.15.179:0/4169455872 shutdown_connections 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e555d0 0x1e29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e526f0 0x1e28c20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 --2- 172.21.15.179:0/4169455872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e28840 0x1e2dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- 172.21.15.179:0/4169455872 >> 172.21.15.179:0/4169455872 conn(0x1bbc5b0 msgr2=0x1e54ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- 172.21.15.179:0/4169455872 shutdown_connections 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- 172.21.15.179:0/4169455872 wait complete. 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 -- start start 2024-05-08T22:36:06.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.971+0000 7fc482bec740 1 Processor -- start 2024-05-08T22:36:06.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e50d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 0x1e51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d97150 con 0x1e28840 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d94eb0 con 0x1e555d0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d528e0 con 0x1e526f0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 10 client.0 _ll_register_callbacks cb 0x1d31670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34028/0 (socket says 172.21.15.179:34028) 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 -- 172.21.15.179:0/657813098 learned_addr learned my addr 172.21.15.179:0/657813098 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47ced8700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e50d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47deda700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 0x1e51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 -- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 msgr2=0x1e50d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e50d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 -- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 msgr2=0x1e51270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 0x1e51270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 -- 172.21.15.179:0/657813098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4781451c0 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47deda700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 0x1e51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc470012380 tx=0x7fc470001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d1a38257457097 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470012b00 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc47001b070 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1bc92c0 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc470012c60 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 10 client.6282 fetch_fsmap learned FSMap version 24 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x1d5b290 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1bc92c0 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc470021410 con 0x1e28840 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 10 client.6282 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:36:06.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 10 client.6282 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 20 client.6282 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e3c8e0 0x1e2be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 4 client.6282 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c13540 con 0x1e3c8e0 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47ced8700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e3c8e0 0x1e2be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc457fff700 10 client.6282 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:36:06.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.975+0000 7fc47ced8700 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e3c8e0 0x1e2be70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b500292d6725529 server_cookie=f7191281c1bbaf2d in_seq=0 out_seq=0 2024-05-08T22:36:06.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc457fff700 1 -- 172.21.15.179:0/657813098 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1c13540 con 0x1e3c8e0 2024-05-08T22:36:06.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc457fff700 10 client.6282 handle_command_reply: tid=0 2024-05-08T22:36:06.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 client.6282 shutdown 2024-05-08T22:36:06.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 msgr2=0x1e3d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc470012380 tx=0x7fc470001250 comp rx=0 tx=0).stop 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 shutdown_connections 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e3c8e0 0x1e2be70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e555d0 0x1e51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc457fff700 0 client.6282 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 --2- 172.21.15.179:0/657813098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e3d840 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 >> 172.21.15.179:0/657813098 conn(0x1bbc5b0 msgr2=0x1e26db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 shutdown_connections 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 1 -- 172.21.15.179:0/657813098 wait complete. 2024-05-08T22:36:06.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.979+0000 7fc482bec740 20 client.6282 trim_cache size 0 max 16384 2024-05-08T22:36:07.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 Processor -- start 2024-05-08T22:36:07.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 -- start start 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e58100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e58640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c40b80 con 0x1e28840 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d39bd0 con 0x1e3c8e0 2024-05-08T22:36:07.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d95760 con 0x1e526f0 2024-05-08T22:36:07.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34030/0 (socket says 172.21.15.179:34030) 2024-05-08T22:36:07.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 -- 172.21.15.179:0/2844924135 learned_addr learned my addr 172.21.15.179:0/2844924135 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47deda700 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e58640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47ced8700 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e58100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 -- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 msgr2=0x1e58640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e58640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 -- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 msgr2=0x1e58100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e58100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 -- 172.21.15.179:0/2844924135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d414c0 con 0x1e28840 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc47002c040 tx=0x7fc470021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c891fe962f7db795 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc45f7fe700 1 -- 172.21.15.179:0/2844924135 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470037070 con 0x1e28840 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc45f7fe700 1 -- 172.21.15.179:0/2844924135 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000b070 con 0x1e28840 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:06.999+0000 7fc45f7fe700 1 -- 172.21.15.179:0/2844924135 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470012740 con 0x1e28840 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/2844924135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 msgr2=0x1e2d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc47002c040 tx=0x7fc470021000 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/2844924135 shutdown_connections 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e58100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e58640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- 172.21.15.179:0/2844924135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1e2d8f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/2844924135 >> 172.21.15.179:0/2844924135 conn(0x1bbc5b0 msgr2=0x1e8bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/2844924135 shutdown_connections 2024-05-08T22:36:07.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/2844924135 wait complete. 2024-05-08T22:36:07.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- start start 2024-05-08T22:36:07.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 Processor -- start 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e92ab0 con 0x1e28840 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e59cb0 con 0x1e3c8e0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e91a90 con 0x1e526f0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 10 client.0 _ll_register_callbacks cb 0x1d4ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47ced8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34046/0 (socket says 172.21.15.179:34046) 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 -- 172.21.15.179:0/816656344 learned_addr learned my addr 172.21.15.179:0/816656344 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47deda700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 -- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 msgr2=0x1e595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 -- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 msgr2=0x1e590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 -- 172.21.15.179:0/816656344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d5b290 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc47d6d9700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc470021800 tx=0x7fc47000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6b9473441062b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc457fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470021410 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc470012b00 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc46c05d570 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.003+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc470021570 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 10 client.6288 fetch_fsmap learned FSMap version 24 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x1d35790 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c05d570 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc470012cc0 con 0x1e28840 2024-05-08T22:36:07.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 10 client.6288 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 10 client.6288 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 20 client.6288 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e555d0 0x1e31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 4 client.6288 mds_command: new command op to 55412 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 --> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1c382c0 con 0x1e555d0 2024-05-08T22:36:07.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47ced8700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e555d0 0x1e31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47ced8700 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e555d0 0x1e31660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f5ded3f41c5244 server_cookie=9645a0431125020d in_seq=0 out_seq=0 2024-05-08T22:36:07.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 10 client.6288 ms_handle_connect on v2:172.21.15.195:6836/1172092943 2024-05-08T22:36:07.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 1 -- 172.21.15.179:0/816656344 <== mds.0 v2:172.21.15.195:6836/1172092943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1c382c0 con 0x1e555d0 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 10 client.6288 handle_command_reply: tid=0 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 client.6288 shutdown 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 msgr2=0x1ed4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc470021800 tx=0x7fc47000a000 comp rx=0 tx=0).stop 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 shutdown_connections 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:6836/1172092943,v1:172.21.15.195:6837/1172092943] conn(0x1e555d0 0x1e31660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1e3c8e0 0x1e590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1e526f0 0x1e595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc457fff700 0 client.6288 ms_handle_reset on v2:172.21.15.195:6836/1172092943 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 --2- 172.21.15.179:0/816656344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1e28840 0x1ed4560 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 >> 172.21.15.179:0/816656344 conn(0x1bbc5b0 msgr2=0x1e2cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 shutdown_connections 2024-05-08T22:36:07.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 1 -- 172.21.15.179:0/816656344 wait complete. 2024-05-08T22:36:07.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc482bec740 20 client.6288 trim_cache size 0 max 16384 2024-05-08T22:36:07.016 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:07.016 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:36:07.016 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:36:07.016 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 msgr2=0x7fc464068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 0x7fc464068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4700012e0 tx=0x7fc470001aa0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 msgr2=0x7fc47814a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e9d0 tx=0x7fc46c00ece0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 shutdown_connections 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc464066730 0x7fc464068bf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc478157040 0x7fc47806b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc47806e6b0 0x7fc47814a6a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 --2- 172.21.15.179:0/1742537336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47806c2d0 0x7fc47814fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 >> 172.21.15.179:0/1742537336 conn(0x7fc478070d80 msgr2=0x7fc478148660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 shutdown_connections 2024-05-08T22:36:07.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.007+0000 7fc47e6db700 1 -- 172.21.15.179:0/1742537336 wait complete. 2024-05-08T22:36:07.032 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:36:07.133 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:07.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 Processor -- start 2024-05-08T22:36:07.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 -- start start 2024-05-08T22:36:07.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 0x7fd36006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd36006dc20 0x7fd36006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 0x7fd36006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600cd200 con 0x7fd36006dc20 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600cd4e0 con 0x7fd3601434d0 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600cd7c0 con 0x7fd3600ce2a0 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd36006dc20 0x7fd36006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd36006dc20 0x7fd36006e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34050/0 (socket says 172.21.15.179:34050) 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 -- 172.21.15.179:0/1123677371 learned_addr learned my addr 172.21.15.179:0/1123677371 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35f7fe700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 0x7fd36006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35effd700 1 --2- 172.21.15.179:0/1123677371 >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 0x7fd36006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 -- 172.21.15.179:0/1123677371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 msgr2=0x7fd36006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 0x7fd36006eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 -- 172.21.15.179:0/1123677371 >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 msgr2=0x7fd36006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/1123677371 >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 0x7fd36006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 -- 172.21.15.179:0/1123677371 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600cdaa0 con 0x7fd36006dc20 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35f7fe700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 0x7fd36006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:07.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35effd700 1 --2- 172.21.15.179:0/1123677371 >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 0x7fd36006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd36006dc20 0x7fd36006e640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd360070800 tx=0x7fd348009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14937724689874cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35dffb700 1 -- 172.21.15.179:0/1123677371 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800b8a0 con 0x7fd36006dc20 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35dffb700 1 -- 172.21.15.179:0/1123677371 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800ba00 con 0x7fd36006dc20 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.251+0000 7fd35dffb700 1 -- 172.21.15.179:0/1123677371 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800bd90 con 0x7fd36006dc20 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 -- 172.21.15.179:0/1123677371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd36006dc20 msgr2=0x7fd36006e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd36006dc20 0x7fd36006e640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd360070800 tx=0x7fd348009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 -- 172.21.15.179:0/1123677371 shutdown_connections 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3600ce2a0 0x7fd36006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 --2- 172.21.15.179:0/1123677371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd36006dc20 0x7fd36006e640 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 --2- 172.21.15.179:0/1123677371 >> v2:172.21.15.195:3300/0 conn(0x7fd3601434d0 0x7fd36006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 -- 172.21.15.179:0/1123677371 >> 172.21.15.179:0/1123677371 conn(0x7fd3600ce9d0 msgr2=0x7fd3600cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 -- 172.21.15.179:0/1123677371 shutdown_connections 2024-05-08T22:36:07.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.255+0000 7fd35ffff700 1 -- 172.21.15.179:0/1123677371 wait complete. 2024-05-08T22:36:07.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 Processor -- start 2024-05-08T22:36:07.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- start start 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3600ce2a0 0x7fd36015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36013b850 0x7fd360140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36013a9c0 con 0x7fd3601434d0 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36013ae50 con 0x7fd3600ce2a0 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36013a090 con 0x7fd36013b850 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34054/0 (socket says 172.21.15.179:34054) 2024-05-08T22:36:07.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 -- 172.21.15.179:0/3106361395 learned_addr learned my addr 172.21.15.179:0/3106361395 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35effd700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3600ce2a0 0x7fd36015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35f7fe700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36013b850 0x7fd360140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 -- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36013b850 msgr2=0x7fd360140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36013b850 0x7fd360140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 -- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3600ce2a0 msgr2=0x7fd36015b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3600ce2a0 0x7fd36015b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 -- 172.21.15.179:0/3106361395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009570 con 0x7fd3601434d0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35e7fc700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd348000c00 tx=0x7fd348004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480038c0 con 0x7fd3601434d0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34801e070 con 0x7fd3601434d0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36013a370 con 0x7fd3601434d0 2024-05-08T22:36:07.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348003c80 con 0x7fd3601434d0 2024-05-08T22:36:07.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.259+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36013bd00 con 0x7fd3601434d0 2024-05-08T22:36:07.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36013ab70 con 0x7fd3601434d0 2024-05-08T22:36:07.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd348011420 con 0x7fd3601434d0 2024-05-08T22:36:07.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd357fff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 0x7fd340068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd348020070 con 0x7fd3601434d0 2024-05-08T22:36:07.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd35effd700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 0x7fd340068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.263+0000 7fd35effd700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 0x7fd340068c20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd350006fd0 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.271+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd36013ab70 con 0x7fd3601434d0 2024-05-08T22:36:07.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd360067740 con 0x7fd3601434d0 2024-05-08T22:36:07.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd357fff700 1 -- 172.21.15.179:0/3106361395 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7fd348052c10 con 0x7fd3601434d0 2024-05-08T22:36:07.487 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:07.487 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 msgr2=0x7fd340068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 0x7fd340068c20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd350006fd0 tx=0x7fd350008040 comp rx=0 tx=0).stop 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 msgr2=0x7fd36013b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.487+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd348000c00 tx=0x7fd348004770 comp rx=0 tx=0).stop 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 shutdown_connections 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd340066760 0x7fd340068c20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd36013b850 0x7fd360140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3601434d0 0x7fd36013b310 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 --2- 172.21.15.179:0/3106361395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3600ce2a0 0x7fd36015b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 >> 172.21.15.179:0/3106361395 conn(0x7fd3600ce9d0 msgr2=0x7fd36006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 shutdown_connections 2024-05-08T22:36:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.491+0000 7fd35ffff700 1 -- 172.21.15.179:0/3106361395 wait complete. 2024-05-08T22:36:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:36:07.502 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T22:36:07.648 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:36:07.649 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:36:07.649 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T22:36:07.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 Processor -- start 2024-05-08T22:36:07.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 -- start start 2024-05-08T22:36:07.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e923c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 0x56527e9c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527e889a00 con 0x56527e9c9700 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527e8852f0 con 0x56527e8f21e0 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3142cb9140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56527e8418e0 con 0x56527e925a80 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f313ffed700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e923c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f3140fef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 0x56527e9c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f31407ee700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f313ffed700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e923c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:50352/0 (socket says 172.21.15.179:50352) 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f31407ee700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:43668/0 (socket says 172.21.15.179:43668) 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.691+0000 7f313ffed700 1 -- 172.21.15.179:0/4160977237 learned_addr learned my addr 172.21.15.179:0/4160977237 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 -- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 msgr2=0x56527e923c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e923c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 -- 172.21.15.179:0/4160977237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 msgr2=0x56527e9c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 0x56527e9c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 -- 172.21.15.179:0/4160977237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313000a100 con 0x56527e8f21e0 2024-05-08T22:36:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3140fef700 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 0x56527e9c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f31407ee700 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3138001cd0 tx=0x7f313800b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26b92879f177c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f313efeb700 1 -- 172.21.15.179:0/4160977237 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138014070 con 0x56527e8f21e0 2024-05-08T22:36:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f313efeb700 1 -- 172.21.15.179:0/4160977237 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138008030 con 0x56527e8f21e0 2024-05-08T22:36:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f313efeb700 1 -- 172.21.15.179:0/4160977237 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800bc80 con 0x56527e8f21e0 2024-05-08T22:36:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 msgr2=0x56527e936d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3138001cd0 tx=0x7f313800b4c0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- 172.21.15.179:0/4160977237 shutdown_connections 2024-05-08T22:36:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e9c9700 0x56527e9c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e923c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 --2- 172.21.15.179:0/4160977237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e8f21e0 0x56527e936d50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- 172.21.15.179:0/4160977237 >> 172.21.15.179:0/4160977237 conn(0x56527e8f8740 msgr2=0x56527e924c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- 172.21.15.179:0/4160977237 shutdown_connections 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- 172.21.15.179:0/4160977237 wait complete. 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 Processor -- start 2024-05-08T22:36:07.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.695+0000 7f3142cb9140 1 -- start start 2024-05-08T22:36:07.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 0x56527e9aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e9c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e9c9700 0x56527e9c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527e841580 con 0x56527e8f21e0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527e8178d0 con 0x56527e9c9700 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56527e9c31f0 con 0x56527e925a80 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 0x56527e9aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 0x56527e9aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34064/0 (socket says 172.21.15.179:34064) 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 -- 172.21.15.179:0/1545971296 learned_addr learned my addr 172.21.15.179:0/1545971296 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f313ffed700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e9c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3140fef700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e9c9700 0x56527e9c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 -- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 msgr2=0x56527e9c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56527e925a80 0x56527e9c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 -- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e9c9700 msgr2=0x56527e9c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56527e9c9700 0x56527e9c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 -- 172.21.15.179:0/1545971296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56527e82fb00 con 0x56527e8f21e0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f31407ee700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 0x56527e9aba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f31380011e0 tx=0x7f3138000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f313dfe9700 1 -- 172.21.15.179:0/1545971296 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138014040 con 0x56527e8f21e0 2024-05-08T22:36:07.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313800a040 con 0x56527e8f21e0 2024-05-08T22:36:07.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f313dfe9700 1 -- 172.21.15.179:0/1545971296 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800fe90 con 0x56527e8f21e0 2024-05-08T22:36:07.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56527e822f40 con 0x56527e8f21e0 2024-05-08T22:36:07.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.699+0000 7f313dfe9700 1 -- 172.21.15.179:0/1545971296 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313801b630 con 0x56527e8f21e0 2024-05-08T22:36:07.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f313dfe9700 1 -- 172.21.15.179:0/1545971296 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f313801b790 con 0x56527e8f21e0 2024-05-08T22:36:07.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f313dfe9700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3120065090 0x7f3120067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f313dfe9700 1 -- 172.21.15.179:0/1545971296 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f31380840e0 con 0x56527e8f21e0 2024-05-08T22:36:07.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f313ffed700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3120065090 0x7f3120067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x56527e9c1e60 0x56527e9c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:07.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) v8 -- 0x56527e9c27b0 con 0x56527e9c1e60 2024-05-08T22:36:07.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3140fef700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x56527e9c1e60 0x56527e9c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f313ffed700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3120065090 0x7f3120067550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3128001ed0 tx=0x7f3128001e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3140fef700 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x56527e9c1e60 0x56527e9c2270 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3140fef700 1 -- 172.21.15.179:0/1545971296 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f3130017040 con 0x56527e9c1e60 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x56527e9c1e60 msgr2=0x56527e9c2270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x56527e9c1e60 0x56527e9c2270 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3120065090 msgr2=0x7f3120067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3120065090 0x7f3120067550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3128001ed0 tx=0x7f3128001e60 comp rx=0 tx=0).stop 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 msgr2=0x56527e9aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 --2- 172.21.15.179:0/1545971296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56527e8f21e0 0x56527e9aba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f31380011e0 tx=0x7f3138000ed0 comp rx=0 tx=0).stop 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3140fef700 1 -- 172.21.15.179:0/1545971296 reap_dead start 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 shutdown_connections 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 >> 172.21.15.179:0/1545971296 conn(0x56527e8f8740 msgr2=0x56527e923c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 shutdown_connections 2024-05-08T22:36:07.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:07.703+0000 7f3142cb9140 1 -- 172.21.15.179:0/1545971296 wait complete. 2024-05-08T22:36:07.708 INFO:teuthology.orchestra.run.smithi179.stdout:Yalphaparent_a 2024-05-08T22:36:07.708 DEBUG:teuthology.orchestra.run.smithi179:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T22:36:07.810 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "ancestors": [ 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1099511627776, 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "alpha", 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 4 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1, 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "parent_a", 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 7 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:07.811 INFO:teuthology.orchestra.run.smithi179.stdout: ], 2024-05-08T22:36:07.812 INFO:teuthology.orchestra.run.smithi179.stdout: "pool": 8, 2024-05-08T22:36:07.812 INFO:teuthology.orchestra.run.smithi179.stdout: "old_pools": [] 2024-05-08T22:36:07.812 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:07.812 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:07.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:36:07.814 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:36:07.832 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:36:07.854 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:36:07.854 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:36:07.854 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:07.862 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 Processor -- start 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 -- start start 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb898161790 0x7fb898164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb898165d50 con 0x7fb89815ef80 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89806ad70 con 0x7fb89806de70 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89d9f0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89806e4b0 con 0x7fb898161790 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89c9ee700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb89815ef80 0x7fb89806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38806/0 (socket says 172.21.15.179:38806) 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.023+0000 7fb89c9ee700 1 -- 172.21.15.179:0/621467736 learned_addr learned my addr 172.21.15.179:0/621467736 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d1ef700 1 --2- 172.21.15.179:0/621467736 >> v2:172.21.15.195:3300/0 conn(0x7fb898161790 0x7fb898164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb8977fe700 1 -- 172.21.15.179:0/621467736 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560776551 0 0) 0x7fb898165d50 con 0x7fb89815ef80 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb8977fe700 1 -- 172.21.15.179:0/621467736 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8840035d0 con 0x7fb89815ef80 2024-05-08T22:36:08.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d1ef700 1 -- 172.21.15.179:0/621467736 >> v1:172.21.15.195:6789/0 conn(0x7fb89806de70 legacy=0x7fb89806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d1ef700 1 -- 172.21.15.179:0/621467736 >> v1:172.21.15.179:6789/0 conn(0x7fb89815ef80 legacy=0x7fb89806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d1ef700 1 -- 172.21.15.179:0/621467736 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8981654b0 con 0x7fb898161790 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d1ef700 1 --2- 172.21.15.179:0/621467736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb898161790 0x7fb898164db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097d0 tx=0x7fb890009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c923443933ab15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb8977fe700 1 -- 172.21.15.179:0/621467736 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890004230 con 0x7fb898161790 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb8977fe700 1 -- 172.21.15.179:0/621467736 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890004390 con 0x7fb898161790 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb8977fe700 1 -- 172.21.15.179:0/621467736 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8900036c0 con 0x7fb898161790 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- 172.21.15.179:0/621467736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb898161790 msgr2=0x7fb898164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/621467736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb898161790 0x7fb898164db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097d0 tx=0x7fb890009ae0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- 172.21.15.179:0/621467736 shutdown_connections 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/621467736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb898161790 0x7fb898164db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- 172.21.15.179:0/621467736 >> 172.21.15.179:0/621467736 conn(0x7fb8980ce9d0 msgr2=0x7fb8980cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- 172.21.15.179:0/621467736 shutdown_connections 2024-05-08T22:36:08.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- 172.21.15.179:0/621467736 wait complete. 2024-05-08T22:36:08.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 Processor -- start 2024-05-08T22:36:08.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.027+0000 7fb89d9f0700 1 -- start start 2024-05-08T22:36:08.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb89815ef80 0x7fb898161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb898161790 0x7fb8981620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980c51b0 con 0x7fb89806de70 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb898151400 con 0x7fb89815ef80 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89815b230 con 0x7fb898161790 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34074/0 (socket says 172.21.15.179:34074) 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 -- 172.21.15.179:0/1202629536 learned_addr learned my addr 172.21.15.179:0/1202629536 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb897fff700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb89815ef80 0x7fb898161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d1ef700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb898161790 0x7fb8981620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 -- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb898161790 msgr2=0x7fb8981620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb898161790 0x7fb8981620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 -- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb89815ef80 msgr2=0x7fb898161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb89815ef80 0x7fb898161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 -- 172.21.15.179:0/1202629536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900094b0 con 0x7fb89806de70 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89c9ee700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c700 tx=0x7fb88800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888005d90 con 0x7fb89806de70 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8980c4530 con 0x7fb89806de70 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980c4af0 con 0x7fb89806de70 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8880012f0 con 0x7fb89806de70 2024-05-08T22:36:08.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8880072d0 con 0x7fb89806de70 2024-05-08T22:36:08.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.031+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c005310 con 0x7fb89806de70 2024-05-08T22:36:08.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.035+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb888007430 con 0x7fb89806de70 2024-05-08T22:36:08.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.035+0000 7fb8957fa700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 0x7fb878068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.035+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb888083690 con 0x7fb89806de70 2024-05-08T22:36:08.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.035+0000 7fb897fff700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 0x7fb878068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.035+0000 7fb897fff700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 0x7fb878068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb880002230 tx=0x7fb880009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.039+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb888050810 con 0x7fb89806de70 2024-05-08T22:36:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.211+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb87c0055c0 con 0x7fb89806de70 2024-05-08T22:36:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.215+0000 7fb8957fa700 1 -- 172.21.15.179:0/1202629536 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16394 (secure 0 0 0) 0x7fb88805a070 con 0x7fb89806de70 2024-05-08T22:36:08.216 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:08.217 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":57,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:05.576343+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 msgr2=0x7fb878068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 0x7fb878068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb880002230 tx=0x7fb880009040 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 msgr2=0x7fb898150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c700 tx=0x7fb88800ca10 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 shutdown_connections 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb878066730 0x7fb878068bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb89815ef80 0x7fb898161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb898161790 0x7fb8981620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 --2- 172.21.15.179:0/1202629536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb89806de70 0x7fb898150ec0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 >> 172.21.15.179:0/1202629536 conn(0x7fb8980ce9d0 msgr2=0x7fb898160200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 shutdown_connections 2024-05-08T22:36:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.219+0000 7fb89d9f0700 1 -- 172.21.15.179:0/1202629536 wait complete. 2024-05-08T22:36:08.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:25.626475+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:26.294593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:08.236 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 Processor -- start 2024-05-08T22:36:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 -- start start 2024-05-08T22:36:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa12815d000 0x7fa1280cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1280ca880 0x7fa1280cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1280cd070 con 0x7fa1280b47f0 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280b3b00 con 0x7fa1280ca880 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.355+0000 7fa12ea3f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280b3d10 con 0x7fa12815d000 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1280ca880 0x7fa1280cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1280ca880 0x7fa1280cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34076/0 (socket says 172.21.15.179:34076) 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 -- 172.21.15.179:0/380284985 learned_addr learned my addr 172.21.15.179:0/380284985 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12d23c700 1 --1- 172.21.15.179:0/380284985 >> v1:172.21.15.179:6789/0 conn(0x7fa1280b47f0 0x7fa128160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38814/0 (socket says 172.21.15.179:38814) 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12da3d700 1 --2- 172.21.15.179:0/380284985 >> v2:172.21.15.195:3300/0 conn(0x7fa12815d000 0x7fa1280cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 -- 172.21.15.179:0/380284985 >> v2:172.21.15.195:3300/0 conn(0x7fa12815d000 msgr2=0x7fa1280cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 --2- 172.21.15.179:0/380284985 >> v2:172.21.15.195:3300/0 conn(0x7fa12815d000 0x7fa1280cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 -- 172.21.15.179:0/380284985 >> v1:172.21.15.179:6789/0 conn(0x7fa1280b47f0 legacy=0x7fa128160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 -- 172.21.15.179:0/380284985 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280b3ff0 con 0x7fa1280ca880 2024-05-08T22:36:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ca3b700 1 -- 172.21.15.179:0/380284985 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708897150 0 0) 0x7fa1280cd070 con 0x7fa1280b47f0 2024-05-08T22:36:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12e23e700 1 --2- 172.21.15.179:0/380284985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1280ca880 0x7fa1280cfdf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa1280d0c30 tx=0x7fa12000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd21d1f17a5a707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ca3b700 1 -- 172.21.15.179:0/380284985 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000e070 con 0x7fa1280ca880 2024-05-08T22:36:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ca3b700 1 -- 172.21.15.179:0/380284985 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000b9c0 con 0x7fa1280ca880 2024-05-08T22:36:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ca3b700 1 -- 172.21.15.179:0/380284985 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000bd70 con 0x7fa1280ca880 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 -- 172.21.15.179:0/380284985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1280ca880 msgr2=0x7fa1280cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/380284985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1280ca880 0x7fa1280cfdf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa1280d0c30 tx=0x7fa12000b330 comp rx=0 tx=0).stop 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 -- 172.21.15.179:0/380284985 shutdown_connections 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/380284985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1280ca880 0x7fa1280cfdf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/380284985 >> v2:172.21.15.195:3300/0 conn(0x7fa12815d000 0x7fa1280cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 -- 172.21.15.179:0/380284985 >> 172.21.15.179:0/380284985 conn(0x7fa1280b27c0 msgr2=0x7fa1280b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 -- 172.21.15.179:0/380284985 shutdown_connections 2024-05-08T22:36:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 -- 172.21.15.179:0/380284985 wait complete. 2024-05-08T22:36:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.359+0000 7fa12ea3f700 1 Processor -- start 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- start start 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1280b47f0 0x7fa1281f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1280ca880 0x7fa1280cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c5770 con 0x7fa12815d000 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281f6cd0 con 0x7fa1280ca880 2024-05-08T22:36:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c4af0 con 0x7fa1280b47f0 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34086/0 (socket says 172.21.15.179:34086) 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 -- 172.21.15.179:0/178727442 learned_addr learned my addr 172.21.15.179:0/178727442 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12d23c700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1280ca880 0x7fa1280cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12da3d700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1280b47f0 0x7fa1281f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 -- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1280b47f0 msgr2=0x7fa1281f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1280b47f0 0x7fa1281f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 -- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1280ca880 msgr2=0x7fa1280cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1280ca880 0x7fa1280cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 -- 172.21.15.179:0/178727442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120009b30 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12e23e700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa12806bc60 tx=0x7fa120004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120004840 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1200099b0 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280c4dd0 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12001c680 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.363+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1280c5390 con 0x7fa12815d000 2024-05-08T22:36:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280c58b0 con 0x7fa12815d000 2024-05-08T22:36:08.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa12001c7e0 con 0x7fa12815d000 2024-05-08T22:36:08.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa11e7fc700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 0x7fa10c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa120085db0 con 0x7fa12815d000 2024-05-08T22:36:08.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa12da3d700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 0x7fa10c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.367+0000 7fa12da3d700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 0x7fa10c068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa118007a60 tx=0x7fa118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.375+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1280c58b0 con 0x7fa12815d000 2024-05-08T22:36:08.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1280c58b0 con 0x7fa12815d000 2024-05-08T22:36:08.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa11e7fc700 1 -- 172.21.15.179:0/178727442 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1280c58b0 con 0x7fa12815d000 2024-05-08T22:36:08.543 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:36:08.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 msgr2=0x7fa10c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 0x7fa10c068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa118007a60 tx=0x7fa118008040 comp rx=0 tx=0).stop 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 msgr2=0x7fa1280cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa12806bc60 tx=0x7fa120004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 shutdown_connections 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa10c066730 0x7fa10c068bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1280ca880 0x7fa1280cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa12815d000 0x7fa1280cfcf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.543+0000 7fa12ea3f700 1 --2- 172.21.15.179:0/178727442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1280b47f0 0x7fa1281f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.547+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 >> 172.21.15.179:0/178727442 conn(0x7fa1280b27c0 msgr2=0x7fa1280b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.547+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 shutdown_connections 2024-05-08T22:36:08.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.547+0000 7fa12ea3f700 1 -- 172.21.15.179:0/178727442 wait complete. 2024-05-08T22:36:08.559 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:36:08.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 Processor -- start 2024-05-08T22:36:08.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 -- start start 2024-05-08T22:36:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1f100cffc0 0x7f1f100cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100cd070 con 0x7f1f1015d000 2024-05-08T22:36:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100b3d10 con 0x7f1f100b47f0 2024-05-08T22:36:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1735f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100b3b00 con 0x7f1f100cffc0 2024-05-08T22:36:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1635d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1f1015d000 0x7f1f100cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38820/0 (socket says 172.21.15.179:38820) 2024-05-08T22:36:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f1635d700 1 -- 172.21.15.179:0/2713514620 learned_addr learned my addr 172.21.15.179:0/2713514620 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.675+0000 7f1f15b5c700 1 --2- 172.21.15.179:0/2713514620 >> v2:172.21.15.195:3300/0 conn(0x7f1f100cffc0 0x7f1f100cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1535b700 1 -- 172.21.15.179:0/2713514620 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511346887 0 0) 0x7f1f100cd070 con 0x7f1f1015d000 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1535b700 1 -- 172.21.15.179:0/2713514620 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1efc0035d0 con 0x7f1f1015d000 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2713514620 >> v1:172.21.15.195:6789/0 conn(0x7f1f100b47f0 legacy=0x7f1f100cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2713514620 >> v1:172.21.15.179:6789/0 conn(0x7f1f1015d000 legacy=0x7f1f100cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2713514620 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100b3ff0 con 0x7f1f100cffc0 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f15b5c700 1 --2- 172.21.15.179:0/2713514620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100cffc0 0x7f1f100cb320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009850 tx=0x7f1f0c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbeac9f6773a52b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1535b700 1 -- 172.21.15.179:0/2713514620 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c011830 con 0x7f1f100cffc0 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1535b700 1 -- 172.21.15.179:0/2713514620 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c011990 con 0x7f1f100cffc0 2024-05-08T22:36:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1535b700 1 -- 172.21.15.179:0/2713514620 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c011d00 con 0x7f1f100cffc0 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- 172.21.15.179:0/2713514620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100cffc0 msgr2=0x7f1f100cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2713514620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100cffc0 0x7f1f100cb320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009850 tx=0x7f1f0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- 172.21.15.179:0/2713514620 shutdown_connections 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2713514620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100cffc0 0x7f1f100cb320 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- 172.21.15.179:0/2713514620 >> 172.21.15.179:0/2713514620 conn(0x7f1f100b27c0 msgr2=0x7f1f100b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- 172.21.15.179:0/2713514620 shutdown_connections 2024-05-08T22:36:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- 172.21.15.179:0/2713514620 wait complete. 2024-05-08T22:36:08.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 Processor -- start 2024-05-08T22:36:08.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.679+0000 7f1f1735f700 1 -- start start 2024-05-08T22:36:08.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100b47f0 0x7f1f1014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 0x7f1f100ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100afc10 con 0x7f1f100cffc0 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100af000 con 0x7f1f100b47f0 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100af2e0 con 0x7f1f1015d000 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34094/0 (socket says 172.21.15.179:34094) 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2464363425 learned_addr learned my addr 172.21.15.179:0/2464363425 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f16b5e700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 0x7f1f100ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1635d700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100b47f0 0x7f1f1014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 msgr2=0x7f1f100ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 0x7f1f100ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100b47f0 msgr2=0x7f1f1014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100b47f0 0x7f1f1014d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 -- 172.21.15.179:0/2464363425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0094b0 con 0x7f1f100cffc0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f16b5e700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 0x7f1f100ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f15b5c700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009ba0 tx=0x7f1f0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0036c0 con 0x7f1f100cffc0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c01f070 con 0x7f1f100cffc0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f100af5c0 con 0x7f1f100cffc0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003a60 con 0x7f1f100cffc0 2024-05-08T22:36:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.683+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1014d540 con 0x7f1f100cffc0 2024-05-08T22:36:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f10067740 con 0x7f1f100cffc0 2024-05-08T22:36:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f0c003bc0 con 0x7f1f100cffc0 2024-05-08T22:36:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f06ffd700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 0x7f1ef4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f1f0c021070 con 0x7f1f100cffc0 2024-05-08T22:36:08.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f1635d700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 0x7f1ef4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.687+0000 7f1f1635d700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 0x7f1ef4068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1f000011e0 tx=0x7f1f00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.695+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1f0c052a30 con 0x7f1f100cffc0 2024-05-08T22:36:08.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.863+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f1004e700 con 0x7f1f100cffc0 2024-05-08T22:36:08.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.863+0000 7f1f06ffd700 1 -- 172.21.15.179:0/2464363425 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f0c015020 con 0x7f1f100cffc0 2024-05-08T22:36:08.865 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:36:08.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 msgr2=0x7f1ef4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 0x7f1ef4068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1f000011e0 tx=0x7f1f00009040 comp rx=0 tx=0).stop 2024-05-08T22:36:08.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 msgr2=0x7f1f100ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009ba0 tx=0x7f1f0c004770 comp rx=0 tx=0).stop 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 shutdown_connections 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1ef4066730 0x7f1ef4068bf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f1015d000 0x7f1f100ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f100cffc0 0x7f1f100ae4b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 --2- 172.21.15.179:0/2464363425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f100b47f0 0x7f1f1014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 >> 172.21.15.179:0/2464363425 conn(0x7f1f100b27c0 msgr2=0x7f1f100c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 shutdown_connections 2024-05-08T22:36:08.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:08.867+0000 7f1f1735f700 1 -- 172.21.15.179:0/2464363425 wait complete. 2024-05-08T22:36:08.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:36:09.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 Processor -- start 2024-05-08T22:36:09.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 -- start start 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc7840b8a80 0x7fc7840cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7840ce0e0 0x7fc78406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7840b8ee0 con 0x7fc78415d0c0 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7840ce570 con 0x7fc7840ce0e0 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc783fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78415d4a0 con 0x7fc7840b8a80 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7840ce0e0 0x7fc78406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7840ce0e0 0x7fc78406b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34108/0 (socket says 172.21.15.179:34108) 2024-05-08T22:36:09.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 -- 172.21.15.179:0/1451512050 learned_addr learned my addr 172.21.15.179:0/1451512050 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7827fc700 1 --2- 172.21.15.179:0/1451512050 >> v2:172.21.15.195:3300/0 conn(0x7fc7840b8a80 0x7fc7840cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 -- 172.21.15.179:0/1451512050 >> v2:172.21.15.195:3300/0 conn(0x7fc7840b8a80 msgr2=0x7fc7840cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 --2- 172.21.15.179:0/1451512050 >> v2:172.21.15.195:3300/0 conn(0x7fc7840b8a80 0x7fc7840cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 -- 172.21.15.179:0/1451512050 >> v1:172.21.15.179:6789/0 conn(0x7fc78415d0c0 legacy=0x7fc7840d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.003+0000 7fc7837fe700 1 -- 172.21.15.179:0/1451512050 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78406abb0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc7837fe700 1 --2- 172.21.15.179:0/1451512050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc78406b370 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc784160b00 tx=0x7fc77c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e920714317b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc781ffb700 1 -- 172.21.15.179:0/1451512050 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00b8a0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc781ffb700 1 -- 172.21.15.179:0/1451512050 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00ba00 con 0x7fc7840ce0e0 2024-05-08T22:36:09.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc781ffb700 1 -- 172.21.15.179:0/1451512050 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00bd90 con 0x7fc7840ce0e0 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- 172.21.15.179:0/1451512050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 msgr2=0x7fc78406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 --2- 172.21.15.179:0/1451512050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc78406b370 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc784160b00 tx=0x7fc77c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- 172.21.15.179:0/1451512050 shutdown_connections 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 --2- 172.21.15.179:0/1451512050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc78406b370 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 --2- 172.21.15.179:0/1451512050 >> v2:172.21.15.195:3300/0 conn(0x7fc7840b8a80 0x7fc7840cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- 172.21.15.179:0/1451512050 >> 172.21.15.179:0/1451512050 conn(0x7fc7840d2ed0 msgr2=0x7fc7840d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- 172.21.15.179:0/1451512050 shutdown_connections 2024-05-08T22:36:09.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- 172.21.15.179:0/1451512050 wait complete. 2024-05-08T22:36:09.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 Processor -- start 2024-05-08T22:36:09.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.007+0000 7fc783fff700 1 -- start start 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7840b8a80 0x7fc7840c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78415d0c0 0x7fc784154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840cb3f0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784155240 con 0x7fc78415d0c0 2024-05-08T22:36:09.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840cb5d0 con 0x7fc7840b8a80 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34110/0 (socket says 172.21.15.179:34110) 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 -- 172.21.15.179:0/4077136515 learned_addr learned my addr 172.21.15.179:0/4077136515 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc782ffd700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7840b8a80 0x7fc7840c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7837fe700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78415d0c0 0x7fc784154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 -- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7840b8a80 msgr2=0x7fc7840c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7840b8a80 0x7fc7840c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 -- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78415d0c0 msgr2=0x7fc784154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78415d0c0 0x7fc784154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 -- 172.21.15.179:0/4077136515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c009570 con 0x7fc7840ce0e0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc7827fc700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e760 tx=0x7fc76c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c013610 con 0x7fc7840ce0e0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c013770 con 0x7fc7840ce0e0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7840cb8b0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c013ae0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc784151ee0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7840d27c0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.011+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc76c013c40 con 0x7fc7840ce0e0 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.015+0000 7fc77bfff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 0x7fc764068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.015+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc76c084050 con 0x7fc7840ce0e0 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.015+0000 7fc782ffd700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 0x7fc764068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.015+0000 7fc782ffd700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 0x7fc764068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc774002130 tx=0x7fc774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.019+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7840d27c0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7840d27c0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc77bfff700 1 -- 172.21.15.179:0/4077136515 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7840d27c0 con 0x7fc7840ce0e0 2024-05-08T22:36:09.191 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:36:09.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 msgr2=0x7fc764068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 0x7fc764068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc774002130 tx=0x7fc774009040 comp rx=0 tx=0).stop 2024-05-08T22:36:09.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 msgr2=0x7fc7841519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e760 tx=0x7fc76c00eb20 comp rx=0 tx=0).stop 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 shutdown_connections 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc764066730 0x7fc764068bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc78415d0c0 0x7fc784154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7840ce0e0 0x7fc7841519a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 --2- 172.21.15.179:0/4077136515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7840b8a80 0x7fc7840c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 >> 172.21.15.179:0/4077136515 conn(0x7fc7840d2ed0 msgr2=0x7fc78406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 shutdown_connections 2024-05-08T22:36:09.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.191+0000 7fc783fff700 1 -- 172.21.15.179:0/4077136515 wait complete. 2024-05-08T22:36:09.206 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:36:09.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.315+0000 7f62f9db4700 1 Processor -- start 2024-05-08T22:36:09.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.315+0000 7f62f9db4700 1 -- start start 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f62f40b4f70 0x7f62f40ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40cd6d0 con 0x7f62f415cf60 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f4161b30 con 0x7f62f415d340 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40cd970 con 0x7f62f40b4f70 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f8db2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f62f415cf60 0x7f62f40b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38844/0 (socket says 172.21.15.179:38844) 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f8db2700 1 -- 172.21.15.179:0/1669500927 learned_addr learned my addr 172.21.15.179:0/1669500927 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f3fff700 1 --2- 172.21.15.179:0/1669500927 >> v2:172.21.15.195:3300/0 conn(0x7f62f40b4f70 0x7f62f40ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735144656 0 0) 0x7f62f40cd6d0 con 0x7f62f415cf60 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e00035d0 con 0x7f62f415cf60 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630748563 0 0) 0x7f62f4161b30 con 0x7f62f415d340 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f40cd6d0 con 0x7f62f415d340 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f3fff700 1 -- 172.21.15.179:0/1669500927 >> v1:172.21.15.195:6789/0 conn(0x7f62f415d340 legacy=0x7f62f4160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f3fff700 1 -- 172.21.15.179:0/1669500927 >> v1:172.21.15.179:6789/0 conn(0x7f62f415cf60 legacy=0x7f62f40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f3fff700 1 -- 172.21.15.179:0/1669500927 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40cdc50 con 0x7f62f40b4f70 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f3fff700 1 --2- 172.21.15.179:0/1669500927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f40ce450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009850 tx=0x7f62e4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c53628b5bb0806 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011830 con 0x7f62f40b4f70 2024-05-08T22:36:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e4011990 con 0x7f62f40b4f70 2024-05-08T22:36:09.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f37fe700 1 -- 172.21.15.179:0/1669500927 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011d40 con 0x7f62f40b4f70 2024-05-08T22:36:09.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- 172.21.15.179:0/1669500927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 msgr2=0x7f62f40ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 --2- 172.21.15.179:0/1669500927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f40ce450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009850 tx=0x7f62e4009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- 172.21.15.179:0/1669500927 shutdown_connections 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 --2- 172.21.15.179:0/1669500927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f40ce450 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- 172.21.15.179:0/1669500927 >> 172.21.15.179:0/1669500927 conn(0x7f62f40cf860 msgr2=0x7f62f40cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- 172.21.15.179:0/1669500927 shutdown_connections 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.319+0000 7f62f9db4700 1 -- 172.21.15.179:0/1669500927 wait complete. 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 Processor -- start 2024-05-08T22:36:09.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 -- start start 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62f415d340 0x7f62f41f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40aa430 con 0x7f62f415cf60 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f414a7c0 con 0x7f62f40b4f70 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f9db4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40a97b0 con 0x7f62f415d340 2024-05-08T22:36:09.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34124/0 (socket says 172.21.15.179:34124) 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 -- 172.21.15.179:0/3305155798 learned_addr learned my addr 172.21.15.179:0/3305155798 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f8db2700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f95b3700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62f415d340 0x7f62f41f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 -- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62f415d340 msgr2=0x7f62f41f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62f415d340 0x7f62f41f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 -- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 msgr2=0x7f62f414a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f414a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f3fff700 1 -- 172.21.15.179:0/3305155798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40094b0 con 0x7f62f415cf60 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.323+0000 7f62f8db2700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f414a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f3fff700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f62f40b5680 tx=0x7f62e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e401aac0 con 0x7f62f415cf60 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e401f070 con 0x7f62f415cf60 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f40a9a90 con 0x7f62f415cf60 2024-05-08T22:36:09.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e401ae60 con 0x7f62f415cf60 2024-05-08T22:36:09.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f40aa050 con 0x7f62f415cf60 2024-05-08T22:36:09.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f40aa570 con 0x7f62f415cf60 2024-05-08T22:36:09.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62e401b430 con 0x7f62f415cf60 2024-05-08T22:36:09.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 0x7f62d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.327+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f62e4085320 con 0x7f62f415cf60 2024-05-08T22:36:09.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.331+0000 7f62f8db2700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 0x7f62d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.331+0000 7f62f8db2700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 0x7f62d4068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000f60 tx=0x7f62dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.339+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f62f40aa570 con 0x7f62f415cf60 2024-05-08T22:36:09.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.515+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62f40aa570 con 0x7f62f415cf60 2024-05-08T22:36:09.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.515+0000 7f62f17fa700 1 -- 172.21.15.179:0/3305155798 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f62f40aa570 con 0x7f62f415cf60 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 msgr2=0x7f62d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 0x7f62d4068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000f60 tx=0x7f62dc009040 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 msgr2=0x7f62f40aa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f62f40b5680 tx=0x7f62e4004770 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 shutdown_connections 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f62d4066730 0x7f62d4068bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62f415d340 0x7f62f41f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62f415cf60 0x7f62f40aa850 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 --2- 172.21.15.179:0/3305155798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62f40b4f70 0x7f62f414a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 >> 172.21.15.179:0/3305155798 conn(0x7f62f40cf860 msgr2=0x7f62f406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 shutdown_connections 2024-05-08T22:36:09.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.519+0000 7f62f9db4700 1 -- 172.21.15.179:0/3305155798 wait complete. 2024-05-08T22:36:09.521 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:36:09.532 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:36:09.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:09.532 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:09.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[38952]: fuse finished with error 0 and tester_r 0 2024-05-08T22:36:09.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:36:09.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:09.543 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:36:09.669 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:36:09.669 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:09.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:36:09.677 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:36:09.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:09.677 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:36:09.683 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:36:09.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:36:09.685 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:36:09.685 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:09.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:36:09.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:36:09.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:09.717 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:36:09.723 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:36:09.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:36:09.724 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:36:09.724 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:09.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:36:09.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:36:09.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:09.732 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:36:09.782 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:36:09.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:36:09.783 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:09.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.903+0000 7f497ce59700 1 Processor -- start 2024-05-08T22:36:09.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- start start 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f497806e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497806efd0 con 0x7f49781434d0 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497806ddf0 con 0x7f497806eb80 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780cd200 con 0x7f49780ce2a0 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49777fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f49781434d0 0x7f497806d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38858/0 (socket says 172.21.15.179:38858) 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49777fe700 1 -- 172.21.15.179:0/17995088 learned_addr learned my addr 172.21.15.179:0/17995088 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:09.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f4976ffd700 1 --2- 172.21.15.179:0/17995088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f497806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937116223 0 0) 0x7f4960001110 con 0x7f49781434d0 2024-05-08T22:36:09.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49640035d0 con 0x7f49781434d0 2024-05-08T22:36:09.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023350190 0 0) 0x7f497806ddf0 con 0x7f497806eb80 2024-05-08T22:36:09.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4960001110 con 0x7f497806eb80 2024-05-08T22:36:09.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f4976ffd700 1 -- 172.21.15.179:0/17995088 >> v1:172.21.15.195:6789/0 conn(0x7f497806eb80 legacy=0x7f49780b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f4976ffd700 1 -- 172.21.15.179:0/17995088 >> v1:172.21.15.179:6789/0 conn(0x7f49781434d0 legacy=0x7f497806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f4976ffd700 1 -- 172.21.15.179:0/17995088 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780cd4e0 con 0x7f49780ce2a0 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f4976ffd700 1 --2- 172.21.15.179:0/17995088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f497806e640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4968009880 tx=0x7f4968009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda4144d47d4569e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968011830 con 0x7f49780ce2a0 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968011990 con 0x7f49780ce2a0 2024-05-08T22:36:09.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f49767fc700 1 -- 172.21.15.179:0/17995088 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968011d00 con 0x7f49780ce2a0 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- 172.21.15.179:0/17995088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 msgr2=0x7f497806e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 --2- 172.21.15.179:0/17995088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f497806e640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4968009880 tx=0x7f4968009b90 comp rx=0 tx=0).stop 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- 172.21.15.179:0/17995088 shutdown_connections 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 --2- 172.21.15.179:0/17995088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f497806e640 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- 172.21.15.179:0/17995088 >> 172.21.15.179:0/17995088 conn(0x7f49780ce9d0 msgr2=0x7f49780cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- 172.21.15.179:0/17995088 shutdown_connections 2024-05-08T22:36:09.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.907+0000 7f497ce59700 1 -- 172.21.15.179:0/17995088 wait complete. 2024-05-08T22:36:09.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 Processor -- start 2024-05-08T22:36:09.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 -- start start 2024-05-08T22:36:09.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 0x7f49781594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f49781e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49781434d0 0x7f49780cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781e7b20 con 0x7f497806eb80 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c8280 con 0x7f49781434d0 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f497ce59700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c8560 con 0x7f49780ce2a0 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f49777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 0x7f49781594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f49777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 0x7f49781594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34140/0 (socket says 172.21.15.179:34140) 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f49777fe700 1 -- 172.21.15.179:0/3147609910 learned_addr learned my addr 172.21.15.179:0/3147609910 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f49781e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4977fff700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49781434d0 0x7f49780cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 -- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49781434d0 msgr2=0x7f49780cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49781434d0 0x7f49780cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 -- 172.21.15.179:0/3147609910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 msgr2=0x7f49781594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 0x7f49781594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.911+0000 7f4976ffd700 1 -- 172.21.15.179:0/3147609910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680094b0 con 0x7f49780ce2a0 2024-05-08T22:36:09.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f4976ffd700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f49781e71d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4968015000 tx=0x7f4968004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968004450 con 0x7f49780ce2a0 2024-05-08T22:36:09.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680045b0 con 0x7f49780ce2a0 2024-05-08T22:36:09.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49780c8840 con 0x7f49780ce2a0 2024-05-08T22:36:09.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680038e0 con 0x7f49780ce2a0 2024-05-08T22:36:09.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781e7710 con 0x7f49780ce2a0 2024-05-08T22:36:09.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.915+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49780c8d60 con 0x7f49780ce2a0 2024-05-08T22:36:09.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.919+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4968003a40 con 0x7f49780ce2a0 2024-05-08T22:36:09.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.919+0000 7f496ffff700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 0x7f4958068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.919+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4968085890 con 0x7f49780ce2a0 2024-05-08T22:36:09.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.919+0000 7f49777fe700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 0x7f4958068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.919+0000 7f49777fe700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 0x7f4958068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:09.927+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4968052bf0 con 0x7f49780ce2a0 2024-05-08T22:36:10.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.135+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49780c8d60 con 0x7f49780ce2a0 2024-05-08T22:36:10.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.135+0000 7f496ffff700 1 -- 172.21.15.179:0/3147609910 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f49780c8d60 con 0x7f49780ce2a0 2024-05-08T22:36:10.136 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:10.136 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:36:10.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 msgr2=0x7f4958068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 0x7f4958068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960007ec0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 msgr2=0x7f49781e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f49781e71d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4968015000 tx=0x7f4968004770 comp rx=0 tx=0).stop 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 shutdown_connections 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4958066730 0x7f4958068bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49781434d0 0x7f49780cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49780ce2a0 0x7f49781e71d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 --2- 172.21.15.179:0/3147609910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497806eb80 0x7f49781594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 >> 172.21.15.179:0/3147609910 conn(0x7f49780ce9d0 msgr2=0x7f49781425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 shutdown_connections 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.139+0000 7f497ce59700 1 -- 172.21.15.179:0/3147609910 wait complete. 2024-05-08T22:36:10.139 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:36:10.151 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:10.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.259+0000 7fa187152700 1 Processor -- start 2024-05-08T22:36:10.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.259+0000 7fa187152700 1 -- start start 2024-05-08T22:36:10.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa18015ef80 0x7fa180161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa180164db0 0x7fa18006d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa180165d50 con 0x7fa18006dca0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006e130 con 0x7fa180164db0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006ad70 con 0x7fa18015ef80 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa186951700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa18006dca0 0x7fa1800b2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38866/0 (socket says 172.21.15.179:38866) 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa186951700 1 -- 172.21.15.179:0/2393487006 learned_addr learned my addr 172.21.15.179:0/2393487006 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 --2- 172.21.15.179:0/2393487006 >> v2:172.21.15.179:3300/0 conn(0x7fa180164db0 0x7fa18006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa186150700 1 --2- 172.21.15.179:0/2393487006 >> v2:172.21.15.195:3300/0 conn(0x7fa18015ef80 0x7fa180161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18514e700 1 -- 172.21.15.179:0/2393487006 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318069428 0 0) 0x7fa180165d50 con 0x7fa18006dca0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18514e700 1 -- 172.21.15.179:0/2393487006 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa16c0035d0 con 0x7fa18006dca0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 -- 172.21.15.179:0/2393487006 >> v2:172.21.15.195:3300/0 conn(0x7fa18015ef80 msgr2=0x7fa180161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 --2- 172.21.15.179:0/2393487006 >> v2:172.21.15.195:3300/0 conn(0x7fa18015ef80 0x7fa180161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 -- 172.21.15.179:0/2393487006 >> v1:172.21.15.179:6789/0 conn(0x7fa18006dca0 legacy=0x7fa1800b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 -- 172.21.15.179:0/2393487006 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180165260 con 0x7fa180164db0 2024-05-08T22:36:10.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18594f700 1 --2- 172.21.15.179:0/2393487006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa18006d760 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa180071ff0 tx=0x7fa170009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49920a10ffa639 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18514e700 1 -- 172.21.15.179:0/2393487006 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011830 con 0x7fa180164db0 2024-05-08T22:36:10.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18514e700 1 -- 172.21.15.179:0/2393487006 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170011990 con 0x7fa180164db0 2024-05-08T22:36:10.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa18514e700 1 -- 172.21.15.179:0/2393487006 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011d20 con 0x7fa180164db0 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- 172.21.15.179:0/2393487006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 msgr2=0x7fa18006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 --2- 172.21.15.179:0/2393487006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa18006d760 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa180071ff0 tx=0x7fa170009b30 comp rx=0 tx=0).stop 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- 172.21.15.179:0/2393487006 shutdown_connections 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 --2- 172.21.15.179:0/2393487006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa18006d760 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 --2- 172.21.15.179:0/2393487006 >> v2:172.21.15.195:3300/0 conn(0x7fa18015ef80 0x7fa180161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- 172.21.15.179:0/2393487006 >> 172.21.15.179:0/2393487006 conn(0x7fa1800ce9d0 msgr2=0x7fa1800cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- 172.21.15.179:0/2393487006 shutdown_connections 2024-05-08T22:36:10.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- 172.21.15.179:0/2393487006 wait complete. 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 Processor -- start 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.263+0000 7fa187152700 1 -- start start 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 0x7fa180158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa1800a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800a7570 con 0x7fa180164db0 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800a7850 con 0x7fa18006dca0 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800a7b30 con 0x7fa18015ef80 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa186951700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa1800a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa186150700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 0x7fa180158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:50434/0 (socket says 172.21.15.179:50434) 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa186150700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 0x7fa180158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:43814/0 (socket says 172.21.15.179:43814) 2024-05-08T22:36:10.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 -- 172.21.15.179:0/2379636274 learned_addr learned my addr 172.21.15.179:0/2379636274 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa186951700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa1800a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34158/0 (socket says 172.21.15.179:34158) 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 -- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 msgr2=0x7fa180158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 0x7fa180158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 -- 172.21.15.179:0/2379636274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 msgr2=0x7fa1800a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa1800a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 -- 172.21.15.179:0/2379636274 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700094b0 con 0x7fa18015ef80 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa18594f700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa170014040 tx=0x7fa170004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170004510 con 0x7fa18015ef80 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17001e070 con 0x7fa18015ef80 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1800a7e10 con 0x7fa18015ef80 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170003900 con 0x7fa18015ef80 2024-05-08T22:36:10.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180158cd0 con 0x7fa18015ef80 2024-05-08T22:36:10.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.267+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1800a8330 con 0x7fa18015ef80 2024-05-08T22:36:10.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.271+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa170003a60 con 0x7fa18015ef80 2024-05-08T22:36:10.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.271+0000 7fa176ffd700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.271+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa170085550 con 0x7fa18015ef80 2024-05-08T22:36:10.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.271+0000 7fa186150700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.271+0000 7fa186150700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009bf0 tx=0x7fa17c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.283+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1700526b0 con 0x7fa18015ef80 2024-05-08T22:36:10.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.451+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1800a8330 con 0x7fa18015ef80 2024-05-08T22:36:10.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.455+0000 7fa176ffd700 1 -- 172.21.15.179:0/2379636274 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16394 (secure 0 0 0) 0x7fa1800a8330 con 0x7fa18015ef80 2024-05-08T22:36:10.456 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:10.456 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":57,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:05.576343+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 msgr2=0x7fa164068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009bf0 tx=0x7fa17c009240 comp rx=0 tx=0).stop 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 msgr2=0x7fa1800abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa170014040 tx=0x7fa170004770 comp rx=0 tx=0).stop 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 shutdown_connections 2024-05-08T22:36:10.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180164db0 0x7fa1800a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18015ef80 0x7fa1800abe10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 --2- 172.21.15.179:0/2379636274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa18006dca0 0x7fa180158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 >> 172.21.15.179:0/2379636274 conn(0x7fa1800ce9d0 msgr2=0x7fa1800af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 shutdown_connections 2024-05-08T22:36:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.459+0000 7fa187152700 1 -- 172.21.15.179:0/2379636274 wait complete. 2024-05-08T22:36:10.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:25.626475+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:26.294593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:10.472 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:10.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 Processor -- start 2024-05-08T22:36:10.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 -- start start 2024-05-08T22:36:10.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd9dc15d0c0 0x7fd9dc0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd9dc0d1d20 0x7fd9dc0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc15d4a0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc0ce570 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9db59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc06abb0 con 0x7fd9dc0d1d20 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd9dc15d0c0 0x7fd9dc0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9dad9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd9dc0ce0e0 0x7fd9dc160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38870/0 (socket says 172.21.15.179:38870) 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9dad9d700 1 -- 172.21.15.179:0/742992942 learned_addr learned my addr 172.21.15.179:0/742992942 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9d1d9b700 1 --2- 172.21.15.179:0/742992942 >> v2:172.21.15.195:3300/0 conn(0x7fd9dc0d1d20 0x7fd9dc0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 -- 172.21.15.179:0/742992942 >> v2:172.21.15.195:3300/0 conn(0x7fd9dc0d1d20 msgr2=0x7fd9dc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 --2- 172.21.15.179:0/742992942 >> v2:172.21.15.195:3300/0 conn(0x7fd9dc0d1d20 0x7fd9dc0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 -- 172.21.15.179:0/742992942 >> v1:172.21.15.179:6789/0 conn(0x7fd9dc0ce0e0 legacy=0x7fd9dc160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 -- 172.21.15.179:0/742992942 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0d2500 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9d9d9b700 1 -- 172.21.15.179:0/742992942 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570699978 0 0) 0x7fd9dc15d4a0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9da59c700 1 --2- 172.21.15.179:0/742992942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0b8a80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0d3e20 tx=0x7fd9cc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bac005f041573b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9d9d9b700 1 -- 172.21.15.179:0/742992942 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc011830 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9d9d9b700 1 -- 172.21.15.179:0/742992942 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc011990 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.591+0000 7fd9d9d9b700 1 -- 172.21.15.179:0/742992942 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc011d20 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- 172.21.15.179:0/742992942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc15d0c0 msgr2=0x7fd9dc0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 --2- 172.21.15.179:0/742992942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0b8a80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0d3e20 tx=0x7fd9cc009b30 comp rx=0 tx=0).stop 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- 172.21.15.179:0/742992942 shutdown_connections 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 --2- 172.21.15.179:0/742992942 >> v2:172.21.15.195:3300/0 conn(0x7fd9dc0d1d20 0x7fd9dc0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 --2- 172.21.15.179:0/742992942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0b8a80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- 172.21.15.179:0/742992942 >> 172.21.15.179:0/742992942 conn(0x7fd9dc0d2ed0 msgr2=0x7fd9dc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- 172.21.15.179:0/742992942 shutdown_connections 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- 172.21.15.179:0/742992942 wait complete. 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 Processor -- start 2024-05-08T22:36:10.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.595+0000 7fd9db59e700 1 -- start start 2024-05-08T22:36:10.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 0x7fd9dc15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cc200 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc159680 con 0x7fd9dc15d0c0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cb960 con 0x7fd9dc0d1d20 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34186/0 (socket says 172.21.15.179:34186) 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 -- 172.21.15.179:0/3329714263 learned_addr learned my addr 172.21.15.179:0/3329714263 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9d1d9b700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 0x7fd9dc15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9dad9d700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 -- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 msgr2=0x7fd9dc15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 0x7fd9dc15f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 -- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 msgr2=0x7fd9dc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 -- 172.21.15.179:0/3329714263 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0094b0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9d1d9b700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 0x7fd9dc15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9dad9d700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9da59c700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc014040 tx=0x7fd9cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0038c0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc01e070 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc0cbc40 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc003c80 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.599+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc15fde0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc159470 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9cc02d450 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9d37fe700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 0x7fd9bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd9cc0851b0 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9d1d9b700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 0x7fd9bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.603+0000 7fd9d1d9b700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 0x7fd9bc068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc1f6ce0 tx=0x7fd9c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.607+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd9cc052430 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.775+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9dc15ff70 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.775+0000 7fd9d37fe700 1 -- 172.21.15.179:0/3329714263 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9cc055a50 con 0x7fd9dc0ce0e0 2024-05-08T22:36:10.777 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 msgr2=0x7fd9bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 0x7fd9bc068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc1f6ce0 tx=0x7fd9c4008040 comp rx=0 tx=0).stop 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 msgr2=0x7fd9dc145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc014040 tx=0x7fd9cc004770 comp rx=0 tx=0).stop 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 shutdown_connections 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9bc066730 0x7fd9bc068bf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9dc15d0c0 0x7fd9dc0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9dc0d1d20 0x7fd9dc15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 --2- 172.21.15.179:0/3329714263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9dc0ce0e0 0x7fd9dc145ab0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 >> 172.21.15.179:0/3329714263 conn(0x7fd9dc0d2ed0 msgr2=0x7fd9dc0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 shutdown_connections 2024-05-08T22:36:10.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.779+0000 7fd9db59e700 1 -- 172.21.15.179:0/3329714263 wait complete. 2024-05-08T22:36:10.792 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:36:10.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 Processor -- start 2024-05-08T22:36:10.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 -- start start 2024-05-08T22:36:10.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee5014ae30 0x7fee50165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee5006abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee50166090 con 0x7fee500b1a50 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5006e9d0 con 0x7fee5014ae30 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56a7a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5006dd50 con 0x7fee500b2de0 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee55a78700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee5014ae30 0x7fee50165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee55a78700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee5014ae30 0x7fee50165b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34190/0 (socket says 172.21.15.179:34190) 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee55a78700 1 -- 172.21.15.179:0/1825655735 learned_addr learned my addr 172.21.15.179:0/1825655735 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.911+0000 7fee56279700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee5006abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee55a78700 1 -- 172.21.15.179:0/1825655735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 msgr2=0x7fee5006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee55a78700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee5006abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee55a78700 1 -- 172.21.15.179:0/1825655735 >> v1:172.21.15.195:6789/0 conn(0x7fee500b1a50 legacy=0x7fee500b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee55a78700 1 -- 172.21.15.179:0/1825655735 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5006e030 con 0x7fee5014ae30 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee55a78700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5014ae30 0x7fee50165b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fee48009e50 tx=0x7fee4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6994dd8b3267dfb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee54a76700 1 -- 172.21.15.179:0/1825655735 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4800e070 con 0x7fee5014ae30 2024-05-08T22:36:10.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee54a76700 1 -- 172.21.15.179:0/1825655735 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4800b9c0 con 0x7fee5014ae30 2024-05-08T22:36:10.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee54a76700 1 -- 172.21.15.179:0/1825655735 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4800bd70 con 0x7fee5014ae30 2024-05-08T22:36:10.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- 172.21.15.179:0/1825655735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5014ae30 msgr2=0x7fee50165b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5014ae30 0x7fee50165b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fee48009e50 tx=0x7fee4800b330 comp rx=0 tx=0).stop 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- 172.21.15.179:0/1825655735 shutdown_connections 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee5006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 --2- 172.21.15.179:0/1825655735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5014ae30 0x7fee50165b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- 172.21.15.179:0/1825655735 >> 172.21.15.179:0/1825655735 conn(0x7fee500cea30 msgr2=0x7fee500cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- 172.21.15.179:0/1825655735 shutdown_connections 2024-05-08T22:36:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- 172.21.15.179:0/1825655735 wait complete. 2024-05-08T22:36:10.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 Processor -- start 2024-05-08T22:36:10.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.915+0000 7fee56a7a700 1 -- start start 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee501453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee5014ae30 0x7fee500a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee500a7cc0 con 0x7fee500b1a50 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee500a9840 con 0x7fee5014ae30 2024-05-08T22:36:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500a9b20 con 0x7fee500b2de0 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34192/0 (socket says 172.21.15.179:34192) 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 -- 172.21.15.179:0/2852402185 learned_addr learned my addr 172.21.15.179:0/2852402185 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56279700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee5014ae30 0x7fee500a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55277700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee501453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 -- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 msgr2=0x7fee501453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee501453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 -- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee5014ae30 msgr2=0x7fee500a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee5014ae30 0x7fee500a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 -- 172.21.15.179:0/2852402185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009b30 con 0x7fee500b1a50 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55277700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee501453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee55a78700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fee4800b370 tx=0x7fee48004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480092e0 con 0x7fee500b1a50 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4801b430 con 0x7fee500b1a50 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee500a7670 con 0x7fee500b1a50 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4801b7e0 con 0x7fee500b1a50 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.919+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee50160f60 con 0x7fee500b1a50 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee500a74d0 con 0x7fee500b1a50 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee4801b940 con 0x7fee500b1a50 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee467fc700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 0x7fee34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fee48085880 con 0x7fee500b1a50 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee55277700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 0x7fee34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.923+0000 7fee55277700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 0x7fee34068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee50160c60 tx=0x7fee40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:10.931+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fee500a74d0 con 0x7fee500b1a50 2024-05-08T22:36:11.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.095+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee500a74d0 con 0x7fee500b1a50 2024-05-08T22:36:11.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.095+0000 7fee467fc700 1 -- 172.21.15.179:0/2852402185 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee500a74d0 con 0x7fee500b1a50 2024-05-08T22:36:11.097 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:36:11.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 msgr2=0x7fee34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 0x7fee34068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee50160c60 tx=0x7fee40009040 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 msgr2=0x7fee500a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fee4800b370 tx=0x7fee48004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 shutdown_connections 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee34066730 0x7fee34068bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee5014ae30 0x7fee500a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee500b2de0 0x7fee501453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 --2- 172.21.15.179:0/2852402185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500b1a50 0x7fee500a8150 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 >> 172.21.15.179:0/2852402185 conn(0x7fee500cea30 msgr2=0x7fee5014a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 shutdown_connections 2024-05-08T22:36:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.099+0000 7fee56a7a700 1 -- 172.21.15.179:0/2852402185 wait complete. 2024-05-08T22:36:11.112 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:36:11.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 Processor -- start 2024-05-08T22:36:11.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 -- start start 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c06dd50 con 0x7fb23c14ae30 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c06e030 con 0x7fb23c0b2c10 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb24184a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c06e9d0 con 0x7fb23c0b1a50 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb240848700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb23c14ae30 0x7fb23c165b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38876/0 (socket says 172.21.15.179:38876) 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb240848700 1 -- 172.21.15.179:0/2493870849 learned_addr learned my addr 172.21.15.179:0/2493870849 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:11.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb23bfff700 1 --2- 172.21.15.179:0/2493870849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095923487 0 0) 0x7fb23c06dd50 con 0x7fb23c14ae30 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.235+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2240035d0 con 0x7fb23c14ae30 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541928246 0 0) 0x7fb23c06e030 con 0x7fb23c0b2c10 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb23c06dd50 con 0x7fb23c0b2c10 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23bfff700 1 -- 172.21.15.179:0/2493870849 >> v1:172.21.15.195:6789/0 conn(0x7fb23c0b2c10 legacy=0x7fb23c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23bfff700 1 -- 172.21.15.179:0/2493870849 >> v1:172.21.15.179:6789/0 conn(0x7fb23c14ae30 legacy=0x7fb23c165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23bfff700 1 -- 172.21.15.179:0/2493870849 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c06e310 con 0x7fb23c0b1a50 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23bfff700 1 --2- 172.21.15.179:0/2493870849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c0b26d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009880 tx=0x7fb22c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16ecd835d3423a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c011830 con 0x7fb23c0b1a50 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c011990 con 0x7fb23c0b1a50 2024-05-08T22:36:11.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb23b7fe700 1 -- 172.21.15.179:0/2493870849 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c011d00 con 0x7fb23c0b1a50 2024-05-08T22:36:11.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 -- 172.21.15.179:0/2493870849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 msgr2=0x7fb23c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 --2- 172.21.15.179:0/2493870849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c0b26d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009880 tx=0x7fb22c009b90 comp rx=0 tx=0).stop 2024-05-08T22:36:11.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 -- 172.21.15.179:0/2493870849 shutdown_connections 2024-05-08T22:36:11.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 --2- 172.21.15.179:0/2493870849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c0b26d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 -- 172.21.15.179:0/2493870849 >> 172.21.15.179:0/2493870849 conn(0x7fb23c0cea30 msgr2=0x7fb23c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 -- 172.21.15.179:0/2493870849 shutdown_connections 2024-05-08T22:36:11.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 -- 172.21.15.179:0/2493870849 wait complete. 2024-05-08T22:36:11.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.239+0000 7fb24184a700 1 Processor -- start 2024-05-08T22:36:11.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- start start 2024-05-08T22:36:11.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb23c14ae30 0x7fb23c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0b30c0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c15ef20 con 0x7fb23c14ae30 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c140e10 con 0x7fb23c0b1a50 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34196/0 (socket says 172.21.15.179:34196) 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 -- 172.21.15.179:0/2227727014 learned_addr learned my addr 172.21.15.179:0/2227727014 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb240848700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb241049700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb23c14ae30 0x7fb23c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 -- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 msgr2=0x7fb23c1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c1445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 -- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb23c14ae30 msgr2=0x7fb23c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb23c14ae30 0x7fb23c0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 -- 172.21.15.179:0/2227727014 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0094b0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb23bfff700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0090b0 tx=0x7fb22c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004610 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c0036c0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c1410f0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c003a70 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c1413d0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.243+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c15f060 con 0x7fb23c0b2c10 2024-05-08T22:36:11.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.247+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb22c003bd0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.247+0000 7fb2397fa700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 0x7fb228068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.247+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb22c0856f0 con 0x7fb23c0b2c10 2024-05-08T22:36:11.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.247+0000 7fb240848700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 0x7fb228068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.247+0000 7fb240848700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 0x7fb228068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb230000f60 tx=0x7fb230009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.251+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb23c15f060 con 0x7fb23c0b2c10 2024-05-08T22:36:11.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.423+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb23c15f060 con 0x7fb23c0b2c10 2024-05-08T22:36:11.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.423+0000 7fb2397fa700 1 -- 172.21.15.179:0/2227727014 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb23c15f060 con 0x7fb23c0b2c10 2024-05-08T22:36:11.424 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 msgr2=0x7fb228068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 0x7fb228068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb230000f60 tx=0x7fb230009040 comp rx=0 tx=0).stop 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 msgr2=0x7fb23c1f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0090b0 tx=0x7fb22c004770 comp rx=0 tx=0).stop 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 shutdown_connections 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb228066730 0x7fb228068bf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb23c14ae30 0x7fb23c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb23c0b2c10 0x7fb23c1f69b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 --2- 172.21.15.179:0/2227727014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb23c0b1a50 0x7fb23c1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 >> 172.21.15.179:0/2227727014 conn(0x7fb23c0cea30 msgr2=0x7fb23c13d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 shutdown_connections 2024-05-08T22:36:11.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.427+0000 7fb24184a700 1 -- 172.21.15.179:0/2227727014 wait complete. 2024-05-08T22:36:11.440 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:11.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.547+0000 7fd3bffff700 1 Processor -- start 2024-05-08T22:36:11.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.547+0000 7fd3bffff700 1 -- start start 2024-05-08T22:36:11.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd3c01435d0 0x7fd3c006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c006d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c01439b0 con 0x7fd3c006d670 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00ce240 con 0x7fd3c01435d0 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c006db00 con 0x7fd3c006ccd0 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd3c01435d0 0x7fd3c006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3be7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd3c01435d0 0x7fd3c006e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:43872/0 (socket says 172.21.15.179:43872) 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 -- 172.21.15.179:0/689292032 learned_addr learned my addr 172.21.15.179:0/689292032 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:11.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bf7fe700 1 --1- 172.21.15.179:0/689292032 >> v1:172.21.15.195:6789/0 conn(0x7fd3c006d670 0x7fd3c015d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:40306/0 (socket says 172.21.15.179:40306) 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 -- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 msgr2=0x7fd3c006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c006d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 -- 172.21.15.179:0/689292032 >> v1:172.21.15.195:6789/0 conn(0x7fd3c006d670 legacy=0x7fd3c015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bdffb700 1 -- 172.21.15.179:0/689292032 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443778699 0 0) 0x7fd3c01439b0 con 0x7fd3c006d670 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 -- 172.21.15.179:0/689292032 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c006efb0 con 0x7fd3c01435d0 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3be7fc700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3beffd700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c01435d0 0x7fd3c006e8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e50 tx=0x7fd3b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c749a73a4be061ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bdffb700 1 -- 172.21.15.179:0/689292032 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800e070 con 0x7fd3c01435d0 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bdffb700 1 -- 172.21.15.179:0/689292032 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b800b910 con 0x7fd3c01435d0 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bdffb700 1 -- 172.21.15.179:0/689292032 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800bcc0 con 0x7fd3c01435d0 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c01435d0 msgr2=0x7fd3c006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c01435d0 0x7fd3c006e8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e50 tx=0x7fd3b800b330 comp rx=0 tx=0).stop 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- 172.21.15.179:0/689292032 shutdown_connections 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 --2- 172.21.15.179:0/689292032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c01435d0 0x7fd3c006e8b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- 172.21.15.179:0/689292032 >> 172.21.15.179:0/689292032 conn(0x7fd3c00ce9d0 msgr2=0x7fd3c00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- 172.21.15.179:0/689292032 shutdown_connections 2024-05-08T22:36:11.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.551+0000 7fd3bffff700 1 -- 172.21.15.179:0/689292032 wait complete. 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 Processor -- start 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- start start 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 0x7fd3c00ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00ae3d0 con 0x7fd3c01435d0 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00ad8b0 con 0x7fd3c006d670 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00adb30 con 0x7fd3c006ccd0 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34208/0 (socket says 172.21.15.179:34208) 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 -- 172.21.15.179:0/2368588499 learned_addr learned my addr 172.21.15.179:0/2368588499 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3beffd700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3be7fc700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 0x7fd3c00ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 -- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 msgr2=0x7fd3c015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 -- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 msgr2=0x7fd3c00ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 0x7fd3c00ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 -- 172.21.15.179:0/2368588499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40094b0 con 0x7fd3c01435d0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3be7fc700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 0x7fd3c00ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bf7fe700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c550 tx=0x7fd3b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004ca0 con 0x7fd3c01435d0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0004e20 con 0x7fd3c01435d0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b8009b30 con 0x7fd3c01435d0 2024-05-08T22:36:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0007310 con 0x7fd3c01435d0 2024-05-08T22:36:11.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.555+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c00ae0f0 con 0x7fd3c01435d0 2024-05-08T22:36:11.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c0067740 con 0x7fd3c01435d0 2024-05-08T22:36:11.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd3b0007470 con 0x7fd3c01435d0 2024-05-08T22:36:11.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3a77fe700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 0x7fd3ac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd3b0083f40 con 0x7fd3c01435d0 2024-05-08T22:36:11.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3beffd700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 0x7fd3ac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.559+0000 7fd3beffd700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 0x7fd3ac068c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000f80 tx=0x7fd3b8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.563+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd3b00511c0 con 0x7fd3c01435d0 2024-05-08T22:36:11.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.771+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3c004e700 con 0x7fd3c01435d0 2024-05-08T22:36:11.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.771+0000 7fd3a77fe700 1 -- 172.21.15.179:0/2368588499 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7fd3b0007720 con 0x7fd3c01435d0 2024-05-08T22:36:11.773 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:11.773 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 msgr2=0x7fd3ac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 0x7fd3ac068c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000f80 tx=0x7fd3b8009000 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 msgr2=0x7fd3c0134220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c550 tx=0x7fd3b000c910 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 shutdown_connections 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd3ac066760 0x7fd3ac068c20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3c006d670 0x7fd3c00ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3c01435d0 0x7fd3c0134220 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 --2- 172.21.15.179:0/2368588499 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3c006ccd0 0x7fd3c015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 >> 172.21.15.179:0/2368588499 conn(0x7fd3c00ce9d0 msgr2=0x7fd3c00af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 shutdown_connections 2024-05-08T22:36:11.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.775+0000 7fd3bffff700 1 -- 172.21.15.179:0/2368588499 wait complete. 2024-05-08T22:36:11.777 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:36:11.788 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:11.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 Processor -- start 2024-05-08T22:36:11.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 -- start start 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79f01539c0 0x7f79f006b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f79f006bc50 0x7f79f00b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f0159b60 con 0x7f79f00b19e0 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f0159e40 con 0x7f79f006bc50 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79f4d3d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f015a120 con 0x7f79f01539c0 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.899+0000 7f79eeffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f79f006bc50 0x7f79f00b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79ef7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79f01539c0 0x7f79f006b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f79f006bc50 0x7f79f00b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34218/0 (socket says 172.21.15.179:34218) 2024-05-08T22:36:11.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 -- 172.21.15.179:0/3631019330 learned_addr learned my addr 172.21.15.179:0/3631019330 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:11.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 -- 172.21.15.179:0/3631019330 >> v2:172.21.15.195:3300/0 conn(0x7f79f01539c0 msgr2=0x7f79f006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:11.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 --2- 172.21.15.179:0/3631019330 >> v2:172.21.15.195:3300/0 conn(0x7f79f01539c0 0x7f79f006b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 -- 172.21.15.179:0/3631019330 >> v1:172.21.15.195:6789/0 conn(0x7f79f00b19e0 legacy=0x7f79f015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 -- 172.21.15.179:0/3631019330 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f015a400 con 0x7f79f006bc50 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79eeffd700 1 --2- 172.21.15.179:0/3631019330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f00b3010 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1650 tx=0x7f79d8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a1a898a20269c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79ee7fc700 1 -- 172.21.15.179:0/3631019330 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8011830 con 0x7f79f006bc50 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79ee7fc700 1 -- 172.21.15.179:0/3631019330 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8011990 con 0x7f79f006bc50 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79ee7fc700 1 -- 172.21.15.179:0/3631019330 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8011d20 con 0x7f79f006bc50 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3631019330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 msgr2=0x7f79f00b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3631019330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f00b3010 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1650 tx=0x7f79d8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3631019330 shutdown_connections 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3631019330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f00b3010 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3631019330 >> v2:172.21.15.195:3300/0 conn(0x7f79f01539c0 0x7f79f006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3631019330 >> 172.21.15.179:0/3631019330 conn(0x7f79f00ce9d0 msgr2=0x7f79f00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3631019330 shutdown_connections 2024-05-08T22:36:11.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.903+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3631019330 wait complete. 2024-05-08T22:36:11.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 Processor -- start 2024-05-08T22:36:11.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 -- start start 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f0147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f00b19e0 0x7f79f0145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 0x7f79f00a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0146500 con 0x7f79f006bc50 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01493c0 con 0x7f79f01539c0 2024-05-08T22:36:11.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79f4d3d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01496a0 con 0x7f79f00b19e0 2024-05-08T22:36:11.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.907+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f0147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f00b19e0 0x7f79f0145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 0x7f79f00a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f0147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34226/0 (socket says 172.21.15.179:34226) 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79ef7fe700 1 -- 172.21.15.179:0/3165879917 learned_addr learned my addr 172.21.15.179:0/3165879917 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 -- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f00b19e0 msgr2=0x7f79f0145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f00b19e0 0x7f79f0145920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 -- 172.21.15.179:0/3165879917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 msgr2=0x7f79f0147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f0147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 -- 172.21.15.179:0/3165879917 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8009570 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79effff700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 0x7f79f00a9d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79e8001eb0 tx=0x7f79e800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80052d0 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8005430 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f0149980 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80055a0 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f0149f40 con 0x7f79f01539c0 2024-05-08T22:36:11.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.911+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0067740 con 0x7f79f01539c0 2024-05-08T22:36:11.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.915+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f79e8003af0 con 0x7f79f01539c0 2024-05-08T22:36:11.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.915+0000 7f79e7fff700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 0x7f79d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.915+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f79e80837b0 con 0x7f79f01539c0 2024-05-08T22:36:11.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.915+0000 7f79ef7fe700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 0x7f79d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.915+0000 7f79ef7fe700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 0x7f79d4068bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79e00094c0 tx=0x7f79e000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:11.927+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f79e8050900 con 0x7f79f01539c0 2024-05-08T22:36:12.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.099+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f004e700 con 0x7f79f01539c0 2024-05-08T22:36:12.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.103+0000 7f79e7fff700 1 -- 172.21.15.179:0/3165879917 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16394 (secure 0 0 0) 0x7f79e8053f20 con 0x7f79f01539c0 2024-05-08T22:36:12.104 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:12.105 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":57,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:05.576343+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 msgr2=0x7f79d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 0x7f79d4068bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79e00094c0 tx=0x7f79e000b040 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 msgr2=0x7f79f00a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 0x7f79f00a9d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79e8001eb0 tx=0x7f79e800c6e0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 shutdown_connections 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f79d4066730 0x7f79d4068bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f01539c0 0x7f79f00a9d30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f00b19e0 0x7f79f0145920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 --2- 172.21.15.179:0/3165879917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f006bc50 0x7f79f0147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 >> 172.21.15.179:0/3165879917 conn(0x7f79f00ce9d0 msgr2=0x7f79f00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 shutdown_connections 2024-05-08T22:36:12.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.107+0000 7f79f4d3d700 1 -- 172.21.15.179:0/3165879917 wait complete. 2024-05-08T22:36:12.122 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:36:12.123 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:12.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 Processor -- start 2024-05-08T22:36:12.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 -- start start 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc790161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc79006de70 0x7fc79006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc790165d50 con 0x7fc790164db0 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79006ad70 con 0x7fc79006de70 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.231+0000 7fc78f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc790165230 con 0x7fc79015ef80 2024-05-08T22:36:12.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc79006de70 0x7fc79006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc79006de70 0x7fc79006b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34234/0 (socket says 172.21.15.179:34234) 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2927295607 learned_addr learned my addr 172.21.15.179:0/2927295607 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78e59c700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc790161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2927295607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 msgr2=0x7fc790161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc790161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2927295607 >> v1:172.21.15.195:6789/0 conn(0x7fc790164db0 legacy=0x7fc79006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2927295607 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79006e4b0 con 0x7fc79006de70 2024-05-08T22:36:12.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78ed9d700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79006de70 0x7fc79006b200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc788009e50 tx=0x7fc78800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704eb6d0be4f4536 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78d59a700 1 -- 172.21.15.179:0/2927295607 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800e070 con 0x7fc79006de70 2024-05-08T22:36:12.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78d59a700 1 -- 172.21.15.179:0/2927295607 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800b910 con 0x7fc79006de70 2024-05-08T22:36:12.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78d59a700 1 -- 172.21.15.179:0/2927295607 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800bcc0 con 0x7fc79006de70 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 -- 172.21.15.179:0/2927295607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79006de70 msgr2=0x7fc79006b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79006de70 0x7fc79006b200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc788009e50 tx=0x7fc78800b330 comp rx=0 tx=0).stop 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 -- 172.21.15.179:0/2927295607 shutdown_connections 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79006de70 0x7fc79006b200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2927295607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc790161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 -- 172.21.15.179:0/2927295607 >> 172.21.15.179:0/2927295607 conn(0x7fc7900ce9d0 msgr2=0x7fc7900cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 -- 172.21.15.179:0/2927295607 shutdown_connections 2024-05-08T22:36:12.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.235+0000 7fc78f59e700 1 -- 172.21.15.179:0/2927295607 wait complete. 2024-05-08T22:36:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 Processor -- start 2024-05-08T22:36:12.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- start start 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79006de70 0x7fc790157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc7900ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900cdd60 con 0x7fc790164db0 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79015f8f0 con 0x7fc79006de70 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901590f0 con 0x7fc79015ef80 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34248/0 (socket says 172.21.15.179:34248) 2024-05-08T22:36:12.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2191559467 learned_addr learned my addr 172.21.15.179:0/2191559467 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78e59c700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79006de70 0x7fc790157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78dd9b700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc7900ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 msgr2=0x7fc7900ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc7900ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79006de70 msgr2=0x7fc790157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79006de70 0x7fc790157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 -- 172.21.15.179:0/2191559467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009b30 con 0x7fc790164db0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78ed9d700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b680 tx=0x7fc788004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880048b0 con 0x7fc790164db0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7880099b0 con 0x7fc790164db0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900cd4c0 con 0x7fc790164db0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78801c680 con 0x7fc790164db0 2024-05-08T22:36:12.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.239+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900cda80 con 0x7fc790164db0 2024-05-08T22:36:12.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.243+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790067740 con 0x7fc790164db0 2024-05-08T22:36:12.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.243+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc78801c7e0 con 0x7fc790164db0 2024-05-08T22:36:12.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.243+0000 7fc786ffd700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.243+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc788086460 con 0x7fc790164db0 2024-05-08T22:36:12.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.243+0000 7fc78e59c700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.247+0000 7fc78e59c700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 0x7fc770068bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc780009bf0 tx=0x7fc780009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.247+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7880536c0 con 0x7fc790164db0 2024-05-08T22:36:12.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.451+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7900af070 con 0x7fc790164db0 2024-05-08T22:36:12.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.451+0000 7fc786ffd700 1 -- 172.21.15.179:0/2191559467 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7fc788056ce0 con 0x7fc790164db0 2024-05-08T22:36:12.452 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:12.452 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:36:12.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 msgr2=0x7fc770068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 0x7fc770068bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc780009bf0 tx=0x7fc780009240 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 msgr2=0x7fc790163cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b680 tx=0x7fc788004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 shutdown_connections 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc790164db0 0x7fc790163cc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc79015ef80 0x7fc7900ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 --2- 172.21.15.179:0/2191559467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc79006de70 0x7fc790157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 >> 172.21.15.179:0/2191559467 conn(0x7fc7900ce9d0 msgr2=0x7fc7900b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 shutdown_connections 2024-05-08T22:36:12.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.455+0000 7fc78f59e700 1 -- 172.21.15.179:0/2191559467 wait complete. 2024-05-08T22:36:12.456 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:36:12.466 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:12.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.579+0000 7f2efe564700 1 Processor -- start 2024-05-08T22:36:12.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.579+0000 7f2efe564700 1 -- start start 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef80ce060 0x7f2ef806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ef80cc310 0x7f2ef80ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef8161ac0 con 0x7f2ef80add40 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef80ae550 con 0x7f2ef80ce060 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef806abb0 con 0x7f2ef80cc310 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef80ce060 0x7f2ef806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef77fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ef80cc310 0x7f2ef80ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef80ce060 0x7f2ef806ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34254/0 (socket says 172.21.15.179:34254) 2024-05-08T22:36:12.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 -- 172.21.15.179:0/3068243857 learned_addr learned my addr 172.21.15.179:0/3068243857 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 -- 172.21.15.179:0/3068243857 >> v2:172.21.15.195:3300/0 conn(0x7f2ef80cc310 msgr2=0x7f2ef80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:12.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 --2- 172.21.15.179:0/3068243857 >> v2:172.21.15.195:3300/0 conn(0x7f2ef80cc310 0x7f2ef80ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 -- 172.21.15.179:0/3068243857 >> v1:172.21.15.195:6789/0 conn(0x7f2ef80add40 legacy=0x7f2ef806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 -- 172.21.15.179:0/3068243857 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80ce980 con 0x7f2ef80ce060 2024-05-08T22:36:12.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef7fff700 1 --2- 172.21.15.179:0/3068243857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80ce060 0x7f2ef806ce50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8073c10 tx=0x7f2eec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52788569831ce830 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef6ffd700 1 -- 172.21.15.179:0/3068243857 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec011830 con 0x7f2ef80ce060 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef6ffd700 1 -- 172.21.15.179:0/3068243857 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec011990 con 0x7f2ef80ce060 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2ef6ffd700 1 -- 172.21.15.179:0/3068243857 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec011d20 con 0x7f2ef80ce060 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- 172.21.15.179:0/3068243857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80ce060 msgr2=0x7f2ef806ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 --2- 172.21.15.179:0/3068243857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80ce060 0x7f2ef806ce50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8073c10 tx=0x7f2eec009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- 172.21.15.179:0/3068243857 shutdown_connections 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 --2- 172.21.15.179:0/3068243857 >> v2:172.21.15.195:3300/0 conn(0x7f2ef80cc310 0x7f2ef80ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 --2- 172.21.15.179:0/3068243857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80ce060 0x7f2ef806ce50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- 172.21.15.179:0/3068243857 >> 172.21.15.179:0/3068243857 conn(0x7f2ef80cb660 msgr2=0x7f2ef80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- 172.21.15.179:0/3068243857 shutdown_connections 2024-05-08T22:36:12.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.583+0000 7f2efe564700 1 -- 172.21.15.179:0/3068243857 wait complete. 2024-05-08T22:36:12.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 Processor -- start 2024-05-08T22:36:12.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- start start 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef80cc310 0x7f2ef80c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef80ce060 0x7f2ef80c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80c5150 con 0x7f2ef80add40 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80ab290 con 0x7f2ef80cc310 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef81f6e20 con 0x7f2ef80ce060 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34264/0 (socket says 172.21.15.179:34264) 2024-05-08T22:36:12.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 -- 172.21.15.179:0/207390832 learned_addr learned my addr 172.21.15.179:0/207390832 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef77fe700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef80cc310 0x7f2ef80c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efcb01700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef80ce060 0x7f2ef80c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 -- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef80ce060 msgr2=0x7f2ef80c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef80ce060 0x7f2ef80c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 -- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef80cc310 msgr2=0x7f2ef80c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef80cc310 0x7f2ef80c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 -- 172.21.15.179:0/207390832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec009570 con 0x7f2ef80add40 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef7fff700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8067e10 tx=0x7f2eec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec004400 con 0x7f2ef80add40 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec01f070 con 0x7f2ef80add40 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef80a9a00 con 0x7f2ef80add40 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0036c0 con 0x7f2ef80add40 2024-05-08T22:36:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.587+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef814ce30 con 0x7f2ef80add40 2024-05-08T22:36:12.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef8067770 con 0x7f2ef80add40 2024-05-08T22:36:12.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2eec021030 con 0x7f2ef80add40 2024-05-08T22:36:12.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2ef4ff9700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 0x7f2ee4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2eec085220 con 0x7f2ef80add40 2024-05-08T22:36:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2ef77fe700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 0x7f2ee4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.591+0000 7f2ef77fe700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 0x7f2ee4068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80c5400 tx=0x7f2ee800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.599+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2eec052530 con 0x7f2ef80add40 2024-05-08T22:36:12.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.807+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ef814c980 con 0x7f2ef80add40 2024-05-08T22:36:12.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.807+0000 7f2ef4ff9700 1 -- 172.21.15.179:0/207390832 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f2eec055b50 con 0x7f2ef80add40 2024-05-08T22:36:12.809 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:12.809 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:35:27.295084+0000","modified":"2024-05-08T22:35:30.308670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55412":{"gid":55412,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/1172092943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1172092943},{"type":"v1","addr":"172.21.15.195:6837","nonce":1172092943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 msgr2=0x7f2ee4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 0x7f2ee4068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80c5400 tx=0x7f2ee800b040 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 msgr2=0x7f2ef80c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8067e10 tx=0x7f2eec004770 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 shutdown_connections 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ee4066730 0x7f2ee4068bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef80cc310 0x7f2ef80c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef80ce060 0x7f2ef80c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 --2- 172.21.15.179:0/207390832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef80add40 0x7f2ef80c4c10 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 >> 172.21.15.179:0/207390832 conn(0x7f2ef80cb660 msgr2=0x7f2ef80c7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 shutdown_connections 2024-05-08T22:36:12.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.811+0000 7f2efe564700 1 -- 172.21.15.179:0/207390832 wait complete. 2024-05-08T22:36:12.813 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 24 2024-05-08T22:36:12.824 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:12.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 Processor -- start 2024-05-08T22:36:12.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 -- start start 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdf8c153b00 0x7fdf8c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c06b440 0x7fdf8c06b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdf8c15cef0 0x7fdf8c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8c06be30 con 0x7fdf8c153b00 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8c0b1b10 con 0x7fdf8c15cef0 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf92493700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0b1db0 con 0x7fdf8c06b440 2024-05-08T22:36:12.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdf8c153b00 0x7fdf8c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdf8c153b00 0x7fdf8c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34276/0 (socket says 172.21.15.179:34276) 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 -- 172.21.15.179:0/910339278 learned_addr learned my addr 172.21.15.179:0/910339278 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf90c90700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c06b440 0x7fdf8c06b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91c92700 1 --2- 172.21.15.179:0/910339278 >> v2:172.21.15.195:3300/0 conn(0x7fdf8c15cef0 0x7fdf8c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 -- 172.21.15.179:0/910339278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c06b440 msgr2=0x7fdf8c06b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c06b440 0x7fdf8c06b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 -- 172.21.15.179:0/910339278 >> v2:172.21.15.195:3300/0 conn(0x7fdf8c15cef0 msgr2=0x7fdf8c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 --2- 172.21.15.179:0/910339278 >> v2:172.21.15.195:3300/0 conn(0x7fdf8c15cef0 0x7fdf8c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 -- 172.21.15.179:0/910339278 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c0b2090 con 0x7fdf8c153b00 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf91491700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c153b00 0x7fdf8c0b31a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c070a30 tx=0x7fdf7c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946aa25ff7ec7388 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf83fff700 1 -- 172.21.15.179:0/910339278 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c011830 con 0x7fdf8c153b00 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf83fff700 1 -- 172.21.15.179:0/910339278 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c011990 con 0x7fdf8c153b00 2024-05-08T22:36:12.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.935+0000 7fdf83fff700 1 -- 172.21.15.179:0/910339278 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c011d20 con 0x7fdf8c153b00 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- 172.21.15.179:0/910339278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c153b00 msgr2=0x7fdf8c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c153b00 0x7fdf8c0b31a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c070a30 tx=0x7fdf7c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- 172.21.15.179:0/910339278 shutdown_connections 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- 172.21.15.179:0/910339278 >> v2:172.21.15.195:3300/0 conn(0x7fdf8c15cef0 0x7fdf8c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c06b440 0x7fdf8c06b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- 172.21.15.179:0/910339278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c153b00 0x7fdf8c0b31a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- 172.21.15.179:0/910339278 >> 172.21.15.179:0/910339278 conn(0x7fdf8c0ce9d0 msgr2=0x7fdf8c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- 172.21.15.179:0/910339278 shutdown_connections 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- 172.21.15.179:0/910339278 wait complete. 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 Processor -- start 2024-05-08T22:36:12.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- start start 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf8c153b00 0x7fdf8c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 0x7fdf8c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c06d210 con 0x7fdf8c06b440 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c06e070 con 0x7fdf8c153b00 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf92493700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c06d460 con 0x7fdf8c15cef0 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf91491700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf91491700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34280/0 (socket says 172.21.15.179:34280) 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf91491700 1 -- 172.21.15.179:0/2037475115 learned_addr learned my addr 172.21.15.179:0/2037475115 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf91c92700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 0x7fdf8c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.939+0000 7fdf90c90700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf8c153b00 0x7fdf8c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 -- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 msgr2=0x7fdf8c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 0x7fdf8c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 -- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf8c153b00 msgr2=0x7fdf8c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf8c153b00 0x7fdf8c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 -- 172.21.15.179:0/2037475115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c009570 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91c92700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 0x7fdf8c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf91491700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009f40 tx=0x7fdf7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004240 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c01f070 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004600 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c06d740 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c06dd00 con 0x7fdf8c06b440 2024-05-08T22:36:12.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.943+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c067740 con 0x7fdf8c06b440 2024-05-08T22:36:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.947+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf7c021030 con 0x7fdf8c06b440 2024-05-08T22:36:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.947+0000 7fdf81ffb700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 0x7fdf78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.947+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fdf7c085290 con 0x7fdf8c06b440 2024-05-08T22:36:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.947+0000 7fdf90c90700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 0x7fdf78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.947+0000 7fdf90c90700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 0x7fdf78068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ceeb0 tx=0x7fdf8800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:12.955+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf7c0525d0 con 0x7fdf8c06b440 2024-05-08T22:36:13.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.127+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf8c04e700 con 0x7fdf8c06b440 2024-05-08T22:36:13.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf81ffb700 1 -- 172.21.15.179:0/2037475115 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16394 (secure 0 0 0) 0x7fdf7c055bf0 con 0x7fdf8c06b440 2024-05-08T22:36:13.131 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:13.131 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":57,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:05.576343+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:25.626475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:26.294593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 msgr2=0x7fdf78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 0x7fdf78068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ceeb0 tx=0x7fdf8800b040 comp rx=0 tx=0).stop 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 msgr2=0x7fdf8c1400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009f40 tx=0x7fdf7c004770 comp rx=0 tx=0).stop 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 shutdown_connections 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdf78066730 0x7fdf78068bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf8c153b00 0x7fdf8c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf8c15cef0 0x7fdf8c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 --2- 172.21.15.179:0/2037475115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8c06b440 0x7fdf8c1400c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 >> 172.21.15.179:0/2037475115 conn(0x7fdf8c0ce9d0 msgr2=0x7fdf8c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 shutdown_connections 2024-05-08T22:36:13.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.131+0000 7fdf92493700 1 -- 172.21.15.179:0/2037475115 wait complete. 2024-05-08T22:36:13.146 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:36:13.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.263+0000 7f2b3f59e700 1 Processor -- start 2024-05-08T22:36:13.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.263+0000 7f2b3f59e700 1 -- start start 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b400b2a50 0x7f2b400b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 0x7f2b400ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b400b3370 con 0x7f2b400ca360 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b400cbb50 con 0x7f2b400b2a50 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b400caed0 con 0x7f2b400c5b40 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b400b2a50 0x7f2b400b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b400b2a50 0x7f2b400b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34284/0 (socket says 172.21.15.179:34284) 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 -- 172.21.15.179:0/231366202 learned_addr learned my addr 172.21.15.179:0/231366202 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3ed9d700 1 --2- 172.21.15.179:0/231366202 >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 0x7f2b400ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 -- 172.21.15.179:0/231366202 >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 msgr2=0x7f2b400ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 --2- 172.21.15.179:0/231366202 >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 0x7f2b400ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 -- 172.21.15.179:0/231366202 >> v1:172.21.15.195:6789/0 conn(0x7f2b400ca360 legacy=0x7f2b4015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 -- 172.21.15.179:0/231366202 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400cb1b0 con 0x7f2b400b2a50 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3ed9d700 1 --2- 172.21.15.179:0/231366202 >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 0x7f2b400ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3e59c700 1 --2- 172.21.15.179:0/231366202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400b2a50 0x7f2b400b2e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2b4006fdc0 tx=0x7f2b30009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4187b61c636c41e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3d59a700 1 -- 172.21.15.179:0/231366202 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000b8a0 con 0x7f2b400b2a50 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3d59a700 1 -- 172.21.15.179:0/231366202 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3000ba00 con 0x7f2b400b2a50 2024-05-08T22:36:13.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3d59a700 1 -- 172.21.15.179:0/231366202 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000bd90 con 0x7f2b400b2a50 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- 172.21.15.179:0/231366202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400b2a50 msgr2=0x7f2b400b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/231366202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400b2a50 0x7f2b400b2e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2b4006fdc0 tx=0x7f2b30009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- 172.21.15.179:0/231366202 shutdown_connections 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/231366202 >> v2:172.21.15.195:3300/0 conn(0x7f2b400c5b40 0x7f2b400ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/231366202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400b2a50 0x7f2b400b2e30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- 172.21.15.179:0/231366202 >> 172.21.15.179:0/231366202 conn(0x7f2b400cfb00 msgr2=0x7f2b400cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- 172.21.15.179:0/231366202 shutdown_connections 2024-05-08T22:36:13.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.267+0000 7f2b3f59e700 1 -- 172.21.15.179:0/231366202 wait complete. 2024-05-08T22:36:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 Processor -- start 2024-05-08T22:36:13.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 -- start start 2024-05-08T22:36:13.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b400b2a50 0x7f2b40143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b400c5b40 0x7f2b401f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400acd20 con 0x7f2b400ca360 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40143ea0 con 0x7f2b400b2a50 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400abc80 con 0x7f2b400c5b40 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34294/0 (socket says 172.21.15.179:34294) 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3ed9d700 1 -- 172.21.15.179:0/1828895472 learned_addr learned my addr 172.21.15.179:0/1828895472 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3dd9b700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b400c5b40 0x7f2b401f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.271+0000 7f2b3e59c700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b400b2a50 0x7f2b40143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 -- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b400c5b40 msgr2=0x7f2b401f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b400c5b40 0x7f2b401f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 -- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b400b2a50 msgr2=0x7f2b40143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b400b2a50 0x7f2b40143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 -- 172.21.15.179:0/1828895472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009570 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3ed9d700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2b38007e70 tx=0x7f2b3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800dd20 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800f040 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b400abf60 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38015640 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400ac520 con 0x7f2b400ca360 2024-05-08T22:36:13.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b400b3e50 con 0x7f2b400ca360 2024-05-08T22:36:13.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.275+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b3801e3b0 con 0x7f2b400ca360 2024-05-08T22:36:13.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.279+0000 7f2b2effd700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 0x7f2b20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.279+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2b38084240 con 0x7f2b400ca360 2024-05-08T22:36:13.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.279+0000 7f2b3e59c700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 0x7f2b20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.279+0000 7f2b3e59c700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 0x7f2b20068c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000c00 tx=0x7f2b30008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.287+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2b380514c0 con 0x7f2b400ca360 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.499+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2b4004e700 con 0x7f2b400ca360 2024-05-08T22:36:13.501 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:36:13.499+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:36:13.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.515+0000 7f2b2effd700 1 -- 172.21.15.179:0/1828895472 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2b38054ae0 con 0x7f2b400ca360 2024-05-08T22:36:13.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 msgr2=0x7f2b20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 0x7f2b20068c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000c00 tx=0x7f2b30008030 comp rx=0 tx=0).stop 2024-05-08T22:36:13.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 msgr2=0x7f2b400ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2b38007e70 tx=0x7f2b3800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 shutdown_connections 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b20066760 0x7f2b20068c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b400ca360 0x7f2b400ad040 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b400c5b40 0x7f2b401f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 --2- 172.21.15.179:0/1828895472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b400b2a50 0x7f2b40143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 >> 172.21.15.179:0/1828895472 conn(0x7f2b400cfb00 msgr2=0x7f2b400cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 shutdown_connections 2024-05-08T22:36:13.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.519+0000 7f2b3f59e700 1 -- 172.21.15.179:0/1828895472 wait complete. 2024-05-08T22:36:13.520 INFO:tasks.ceph.mds.d.smithi195.stderr: -4746> 2024-05-08T22:35:28.239+0000 7f5d381a1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:36:13.520 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:36:13.522 INFO:tasks.ceph.mds.d.smithi195.stderr: -3034> 2024-05-08T22:35:36.619+0000 7f5d3499a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6142=0-4194304@1} caps={6142=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6142 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560385257700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:36:13.534 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:36:13.555 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.647+0000 7f6133520700 1 Processor -- start 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.647+0000 7f6133520700 1 -- start start 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f612c06c2d0 0x7f612c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c159b00 con 0x7f612c157040 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c159d80 con 0x7f612c06e6b0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c15a060 con 0x7f612c06c2d0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f612c157040 0x7f612c06bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:38892/0 (socket says 172.21.15.179:38892) 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 -- 172.21.15.179:0/2629250823 learned_addr learned my addr 172.21.15.179:0/2629250823 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6131d1d700 1 --2- 172.21.15.179:0/2629250823 >> v2:172.21.15.195:3300/0 conn(0x7f612c06c2d0 0x7f612c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57420241 0 0) 0x7f612c159b00 con 0x7f612c157040 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61180035d0 con 0x7f612c157040 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445469281 0 0) 0x7f61180035d0 con 0x7f612c157040 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 >> v2:172.21.15.195:3300/0 conn(0x7f612c06c2d0 msgr2=0x7f612c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 --2- 172.21.15.179:0/2629250823 >> v2:172.21.15.195:3300/0 conn(0x7f612c06c2d0 0x7f612c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 >> v1:172.21.15.195:6789/0 conn(0x7f612c06e6b0 legacy=0x7f612c06eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c15a340 con 0x7f612c157040 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f61240019f0 con 0x7f612c157040 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 >> v1:172.21.15.179:6789/0 conn(0x7f612c157040 legacy=0x7f612c06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f6118003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6118004870 0x7f6118004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6118005230 0x7f611800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c15a060 con 0x7f6118003a20 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611800e2e0 con 0x7f6118005230 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611800e4a0 con 0x7f6118004870 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f6118003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6131d1d700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6118004870 0x7f6118004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6132d1f700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6118005230 0x7f611800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 -- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6118004870 msgr2=0x7f6118004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6118004870 0x7f6118004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 -- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6118005230 msgr2=0x7f611800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6118005230 0x7f611800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6132d1f700 1 -- 172.21.15.179:0/2629250823 reap_dead start 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 -- 172.21.15.179:0/2629250823 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180035d0 con 0x7f6118003a20 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613251e700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f6118003e40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f612400a830 tx=0x7f61180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381deba9083e56b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400a2f0 con 0x7f6118003a20 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61240072f0 con 0x7f6118003a20 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f613151c700 1 -- 172.21.15.179:0/2629250823 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240076a0 con 0x7f6118003a20 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 msgr2=0x7f6118003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f6118003e40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f612400a830 tx=0x7f61180058a0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- 172.21.15.179:0/2629250823 shutdown_connections 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 --2- 172.21.15.179:0/2629250823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f6118003e40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.651+0000 7f6133520700 1 -- 172.21.15.179:0/2629250823 >> 172.21.15.179:0/2629250823 conn(0x7f612c070d80 msgr2=0x7f612c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.655+0000 7f6133520700 1 -- 172.21.15.179:0/2629250823 shutdown_connections 2024-05-08T22:36:13.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.655+0000 7f6133520700 1 -- 172.21.15.179:0/2629250823 wait complete. 2024-05-08T22:36:13.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.655+0000 7f6133520700 1 Processor -- start 2024-05-08T22:36:13.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.655+0000 7f6133520700 1 -- start start 2024-05-08T22:36:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612c14c710 0x7f612c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f612c14ed00 0x7f612c14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c14f6c0 con 0x7f6118003a20 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c14d890 con 0x7f612c14c710 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c14e8b0 con 0x7f612c14ed00 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34300/0 (socket says 172.21.15.179:34300) 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 -- 172.21.15.179:0/3673999533 learned_addr learned my addr 172.21.15.179:0/3673999533 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6132d1f700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f612c14ed00 0x7f612c14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6131d1d700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612c14c710 0x7f612c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 -- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f612c14ed00 msgr2=0x7f612c14f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f612c14ed00 0x7f612c14f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 -- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612c14c710 msgr2=0x7f612c14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612c14c710 0x7f612c14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 -- 172.21.15.179:0/3673999533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c14cc80 con 0x7f6118003a20 2024-05-08T22:36:13.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f613251e700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f611800c0a0 tx=0x7f6124001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240017f0 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124001970 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c14cf60 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124012640 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c14d520 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.659+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c04e700 con 0x7f6118003a20 2024-05-08T22:36:13.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.663+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f612401a070 con 0x7f6118003a20 2024-05-08T22:36:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.663+0000 7f6122ffd700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 0x7f6110068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.663+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f6124086070 con 0x7f6118003a20 2024-05-08T22:36:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.663+0000 7f6131d1d700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 0x7f6110068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.663+0000 7f6131d1d700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 0x7f6110068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f612c1f3720 tx=0x7f611c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.671+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61240532a0 con 0x7f6118003a20 2024-05-08T22:36:13.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:13.879+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f612c0b2e30 con 0x7f6118003a20 2024-05-08T22:36:14.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.519+0000 7f6122ffd700 1 -- 172.21.15.179:0/3673999533 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61240568c0 con 0x7f6118003a20 2024-05-08T22:36:14.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 msgr2=0x7f6110068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 0x7f6110068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f612c1f3720 tx=0x7f611c008040 comp rx=0 tx=0).stop 2024-05-08T22:36:14.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 msgr2=0x7f612c13f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f611800c0a0 tx=0x7f6124001d80 comp rx=0 tx=0).stop 2024-05-08T22:36:14.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 shutdown_connections 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6110066730 0x7f6110068bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612c14c710 0x7f612c14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f612c14ed00 0x7f612c14f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 --2- 172.21.15.179:0/3673999533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118003a20 0x7f612c13f640 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 >> 172.21.15.179:0/3673999533 conn(0x7f612c070d80 msgr2=0x7f612c0cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 shutdown_connections 2024-05-08T22:36:14.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.523+0000 7f6133520700 1 -- 172.21.15.179:0/3673999533 wait complete. 2024-05-08T22:36:14.542 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:36:14.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 Processor -- start 2024-05-08T22:36:14.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 -- start start 2024-05-08T22:36:14.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe2c81594a0 0x7fe2c80b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c80b3590 con 0x7fe2c80cc760 2024-05-08T22:36:14.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80b4b70 con 0x7fe2c81594a0 2024-05-08T22:36:14.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.671+0000 7fe2ce0df700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80b3870 con 0x7fe2c815d170 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c6ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c6ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c80b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:50584/0 (socket says 172.21.15.179:50584) 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c6ffd700 1 -- 172.21.15.179:0/4068705522 learned_addr learned my addr 172.21.15.179:0/4068705522 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 --2- 172.21.15.179:0/4068705522 >> v2:172.21.15.195:3300/0 conn(0x7fe2c81594a0 0x7fe2c80b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 -- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 msgr2=0x7fe2c80b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 --2- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c80b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 -- 172.21.15.179:0/4068705522 >> v1:172.21.15.195:6789/0 conn(0x7fe2c80cc760 legacy=0x7fe2c80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 -- 172.21.15.179:0/4068705522 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80b3b50 con 0x7fe2c81594a0 2024-05-08T22:36:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c77fe700 1 --2- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c81594a0 0x7fe2c80b4630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009850 tx=0x7fe2c0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dbe666e87758e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c67fc700 1 -- 172.21.15.179:0/4068705522 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0011830 con 0x7fe2c81594a0 2024-05-08T22:36:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c67fc700 1 -- 172.21.15.179:0/4068705522 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0011990 con 0x7fe2c81594a0 2024-05-08T22:36:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2c67fc700 1 -- 172.21.15.179:0/4068705522 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0011d40 con 0x7fe2c81594a0 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 -- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c81594a0 msgr2=0x7fe2c80b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c81594a0 0x7fe2c80b4630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009850 tx=0x7fe2c0009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 -- 172.21.15.179:0/4068705522 shutdown_connections 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c80b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/4068705522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c81594a0 0x7fe2c80b4630 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 -- 172.21.15.179:0/4068705522 >> 172.21.15.179:0/4068705522 conn(0x7fe2c80cb610 msgr2=0x7fe2c80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 -- 172.21.15.179:0/4068705522 shutdown_connections 2024-05-08T22:36:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.675+0000 7fe2ce0df700 1 -- 172.21.15.179:0/4068705522 wait complete. 2024-05-08T22:36:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 Processor -- start 2024-05-08T22:36:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 -- start start 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c80cc760 0x7fe2c814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c806b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c8150d20 con 0x7fe2c81594a0 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c8150110 con 0x7fe2c80cc760 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81503f0 con 0x7fe2c815d170 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34310/0 (socket says 172.21.15.179:34310) 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c80cc760 0x7fe2c814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 -- 172.21.15.179:0/1670389451 learned_addr learned my addr 172.21.15.179:0/1670389451 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 -- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 msgr2=0x7fe2c806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c806b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 -- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c80cc760 msgr2=0x7fe2c814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c80cc760 0x7fe2c814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 -- 172.21.15.179:0/1670389451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00094b0 con 0x7fe2c81594a0 2024-05-08T22:36:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2c6ffd700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e890 tx=0x7fe2b000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0009620 con 0x7fe2c81594a0 2024-05-08T22:36:14.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0013680 con 0x7fe2c81594a0 2024-05-08T22:36:14.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.679+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c81506d0 con 0x7fe2c81594a0 2024-05-08T22:36:14.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0013a30 con 0x7fe2c81594a0 2024-05-08T22:36:14.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c81519e0 con 0x7fe2c81594a0 2024-05-08T22:36:14.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c8150ed0 con 0x7fe2c81594a0 2024-05-08T22:36:14.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2b0013b90 con 0x7fe2c81594a0 2024-05-08T22:36:14.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2bffff700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 0x7fe2a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fe2b0084110 con 0x7fe2c81594a0 2024-05-08T22:36:14.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.683+0000 7fe2c77fe700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 0x7fe2a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.687+0000 7fe2c77fe700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 0x7fe2a8068bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00090b0 tx=0x7fe2c0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.695+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe2c8150ed0 con 0x7fe2c81594a0 2024-05-08T22:36:14.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:14.871+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2c8073290 con 0x7fe2c81594a0 2024-05-08T22:36:15.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.647+0000 7fe2bffff700 1 -- 172.21.15.179:0/1670389451 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe2b0054dd0 con 0x7fe2c81594a0 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 msgr2=0x7fe2a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 0x7fe2a8068bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00090b0 tx=0x7fe2c0008030 comp rx=0 tx=0).stop 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 msgr2=0x7fe2c806b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e890 tx=0x7fe2b000eba0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 shutdown_connections 2024-05-08T22:36:15.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe2a8066730 0x7fe2a8068bf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2c815d170 0x7fe2c806b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2c81594a0 0x7fe2c806b030 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 --2- 172.21.15.179:0/1670389451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2c80cc760 0x7fe2c814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 >> 172.21.15.179:0/1670389451 conn(0x7fe2c80cb610 msgr2=0x7fe2c806ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 shutdown_connections 2024-05-08T22:36:15.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.651+0000 7fe2ce0df700 1 -- 172.21.15.179:0/1670389451 wait complete. 2024-05-08T22:36:15.653 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:36:15.669 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:36:15.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 Processor -- start 2024-05-08T22:36:15.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 -- start start 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34f80b43c0 0x7f34f8161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f80cd3a0 con 0x7f34f815d640 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f815db90 con 0x7f34f815ad20 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34ffa8b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f8161b80 con 0x7f34f80b43c0 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34fe028700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f34f815d640 0x7f34f80cbba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43626/0 (socket says 172.21.15.179:43626) 2024-05-08T22:36:15.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34fe028700 1 -- 172.21.15.179:0/1713708633 learned_addr learned my addr 172.21.15.179:0/1713708633 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.795+0000 7f34fd026700 1 --2- 172.21.15.179:0/1713708633 >> v2:172.21.15.195:3300/0 conn(0x7f34f80b43c0 0x7f34f8161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822650257 0 0) 0x7f34f80cd3a0 con 0x7f34f815d640 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e40035d0 con 0x7f34f815d640 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138752366 0 0) 0x7f34f815db90 con 0x7f34f815ad20 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f80cd3a0 con 0x7f34f815ad20 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704739150 0 0) 0x7f34e40035d0 con 0x7f34f815d640 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 >> v2:172.21.15.195:3300/0 conn(0x7f34f80b43c0 msgr2=0x7f34f8161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 --2- 172.21.15.179:0/1713708633 >> v2:172.21.15.195:3300/0 conn(0x7f34f80b43c0 0x7f34f8161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 >> v1:172.21.15.195:6789/0 conn(0x7f34f815ad20 legacy=0x7f34f80b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f815de70 con 0x7f34f815d640 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f34f00019f0 con 0x7f34f815d640 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 >> v1:172.21.15.179:6789/0 conn(0x7f34f815d640 legacy=0x7f34f80cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e4003bb0 0x7f34e4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 0x7f34e4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4005230 0x7f34e400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e400c080 con 0x7f34e4005230 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e400c1f0 con 0x7f34e4004870 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e400c3b0 con 0x7f34e4003bb0 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 0x7f34e4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fe028700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4005230 0x7f34e400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd827700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e4003bb0 0x7f34e4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 -- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e4003bb0 msgr2=0x7f34e4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e4003bb0 0x7f34e4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 -- 172.21.15.179:0/1713708633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4005230 msgr2=0x7f34e400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4005230 0x7f34e400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fe028700 1 -- 172.21.15.179:0/1713708633 reap_dead start 2024-05-08T22:36:15.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 -- 172.21.15.179:0/1713708633 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40035d0 con 0x7f34e4004870 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fd026700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 0x7f34e4004cf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009b70 tx=0x7f34e40058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5994250333c4dfff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400e6c0 con 0x7f34e4004870 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400e820 con 0x7f34e4004870 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34fc825700 1 -- 172.21.15.179:0/1713708633 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400eb90 con 0x7f34e4004870 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 -- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 msgr2=0x7f34e4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 0x7f34e4004cf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009b70 tx=0x7f34e40058a0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 -- 172.21.15.179:0/1713708633 shutdown_connections 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/1713708633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e4004870 0x7f34e4004cf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 -- 172.21.15.179:0/1713708633 >> 172.21.15.179:0/1713708633 conn(0x7f34f80d0e60 msgr2=0x7f34f80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 -- 172.21.15.179:0/1713708633 shutdown_connections 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.799+0000 7f34ffa8b700 1 -- 172.21.15.179:0/1713708633 wait complete. 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 Processor -- start 2024-05-08T22:36:15.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- start start 2024-05-08T22:36:15.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 0x7f34f806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 0x7f34f8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8158980 con 0x7f34e4004870 2024-05-08T22:36:15.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806e750 con 0x7f34f81580e0 2024-05-08T22:36:15.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806c550 con 0x7f34f814e010 2024-05-08T22:36:15.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd026700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 0x7f34f806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49570/0 (socket says 172.21.15.179:49570) 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 -- 172.21.15.179:0/2213560936 learned_addr learned my addr 172.21.15.179:0/2213560936 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fe028700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 0x7f34f8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 -- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 msgr2=0x7f34f8158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 0x7f34f8158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 -- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 msgr2=0x7f34f806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 0x7f34f806cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 -- 172.21.15.179:0/2213560936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f806cb10 con 0x7f34e4004870 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd026700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 0x7f34f806cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fe028700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 0x7f34f8158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34fd827700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34e800f870 tx=0x7f34e800fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80052e0 con 0x7f34e4004870 2024-05-08T22:36:15.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e8005460 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80038e0 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f806e980 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f806c7a0 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.803+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f806eea0 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.807+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f34e8003a40 con 0x7f34e4004870 2024-05-08T22:36:15.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.807+0000 7f34ee7fc700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 0x7f34dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.807+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5735+0+0 (secure 0 0 0) 0x7f34e8084b50 con 0x7f34e4004870 2024-05-08T22:36:15.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.807+0000 7f34fd026700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 0x7f34dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.807+0000 7f34fd026700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 0x7f34dc068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f40090b0 tx=0x7f34f401e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:15.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.815+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f34e8051e10 con 0x7f34e4004870 2024-05-08T22:36:15.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:15.987+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34f8073290 con 0x7f34e4004870 2024-05-08T22:36:16.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.651+0000 7f34ee7fc700 1 -- 172.21.15.179:0/2213560936 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7f34e8055430 con 0x7f34e4004870 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 msgr2=0x7f34dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 0x7f34dc068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f40090b0 tx=0x7f34f401e040 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 msgr2=0x7f34f806e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34e800f870 tx=0x7f34e800fb80 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 shutdown_connections 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f34dc066730 0x7f34dc068bf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34f81580e0 0x7f34f806cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34f814e010 0x7f34f8158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 --2- 172.21.15.179:0/2213560936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e4004870 0x7f34f806e210 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 >> 172.21.15.179:0/2213560936 conn(0x7f34f80d0e60 msgr2=0x7f34f80b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:16.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 shutdown_connections 2024-05-08T22:36:16.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.655+0000 7f34ffa8b700 1 -- 172.21.15.179:0/2213560936 wait complete. 2024-05-08T22:36:16.658 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:36:16.675 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:36:16.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 Processor -- start 2024-05-08T22:36:16.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 -- start start 2024-05-08T22:36:16.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe4e415c0b0 0x7fe4e40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40cbfd0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40cb3c0 con 0x7fe4e41448b0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e40cd970 con 0x7fe4e415c0b0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ea555700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe4e40cc9a0 0x7fe4e406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43636/0 (socket says 172.21.15.179:43636) 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ea555700 1 -- 172.21.15.179:0/3747086611 learned_addr learned my addr 172.21.15.179:0/3747086611 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9553700 1 --2- 172.21.15.179:0/3747086611 >> v2:172.21.15.195:3300/0 conn(0x7fe4e415c0b0 0x7fe4e40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358087484 0 0) 0x7fe4e40cbfd0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d00035d0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35890514 0 0) 0x7fe4d00035d0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 >> v2:172.21.15.195:3300/0 conn(0x7fe4e415c0b0 msgr2=0x7fe4e40cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 --2- 172.21.15.179:0/3747086611 >> v2:172.21.15.195:3300/0 conn(0x7fe4e415c0b0 0x7fe4e40cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 >> v1:172.21.15.195:6789/0 conn(0x7fe4e41448b0 legacy=0x7fe4e40ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40cb6a0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe4dc0019f0 con 0x7fe4e40cc9a0 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 >> v1:172.21.15.179:6789/0 conn(0x7fe4e40cc9a0 legacy=0x7fe4e406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 0x7fe4d0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0004870 0x7fe4d0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d0005230 0x7fe4d000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d000c0a0 con 0x7fe4d0003a20 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d000c210 con 0x7fe4d0004870 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d000c3d0 con 0x7fe4d0005230 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 0x7fe4d0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9553700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0004870 0x7fe4d0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 -- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d0005230 msgr2=0x7fe4d000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4d0005230 0x7fe4d000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 -- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0004870 msgr2=0x7fe4d0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0004870 0x7fe4d0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 -- 172.21.15.179:0/3747086611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00035d0 con 0x7fe4d0003a20 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9553700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0004870 0x7fe4d0004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ea555700 1 -- 172.21.15.179:0/3747086611 reap_dead start 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 0x7fe4d0003e40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40017e0 tx=0x7fe4d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1c0e091b4c60bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40052d0 con 0x7fe4d0003a20 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4005430 con 0x7fe4d0003a20 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4e8d52700 1 -- 172.21.15.179:0/3747086611 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40055a0 con 0x7fe4d0003a20 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 -- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 msgr2=0x7fe4d0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.803+0000 7fe4ead56700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 0x7fe4d0003e40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40017e0 tx=0x7fe4d00058a0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- 172.21.15.179:0/3747086611 shutdown_connections 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 --2- 172.21.15.179:0/3747086611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4d0003a20 0x7fe4d0003e40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- 172.21.15.179:0/3747086611 >> 172.21.15.179:0/3747086611 conn(0x7fe4e4070d80 msgr2=0x7fe4e4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- 172.21.15.179:0/3747086611 shutdown_connections 2024-05-08T22:36:16.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- 172.21.15.179:0/3747086611 wait complete. 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 Processor -- start 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- start start 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0003a20 0x7fe4e40a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4e406e040 0x7fe4e415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415b990 con 0x7fe4e406dbe0 2024-05-08T22:36:16.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40a84f0 con 0x7fe4d0003a20 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ead56700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415abe0 con 0x7fe4e406e040 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49584/0 (socket says 172.21.15.179:49584) 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 -- 172.21.15.179:0/2045494413 learned_addr learned my addr 172.21.15.179:0/2045494413 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0003a20 0x7fe4e40a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4ea555700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4e406e040 0x7fe4e415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 -- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4e406e040 msgr2=0x7fe4e415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4e406e040 0x7fe4e415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 -- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0003a20 msgr2=0x7fe4e40a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0003a20 0x7fe4e40a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 -- 172.21.15.179:0/2045494413 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e415aec0 con 0x7fe4e406dbe0 2024-05-08T22:36:16.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4e9553700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0010820 tx=0x7fe4e0010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.807+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004220 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0004380 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e40a74d0 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00036c0 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e40a7a90 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e415c570 con 0x7fe4e406dbe0 2024-05-08T22:36:16.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4e0003820 con 0x7fe4e406dbe0 2024-05-08T22:36:16.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4da7fc700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 0x7fe4c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:16.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe4e0083dd0 con 0x7fe4e406dbe0 2024-05-08T22:36:16.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.811+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 0x7fe4c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:16.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.815+0000 7fe4e9d54700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 0x7fe4c8068bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4015040 tx=0x7fe4d4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:16.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.819+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe4e0051310 con 0x7fe4e406dbe0 2024-05-08T22:36:16.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:16.999+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe4e404e700 con 0x7fe4e406dbe0 2024-05-08T22:36:17.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.003+0000 7fe4da7fc700 1 -- 172.21.15.179:0/2045494413 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe4e0054930 con 0x7fe4e406dbe0 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.003+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 msgr2=0x7fe4c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.003+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 0x7fe4c8068bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4015040 tx=0x7fe4d4009030 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 msgr2=0x7fe4e415a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0010820 tx=0x7fe4e0010b30 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 shutdown_connections 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4c8066730 0x7fe4c8068bf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4e406e040 0x7fe4e415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4e406dbe0 0x7fe4e415a040 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 --2- 172.21.15.179:0/2045494413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4d0003a20 0x7fe4e40a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 >> 172.21.15.179:0/2045494413 conn(0x7fe4e4070d80 msgr2=0x7fe4e40b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 shutdown_connections 2024-05-08T22:36:17.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.007+0000 7fe4ead56700 1 -- 172.21.15.179:0/2045494413 wait complete. 2024-05-08T22:36:17.017 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-08T22:36:17.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.123+0000 7fdaae8f0700 1 Processor -- start 2024-05-08T22:36:17.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.123+0000 7fdaae8f0700 1 -- start start 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdaa806de70 0x7fdaa8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdaa8164db0 0x7fdaa806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa8165d50 con 0x7fdaa815ef80 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa806ad70 con 0x7fdaa8164db0 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa8165220 con 0x7fdaa806de70 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdaa8164db0 0x7fdaa806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdaa8164db0 0x7fdaa806b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49592/0 (socket says 172.21.15.179:49592) 2024-05-08T22:36:17.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3560435219 learned_addr learned my addr 172.21.15.179:0/3560435219 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaad0ed700 1 --2- 172.21.15.179:0/3560435219 >> v2:172.21.15.195:3300/0 conn(0x7fdaa806de70 0x7fdaa8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3560435219 >> v2:172.21.15.195:3300/0 conn(0x7fdaa806de70 msgr2=0x7fdaa8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 --2- 172.21.15.179:0/3560435219 >> v2:172.21.15.195:3300/0 conn(0x7fdaa806de70 0x7fdaa8161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3560435219 >> v1:172.21.15.195:6789/0 conn(0x7fdaa815ef80 legacy=0x7fdaa806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3560435219 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa806e4b0 con 0x7fdaa8164db0 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae0ef700 1 --2- 172.21.15.179:0/3560435219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa806b200 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b2020 tx=0x7fdaa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c738c2b1a218862 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaac8ec700 1 -- 172.21.15.179:0/3560435219 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e070 con 0x7fdaa8164db0 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaac8ec700 1 -- 172.21.15.179:0/3560435219 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa000b910 con 0x7fdaa8164db0 2024-05-08T22:36:17.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaac8ec700 1 -- 172.21.15.179:0/3560435219 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000bcc0 con 0x7fdaa8164db0 2024-05-08T22:36:17.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3560435219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 msgr2=0x7fdaa806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3560435219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa806b200 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b2020 tx=0x7fdaa000b330 comp rx=0 tx=0).stop 2024-05-08T22:36:17.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3560435219 shutdown_connections 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3560435219 >> v2:172.21.15.195:3300/0 conn(0x7fdaa806de70 0x7fdaa8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3560435219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa806b200 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3560435219 >> 172.21.15.179:0/3560435219 conn(0x7fdaa80ce9d0 msgr2=0x7fdaa80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3560435219 shutdown_connections 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3560435219 wait complete. 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 Processor -- start 2024-05-08T22:36:17.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.127+0000 7fdaae8f0700 1 -- start start 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaa806de70 0x7fdaa8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa815ef80 0x7fdaa8158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81eaf60 con 0x7fdaa8164db0 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81eac10 con 0x7fdaa815ef80 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81550d0 con 0x7fdaa806de70 2024-05-08T22:36:17.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49608/0 (socket says 172.21.15.179:49608) 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3687988641 learned_addr learned my addr 172.21.15.179:0/3687988641 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaad0ed700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa815ef80 0x7fdaa8158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaad8ee700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaa806de70 0x7fdaa8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaa806de70 msgr2=0x7fdaa8154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaa806de70 0x7fdaa8154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa815ef80 msgr2=0x7fdaa8158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa815ef80 0x7fdaa8158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 -- 172.21.15.179:0/3687988641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009b30 con 0x7fdaa8164db0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae0ef700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8072020 tx=0x7fdaa0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0004880 con 0x7fdaa8164db0 2024-05-08T22:36:17.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa00099b0 con 0x7fdaa8164db0 2024-05-08T22:36:17.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa8154ba0 con 0x7fdaa8164db0 2024-05-08T22:36:17.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa001b670 con 0x7fdaa8164db0 2024-05-08T22:36:17.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa8159a50 con 0x7fdaa8164db0 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.131+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa81eb0a0 con 0x7fdaa8164db0 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.135+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdaa001b7d0 con 0x7fdaa8164db0 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.135+0000 7fda9a7fc700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 0x7fda8c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.135+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fdaa0085450 con 0x7fdaa8164db0 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.135+0000 7fdaad8ee700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 0x7fda8c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.135+0000 7fdaad8ee700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 0x7fda8c068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8158010 tx=0x7fdaa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.143+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdaa81eb0a0 con 0x7fdaa8164db0 2024-05-08T22:36:17.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.375+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fdaa81eb0a0 con 0x7fdaa8164db0 2024-05-08T22:36:17.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.655+0000 7fda9a7fc700 1 -- 172.21.15.179:0/3687988641 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v168) v1 ==== 149+0+0 (secure 0 0 0) 0x7fdaa81eb0a0 con 0x7fdaa8164db0 2024-05-08T22:36:17.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 msgr2=0x7fda8c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 0x7fda8c068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8158010 tx=0x7fdaa4009040 comp rx=0 tx=0).stop 2024-05-08T22:36:17.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 msgr2=0x7fdaa81eb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8072020 tx=0x7fdaa0004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 shutdown_connections 2024-05-08T22:36:17.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda8c066760 0x7fda8c068c20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa815ef80 0x7fdaa8158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa8164db0 0x7fdaa81eb380 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 --2- 172.21.15.179:0/3687988641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaa806de70 0x7fdaa8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 >> 172.21.15.179:0/3687988641 conn(0x7fdaa80ce9d0 msgr2=0x7fdaa81606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 shutdown_connections 2024-05-08T22:36:17.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.659+0000 7fdaae8f0700 1 -- 172.21.15.179:0/3687988641 wait complete. 2024-05-08T22:36:17.677 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-08T22:36:17.677 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T22:36:17.677 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:36:17.678 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:17.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 Processor -- start 2024-05-08T22:36:17.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- start start 2024-05-08T22:36:17.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f059415ef80 0x7f0594161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f059406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f059406dca0 0x7f05940b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406e130 con 0x7f059406dca0 2024-05-08T22:36:17.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0594165d50 con 0x7f059415ef80 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059406ad70 con 0x7f0594164db0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f059406dca0 0x7f05940b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f059406dca0 0x7f05940b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49616/0 (socket says 172.21.15.179:49616) 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 -- 172.21.15.179:0/4081328282 learned_addr learned my addr 172.21.15.179:0/4081328282 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0592ffd700 1 --2- 172.21.15.179:0/4081328282 >> v2:172.21.15.195:3300/0 conn(0x7f059415ef80 0x7f0594161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05927fc700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f059406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 -- 172.21.15.179:0/4081328282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 msgr2=0x7f059406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f059406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 -- 172.21.15.179:0/4081328282 >> v2:172.21.15.195:3300/0 conn(0x7f059415ef80 msgr2=0x7f0594161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 --2- 172.21.15.179:0/4081328282 >> v2:172.21.15.195:3300/0 conn(0x7f059415ef80 0x7f0594161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 -- 172.21.15.179:0/4081328282 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594165230 con 0x7f059406dca0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f05937fe700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f05940b2a20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0594071ff0 tx=0x7f058c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8704f7ff125982c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0591ffb700 1 -- 172.21.15.179:0/4081328282 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011830 con 0x7f059406dca0 2024-05-08T22:36:17.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0591ffb700 1 -- 172.21.15.179:0/4081328282 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c011990 con 0x7f059406dca0 2024-05-08T22:36:17.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0591ffb700 1 -- 172.21.15.179:0/4081328282 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011d20 con 0x7f059406dca0 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- 172.21.15.179:0/4081328282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 msgr2=0x7f05940b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f05940b2a20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0594071ff0 tx=0x7f058c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- 172.21.15.179:0/4081328282 shutdown_connections 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f05940b2a20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- 172.21.15.179:0/4081328282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f059406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 --2- 172.21.15.179:0/4081328282 >> v2:172.21.15.195:3300/0 conn(0x7f059415ef80 0x7f0594161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- 172.21.15.179:0/4081328282 >> 172.21.15.179:0/4081328282 conn(0x7f05940ce9d0 msgr2=0x7f05940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.795+0000 7f0593fff700 1 -- 172.21.15.179:0/4081328282 shutdown_connections 2024-05-08T22:36:17.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.799+0000 7f0593fff700 1 -- 172.21.15.179:0/4081328282 wait complete. 2024-05-08T22:36:17.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.799+0000 7f0593fff700 1 Processor -- start 2024-05-08T22:36:17.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.799+0000 7f0593fff700 1 -- start start 2024-05-08T22:36:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 0x7f05941541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f05941628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594163950 con 0x7f059406dca0 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594162e10 con 0x7f059415ef80 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05941630b0 con 0x7f0594164db0 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f05927fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 0x7f05941541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49624/0 (socket says 172.21.15.179:49624) 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 -- 172.21.15.179:0/4121674272 learned_addr learned my addr 172.21.15.179:0/4121674272 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f05937fe700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f05941628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 -- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 msgr2=0x7f05941628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f05941628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 -- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 msgr2=0x7f05941541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 0x7f05941541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 -- 172.21.15.179:0/4121674272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c009570 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f05927fc700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 0x7f05941541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0592ffd700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f058800e940 tx=0x7f058800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588013680 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05880137e0 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588013b70 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594163390 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.803+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0594157e80 con 0x7f059406dca0 2024-05-08T22:36:17.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.807+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0594067740 con 0x7f059406dca0 2024-05-08T22:36:17.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.807+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0588013cd0 con 0x7f059406dca0 2024-05-08T22:36:17.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.807+0000 7f057b7fe700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 0x7f0580068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.807+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0588083fa0 con 0x7f059406dca0 2024-05-08T22:36:17.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.807+0000 7f05927fc700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 0x7f0580068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.811+0000 7f05927fc700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 0x7f0580068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0584009fd0 tx=0x7f058400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.815+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f05880514c0 con 0x7f059406dca0 2024-05-08T22:36:17.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.987+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f059404e700 con 0x7f059406dca0 2024-05-08T22:36:17.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.991+0000 7f057b7fe700 1 -- 172.21.15.179:0/4121674272 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f0588017090 con 0x7f059406dca0 2024-05-08T22:36:17.990 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:17.991 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":60,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:16.640437+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/1172092943":"2024-05-09T22:36:13.501978+0000","172.21.15.195:6836/1172092943":"2024-05-09T22:36:13.501978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 msgr2=0x7f0580068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 0x7f0580068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0584009fd0 tx=0x7f058400b040 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 msgr2=0x7f0594164530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f058800e940 tx=0x7f058800ec50 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 shutdown_connections 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0580066730 0x7f0580068bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f059415ef80 0x7f05941541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0594164db0 0x7f05941628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 --2- 172.21.15.179:0/4121674272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f059406dca0 0x7f0594164530 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 >> 172.21.15.179:0/4121674272 conn(0x7f05940ce9d0 msgr2=0x7f05940b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 shutdown_connections 2024-05-08T22:36:17.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:17.995+0000 7f0593fff700 1 -- 172.21.15.179:0/4121674272 wait complete. 2024-05-08T22:36:18.010 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:18.010 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:18.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.123+0000 7f75aced5700 1 Processor -- start 2024-05-08T22:36:18.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.123+0000 7f75aced5700 1 -- start start 2024-05-08T22:36:18.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a815d270 0x7f75a80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75a80af230 0x7f75a80ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80cef40 con 0x7f75a80c5eb0 2024-05-08T22:36:18.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a815d650 con 0x7f75a80af230 2024-05-08T22:36:18.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80ce330 con 0x7f75a815d270 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75a80af230 0x7f75a80ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a77fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a815d270 0x7f75a80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75a80af230 0x7f75a80ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:38534/0 (socket says 172.21.15.179:38534) 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 -- 172.21.15.179:0/3847682604 learned_addr learned my addr 172.21.15.179:0/3847682604 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 -- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a815d270 msgr2=0x7f75a80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 --2- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a815d270 0x7f75a80cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 -- 172.21.15.179:0/3847682604 >> v1:172.21.15.195:6789/0 conn(0x7f75a80c5eb0 legacy=0x7f75a80aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 -- 172.21.15.179:0/3847682604 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80ce610 con 0x7f75a80af230 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a67fc700 1 -- 172.21.15.179:0/3847682604 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570714831 0 0) 0x7f75a80cef40 con 0x7f75a80c5eb0 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a7fff700 1 --2- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a80af230 0x7f75a80ab8e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009e50 tx=0x7f75a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114249e1e8f854af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a67fc700 1 -- 172.21.15.179:0/3847682604 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000e070 con 0x7f75a80af230 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a67fc700 1 -- 172.21.15.179:0/3847682604 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000b910 con 0x7f75a80af230 2024-05-08T22:36:18.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75a67fc700 1 -- 172.21.15.179:0/3847682604 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000bcc0 con 0x7f75a80af230 2024-05-08T22:36:18.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a80af230 msgr2=0x7f75a80ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 --2- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a80af230 0x7f75a80ab8e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009e50 tx=0x7f75a000b330 comp rx=0 tx=0).stop 2024-05-08T22:36:18.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- 172.21.15.179:0/3847682604 shutdown_connections 2024-05-08T22:36:18.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 --2- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a80af230 0x7f75a80ab8e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 --2- 172.21.15.179:0/3847682604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a815d270 0x7f75a80cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- 172.21.15.179:0/3847682604 >> 172.21.15.179:0/3847682604 conn(0x7f75a80cc5d0 msgr2=0x7f75a80cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:18.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- 172.21.15.179:0/3847682604 shutdown_connections 2024-05-08T22:36:18.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.127+0000 7f75aced5700 1 -- 172.21.15.179:0/3847682604 wait complete. 2024-05-08T22:36:18.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 Processor -- start 2024-05-08T22:36:18.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 -- start start 2024-05-08T22:36:18.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 0x7f75a815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a815d270 0x7f75a80ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a815f710 con 0x7f75a80af230 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a815f9f0 con 0x7f75a815d270 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75aced5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a815fcd0 con 0x7f75a80c5eb0 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49638/0 (socket says 172.21.15.179:49638) 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75a77fe700 1 -- 172.21.15.179:0/308359806 learned_addr learned my addr 172.21.15.179:0/308359806 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:18.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75a7fff700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a815d270 0x7f75a80ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.131+0000 7f75a6ffd700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 0x7f75a815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 -- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 msgr2=0x7f75a815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 0x7f75a815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 -- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a815d270 msgr2=0x7f75a80ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a815d270 0x7f75a80ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 -- 172.21.15.179:0/308359806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a0009b30 con 0x7f75a80af230 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a6ffd700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 0x7f75a815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75a77fe700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f759800e840 tx=0x7f759800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980095a0 con 0x7f75a80af230 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598013680 con 0x7f75a80af230 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a815ffb0 con 0x7f75a80af230 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980139f0 con 0x7f75a80af230 2024-05-08T22:36:18.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a81f27b0 con 0x7f75a80af230 2024-05-08T22:36:18.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a804e700 con 0x7f75a80af230 2024-05-08T22:36:18.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.135+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7598013b50 con 0x7f75a80af230 2024-05-08T22:36:18.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.139+0000 7f759ffff700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 0x7f758c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.139+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7598083d50 con 0x7f75a80af230 2024-05-08T22:36:18.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.139+0000 7f75a6ffd700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 0x7f758c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.139+0000 7f75a6ffd700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 0x7f758c068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f75a81f5e70 tx=0x7f7590009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.147+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7598087030 con 0x7f75a80af230 2024-05-08T22:36:18.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.315+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75a80b04b0 con 0x7f75a80af230 2024-05-08T22:36:18.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.315+0000 7f759ffff700 1 -- 172.21.15.179:0/308359806 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7598054da0 con 0x7f75a80af230 2024-05-08T22:36:18.317 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:36:18.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 msgr2=0x7f758c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 0x7f758c068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f75a81f5e70 tx=0x7f7590009040 comp rx=0 tx=0).stop 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 msgr2=0x7f75a81f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f759800e840 tx=0x7f759800eb50 comp rx=0 tx=0).stop 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 shutdown_connections 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f758c066730 0x7f758c068bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a815d270 0x7f75a80ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75a80c5eb0 0x7f75a815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 --2- 172.21.15.179:0/308359806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75a80af230 0x7f75a81f2270 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 >> 172.21.15.179:0/308359806 conn(0x7f75a80cc5d0 msgr2=0x7f75a80ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:18.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 shutdown_connections 2024-05-08T22:36:18.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.319+0000 7f75aced5700 1 -- 172.21.15.179:0/308359806 wait complete. 2024-05-08T22:36:18.337 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-08T22:36:18.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 Processor -- start 2024-05-08T22:36:18.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 -- start start 2024-05-08T22:36:18.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 0x7ff4480ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff44815b1b0 0x7ff4480cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44806abb0 con 0x7ff4480cb510 2024-05-08T22:36:18.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4480cb980 con 0x7ff44815b1b0 2024-05-08T22:36:18.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff44d457700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44806ae90 con 0x7ff4480cc700 2024-05-08T22:36:18.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff4477fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff44815b1b0 0x7ff4480cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff4477fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff44815b1b0 0x7ff4480cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49644/0 (socket says 172.21.15.179:49644) 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.455+0000 7ff4477fe700 1 -- 172.21.15.179:0/3007434319 learned_addr learned my addr 172.21.15.179:0/3007434319 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff447fff700 1 --2- 172.21.15.179:0/3007434319 >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 0x7ff4480ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff4477fe700 1 -- 172.21.15.179:0/3007434319 >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 msgr2=0x7ff4480ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff4477fe700 1 --2- 172.21.15.179:0/3007434319 >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 0x7ff4480ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff4477fe700 1 -- 172.21.15.179:0/3007434319 >> v1:172.21.15.195:6789/0 conn(0x7ff4480cb510 legacy=0x7ff44815e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff4477fe700 1 -- 172.21.15.179:0/3007434319 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44815b9c0 con 0x7ff44815b1b0 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff447fff700 1 --2- 172.21.15.179:0/3007434319 >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 0x7ff4480ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:18.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff4477fe700 1 --2- 172.21.15.179:0/3007434319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff44815b1b0 0x7ff4480cafd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff448070800 tx=0x7ff430009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fa5a476e942774 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff446ffd700 1 -- 172.21.15.179:0/3007434319 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430004230 con 0x7ff44815b1b0 2024-05-08T22:36:18.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff446ffd700 1 -- 172.21.15.179:0/3007434319 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff430004390 con 0x7ff44815b1b0 2024-05-08T22:36:18.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff446ffd700 1 -- 172.21.15.179:0/3007434319 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4300036c0 con 0x7ff44815b1b0 2024-05-08T22:36:18.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- 172.21.15.179:0/3007434319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff44815b1b0 msgr2=0x7ff4480cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 --2- 172.21.15.179:0/3007434319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff44815b1b0 0x7ff4480cafd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff448070800 tx=0x7ff430009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- 172.21.15.179:0/3007434319 shutdown_connections 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 --2- 172.21.15.179:0/3007434319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff44815b1b0 0x7ff4480cafd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 --2- 172.21.15.179:0/3007434319 >> v2:172.21.15.195:3300/0 conn(0x7ff4480cc700 0x7ff4480ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- 172.21.15.179:0/3007434319 >> 172.21.15.179:0/3007434319 conn(0x7ff4480ce9d0 msgr2=0x7ff4480cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- 172.21.15.179:0/3007434319 shutdown_connections 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- 172.21.15.179:0/3007434319 wait complete. 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 Processor -- start 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.459+0000 7ff44d457700 1 -- start start 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 0x7ff44813ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 0x7ff4480ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481401c0 con 0x7ff4480cc700 2024-05-08T22:36:18.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44813cf40 con 0x7ff4480cb510 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44814d6a0 con 0x7ff44815b1b0 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49650/0 (socket says 172.21.15.179:49650) 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 -- 172.21.15.179:0/546138399 learned_addr learned my addr 172.21.15.179:0/546138399 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff447fff700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 0x7ff44813ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44cc56700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 0x7ff4480ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 -- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 msgr2=0x7ff4480ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 0x7ff4480ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 -- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 msgr2=0x7ff44813ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 0x7ff44813ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 -- 172.21.15.179:0/546138399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff430009570 con 0x7ff4480cc700 2024-05-08T22:36:18.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff447fff700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 0x7ff44813ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44cc56700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 0x7ff4480ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff4477fe700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff4480b3e80 tx=0x7ff430004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430003e60 con 0x7ff4480cc700 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43001f070 con 0x7ff4480cc700 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44813f400 con 0x7ff4480cc700 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43000b8a0 con 0x7ff4480cc700 2024-05-08T22:36:18.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44813f9c0 con 0x7ff4480cc700 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.463+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff430021030 con 0x7ff4480cc700 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.467+0000 7ff444ff9700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 0x7ff42c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.467+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448067740 con 0x7ff4480cc700 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.467+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff430085640 con 0x7ff4480cc700 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.467+0000 7ff447fff700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 0x7ff42c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:18.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.467+0000 7ff447fff700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 0x7ff42c068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff438006fd0 tx=0x7ff438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:18.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.471+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff430056690 con 0x7ff4480cc700 2024-05-08T22:36:18.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:18.703+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7ff448067740 con 0x7ff4480cc700 2024-05-08T22:36:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.663+0000 7ff444ff9700 1 -- 172.21.15.179:0/546138399 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v170) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff43005b610 con 0x7ff4480cc700 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 msgr2=0x7ff42c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 0x7ff42c068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff438006fd0 tx=0x7ff438008040 comp rx=0 tx=0).stop 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 msgr2=0x7ff44814ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff4480b3e80 tx=0x7ff430004770 comp rx=0 tx=0).stop 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 shutdown_connections 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff42c066730 0x7ff42c068bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff44815b1b0 0x7ff4480ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4480cc700 0x7ff44814ccc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 --2- 172.21.15.179:0/546138399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4480cb510 0x7ff44813ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 >> 172.21.15.179:0/546138399 conn(0x7ff4480ce9d0 msgr2=0x7ff4480ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:19.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 shutdown_connections 2024-05-08T22:36:19.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.667+0000 7ff44d457700 1 -- 172.21.15.179:0/546138399 wait complete. 2024-05-08T22:36:19.685 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:36:19.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 Processor -- start 2024-05-08T22:36:19.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 -- start start 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340b4b40 0x7f2b3406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b340700b0 0x7f2b340ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340b3990 con 0x7f2b34159ee0 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b34070540 con 0x7f2b340700b0 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b38c77700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b41a0 con 0x7f2b340b4b40 2024-05-08T22:36:19.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b340700b0 0x7f2b340ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b340700b0 0x7f2b340ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49652/0 (socket says 172.21.15.179:49652) 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 -- 172.21.15.179:0/2857146500 learned_addr learned my addr 172.21.15.179:0/2857146500 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b31d9b700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340b4b40 0x7f2b3406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 -- 172.21.15.179:0/2857146500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340b4b40 msgr2=0x7f2b3406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340b4b40 0x7f2b3406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 -- 172.21.15.179:0/2857146500 >> v1:172.21.15.195:6789/0 conn(0x7f2b34159ee0 legacy=0x7f2b340cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.811+0000 7f2b32d9d700 1 -- 172.21.15.179:0/2857146500 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340b50b0 con 0x7f2b340700b0 2024-05-08T22:36:19.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b32d9d700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b340700b0 0x7f2b340ce8c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009e50 tx=0x7f2b2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f35d2b49063f62d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b3159a700 1 -- 172.21.15.179:0/2857146500 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00e070 con 0x7f2b340700b0 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b3159a700 1 -- 172.21.15.179:0/2857146500 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c00b910 con 0x7f2b340700b0 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b3159a700 1 -- 172.21.15.179:0/2857146500 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00bcc0 con 0x7f2b340700b0 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- 172.21.15.179:0/2857146500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b340700b0 msgr2=0x7f2b340ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b340700b0 0x7f2b340ce8c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009e50 tx=0x7f2b2c00b330 comp rx=0 tx=0).stop 2024-05-08T22:36:19.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- 172.21.15.179:0/2857146500 shutdown_connections 2024-05-08T22:36:19.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340b4b40 0x7f2b3406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 --2- 172.21.15.179:0/2857146500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b340700b0 0x7f2b340ce8c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- 172.21.15.179:0/2857146500 >> 172.21.15.179:0/2857146500 conn(0x7f2b3406e4e0 msgr2=0x7f2b3406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:19.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- 172.21.15.179:0/2857146500 shutdown_connections 2024-05-08T22:36:19.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- 172.21.15.179:0/2857146500 wait complete. 2024-05-08T22:36:19.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 Processor -- start 2024-05-08T22:36:19.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.815+0000 7f2b38c77700 1 -- start start 2024-05-08T22:36:19.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340700b0 0x7f2b3414db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 0x7f2b340b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341530b0 con 0x7f2b34159ee0 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b2f20 con 0x7f2b340b4b40 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b2c80 con 0x7f2b340700b0 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49654/0 (socket says 172.21.15.179:49654) 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 -- 172.21.15.179:0/766290181 learned_addr learned my addr 172.21.15.179:0/766290181 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b31d9b700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 0x7f2b340b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b3259c700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340700b0 0x7f2b3414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 -- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340700b0 msgr2=0x7f2b3414db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340700b0 0x7f2b3414db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 -- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 msgr2=0x7f2b340b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 0x7f2b340b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 -- 172.21.15.179:0/766290181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c009b30 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b31d9b700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 0x7f2b340b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b32d9d700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00b680 tx=0x7f2b2c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004830 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c0099b0 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b34152780 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004990 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b34152d40 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.819+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34152f00 con 0x7f2b34159ee0 2024-05-08T22:36:19.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.823+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b2c01c680 con 0x7f2b34159ee0 2024-05-08T22:36:19.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.823+0000 7f2b22ffd700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 0x7f2b14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:19.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.823+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2b2c085ac0 con 0x7f2b34159ee0 2024-05-08T22:36:19.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.823+0000 7f2b3259c700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 0x7f2b14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:19.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.823+0000 7f2b3259c700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 0x7f2b14068bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2b28000f60 tx=0x7f2b28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:19.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.831+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2b2c053080 con 0x7f2b34159ee0 2024-05-08T22:36:20.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:19.999+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2b34073290 con 0x7f2b34159ee0 2024-05-08T22:36:20.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.007+0000 7f2b22ffd700 1 -- 172.21.15.179:0/766290181 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b2c0566a0 con 0x7f2b34159ee0 2024-05-08T22:36:20.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 msgr2=0x7f2b14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 0x7f2b14068bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2b28000f60 tx=0x7f2b28009040 comp rx=0 tx=0).stop 2024-05-08T22:36:20.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 msgr2=0x7f2b340b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00b680 tx=0x7f2b2c004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 shutdown_connections 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b14066730 0x7f2b14068bf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b340b4b40 0x7f2b340b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b34159ee0 0x7f2b340b2740 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 --2- 172.21.15.179:0/766290181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b340700b0 0x7f2b3414db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 >> 172.21.15.179:0/766290181 conn(0x7f2b3406e4e0 msgr2=0x7f2b34156970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:20.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 shutdown_connections 2024-05-08T22:36:20.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.011+0000 7f2b38c77700 1 -- 172.21.15.179:0/766290181 wait complete. 2024-05-08T22:36:20.025 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:36:20.026 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:36:20.038 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:36:20.060 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:36:20.060 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:36:20.060 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:20.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:36:20.068 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:36:20.068 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:20.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:36:20.074 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:36:20.074 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:20.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:36:20.116 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:36:20.116 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:20.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:36:20.123 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:36:20.124 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.243+0000 7f4bf5c37700 1 Processor -- start 2024-05-08T22:36:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.243+0000 7f4bf5c37700 1 -- start start 2024-05-08T22:36:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.243+0000 7f4bf5c37700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4bf00ce2a0 0x7f4bf006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.243+0000 7f4bf5c37700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4bf006eb80 0x7f4bf00b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf006ddf0 con 0x7f4bf01434d0 2024-05-08T22:36:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf006f010 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00cd200 con 0x7f4bf006eb80 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4bf00ce2a0 0x7f4bf006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4bf00ce2a0 0x7f4bf006e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49662/0 (socket says 172.21.15.179:49662) 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 -- 172.21.15.179:0/68225959 learned_addr learned my addr 172.21.15.179:0/68225959 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5436700 1 --2- 172.21.15.179:0/68225959 >> v2:172.21.15.195:3300/0 conn(0x7f4bf006eb80 0x7f4bf00b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 -- 172.21.15.179:0/68225959 >> v2:172.21.15.195:3300/0 conn(0x7f4bf006eb80 msgr2=0x7f4bf00b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 --2- 172.21.15.179:0/68225959 >> v2:172.21.15.195:3300/0 conn(0x7f4bf006eb80 0x7f4bf00b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 -- 172.21.15.179:0/68225959 >> v1:172.21.15.195:6789/0 conn(0x7f4bf01434d0 legacy=0x7f4bf006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 -- 172.21.15.179:0/68225959 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00cd4e0 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4beffff700 1 --2- 172.21.15.179:0/68225959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf00ce2a0 0x7f4bf006e640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00cc050 tx=0x7f4bd8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5061272ef5bae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bef7fe700 1 -- 172.21.15.179:0/68225959 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800b8a0 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bef7fe700 1 -- 172.21.15.179:0/68225959 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800ba00 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bef7fe700 1 -- 172.21.15.179:0/68225959 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800bd90 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- 172.21.15.179:0/68225959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf00ce2a0 msgr2=0x7f4bf006e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/68225959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf00ce2a0 0x7f4bf006e640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00cc050 tx=0x7f4bd8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- 172.21.15.179:0/68225959 shutdown_connections 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/68225959 >> v2:172.21.15.195:3300/0 conn(0x7f4bf006eb80 0x7f4bf00b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/68225959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf00ce2a0 0x7f4bf006e640 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- 172.21.15.179:0/68225959 >> 172.21.15.179:0/68225959 conn(0x7f4bf00ce9d0 msgr2=0x7f4bf00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- 172.21.15.179:0/68225959 shutdown_connections 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.247+0000 7f4bf5c37700 1 -- 172.21.15.179:0/68225959 wait complete. 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 Processor -- start 2024-05-08T22:36:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- start start 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 0x7f4bf0134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bf00ce2a0 0x7f4bf0136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0136750 con 0x7f4bf01434d0 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01540a0 con 0x7f4bf006eb80 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0154380 con 0x7f4bf00ce2a0 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49664/0 (socket says 172.21.15.179:49664) 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 -- 172.21.15.179:0/2477735757 learned_addr learned my addr 172.21.15.179:0/2477735757 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4beffff700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bf00ce2a0 0x7f4bf0136210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf4c35700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 0x7f4bf0134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 -- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bf00ce2a0 msgr2=0x7f4bf0136210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bf00ce2a0 0x7f4bf0136210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 -- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 msgr2=0x7f4bf0134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 0x7f4bf0134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 -- 172.21.15.179:0/2477735757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8009570 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf4c35700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 0x7f4bf0134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5436700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4be8007e60 tx=0x7f4be800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800ede0 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8014420 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0154660 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80147d0 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf0136e00 con 0x7f4bf01434d0 2024-05-08T22:36:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.251+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0067740 con 0x7f4bf01434d0 2024-05-08T22:36:20.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.255+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4be8014930 con 0x7f4bf01434d0 2024-05-08T22:36:20.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.255+0000 7f4bed7fa700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 0x7f4bd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.255+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4be8084660 con 0x7f4bf01434d0 2024-05-08T22:36:20.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.255+0000 7f4bf4c35700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 0x7f4bd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.259+0000 7f4bf4c35700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 0x7f4bd0068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0136ad0 tx=0x7f4be0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.259+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4be8051b90 con 0x7f4bf01434d0 2024-05-08T22:36:20.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.467+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bf006c290 con 0x7f4bf01434d0 2024-05-08T22:36:20.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.467+0000 7f4bed7fa700 1 -- 172.21.15.179:0/2477735757 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3539 (secure 0 0 0) 0x7f4be8084020 con 0x7f4bf01434d0 2024-05-08T22:36:20.468 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:20.469 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2961587843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2961587843},{"type":"v1","addr":"172.21.15.195:6835","nonce":2961587843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6015,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/4039131439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":4039131439},{"type":"v1","addr":"172.21.15.179:6833","nonce":4039131439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2604632804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2604632804},{"type":"v1","addr":"172.21.15.179:6835","nonce":2604632804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":55664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/693641241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":693641241},{"type":"v1","addr":"172.21.15.195:6837","nonce":693641241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 msgr2=0x7f4bd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 0x7f4bd0068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0136ad0 tx=0x7f4be0009040 comp rx=0 tx=0).stop 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 msgr2=0x7f4bf0138920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4be8007e60 tx=0x7f4be800e3c0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 shutdown_connections 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4bd0066730 0x7f4bd0068bf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bf01434d0 0x7f4bf0138920 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bf00ce2a0 0x7f4bf0136210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 --2- 172.21.15.179:0/2477735757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf006eb80 0x7f4bf0134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 >> 172.21.15.179:0/2477735757 conn(0x7f4bf00ce9d0 msgr2=0x7f4bf00b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:20.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 shutdown_connections 2024-05-08T22:36:20.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.471+0000 7f4bf5c37700 1 -- 172.21.15.179:0/2477735757 wait complete. 2024-05-08T22:36:20.472 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 26 2024-05-08T22:36:20.483 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:36:20.483 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:36:20.484 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:36:20.484 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:36:20.484 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:36:20.484 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:36:20.484 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:36:20.484 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:36:20.484 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:36:20.484 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:36:20.485 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:36:20.485 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:36:20.485 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:36:20.485 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:36:20.485 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:36:20.485 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:36:20.495 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:36:20.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:36:20.498 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:36:20.498 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:36:20.499 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:36:20.499 DEBUG:teuthology.parallel:result is None 2024-05-08T22:36:20.500 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:36:20.500 DEBUG:teuthology.parallel:result is None 2024-05-08T22:36:20.501 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:36:20.502 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:36:20.502 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:36:20.503 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:36:20.504 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:36:20.504 DEBUG:teuthology.parallel:result is None 2024-05-08T22:36:20.505 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:36:20.505 DEBUG:teuthology.parallel:result is None 2024-05-08T22:36:20.505 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:36:20.574 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:36:20.578 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:36:20.578 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:36:20.587 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:36:20.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 Processor -- start 2024-05-08T22:36:20.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- start start 2024-05-08T22:36:20.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 0x7f1e500cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e5006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e5006c100 0x7f1e50165190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e500cdea0 con 0x7f1e5006c100 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e50161af0 con 0x7f1e500c2170 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500ca670 con 0x7f1e500cda40 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e5006c100 0x7f1e50165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e5006c100 0x7f1e50165190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49710/0 (socket says 172.21.15.179:49710) 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 -- 172.21.15.179:0/994421049 learned_addr learned my addr 172.21.15.179:0/994421049 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4ed9d700 1 --2- 172.21.15.179:0/994421049 >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 0x7f1e500cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4e59c700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e5006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 -- 172.21.15.179:0/994421049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 msgr2=0x7f1e5006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e5006bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 -- 172.21.15.179:0/994421049 >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 msgr2=0x7f1e500cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 --2- 172.21.15.179:0/994421049 >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 0x7f1e500cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 -- 172.21.15.179:0/994421049 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500ca950 con 0x7f1e5006c100 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4ed9d700 1 --2- 172.21.15.179:0/994421049 >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 0x7f1e500cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4f59e700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e50165190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e50067c80 tx=0x7f1e4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174f802c6b922bed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4dd9b700 1 -- 172.21.15.179:0/994421049 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800e070 con 0x7f1e5006c100 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4dd9b700 1 -- 172.21.15.179:0/994421049 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4800b9c0 con 0x7f1e5006c100 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e4dd9b700 1 -- 172.21.15.179:0/994421049 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800bd70 con 0x7f1e5006c100 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- 172.21.15.179:0/994421049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 msgr2=0x7f1e50165190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e50165190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e50067c80 tx=0x7f1e4800b330 comp rx=0 tx=0).stop 2024-05-08T22:36:20.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- 172.21.15.179:0/994421049 shutdown_connections 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e50165190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- 172.21.15.179:0/994421049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e5006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 --2- 172.21.15.179:0/994421049 >> v2:172.21.15.195:3300/0 conn(0x7f1e500c2170 0x7f1e500cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- 172.21.15.179:0/994421049 >> 172.21.15.179:0/994421049 conn(0x7f1e50070e50 msgr2=0x7f1e50071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- 172.21.15.179:0/994421049 shutdown_connections 2024-05-08T22:36:20.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.623+0000 7f1e55217700 1 -- 172.21.15.179:0/994421049 wait complete. 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 Processor -- start 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 -- start start 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e500c2170 0x7f1e50151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e50151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500c4900 con 0x7f1e5006c100 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500c42f0 con 0x7f1e500c2170 2024-05-08T22:36:20.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500c60c0 con 0x7f1e500cda40 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49718/0 (socket says 172.21.15.179:49718) 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 -- 172.21.15.179:0/1226459498 learned_addr learned my addr 172.21.15.179:0/1226459498 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4e59c700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e500c2170 0x7f1e50151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4f59e700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e50151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 -- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 msgr2=0x7f1e50151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e50151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 -- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e500c2170 msgr2=0x7f1e50151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e500c2170 0x7f1e50151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 -- 172.21.15.179:0/1226459498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e48009b30 con 0x7f1e5006c100 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4f59e700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e50151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e4ed9d700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d800 tx=0x7f1e4400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004030 con 0x7f1e5006c100 2024-05-08T22:36:20.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44004190 con 0x7f1e5006c100 2024-05-08T22:36:20.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004540 con 0x7f1e5006c100 2024-05-08T22:36:20.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.627+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e500c4590 con 0x7f1e5006c100 2024-05-08T22:36:20.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5014f210 con 0x7f1e5006c100 2024-05-08T22:36:20.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e50067770 con 0x7f1e5006c100 2024-05-08T22:36:20.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e440036c0 con 0x7f1e5006c100 2024-05-08T22:36:20.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e377fe700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 0x7f1e3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:20.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1e440848f0 con 0x7f1e5006c100 2024-05-08T22:36:20.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.631+0000 7f1e4e59c700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 0x7f1e3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:20.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.635+0000 7f1e4e59c700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 0x7f1e3c068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1e500c7640 tx=0x7f1e40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:20.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.639+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e44051e00 con 0x7f1e5006c100 2024-05-08T22:36:20.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:20.815+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1e50073290 con 0x7f1e5006c100 2024-05-08T22:36:21.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.675+0000 7f1e377fe700 1 -- 172.21.15.179:0/1226459498 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1e44055420 con 0x7f1e5006c100 2024-05-08T22:36:21.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 msgr2=0x7f1e3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 0x7f1e3c068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1e500c7640 tx=0x7f1e40008040 comp rx=0 tx=0).stop 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 msgr2=0x7f1e5014ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d800 tx=0x7f1e4400db10 comp rx=0 tx=0).stop 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 shutdown_connections 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e3c066730 0x7f1e3c068bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e500c2170 0x7f1e50151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e500cda40 0x7f1e50151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 --2- 172.21.15.179:0/1226459498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e5006c100 0x7f1e5014ecd0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 >> 172.21.15.179:0/1226459498 conn(0x7f1e50070e50 msgr2=0x7f1e50161260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:21.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 shutdown_connections 2024-05-08T22:36:21.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.679+0000 7f1e55217700 1 -- 172.21.15.179:0/1226459498 wait complete. 2024-05-08T22:36:21.682 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:36:21.696 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:36:21.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.815+0000 7f88c1709700 1 Processor -- start 2024-05-08T22:36:21.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.815+0000 7f88c1709700 1 -- start start 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88bc15d970 0x7f88bc15de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc15e340 con 0x7f88bc143640 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc06ce80 con 0x7f88bc15d970 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc143a20 con 0x7f88bc0cea40 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88bbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88bc15d970 0x7f88bc15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88bc15d970 0x7f88bc15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:38618/0 (socket says 172.21.15.179:38618) 2024-05-08T22:36:21.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 -- 172.21.15.179:0/3380905993 learned_addr learned my addr 172.21.15.179:0/3380905993 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88bbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49720/0 (socket says 172.21.15.179:49720) 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 -- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 msgr2=0x7f88bc0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 -- 172.21.15.179:0/3380905993 >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 msgr2=0x7f88bc06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 --2- 172.21.15.179:0/3380905993 >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 -- 172.21.15.179:0/3380905993 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0cf220 con 0x7f88bc15d970 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3380905993 >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c0f08700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc15d970 0x7f88bc15de00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88bc15d8b0 tx=0x7f88b400be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f8d54707114f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88baffd700 1 -- 172.21.15.179:0/3380905993 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400dd30 con 0x7f88bc15d970 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88baffd700 1 -- 172.21.15.179:0/3380905993 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400de90 con 0x7f88bc15d970 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88baffd700 1 -- 172.21.15.179:0/3380905993 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4010690 con 0x7f88bc15d970 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc15d970 msgr2=0x7f88bc15de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:21.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc15d970 0x7f88bc15de00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88bc15d8b0 tx=0x7f88b400be90 comp rx=0 tx=0).stop 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- 172.21.15.179:0/3380905993 shutdown_connections 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc15d970 0x7f88bc15de00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- 172.21.15.179:0/3380905993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 --2- 172.21.15.179:0/3380905993 >> v2:172.21.15.179:3300/0 conn(0x7f88bc143640 0x7f88bc06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- 172.21.15.179:0/3380905993 >> 172.21.15.179:0/3380905993 conn(0x7f88bc0cfae0 msgr2=0x7f88bc0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- 172.21.15.179:0/3380905993 shutdown_connections 2024-05-08T22:36:21.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.819+0000 7f88c1709700 1 -- 172.21.15.179:0/3380905993 wait complete. 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 Processor -- start 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 -- start start 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 0x7f88bc13d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc15ab80 con 0x7f88bc15d970 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc13dab0 con 0x7f88bc143640 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c1709700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc156ef0 con 0x7f88bc0cea40 2024-05-08T22:36:21.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c0f08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c0f08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49736/0 (socket says 172.21.15.179:49736) 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88c0f08700 1 -- 172.21.15.179:0/229869175 learned_addr learned my addr 172.21.15.179:0/229869175 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bbfff700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 0x7f88bc13d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bb7fe700 1 -- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 msgr2=0x7f88bc15a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc15a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bb7fe700 1 -- 172.21.15.179:0/229869175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 msgr2=0x7f88bc141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.823+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88bb7fe700 1 -- 172.21.15.179:0/229869175 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00094b0 con 0x7f88bc143640 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88c0f08700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:21.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 0x7f88bc13d570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009d90 tx=0x7f88ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:21.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac009c00 con 0x7f88bc143640 2024-05-08T22:36:21.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac00f040 con 0x7f88bc143640 2024-05-08T22:36:21.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b400b810 con 0x7f88bc143640 2024-05-08T22:36:21.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac014480 con 0x7f88bc143640 2024-05-08T22:36:21.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc1574b0 con 0x7f88bc143640 2024-05-08T22:36:21.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88ac0145e0 con 0x7f88bc143640 2024-05-08T22:36:21.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.827+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc04e700 con 0x7f88bc143640 2024-05-08T22:36:21.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.831+0000 7f88b8ff9700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 0x7f88a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:21.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.831+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f88ac084a00 con 0x7f88bc143640 2024-05-08T22:36:21.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.831+0000 7f88bbfff700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 0x7f88a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:21.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.831+0000 7f88bbfff700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 0x7f88a8068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d00 tx=0x7f88b0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:21.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:21.839+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f88ac052010 con 0x7f88bc143640 2024-05-08T22:36:21.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:36:21.882 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:36:22.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.071+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f88bc0b4b90 con 0x7f88bc143640 2024-05-08T22:36:22.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88b8ff9700 1 -- 172.21.15.179:0/229869175 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f88ac018020 con 0x7f88bc143640 2024-05-08T22:36:22.074 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.075 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.076 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.077 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.078 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.079 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.080 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:36:22.081 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:36:22.082 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.083 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.084 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:36:22.085 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.087 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:36:22.088 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:36:22.089 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.090 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:36:22.091 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:36:22.092 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:36:22.093 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:22.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 msgr2=0x7f88a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 0x7f88a8068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d00 tx=0x7f88b0009240 comp rx=0 tx=0).stop 2024-05-08T22:36:22.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 msgr2=0x7f88bc13d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 0x7f88bc13d570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009d90 tx=0x7f88ac00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 shutdown_connections 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f88a8066730 0x7f88a8068bf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc143640 0x7f88bc13d570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15d970 0x7f88bc141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 --2- 172.21.15.179:0/229869175 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc0cea40 0x7f88bc15a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 >> 172.21.15.179:0/229869175 conn(0x7f88bc0cfae0 msgr2=0x7f88bc140890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 shutdown_connections 2024-05-08T22:36:22.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.075+0000 7f88c1709700 1 -- 172.21.15.179:0/229869175 wait complete. 2024-05-08T22:36:22.099 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:22.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.215+0000 7fc274923700 1 Processor -- start 2024-05-08T22:36:22.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.215+0000 7fc274923700 1 -- start start 2024-05-08T22:36:22.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.215+0000 7fc274923700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc270146800 0x7fc27006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.215+0000 7fc274923700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc2700caed0 0x7fc2700c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.215+0000 7fc274923700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc270160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2700c88d0 con 0x7fc270146800 2024-05-08T22:36:22.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27006d370 con 0x7fc2700caed0 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cb350 con 0x7fc2700c8480 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc270146800 0x7fc27006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc270146800 0x7fc27006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49744/0 (socket says 172.21.15.179:49744) 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 -- 172.21.15.179:0/292961232 learned_addr learned my addr 172.21.15.179:0/292961232 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc266ffd700 1 --2- 172.21.15.179:0/292961232 >> v2:172.21.15.195:3300/0 conn(0x7fc2700caed0 0x7fc2700c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26ffff700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc270160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 -- 172.21.15.179:0/292961232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 msgr2=0x7fc270160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc270160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 -- 172.21.15.179:0/292961232 >> v2:172.21.15.195:3300/0 conn(0x7fc2700caed0 msgr2=0x7fc2700c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/292961232 >> v2:172.21.15.195:3300/0 conn(0x7fc2700caed0 0x7fc2700c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 -- 172.21.15.179:0/292961232 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270146be0 con 0x7fc270146800 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26ffff700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc270160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270146800 0x7fc27006b5c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc2700cb5d0 tx=0x7fc260009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44df7295c03aecf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26effd700 1 -- 172.21.15.179:0/292961232 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260011830 con 0x7fc270146800 2024-05-08T22:36:22.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26effd700 1 -- 172.21.15.179:0/292961232 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260011990 con 0x7fc270146800 2024-05-08T22:36:22.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc26effd700 1 -- 172.21.15.179:0/292961232 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260011d20 con 0x7fc270146800 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- 172.21.15.179:0/292961232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270146800 msgr2=0x7fc27006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270146800 0x7fc27006b5c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc2700cb5d0 tx=0x7fc260009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- 172.21.15.179:0/292961232 shutdown_connections 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc270160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 --2- 172.21.15.179:0/292961232 >> v2:172.21.15.195:3300/0 conn(0x7fc2700caed0 0x7fc2700c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 --2- 172.21.15.179:0/292961232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270146800 0x7fc27006b5c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- 172.21.15.179:0/292961232 >> 172.21.15.179:0/292961232 conn(0x7fc2700cb670 msgr2=0x7fc2700cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- 172.21.15.179:0/292961232 shutdown_connections 2024-05-08T22:36:22.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.219+0000 7fc274923700 1 -- 172.21.15.179:0/292961232 wait complete. 2024-05-08T22:36:22.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 Processor -- start 2024-05-08T22:36:22.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- start start 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc2700b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc270146800 0x7fc27013b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700b4350 con 0x7fc2700caed0 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700b2d70 con 0x7fc270146800 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700b3050 con 0x7fc2700c8480 2024-05-08T22:36:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49748/0 (socket says 172.21.15.179:49748) 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 -- 172.21.15.179:0/2782206768 learned_addr learned my addr 172.21.15.179:0/2782206768 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc26ffff700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc270146800 0x7fc27013b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc2700b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 -- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 msgr2=0x7fc2700b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc2700b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 -- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc270146800 msgr2=0x7fc27013b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc270146800 0x7fc27013b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 -- 172.21.15.179:0/2782206768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009570 con 0x7fc2700caed0 2024-05-08T22:36:22.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc266ffd700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9d0 tx=0x7fc25800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009720 con 0x7fc2700caed0 2024-05-08T22:36:22.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800c850 con 0x7fc2700caed0 2024-05-08T22:36:22.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2700b3330 con 0x7fc2700caed0 2024-05-08T22:36:22.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800cbc0 con 0x7fc2700caed0 2024-05-08T22:36:22.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.223+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2700b2a00 con 0x7fc2700caed0 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2700b3b30 con 0x7fc2700caed0 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc25800cd20 con 0x7fc2700caed0 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc26cff9700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 0x7fc254068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc258084010 con 0x7fc2700caed0 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 0x7fc254068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.227+0000 7fc26f7fe700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 0x7fc254068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc260000c00 tx=0x7fc260008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.231+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc2700b3b30 con 0x7fc2700caed0 2024-05-08T22:36:22.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.403+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2700b3b30 con 0x7fc2700caed0 2024-05-08T22:36:22.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.407+0000 7fc26cff9700 1 -- 172.21.15.179:0/2782206768 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12642 (secure 0 0 0) 0x7fc2700b3b30 con 0x7fc2700caed0 2024-05-08T22:36:22.408 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:22.408 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":65,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:21.664387+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 msgr2=0x7fc254068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 0x7fc254068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc260000c00 tx=0x7fc260008030 comp rx=0 tx=0).stop 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 msgr2=0x7fc27013b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9d0 tx=0x7fc25800ece0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 shutdown_connections 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc254066730 0x7fc254068bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc270146800 0x7fc27013b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2700caed0 0x7fc27013b460 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 --2- 172.21.15.179:0/2782206768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2700c8480 0x7fc2700b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 >> 172.21.15.179:0/2782206768 conn(0x7fc2700cb670 msgr2=0x7fc27006a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 shutdown_connections 2024-05-08T22:36:22.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.411+0000 7fc274923700 1 -- 172.21.15.179:0/2782206768 wait complete. 2024-05-08T22:36:22.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:22.423 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:22.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.543+0000 7f8b1d978700 1 Processor -- start 2024-05-08T22:36:22.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.543+0000 7f8b1d978700 1 -- start start 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b1815b020 0x7f8b180b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b180ccd90 0x7f8b1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b180cd1f0 0x7f8b1806f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1806f860 con 0x7f8b180ccd90 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1806e930 con 0x7f8b180cd1f0 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180cc900 con 0x7f8b1815b020 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b180ccd90 0x7f8b1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b180ccd90 0x7f8b1806ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49756/0 (socket says 172.21.15.179:49756) 2024-05-08T22:36:22.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 -- 172.21.15.179:0/3992519669 learned_addr learned my addr 172.21.15.179:0/3992519669 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d177700 1 --2- 172.21.15.179:0/3992519669 >> v2:172.21.15.195:3300/0 conn(0x7f8b180cd1f0 0x7f8b1806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1c976700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b1815b020 0x7f8b180b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 -- 172.21.15.179:0/3992519669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b1815b020 msgr2=0x7f8b180b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b1815b020 0x7f8b180b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 -- 172.21.15.179:0/3992519669 >> v2:172.21.15.195:3300/0 conn(0x7f8b180cd1f0 msgr2=0x7f8b1806f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 --2- 172.21.15.179:0/3992519669 >> v2:172.21.15.195:3300/0 conn(0x7f8b180cd1f0 0x7f8b1806f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 -- 172.21.15.179:0/3992519669 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180cba10 con 0x7f8b180ccd90 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b17fff700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180ccd90 0x7f8b1806ede0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8b1806c0d0 tx=0x7f8b08009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221171acc636c79b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b177fe700 1 -- 172.21.15.179:0/3992519669 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800ba20 con 0x7f8b180ccd90 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b177fe700 1 -- 172.21.15.179:0/3992519669 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0800bb80 con 0x7f8b180ccd90 2024-05-08T22:36:22.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b177fe700 1 -- 172.21.15.179:0/3992519669 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08009e60 con 0x7f8b180ccd90 2024-05-08T22:36:22.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- 172.21.15.179:0/3992519669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180ccd90 msgr2=0x7f8b1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180ccd90 0x7f8b1806ede0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8b1806c0d0 tx=0x7f8b08009c20 comp rx=0 tx=0).stop 2024-05-08T22:36:22.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- 172.21.15.179:0/3992519669 shutdown_connections 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- 172.21.15.179:0/3992519669 >> v2:172.21.15.195:3300/0 conn(0x7f8b180cd1f0 0x7f8b1806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180ccd90 0x7f8b1806ede0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 --2- 172.21.15.179:0/3992519669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b1815b020 0x7f8b180b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- 172.21.15.179:0/3992519669 >> 172.21.15.179:0/3992519669 conn(0x7f8b180d0ca0 msgr2=0x7f8b180d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- 172.21.15.179:0/3992519669 shutdown_connections 2024-05-08T22:36:22.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.547+0000 7f8b1d978700 1 -- 172.21.15.179:0/3992519669 wait complete. 2024-05-08T22:36:22.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 Processor -- start 2024-05-08T22:36:22.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 -- start start 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b180ccd90 0x7f8b180c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 0x7f8b180c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1806cc40 con 0x7f8b180cd1f0 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180c5550 con 0x7f8b1815b020 2024-05-08T22:36:22.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d978700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180c5830 con 0x7f8b180ccd90 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49766/0 (socket says 172.21.15.179:49766) 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 -- 172.21.15.179:0/1284058748 learned_addr learned my addr 172.21.15.179:0/1284058748 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1c976700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b180ccd90 0x7f8b180c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d177700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 0x7f8b180c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 -- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b180ccd90 msgr2=0x7f8b180c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b180ccd90 0x7f8b180c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 -- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 msgr2=0x7f8b180c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 0x7f8b180c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 -- 172.21.15.179:0/1284058748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08009570 con 0x7f8b180cd1f0 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b1d177700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 0x7f8b180c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b17fff700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.551+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080044d0 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0801f070 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1806c580 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08003900 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b180c5250 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1806cd80 con 0x7f8b180cd1f0 2024-05-08T22:36:22.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b08003a60 con 0x7f8b180cd1f0 2024-05-08T22:36:22.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b157fa700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 0x7f8af8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8b08021070 con 0x7f8b180cd1f0 2024-05-08T22:36:22.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.555+0000 7f8b1c976700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 0x7f8af8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.559+0000 7f8b1c976700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 0x7f8af8068c20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009790 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.567+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b1806cd80 con 0x7f8b180cd1f0 2024-05-08T22:36:22.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.735+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b1806cd80 con 0x7f8b180cd1f0 2024-05-08T22:36:22.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b157fa700 1 -- 172.21.15.179:0/1284058748 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b1806cd80 con 0x7f8b180cd1f0 2024-05-08T22:36:22.738 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 msgr2=0x7f8af8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 0x7f8af8068c20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009790 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 msgr2=0x7f8b1815def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b08004770 comp rx=0 tx=0).stop 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 shutdown_connections 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8af8066760 0x7f8af8068c20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1815b020 0x7f8b180c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b180cd1f0 0x7f8b1815def0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 --2- 172.21.15.179:0/1284058748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b180ccd90 0x7f8b180c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 >> 172.21.15.179:0/1284058748 conn(0x7f8b180d0ca0 msgr2=0x7f8b1806ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 shutdown_connections 2024-05-08T22:36:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.739+0000 7f8b1d978700 1 -- 172.21.15.179:0/1284058748 wait complete. 2024-05-08T22:36:22.753 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:36:22.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 Processor -- start 2024-05-08T22:36:22.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 -- start start 2024-05-08T22:36:22.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff79c15ef80 0x7ff79c161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c165d50 con 0x7ff79c06de70 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c06ad70 con 0x7ff79c164db0 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a23f7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79c165260 con 0x7ff79c15ef80 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a1bf6700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff79c06de70 0x7ff79c06b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43654/0 (socket says 172.21.15.179:43654) 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a1bf6700 1 -- 172.21.15.179:0/1839500691 learned_addr learned my addr 172.21.15.179:0/1839500691 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a13f5700 1 --2- 172.21.15.179:0/1839500691 >> v2:172.21.15.195:3300/0 conn(0x7ff79c15ef80 0x7ff79c161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615502542 0 0) 0x7ff79c165d50 con 0x7ff79c06de70 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7880035d0 con 0x7ff79c06de70 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408498495 0 0) 0x7ff79c06ad70 con 0x7ff79c164db0 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff79c165d50 con 0x7ff79c164db0 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a13f5700 1 -- 172.21.15.179:0/1839500691 >> v1:172.21.15.195:6789/0 conn(0x7ff79c164db0 legacy=0x7ff79c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a13f5700 1 -- 172.21.15.179:0/1839500691 >> v1:172.21.15.179:6789/0 conn(0x7ff79c06de70 legacy=0x7ff79c06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755915339 0 0) 0x7ff7880035d0 con 0x7ff79c06de70 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a13f5700 1 -- 172.21.15.179:0/1839500691 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c06e4b0 con 0x7ff79c15ef80 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff7a13f5700 1 --2- 172.21.15.179:0/1839500691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c15ef80 0x7ff79c161790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff794009d90 tx=0x7ff79400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23915f122bac58fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79400e070 con 0x7ff79c15ef80 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794003a40 con 0x7ff79c15ef80 2024-05-08T22:36:22.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.867+0000 7ff78ffff700 1 -- 172.21.15.179:0/1839500691 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798001670 con 0x7ff79c15ef80 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- 172.21.15.179:0/1839500691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c15ef80 msgr2=0x7ff79c161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/1839500691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c15ef80 0x7ff79c161790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff794009d90 tx=0x7ff79400b330 comp rx=0 tx=0).stop 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- 172.21.15.179:0/1839500691 shutdown_connections 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/1839500691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c15ef80 0x7ff79c161790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- 172.21.15.179:0/1839500691 >> 172.21.15.179:0/1839500691 conn(0x7ff79c0ce9d0 msgr2=0x7ff79c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- 172.21.15.179:0/1839500691 shutdown_connections 2024-05-08T22:36:22.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- 172.21.15.179:0/1839500691 wait complete. 2024-05-08T22:36:22.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 Processor -- start 2024-05-08T22:36:22.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- start start 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c06de70 0x7ff79c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff79c164db0 0x7ff79c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15bd70 con 0x7ff79c15ef80 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15b160 con 0x7ff79c06de70 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a23f7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15b440 con 0x7ff79c164db0 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a0bf4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a0bf4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49772/0 (socket says 172.21.15.179:49772) 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a13f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c06de70 0x7ff79c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a0bf4700 1 -- 172.21.15.179:0/2220981907 learned_addr learned my addr 172.21.15.179:0/2220981907 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.871+0000 7ff7a1bf6700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff79c164db0 0x7ff79c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 -- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff79c164db0 msgr2=0x7ff79c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff79c164db0 0x7ff79c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 -- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c06de70 msgr2=0x7ff79c15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:22.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c06de70 0x7ff79c15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:22.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 -- 172.21.15.179:0/2220981907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794009a70 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a0bf4700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c490 tx=0x7ff79000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000cea0 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7900052f0 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900036a0 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c15b720 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c1470d0 con 0x7ff79c15ef80 2024-05-08T22:36:22.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.875+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c067740 con 0x7ff79c15ef80 2024-05-08T22:36:22.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.879+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff790003800 con 0x7ff79c15ef80 2024-05-08T22:36:22.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.879+0000 7ff78dffb700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 0x7ff780068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:22.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.879+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff7900831d0 con 0x7ff79c15ef80 2024-05-08T22:36:22.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.879+0000 7ff7a13f5700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 0x7ff780068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:22.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.879+0000 7ff7a13f5700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 0x7ff780068c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b370 tx=0x7ff794011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:22.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:22.883+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff7900507e0 con 0x7ff79c15ef80 2024-05-08T22:36:23.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.067+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff79c04e700 con 0x7ff79c15ef80 2024-05-08T22:36:23.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.067+0000 7ff78dffb700 1 -- 172.21.15.179:0/2220981907 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff790053e00 con 0x7ff79c15ef80 2024-05-08T22:36:23.067 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:23.067 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:36:23.068 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:23.068 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:36:23.068 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:36:23.068 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:23.068 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:36:23.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 msgr2=0x7ff780068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 0x7ff780068c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b370 tx=0x7ff794011000 comp rx=0 tx=0).stop 2024-05-08T22:36:23.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 msgr2=0x7ff79c146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c490 tx=0x7ff79000c7a0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 shutdown_connections 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff780066760 0x7ff780068c20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff79c164db0 0x7ff79c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff79c15ef80 0x7ff79c146b90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 --2- 172.21.15.179:0/2220981907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff79c06de70 0x7ff79c15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 >> 172.21.15.179:0/2220981907 conn(0x7ff79c0ce9d0 msgr2=0x7ff79c0ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 shutdown_connections 2024-05-08T22:36:23.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.071+0000 7ff7a23f7700 1 -- 172.21.15.179:0/2220981907 wait complete. 2024-05-08T22:36:23.082 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:36:23.083 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:36:23.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.199+0000 7f0f5bfff700 1 Processor -- start 2024-05-08T22:36:23.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.199+0000 7f0f5bfff700 1 -- start start 2024-05-08T22:36:23.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 0x7f0f5c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 0x7f0f5c0d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c15d870 con 0x7f0f5c06c2e0 2024-05-08T22:36:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0d2740 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b7300 con 0x7f0f5c15d490 2024-05-08T22:36:23.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 0x7f0f5c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5b7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 0x7f0f5c0d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5a7fc700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f0f5c06c2e0 0x7f0f5c0cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43234/0 (socket says 172.21.15.179:43234) 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5a7fc700 1 -- 172.21.15.179:0/764170961 learned_addr learned my addr 172.21.15.179:0/764170961 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f59ffb700 1 -- 172.21.15.179:0/764170961 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701797701 0 0) 0x7f0f5c15d870 con 0x7f0f5c06c2e0 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f59ffb700 1 -- 172.21.15.179:0/764170961 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f440035d0 con 0x7f0f5c06c2e0 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 -- 172.21.15.179:0/764170961 >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 msgr2=0x7f0f5c0d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 --2- 172.21.15.179:0/764170961 >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 0x7f0f5c0d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 -- 172.21.15.179:0/764170961 >> v1:172.21.15.195:6789/0 conn(0x7f0f5c06c2e0 legacy=0x7f0f5c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 -- 172.21.15.179:0/764170961 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c06b240 con 0x7f0f5c15d490 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5b7fe700 1 --2- 172.21.15.179:0/764170961 >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 0x7f0f5c0d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5affd700 1 --2- 172.21.15.179:0/764170961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 0x7f0f5c06abb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009850 tx=0x7f0f50009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be64447258053115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f59ffb700 1 -- 172.21.15.179:0/764170961 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50011830 con 0x7f0f5c15d490 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f59ffb700 1 -- 172.21.15.179:0/764170961 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f50011990 con 0x7f0f5c15d490 2024-05-08T22:36:23.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f59ffb700 1 -- 172.21.15.179:0/764170961 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50011d00 con 0x7f0f5c15d490 2024-05-08T22:36:23.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 -- 172.21.15.179:0/764170961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 msgr2=0x7f0f5c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.203+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/764170961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 0x7f0f5c06abb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009850 tx=0x7f0f50009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:23.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 -- 172.21.15.179:0/764170961 shutdown_connections 2024-05-08T22:36:23.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/764170961 >> v2:172.21.15.195:3300/0 conn(0x7f0f5c0ce5a0 0x7f0f5c0d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/764170961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c15d490 0x7f0f5c06abb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 -- 172.21.15.179:0/764170961 >> 172.21.15.179:0/764170961 conn(0x7f0f5c0b4940 msgr2=0x7f0f5c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:23.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 -- 172.21.15.179:0/764170961 shutdown_connections 2024-05-08T22:36:23.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 -- 172.21.15.179:0/764170961 wait complete. 2024-05-08T22:36:23.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 Processor -- start 2024-05-08T22:36:23.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.207+0000 7f0f5bfff700 1 -- start start 2024-05-08T22:36:23.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c06c2e0 0x7f0f5c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f5c15d490 0x7f0f5c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c14e8e0 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0cba50 con 0x7f0f5c15d490 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0cbd30 con 0x7f0f5c06c2e0 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49784/0 (socket says 172.21.15.179:49784) 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 -- 172.21.15.179:0/3050428718 learned_addr learned my addr 172.21.15.179:0/3050428718 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5affd700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c06c2e0 0x7f0f5c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5b7fe700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f5c15d490 0x7f0f5c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 -- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c06c2e0 msgr2=0x7f0f5c14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c06c2e0 0x7f0f5c14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 -- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f5c15d490 msgr2=0x7f0f5c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f5c15d490 0x7f0f5c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 -- 172.21.15.179:0/3050428718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009a70 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5a7fc700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005930 tx=0x7f0f4c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c005d90 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0070d0 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500094b0 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c007480 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.211+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c0cc2f0 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f4c0075e0 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c04e770 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f43fff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 0x7f0f48068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0f4c083260 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f5affd700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 0x7f0f48068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.215+0000 7f0f5affd700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 0x7f0f48068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0f500090b0 tx=0x7f0f50008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.223+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f4c050870 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.395+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0f5c06ac90 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.691+0000 7f0f43fff700 1 -- 172.21.15.179:0/3050428718 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0f4c053e90 con 0x7f0f5c0ce5a0 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 msgr2=0x7f0f48068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 0x7f0f48068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0f500090b0 tx=0x7f0f50008030 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 msgr2=0x7f0f5c1f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005930 tx=0x7f0f4c00c7b0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 shutdown_connections 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0f48066730 0x7f0f48068bf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f5c15d490 0x7f0f5c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c0ce5a0 0x7f0f5c1f2d80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 --2- 172.21.15.179:0/3050428718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f5c06c2e0 0x7f0f5c14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.703+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 >> 172.21.15.179:0/3050428718 conn(0x7f0f5c0b4940 msgr2=0x7f0f5c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.707+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 shutdown_connections 2024-05-08T22:36:23.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.707+0000 7f0f5bfff700 1 -- 172.21.15.179:0/3050428718 wait complete. 2024-05-08T22:36:23.707 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:36:23.719 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:36:23.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 Processor -- start 2024-05-08T22:36:23.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 -- start start 2024-05-08T22:36:23.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 0x7f24bc0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24bc0b4a90 con 0x7f24bc15b930 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24bc0b4d10 con 0x7f24bc0b2ac0 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c2f4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc15c1e0 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c1f4b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 0x7f24bc0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c274c700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f24bc15b930 0x7f24bc0cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43240/0 (socket says 172.21.15.179:43240) 2024-05-08T22:36:23.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c274c700 1 -- 172.21.15.179:0/1166725991 learned_addr learned my addr 172.21.15.179:0/1166725991 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:23.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 -- 172.21.15.179:0/1166725991 >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 msgr2=0x7f24bc0b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:23.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 --2- 172.21.15.179:0/1166725991 >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 0x7f24bc0b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 -- 172.21.15.179:0/1166725991 >> v1:172.21.15.195:6789/0 conn(0x7f24bc15b930 legacy=0x7f24bc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 -- 172.21.15.179:0/1166725991 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc0b4fd0 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c1f4b700 1 --2- 172.21.15.179:0/1166725991 >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 0x7f24bc0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:23.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.843+0000 7f24c174a700 1 --2- 172.21.15.179:0/1166725991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc15b3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009850 tx=0x7f24ac009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77280e7ff1017378 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c0f49700 1 -- 172.21.15.179:0/1166725991 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac011830 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c0f49700 1 -- 172.21.15.179:0/1166725991 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac011990 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c0f49700 1 -- 172.21.15.179:0/1166725991 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac011d40 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 -- 172.21.15.179:0/1166725991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 msgr2=0x7f24bc15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/1166725991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc15b3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009850 tx=0x7f24ac009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 -- 172.21.15.179:0/1166725991 shutdown_connections 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/1166725991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc15b3f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/1166725991 >> v2:172.21.15.195:3300/0 conn(0x7f24bc0b2ac0 0x7f24bc0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 -- 172.21.15.179:0/1166725991 >> 172.21.15.179:0/1166725991 conn(0x7f24bc0cc5d0 msgr2=0x7f24bc0cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 -- 172.21.15.179:0/1166725991 shutdown_connections 2024-05-08T22:36:23.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.847+0000 7f24c2f4d700 1 -- 172.21.15.179:0/1166725991 wait complete. 2024-05-08T22:36:23.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 Processor -- start 2024-05-08T22:36:23.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- start start 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc0b2ac0 0x7f24bc15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc06c7a0 con 0x7f24bc15b930 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc15ddc0 con 0x7f24bc0b2ac0 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc06e250 con 0x7f24bc0b2ea0 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49790/0 (socket says 172.21.15.179:49790) 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 -- 172.21.15.179:0/3933468052 learned_addr learned my addr 172.21.15.179:0/3933468052 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c1f4b700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc0b2ac0 0x7f24bc15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c174a700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 -- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 msgr2=0x7f24bc06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 -- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc0b2ac0 msgr2=0x7f24bc15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc0b2ac0 0x7f24bc15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 -- 172.21.15.179:0/3933468052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80094b0 con 0x7f24bc15b930 2024-05-08T22:36:23.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c274c700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ca80 tx=0x7f24b400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4007480 con 0x7f24bc15b930 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40075e0 con 0x7f24bc15b930 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac0094b0 con 0x7f24bc15b930 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4007950 con 0x7f24bc15b930 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.851+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc06d230 con 0x7f24bc15b930 2024-05-08T22:36:23.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc04e700 con 0x7f24bc15b930 2024-05-08T22:36:23.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24b401b3b0 con 0x7f24bc15b930 2024-05-08T22:36:23.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24b27fc700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 0x7f24a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:23.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f24b4083870 con 0x7f24bc15b930 2024-05-08T22:36:23.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24c1f4b700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 0x7f24a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:23.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.855+0000 7f24c1f4b700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 0x7f24a0068bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009d80 tx=0x7f24b8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:23.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:23.863+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f24b4050cb0 con 0x7f24bc15b930 2024-05-08T22:36:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.031+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f24bc0cd2e0 con 0x7f24bc15b930 2024-05-08T22:36:24.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.695+0000 7f24b27fc700 1 -- 172.21.15.179:0/3933468052 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7f24b40170d0 con 0x7f24bc15b930 2024-05-08T22:36:24.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.711+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 msgr2=0x7f24a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.711+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 0x7f24a0068bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009d80 tx=0x7f24b8009240 comp rx=0 tx=0).stop 2024-05-08T22:36:24.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 msgr2=0x7f24bc06be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ca80 tx=0x7f24b400cab0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 shutdown_connections 2024-05-08T22:36:24.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24a0066730 0x7f24a0068bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24bc15b930 0x7f24bc06be00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24bc0b2ea0 0x7f24bc06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 --2- 172.21.15.179:0/3933468052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc0b2ac0 0x7f24bc15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 >> 172.21.15.179:0/3933468052 conn(0x7f24bc0cc5d0 msgr2=0x7f24bc15e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 shutdown_connections 2024-05-08T22:36:24.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.715+0000 7f24c2f4d700 1 -- 172.21.15.179:0/3933468052 wait complete. 2024-05-08T22:36:24.719 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:36:24.731 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:36:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 Processor -- start 2024-05-08T22:36:24.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 -- start start 2024-05-08T22:36:24.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f26a80b19e0 0x7f26a80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a8159700 0x7f26a806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a806cda0 con 0x7f26a8159320 2024-05-08T22:36:24.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a80cb4c0 con 0x7f26a80b19e0 2024-05-08T22:36:24.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae88b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80b1f10 con 0x7f26a8159700 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad889700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f26a8159320 0x7f26a80cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43670/0 (socket says 172.21.15.179:43670) 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad889700 1 -- 172.21.15.179:0/33343309 learned_addr learned my addr 172.21.15.179:0/33343309 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad088700 1 --2- 172.21.15.179:0/33343309 >> v2:172.21.15.179:3300/0 conn(0x7f26a80b19e0 0x7f26a80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ae08a700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a8159700 0x7f26a806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846019422 0 0) 0x7f26a806cda0 con 0x7f26a8159320 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26940035d0 con 0x7f26a8159320 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad088700 1 -- 172.21.15.179:0/33343309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a8159700 msgr2=0x7f26a806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 603770604 0 0) 0x7f26940035d0 con 0x7f26a8159320 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad088700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a8159700 0x7f26a806b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad088700 1 -- 172.21.15.179:0/33343309 >> v1:172.21.15.179:6789/0 conn(0x7f26a8159320 legacy=0x7f26a80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.847+0000 7f26ad088700 1 -- 172.21.15.179:0/33343309 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80b21f0 con 0x7f26a80b19e0 2024-05-08T22:36:24.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ad088700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a80b19e0 0x7f26a80cc630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2698009850 tx=0x7f2698009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a434b5c47042cded server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:24.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00019f0 con 0x7f26a80b19e0 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698004030 con 0x7f26a80b19e0 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ac887700 1 -- 172.21.15.179:0/33343309 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980043e0 con 0x7f26a80b19e0 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 -- 172.21.15.179:0/33343309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a80b19e0 msgr2=0x7f26a80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a80b19e0 0x7f26a80cc630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2698009850 tx=0x7f2698009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 -- 172.21.15.179:0/33343309 shutdown_connections 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a8159700 0x7f26a806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 --2- 172.21.15.179:0/33343309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a80b19e0 0x7f26a80cc630 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 -- 172.21.15.179:0/33343309 >> 172.21.15.179:0/33343309 conn(0x7f26a80cea30 msgr2=0x7f26a80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 -- 172.21.15.179:0/33343309 shutdown_connections 2024-05-08T22:36:24.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.851+0000 7f26ae88b700 1 -- 172.21.15.179:0/33343309 wait complete. 2024-05-08T22:36:24.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 Processor -- start 2024-05-08T22:36:24.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 -- start start 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a80b19e0 0x7f26a80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a8159700 0x7f26a8143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8143d70 con 0x7f26a8159320 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a814e170 con 0x7f26a8159700 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae88b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a814e450 con 0x7f26a80b19e0 2024-05-08T22:36:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49810/0 (socket says 172.21.15.179:49810) 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 -- 172.21.15.179:0/491307557 learned_addr learned my addr 172.21.15.179:0/491307557 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad889700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a80b19e0 0x7f26a80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ae08a700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a8159700 0x7f26a8143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 -- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a80b19e0 msgr2=0x7f26a80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a80b19e0 0x7f26a80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 -- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a8159700 msgr2=0x7f26a8143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a8159700 0x7f26a8143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 -- 172.21.15.179:0/491307557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980094b0 con 0x7f26a8159320 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.855+0000 7f26ad088700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2698000c00 tx=0x7f2698004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698003b00 con 0x7f26a8159320 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269801f070 con 0x7f26a8159320 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a814e730 con 0x7f26a8159320 2024-05-08T22:36:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011630 con 0x7f26a8159320 2024-05-08T22:36:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a814ecf0 con 0x7f26a8159320 2024-05-08T22:36:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a804e700 con 0x7f26a8159320 2024-05-08T22:36:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2698021030 con 0x7f26a8159320 2024-05-08T22:36:24.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 0x7f268c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:24.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f2698085310 con 0x7f26a8159320 2024-05-08T22:36:24.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.859+0000 7f26ad889700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 0x7f268c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.863+0000 7f26ad889700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 0x7f268c068c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26a00012e0 tx=0x7f26a00015b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:24.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:24.867+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2698052630 con 0x7f26a8159320 2024-05-08T22:36:25.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.075+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f26a814cea0 con 0x7f26a8159320 2024-05-08T22:36:25.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.083+0000 7f269e7fc700 1 -- 172.21.15.179:0/491307557 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2698055c50 con 0x7f26a8159320 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 msgr2=0x7f268c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 0x7f268c068c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26a00012e0 tx=0x7f26a00015b0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 msgr2=0x7f26a81432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2698000c00 tx=0x7f2698004770 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 shutdown_connections 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f268c066760 0x7f268c068c20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a8159700 0x7f26a8143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26a8159320 0x7f26a81432f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 --2- 172.21.15.179:0/491307557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26a80b19e0 0x7f26a80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 >> 172.21.15.179:0/491307557 conn(0x7f26a80cea30 msgr2=0x7f26a80ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 shutdown_connections 2024-05-08T22:36:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.087+0000 7f26ae88b700 1 -- 172.21.15.179:0/491307557 wait complete. 2024-05-08T22:36:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 9 and data pool 10 2024-05-08T22:36:25.107 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:25.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 Processor -- start 2024-05-08T22:36:25.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 -- start start 2024-05-08T22:36:25.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 0x7f17540cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17540c9230 0x7f17540c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17540c9bd0 con 0x7f1754160cb0 2024-05-08T22:36:25.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17540cd340 con 0x7f17540c9230 2024-05-08T22:36:25.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f175a0ef700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17540cd620 con 0x7f175415d130 2024-05-08T22:36:25.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f1753fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1754160cb0 0x7f1754161140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43684/0 (socket says 172.21.15.179:43684) 2024-05-08T22:36:25.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f1753fff700 1 -- 172.21.15.179:0/2594229881 learned_addr learned my addr 172.21.15.179:0/2594229881 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:25.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 --2- 172.21.15.179:0/2594229881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 0x7f17540cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f1752ffd700 1 --2- 172.21.15.179:0/2594229881 >> v2:172.21.15.195:3300/0 conn(0x7f17540c9230 0x7f17540c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166971402 0 0) 0x7f17540c9bd0 con 0x7f1754160cb0 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f173c0035d0 con 0x7f1754160cb0 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 -- 172.21.15.179:0/2594229881 >> v2:172.21.15.195:3300/0 conn(0x7f17540c9230 msgr2=0x7f17540c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 --2- 172.21.15.179:0/2594229881 >> v2:172.21.15.195:3300/0 conn(0x7f17540c9230 0x7f17540c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 -- 172.21.15.179:0/2594229881 >> v1:172.21.15.179:6789/0 conn(0x7f1754160cb0 legacy=0x7f1754161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216316894 0 0) 0x7f173c0035d0 con 0x7f1754160cb0 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 -- 172.21.15.179:0/2594229881 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540cd900 con 0x7f175415d130 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.207+0000 7f17537fe700 1 --2- 172.21.15.179:0/2594229881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 0x7f17540cdf40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f174c009d90 tx=0x7f174c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab494bb484effc90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00e070 con 0x7f175415d130 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c00bb20 con 0x7f175415d130 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f17527fc700 1 -- 172.21.15.179:0/2594229881 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00be90 con 0x7f175415d130 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- 172.21.15.179:0/2594229881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 msgr2=0x7f17540cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 --2- 172.21.15.179:0/2594229881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 0x7f17540cdf40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f174c009d90 tx=0x7f174c00b330 comp rx=0 tx=0).stop 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- 172.21.15.179:0/2594229881 shutdown_connections 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 --2- 172.21.15.179:0/2594229881 >> v2:172.21.15.195:3300/0 conn(0x7f17540c9230 0x7f17540c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 --2- 172.21.15.179:0/2594229881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175415d130 0x7f17540cdf40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- 172.21.15.179:0/2594229881 >> 172.21.15.179:0/2594229881 conn(0x7f17540cb660 msgr2=0x7f17540cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- 172.21.15.179:0/2594229881 shutdown_connections 2024-05-08T22:36:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- 172.21.15.179:0/2594229881 wait complete. 2024-05-08T22:36:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 Processor -- start 2024-05-08T22:36:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.211+0000 7f175a0ef700 1 -- start start 2024-05-08T22:36:25.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17540c9230 0x7f1754154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1754160cb0 0x7f1754159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17540c4160 con 0x7f175415d130 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175415a150 con 0x7f17540c9230 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175415a430 con 0x7f1754160cb0 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49814/0 (socket says 172.21.15.179:49814) 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 -- 172.21.15.179:0/1652121812 learned_addr learned my addr 172.21.15.179:0/1652121812 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f17537fe700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17540c9230 0x7f1754154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1753fff700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1754160cb0 0x7f1754159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 -- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1754160cb0 msgr2=0x7f1754159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1754160cb0 0x7f1754159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 -- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17540c9230 msgr2=0x7f1754154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17540c9230 0x7f1754154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 -- 172.21.15.179:0/1652121812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c009a70 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f1752ffd700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f174800e9d0 tx=0x7f174800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748009720 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174800ee70 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17540c3b10 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800ca50 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1754159d60 con 0x7f175415d130 2024-05-08T22:36:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.215+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1754067770 con 0x7f175415d130 2024-05-08T22:36:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.219+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f174800cbb0 con 0x7f175415d130 2024-05-08T22:36:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.219+0000 7f173bfff700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 0x7f1740068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.219+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f17480842c0 con 0x7f175415d130 2024-05-08T22:36:25.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.219+0000 7f17537fe700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 0x7f1740068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.219+0000 7f17537fe700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 0x7f1740068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f174c014040 tx=0x7f174c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.223+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17480514f0 con 0x7f175415d130 2024-05-08T22:36:25.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.395+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1754073290 con 0x7f175415d130 2024-05-08T22:36:25.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f173bfff700 1 -- 172.21.15.179:0/1652121812 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f174800ce60 con 0x7f175415d130 2024-05-08T22:36:25.399 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:25.399 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:25.076881+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:25.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 msgr2=0x7f1740068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 0x7f1740068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f174c014040 tx=0x7f174c011000 comp rx=0 tx=0).stop 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 msgr2=0x7f17540b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f174800e9d0 tx=0x7f174800ece0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 shutdown_connections 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1740066730 0x7f1740068bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1754160cb0 0x7f1754159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f175415d130 0x7f17540b1840 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 --2- 172.21.15.179:0/1652121812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17540c9230 0x7f1754154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 >> 172.21.15.179:0/1652121812 conn(0x7f17540cb660 msgr2=0x7f175406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 shutdown_connections 2024-05-08T22:36:25.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.399+0000 7f175a0ef700 1 -- 172.21.15.179:0/1652121812 wait complete. 2024-05-08T22:36:25.414 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:25.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 Processor -- start 2024-05-08T22:36:25.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- start start 2024-05-08T22:36:25.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 0x7f53280ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532815b1b0 0x7f53280cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53280cb510 0x7f532815e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280cb9a0 con 0x7f53280cc700 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532806abb0 con 0x7f53280cb510 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532806ae90 con 0x7f532815b1b0 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5326d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 0x7f53280ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53280cb510 0x7f532815e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5326d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 0x7f53280ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55796/0 (socket says 172.21.15.179:55796) 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5326d9d700 1 -- 172.21.15.179:0/2777793307 learned_addr learned my addr 172.21.15.179:0/2777793307 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532659c700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532815b1b0 0x7f53280cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 -- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532815b1b0 msgr2=0x7f53280cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:25.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532815b1b0 0x7f53280cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 -- 172.21.15.179:0/2777793307 >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 msgr2=0x7f53280ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:25.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 --2- 172.21.15.179:0/2777793307 >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 0x7f53280ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 -- 172.21.15.179:0/2777793307 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532815b9c0 con 0x7f53280cb510 2024-05-08T22:36:25.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532759e700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53280cb510 0x7f532815e7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f53280708c0 tx=0x7f532000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a16a613218a65bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5325d9b700 1 -- 172.21.15.179:0/2777793307 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000e070 con 0x7f53280cb510 2024-05-08T22:36:25.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5325d9b700 1 -- 172.21.15.179:0/2777793307 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532000bb20 con 0x7f53280cb510 2024-05-08T22:36:25.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f5325d9b700 1 -- 172.21.15.179:0/2777793307 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320012430 con 0x7f53280cb510 2024-05-08T22:36:25.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53280cb510 msgr2=0x7f532815e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53280cb510 0x7f532815e7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f53280708c0 tx=0x7f532000b330 comp rx=0 tx=0).stop 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- 172.21.15.179:0/2777793307 shutdown_connections 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53280cb510 0x7f532815e7d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- 172.21.15.179:0/2777793307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532815b1b0 0x7f53280cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 --2- 172.21.15.179:0/2777793307 >> v2:172.21.15.179:3300/0 conn(0x7f53280cc700 0x7f53280ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- 172.21.15.179:0/2777793307 >> 172.21.15.179:0/2777793307 conn(0x7f53280ce9d0 msgr2=0x7f53280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- 172.21.15.179:0/2777793307 shutdown_connections 2024-05-08T22:36:25.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.527+0000 7f532c962700 1 -- 172.21.15.179:0/2777793307 wait complete. 2024-05-08T22:36:25.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.531+0000 7f532c962700 1 Processor -- start 2024-05-08T22:36:25.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.531+0000 7f532c962700 1 -- start start 2024-05-08T22:36:25.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 0x7f53281575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 0x7f5328157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328153c00 con 0x7f53280cb510 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328140be0 con 0x7f532815b1b0 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5328153080 con 0x7f53280cc700 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55798/0 (socket says 172.21.15.179:55798) 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 -- 172.21.15.179:0/1464797030 learned_addr learned my addr 172.21.15.179:0/1464797030 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532759e700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 0x7f5328157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532659c700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 0x7f53281575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 -- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 msgr2=0x7f53281575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 0x7f53281575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 -- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 msgr2=0x7f5328157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 0x7f5328157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 -- 172.21.15.179:0/1464797030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5320009a70 con 0x7f53280cb510 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532759e700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 0x7f5328157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532659c700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 0x7f53281575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f5326d9d700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f531000e960 tx=0x7f531000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53100096c0 con 0x7f53280cb510 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531000c850 con 0x7f53280cb510 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5328153360 con 0x7f53280cb510 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000cc00 con 0x7f53280cb510 2024-05-08T22:36:25.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5328153920 con 0x7f53280cb510 2024-05-08T22:36:25.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.535+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5328067740 con 0x7f53280cb510 2024-05-08T22:36:25.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.539+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f531000cd60 con 0x7f53280cb510 2024-05-08T22:36:25.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.539+0000 7f531f7fe700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 0x7f5308068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.539+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5310084370 con 0x7f53280cb510 2024-05-08T22:36:25.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.539+0000 7f532659c700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 0x7f5308068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.539+0000 7f532659c700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 0x7f5308068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f532806a750 tx=0x7f5318008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.543+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f53100515a0 con 0x7f53280cb510 2024-05-08T22:36:25.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.719+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f532804e700 con 0x7f53280cb510 2024-05-08T22:36:25.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.719+0000 7f531f7fe700 1 -- 172.21.15.179:0/1464797030 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f5310054bc0 con 0x7f53280cb510 2024-05-08T22:36:25.721 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:25.721 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:25.076881+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 msgr2=0x7f5308068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 0x7f5308068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f532806a750 tx=0x7f5318008040 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 msgr2=0x7f53281406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f531000e960 tx=0x7f531000ed20 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 shutdown_connections 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066730 0x7f5308068bf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532815b1b0 0x7f5328157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280cc700 0x7f53281575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 --2- 172.21.15.179:0/1464797030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cb510 0x7f53281406a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 >> 172.21.15.179:0/1464797030 conn(0x7f53280ce9d0 msgr2=0x7f53281584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 shutdown_connections 2024-05-08T22:36:25.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.723+0000 7f532c962700 1 -- 172.21.15.179:0/1464797030 wait complete. 2024-05-08T22:36:25.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:36:25.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.851+0000 7f338f293700 1 Processor -- start 2024-05-08T22:36:25.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.851+0000 7f338f293700 1 -- start start 2024-05-08T22:36:25.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33880d16b0 con 0x7f338815d130 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33880ce0d0 con 0x7f33880cd710 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880d1990 con 0x7f33880ce2b0 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338ea92700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338e291700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f338815d130 0x7f33880d2750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:56470/0 (socket says 172.21.15.179:56470) 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338ea92700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:35032/0 (socket says 172.21.15.179:35032) 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:37834/0 (socket says 172.21.15.179:37834) 2024-05-08T22:36:25.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338e291700 1 -- 172.21.15.179:0/2050689670 learned_addr learned my addr 172.21.15.179:0/2050689670 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:25.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 -- 172.21.15.179:0/2050689670 >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 msgr2=0x7f33880cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:25.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 --2- 172.21.15.179:0/2050689670 >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 -- 172.21.15.179:0/2050689670 >> v1:172.21.15.195:6789/0 conn(0x7f338815d130 legacy=0x7f33880d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 -- 172.21.15.179:0/2050689670 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880d1c70 con 0x7f33880ce2b0 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338d28f700 1 -- 172.21.15.179:0/2050689670 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564547995 0 0) 0x7f33880d16b0 con 0x7f338815d130 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338ea92700 1 --2- 172.21.15.179:0/2050689670 >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338da90700 1 --2- 172.21.15.179:0/2050689670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3384009850 tx=0x7f3384009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85afcce19aaaf25b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338d28f700 1 -- 172.21.15.179:0/2050689670 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384011830 con 0x7f33880ce2b0 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338d28f700 1 -- 172.21.15.179:0/2050689670 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3384011990 con 0x7f33880ce2b0 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338d28f700 1 -- 172.21.15.179:0/2050689670 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384011d40 con 0x7f33880ce2b0 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- 172.21.15.179:0/2050689670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 msgr2=0x7f33880d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 --2- 172.21.15.179:0/2050689670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3384009850 tx=0x7f3384009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- 172.21.15.179:0/2050689670 shutdown_connections 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 --2- 172.21.15.179:0/2050689670 >> v2:172.21.15.195:3300/0 conn(0x7f33880cd710 0x7f33880cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 --2- 172.21.15.179:0/2050689670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f33880d0db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- 172.21.15.179:0/2050689670 >> 172.21.15.179:0/2050689670 conn(0x7f33880b49a0 msgr2=0x7f33880b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:25.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- 172.21.15.179:0/2050689670 shutdown_connections 2024-05-08T22:36:25.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.855+0000 7f338f293700 1 -- 172.21.15.179:0/2050689670 wait complete. 2024-05-08T22:36:25.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 Processor -- start 2024-05-08T22:36:25.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 -- start start 2024-05-08T22:36:25.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f3388147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f338815d130 0x7f338806bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33881464f0 con 0x7f33880cd710 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3388148a70 con 0x7f338815d130 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338f293700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3388148d50 con 0x7f33880ce2b0 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55804/0 (socket says 172.21.15.179:55804) 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 -- 172.21.15.179:0/3292391168 learned_addr learned my addr 172.21.15.179:0/3292391168 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338da90700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f3388147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338ea92700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f338815d130 0x7f338806bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 -- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 msgr2=0x7f3388147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f3388147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 -- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f338815d130 msgr2=0x7f338806bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:25.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f338815d130 0x7f338806bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 -- 172.21.15.179:0/3292391168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009a70 con 0x7f33880cd710 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f338e291700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f337800c4e0 tx=0x7f337800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.859+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378004ca0 con 0x7f33880cd710 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33840094b0 con 0x7f33880cd710 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378004e20 con 0x7f33880cd710 2024-05-08T22:36:25.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338815d690 con 0x7f33880cd710 2024-05-08T22:36:25.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780072d0 con 0x7f33880cd710 2024-05-08T22:36:25.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3388148f60 con 0x7f33880cd710 2024-05-08T22:36:25.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3378007430 con 0x7f33880cd710 2024-05-08T22:36:25.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f337effd700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 0x7f336c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:25.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3378083db0 con 0x7f33880cd710 2024-05-08T22:36:25.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f338da90700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 0x7f336c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:25.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.863+0000 7f338da90700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 0x7f336c068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3384009850 tx=0x7f3384008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:25.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:25.867+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3378050ed0 con 0x7f33880cd710 2024-05-08T22:36:26.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.075+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f338804e700 con 0x7f33880cd710 2024-05-08T22:36:26.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.087+0000 7f337effd700 1 -- 172.21.15.179:0/3292391168 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33780544f0 con 0x7f33880cd710 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 msgr2=0x7f336c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 0x7f336c068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3384009850 tx=0x7f3384008030 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 msgr2=0x7f33880b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f337800c4e0 tx=0x7f337800c7f0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 shutdown_connections 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f336c066730 0x7f336c068bf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f338815d130 0x7f338806bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33880ce2b0 0x7f3388147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 --2- 172.21.15.179:0/3292391168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33880cd710 0x7f33880b1580 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 >> 172.21.15.179:0/3292391168 conn(0x7f33880b49a0 msgr2=0x7f33880cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 shutdown_connections 2024-05-08T22:36:26.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.091+0000 7f338f293700 1 -- 172.21.15.179:0/3292391168 wait complete. 2024-05-08T22:36:26.111 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:36:26.112 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:36:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 Processor -- start 2024-05-08T22:36:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 -- start start 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91b815d130 0x7f91b80d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80d2c90 con 0x7f91b80ce2b0 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80d16b0 con 0x7f91b80cd710 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bddf7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80d14c0 con 0x7f91b815d130 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91b7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f91b80ce2b0 0x7f91b80d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54700/0 (socket says 172.21.15.179:54700) 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91b7fff700 1 -- 172.21.15.179:0/1290393875 learned_addr learned my addr 172.21.15.179:0/1290393875 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bcdf5700 1 --2- 172.21.15.179:0/1290393875 >> v2:172.21.15.179:3300/0 conn(0x7f91b815d130 0x7f91b80d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:26.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bcdf5700 1 -- 172.21.15.179:0/1290393875 >> v1:172.21.15.195:6789/0 conn(0x7f91b80cd710 legacy=0x7f91b80b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bcdf5700 1 -- 172.21.15.179:0/1290393875 >> v1:172.21.15.179:6789/0 conn(0x7f91b80ce2b0 legacy=0x7f91b80d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91bcdf5700 1 -- 172.21.15.179:0/1290393875 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80d1990 con 0x7f91b815d130 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.235+0000 7f91b77fe700 1 -- 172.21.15.179:0/1290393875 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805284145 0 0) 0x7f91b80d2c90 con 0x7f91b80ce2b0 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bcdf5700 1 --2- 172.21.15.179:0/1290393875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b80d2750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009850 tx=0x7f91ac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8316ad30ff30752c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91b77fe700 1 -- 172.21.15.179:0/1290393875 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011830 con 0x7f91b815d130 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91b77fe700 1 -- 172.21.15.179:0/1290393875 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac011990 con 0x7f91b815d130 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91b77fe700 1 -- 172.21.15.179:0/1290393875 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011d20 con 0x7f91b815d130 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- 172.21.15.179:0/1290393875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 msgr2=0x7f91b80d2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 --2- 172.21.15.179:0/1290393875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b80d2750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009850 tx=0x7f91ac009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- 172.21.15.179:0/1290393875 shutdown_connections 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 --2- 172.21.15.179:0/1290393875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b80d2750 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- 172.21.15.179:0/1290393875 >> 172.21.15.179:0/1290393875 conn(0x7f91b80b49a0 msgr2=0x7f91b80b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- 172.21.15.179:0/1290393875 shutdown_connections 2024-05-08T22:36:26.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- 172.21.15.179:0/1290393875 wait complete. 2024-05-08T22:36:26.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 Processor -- start 2024-05-08T22:36:26.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.239+0000 7f91bddf7700 1 -- start start 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91b80cd710 0x7f91b815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91b80ce2b0 0x7f91b8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8157750 con 0x7f91b815d130 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815ea00 con 0x7f91b80ce2b0 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8156b40 con 0x7f91b80cd710 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:26.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55820/0 (socket says 172.21.15.179:55820) 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 -- 172.21.15.179:0/786438340 learned_addr learned my addr 172.21.15.179:0/786438340 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91b7fff700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91b80ce2b0 0x7f91b8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bcdf5700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91b80cd710 0x7f91b815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 -- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91b80cd710 msgr2=0x7f91b815e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91b80cd710 0x7f91b815e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 -- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91b80ce2b0 msgr2=0x7f91b8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91b80ce2b0 0x7f91b8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 -- 172.21.15.179:0/786438340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0094b0 con 0x7f91b815d130 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bd5f6700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f91b000cb50 tx=0x7f91b000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003c20 con 0x7f91b815d130 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b0003d80 con 0x7f91b815d130 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b8156e20 con 0x7f91b815d130 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000aac0 con 0x7f91b815d130 2024-05-08T22:36:26.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81573e0 con 0x7f91b815d130 2024-05-08T22:36:26.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.243+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b81578c0 con 0x7f91b815d130 2024-05-08T22:36:26.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.247+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91b000ac20 con 0x7f91b815d130 2024-05-08T22:36:26.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.247+0000 7f91b57fa700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 0x7f91a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:26.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.247+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f91b0084240 con 0x7f91b815d130 2024-05-08T22:36:26.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.247+0000 7f91bcdf5700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 0x7f91a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:26.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.247+0000 7f91bcdf5700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 0x7f91a4068bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0090b0 tx=0x7f91ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:26.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.255+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f91b0051500 con 0x7f91b815d130 2024-05-08T22:36:26.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:26.463+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91b804e700 con 0x7f91b815d130 2024-05-08T22:36:27.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.091+0000 7f91b57fa700 1 -- 172.21.15.179:0/786438340 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91b0054b20 con 0x7f91b815d130 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 msgr2=0x7f91a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 0x7f91a4068bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0090b0 tx=0x7f91ac008030 comp rx=0 tx=0).stop 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 msgr2=0x7f91b8159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f91b000cb50 tx=0x7f91b000ce60 comp rx=0 tx=0).stop 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 shutdown_connections 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f91a4066730 0x7f91a4068bf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91b80ce2b0 0x7f91b8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b815d130 0x7f91b8159530 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 --2- 172.21.15.179:0/786438340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91b80cd710 0x7f91b815e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 >> 172.21.15.179:0/786438340 conn(0x7f91b80b49a0 msgr2=0x7f91b815c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 shutdown_connections 2024-05-08T22:36:27.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.099+0000 7f91bddf7700 1 -- 172.21.15.179:0/786438340 wait complete. 2024-05-08T22:36:27.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:36:27.116 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 Processor -- start 2024-05-08T22:36:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- start start 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f97ac06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac161ac0 con 0x7f97ac0cc310 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0cdaa0 con 0x7f97ac0add40 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ae550 con 0x7f97ac0ce060 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f97ac0cc310 0x7f97ac0ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54714/0 (socket says 172.21.15.179:54714) 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 -- 172.21.15.179:0/3787728481 learned_addr learned my addr 172.21.15.179:0/3787728481 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587987290 0 0) 0x7f97ac161ac0 con 0x7f97ac0cc310 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97940035d0 con 0x7f97ac0cc310 2024-05-08T22:36:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97abfff700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f97ac06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831055319 0 0) 0x7f97940035d0 con 0x7f97ac0cc310 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 msgr2=0x7f97ac06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f97ac06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 >> v1:172.21.15.195:6789/0 conn(0x7f97ac0add40 legacy=0x7f97ac06b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac06adc0 con 0x7f97ac0cc310 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f979c0019f0 con 0x7f97ac0cc310 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 >> v1:172.21.15.179:6789/0 conn(0x7f97ac0cc310 legacy=0x7f97ac0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9794003a50 0x7f9794003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9794004870 0x7f9794004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f9794005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ae550 con 0x7f9794004870 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979400bb60 con 0x7f9794003a50 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979400bd20 con 0x7f97ac0ce060 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9794004870 0x7f9794004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97abfff700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9794003a50 0x7f9794003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b09ae700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f9794005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 -- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 msgr2=0x7f9794005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f9794005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 -- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9794003a50 msgr2=0x7f9794003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9794003a50 0x7f9794003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 -- 172.21.15.179:0/3787728481 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940035d0 con 0x7f9794004870 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b09ae700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97ac0ce060 0x7f9794005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97ab7fe700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9794004870 0x7f9794004cf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f979c00a830 tx=0x7f97940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d65ea12e668332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00a2f0 con 0x7f9794004870 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0072f0 con 0x7f9794004870 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97aaffd700 1 -- 172.21.15.179:0/3787728481 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c0076a0 con 0x7f9794004870 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- 172.21.15.179:0/3787728481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9794004870 msgr2=0x7f9794004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 --2- 172.21.15.179:0/3787728481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9794004870 0x7f9794004cf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f979c00a830 tx=0x7f97940058a0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b09ae700 1 -- 172.21.15.179:0/3787728481 reap_dead start 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- 172.21.15.179:0/3787728481 shutdown_connections 2024-05-08T22:36:27.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.239+0000 7f97b2411700 1 -- 172.21.15.179:0/3787728481 >> 172.21.15.179:0/3787728481 conn(0x7f97ac0cb660 msgr2=0x7f97ac0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- 172.21.15.179:0/3787728481 shutdown_connections 2024-05-08T22:36:27.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- 172.21.15.179:0/3787728481 wait complete. 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 Processor -- start 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- start start 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9794004870 0x7f97ac0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97ac0c4af0 0x7f97ac0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b40c0 con 0x7f97ac1587a0 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac159190 con 0x7f97ac0c4af0 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b2411700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b2120 con 0x7f9794004870 2024-05-08T22:36:27.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55832/0 (socket says 172.21.15.179:55832) 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 -- 172.21.15.179:0/2113277541 learned_addr learned my addr 172.21.15.179:0/2113277541 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97abfff700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9794004870 0x7f97ac0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97ab7fe700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97ac0c4af0 0x7f97ac0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 -- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9794004870 msgr2=0x7f97ac0b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9794004870 0x7f97ac0b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 -- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97ac0c4af0 msgr2=0x7f97ac0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97ac0c4af0 0x7f97ac0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.243+0000 7f97b09ae700 1 -- 172.21.15.179:0/2113277541 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0b2400 con 0x7f97ac1587a0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97b09ae700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f97ac067690 tx=0x7f97a400d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400b070 con 0x7f97ac1587a0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4005310 con 0x7f97ac1587a0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac0b26e0 con 0x7f97ac1587a0 2024-05-08T22:36:27.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400dee0 con 0x7f97ac1587a0 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac0b1d80 con 0x7f97ac1587a0 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac067770 con 0x7f97ac1587a0 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.247+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97a4006880 con 0x7f97ac1587a0 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.251+0000 7f97a8ff9700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 0x7f9798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.251+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97a4083a60 con 0x7f97ac1587a0 2024-05-08T22:36:27.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.251+0000 7f97abfff700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 0x7f9798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.251+0000 7f97abfff700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 0x7f9798068bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009930 tx=0x7f97a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.259+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97a4050d20 con 0x7f97ac1587a0 2024-05-08T22:36:27.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ac0b2c00 con 0x7f97ac1587a0 2024-05-08T22:36:27.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97a8ff9700 1 -- 172.21.15.179:0/2113277541 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f97a4054340 con 0x7f97ac1587a0 2024-05-08T22:36:27.463 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:27.463 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:36:27.087228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 msgr2=0x7f9798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 0x7f9798068bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009930 tx=0x7f97a0008040 comp rx=0 tx=0).stop 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 msgr2=0x7f97ac158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f97ac067690 tx=0x7f97a400d6e0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 shutdown_connections 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9798066730 0x7f9798068bf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97ac0c4af0 0x7f97ac0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ac1587a0 0x7f97ac158c50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 --2- 172.21.15.179:0/2113277541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9794004870 0x7f97ac0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 >> 172.21.15.179:0/2113277541 conn(0x7f97ac0cb660 msgr2=0x7f97ac0af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 shutdown_connections 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.463+0000 7f97b2411700 1 -- 172.21.15.179:0/2113277541 wait complete. 2024-05-08T22:36:27.467 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 30 2024-05-08T22:36:27.478 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:27.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 Processor -- start 2024-05-08T22:36:27.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- start start 2024-05-08T22:36:27.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0db40d0650 0x7f0db40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 0x7f0db40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db40b5500 con 0x7f0db41447c0 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db40b4630 con 0x7f0db40d0650 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40b48f0 con 0x7f0db40b41c0 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0db40d0650 0x7f0db40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0db40d0650 0x7f0db40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55848/0 (socket says 172.21.15.179:55848) 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 -- 172.21.15.179:0/2668070334 learned_addr learned my addr 172.21.15.179:0/2668070334 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db9fce700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 0x7f0db40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 -- 172.21.15.179:0/2668070334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 msgr2=0x7f0db40b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 0x7f0db40b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 -- 172.21.15.179:0/2668070334 >> v1:172.21.15.195:6789/0 conn(0x7f0db41447c0 legacy=0x7f0db40cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 -- 172.21.15.179:0/2668070334 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db40b4bd0 con 0x7f0db40d0650 2024-05-08T22:36:27.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db9fce700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 0x7f0db40b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:27.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0db8fcc700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db40ce700 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009910 tx=0x7f0da4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865a2226f58a29a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dabfff700 1 -- 172.21.15.179:0/2668070334 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da400ba60 con 0x7f0db40d0650 2024-05-08T22:36:27.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dabfff700 1 -- 172.21.15.179:0/2668070334 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da400bbc0 con 0x7f0db40d0650 2024-05-08T22:36:27.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dabfff700 1 -- 172.21.15.179:0/2668070334 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4009e60 con 0x7f0db40d0650 2024-05-08T22:36:27.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- 172.21.15.179:0/2668070334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 msgr2=0x7f0db40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db40ce700 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009910 tx=0x7f0da4009c20 comp rx=0 tx=0).stop 2024-05-08T22:36:27.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- 172.21.15.179:0/2668070334 shutdown_connections 2024-05-08T22:36:27.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db40b41c0 0x7f0db40b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/2668070334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db40ce700 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- 172.21.15.179:0/2668070334 >> 172.21.15.179:0/2668070334 conn(0x7f0db40b2840 msgr2=0x7f0db40b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- 172.21.15.179:0/2668070334 shutdown_connections 2024-05-08T22:36:27.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.591+0000 7f0dba7cf700 1 -- 172.21.15.179:0/2668070334 wait complete. 2024-05-08T22:36:27.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.595+0000 7f0dba7cf700 1 Processor -- start 2024-05-08T22:36:27.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.595+0000 7f0dba7cf700 1 -- start start 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0db40b41c0 0x7f0db413d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db41447c0 0x7f0db413f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db413ec60 con 0x7f0db40d0650 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4140720 con 0x7f0db40b41c0 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db413eeb0 con 0x7f0db41447c0 2024-05-08T22:36:27.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55860/0 (socket says 172.21.15.179:55860) 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 -- 172.21.15.179:0/1728781661 learned_addr learned my addr 172.21.15.179:0/1728781661 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db97cd700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0db40b41c0 0x7f0db413d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db9fce700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db41447c0 0x7f0db413f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 -- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db41447c0 msgr2=0x7f0db413f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db41447c0 0x7f0db413f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 -- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0db40b41c0 msgr2=0x7f0db413d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0db40b41c0 0x7f0db413d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 -- 172.21.15.179:0/1728781661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4009570 con 0x7f0db40d0650 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0db8fcc700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0db4067a00 tx=0x7f0da4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4004590 con 0x7f0db40d0650 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da401f070 con 0x7f0db40d0650 2024-05-08T22:36:27.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db413f190 con 0x7f0db40d0650 2024-05-08T22:36:27.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4003900 con 0x7f0db40d0650 2024-05-08T22:36:27.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db413fed0 con 0x7f0db40d0650 2024-05-08T22:36:27.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.599+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db413f650 con 0x7f0db40d0650 2024-05-08T22:36:27.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.603+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0da4003a60 con 0x7f0db40d0650 2024-05-08T22:36:27.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.603+0000 7f0da9ffb700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 0x7f0d98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.603+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0da4021070 con 0x7f0db40d0650 2024-05-08T22:36:27.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.603+0000 7f0db97cd700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 0x7f0d98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.603+0000 7f0db97cd700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 0x7f0d98068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000f60 tx=0x7f0db0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.611+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0da4052d70 con 0x7f0db40d0650 2024-05-08T22:36:27.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.783+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0db404e700 con 0x7f0db40d0650 2024-05-08T22:36:27.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.787+0000 7f0da9ffb700 1 -- 172.21.15.179:0/1728781661 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16296 (secure 0 0 0) 0x7f0da4015090 con 0x7f0db40d0650 2024-05-08T22:36:27.788 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:27.789 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":69,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:36:26.080121+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 msgr2=0x7f0d98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 0x7f0d98068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000f60 tx=0x7f0db0009040 comp rx=0 tx=0).stop 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 msgr2=0x7f0db413d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0db4067a00 tx=0x7f0da4004770 comp rx=0 tx=0).stop 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 shutdown_connections 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0d98066730 0x7f0d98068bf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0db41447c0 0x7f0db413f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db40d0650 0x7f0db413d930 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 --2- 172.21.15.179:0/1728781661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0db40b41c0 0x7f0db413d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 >> 172.21.15.179:0/1728781661 conn(0x7f0db40b2840 msgr2=0x7f0db40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 shutdown_connections 2024-05-08T22:36:27.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.791+0000 7f0dba7cf700 1 -- 172.21.15.179:0/1728781661 wait complete. 2024-05-08T22:36:27.805 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:36:27.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:36:27.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 Processor -- start 2024-05-08T22:36:27.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- start start 2024-05-08T22:36:27.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf2c0cc700 0x7fcf2c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf2c15b1b0 0x7fcf2c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c06abb0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c0cb980 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c06ae90 con 0x7fcf2c0cc700 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf2c15b1b0 0x7fcf2c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf30b6f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcf2c0cb510 0x7fcf2c15e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54726/0 (socket says 172.21.15.179:54726) 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf30b6f700 1 -- 172.21.15.179:0/700921209 learned_addr learned my addr 172.21.15.179:0/700921209 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2bfff700 1 --2- 172.21.15.179:0/700921209 >> v2:172.21.15.195:3300/0 conn(0x7fcf2c0cc700 0x7fcf2c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 -- 172.21.15.179:0/700921209 >> v2:172.21.15.195:3300/0 conn(0x7fcf2c0cc700 msgr2=0x7fcf2c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 --2- 172.21.15.179:0/700921209 >> v2:172.21.15.195:3300/0 conn(0x7fcf2c0cc700 0x7fcf2c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 -- 172.21.15.179:0/700921209 >> v1:172.21.15.179:6789/0 conn(0x7fcf2c0cb510 legacy=0x7fcf2c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 -- 172.21.15.179:0/700921209 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c15b9c0 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2b7fe700 1 --2- 172.21.15.179:0/700921209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c0cafd0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c070800 tx=0x7fcf1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad7d42d93f8f79d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2affd700 1 -- 172.21.15.179:0/700921209 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00e070 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2affd700 1 -- 172.21.15.179:0/700921209 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c00bbd0 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf2affd700 1 -- 172.21.15.179:0/700921209 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c012430 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- 172.21.15.179:0/700921209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c15b1b0 msgr2=0x7fcf2c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 --2- 172.21.15.179:0/700921209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c0cafd0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c070800 tx=0x7fcf1c00b330 comp rx=0 tx=0).stop 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- 172.21.15.179:0/700921209 shutdown_connections 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 --2- 172.21.15.179:0/700921209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c0cafd0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 --2- 172.21.15.179:0/700921209 >> v2:172.21.15.195:3300/0 conn(0x7fcf2c0cc700 0x7fcf2c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.919+0000 7fcf31370700 1 -- 172.21.15.179:0/700921209 >> 172.21.15.179:0/700921209 conn(0x7fcf2c0ce9d0 msgr2=0x7fcf2c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:27.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- 172.21.15.179:0/700921209 shutdown_connections 2024-05-08T22:36:27.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- 172.21.15.179:0/700921209 wait complete. 2024-05-08T22:36:27.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 Processor -- start 2024-05-08T22:36:27.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- start start 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf2c0cc700 0x7fcf2c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c13def0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0a8470 con 0x7fcf2c15b1b0 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf31370700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c13d5c0 con 0x7fcf2c0cc700 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55886/0 (socket says 172.21.15.179:55886) 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf2bfff700 1 -- 172.21.15.179:0/3491158577 learned_addr learned my addr 172.21.15.179:0/3491158577 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf30b6f700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.923+0000 7fcf2b7fe700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf2c0cc700 0x7fcf2c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 -- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf2c0cc700 msgr2=0x7fcf2c06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf2c0cc700 0x7fcf2c06e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 -- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf2c15b1b0 msgr2=0x7fcf2c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 -- 172.21.15.179:0/3491158577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c009a70 con 0x7fcf2c0cb510 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf2bfff700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400d800 tx=0x7fcf1400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140125f0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf14012750 con 0x7fcf2c0cb510 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14012ac0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c13d8a0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c13d030 con 0x7fcf2c0cb510 2024-05-08T22:36:27.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c067740 con 0x7fcf2c0cb510 2024-05-08T22:36:27.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.927+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf14012c20 con 0x7fcf2c0cb510 2024-05-08T22:36:27.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.931+0000 7fcf28ff9700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 0x7fcf0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:27.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.931+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcf140842b0 con 0x7fcf2c0cb510 2024-05-08T22:36:27.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.931+0000 7fcf2b7fe700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 0x7fcf0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:27.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.931+0000 7fcf2b7fe700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 0x7fcf0c068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c014040 tx=0x7fcf1c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:27.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:27.935+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcf140515a0 con 0x7fcf2c0cb510 2024-05-08T22:36:28.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.103+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf2c13d220 con 0x7fcf0c066730 2024-05-08T22:36:28.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.115+0000 7fcf28ff9700 1 -- 172.21.15.179:0/3491158577 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274273 (secure 0 0 0) 0x7fcf2c13d220 con 0x7fcf0c066730 2024-05-08T22:36:28.120 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 msgr2=0x7fcf0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 0x7fcf0c068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c014040 tx=0x7fcf1c011000 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 msgr2=0x7fcf2c13caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400d800 tx=0x7fcf1400db10 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 shutdown_connections 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcf0c066730 0x7fcf0c068bf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf2c15b1b0 0x7fcf2c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf2c0cc700 0x7fcf2c06e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 --2- 172.21.15.179:0/3491158577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c0cb510 0x7fcf2c13caf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 >> 172.21.15.179:0/3491158577 conn(0x7fcf2c0ce9d0 msgr2=0x7fcf2c15e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 shutdown_connections 2024-05-08T22:36:28.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.119+0000 7fcf31370700 1 -- 172.21.15.179:0/3491158577 wait complete. 2024-05-08T22:36:28.124 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:36:28.135 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-08T22:36:27.444758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":243948,"kb_used_data":7252,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730772,"statfs":{"total":773094113280,"available":772844310528,"internally_reserved":0,"allocated":7426048,"data_stored":3947305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001101"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.698577+0000","last_change":"2024-05-08T22:36:23.698577+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.698577+0000","last_undegraded":"2024-05-08T22:36:23.698577+0000","last_fullsized":"2024-05-08T22:36:23.698577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699742+0000","last_change":"2024-05-08T22:36:23.699742+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699742+0000","last_undegraded":"2024-05-08T22:36:23.699742+0000","last_fullsized":"2024-05-08T22:36:23.699742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.700977+0000","last_change":"2024-05-08T22:36:23.700977+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.700977+0000","last_undegraded":"2024-05-08T22:36:23.700977+0000","last_fullsized":"2024-05-08T22:36:23.700977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701304+0000","last_change":"2024-05-08T22:36:23.701304+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701304+0000","last_undegraded":"2024-05-08T22:36:23.701304+0000","last_fullsized":"2024-05-08T22:36:23.701304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.697706+0000","last_change":"2024-05-08T22:36:23.697706+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.697706+0000","last_undegraded":"2024-05-08T22:36:23.697706+0000","last_fullsized":"2024-05-08T22:36:23.697706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701388+0000","last_change":"2024-05-08T22:36:23.701388+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701388+0000","last_undegraded":"2024-05-08T22:36:23.701388+0000","last_fullsized":"2024-05-08T22:36:23.701388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696746+0000","last_change":"2024-05-08T22:36:23.696746+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696746+0000","last_undegraded":"2024-05-08T22:36:23.696746+0000","last_fullsized":"2024-05-08T22:36:23.696746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699737+0000","last_change":"2024-05-08T22:36:23.699737+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699737+0000","last_undegraded":"2024-05-08T22:36:23.699737+0000","last_fullsized":"2024-05-08T22:36:23.699737+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696747+0000","last_change":"2024-05-08T22:36:23.696747+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696747+0000","last_undegraded":"2024-05-08T22:36:23.696747+0000","last_fullsized":"2024-05-08T22:36:23.696747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T22:36:20.613925+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:36:20.613925+0000","last_peered":"2024-05-08T22:36:20.613925+0000","last_clean":"2024-05-08T22:36:20.613925+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:36:20.613925+0000","last_undegraded":"2024-05-08T22:36:20.613925+0000","last_fullsized":"2024-05-08T22:36:20.613925+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63},"min":{"1min":0.44500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.012},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0529999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97099999999999997,"5min":1.101,"15min":1.101},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.51000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0089999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0649999999999999,"5min":1.21,"15min":1.21},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.92100000000000004},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2330000000000001,"5min":1.306,"15min":1.306},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":0.94099999999999995},"min":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.234,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.008,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2649999999999999,"5min":1.476,"15min":1.476},"last":1.097}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.68600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3069999999999999,"5min":1.446,"15min":1.446},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2809999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.095,"15min":1.107},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4510000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.167},{"interface":"front","average":{"1min":1,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.347,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.121}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":351191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.04,"15min":1.04},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.046,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.371,"15min":0.371},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.3280000000000001,"5min":1.431,"15min":1.431},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.96699999999999997},"min":{"1min":0.75,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.226,"5min":1.508,"15min":1.508},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.268,"5min":1.327,"15min":1.327},"last":1.042},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.093},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.268,"5min":1.726,"15min":1.726},"last":1.397}]},{"osd":6,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.83899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.256,"5min":1.659,"15min":1.659},"last":1.446}]},{"osd":7,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4319999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.79900000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.131}]}]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.871},{"interface":"front","average":{"1min":0.879,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90500000000000003,"15min":0.92100000000000004},"min":{"1min":0.69399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1220000000000001,"5min":1.419,"15min":1.419},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.151,"5min":1.337,"15min":1.337},"last":0.77900000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0629999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.754,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1619999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.109,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3799999999999999,"5min":1.482,"15min":1.482},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0489999999999999},"min":{"1min":0.77600000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.135}]},{"osd":6,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.071,"15min":1.089},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.1599999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31892,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.141},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.88200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.997,"5min":1.758,"15min":1.758},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.216,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.042}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":0.99199999999999999},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.93100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1559999999999999,"5min":1.419,"15min":1.419},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.071,"15min":1.109},"min":{"1min":0.76000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3580000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.97099999999999997,"15min":0.999},"min":{"1min":0.68000000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.216,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.877}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4370000000000001,"5min":1.538,"15min":1.538},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.69499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.407,"5min":1.458,"15min":1.458},"last":0.97599999999999998}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.528,"5min":1.681,"15min":1.681},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1519999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3160000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.079}]},{"osd":7,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0549999999999999,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2509999999999999,"5min":1.835,"15min":1.835},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.218,"15min":1.248},"min":{"1min":0.87,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.595,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.2010000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31876,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.53400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.014,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.123,"5min":1.52,"15min":1.52},"last":1.145}]},{"osd":1,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81899999999999995,"15min":0.80100000000000005},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1559999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.266}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1759999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.57499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.86599999999999999},"min":{"1min":0.60099999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.0449999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97099999999999997,"15min":0.96399999999999997},"min":{"1min":0.82499999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.0580000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.63900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1739999999999999,"5min":1.304,"15min":1.304},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97899999999999998},"min":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.149,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.90600000000000003},"min":{"1min":0.71699999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.605,"15min":1.605},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.90100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.92400000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2330000000000001,"5min":1.538,"15min":1.538},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0529999999999999,"15min":1.0169999999999999},"min":{"1min":0.90300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.278,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3089999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.28,"15min":1.28},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.64800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.872,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1819999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.67000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.214,"5min":1.29,"15min":1.29},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.351,"5min":1.637,"15min":1.637},"last":0.75}]},{"osd":3,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.337,"5min":2.601,"15min":2.601},"last":0.877},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.073,"15min":1.119},"min":{"1min":0.81000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4259999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2929999999999999,"5min":1.792,"15min":1.792},"last":1.02},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.034},"min":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3839999999999999,"5min":1.575,"15min":1.575},"last":1.046}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0800000000000001,"15min":1.1000000000000001},"min":{"1min":0.71299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2849999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":1.012,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.347,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.099,"15min":1.1140000000000001},"min":{"1min":0.83399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5169999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.23},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1759999999999999,"15min":1.1830000000000001},"min":{"1min":0.85799999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.405,"5min":1.944,"15min":1.944},"last":1.1659999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.875},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85299999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.93400000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.016}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.008,"15min":1.008},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59299999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90700000000000003},"min":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3029999999999999,"5min":1.363,"15min":1.363},"last":1.333},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0069999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.204,"5min":1.282,"15min":1.282},"last":1.1120000000000001}]},{"osd":5,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.347,"5min":1.484,"15min":1.484},"last":1.407},{"interface":"front","average":{"1min":1,"5min":1.0149999999999999,"15min":1.02},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2509999999999999,"5min":1.391,"15min":1.391},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.052,"15min":1.0609999999999999},"min":{"1min":0.80200000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.147},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3120000000000001,"5min":1.673,"15min":1.673},"last":1.4450000000000001}]},{"osd":7,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0700000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.534},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2849999999999999,"5min":1.409,"15min":1.409},"last":1.1879999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.012,"5min":1.617,"15min":1.617},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.446,"15min":1.446},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.032,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81399999999999995},"min":{"1min":0.55300000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":1.714,"15min":1.714},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.327,"15min":1.327},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.70699999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2110000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.22,"5min":1.669,"15min":1.669},"last":0.98699999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70099999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0289999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":1},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0609999999999999,"5min":1.474,"15min":1.474},"last":0.92600000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95299999999999996,"15min":0.96799999999999997},"min":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1579999999999999,"5min":1.397,"15min":1.397},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0349999999999999,"15min":1.0529999999999999},"min":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1759999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.96399999999999997}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.85699999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.2909999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.01,"15min":1.022},"min":{"1min":0.85999999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1359999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.355}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:36:28.140 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:36:28.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 Processor -- start 2024-05-08T22:36:28.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 -- start start 2024-05-08T22:36:28.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3801434d0 0x7ff38006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff38006dc20 0x7ff38006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3800ce2a0 0x7ff38006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3800cd200 con 0x7ff3800ce2a0 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3800cd4e0 con 0x7ff3801434d0 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff38754b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3800cd7c0 con 0x7ff38006dc20 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff386d4a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3800ce2a0 0x7ff38006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff386d4a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3800ce2a0 0x7ff38006eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55892/0 (socket says 172.21.15.179:55892) 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff386d4a700 1 -- 172.21.15.179:0/1585371493 learned_addr learned my addr 172.21.15.179:0/1585371493 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff386549700 1 --2- 172.21.15.179:0/1585371493 >> v2:172.21.15.195:3300/0 conn(0x7ff3801434d0 0x7ff38006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.251+0000 7ff385d48700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff38006dc20 0x7ff38006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 -- 172.21.15.179:0/1585371493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff38006dc20 msgr2=0x7ff38006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff38006dc20 0x7ff38006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 -- 172.21.15.179:0/1585371493 >> v2:172.21.15.195:3300/0 conn(0x7ff3801434d0 msgr2=0x7ff38006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 --2- 172.21.15.179:0/1585371493 >> v2:172.21.15.195:3300/0 conn(0x7ff3801434d0 0x7ff38006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 -- 172.21.15.179:0/1585371493 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800cdaa0 con 0x7ff3800ce2a0 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff386d4a700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff38006eb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff380070800 tx=0x7ff370009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6570b257fe0d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff385547700 1 -- 172.21.15.179:0/1585371493 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370011830 con 0x7ff3800ce2a0 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff385547700 1 -- 172.21.15.179:0/1585371493 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff370011990 con 0x7ff3800ce2a0 2024-05-08T22:36:28.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff385547700 1 -- 172.21.15.179:0/1585371493 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370011d20 con 0x7ff3800ce2a0 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- 172.21.15.179:0/1585371493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 msgr2=0x7ff38006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff38006eb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff380070800 tx=0x7ff370009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- 172.21.15.179:0/1585371493 shutdown_connections 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff38006eb80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 --2- 172.21.15.179:0/1585371493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff38006dc20 0x7ff38006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 --2- 172.21.15.179:0/1585371493 >> v2:172.21.15.195:3300/0 conn(0x7ff3801434d0 0x7ff38006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- 172.21.15.179:0/1585371493 >> 172.21.15.179:0/1585371493 conn(0x7ff3800ce9d0 msgr2=0x7ff3800cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- 172.21.15.179:0/1585371493 shutdown_connections 2024-05-08T22:36:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- 172.21.15.179:0/1585371493 wait complete. 2024-05-08T22:36:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 Processor -- start 2024-05-08T22:36:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.255+0000 7ff38754b700 1 -- start start 2024-05-08T22:36:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3801434d0 0x7ff380140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 0x7ff380141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38013c730 con 0x7ff3800ce2a0 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38013d740 con 0x7ff3801f0200 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38013c980 con 0x7ff3801434d0 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55898/0 (socket says 172.21.15.179:55898) 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 -- 172.21.15.179:0/405039800 learned_addr learned my addr 172.21.15.179:0/405039800 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff385d48700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3801434d0 0x7ff380140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386d4a700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 0x7ff380141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 -- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3801434d0 msgr2=0x7ff380140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3801434d0 0x7ff380140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 -- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 msgr2=0x7ff380141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 0x7ff380141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 -- 172.21.15.179:0/405039800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff370009570 con 0x7ff3800ce2a0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386d4a700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 0x7ff380141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff386549700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff37800efc0 tx=0x7ff37800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37800cb50 con 0x7ff3800ce2a0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37800f040 con 0x7ff3800ce2a0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38013cc60 con 0x7ff3800ce2a0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378014480 con 0x7ff3800ce2a0 2024-05-08T22:36:28.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.259+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38013d220 con 0x7ff3800ce2a0 2024-05-08T22:36:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff380067740 con 0x7ff3800ce2a0 2024-05-08T22:36:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3780145e0 con 0x7ff3800ce2a0 2024-05-08T22:36:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff376ffd700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 0x7ff36c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff378084cd0 con 0x7ff3800ce2a0 2024-05-08T22:36:28.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff385d48700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 0x7ff36c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.263+0000 7ff385d48700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 0x7ff36c068c20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff38013dd80 tx=0x7ff37c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.267+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff378051f10 con 0x7ff3800ce2a0 2024-05-08T22:36:28.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.435+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3801438b0 con 0x7ff36c066760 2024-05-08T22:36:28.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.447+0000 7ff376ffd700 1 -- 172.21.15.179:0/405039800 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274273 (secure 0 0 0) 0x7ff37c0117e0 con 0x7ff36c066760 2024-05-08T22:36:28.454 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:28.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 msgr2=0x7ff36c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 0x7ff36c068c20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff38013dd80 tx=0x7ff37c009240 comp rx=0 tx=0).stop 2024-05-08T22:36:28.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 msgr2=0x7ff3801efcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff37800efc0 tx=0x7ff37800c0c0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 shutdown_connections 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff36c066760 0x7ff36c068c20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3801f0200 0x7ff380141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3801434d0 0x7ff380140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 --2- 172.21.15.179:0/405039800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3800ce2a0 0x7ff3801efcc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 >> 172.21.15.179:0/405039800 conn(0x7ff3800ce9d0 msgr2=0x7ff38006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 shutdown_connections 2024-05-08T22:36:28.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.455+0000 7ff38754b700 1 -- 172.21.15.179:0/405039800 wait complete. 2024-05-08T22:36:28.458 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:36:28.469 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-08T22:36:27.444758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":243948,"kb_used_data":7252,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730772,"statfs":{"total":773094113280,"available":772844310528,"internally_reserved":0,"allocated":7426048,"data_stored":3947305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001101"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.698577+0000","last_change":"2024-05-08T22:36:23.698577+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.698577+0000","last_undegraded":"2024-05-08T22:36:23.698577+0000","last_fullsized":"2024-05-08T22:36:23.698577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699742+0000","last_change":"2024-05-08T22:36:23.699742+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699742+0000","last_undegraded":"2024-05-08T22:36:23.699742+0000","last_fullsized":"2024-05-08T22:36:23.699742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.700977+0000","last_change":"2024-05-08T22:36:23.700977+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.700977+0000","last_undegraded":"2024-05-08T22:36:23.700977+0000","last_fullsized":"2024-05-08T22:36:23.700977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701304+0000","last_change":"2024-05-08T22:36:23.701304+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701304+0000","last_undegraded":"2024-05-08T22:36:23.701304+0000","last_fullsized":"2024-05-08T22:36:23.701304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.697706+0000","last_change":"2024-05-08T22:36:23.697706+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.697706+0000","last_undegraded":"2024-05-08T22:36:23.697706+0000","last_fullsized":"2024-05-08T22:36:23.697706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701388+0000","last_change":"2024-05-08T22:36:23.701388+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701388+0000","last_undegraded":"2024-05-08T22:36:23.701388+0000","last_fullsized":"2024-05-08T22:36:23.701388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696746+0000","last_change":"2024-05-08T22:36:23.696746+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696746+0000","last_undegraded":"2024-05-08T22:36:23.696746+0000","last_fullsized":"2024-05-08T22:36:23.696746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699737+0000","last_change":"2024-05-08T22:36:23.699737+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699737+0000","last_undegraded":"2024-05-08T22:36:23.699737+0000","last_fullsized":"2024-05-08T22:36:23.699737+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696747+0000","last_change":"2024-05-08T22:36:23.696747+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696747+0000","last_undegraded":"2024-05-08T22:36:23.696747+0000","last_fullsized":"2024-05-08T22:36:23.696747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T22:36:20.613925+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:36:20.613925+0000","last_peered":"2024-05-08T22:36:20.613925+0000","last_clean":"2024-05-08T22:36:20.613925+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:36:20.613925+0000","last_undegraded":"2024-05-08T22:36:20.613925+0000","last_fullsized":"2024-05-08T22:36:20.613925+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63},"min":{"1min":0.44500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.012},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0529999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97099999999999997,"5min":1.101,"15min":1.101},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.51000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0089999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0649999999999999,"5min":1.21,"15min":1.21},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.92100000000000004},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2330000000000001,"5min":1.306,"15min":1.306},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":0.94099999999999995},"min":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.234,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.008,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2649999999999999,"5min":1.476,"15min":1.476},"last":1.097}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.68600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3069999999999999,"5min":1.446,"15min":1.446},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2809999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.095,"15min":1.107},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4510000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.167},{"interface":"front","average":{"1min":1,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.347,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.121}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":351191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.04,"15min":1.04},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.046,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.371,"15min":0.371},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.3280000000000001,"5min":1.431,"15min":1.431},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.96699999999999997},"min":{"1min":0.75,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.226,"5min":1.508,"15min":1.508},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.268,"5min":1.327,"15min":1.327},"last":1.042},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.093},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.268,"5min":1.726,"15min":1.726},"last":1.397}]},{"osd":6,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.83899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.256,"5min":1.659,"15min":1.659},"last":1.446}]},{"osd":7,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4319999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.79900000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.131}]}]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.871},{"interface":"front","average":{"1min":0.879,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90500000000000003,"15min":0.92100000000000004},"min":{"1min":0.69399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1220000000000001,"5min":1.419,"15min":1.419},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.151,"5min":1.337,"15min":1.337},"last":0.77900000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0629999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.754,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1619999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.109,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3799999999999999,"5min":1.482,"15min":1.482},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0489999999999999},"min":{"1min":0.77600000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.135}]},{"osd":6,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.071,"15min":1.089},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.1599999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31892,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.141},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.88200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.997,"5min":1.758,"15min":1.758},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.216,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.042}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":0.99199999999999999},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.93100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1559999999999999,"5min":1.419,"15min":1.419},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.071,"15min":1.109},"min":{"1min":0.76000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3580000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.97099999999999997,"15min":0.999},"min":{"1min":0.68000000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.216,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.877}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4370000000000001,"5min":1.538,"15min":1.538},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.69499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.407,"5min":1.458,"15min":1.458},"last":0.97599999999999998}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.528,"5min":1.681,"15min":1.681},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1519999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3160000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.079}]},{"osd":7,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0549999999999999,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2509999999999999,"5min":1.835,"15min":1.835},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.218,"15min":1.248},"min":{"1min":0.87,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.595,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.2010000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31876,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.53400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.014,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.123,"5min":1.52,"15min":1.52},"last":1.145}]},{"osd":1,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81899999999999995,"15min":0.80100000000000005},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1559999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.266}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1759999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.57499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.86599999999999999},"min":{"1min":0.60099999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.0449999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97099999999999997,"15min":0.96399999999999997},"min":{"1min":0.82499999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.0580000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.63900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1739999999999999,"5min":1.304,"15min":1.304},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97899999999999998},"min":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.149,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.90600000000000003},"min":{"1min":0.71699999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.605,"15min":1.605},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.90100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.92400000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2330000000000001,"5min":1.538,"15min":1.538},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0529999999999999,"15min":1.0169999999999999},"min":{"1min":0.90300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.278,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3089999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.28,"15min":1.28},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.64800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.872,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1819999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.67000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.214,"5min":1.29,"15min":1.29},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.351,"5min":1.637,"15min":1.637},"last":0.75}]},{"osd":3,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.337,"5min":2.601,"15min":2.601},"last":0.877},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.073,"15min":1.119},"min":{"1min":0.81000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4259999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2929999999999999,"5min":1.792,"15min":1.792},"last":1.02},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.034},"min":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3839999999999999,"5min":1.575,"15min":1.575},"last":1.046}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0800000000000001,"15min":1.1000000000000001},"min":{"1min":0.71299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2849999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":1.012,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.347,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.099,"15min":1.1140000000000001},"min":{"1min":0.83399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5169999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.23},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1759999999999999,"15min":1.1830000000000001},"min":{"1min":0.85799999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.405,"5min":1.944,"15min":1.944},"last":1.1659999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.875},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85299999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.93400000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.016}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.008,"15min":1.008},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59299999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90700000000000003},"min":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3029999999999999,"5min":1.363,"15min":1.363},"last":1.333},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0069999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.204,"5min":1.282,"15min":1.282},"last":1.1120000000000001}]},{"osd":5,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.347,"5min":1.484,"15min":1.484},"last":1.407},{"interface":"front","average":{"1min":1,"5min":1.0149999999999999,"15min":1.02},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2509999999999999,"5min":1.391,"15min":1.391},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.052,"15min":1.0609999999999999},"min":{"1min":0.80200000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.147},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3120000000000001,"5min":1.673,"15min":1.673},"last":1.4450000000000001}]},{"osd":7,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0700000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.534},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2849999999999999,"5min":1.409,"15min":1.409},"last":1.1879999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.012,"5min":1.617,"15min":1.617},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.446,"15min":1.446},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.032,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81399999999999995},"min":{"1min":0.55300000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":1.714,"15min":1.714},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.327,"15min":1.327},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.70699999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2110000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.22,"5min":1.669,"15min":1.669},"last":0.98699999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70099999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0289999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":1},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0609999999999999,"5min":1.474,"15min":1.474},"last":0.92600000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95299999999999996,"15min":0.96799999999999997},"min":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1579999999999999,"5min":1.397,"15min":1.397},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0349999999999999,"15min":1.0529999999999999},"min":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1759999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.96399999999999997}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.85699999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.2909999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.01,"15min":1.022},"min":{"1min":0.85999999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1359999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.355}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:36:28.474 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:36:28.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 Processor -- start 2024-05-08T22:36:28.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- start start 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d10158b80 0x7f3d1015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d10158f60 con 0x7f3d100c8d70 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1006b860 con 0x7f3d100c9230 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1006ac50 con 0x7f3d10158b80 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d151a9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d10158b80 0x7f3d1015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d149a8700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3d100c8d70 0x7f3d1006bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54740/0 (socket says 172.21.15.179:54740) 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d149a8700 1 -- 172.21.15.179:0/1148128299 learned_addr learned my addr 172.21.15.179:0/1148128299 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d151a9700 1 -- 172.21.15.179:0/1148128299 >> v1:172.21.15.195:6789/0 conn(0x7f3d100c9230 legacy=0x7f3d1006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d151a9700 1 -- 172.21.15.179:0/1148128299 >> v1:172.21.15.179:6789/0 conn(0x7f3d100c8d70 legacy=0x7f3d1006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d151a9700 1 -- 172.21.15.179:0/1148128299 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1006af30 con 0x7f3d10158b80 2024-05-08T22:36:28.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d151a9700 1 --2- 172.21.15.179:0/1148128299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d10158b80 0x7f3d1015b4d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009850 tx=0x7f3d00009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5e1e5d5ac05a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d07fff700 1 -- 172.21.15.179:0/1148128299 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00011830 con 0x7f3d10158b80 2024-05-08T22:36:28.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d07fff700 1 -- 172.21.15.179:0/1148128299 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d00011990 con 0x7f3d10158b80 2024-05-08T22:36:28.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d07fff700 1 -- 172.21.15.179:0/1148128299 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00011d20 con 0x7f3d10158b80 2024-05-08T22:36:28.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- 172.21.15.179:0/1148128299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d10158b80 msgr2=0x7f3d1015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1148128299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d10158b80 0x7f3d1015b4d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009850 tx=0x7f3d00009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:28.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- 172.21.15.179:0/1148128299 shutdown_connections 2024-05-08T22:36:28.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1148128299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d10158b80 0x7f3d1015b4d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- 172.21.15.179:0/1148128299 >> 172.21.15.179:0/1148128299 conn(0x7f3d100cb660 msgr2=0x7f3d100cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- 172.21.15.179:0/1148128299 shutdown_connections 2024-05-08T22:36:28.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.587+0000 7f3d1740d700 1 -- 172.21.15.179:0/1148128299 wait complete. 2024-05-08T22:36:28.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 Processor -- start 2024-05-08T22:36:28.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 -- start start 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d100c9230 0x7f3d101558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 0x7f3d100c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100c41d0 con 0x7f3d100c8d70 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100c35c0 con 0x7f3d100c9230 2024-05-08T22:36:28.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d1740d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100c38a0 con 0x7f3d10158b80 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55920/0 (socket says 172.21.15.179:55920) 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 -- 172.21.15.179:0/1618125740 learned_addr learned my addr 172.21.15.179:0/1618125740 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d149a8700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d100c9230 0x7f3d101558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d159aa700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 0x7f3d100c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 -- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 msgr2=0x7f3d100c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 0x7f3d100c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 -- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d100c9230 msgr2=0x7f3d101558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d100c9230 0x7f3d101558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 -- 172.21.15.179:0/1618125740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000094b0 con 0x7f3d100c8d70 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d159aa700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 0x7f3d100c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:28.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.591+0000 7f3d151a9700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d00004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d000038c0 con 0x7f3d100c8d70 2024-05-08T22:36:28.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0001e070 con 0x7f3d100c8d70 2024-05-08T22:36:28.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d100c3b80 con 0x7f3d100c8d70 2024-05-08T22:36:28.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00003c80 con 0x7f3d100c8d70 2024-05-08T22:36:28.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d10154c90 con 0x7f3d100c8d70 2024-05-08T22:36:28.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d10067770 con 0x7f3d100c8d70 2024-05-08T22:36:28.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d0001a420 con 0x7f3d100c8d70 2024-05-08T22:36:28.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 0x7f3cf4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.595+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3d00020070 con 0x7f3d100c8d70 2024-05-08T22:36:28.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.599+0000 7f3d149a8700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 0x7f3cf4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.599+0000 7f3d149a8700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 0x7f3cf4068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d100b0170 tx=0x7f3d08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.607+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3d00052c50 con 0x7f3d100c8d70 2024-05-08T22:36:28.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.775+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d1006c400 con 0x7f3cf4066730 2024-05-08T22:36:28.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.791+0000 7f3d05ffb700 1 -- 172.21.15.179:0/1618125740 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274273 (secure 0 0 0) 0x7f3d0800e6d0 con 0x7f3cf4066730 2024-05-08T22:36:28.796 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 msgr2=0x7f3cf4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 0x7f3cf4068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d100b0170 tx=0x7f3d08009040 comp rx=0 tx=0).stop 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 msgr2=0x7f3d10154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d00004770 comp rx=0 tx=0).stop 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 shutdown_connections 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3cf4066730 0x7f3cf4068bf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d100c9230 0x7f3d101558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d10158b80 0x7f3d100c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 --2- 172.21.15.179:0/1618125740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d100c8d70 0x7f3d10154750 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 >> 172.21.15.179:0/1618125740 conn(0x7f3d100cb660 msgr2=0x7f3d1015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 shutdown_connections 2024-05-08T22:36:28.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.795+0000 7f3d1740d700 1 -- 172.21.15.179:0/1618125740 wait complete. 2024-05-08T22:36:28.800 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:36:28.811 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-08T22:36:27.444758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":243948,"kb_used_data":7252,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730772,"statfs":{"total":773094113280,"available":772844310528,"internally_reserved":0,"allocated":7426048,"data_stored":3947305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001101"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.698577+0000","last_change":"2024-05-08T22:36:23.698577+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.698577+0000","last_undegraded":"2024-05-08T22:36:23.698577+0000","last_fullsized":"2024-05-08T22:36:23.698577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699742+0000","last_change":"2024-05-08T22:36:23.699742+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699742+0000","last_undegraded":"2024-05-08T22:36:23.699742+0000","last_fullsized":"2024-05-08T22:36:23.699742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.700977+0000","last_change":"2024-05-08T22:36:23.700977+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.700977+0000","last_undegraded":"2024-05-08T22:36:23.700977+0000","last_fullsized":"2024-05-08T22:36:23.700977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701304+0000","last_change":"2024-05-08T22:36:23.701304+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701304+0000","last_undegraded":"2024-05-08T22:36:23.701304+0000","last_fullsized":"2024-05-08T22:36:23.701304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.697706+0000","last_change":"2024-05-08T22:36:23.697706+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.697706+0000","last_undegraded":"2024-05-08T22:36:23.697706+0000","last_fullsized":"2024-05-08T22:36:23.697706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701388+0000","last_change":"2024-05-08T22:36:23.701388+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701388+0000","last_undegraded":"2024-05-08T22:36:23.701388+0000","last_fullsized":"2024-05-08T22:36:23.701388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696746+0000","last_change":"2024-05-08T22:36:23.696746+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696746+0000","last_undegraded":"2024-05-08T22:36:23.696746+0000","last_fullsized":"2024-05-08T22:36:23.696746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:23.676082+0000","last_change":"2024-05-08T22:36:23.676082+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.676082+0000","last_undegraded":"2024-05-08T22:36:23.676082+0000","last_fullsized":"2024-05-08T22:36:23.676082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699737+0000","last_change":"2024-05-08T22:36:23.699737+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699737+0000","last_undegraded":"2024-05-08T22:36:23.699737+0000","last_fullsized":"2024-05-08T22:36:23.699737+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696747+0000","last_change":"2024-05-08T22:36:23.696747+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696747+0000","last_undegraded":"2024-05-08T22:36:23.696747+0000","last_fullsized":"2024-05-08T22:36:23.696747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T22:36:20.613925+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:36:20.613925+0000","last_peered":"2024-05-08T22:36:20.613925+0000","last_clean":"2024-05-08T22:36:20.613925+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:36:20.613925+0000","last_undegraded":"2024-05-08T22:36:20.613925+0000","last_fullsized":"2024-05-08T22:36:20.613925+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63},"min":{"1min":0.44500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.012},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0529999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97099999999999997,"5min":1.101,"15min":1.101},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.51000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0089999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0649999999999999,"5min":1.21,"15min":1.21},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.92100000000000004},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2330000000000001,"5min":1.306,"15min":1.306},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":0.94099999999999995},"min":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.234,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.008,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2649999999999999,"5min":1.476,"15min":1.476},"last":1.097}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.68600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3069999999999999,"5min":1.446,"15min":1.446},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2809999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.095,"15min":1.107},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4510000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.167},{"interface":"front","average":{"1min":1,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.347,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.121}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":351191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.04,"15min":1.04},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.046,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.371,"15min":0.371},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.3280000000000001,"5min":1.431,"15min":1.431},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.96699999999999997},"min":{"1min":0.75,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.226,"5min":1.508,"15min":1.508},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.268,"5min":1.327,"15min":1.327},"last":1.042},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.093},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.268,"5min":1.726,"15min":1.726},"last":1.397}]},{"osd":6,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.83899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.256,"5min":1.659,"15min":1.659},"last":1.446}]},{"osd":7,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4319999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.79900000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.131}]}]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.871},{"interface":"front","average":{"1min":0.879,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90500000000000003,"15min":0.92100000000000004},"min":{"1min":0.69399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1220000000000001,"5min":1.419,"15min":1.419},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.151,"5min":1.337,"15min":1.337},"last":0.77900000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0629999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.754,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1619999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.109,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3799999999999999,"5min":1.482,"15min":1.482},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0489999999999999},"min":{"1min":0.77600000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.135}]},{"osd":6,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.071,"15min":1.089},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.1599999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31892,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.141},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.88200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.997,"5min":1.758,"15min":1.758},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.216,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.042}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":0.99199999999999999},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.93100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1559999999999999,"5min":1.419,"15min":1.419},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.071,"15min":1.109},"min":{"1min":0.76000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3580000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.97099999999999997,"15min":0.999},"min":{"1min":0.68000000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.216,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.877}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4370000000000001,"5min":1.538,"15min":1.538},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.69499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.407,"5min":1.458,"15min":1.458},"last":0.97599999999999998}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.528,"5min":1.681,"15min":1.681},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1519999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3160000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.079}]},{"osd":7,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0549999999999999,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2509999999999999,"5min":1.835,"15min":1.835},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.218,"15min":1.248},"min":{"1min":0.87,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.595,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.2010000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31876,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.53400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.014,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.123,"5min":1.52,"15min":1.52},"last":1.145}]},{"osd":1,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81899999999999995,"15min":0.80100000000000005},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1559999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.266}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1759999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.57499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.86599999999999999},"min":{"1min":0.60099999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.0449999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97099999999999997,"15min":0.96399999999999997},"min":{"1min":0.82499999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.0580000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.63900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1739999999999999,"5min":1.304,"15min":1.304},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97899999999999998},"min":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.149,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.90600000000000003},"min":{"1min":0.71699999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.605,"15min":1.605},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.90100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.92400000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2330000000000001,"5min":1.538,"15min":1.538},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0529999999999999,"15min":1.0169999999999999},"min":{"1min":0.90300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.278,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3089999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.28,"15min":1.28},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.64800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.872,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1819999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.67000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.214,"5min":1.29,"15min":1.29},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.351,"5min":1.637,"15min":1.637},"last":0.75}]},{"osd":3,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.337,"5min":2.601,"15min":2.601},"last":0.877},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.073,"15min":1.119},"min":{"1min":0.81000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4259999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2929999999999999,"5min":1.792,"15min":1.792},"last":1.02},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.034},"min":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3839999999999999,"5min":1.575,"15min":1.575},"last":1.046}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0800000000000001,"15min":1.1000000000000001},"min":{"1min":0.71299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2849999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":1.012,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.347,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.099,"15min":1.1140000000000001},"min":{"1min":0.83399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5169999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.23},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1759999999999999,"15min":1.1830000000000001},"min":{"1min":0.85799999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.405,"5min":1.944,"15min":1.944},"last":1.1659999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.875},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85299999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.93400000000000005}]},{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.016}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.008,"15min":1.008},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59299999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90700000000000003},"min":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3029999999999999,"5min":1.363,"15min":1.363},"last":1.333},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0069999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.204,"5min":1.282,"15min":1.282},"last":1.1120000000000001}]},{"osd":5,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.347,"5min":1.484,"15min":1.484},"last":1.407},{"interface":"front","average":{"1min":1,"5min":1.0149999999999999,"15min":1.02},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2509999999999999,"5min":1.391,"15min":1.391},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.052,"15min":1.0609999999999999},"min":{"1min":0.80200000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.147},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3120000000000001,"5min":1.673,"15min":1.673},"last":1.4450000000000001}]},{"osd":7,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0700000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.534},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2849999999999999,"5min":1.409,"15min":1.409},"last":1.1879999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":935422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.012,"5min":1.617,"15min":1.617},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.446,"15min":1.446},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.032,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81399999999999995},"min":{"1min":0.55300000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":1.714,"15min":1.714},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.327,"15min":1.327},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.70699999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2110000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.22,"5min":1.669,"15min":1.669},"last":0.98699999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70099999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0289999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":1},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0609999999999999,"5min":1.474,"15min":1.474},"last":0.92600000000000005}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95299999999999996,"15min":0.96799999999999997},"min":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1579999999999999,"5min":1.397,"15min":1.397},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0349999999999999,"15min":1.0529999999999999},"min":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1759999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.96399999999999997}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.85699999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.2909999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.01,"15min":1.022},"min":{"1min":0.85999999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1359999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.355}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:36:28.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-05-08T22:36:28.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-05-08T22:36:28.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-05-08T22:36:28.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-05-08T22:36:28.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-05-08T22:36:28.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-05-08T22:36:28.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-05-08T22:36:28.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-05-08T22:36:28.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.698577+0000', 'last_change': '2024-05-08T22:36:23.698577+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.698577+0000', 'last_undegraded': '2024-05-08T22:36:23.698577+0000', 'last_fullsized': '2024-05-08T22:36:23.698577+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-05-08T22:36:28.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.699742+0000', 'last_change': '2024-05-08T22:36:23.699742+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.699742+0000', 'last_undegraded': '2024-05-08T22:36:23.699742+0000', 'last_fullsized': '2024-05-08T22:36:23.699742+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.700977+0000', 'last_change': '2024-05-08T22:36:23.700977+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.700977+0000', 'last_undegraded': '2024-05-08T22:36:23.700977+0000', 'last_fullsized': '2024-05-08T22:36:23.700977+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-05-08T22:36:28.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.701304+0000', 'last_change': '2024-05-08T22:36:23.701304+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.701304+0000', 'last_undegraded': '2024-05-08T22:36:23.701304+0000', 'last_fullsized': '2024-05-08T22:36:23.701304+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-05-08T22:36:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.697706+0000', 'last_change': '2024-05-08T22:36:23.697706+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.697706+0000', 'last_undegraded': '2024-05-08T22:36:23.697706+0000', 'last_fullsized': '2024-05-08T22:36:23.697706+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-05-08T22:36:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-05-08T22:36:28.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-05-08T22:36:28.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-05-08T22:36:28.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-05-08T22:36:28.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.701388+0000', 'last_change': '2024-05-08T22:36:23.701388+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.701388+0000', 'last_undegraded': '2024-05-08T22:36:23.701388+0000', 'last_fullsized': '2024-05-08T22:36:23.701388+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-05-08T22:36:28.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-05-08T22:36:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-05-08T22:36:28.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.696746+0000', 'last_change': '2024-05-08T22:36:23.696746+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.696746+0000', 'last_undegraded': '2024-05-08T22:36:23.696746+0000', 'last_fullsized': '2024-05-08T22:36:23.696746+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:23.676082+0000', 'last_change': '2024-05-08T22:36:23.676082+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.676082+0000', 'last_undegraded': '2024-05-08T22:36:23.676082+0000', 'last_fullsized': '2024-05-08T22:36:23.676082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.699737+0000', 'last_change': '2024-05-08T22:36:23.699737+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.699737+0000', 'last_undegraded': '2024-05-08T22:36:23.699737+0000', 'last_fullsized': '2024-05-08T22:36:23.699737+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.699763+0000', 'last_change': '2024-05-08T22:36:23.699763+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.699763+0000', 'last_undegraded': '2024-05-08T22:36:23.699763+0000', 'last_fullsized': '2024-05-08T22:36:23.699763+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-05-08T22:36:28.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.696747+0000', 'last_change': '2024-05-08T22:36:23.696747+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.696747+0000', 'last_undegraded': '2024-05-08T22:36:23.696747+0000', 'last_fullsized': '2024-05-08T22:36:23.696747+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T22:36:28.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-05-08T22:36:28.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T22:36:24.685710+0000', 'last_change': '2024-05-08T22:36:24.685710+0000', 'last_active': '2024-05-08T22:36:24.685710+0000', 'last_peered': '2024-05-08T22:36:24.685710+0000', 'last_clean': '2024-05-08T22:36:24.685710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:24.685710+0000', 'last_undegraded': '2024-05-08T22:36:24.685710+0000', 'last_fullsized': '2024-05-08T22:36:24.685710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:24.685710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T22:36:28.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-05-08T22:36:28.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:36:23.699763+0000', 'last_change': '2024-05-08T22:36:23.699763+0000', 'last_active': '2024-05-08T22:36:23.676082+0000', 'last_peered': '2024-05-08T22:36:23.676082+0000', 'last_clean': '2024-05-08T22:36:23.676082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:36:23.699763+0000', 'last_undegraded': '2024-05-08T22:36:23.699763+0000', 'last_fullsized': '2024-05-08T22:36:23.699763+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'last_clean_scrub_stamp': '2024-05-08T22:36:23.676082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T22:36:28.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T22:36:28.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 Processor -- start 2024-05-08T22:36:28.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 -- start start 2024-05-08T22:36:28.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b00ca790 0x7f15b00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b015d9c0 con 0x7f15b00ca3b0 2024-05-08T22:36:28.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b015dca0 con 0x7f15b0146260 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b67f4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b015df80 con 0x7f15b00ca790 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b57f2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f15b00ca3b0 0x7f15b00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54754/0 (socket says 172.21.15.179:54754) 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b57f2700 1 -- 172.21.15.179:0/2336403895 learned_addr learned my addr 172.21.15.179:0/2336403895 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b00ca790 0x7f15b00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641776180 0 0) 0x7f15b015d9c0 con 0x7f15b00ca3b0 2024-05-08T22:36:28.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f159c0035d0 con 0x7f15b00ca3b0 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187450804 0 0) 0x7f15b015dca0 con 0x7f15b0146260 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b015d9c0 con 0x7f15b0146260 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324335550 0 0) 0x7f159c0035d0 con 0x7f15b00ca3b0 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b00ca790 msgr2=0x7f15b00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b00ca790 0x7f15b00ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 >> v1:172.21.15.195:6789/0 conn(0x7f15b0146260 legacy=0x7f15b01466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b015e260 con 0x7f15b00ca3b0 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f15ac001670 con 0x7f15b00ca3b0 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 >> v1:172.21.15.179:6789/0 conn(0x7f15b00ca3b0 legacy=0x7f15b00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159c003be0 0x7f159c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c004870 0x7f159c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f159c005230 0x7f159c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b015df80 con 0x7f159c003be0 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c00c050 con 0x7f159c004870 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c00c210 con 0x7f159c005230 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b57f2700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159c003be0 0x7f159c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.951+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b00ca790 0x7f15b00ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 -- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f159c005230 msgr2=0x7f159c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b5ff3700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f159c005230 0x7f159c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f159c005230 0x7f159c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c004870 0x7f159c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 -- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c004870 msgr2=0x7f159c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c004870 0x7f159c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 -- 172.21.15.179:0/2336403895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0035d0 con 0x7f159c003be0 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b5ff3700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f159c005230 0x7f159c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c004870 0x7f159c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b57f2700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159c003be0 0x7f159c003fe0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c160 tx=0x7f159c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b795e698c657bcbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00ccc0 con 0x7f159c003be0 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00ce20 con 0x7f159c003be0 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15a7fff700 1 -- 172.21.15.179:0/2336403895 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac005b80 con 0x7f159c003be0 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 -- 172.21.15.179:0/2336403895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159c003be0 msgr2=0x7f159c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 --2- 172.21.15.179:0/2336403895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159c003be0 0x7f159c003fe0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c160 tx=0x7f159c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b5ff3700 1 -- 172.21.15.179:0/2336403895 reap_dead start 2024-05-08T22:36:28.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 -- 172.21.15.179:0/2336403895 shutdown_connections 2024-05-08T22:36:28.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 -- 172.21.15.179:0/2336403895 >> 172.21.15.179:0/2336403895 conn(0x7f15b00b17c0 msgr2=0x7f15b00b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:28.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 -- 172.21.15.179:0/2336403895 shutdown_connections 2024-05-08T22:36:28.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.955+0000 7f15b67f4700 1 -- 172.21.15.179:0/2336403895 wait complete. 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 Processor -- start 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- start start 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 0x7f15b015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b01447e0 0x7f15b013b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0142080 con 0x7f15b0142510 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0141470 con 0x7f159c003be0 2024-05-08T22:36:28.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0141750 con 0x7f15b01447e0 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55930/0 (socket says 172.21.15.179:55930) 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 -- 172.21.15.179:0/1223997663 learned_addr learned my addr 172.21.15.179:0/1223997663 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b57f2700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 0x7f15b015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b5ff3700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b01447e0 0x7f15b013b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 -- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b01447e0 msgr2=0x7f15b013b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b01447e0 0x7f15b013b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 -- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 msgr2=0x7f15b015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 0x7f15b015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:28.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 -- 172.21.15.179:0/1223997663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0141a30 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b57f2700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 0x7f15b015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b4ff1700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f15a0010b00 tx=0x7f15a0010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000ea70 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000ebd0 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b0141d10 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0009e50 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.959+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b0142990 con 0x7f15b0142510 2024-05-08T22:36:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b004e700 con 0x7f15b0142510 2024-05-08T22:36:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15a001c400 con 0x7f15b0142510 2024-05-08T22:36:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15a5ffb700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 0x7f1594068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15a0084630 con 0x7f15b0142510 2024-05-08T22:36:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15b57f2700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 0x7f1594068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.963+0000 7f15b57f2700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 0x7f1594068c20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f15ac000c00 tx=0x7f15ac007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:28.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:28.971+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f15a0051910 con 0x7f15b0142510 2024-05-08T22:36:29.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f15b00cdf00 con 0x7f15b0142510 2024-05-08T22:36:29.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15a5ffb700 1 -- 172.21.15.179:0/1223997663 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1657 (secure 0 0 0) 0x7f15a0054f30 con 0x7f15b0142510 2024-05-08T22:36:29.203 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:29.204 INFO:teuthology.orchestra.run.smithi179.stdout:{"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1715207588,"num_in_osds":8,"osd_in_since":1715207580,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":85},{"state_name":"creating+peering","count":11},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":249802752,"bytes_avail":772844310528,"bytes_total":773094113280,"unknown_pgs_ratio":0.87628865242004395,"inactive_pgs_ratio":0.11340206116437912},"fsmap":{"epoch":30,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":55718}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-05-08T22:35:43.430090+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 msgr2=0x7f1594068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 0x7f1594068c20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f15ac000c00 tx=0x7f15ac007000 comp rx=0 tx=0).stop 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 msgr2=0x7f15b0140b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.203+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f15a0010b00 tx=0x7f15a0010e10 comp rx=0 tx=0).stop 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 shutdown_connections 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1594066760 0x7f1594068c20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15b01447e0 0x7f15b013b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15b0142510 0x7f15b0140b60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 --2- 172.21.15.179:0/1223997663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f159c003be0 0x7f15b015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 >> 172.21.15.179:0/1223997663 conn(0x7f15b00b17c0 msgr2=0x7f15b00cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:29.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 shutdown_connections 2024-05-08T22:36:29.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.207+0000 7f15b67f4700 1 -- 172.21.15.179:0/1223997663 wait complete. 2024-05-08T22:36:29.218 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T22:36:29.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:36:29.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 Processor -- start 2024-05-08T22:36:29.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 -- start start 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 0x7f897c0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f897c0cbdb0 0x7f897c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0cc380 con 0x7f897c0d0450 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0b13a0 con 0x7f897c0cbdb0 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.331+0000 7f8981614700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b1680 con 0x7f897c15b070 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f897c0cbdb0 0x7f897c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f897c0cbdb0 0x7f897c0b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55942/0 (socket says 172.21.15.179:55942) 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 -- 172.21.15.179:0/1921781868 learned_addr learned my addr 172.21.15.179:0/1921781868 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897affd700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 0x7f897c0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 -- 172.21.15.179:0/1921781868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 msgr2=0x7f897c0b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 0x7f897c0b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 -- 172.21.15.179:0/1921781868 >> v1:172.21.15.195:6789/0 conn(0x7f897c0d0450 legacy=0x7f897c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 -- 172.21.15.179:0/1921781868 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0b1960 con 0x7f897c0cbdb0 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897affd700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 0x7f897c0b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897b7fe700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0cbdb0 0x7f897c0b3ec0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8974009e50 tx=0x7f897400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbba0371938b74c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897a7fc700 1 -- 172.21.15.179:0/1921781868 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400e070 con 0x7f897c0cbdb0 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897a7fc700 1 -- 172.21.15.179:0/1921781868 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400b9c0 con 0x7f897c0cbdb0 2024-05-08T22:36:29.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f897a7fc700 1 -- 172.21.15.179:0/1921781868 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400bd70 con 0x7f897c0cbdb0 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 -- 172.21.15.179:0/1921781868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0cbdb0 msgr2=0x7f897c0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0cbdb0 0x7f897c0b3ec0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8974009e50 tx=0x7f897400b330 comp rx=0 tx=0).stop 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 -- 172.21.15.179:0/1921781868 shutdown_connections 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0cbdb0 0x7f897c0b3ec0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 --2- 172.21.15.179:0/1921781868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c15b070 0x7f897c0b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 -- 172.21.15.179:0/1921781868 >> 172.21.15.179:0/1921781868 conn(0x7f897c0d0dd0 msgr2=0x7f897c0d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:29.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 -- 172.21.15.179:0/1921781868 shutdown_connections 2024-05-08T22:36:29.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.335+0000 7f8981614700 1 -- 172.21.15.179:0/1921781868 wait complete. 2024-05-08T22:36:29.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 Processor -- start 2024-05-08T22:36:29.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- start start 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 0x7f897c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f897c15b070 0x7f897c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c14e570 con 0x7f897c0d0450 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c14d960 con 0x7f897c15b070 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c14dc40 con 0x7f897c0cbdb0 2024-05-08T22:36:29.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f897affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 0x7f897c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55948/0 (socket says 172.21.15.179:55948) 2024-05-08T22:36:29.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f897b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f897c15b070 0x7f897c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 -- 172.21.15.179:0/2726178021 learned_addr learned my addr 172.21.15.179:0/2726178021 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 -- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 msgr2=0x7f897c0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 0x7f897c0c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 -- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f897c15b070 msgr2=0x7f897c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f897c15b070 0x7f897c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 -- 172.21.15.179:0/2726178021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974009b30 con 0x7f897c0d0450 2024-05-08T22:36:29.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f897affd700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 0x7f897c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8973fff700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f896400c500 tx=0x7f896400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640052d0 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964005430 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c14df20 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640055a0 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.339+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c0c6590 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c073290 con 0x7f897c0d0450 2024-05-08T22:36:29.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89640038e0 con 0x7f897c0d0450 2024-05-08T22:36:29.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f89737fe700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 0x7f895c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:29.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89640836d0 con 0x7f897c0d0450 2024-05-08T22:36:29.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f897affd700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 0x7f895c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:29.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.343+0000 7f897affd700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 0x7f895c068bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f896c00af30 tx=0x7f896c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:29.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.351+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8964050870 con 0x7f897c0d0450 2024-05-08T22:36:29.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.519+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f897c0b1f20 con 0x7f895c066730 2024-05-08T22:36:29.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.535+0000 7f89737fe700 1 -- 172.21.15.179:0/2726178021 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285374 (secure 0 0 0) 0x7f896c00c040 con 0x7f895c066730 2024-05-08T22:36:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 msgr2=0x7f895c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 0x7f895c068bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f896c00af30 tx=0x7f896c008040 comp rx=0 tx=0).stop 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 msgr2=0x7f897c14d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f896400c500 tx=0x7f896400c810 comp rx=0 tx=0).stop 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 shutdown_connections 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f895c066730 0x7f895c068bf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f897c15b070 0x7f897c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f897c0d0450 0x7f897c14d030 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 --2- 172.21.15.179:0/2726178021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897c0cbdb0 0x7f897c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:29.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 >> 172.21.15.179:0/2726178021 conn(0x7f897c0d0dd0 msgr2=0x7f897c0ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:29.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 shutdown_connections 2024-05-08T22:36:29.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:29.539+0000 7f8981614700 1 -- 172.21.15.179:0/2726178021 wait complete. 2024-05-08T22:36:29.542 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:36:29.554 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-08T22:36:29.445359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248336,"kb_used_data":7576,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726384,"statfs":{"total":773094113280,"available":772839817216,"internally_reserved":0,"allocated":7757824,"data_stored":4108335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759452"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092560+0000","last_change":"2024-05-08T22:36:25.108945+0000","last_active":"2024-05-08T22:36:26.092560+0000","last_peered":"2024-05-08T22:36:26.092560+0000","last_clean":"2024-05-08T22:36:26.092560+0000","last_became_active":"2024-05-08T22:36:25.108540+0000","last_became_peered":"2024-05-08T22:36:25.108540+0000","last_unstale":"2024-05-08T22:36:26.092560+0000","last_undegraded":"2024-05-08T22:36:26.092560+0000","last_fullsized":"2024-05-08T22:36:26.092560+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.308266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094558+0000","last_change":"2024-05-08T22:36:25.104780+0000","last_active":"2024-05-08T22:36:26.094558+0000","last_peered":"2024-05-08T22:36:26.094558+0000","last_clean":"2024-05-08T22:36:26.094558+0000","last_became_active":"2024-05-08T22:36:25.101532+0000","last_became_peered":"2024-05-08T22:36:25.101532+0000","last_unstale":"2024-05-08T22:36:26.094558+0000","last_undegraded":"2024-05-08T22:36:26.094558+0000","last_fullsized":"2024-05-08T22:36:26.094558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:50.597974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092786+0000","last_change":"2024-05-08T22:36:25.120411+0000","last_active":"2024-05-08T22:36:26.092786+0000","last_peered":"2024-05-08T22:36:26.092786+0000","last_clean":"2024-05-08T22:36:26.092786+0000","last_became_active":"2024-05-08T22:36:25.118888+0000","last_became_peered":"2024-05-08T22:36:25.118888+0000","last_unstale":"2024-05-08T22:36:26.092786+0000","last_undegraded":"2024-05-08T22:36:26.092786+0000","last_fullsized":"2024-05-08T22:36:26.092786+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:03.368207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270900+0000","last_change":"2024-05-08T22:36:25.102907+0000","last_active":"2024-05-08T22:36:26.270900+0000","last_peered":"2024-05-08T22:36:26.270900+0000","last_clean":"2024-05-08T22:36:26.270900+0000","last_became_active":"2024-05-08T22:36:25.102653+0000","last_became_peered":"2024-05-08T22:36:25.102653+0000","last_unstale":"2024-05-08T22:36:26.270900+0000","last_undegraded":"2024-05-08T22:36:26.270900+0000","last_fullsized":"2024-05-08T22:36:26.270900+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:54.091244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270795+0000","last_change":"2024-05-08T22:36:25.111088+0000","last_active":"2024-05-08T22:36:26.270795+0000","last_peered":"2024-05-08T22:36:26.270795+0000","last_clean":"2024-05-08T22:36:26.270795+0000","last_became_active":"2024-05-08T22:36:25.110831+0000","last_became_peered":"2024-05-08T22:36:25.110831+0000","last_unstale":"2024-05-08T22:36:26.270795+0000","last_undegraded":"2024-05-08T22:36:26.270795+0000","last_fullsized":"2024-05-08T22:36:26.270795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:26.098141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269436+0000","last_change":"2024-05-08T22:36:25.104669+0000","last_active":"2024-05-08T22:36:26.269436+0000","last_peered":"2024-05-08T22:36:26.269436+0000","last_clean":"2024-05-08T22:36:26.269436+0000","last_became_active":"2024-05-08T22:36:25.104172+0000","last_became_peered":"2024-05-08T22:36:25.104172+0000","last_unstale":"2024-05-08T22:36:26.269436+0000","last_undegraded":"2024-05-08T22:36:26.269436+0000","last_fullsized":"2024-05-08T22:36:26.269436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:29.315608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093251+0000","last_change":"2024-05-08T22:36:25.112373+0000","last_active":"2024-05-08T22:36:26.093251+0000","last_peered":"2024-05-08T22:36:26.093251+0000","last_clean":"2024-05-08T22:36:26.093251+0000","last_became_active":"2024-05-08T22:36:25.111430+0000","last_became_peered":"2024-05-08T22:36:25.111430+0000","last_unstale":"2024-05-08T22:36:26.093251+0000","last_undegraded":"2024-05-08T22:36:26.093251+0000","last_fullsized":"2024-05-08T22:36:26.093251+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:51.224795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092786+0000","last_change":"2024-05-08T22:36:25.109292+0000","last_active":"2024-05-08T22:36:26.092786+0000","last_peered":"2024-05-08T22:36:26.092786+0000","last_clean":"2024-05-08T22:36:26.092786+0000","last_became_active":"2024-05-08T22:36:25.108914+0000","last_became_peered":"2024-05-08T22:36:25.108914+0000","last_unstale":"2024-05-08T22:36:26.092786+0000","last_undegraded":"2024-05-08T22:36:26.092786+0000","last_fullsized":"2024-05-08T22:36:26.092786+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:58.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094235+0000","last_change":"2024-05-08T22:36:25.100287+0000","last_active":"2024-05-08T22:36:26.094235+0000","last_peered":"2024-05-08T22:36:26.094235+0000","last_clean":"2024-05-08T22:36:26.094235+0000","last_became_active":"2024-05-08T22:36:25.100010+0000","last_became_peered":"2024-05-08T22:36:25.100010+0000","last_unstale":"2024-05-08T22:36:26.094235+0000","last_undegraded":"2024-05-08T22:36:26.094235+0000","last_fullsized":"2024-05-08T22:36:26.094235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:27.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269889+0000","last_change":"2024-05-08T22:36:25.099776+0000","last_active":"2024-05-08T22:36:26.269889+0000","last_peered":"2024-05-08T22:36:26.269889+0000","last_clean":"2024-05-08T22:36:26.269889+0000","last_became_active":"2024-05-08T22:36:25.099500+0000","last_became_peered":"2024-05-08T22:36:25.099500+0000","last_unstale":"2024-05-08T22:36:26.269889+0000","last_undegraded":"2024-05-08T22:36:26.269889+0000","last_fullsized":"2024-05-08T22:36:26.269889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:40.400169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269863+0000","last_change":"2024-05-08T22:36:25.107160+0000","last_active":"2024-05-08T22:36:26.269863+0000","last_peered":"2024-05-08T22:36:26.269863+0000","last_clean":"2024-05-08T22:36:26.269863+0000","last_became_active":"2024-05-08T22:36:25.106917+0000","last_became_peered":"2024-05-08T22:36:25.106917+0000","last_unstale":"2024-05-08T22:36:26.269863+0000","last_undegraded":"2024-05-08T22:36:26.269863+0000","last_fullsized":"2024-05-08T22:36:26.269863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:06.090448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091507+0000","last_change":"2024-05-08T22:36:25.110725+0000","last_active":"2024-05-08T22:36:26.091507+0000","last_peered":"2024-05-08T22:36:26.091507+0000","last_clean":"2024-05-08T22:36:26.091507+0000","last_became_active":"2024-05-08T22:36:25.108936+0000","last_became_peered":"2024-05-08T22:36:25.108936+0000","last_unstale":"2024-05-08T22:36:26.091507+0000","last_undegraded":"2024-05-08T22:36:26.091507+0000","last_fullsized":"2024-05-08T22:36:26.091507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091065+0000","last_change":"2024-05-08T22:36:25.108772+0000","last_active":"2024-05-08T22:36:26.091065+0000","last_peered":"2024-05-08T22:36:26.091065+0000","last_clean":"2024-05-08T22:36:26.091065+0000","last_became_active":"2024-05-08T22:36:25.108475+0000","last_became_peered":"2024-05-08T22:36:25.108475+0000","last_unstale":"2024-05-08T22:36:26.091065+0000","last_undegraded":"2024-05-08T22:36:26.091065+0000","last_fullsized":"2024-05-08T22:36:26.091065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:42.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272404+0000","last_change":"2024-05-08T22:36:25.100558+0000","last_active":"2024-05-08T22:36:26.272404+0000","last_peered":"2024-05-08T22:36:26.272404+0000","last_clean":"2024-05-08T22:36:26.272404+0000","last_became_active":"2024-05-08T22:36:25.100039+0000","last_became_peered":"2024-05-08T22:36:25.100039+0000","last_unstale":"2024-05-08T22:36:26.272404+0000","last_undegraded":"2024-05-08T22:36:26.272404+0000","last_fullsized":"2024-05-08T22:36:26.272404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:38.134712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094371+0000","last_change":"2024-05-08T22:36:25.108952+0000","last_active":"2024-05-08T22:36:26.094371+0000","last_peered":"2024-05-08T22:36:26.094371+0000","last_clean":"2024-05-08T22:36:26.094371+0000","last_became_active":"2024-05-08T22:36:25.108487+0000","last_became_peered":"2024-05-08T22:36:25.108487+0000","last_unstale":"2024-05-08T22:36:26.094371+0000","last_undegraded":"2024-05-08T22:36:26.094371+0000","last_fullsized":"2024-05-08T22:36:26.094371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091953+0000","last_change":"2024-05-08T22:36:25.117445+0000","last_active":"2024-05-08T22:36:26.091953+0000","last_peered":"2024-05-08T22:36:26.091953+0000","last_clean":"2024-05-08T22:36:26.091953+0000","last_became_active":"2024-05-08T22:36:25.116915+0000","last_became_peered":"2024-05-08T22:36:25.116915+0000","last_unstale":"2024-05-08T22:36:26.091953+0000","last_undegraded":"2024-05-08T22:36:26.091953+0000","last_fullsized":"2024-05-08T22:36:26.091953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:12.701134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269926+0000","last_change":"2024-05-08T22:36:25.101874+0000","last_active":"2024-05-08T22:36:26.269926+0000","last_peered":"2024-05-08T22:36:26.269926+0000","last_clean":"2024-05-08T22:36:26.269926+0000","last_became_active":"2024-05-08T22:36:25.101625+0000","last_became_peered":"2024-05-08T22:36:25.101625+0000","last_unstale":"2024-05-08T22:36:26.269926+0000","last_undegraded":"2024-05-08T22:36:26.269926+0000","last_fullsized":"2024-05-08T22:36:26.269926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:24.127115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091444+0000","last_change":"2024-05-08T22:36:25.108561+0000","last_active":"2024-05-08T22:36:26.091444+0000","last_peered":"2024-05-08T22:36:26.091444+0000","last_clean":"2024-05-08T22:36:26.091444+0000","last_became_active":"2024-05-08T22:36:25.106222+0000","last_became_peered":"2024-05-08T22:36:25.106222+0000","last_unstale":"2024-05-08T22:36:26.091444+0000","last_undegraded":"2024-05-08T22:36:26.091444+0000","last_fullsized":"2024-05-08T22:36:26.091444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:27.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270289+0000","last_change":"2024-05-08T22:36:25.102432+0000","last_active":"2024-05-08T22:36:26.270289+0000","last_peered":"2024-05-08T22:36:26.270289+0000","last_clean":"2024-05-08T22:36:26.270289+0000","last_became_active":"2024-05-08T22:36:25.102240+0000","last_became_peered":"2024-05-08T22:36:25.102240+0000","last_unstale":"2024-05-08T22:36:26.270289+0000","last_undegraded":"2024-05-08T22:36:26.270289+0000","last_fullsized":"2024-05-08T22:36:26.270289+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270144+0000","last_change":"2024-05-08T22:36:25.102435+0000","last_active":"2024-05-08T22:36:26.270144+0000","last_peered":"2024-05-08T22:36:26.270144+0000","last_clean":"2024-05-08T22:36:26.270144+0000","last_became_active":"2024-05-08T22:36:25.101968+0000","last_became_peered":"2024-05-08T22:36:25.101968+0000","last_unstale":"2024-05-08T22:36:26.270144+0000","last_undegraded":"2024-05-08T22:36:26.270144+0000","last_fullsized":"2024-05-08T22:36:26.270144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094227+0000","last_change":"2024-05-08T22:36:25.094113+0000","last_active":"2024-05-08T22:36:26.094227+0000","last_peered":"2024-05-08T22:36:26.094227+0000","last_clean":"2024-05-08T22:36:26.094227+0000","last_became_active":"2024-05-08T22:36:25.092770+0000","last_became_peered":"2024-05-08T22:36:25.092770+0000","last_unstale":"2024-05-08T22:36:26.094227+0000","last_undegraded":"2024-05-08T22:36:26.094227+0000","last_fullsized":"2024-05-08T22:36:26.094227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:46.820891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091940+0000","last_change":"2024-05-08T22:36:25.111508+0000","last_active":"2024-05-08T22:36:26.091940+0000","last_peered":"2024-05-08T22:36:26.091940+0000","last_clean":"2024-05-08T22:36:26.091940+0000","last_became_active":"2024-05-08T22:36:25.110986+0000","last_became_peered":"2024-05-08T22:36:25.110986+0000","last_unstale":"2024-05-08T22:36:26.091940+0000","last_undegraded":"2024-05-08T22:36:26.091940+0000","last_fullsized":"2024-05-08T22:36:26.091940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269711+0000","last_change":"2024-05-08T22:36:25.100692+0000","last_active":"2024-05-08T22:36:26.269711+0000","last_peered":"2024-05-08T22:36:26.269711+0000","last_clean":"2024-05-08T22:36:26.269711+0000","last_became_active":"2024-05-08T22:36:25.100398+0000","last_became_peered":"2024-05-08T22:36:25.100398+0000","last_unstale":"2024-05-08T22:36:26.269711+0000","last_undegraded":"2024-05-08T22:36:26.269711+0000","last_fullsized":"2024-05-08T22:36:26.269711+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:50.597974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093793+0000","last_change":"2024-05-08T22:36:25.093321+0000","last_active":"2024-05-08T22:36:26.093793+0000","last_peered":"2024-05-08T22:36:26.093793+0000","last_clean":"2024-05-08T22:36:26.093793+0000","last_became_active":"2024-05-08T22:36:25.093005+0000","last_became_peered":"2024-05-08T22:36:25.093005+0000","last_unstale":"2024-05-08T22:36:26.093793+0000","last_undegraded":"2024-05-08T22:36:26.093793+0000","last_fullsized":"2024-05-08T22:36:26.093793+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:51.224795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270467+0000","last_change":"2024-05-08T22:36:25.103124+0000","last_active":"2024-05-08T22:36:26.270467+0000","last_peered":"2024-05-08T22:36:26.270467+0000","last_clean":"2024-05-08T22:36:26.270467+0000","last_became_active":"2024-05-08T22:36:25.102834+0000","last_became_peered":"2024-05-08T22:36:25.102834+0000","last_unstale":"2024-05-08T22:36:26.270467+0000","last_undegraded":"2024-05-08T22:36:26.270467+0000","last_fullsized":"2024-05-08T22:36:26.270467+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:16.410609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093079+0000","last_change":"2024-05-08T22:36:25.121119+0000","last_active":"2024-05-08T22:36:26.093079+0000","last_peered":"2024-05-08T22:36:26.093079+0000","last_clean":"2024-05-08T22:36:26.093079+0000","last_became_active":"2024-05-08T22:36:25.120887+0000","last_became_peered":"2024-05-08T22:36:25.120887+0000","last_unstale":"2024-05-08T22:36:26.093079+0000","last_undegraded":"2024-05-08T22:36:26.093079+0000","last_fullsized":"2024-05-08T22:36:26.093079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:46.834588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.698577+0000","last_change":"2024-05-08T22:36:23.698577+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.698577+0000","last_undegraded":"2024-05-08T22:36:23.698577+0000","last_fullsized":"2024-05-08T22:36:23.698577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094367+0000","last_change":"2024-05-08T22:36:25.118156+0000","last_active":"2024-05-08T22:36:26.094367+0000","last_peered":"2024-05-08T22:36:26.094367+0000","last_clean":"2024-05-08T22:36:26.094367+0000","last_became_active":"2024-05-08T22:36:25.117911+0000","last_became_peered":"2024-05-08T22:36:25.117911+0000","last_unstale":"2024-05-08T22:36:26.094367+0000","last_undegraded":"2024-05-08T22:36:26.094367+0000","last_fullsized":"2024-05-08T22:36:26.094367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272081+0000","last_change":"2024-05-08T22:36:24.717417+0000","last_active":"2024-05-08T22:36:26.272081+0000","last_peered":"2024-05-08T22:36:26.272081+0000","last_clean":"2024-05-08T22:36:26.272081+0000","last_became_active":"2024-05-08T22:36:24.717141+0000","last_became_peered":"2024-05-08T22:36:24.717141+0000","last_unstale":"2024-05-08T22:36:26.272081+0000","last_undegraded":"2024-05-08T22:36:26.272081+0000","last_fullsized":"2024-05-08T22:36:26.272081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:26.143925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271932+0000","last_change":"2024-05-08T22:36:25.107804+0000","last_active":"2024-05-08T22:36:26.271932+0000","last_peered":"2024-05-08T22:36:26.271932+0000","last_clean":"2024-05-08T22:36:26.271932+0000","last_became_active":"2024-05-08T22:36:25.107520+0000","last_became_peered":"2024-05-08T22:36:25.107520+0000","last_unstale":"2024-05-08T22:36:26.271932+0000","last_undegraded":"2024-05-08T22:36:26.271932+0000","last_fullsized":"2024-05-08T22:36:26.271932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:30.250704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271594+0000","last_change":"2024-05-08T22:36:24.712655+0000","last_active":"2024-05-08T22:36:26.271594+0000","last_peered":"2024-05-08T22:36:26.271594+0000","last_clean":"2024-05-08T22:36:26.271594+0000","last_became_active":"2024-05-08T22:36:24.712395+0000","last_became_peered":"2024-05-08T22:36:24.712395+0000","last_unstale":"2024-05-08T22:36:26.271594+0000","last_undegraded":"2024-05-08T22:36:26.271594+0000","last_fullsized":"2024-05-08T22:36:26.271594+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:50.215167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093039+0000","last_change":"2024-05-08T22:36:25.112465+0000","last_active":"2024-05-08T22:36:26.093039+0000","last_peered":"2024-05-08T22:36:26.093039+0000","last_clean":"2024-05-08T22:36:26.093039+0000","last_became_active":"2024-05-08T22:36:25.111281+0000","last_became_peered":"2024-05-08T22:36:25.111281+0000","last_unstale":"2024-05-08T22:36:26.093039+0000","last_undegraded":"2024-05-08T22:36:26.093039+0000","last_fullsized":"2024-05-08T22:36:26.093039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:46.820891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270837+0000","last_change":"2024-05-08T22:36:24.712269+0000","last_active":"2024-05-08T22:36:26.270837+0000","last_peered":"2024-05-08T22:36:26.270837+0000","last_clean":"2024-05-08T22:36:26.270837+0000","last_became_active":"2024-05-08T22:36:24.711998+0000","last_became_peered":"2024-05-08T22:36:24.711998+0000","last_unstale":"2024-05-08T22:36:26.270837+0000","last_undegraded":"2024-05-08T22:36:26.270837+0000","last_fullsized":"2024-05-08T22:36:26.270837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:41.083742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271598+0000","last_change":"2024-05-08T22:36:25.100675+0000","last_active":"2024-05-08T22:36:26.271598+0000","last_peered":"2024-05-08T22:36:26.271598+0000","last_clean":"2024-05-08T22:36:26.271598+0000","last_became_active":"2024-05-08T22:36:25.100358+0000","last_became_peered":"2024-05-08T22:36:25.100358+0000","last_unstale":"2024-05-08T22:36:26.271598+0000","last_undegraded":"2024-05-08T22:36:26.271598+0000","last_fullsized":"2024-05-08T22:36:26.271598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:53.144716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699742+0000","last_change":"2024-05-08T22:36:23.699742+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699742+0000","last_undegraded":"2024-05-08T22:36:23.699742+0000","last_fullsized":"2024-05-08T22:36:23.699742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.700977+0000","last_change":"2024-05-08T22:36:23.700977+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.700977+0000","last_undegraded":"2024-05-08T22:36:23.700977+0000","last_fullsized":"2024-05-08T22:36:23.700977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092141+0000","last_change":"2024-05-08T22:36:25.120862+0000","last_active":"2024-05-08T22:36:26.092141+0000","last_peered":"2024-05-08T22:36:26.092141+0000","last_clean":"2024-05-08T22:36:26.092141+0000","last_became_active":"2024-05-08T22:36:25.120341+0000","last_became_peered":"2024-05-08T22:36:25.120341+0000","last_unstale":"2024-05-08T22:36:26.092141+0000","last_undegraded":"2024-05-08T22:36:26.092141+0000","last_fullsized":"2024-05-08T22:36:26.092141+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:10.892048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701304+0000","last_change":"2024-05-08T22:36:23.701304+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701304+0000","last_undegraded":"2024-05-08T22:36:23.701304+0000","last_fullsized":"2024-05-08T22:36:23.701304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091484+0000","last_change":"2024-05-08T22:36:24.718276+0000","last_active":"2024-05-08T22:36:26.091484+0000","last_peered":"2024-05-08T22:36:26.091484+0000","last_clean":"2024-05-08T22:36:26.091484+0000","last_became_active":"2024-05-08T22:36:24.717997+0000","last_became_peered":"2024-05-08T22:36:24.717997+0000","last_unstale":"2024-05-08T22:36:26.091484+0000","last_undegraded":"2024-05-08T22:36:26.091484+0000","last_fullsized":"2024-05-08T22:36:26.091484+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:01.753101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093149+0000","last_change":"2024-05-08T22:36:24.716034+0000","last_active":"2024-05-08T22:36:26.093149+0000","last_peered":"2024-05-08T22:36:26.093149+0000","last_clean":"2024-05-08T22:36:26.093149+0000","last_became_active":"2024-05-08T22:36:24.715726+0000","last_became_peered":"2024-05-08T22:36:24.715726+0000","last_unstale":"2024-05-08T22:36:26.093149+0000","last_undegraded":"2024-05-08T22:36:26.093149+0000","last_fullsized":"2024-05-08T22:36:26.093149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:27.544825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094498+0000","last_change":"2024-05-08T22:36:24.710780+0000","last_active":"2024-05-08T22:36:26.094498+0000","last_peered":"2024-05-08T22:36:26.094498+0000","last_clean":"2024-05-08T22:36:26.094498+0000","last_became_active":"2024-05-08T22:36:24.710540+0000","last_became_peered":"2024-05-08T22:36:24.710540+0000","last_unstale":"2024-05-08T22:36:26.094498+0000","last_undegraded":"2024-05-08T22:36:26.094498+0000","last_fullsized":"2024-05-08T22:36:26.094498+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:08.600209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093337+0000","last_change":"2024-05-08T22:36:25.119027+0000","last_active":"2024-05-08T22:36:26.093337+0000","last_peered":"2024-05-08T22:36:26.093337+0000","last_clean":"2024-05-08T22:36:26.093337+0000","last_became_active":"2024-05-08T22:36:25.118346+0000","last_became_peered":"2024-05-08T22:36:25.118346+0000","last_unstale":"2024-05-08T22:36:26.093337+0000","last_undegraded":"2024-05-08T22:36:26.093337+0000","last_fullsized":"2024-05-08T22:36:26.093337+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:36.786424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092653+0000","last_change":"2024-05-08T22:36:24.715817+0000","last_active":"2024-05-08T22:36:26.092653+0000","last_peered":"2024-05-08T22:36:26.092653+0000","last_clean":"2024-05-08T22:36:26.092653+0000","last_became_active":"2024-05-08T22:36:24.715468+0000","last_became_peered":"2024-05-08T22:36:24.715468+0000","last_unstale":"2024-05-08T22:36:26.092653+0000","last_undegraded":"2024-05-08T22:36:26.092653+0000","last_fullsized":"2024-05-08T22:36:26.092653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:08.600209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094776+0000","last_change":"2024-05-08T22:36:25.093373+0000","last_active":"2024-05-08T22:36:26.094776+0000","last_peered":"2024-05-08T22:36:26.094776+0000","last_clean":"2024-05-08T22:36:26.094776+0000","last_became_active":"2024-05-08T22:36:25.092969+0000","last_became_peered":"2024-05-08T22:36:25.092969+0000","last_unstale":"2024-05-08T22:36:26.094776+0000","last_undegraded":"2024-05-08T22:36:26.094776+0000","last_fullsized":"2024-05-08T22:36:26.094776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:58.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.697706+0000","last_change":"2024-05-08T22:36:23.697706+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.697706+0000","last_undegraded":"2024-05-08T22:36:23.697706+0000","last_fullsized":"2024-05-08T22:36:23.697706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093687+0000","last_change":"2024-05-08T22:36:25.093687+0000","last_active":"2024-05-08T22:36:26.093687+0000","last_peered":"2024-05-08T22:36:26.093687+0000","last_clean":"2024-05-08T22:36:26.093687+0000","last_became_active":"2024-05-08T22:36:25.093184+0000","last_became_peered":"2024-05-08T22:36:25.093184+0000","last_unstale":"2024-05-08T22:36:26.093687+0000","last_undegraded":"2024-05-08T22:36:26.093687+0000","last_fullsized":"2024-05-08T22:36:26.093687+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.308266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092159+0000","last_change":"2024-05-08T22:36:24.716121+0000","last_active":"2024-05-08T22:36:26.092159+0000","last_peered":"2024-05-08T22:36:26.092159+0000","last_clean":"2024-05-08T22:36:26.092159+0000","last_became_active":"2024-05-08T22:36:24.715315+0000","last_became_peered":"2024-05-08T22:36:24.715315+0000","last_unstale":"2024-05-08T22:36:26.092159+0000","last_undegraded":"2024-05-08T22:36:26.092159+0000","last_fullsized":"2024-05-08T22:36:26.092159+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:47.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269538+0000","last_change":"2024-05-08T22:36:25.106407+0000","last_active":"2024-05-08T22:36:26.269538+0000","last_peered":"2024-05-08T22:36:26.269538+0000","last_clean":"2024-05-08T22:36:26.269538+0000","last_became_active":"2024-05-08T22:36:25.106141+0000","last_became_peered":"2024-05-08T22:36:25.106141+0000","last_unstale":"2024-05-08T22:36:26.269538+0000","last_undegraded":"2024-05-08T22:36:26.269538+0000","last_fullsized":"2024-05-08T22:36:26.269538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:30.536188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269594+0000","last_change":"2024-05-08T22:36:24.717183+0000","last_active":"2024-05-08T22:36:26.269594+0000","last_peered":"2024-05-08T22:36:26.269594+0000","last_clean":"2024-05-08T22:36:26.269594+0000","last_became_active":"2024-05-08T22:36:24.716915+0000","last_became_peered":"2024-05-08T22:36:24.716915+0000","last_unstale":"2024-05-08T22:36:26.269594+0000","last_undegraded":"2024-05-08T22:36:26.269594+0000","last_fullsized":"2024-05-08T22:36:26.269594+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:41.219331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092299+0000","last_change":"2024-05-08T22:36:25.121236+0000","last_active":"2024-05-08T22:36:26.092299+0000","last_peered":"2024-05-08T22:36:26.092299+0000","last_clean":"2024-05-08T22:36:26.092299+0000","last_became_active":"2024-05-08T22:36:25.120859+0000","last_became_peered":"2024-05-08T22:36:25.120859+0000","last_unstale":"2024-05-08T22:36:26.092299+0000","last_undegraded":"2024-05-08T22:36:26.092299+0000","last_fullsized":"2024-05-08T22:36:26.092299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:56.357291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091622+0000","last_change":"2024-05-08T22:36:24.717384+0000","last_active":"2024-05-08T22:36:26.091622+0000","last_peered":"2024-05-08T22:36:26.091622+0000","last_clean":"2024-05-08T22:36:26.091622+0000","last_became_active":"2024-05-08T22:36:24.716291+0000","last_became_peered":"2024-05-08T22:36:24.716291+0000","last_unstale":"2024-05-08T22:36:26.091622+0000","last_undegraded":"2024-05-08T22:36:26.091622+0000","last_fullsized":"2024-05-08T22:36:26.091622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:59.835562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093718+0000","last_change":"2024-05-08T22:36:24.717124+0000","last_active":"2024-05-08T22:36:26.093718+0000","last_peered":"2024-05-08T22:36:26.093718+0000","last_clean":"2024-05-08T22:36:26.093718+0000","last_became_active":"2024-05-08T22:36:24.716799+0000","last_became_peered":"2024-05-08T22:36:24.716799+0000","last_unstale":"2024-05-08T22:36:26.093718+0000","last_undegraded":"2024-05-08T22:36:26.093718+0000","last_fullsized":"2024-05-08T22:36:26.093718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:27.544825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092607+0000","last_change":"2024-05-08T22:36:25.111351+0000","last_active":"2024-05-08T22:36:26.092607+0000","last_peered":"2024-05-08T22:36:26.092607+0000","last_clean":"2024-05-08T22:36:26.092607+0000","last_became_active":"2024-05-08T22:36:25.111012+0000","last_became_peered":"2024-05-08T22:36:25.111012+0000","last_unstale":"2024-05-08T22:36:26.092607+0000","last_undegraded":"2024-05-08T22:36:26.092607+0000","last_fullsized":"2024-05-08T22:36:26.092607+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270330+0000","last_change":"2024-05-08T22:36:24.717060+0000","last_active":"2024-05-08T22:36:26.270330+0000","last_peered":"2024-05-08T22:36:26.270330+0000","last_clean":"2024-05-08T22:36:26.270330+0000","last_became_active":"2024-05-08T22:36:24.716722+0000","last_became_peered":"2024-05-08T22:36:24.716722+0000","last_unstale":"2024-05-08T22:36:26.270330+0000","last_undegraded":"2024-05-08T22:36:26.270330+0000","last_fullsized":"2024-05-08T22:36:26.270330+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:17.582105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092186+0000","last_change":"2024-05-08T22:36:25.116660+0000","last_active":"2024-05-08T22:36:26.092186+0000","last_peered":"2024-05-08T22:36:26.092186+0000","last_clean":"2024-05-08T22:36:26.092186+0000","last_became_active":"2024-05-08T22:36:25.115172+0000","last_became_peered":"2024-05-08T22:36:25.115172+0000","last_unstale":"2024-05-08T22:36:26.092186+0000","last_undegraded":"2024-05-08T22:36:26.092186+0000","last_fullsized":"2024-05-08T22:36:26.092186+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:27.013184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093028+0000","last_change":"2024-05-08T22:36:24.709860+0000","last_active":"2024-05-08T22:36:26.093028+0000","last_peered":"2024-05-08T22:36:26.093028+0000","last_clean":"2024-05-08T22:36:26.093028+0000","last_became_active":"2024-05-08T22:36:24.709571+0000","last_became_peered":"2024-05-08T22:36:24.709571+0000","last_unstale":"2024-05-08T22:36:26.093028+0000","last_undegraded":"2024-05-08T22:36:26.093028+0000","last_fullsized":"2024-05-08T22:36:26.093028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:17.582105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094496+0000","last_change":"2024-05-08T22:36:25.121558+0000","last_active":"2024-05-08T22:36:26.094496+0000","last_peered":"2024-05-08T22:36:26.094496+0000","last_clean":"2024-05-08T22:36:26.094496+0000","last_became_active":"2024-05-08T22:36:25.121302+0000","last_became_peered":"2024-05-08T22:36:25.121302+0000","last_unstale":"2024-05-08T22:36:26.094496+0000","last_undegraded":"2024-05-08T22:36:26.094496+0000","last_fullsized":"2024-05-08T22:36:26.094496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092838+0000","last_change":"2024-05-08T22:36:24.716063+0000","last_active":"2024-05-08T22:36:26.092838+0000","last_peered":"2024-05-08T22:36:26.092838+0000","last_clean":"2024-05-08T22:36:26.092838+0000","last_became_active":"2024-05-08T22:36:24.715380+0000","last_became_peered":"2024-05-08T22:36:24.715380+0000","last_unstale":"2024-05-08T22:36:26.092838+0000","last_undegraded":"2024-05-08T22:36:26.092838+0000","last_fullsized":"2024-05-08T22:36:26.092838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:14.602371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093606+0000","last_change":"2024-05-08T22:36:25.117186+0000","last_active":"2024-05-08T22:36:26.093606+0000","last_peered":"2024-05-08T22:36:26.093606+0000","last_clean":"2024-05-08T22:36:26.093606+0000","last_became_active":"2024-05-08T22:36:25.116877+0000","last_became_peered":"2024-05-08T22:36:25.116877+0000","last_unstale":"2024-05-08T22:36:26.093606+0000","last_undegraded":"2024-05-08T22:36:26.093606+0000","last_fullsized":"2024-05-08T22:36:26.093606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:56.680502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.701388+0000","last_change":"2024-05-08T22:36:23.701388+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.701388+0000","last_undegraded":"2024-05-08T22:36:23.701388+0000","last_fullsized":"2024-05-08T22:36:23.701388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270688+0000","last_change":"2024-05-08T22:36:25.101322+0000","last_active":"2024-05-08T22:36:26.270688+0000","last_peered":"2024-05-08T22:36:26.270688+0000","last_clean":"2024-05-08T22:36:26.270688+0000","last_became_active":"2024-05-08T22:36:25.101071+0000","last_became_peered":"2024-05-08T22:36:25.101071+0000","last_unstale":"2024-05-08T22:36:26.270688+0000","last_undegraded":"2024-05-08T22:36:26.270688+0000","last_fullsized":"2024-05-08T22:36:26.270688+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:37.353563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271170+0000","last_change":"2024-05-08T22:36:24.710252+0000","last_active":"2024-05-08T22:36:26.271170+0000","last_peered":"2024-05-08T22:36:26.271170+0000","last_clean":"2024-05-08T22:36:26.271170+0000","last_became_active":"2024-05-08T22:36:24.709875+0000","last_became_peered":"2024-05-08T22:36:24.709875+0000","last_unstale":"2024-05-08T22:36:26.271170+0000","last_undegraded":"2024-05-08T22:36:26.271170+0000","last_fullsized":"2024-05-08T22:36:26.271170+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:20.298638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269602+0000","last_change":"2024-05-08T22:36:25.102394+0000","last_active":"2024-05-08T22:36:26.269602+0000","last_peered":"2024-05-08T22:36:26.269602+0000","last_clean":"2024-05-08T22:36:26.269602+0000","last_became_active":"2024-05-08T22:36:25.102050+0000","last_became_peered":"2024-05-08T22:36:25.102050+0000","last_unstale":"2024-05-08T22:36:26.269602+0000","last_undegraded":"2024-05-08T22:36:26.269602+0000","last_fullsized":"2024-05-08T22:36:26.269602+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269732+0000","last_change":"2024-05-08T22:36:24.713531+0000","last_active":"2024-05-08T22:36:26.269732+0000","last_peered":"2024-05-08T22:36:26.269732+0000","last_clean":"2024-05-08T22:36:26.269732+0000","last_became_active":"2024-05-08T22:36:24.713186+0000","last_became_peered":"2024-05-08T22:36:24.713186+0000","last_unstale":"2024-05-08T22:36:26.269732+0000","last_undegraded":"2024-05-08T22:36:26.269732+0000","last_fullsized":"2024-05-08T22:36:26.269732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:47.747304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092358+0000","last_change":"2024-05-08T22:36:25.119253+0000","last_active":"2024-05-08T22:36:26.092358+0000","last_peered":"2024-05-08T22:36:26.092358+0000","last_clean":"2024-05-08T22:36:26.092358+0000","last_became_active":"2024-05-08T22:36:25.118822+0000","last_became_peered":"2024-05-08T22:36:25.118822+0000","last_unstale":"2024-05-08T22:36:26.092358+0000","last_undegraded":"2024-05-08T22:36:26.092358+0000","last_fullsized":"2024-05-08T22:36:26.092358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:23.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270510+0000","last_change":"2024-05-08T22:36:24.717483+0000","last_active":"2024-05-08T22:36:26.270510+0000","last_peered":"2024-05-08T22:36:26.270510+0000","last_clean":"2024-05-08T22:36:26.270510+0000","last_became_active":"2024-05-08T22:36:24.717104+0000","last_became_peered":"2024-05-08T22:36:24.717104+0000","last_unstale":"2024-05-08T22:36:26.270510+0000","last_undegraded":"2024-05-08T22:36:26.270510+0000","last_fullsized":"2024-05-08T22:36:26.270510+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:45.811263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270452+0000","last_change":"2024-05-08T22:36:25.108059+0000","last_active":"2024-05-08T22:36:26.270452+0000","last_peered":"2024-05-08T22:36:26.270452+0000","last_clean":"2024-05-08T22:36:26.270452+0000","last_became_active":"2024-05-08T22:36:25.107772+0000","last_became_peered":"2024-05-08T22:36:25.107772+0000","last_unstale":"2024-05-08T22:36:26.270452+0000","last_undegraded":"2024-05-08T22:36:26.270452+0000","last_fullsized":"2024-05-08T22:36:26.270452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:25.624835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270532+0000","last_change":"2024-05-08T22:36:24.716736+0000","last_active":"2024-05-08T22:36:26.270532+0000","last_peered":"2024-05-08T22:36:26.270532+0000","last_clean":"2024-05-08T22:36:26.270532+0000","last_became_active":"2024-05-08T22:36:24.716457+0000","last_became_peered":"2024-05-08T22:36:24.716457+0000","last_unstale":"2024-05-08T22:36:26.270532+0000","last_undegraded":"2024-05-08T22:36:26.270532+0000","last_fullsized":"2024-05-08T22:36:26.270532+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:01.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272176+0000","last_change":"2024-05-08T22:36:25.107768+0000","last_active":"2024-05-08T22:36:26.272176+0000","last_peered":"2024-05-08T22:36:26.272176+0000","last_clean":"2024-05-08T22:36:26.272176+0000","last_became_active":"2024-05-08T22:36:25.107407+0000","last_became_peered":"2024-05-08T22:36:25.107407+0000","last_unstale":"2024-05-08T22:36:26.272176+0000","last_undegraded":"2024-05-08T22:36:26.272176+0000","last_fullsized":"2024-05-08T22:36:26.272176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:59.987796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270396+0000","last_change":"2024-05-08T22:36:24.713879+0000","last_active":"2024-05-08T22:36:26.270396+0000","last_peered":"2024-05-08T22:36:26.270396+0000","last_clean":"2024-05-08T22:36:26.270396+0000","last_became_active":"2024-05-08T22:36:24.713522+0000","last_became_peered":"2024-05-08T22:36:24.713522+0000","last_unstale":"2024-05-08T22:36:26.270396+0000","last_undegraded":"2024-05-08T22:36:26.270396+0000","last_fullsized":"2024-05-08T22:36:26.270396+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:39.806901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270199+0000","last_change":"2024-05-08T22:36:25.107053+0000","last_active":"2024-05-08T22:36:26.270199+0000","last_peered":"2024-05-08T22:36:26.270199+0000","last_clean":"2024-05-08T22:36:26.270199+0000","last_became_active":"2024-05-08T22:36:25.106800+0000","last_became_peered":"2024-05-08T22:36:25.106800+0000","last_unstale":"2024-05-08T22:36:26.270199+0000","last_undegraded":"2024-05-08T22:36:26.270199+0000","last_fullsized":"2024-05-08T22:36:26.270199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:35.496293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272234+0000","last_change":"2024-05-08T22:36:24.712345+0000","last_active":"2024-05-08T22:36:26.272234+0000","last_peered":"2024-05-08T22:36:26.272234+0000","last_clean":"2024-05-08T22:36:26.272234+0000","last_became_active":"2024-05-08T22:36:24.712059+0000","last_became_peered":"2024-05-08T22:36:24.712059+0000","last_unstale":"2024-05-08T22:36:26.272234+0000","last_undegraded":"2024-05-08T22:36:26.272234+0000","last_fullsized":"2024-05-08T22:36:26.272234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:57.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093423+0000","last_change":"2024-05-08T22:36:25.118885+0000","last_active":"2024-05-08T22:36:26.093423+0000","last_peered":"2024-05-08T22:36:26.093423+0000","last_clean":"2024-05-08T22:36:26.093423+0000","last_became_active":"2024-05-08T22:36:25.118515+0000","last_became_peered":"2024-05-08T22:36:25.118515+0000","last_unstale":"2024-05-08T22:36:26.093423+0000","last_undegraded":"2024-05-08T22:36:26.093423+0000","last_fullsized":"2024-05-08T22:36:26.093423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:38.250241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696746+0000","last_change":"2024-05-08T22:36:23.696746+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696746+0000","last_undegraded":"2024-05-08T22:36:23.696746+0000","last_fullsized":"2024-05-08T22:36:23.696746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091288+0000","last_change":"2024-05-08T22:36:25.121033+0000","last_active":"2024-05-08T22:36:26.091288+0000","last_peered":"2024-05-08T22:36:26.091288+0000","last_clean":"2024-05-08T22:36:26.091288+0000","last_became_active":"2024-05-08T22:36:25.120751+0000","last_became_peered":"2024-05-08T22:36:25.120751+0000","last_unstale":"2024-05-08T22:36:26.091288+0000","last_undegraded":"2024-05-08T22:36:26.091288+0000","last_fullsized":"2024-05-08T22:36:26.091288+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:15.931328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092406+0000","last_change":"2024-05-08T22:36:24.719941+0000","last_active":"2024-05-08T22:36:26.092406+0000","last_peered":"2024-05-08T22:36:26.092406+0000","last_clean":"2024-05-08T22:36:26.092406+0000","last_became_active":"2024-05-08T22:36:24.719701+0000","last_became_peered":"2024-05-08T22:36:24.719701+0000","last_unstale":"2024-05-08T22:36:26.092406+0000","last_undegraded":"2024-05-08T22:36:26.092406+0000","last_fullsized":"2024-05-08T22:36:26.092406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:14.461876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093885+0000","last_change":"2024-05-08T22:36:25.101483+0000","last_active":"2024-05-08T22:36:26.093885+0000","last_peered":"2024-05-08T22:36:26.093885+0000","last_clean":"2024-05-08T22:36:26.093885+0000","last_became_active":"2024-05-08T22:36:25.100871+0000","last_became_peered":"2024-05-08T22:36:25.100871+0000","last_unstale":"2024-05-08T22:36:26.093885+0000","last_undegraded":"2024-05-08T22:36:26.093885+0000","last_fullsized":"2024-05-08T22:36:26.093885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:42.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699737+0000","last_change":"2024-05-08T22:36:23.699737+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699737+0000","last_undegraded":"2024-05-08T22:36:23.699737+0000","last_fullsized":"2024-05-08T22:36:23.699737+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:36:24.685710+0000","last_change":"2024-05-08T22:36:24.685710+0000","last_active":"2024-05-08T22:36:24.685710+0000","last_peered":"2024-05-08T22:36:24.685710+0000","last_clean":"2024-05-08T22:36:24.685710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:24.685710+0000","last_undegraded":"2024-05-08T22:36:24.685710+0000","last_fullsized":"2024-05-08T22:36:24.685710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271234+0000","last_change":"2024-05-08T22:36:25.099537+0000","last_active":"2024-05-08T22:36:26.271234+0000","last_peered":"2024-05-08T22:36:26.271234+0000","last_clean":"2024-05-08T22:36:26.271234+0000","last_became_active":"2024-05-08T22:36:25.099311+0000","last_became_peered":"2024-05-08T22:36:25.099311+0000","last_unstale":"2024-05-08T22:36:26.271234+0000","last_undegraded":"2024-05-08T22:36:26.271234+0000","last_fullsized":"2024-05-08T22:36:26.271234+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:24.620101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.696747+0000","last_change":"2024-05-08T22:36:23.696747+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.696747+0000","last_undegraded":"2024-05-08T22:36:23.696747+0000","last_fullsized":"2024-05-08T22:36:23.696747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092879+0000","last_change":"2024-05-08T22:36:25.117200+0000","last_active":"2024-05-08T22:36:26.092879+0000","last_peered":"2024-05-08T22:36:26.092879+0000","last_clean":"2024-05-08T22:36:26.092879+0000","last_became_active":"2024-05-08T22:36:25.116658+0000","last_became_peered":"2024-05-08T22:36:25.116658+0000","last_unstale":"2024-05-08T22:36:26.092879+0000","last_undegraded":"2024-05-08T22:36:26.092879+0000","last_fullsized":"2024-05-08T22:36:26.092879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:44.204427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-05-08T22:36:23.699763+0000","last_change":"2024-05-08T22:36:23.699763+0000","last_active":"2024-05-08T22:36:23.676082+0000","last_peered":"2024-05-08T22:36:23.676082+0000","last_clean":"2024-05-08T22:36:23.676082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:36:23.699763+0000","last_undegraded":"2024-05-08T22:36:23.699763+0000","last_fullsized":"2024-05-08T22:36:23.699763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.098563+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:36:25.098563+0000","last_peered":"2024-05-08T22:36:25.098563+0000","last_clean":"2024-05-08T22:36:25.098563+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:36:25.098563+0000","last_undegraded":"2024-05-08T22:36:25.098563+0000","last_fullsized":"2024-05-08T22:36:25.098563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":778240,"data_stored":345054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63},"min":{"1min":0.44500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.012},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0529999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97099999999999997,"5min":1.101,"15min":1.101},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.51000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0089999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0649999999999999,"5min":1.21,"15min":1.21},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.92100000000000004},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2330000000000001,"5min":1.306,"15min":1.306},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":0.94099999999999995},"min":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.234,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.008,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2649999999999999,"5min":1.476,"15min":1.476},"last":1.097}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.68600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3069999999999999,"5min":1.446,"15min":1.446},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2809999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.095,"15min":1.107},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4510000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.167},{"interface":"front","average":{"1min":1,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.347,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.121}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":351191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.04,"15min":1.04},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.046,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.371,"15min":0.371},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.3280000000000001,"5min":1.431,"15min":1.431},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.96699999999999997},"min":{"1min":0.75,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.226,"5min":1.508,"15min":1.508},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.268,"5min":1.327,"15min":1.327},"last":1.042},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.093},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.268,"5min":1.726,"15min":1.726},"last":1.397}]},{"osd":6,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.83899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.256,"5min":1.659,"15min":1.659},"last":1.446}]},{"osd":7,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4319999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.79900000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.131}]}]},{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1437696,"data_stored":962637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.032},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.91300000000000003}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90500000000000003,"15min":0.92100000000000004},"min":{"1min":0.69399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1220000000000001,"5min":1.419,"15min":1.419},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.151,"5min":1.337,"15min":1.337},"last":1.177}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0629999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.754,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1619999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.92900000000000005}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.109,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3799999999999999,"5min":1.482,"15min":1.482},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0489999999999999},"min":{"1min":0.77600000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.091},{"interface":"front","average":{"1min":1.077,"5min":1.071,"15min":1.089},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.96699999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":831488,"data_stored":371527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.141},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.88200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.997,"5min":1.758,"15min":1.758},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.216,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":0.99199999999999999},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.93100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1559999999999999,"5min":1.419,"15min":1.419},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.071,"15min":1.109},"min":{"1min":0.76000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3580000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.97099999999999997,"15min":0.999},"min":{"1min":0.68000000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.216,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.1220000000000001}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4370000000000001,"5min":1.538,"15min":1.538},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.69499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.407,"5min":1.458,"15min":1.458},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.528,"5min":1.681,"15min":1.681},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1519999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3160000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1679999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0549999999999999,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2509999999999999,"5min":1.835,"15min":1.835},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.218,"15min":1.248},"min":{"1min":0.87,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.595,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.097}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":831488,"data_stored":371515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.53400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.014,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.123,"5min":1.52,"15min":1.52},"last":0.82599999999999996}]},{"osd":1,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81899999999999995,"15min":0.80100000000000005},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1559999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1759999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.57499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.1299999999999999}]},{"osd":3,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.86599999999999999},"min":{"1min":0.60099999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.0449999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97099999999999997,"15min":0.96399999999999997},"min":{"1min":0.82499999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.0580000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.038}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.63900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1739999999999999,"5min":1.304,"15min":1.304},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97899999999999998},"min":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.149,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1930000000000001}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.90600000000000003},"min":{"1min":0.71699999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.605,"15min":1.605},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.90100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.92400000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2330000000000001,"5min":1.538,"15min":1.538},"last":1.236},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0529999999999999,"15min":1.0169999999999999},"min":{"1min":0.90300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.278,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.075}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":839680,"data_stored":373057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.64800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.94899999999999995}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.872,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1819999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.163},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.67000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.214,"5min":1.29,"15min":1.29},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.351,"5min":1.637,"15min":1.637},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.337,"5min":2.601,"15min":2.601},"last":1.038},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.073,"15min":1.119},"min":{"1min":0.81000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4259999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.083}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2929999999999999,"5min":1.792,"15min":1.792},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.034},"min":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3839999999999999,"5min":1.575,"15min":1.575},"last":1.137}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0800000000000001,"15min":1.1000000000000001},"min":{"1min":0.71299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2849999999999999,"5min":1.831,"15min":1.831},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":1.012,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.347,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.1850000000000001}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.099,"15min":1.1140000000000001},"min":{"1min":0.83399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5169999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1759999999999999,"15min":1.1830000000000001},"min":{"1min":0.85799999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.405,"5min":1.944,"15min":1.944},"last":1.21}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":371493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85299999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.008,"15min":1.008},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59299999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90700000000000003},"min":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3029999999999999,"5min":1.363,"15min":1.363},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0069999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.204,"5min":1.282,"15min":1.282},"last":0.90100000000000002}]},{"osd":5,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.347,"5min":1.484,"15min":1.484},"last":0.97399999999999998},{"interface":"front","average":{"1min":1,"5min":1.0149999999999999,"15min":1.02},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2509999999999999,"5min":1.391,"15min":1.391},"last":1.2270000000000001}]},{"osd":6,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.052,"15min":1.0609999999999999},"min":{"1min":0.80200000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3120000000000001,"5min":1.673,"15min":1.673},"last":0.92700000000000005}]},{"osd":7,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0700000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.997},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2849999999999999,"5min":1.409,"15min":1.409},"last":1.052}]}]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343432,"statfs":{"total":96636764160,"available":96607674368,"internally_reserved":0,"allocated":1421312,"data_stored":961861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.012,"5min":1.617,"15min":1.617},"last":0.495},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.446,"15min":1.446},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.032,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81399999999999995},"min":{"1min":0.55300000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":1.714,"15min":1.714},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.327,"15min":1.327},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.70699999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2110000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.22,"5min":1.669,"15min":1.669},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70099999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0289999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":1},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0609999999999999,"5min":1.474,"15min":1.474},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95299999999999996,"15min":0.96799999999999997},"min":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1579999999999999,"5min":1.397,"15min":1.397},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0349999999999999,"15min":1.0529999999999999},"min":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1759999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.93500000000000005}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.85699999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.2909999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.877},{"interface":"front","average":{"1min":0.996,"5min":1.01,"15min":1.022},"min":{"1min":0.85999999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1359999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:36:32.559 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:36:32.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 Processor -- start 2024-05-08T22:36:32.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 -- start start 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feebc0ce060 0x7feebc06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 0x7feebc0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0add40 0x7feebc06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0ae550 con 0x7feebc0ce060 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc06abb0 con 0x7feebc0cc310 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc161ac0 con 0x7feebc0add40 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feebc0ce060 0x7feebc06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feebc0ce060 0x7feebc06ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55958/0 (socket says 172.21.15.179:55958) 2024-05-08T22:36:32.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 -- 172.21.15.179:0/1585305378 learned_addr learned my addr 172.21.15.179:0/1585305378 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:32.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebb7fe700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0add40 0x7feebc06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feeba7fc700 1 --2- 172.21.15.179:0/1585305378 >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 0x7feebc0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 -- 172.21.15.179:0/1585305378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0add40 msgr2=0x7feebc06b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0add40 0x7feebc06b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 -- 172.21.15.179:0/1585305378 >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 msgr2=0x7feebc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 --2- 172.21.15.179:0/1585305378 >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 0x7feebc0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 -- 172.21.15.179:0/1585305378 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0ce980 con 0x7feebc0ce060 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feeba7fc700 1 --2- 172.21.15.179:0/1585305378 >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 0x7feebc0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feebaffd700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc06ce50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feebc073c10 tx=0x7feea4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a504e0cbb273d3c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feeb9ffb700 1 -- 172.21.15.179:0/1585305378 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400b8a0 con 0x7feebc0ce060 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feeb9ffb700 1 -- 172.21.15.179:0/1585305378 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400ba00 con 0x7feebc0ce060 2024-05-08T22:36:32.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feeb9ffb700 1 -- 172.21.15.179:0/1585305378 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400bd90 con 0x7feebc0ce060 2024-05-08T22:36:32.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 -- 172.21.15.179:0/1585305378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 msgr2=0x7feebc06ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:32.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.682+0000 7feec1279700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc06ce50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feebc073c10 tx=0x7feea4009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 -- 172.21.15.179:0/1585305378 shutdown_connections 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0add40 0x7feebc06b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 --2- 172.21.15.179:0/1585305378 >> v2:172.21.15.195:3300/0 conn(0x7feebc0cc310 0x7feebc0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 --2- 172.21.15.179:0/1585305378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc06ce50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 -- 172.21.15.179:0/1585305378 >> 172.21.15.179:0/1585305378 conn(0x7feebc0cb660 msgr2=0x7feebc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 -- 172.21.15.179:0/1585305378 shutdown_connections 2024-05-08T22:36:32.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 -- 172.21.15.179:0/1585305378 wait complete. 2024-05-08T22:36:32.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 Processor -- start 2024-05-08T22:36:32.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.686+0000 7feec1279700 1 -- start start 2024-05-08T22:36:32.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc0add40 0x7feebc0aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0cc310 0x7feebc15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b2d20 con 0x7feebc0ce060 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b3000 con 0x7feebc0add40 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b32e0 con 0x7feebc0cc310 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55968/0 (socket says 172.21.15.179:55968) 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 -- 172.21.15.179:0/3408333818 learned_addr learned my addr 172.21.15.179:0/3408333818 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feeba7fc700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0cc310 0x7feebc15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebaffd700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc0add40 0x7feebc0aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 -- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0cc310 msgr2=0x7feebc15f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0cc310 0x7feebc15f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 -- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc0add40 msgr2=0x7feebc0aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc0add40 0x7feebc0aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 -- 172.21.15.179:0/3408333818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea4009570 con 0x7feebc0ce060 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feebb7fe700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feeb400d800 tx=0x7feeb400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40125f0 con 0x7feebc0ce060 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4012770 con 0x7feebc0ce060 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0b35c0 con 0x7feebc0ce060 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4012b20 con 0x7feebc0ce060 2024-05-08T22:36:32.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc160010 con 0x7feebc0ce060 2024-05-08T22:36:32.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.690+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc067770 con 0x7feebc0ce060 2024-05-08T22:36:32.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.694+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feeb4012c80 con 0x7feebc0ce060 2024-05-08T22:36:32.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.694+0000 7feeb37fe700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 0x7fee9c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:32.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.694+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7feeb4084100 con 0x7feebc0ce060 2024-05-08T22:36:32.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.694+0000 7feebaffd700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 0x7fee9c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:32.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.694+0000 7feebaffd700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 0x7fee9c068bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feea4000c00 tx=0x7feea4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:32.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.698+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feeb40513c0 con 0x7feebc0ce060 2024-05-08T22:36:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.870+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feebc0ab920 con 0x7fee9c066730 2024-05-08T22:36:32.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.886+0000 7feeb37fe700 1 -- 172.21.15.179:0/3408333818 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287879 (secure 0 0 0) 0x7feebc0ab920 con 0x7fee9c066730 2024-05-08T22:36:32.893 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 msgr2=0x7fee9c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 0x7fee9c068bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feea4000c00 tx=0x7feea4008030 comp rx=0 tx=0).stop 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 msgr2=0x7feebc0b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feeb400d800 tx=0x7feeb400db10 comp rx=0 tx=0).stop 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 shutdown_connections 2024-05-08T22:36:32.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee9c066730 0x7fee9c068bf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feebc0ce060 0x7feebc0b3f00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feebc0cc310 0x7feebc15f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 --2- 172.21.15.179:0/3408333818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feebc0add40 0x7feebc0aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 >> 172.21.15.179:0/3408333818 conn(0x7feebc0cb660 msgr2=0x7feebc06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 shutdown_connections 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:32.894+0000 7feec1279700 1 -- 172.21.15.179:0/3408333818 wait complete. 2024-05-08T22:36:32.896 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:36:32.909 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-05-08T22:36:31.446131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248440,"kb_used_data":7688,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726280,"statfs":{"total":773094113280,"available":772839710720,"internally_reserved":0,"allocated":7872512,"data_stored":4157404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.755603"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273899+0000","last_change":"2024-05-08T22:36:25.099695+0000","last_active":"2024-05-08T22:36:26.273899+0000","last_peered":"2024-05-08T22:36:26.273899+0000","last_clean":"2024-05-08T22:36:26.273899+0000","last_became_active":"2024-05-08T22:36:25.099333+0000","last_became_peered":"2024-05-08T22:36:25.099333+0000","last_unstale":"2024-05-08T22:36:26.273899+0000","last_undegraded":"2024-05-08T22:36:26.273899+0000","last_fullsized":"2024-05-08T22:36:26.273899+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:38.250241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273827+0000","last_change":"2024-05-08T22:36:25.108572+0000","last_active":"2024-05-08T22:36:26.273827+0000","last_peered":"2024-05-08T22:36:26.273827+0000","last_clean":"2024-05-08T22:36:26.273827+0000","last_became_active":"2024-05-08T22:36:25.108324+0000","last_became_peered":"2024-05-08T22:36:25.108324+0000","last_unstale":"2024-05-08T22:36:26.273827+0000","last_undegraded":"2024-05-08T22:36:26.273827+0000","last_fullsized":"2024-05-08T22:36:26.273827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:56.680502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092560+0000","last_change":"2024-05-08T22:36:25.108945+0000","last_active":"2024-05-08T22:36:26.092560+0000","last_peered":"2024-05-08T22:36:26.092560+0000","last_clean":"2024-05-08T22:36:26.092560+0000","last_became_active":"2024-05-08T22:36:25.108540+0000","last_became_peered":"2024-05-08T22:36:25.108540+0000","last_unstale":"2024-05-08T22:36:26.092560+0000","last_undegraded":"2024-05-08T22:36:26.092560+0000","last_fullsized":"2024-05-08T22:36:26.092560+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.308266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094558+0000","last_change":"2024-05-08T22:36:25.104780+0000","last_active":"2024-05-08T22:36:26.094558+0000","last_peered":"2024-05-08T22:36:26.094558+0000","last_clean":"2024-05-08T22:36:26.094558+0000","last_became_active":"2024-05-08T22:36:25.101532+0000","last_became_peered":"2024-05-08T22:36:25.101532+0000","last_unstale":"2024-05-08T22:36:26.094558+0000","last_undegraded":"2024-05-08T22:36:26.094558+0000","last_fullsized":"2024-05-08T22:36:26.094558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:50.597974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092786+0000","last_change":"2024-05-08T22:36:25.120411+0000","last_active":"2024-05-08T22:36:26.092786+0000","last_peered":"2024-05-08T22:36:26.092786+0000","last_clean":"2024-05-08T22:36:26.092786+0000","last_became_active":"2024-05-08T22:36:25.118888+0000","last_became_peered":"2024-05-08T22:36:25.118888+0000","last_unstale":"2024-05-08T22:36:26.092786+0000","last_undegraded":"2024-05-08T22:36:26.092786+0000","last_fullsized":"2024-05-08T22:36:26.092786+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:03.368207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270900+0000","last_change":"2024-05-08T22:36:25.102907+0000","last_active":"2024-05-08T22:36:26.270900+0000","last_peered":"2024-05-08T22:36:26.270900+0000","last_clean":"2024-05-08T22:36:26.270900+0000","last_became_active":"2024-05-08T22:36:25.102653+0000","last_became_peered":"2024-05-08T22:36:25.102653+0000","last_unstale":"2024-05-08T22:36:26.270900+0000","last_undegraded":"2024-05-08T22:36:26.270900+0000","last_fullsized":"2024-05-08T22:36:26.270900+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:54.091244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270795+0000","last_change":"2024-05-08T22:36:25.111088+0000","last_active":"2024-05-08T22:36:26.270795+0000","last_peered":"2024-05-08T22:36:26.270795+0000","last_clean":"2024-05-08T22:36:26.270795+0000","last_became_active":"2024-05-08T22:36:25.110831+0000","last_became_peered":"2024-05-08T22:36:25.110831+0000","last_unstale":"2024-05-08T22:36:26.270795+0000","last_undegraded":"2024-05-08T22:36:26.270795+0000","last_fullsized":"2024-05-08T22:36:26.270795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:26.098141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269436+0000","last_change":"2024-05-08T22:36:25.104669+0000","last_active":"2024-05-08T22:36:26.269436+0000","last_peered":"2024-05-08T22:36:26.269436+0000","last_clean":"2024-05-08T22:36:26.269436+0000","last_became_active":"2024-05-08T22:36:25.104172+0000","last_became_peered":"2024-05-08T22:36:25.104172+0000","last_unstale":"2024-05-08T22:36:26.269436+0000","last_undegraded":"2024-05-08T22:36:26.269436+0000","last_fullsized":"2024-05-08T22:36:26.269436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:29.315608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093251+0000","last_change":"2024-05-08T22:36:25.112373+0000","last_active":"2024-05-08T22:36:26.093251+0000","last_peered":"2024-05-08T22:36:26.093251+0000","last_clean":"2024-05-08T22:36:26.093251+0000","last_became_active":"2024-05-08T22:36:25.111430+0000","last_became_peered":"2024-05-08T22:36:25.111430+0000","last_unstale":"2024-05-08T22:36:26.093251+0000","last_undegraded":"2024-05-08T22:36:26.093251+0000","last_fullsized":"2024-05-08T22:36:26.093251+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:51.224795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092786+0000","last_change":"2024-05-08T22:36:25.109292+0000","last_active":"2024-05-08T22:36:26.092786+0000","last_peered":"2024-05-08T22:36:26.092786+0000","last_clean":"2024-05-08T22:36:26.092786+0000","last_became_active":"2024-05-08T22:36:25.108914+0000","last_became_peered":"2024-05-08T22:36:25.108914+0000","last_unstale":"2024-05-08T22:36:26.092786+0000","last_undegraded":"2024-05-08T22:36:26.092786+0000","last_fullsized":"2024-05-08T22:36:26.092786+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:58.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094235+0000","last_change":"2024-05-08T22:36:25.100287+0000","last_active":"2024-05-08T22:36:26.094235+0000","last_peered":"2024-05-08T22:36:26.094235+0000","last_clean":"2024-05-08T22:36:26.094235+0000","last_became_active":"2024-05-08T22:36:25.100010+0000","last_became_peered":"2024-05-08T22:36:25.100010+0000","last_unstale":"2024-05-08T22:36:26.094235+0000","last_undegraded":"2024-05-08T22:36:26.094235+0000","last_fullsized":"2024-05-08T22:36:26.094235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:27.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269889+0000","last_change":"2024-05-08T22:36:25.099776+0000","last_active":"2024-05-08T22:36:26.269889+0000","last_peered":"2024-05-08T22:36:26.269889+0000","last_clean":"2024-05-08T22:36:26.269889+0000","last_became_active":"2024-05-08T22:36:25.099500+0000","last_became_peered":"2024-05-08T22:36:25.099500+0000","last_unstale":"2024-05-08T22:36:26.269889+0000","last_undegraded":"2024-05-08T22:36:26.269889+0000","last_fullsized":"2024-05-08T22:36:26.269889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:40.400169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269863+0000","last_change":"2024-05-08T22:36:25.107160+0000","last_active":"2024-05-08T22:36:26.269863+0000","last_peered":"2024-05-08T22:36:26.269863+0000","last_clean":"2024-05-08T22:36:26.269863+0000","last_became_active":"2024-05-08T22:36:25.106917+0000","last_became_peered":"2024-05-08T22:36:25.106917+0000","last_unstale":"2024-05-08T22:36:26.269863+0000","last_undegraded":"2024-05-08T22:36:26.269863+0000","last_fullsized":"2024-05-08T22:36:26.269863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:06.090448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091507+0000","last_change":"2024-05-08T22:36:25.110725+0000","last_active":"2024-05-08T22:36:26.091507+0000","last_peered":"2024-05-08T22:36:26.091507+0000","last_clean":"2024-05-08T22:36:26.091507+0000","last_became_active":"2024-05-08T22:36:25.108936+0000","last_became_peered":"2024-05-08T22:36:25.108936+0000","last_unstale":"2024-05-08T22:36:26.091507+0000","last_undegraded":"2024-05-08T22:36:26.091507+0000","last_fullsized":"2024-05-08T22:36:26.091507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091065+0000","last_change":"2024-05-08T22:36:25.108772+0000","last_active":"2024-05-08T22:36:26.091065+0000","last_peered":"2024-05-08T22:36:26.091065+0000","last_clean":"2024-05-08T22:36:26.091065+0000","last_became_active":"2024-05-08T22:36:25.108475+0000","last_became_peered":"2024-05-08T22:36:25.108475+0000","last_unstale":"2024-05-08T22:36:26.091065+0000","last_undegraded":"2024-05-08T22:36:26.091065+0000","last_fullsized":"2024-05-08T22:36:26.091065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:42.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272404+0000","last_change":"2024-05-08T22:36:25.100558+0000","last_active":"2024-05-08T22:36:26.272404+0000","last_peered":"2024-05-08T22:36:26.272404+0000","last_clean":"2024-05-08T22:36:26.272404+0000","last_became_active":"2024-05-08T22:36:25.100039+0000","last_became_peered":"2024-05-08T22:36:25.100039+0000","last_unstale":"2024-05-08T22:36:26.272404+0000","last_undegraded":"2024-05-08T22:36:26.272404+0000","last_fullsized":"2024-05-08T22:36:26.272404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:38.134712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273944+0000","last_change":"2024-05-08T22:36:25.095804+0000","last_active":"2024-05-08T22:36:26.273944+0000","last_peered":"2024-05-08T22:36:26.273944+0000","last_clean":"2024-05-08T22:36:26.273944+0000","last_became_active":"2024-05-08T22:36:25.095620+0000","last_became_peered":"2024-05-08T22:36:25.095620+0000","last_unstale":"2024-05-08T22:36:26.273944+0000","last_undegraded":"2024-05-08T22:36:26.273944+0000","last_fullsized":"2024-05-08T22:36:26.273944+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:44.204427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094371+0000","last_change":"2024-05-08T22:36:25.108952+0000","last_active":"2024-05-08T22:36:26.094371+0000","last_peered":"2024-05-08T22:36:26.094371+0000","last_clean":"2024-05-08T22:36:26.094371+0000","last_became_active":"2024-05-08T22:36:25.108487+0000","last_became_peered":"2024-05-08T22:36:25.108487+0000","last_unstale":"2024-05-08T22:36:26.094371+0000","last_undegraded":"2024-05-08T22:36:26.094371+0000","last_fullsized":"2024-05-08T22:36:26.094371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091953+0000","last_change":"2024-05-08T22:36:25.117445+0000","last_active":"2024-05-08T22:36:26.091953+0000","last_peered":"2024-05-08T22:36:26.091953+0000","last_clean":"2024-05-08T22:36:26.091953+0000","last_became_active":"2024-05-08T22:36:25.116915+0000","last_became_peered":"2024-05-08T22:36:25.116915+0000","last_unstale":"2024-05-08T22:36:26.091953+0000","last_undegraded":"2024-05-08T22:36:26.091953+0000","last_fullsized":"2024-05-08T22:36:26.091953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:12.701134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271684+0000","last_change":"2024-05-08T22:36:25.123709+0000","last_active":"2024-05-08T22:36:26.271684+0000","last_peered":"2024-05-08T22:36:26.271684+0000","last_clean":"2024-05-08T22:36:26.271684+0000","last_became_active":"2024-05-08T22:36:25.123378+0000","last_became_peered":"2024-05-08T22:36:25.123378+0000","last_unstale":"2024-05-08T22:36:26.271684+0000","last_undegraded":"2024-05-08T22:36:26.271684+0000","last_fullsized":"2024-05-08T22:36:26.271684+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:06.311940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269926+0000","last_change":"2024-05-08T22:36:25.101874+0000","last_active":"2024-05-08T22:36:26.269926+0000","last_peered":"2024-05-08T22:36:26.269926+0000","last_clean":"2024-05-08T22:36:26.269926+0000","last_became_active":"2024-05-08T22:36:25.101625+0000","last_became_peered":"2024-05-08T22:36:25.101625+0000","last_unstale":"2024-05-08T22:36:26.269926+0000","last_undegraded":"2024-05-08T22:36:26.269926+0000","last_fullsized":"2024-05-08T22:36:26.269926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:24.127115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272229+0000","last_change":"2024-05-08T22:36:25.099347+0000","last_active":"2024-05-08T22:36:26.272229+0000","last_peered":"2024-05-08T22:36:26.272229+0000","last_clean":"2024-05-08T22:36:26.272229+0000","last_became_active":"2024-05-08T22:36:25.099100+0000","last_became_peered":"2024-05-08T22:36:25.099100+0000","last_unstale":"2024-05-08T22:36:26.272229+0000","last_undegraded":"2024-05-08T22:36:26.272229+0000","last_fullsized":"2024-05-08T22:36:26.272229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:11.265737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271450+0000","last_change":"2024-05-08T22:36:25.100234+0000","last_active":"2024-05-08T22:36:26.271450+0000","last_peered":"2024-05-08T22:36:26.271450+0000","last_clean":"2024-05-08T22:36:26.271450+0000","last_became_active":"2024-05-08T22:36:25.099947+0000","last_became_peered":"2024-05-08T22:36:25.099947+0000","last_unstale":"2024-05-08T22:36:26.271450+0000","last_undegraded":"2024-05-08T22:36:26.271450+0000","last_fullsized":"2024-05-08T22:36:26.271450+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:59.688462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091444+0000","last_change":"2024-05-08T22:36:25.108561+0000","last_active":"2024-05-08T22:36:26.091444+0000","last_peered":"2024-05-08T22:36:26.091444+0000","last_clean":"2024-05-08T22:36:26.091444+0000","last_became_active":"2024-05-08T22:36:25.106222+0000","last_became_peered":"2024-05-08T22:36:25.106222+0000","last_unstale":"2024-05-08T22:36:26.091444+0000","last_undegraded":"2024-05-08T22:36:26.091444+0000","last_fullsized":"2024-05-08T22:36:26.091444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:27.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270289+0000","last_change":"2024-05-08T22:36:25.102432+0000","last_active":"2024-05-08T22:36:26.270289+0000","last_peered":"2024-05-08T22:36:26.270289+0000","last_clean":"2024-05-08T22:36:26.270289+0000","last_became_active":"2024-05-08T22:36:25.102240+0000","last_became_peered":"2024-05-08T22:36:25.102240+0000","last_unstale":"2024-05-08T22:36:26.270289+0000","last_undegraded":"2024-05-08T22:36:26.270289+0000","last_fullsized":"2024-05-08T22:36:26.270289+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270144+0000","last_change":"2024-05-08T22:36:25.102435+0000","last_active":"2024-05-08T22:36:26.270144+0000","last_peered":"2024-05-08T22:36:26.270144+0000","last_clean":"2024-05-08T22:36:26.270144+0000","last_became_active":"2024-05-08T22:36:25.101968+0000","last_became_peered":"2024-05-08T22:36:25.101968+0000","last_unstale":"2024-05-08T22:36:26.270144+0000","last_undegraded":"2024-05-08T22:36:26.270144+0000","last_fullsized":"2024-05-08T22:36:26.270144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094227+0000","last_change":"2024-05-08T22:36:25.094113+0000","last_active":"2024-05-08T22:36:26.094227+0000","last_peered":"2024-05-08T22:36:26.094227+0000","last_clean":"2024-05-08T22:36:26.094227+0000","last_became_active":"2024-05-08T22:36:25.092770+0000","last_became_peered":"2024-05-08T22:36:25.092770+0000","last_unstale":"2024-05-08T22:36:26.094227+0000","last_undegraded":"2024-05-08T22:36:26.094227+0000","last_fullsized":"2024-05-08T22:36:26.094227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:46.820891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273867+0000","last_change":"2024-05-08T22:36:25.124774+0000","last_active":"2024-05-08T22:36:26.273867+0000","last_peered":"2024-05-08T22:36:26.273867+0000","last_clean":"2024-05-08T22:36:26.273867+0000","last_became_active":"2024-05-08T22:36:25.124494+0000","last_became_peered":"2024-05-08T22:36:25.124494+0000","last_unstale":"2024-05-08T22:36:26.273867+0000","last_undegraded":"2024-05-08T22:36:26.273867+0000","last_fullsized":"2024-05-08T22:36:26.273867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:46.834588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091940+0000","last_change":"2024-05-08T22:36:25.111508+0000","last_active":"2024-05-08T22:36:26.091940+0000","last_peered":"2024-05-08T22:36:26.091940+0000","last_clean":"2024-05-08T22:36:26.091940+0000","last_became_active":"2024-05-08T22:36:25.110986+0000","last_became_peered":"2024-05-08T22:36:25.110986+0000","last_unstale":"2024-05-08T22:36:26.091940+0000","last_undegraded":"2024-05-08T22:36:26.091940+0000","last_fullsized":"2024-05-08T22:36:26.091940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269711+0000","last_change":"2024-05-08T22:36:25.100692+0000","last_active":"2024-05-08T22:36:26.269711+0000","last_peered":"2024-05-08T22:36:26.269711+0000","last_clean":"2024-05-08T22:36:26.269711+0000","last_became_active":"2024-05-08T22:36:25.100398+0000","last_became_peered":"2024-05-08T22:36:25.100398+0000","last_unstale":"2024-05-08T22:36:26.269711+0000","last_undegraded":"2024-05-08T22:36:26.269711+0000","last_fullsized":"2024-05-08T22:36:26.269711+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:50.597974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093793+0000","last_change":"2024-05-08T22:36:25.093321+0000","last_active":"2024-05-08T22:36:26.093793+0000","last_peered":"2024-05-08T22:36:26.093793+0000","last_clean":"2024-05-08T22:36:26.093793+0000","last_became_active":"2024-05-08T22:36:25.093005+0000","last_became_peered":"2024-05-08T22:36:25.093005+0000","last_unstale":"2024-05-08T22:36:26.093793+0000","last_undegraded":"2024-05-08T22:36:26.093793+0000","last_fullsized":"2024-05-08T22:36:26.093793+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:51.224795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270467+0000","last_change":"2024-05-08T22:36:25.103124+0000","last_active":"2024-05-08T22:36:26.270467+0000","last_peered":"2024-05-08T22:36:26.270467+0000","last_clean":"2024-05-08T22:36:26.270467+0000","last_became_active":"2024-05-08T22:36:25.102834+0000","last_became_peered":"2024-05-08T22:36:25.102834+0000","last_unstale":"2024-05-08T22:36:26.270467+0000","last_undegraded":"2024-05-08T22:36:26.270467+0000","last_fullsized":"2024-05-08T22:36:26.270467+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:16.410609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093079+0000","last_change":"2024-05-08T22:36:25.121119+0000","last_active":"2024-05-08T22:36:26.093079+0000","last_peered":"2024-05-08T22:36:26.093079+0000","last_clean":"2024-05-08T22:36:26.093079+0000","last_became_active":"2024-05-08T22:36:25.120887+0000","last_became_peered":"2024-05-08T22:36:25.120887+0000","last_unstale":"2024-05-08T22:36:26.093079+0000","last_undegraded":"2024-05-08T22:36:26.093079+0000","last_fullsized":"2024-05-08T22:36:26.093079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:46.834588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"68'4","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.114298+0000","last_change":"2024-05-08T22:36:24.719668+0000","last_active":"2024-05-08T22:36:25.114298+0000","last_peered":"2024-05-08T22:36:25.114298+0000","last_clean":"2024-05-08T22:36:25.114298+0000","last_became_active":"2024-05-08T22:36:24.719415+0000","last_became_peered":"2024-05-08T22:36:24.719415+0000","last_unstale":"2024-05-08T22:36:25.114298+0000","last_undegraded":"2024-05-08T22:36:25.114298+0000","last_fullsized":"2024-05-08T22:36:25.114298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:36:36.656860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094367+0000","last_change":"2024-05-08T22:36:25.118156+0000","last_active":"2024-05-08T22:36:26.094367+0000","last_peered":"2024-05-08T22:36:26.094367+0000","last_clean":"2024-05-08T22:36:26.094367+0000","last_became_active":"2024-05-08T22:36:25.117911+0000","last_became_peered":"2024-05-08T22:36:25.117911+0000","last_unstale":"2024-05-08T22:36:26.094367+0000","last_undegraded":"2024-05-08T22:36:26.094367+0000","last_fullsized":"2024-05-08T22:36:26.094367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:23.130747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272081+0000","last_change":"2024-05-08T22:36:24.717417+0000","last_active":"2024-05-08T22:36:26.272081+0000","last_peered":"2024-05-08T22:36:26.272081+0000","last_clean":"2024-05-08T22:36:26.272081+0000","last_became_active":"2024-05-08T22:36:24.717141+0000","last_became_peered":"2024-05-08T22:36:24.717141+0000","last_unstale":"2024-05-08T22:36:26.272081+0000","last_undegraded":"2024-05-08T22:36:26.272081+0000","last_fullsized":"2024-05-08T22:36:26.272081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:26.143925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271932+0000","last_change":"2024-05-08T22:36:25.107804+0000","last_active":"2024-05-08T22:36:26.271932+0000","last_peered":"2024-05-08T22:36:26.271932+0000","last_clean":"2024-05-08T22:36:26.271932+0000","last_became_active":"2024-05-08T22:36:25.107520+0000","last_became_peered":"2024-05-08T22:36:25.107520+0000","last_unstale":"2024-05-08T22:36:26.271932+0000","last_undegraded":"2024-05-08T22:36:26.271932+0000","last_fullsized":"2024-05-08T22:36:26.271932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:30.250704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271594+0000","last_change":"2024-05-08T22:36:24.712655+0000","last_active":"2024-05-08T22:36:26.271594+0000","last_peered":"2024-05-08T22:36:26.271594+0000","last_clean":"2024-05-08T22:36:26.271594+0000","last_became_active":"2024-05-08T22:36:24.712395+0000","last_became_peered":"2024-05-08T22:36:24.712395+0000","last_unstale":"2024-05-08T22:36:26.271594+0000","last_undegraded":"2024-05-08T22:36:26.271594+0000","last_fullsized":"2024-05-08T22:36:26.271594+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:50.215167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093039+0000","last_change":"2024-05-08T22:36:25.112465+0000","last_active":"2024-05-08T22:36:26.093039+0000","last_peered":"2024-05-08T22:36:26.093039+0000","last_clean":"2024-05-08T22:36:26.093039+0000","last_became_active":"2024-05-08T22:36:25.111281+0000","last_became_peered":"2024-05-08T22:36:25.111281+0000","last_unstale":"2024-05-08T22:36:26.093039+0000","last_undegraded":"2024-05-08T22:36:26.093039+0000","last_fullsized":"2024-05-08T22:36:26.093039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:46.820891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270837+0000","last_change":"2024-05-08T22:36:24.712269+0000","last_active":"2024-05-08T22:36:26.270837+0000","last_peered":"2024-05-08T22:36:26.270837+0000","last_clean":"2024-05-08T22:36:26.270837+0000","last_became_active":"2024-05-08T22:36:24.711998+0000","last_became_peered":"2024-05-08T22:36:24.711998+0000","last_unstale":"2024-05-08T22:36:26.270837+0000","last_undegraded":"2024-05-08T22:36:26.270837+0000","last_fullsized":"2024-05-08T22:36:26.270837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:41.083742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271598+0000","last_change":"2024-05-08T22:36:25.100675+0000","last_active":"2024-05-08T22:36:26.271598+0000","last_peered":"2024-05-08T22:36:26.271598+0000","last_clean":"2024-05-08T22:36:26.271598+0000","last_became_active":"2024-05-08T22:36:25.100358+0000","last_became_peered":"2024-05-08T22:36:25.100358+0000","last_unstale":"2024-05-08T22:36:26.271598+0000","last_undegraded":"2024-05-08T22:36:26.271598+0000","last_fullsized":"2024-05-08T22:36:26.271598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:53.144716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"68'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.107785+0000","last_change":"2024-05-08T22:36:24.717729+0000","last_active":"2024-05-08T22:36:25.107785+0000","last_peered":"2024-05-08T22:36:25.107785+0000","last_clean":"2024-05-08T22:36:25.107785+0000","last_became_active":"2024-05-08T22:36:24.717480+0000","last_became_peered":"2024-05-08T22:36:24.717480+0000","last_unstale":"2024-05-08T22:36:25.107785+0000","last_undegraded":"2024-05-08T22:36:25.107785+0000","last_fullsized":"2024-05-08T22:36:25.107785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:24.436027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273130+0000","last_change":"2024-05-08T22:36:25.093897+0000","last_active":"2024-05-08T22:36:26.273130+0000","last_peered":"2024-05-08T22:36:26.273130+0000","last_clean":"2024-05-08T22:36:26.273130+0000","last_became_active":"2024-05-08T22:36:25.092951+0000","last_became_peered":"2024-05-08T22:36:25.092951+0000","last_unstale":"2024-05-08T22:36:26.273130+0000","last_undegraded":"2024-05-08T22:36:26.273130+0000","last_fullsized":"2024-05-08T22:36:26.273130+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:10.892048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"68'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.106736+0000","last_change":"2024-05-08T22:36:24.717366+0000","last_active":"2024-05-08T22:36:25.106736+0000","last_peered":"2024-05-08T22:36:25.106736+0000","last_clean":"2024-05-08T22:36:25.106736+0000","last_became_active":"2024-05-08T22:36:24.716770+0000","last_became_peered":"2024-05-08T22:36:24.716770+0000","last_unstale":"2024-05-08T22:36:25.106736+0000","last_undegraded":"2024-05-08T22:36:25.106736+0000","last_fullsized":"2024-05-08T22:36:25.106736+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:40.817715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092141+0000","last_change":"2024-05-08T22:36:25.120862+0000","last_active":"2024-05-08T22:36:26.092141+0000","last_peered":"2024-05-08T22:36:26.092141+0000","last_clean":"2024-05-08T22:36:26.092141+0000","last_became_active":"2024-05-08T22:36:25.120341+0000","last_became_peered":"2024-05-08T22:36:25.120341+0000","last_unstale":"2024-05-08T22:36:26.092141+0000","last_undegraded":"2024-05-08T22:36:26.092141+0000","last_fullsized":"2024-05-08T22:36:26.092141+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:10.892048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.087075+0000","last_change":"2024-05-08T22:36:24.718881+0000","last_active":"2024-05-08T22:36:25.087075+0000","last_peered":"2024-05-08T22:36:25.087075+0000","last_clean":"2024-05-08T22:36:25.087075+0000","last_became_active":"2024-05-08T22:36:24.718031+0000","last_became_peered":"2024-05-08T22:36:24.718031+0000","last_unstale":"2024-05-08T22:36:25.087075+0000","last_undegraded":"2024-05-08T22:36:25.087075+0000","last_fullsized":"2024-05-08T22:36:25.087075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:19.727656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272758+0000","last_change":"2024-05-08T22:36:25.093313+0000","last_active":"2024-05-08T22:36:26.272758+0000","last_peered":"2024-05-08T22:36:26.272758+0000","last_clean":"2024-05-08T22:36:26.272758+0000","last_became_active":"2024-05-08T22:36:25.093067+0000","last_became_peered":"2024-05-08T22:36:25.093067+0000","last_unstale":"2024-05-08T22:36:26.272758+0000","last_undegraded":"2024-05-08T22:36:26.272758+0000","last_fullsized":"2024-05-08T22:36:26.272758+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:08.043658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091484+0000","last_change":"2024-05-08T22:36:24.718276+0000","last_active":"2024-05-08T22:36:26.091484+0000","last_peered":"2024-05-08T22:36:26.091484+0000","last_clean":"2024-05-08T22:36:26.091484+0000","last_became_active":"2024-05-08T22:36:24.717997+0000","last_became_peered":"2024-05-08T22:36:24.717997+0000","last_unstale":"2024-05-08T22:36:26.091484+0000","last_undegraded":"2024-05-08T22:36:26.091484+0000","last_fullsized":"2024-05-08T22:36:26.091484+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:01.753101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273800+0000","last_change":"2024-05-08T22:36:25.094391+0000","last_active":"2024-05-08T22:36:26.273800+0000","last_peered":"2024-05-08T22:36:26.273800+0000","last_clean":"2024-05-08T22:36:26.273800+0000","last_became_active":"2024-05-08T22:36:25.094159+0000","last_became_peered":"2024-05-08T22:36:25.094159+0000","last_unstale":"2024-05-08T22:36:26.273800+0000","last_undegraded":"2024-05-08T22:36:26.273800+0000","last_fullsized":"2024-05-08T22:36:26.273800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:14.666835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093149+0000","last_change":"2024-05-08T22:36:24.716034+0000","last_active":"2024-05-08T22:36:26.093149+0000","last_peered":"2024-05-08T22:36:26.093149+0000","last_clean":"2024-05-08T22:36:26.093149+0000","last_became_active":"2024-05-08T22:36:24.715726+0000","last_became_peered":"2024-05-08T22:36:24.715726+0000","last_unstale":"2024-05-08T22:36:26.093149+0000","last_undegraded":"2024-05-08T22:36:26.093149+0000","last_fullsized":"2024-05-08T22:36:26.093149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:27.544825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271455+0000","last_change":"2024-05-08T22:36:25.099953+0000","last_active":"2024-05-08T22:36:26.271455+0000","last_peered":"2024-05-08T22:36:26.271455+0000","last_clean":"2024-05-08T22:36:26.271455+0000","last_became_active":"2024-05-08T22:36:25.099747+0000","last_became_peered":"2024-05-08T22:36:25.099747+0000","last_unstale":"2024-05-08T22:36:26.271455+0000","last_undegraded":"2024-05-08T22:36:26.271455+0000","last_fullsized":"2024-05-08T22:36:26.271455+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:45.894780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094498+0000","last_change":"2024-05-08T22:36:24.710780+0000","last_active":"2024-05-08T22:36:26.094498+0000","last_peered":"2024-05-08T22:36:26.094498+0000","last_clean":"2024-05-08T22:36:26.094498+0000","last_became_active":"2024-05-08T22:36:24.710540+0000","last_became_peered":"2024-05-08T22:36:24.710540+0000","last_unstale":"2024-05-08T22:36:26.094498+0000","last_undegraded":"2024-05-08T22:36:26.094498+0000","last_fullsized":"2024-05-08T22:36:26.094498+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:08.600209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093337+0000","last_change":"2024-05-08T22:36:25.119027+0000","last_active":"2024-05-08T22:36:26.093337+0000","last_peered":"2024-05-08T22:36:26.093337+0000","last_clean":"2024-05-08T22:36:26.093337+0000","last_became_active":"2024-05-08T22:36:25.118346+0000","last_became_peered":"2024-05-08T22:36:25.118346+0000","last_unstale":"2024-05-08T22:36:26.093337+0000","last_undegraded":"2024-05-08T22:36:26.093337+0000","last_fullsized":"2024-05-08T22:36:26.093337+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:36.786424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092653+0000","last_change":"2024-05-08T22:36:24.715817+0000","last_active":"2024-05-08T22:36:26.092653+0000","last_peered":"2024-05-08T22:36:26.092653+0000","last_clean":"2024-05-08T22:36:26.092653+0000","last_became_active":"2024-05-08T22:36:24.715468+0000","last_became_peered":"2024-05-08T22:36:24.715468+0000","last_unstale":"2024-05-08T22:36:26.092653+0000","last_undegraded":"2024-05-08T22:36:26.092653+0000","last_fullsized":"2024-05-08T22:36:26.092653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:08.600209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094776+0000","last_change":"2024-05-08T22:36:25.093373+0000","last_active":"2024-05-08T22:36:26.094776+0000","last_peered":"2024-05-08T22:36:26.094776+0000","last_clean":"2024-05-08T22:36:26.094776+0000","last_became_active":"2024-05-08T22:36:25.092969+0000","last_became_peered":"2024-05-08T22:36:25.092969+0000","last_unstale":"2024-05-08T22:36:26.094776+0000","last_undegraded":"2024-05-08T22:36:26.094776+0000","last_fullsized":"2024-05-08T22:36:26.094776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:58.224189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"68'6","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.116710+0000","last_change":"2024-05-08T22:36:24.708425+0000","last_active":"2024-05-08T22:36:25.116710+0000","last_peered":"2024-05-08T22:36:25.116710+0000","last_clean":"2024-05-08T22:36:25.116710+0000","last_became_active":"2024-05-08T22:36:24.708180+0000","last_became_peered":"2024-05-08T22:36:24.708180+0000","last_unstale":"2024-05-08T22:36:25.116710+0000","last_undegraded":"2024-05-08T22:36:25.116710+0000","last_fullsized":"2024-05-08T22:36:25.116710+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:07.871934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093687+0000","last_change":"2024-05-08T22:36:25.093687+0000","last_active":"2024-05-08T22:36:26.093687+0000","last_peered":"2024-05-08T22:36:26.093687+0000","last_clean":"2024-05-08T22:36:26.093687+0000","last_became_active":"2024-05-08T22:36:25.093184+0000","last_became_peered":"2024-05-08T22:36:25.093184+0000","last_unstale":"2024-05-08T22:36:26.093687+0000","last_undegraded":"2024-05-08T22:36:26.093687+0000","last_fullsized":"2024-05-08T22:36:26.093687+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.308266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092159+0000","last_change":"2024-05-08T22:36:24.716121+0000","last_active":"2024-05-08T22:36:26.092159+0000","last_peered":"2024-05-08T22:36:26.092159+0000","last_clean":"2024-05-08T22:36:26.092159+0000","last_became_active":"2024-05-08T22:36:24.715315+0000","last_became_peered":"2024-05-08T22:36:24.715315+0000","last_unstale":"2024-05-08T22:36:26.092159+0000","last_undegraded":"2024-05-08T22:36:26.092159+0000","last_fullsized":"2024-05-08T22:36:26.092159+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:47.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269538+0000","last_change":"2024-05-08T22:36:25.106407+0000","last_active":"2024-05-08T22:36:26.269538+0000","last_peered":"2024-05-08T22:36:26.269538+0000","last_clean":"2024-05-08T22:36:26.269538+0000","last_became_active":"2024-05-08T22:36:25.106141+0000","last_became_peered":"2024-05-08T22:36:25.106141+0000","last_unstale":"2024-05-08T22:36:26.269538+0000","last_undegraded":"2024-05-08T22:36:26.269538+0000","last_fullsized":"2024-05-08T22:36:26.269538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:30.536188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269594+0000","last_change":"2024-05-08T22:36:24.717183+0000","last_active":"2024-05-08T22:36:26.269594+0000","last_peered":"2024-05-08T22:36:26.269594+0000","last_clean":"2024-05-08T22:36:26.269594+0000","last_became_active":"2024-05-08T22:36:24.716915+0000","last_became_peered":"2024-05-08T22:36:24.716915+0000","last_unstale":"2024-05-08T22:36:26.269594+0000","last_undegraded":"2024-05-08T22:36:26.269594+0000","last_fullsized":"2024-05-08T22:36:26.269594+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:41.219331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092299+0000","last_change":"2024-05-08T22:36:25.121236+0000","last_active":"2024-05-08T22:36:26.092299+0000","last_peered":"2024-05-08T22:36:26.092299+0000","last_clean":"2024-05-08T22:36:26.092299+0000","last_became_active":"2024-05-08T22:36:25.120859+0000","last_became_peered":"2024-05-08T22:36:25.120859+0000","last_unstale":"2024-05-08T22:36:26.092299+0000","last_undegraded":"2024-05-08T22:36:26.092299+0000","last_fullsized":"2024-05-08T22:36:26.092299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:56.357291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091622+0000","last_change":"2024-05-08T22:36:24.717384+0000","last_active":"2024-05-08T22:36:26.091622+0000","last_peered":"2024-05-08T22:36:26.091622+0000","last_clean":"2024-05-08T22:36:26.091622+0000","last_became_active":"2024-05-08T22:36:24.716291+0000","last_became_peered":"2024-05-08T22:36:24.716291+0000","last_unstale":"2024-05-08T22:36:26.091622+0000","last_undegraded":"2024-05-08T22:36:26.091622+0000","last_fullsized":"2024-05-08T22:36:26.091622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:59.835562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273624+0000","last_change":"2024-05-08T22:36:25.099295+0000","last_active":"2024-05-08T22:36:26.273624+0000","last_peered":"2024-05-08T22:36:26.273624+0000","last_clean":"2024-05-08T22:36:26.273624+0000","last_became_active":"2024-05-08T22:36:25.098449+0000","last_became_peered":"2024-05-08T22:36:25.098449+0000","last_unstale":"2024-05-08T22:36:26.273624+0000","last_undegraded":"2024-05-08T22:36:26.273624+0000","last_fullsized":"2024-05-08T22:36:26.273624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:56.357291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093718+0000","last_change":"2024-05-08T22:36:24.717124+0000","last_active":"2024-05-08T22:36:26.093718+0000","last_peered":"2024-05-08T22:36:26.093718+0000","last_clean":"2024-05-08T22:36:26.093718+0000","last_became_active":"2024-05-08T22:36:24.716799+0000","last_became_peered":"2024-05-08T22:36:24.716799+0000","last_unstale":"2024-05-08T22:36:26.093718+0000","last_undegraded":"2024-05-08T22:36:26.093718+0000","last_fullsized":"2024-05-08T22:36:26.093718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:27.544825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092607+0000","last_change":"2024-05-08T22:36:25.111351+0000","last_active":"2024-05-08T22:36:26.092607+0000","last_peered":"2024-05-08T22:36:26.092607+0000","last_clean":"2024-05-08T22:36:26.092607+0000","last_became_active":"2024-05-08T22:36:25.111012+0000","last_became_peered":"2024-05-08T22:36:25.111012+0000","last_unstale":"2024-05-08T22:36:26.092607+0000","last_undegraded":"2024-05-08T22:36:26.092607+0000","last_fullsized":"2024-05-08T22:36:26.092607+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:33.244308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270330+0000","last_change":"2024-05-08T22:36:24.717060+0000","last_active":"2024-05-08T22:36:26.270330+0000","last_peered":"2024-05-08T22:36:26.270330+0000","last_clean":"2024-05-08T22:36:26.270330+0000","last_became_active":"2024-05-08T22:36:24.716722+0000","last_became_peered":"2024-05-08T22:36:24.716722+0000","last_unstale":"2024-05-08T22:36:26.270330+0000","last_undegraded":"2024-05-08T22:36:26.270330+0000","last_fullsized":"2024-05-08T22:36:26.270330+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:17.582105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092186+0000","last_change":"2024-05-08T22:36:25.116660+0000","last_active":"2024-05-08T22:36:26.092186+0000","last_peered":"2024-05-08T22:36:26.092186+0000","last_clean":"2024-05-08T22:36:26.092186+0000","last_became_active":"2024-05-08T22:36:25.115172+0000","last_became_peered":"2024-05-08T22:36:25.115172+0000","last_unstale":"2024-05-08T22:36:26.092186+0000","last_undegraded":"2024-05-08T22:36:26.092186+0000","last_fullsized":"2024-05-08T22:36:26.092186+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:27.013184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093028+0000","last_change":"2024-05-08T22:36:24.709860+0000","last_active":"2024-05-08T22:36:26.093028+0000","last_peered":"2024-05-08T22:36:26.093028+0000","last_clean":"2024-05-08T22:36:26.093028+0000","last_became_active":"2024-05-08T22:36:24.709571+0000","last_became_peered":"2024-05-08T22:36:24.709571+0000","last_unstale":"2024-05-08T22:36:26.093028+0000","last_undegraded":"2024-05-08T22:36:26.093028+0000","last_fullsized":"2024-05-08T22:36:26.093028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:17.582105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.094496+0000","last_change":"2024-05-08T22:36:25.121558+0000","last_active":"2024-05-08T22:36:26.094496+0000","last_peered":"2024-05-08T22:36:26.094496+0000","last_clean":"2024-05-08T22:36:26.094496+0000","last_became_active":"2024-05-08T22:36:25.121302+0000","last_became_peered":"2024-05-08T22:36:25.121302+0000","last_unstale":"2024-05-08T22:36:26.094496+0000","last_undegraded":"2024-05-08T22:36:26.094496+0000","last_fullsized":"2024-05-08T22:36:26.094496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092838+0000","last_change":"2024-05-08T22:36:24.716063+0000","last_active":"2024-05-08T22:36:26.092838+0000","last_peered":"2024-05-08T22:36:26.092838+0000","last_clean":"2024-05-08T22:36:26.092838+0000","last_became_active":"2024-05-08T22:36:24.715380+0000","last_became_peered":"2024-05-08T22:36:24.715380+0000","last_unstale":"2024-05-08T22:36:26.092838+0000","last_undegraded":"2024-05-08T22:36:26.092838+0000","last_fullsized":"2024-05-08T22:36:26.092838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:14.602371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093606+0000","last_change":"2024-05-08T22:36:25.117186+0000","last_active":"2024-05-08T22:36:26.093606+0000","last_peered":"2024-05-08T22:36:26.093606+0000","last_clean":"2024-05-08T22:36:26.093606+0000","last_became_active":"2024-05-08T22:36:25.116877+0000","last_became_peered":"2024-05-08T22:36:25.116877+0000","last_unstale":"2024-05-08T22:36:26.093606+0000","last_undegraded":"2024-05-08T22:36:26.093606+0000","last_fullsized":"2024-05-08T22:36:26.093606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:08:56.680502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.087755+0000","last_change":"2024-05-08T22:36:24.717845+0000","last_active":"2024-05-08T22:36:25.087755+0000","last_peered":"2024-05-08T22:36:25.087755+0000","last_clean":"2024-05-08T22:36:25.087755+0000","last_became_active":"2024-05-08T22:36:24.716885+0000","last_became_peered":"2024-05-08T22:36:24.716885+0000","last_unstale":"2024-05-08T22:36:25.087755+0000","last_undegraded":"2024-05-08T22:36:25.087755+0000","last_fullsized":"2024-05-08T22:36:25.087755+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:01.354846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270688+0000","last_change":"2024-05-08T22:36:25.101322+0000","last_active":"2024-05-08T22:36:26.270688+0000","last_peered":"2024-05-08T22:36:26.270688+0000","last_clean":"2024-05-08T22:36:26.270688+0000","last_became_active":"2024-05-08T22:36:25.101071+0000","last_became_peered":"2024-05-08T22:36:25.101071+0000","last_unstale":"2024-05-08T22:36:26.270688+0000","last_undegraded":"2024-05-08T22:36:26.270688+0000","last_fullsized":"2024-05-08T22:36:26.270688+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:37.353563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271170+0000","last_change":"2024-05-08T22:36:24.710252+0000","last_active":"2024-05-08T22:36:26.271170+0000","last_peered":"2024-05-08T22:36:26.271170+0000","last_clean":"2024-05-08T22:36:26.271170+0000","last_became_active":"2024-05-08T22:36:24.709875+0000","last_became_peered":"2024-05-08T22:36:24.709875+0000","last_unstale":"2024-05-08T22:36:26.271170+0000","last_undegraded":"2024-05-08T22:36:26.271170+0000","last_fullsized":"2024-05-08T22:36:26.271170+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:20.298638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269602+0000","last_change":"2024-05-08T22:36:25.102394+0000","last_active":"2024-05-08T22:36:26.269602+0000","last_peered":"2024-05-08T22:36:26.269602+0000","last_clean":"2024-05-08T22:36:26.269602+0000","last_became_active":"2024-05-08T22:36:25.102050+0000","last_became_peered":"2024-05-08T22:36:25.102050+0000","last_unstale":"2024-05-08T22:36:26.269602+0000","last_undegraded":"2024-05-08T22:36:26.269602+0000","last_fullsized":"2024-05-08T22:36:26.269602+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:38.384417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.269732+0000","last_change":"2024-05-08T22:36:24.713531+0000","last_active":"2024-05-08T22:36:26.269732+0000","last_peered":"2024-05-08T22:36:26.269732+0000","last_clean":"2024-05-08T22:36:26.269732+0000","last_became_active":"2024-05-08T22:36:24.713186+0000","last_became_peered":"2024-05-08T22:36:24.713186+0000","last_unstale":"2024-05-08T22:36:26.269732+0000","last_undegraded":"2024-05-08T22:36:26.269732+0000","last_fullsized":"2024-05-08T22:36:26.269732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:47.747304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092358+0000","last_change":"2024-05-08T22:36:25.119253+0000","last_active":"2024-05-08T22:36:26.092358+0000","last_peered":"2024-05-08T22:36:26.092358+0000","last_clean":"2024-05-08T22:36:26.092358+0000","last_became_active":"2024-05-08T22:36:25.118822+0000","last_became_peered":"2024-05-08T22:36:25.118822+0000","last_unstale":"2024-05-08T22:36:26.092358+0000","last_undegraded":"2024-05-08T22:36:26.092358+0000","last_fullsized":"2024-05-08T22:36:26.092358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:23.416109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270510+0000","last_change":"2024-05-08T22:36:24.717483+0000","last_active":"2024-05-08T22:36:26.270510+0000","last_peered":"2024-05-08T22:36:26.270510+0000","last_clean":"2024-05-08T22:36:26.270510+0000","last_became_active":"2024-05-08T22:36:24.717104+0000","last_became_peered":"2024-05-08T22:36:24.717104+0000","last_unstale":"2024-05-08T22:36:26.270510+0000","last_undegraded":"2024-05-08T22:36:26.270510+0000","last_fullsized":"2024-05-08T22:36:26.270510+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:45.811263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270452+0000","last_change":"2024-05-08T22:36:25.108059+0000","last_active":"2024-05-08T22:36:26.270452+0000","last_peered":"2024-05-08T22:36:26.270452+0000","last_clean":"2024-05-08T22:36:26.270452+0000","last_became_active":"2024-05-08T22:36:25.107772+0000","last_became_peered":"2024-05-08T22:36:25.107772+0000","last_unstale":"2024-05-08T22:36:26.270452+0000","last_undegraded":"2024-05-08T22:36:26.270452+0000","last_fullsized":"2024-05-08T22:36:26.270452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:25.624835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270532+0000","last_change":"2024-05-08T22:36:24.716736+0000","last_active":"2024-05-08T22:36:26.270532+0000","last_peered":"2024-05-08T22:36:26.270532+0000","last_clean":"2024-05-08T22:36:26.270532+0000","last_became_active":"2024-05-08T22:36:24.716457+0000","last_became_peered":"2024-05-08T22:36:24.716457+0000","last_unstale":"2024-05-08T22:36:26.270532+0000","last_undegraded":"2024-05-08T22:36:26.270532+0000","last_fullsized":"2024-05-08T22:36:26.270532+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:01.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272176+0000","last_change":"2024-05-08T22:36:25.107768+0000","last_active":"2024-05-08T22:36:26.272176+0000","last_peered":"2024-05-08T22:36:26.272176+0000","last_clean":"2024-05-08T22:36:26.272176+0000","last_became_active":"2024-05-08T22:36:25.107407+0000","last_became_peered":"2024-05-08T22:36:25.107407+0000","last_unstale":"2024-05-08T22:36:26.272176+0000","last_undegraded":"2024-05-08T22:36:26.272176+0000","last_fullsized":"2024-05-08T22:36:26.272176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:59.987796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270396+0000","last_change":"2024-05-08T22:36:24.713879+0000","last_active":"2024-05-08T22:36:26.270396+0000","last_peered":"2024-05-08T22:36:26.270396+0000","last_clean":"2024-05-08T22:36:26.270396+0000","last_became_active":"2024-05-08T22:36:24.713522+0000","last_became_peered":"2024-05-08T22:36:24.713522+0000","last_unstale":"2024-05-08T22:36:26.270396+0000","last_undegraded":"2024-05-08T22:36:26.270396+0000","last_fullsized":"2024-05-08T22:36:26.270396+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:39.806901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.270199+0000","last_change":"2024-05-08T22:36:25.107053+0000","last_active":"2024-05-08T22:36:26.270199+0000","last_peered":"2024-05-08T22:36:26.270199+0000","last_clean":"2024-05-08T22:36:26.270199+0000","last_became_active":"2024-05-08T22:36:25.106800+0000","last_became_peered":"2024-05-08T22:36:25.106800+0000","last_unstale":"2024-05-08T22:36:26.270199+0000","last_undegraded":"2024-05-08T22:36:26.270199+0000","last_fullsized":"2024-05-08T22:36:26.270199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:35.496293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.272234+0000","last_change":"2024-05-08T22:36:24.712345+0000","last_active":"2024-05-08T22:36:26.272234+0000","last_peered":"2024-05-08T22:36:26.272234+0000","last_clean":"2024-05-08T22:36:26.272234+0000","last_became_active":"2024-05-08T22:36:24.712059+0000","last_became_peered":"2024-05-08T22:36:24.712059+0000","last_unstale":"2024-05-08T22:36:26.272234+0000","last_undegraded":"2024-05-08T22:36:26.272234+0000","last_fullsized":"2024-05-08T22:36:26.272234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:57.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093423+0000","last_change":"2024-05-08T22:36:25.118885+0000","last_active":"2024-05-08T22:36:26.093423+0000","last_peered":"2024-05-08T22:36:26.093423+0000","last_clean":"2024-05-08T22:36:26.093423+0000","last_became_active":"2024-05-08T22:36:25.118515+0000","last_became_peered":"2024-05-08T22:36:25.118515+0000","last_unstale":"2024-05-08T22:36:26.093423+0000","last_undegraded":"2024-05-08T22:36:26.093423+0000","last_fullsized":"2024-05-08T22:36:26.093423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:38.250241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.085621+0000","last_change":"2024-05-08T22:36:24.710341+0000","last_active":"2024-05-08T22:36:25.085621+0000","last_peered":"2024-05-08T22:36:25.085621+0000","last_clean":"2024-05-08T22:36:25.085621+0000","last_became_active":"2024-05-08T22:36:24.710016+0000","last_became_peered":"2024-05-08T22:36:24.710016+0000","last_unstale":"2024-05-08T22:36:25.085621+0000","last_undegraded":"2024-05-08T22:36:25.085621+0000","last_fullsized":"2024-05-08T22:36:25.085621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:19.890711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.091288+0000","last_change":"2024-05-08T22:36:25.121033+0000","last_active":"2024-05-08T22:36:26.091288+0000","last_peered":"2024-05-08T22:36:26.091288+0000","last_clean":"2024-05-08T22:36:26.091288+0000","last_became_active":"2024-05-08T22:36:25.120751+0000","last_became_peered":"2024-05-08T22:36:25.120751+0000","last_unstale":"2024-05-08T22:36:26.091288+0000","last_undegraded":"2024-05-08T22:36:26.091288+0000","last_fullsized":"2024-05-08T22:36:26.091288+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:15.931328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092406+0000","last_change":"2024-05-08T22:36:24.719941+0000","last_active":"2024-05-08T22:36:26.092406+0000","last_peered":"2024-05-08T22:36:26.092406+0000","last_clean":"2024-05-08T22:36:26.092406+0000","last_became_active":"2024-05-08T22:36:24.719701+0000","last_became_peered":"2024-05-08T22:36:24.719701+0000","last_unstale":"2024-05-08T22:36:26.092406+0000","last_undegraded":"2024-05-08T22:36:26.092406+0000","last_fullsized":"2024-05-08T22:36:26.092406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:14.461876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.093885+0000","last_change":"2024-05-08T22:36:25.101483+0000","last_active":"2024-05-08T22:36:26.093885+0000","last_peered":"2024-05-08T22:36:26.093885+0000","last_clean":"2024-05-08T22:36:26.093885+0000","last_became_active":"2024-05-08T22:36:25.100871+0000","last_became_peered":"2024-05-08T22:36:25.100871+0000","last_unstale":"2024-05-08T22:36:26.093885+0000","last_undegraded":"2024-05-08T22:36:26.093885+0000","last_fullsized":"2024-05-08T22:36:26.093885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:42.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.088485+0000","last_change":"2024-05-08T22:36:24.720832+0000","last_active":"2024-05-08T22:36:25.088485+0000","last_peered":"2024-05-08T22:36:25.088485+0000","last_clean":"2024-05-08T22:36:25.088485+0000","last_became_active":"2024-05-08T22:36:24.720552+0000","last_became_peered":"2024-05-08T22:36:24.720552+0000","last_unstale":"2024-05-08T22:36:25.088485+0000","last_undegraded":"2024-05-08T22:36:25.088485+0000","last_fullsized":"2024-05-08T22:36:25.088485+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:51.731764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.273413+0000","last_change":"2024-05-08T22:36:25.093523+0000","last_active":"2024-05-08T22:36:26.273413+0000","last_peered":"2024-05-08T22:36:26.273413+0000","last_clean":"2024-05-08T22:36:26.273413+0000","last_became_active":"2024-05-08T22:36:25.092961+0000","last_became_peered":"2024-05-08T22:36:25.092961+0000","last_unstale":"2024-05-08T22:36:26.273413+0000","last_undegraded":"2024-05-08T22:36:26.273413+0000","last_fullsized":"2024-05-08T22:36:26.273413+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:27.013184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.087271+0000","last_change":"2024-05-08T22:36:24.715429+0000","last_active":"2024-05-08T22:36:25.087271+0000","last_peered":"2024-05-08T22:36:25.087271+0000","last_clean":"2024-05-08T22:36:25.087271+0000","last_became_active":"2024-05-08T22:36:24.715082+0000","last_became_peered":"2024-05-08T22:36:24.715082+0000","last_unstale":"2024-05-08T22:36:25.087271+0000","last_undegraded":"2024-05-08T22:36:25.087271+0000","last_fullsized":"2024-05-08T22:36:25.087271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:42.634203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.271234+0000","last_change":"2024-05-08T22:36:25.099537+0000","last_active":"2024-05-08T22:36:26.271234+0000","last_peered":"2024-05-08T22:36:26.271234+0000","last_clean":"2024-05-08T22:36:26.271234+0000","last_became_active":"2024-05-08T22:36:25.099311+0000","last_became_peered":"2024-05-08T22:36:25.099311+0000","last_unstale":"2024-05-08T22:36:26.271234+0000","last_undegraded":"2024-05-08T22:36:26.271234+0000","last_fullsized":"2024-05-08T22:36:26.271234+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:24.620101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.084047+0000","last_change":"2024-05-08T22:36:24.710788+0000","last_active":"2024-05-08T22:36:25.084047+0000","last_peered":"2024-05-08T22:36:25.084047+0000","last_clean":"2024-05-08T22:36:25.084047+0000","last_became_active":"2024-05-08T22:36:24.710408+0000","last_became_peered":"2024-05-08T22:36:24.710408+0000","last_unstale":"2024-05-08T22:36:25.084047+0000","last_undegraded":"2024-05-08T22:36:25.084047+0000","last_fullsized":"2024-05-08T22:36:25.084047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:44.895538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-08T22:36:26.092879+0000","last_change":"2024-05-08T22:36:25.117200+0000","last_active":"2024-05-08T22:36:26.092879+0000","last_peered":"2024-05-08T22:36:26.092879+0000","last_clean":"2024-05-08T22:36:26.092879+0000","last_became_active":"2024-05-08T22:36:25.116658+0000","last_became_peered":"2024-05-08T22:36:25.116658+0000","last_unstale":"2024-05-08T22:36:26.092879+0000","last_undegraded":"2024-05-08T22:36:26.092879+0000","last_fullsized":"2024-05-08T22:36:26.092879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:24.685710+0000","last_clean_scrub_stamp":"2024-05-08T22:36:24.685710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:44.204427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.086617+0000","last_change":"2024-05-08T22:36:24.712890+0000","last_active":"2024-05-08T22:36:25.086617+0000","last_peered":"2024-05-08T22:36:25.086617+0000","last_clean":"2024-05-08T22:36:25.086617+0000","last_became_active":"2024-05-08T22:36:24.712593+0000","last_became_peered":"2024-05-08T22:36:24.712593+0000","last_unstale":"2024-05-08T22:36:25.086617+0000","last_undegraded":"2024-05-08T22:36:25.086617+0000","last_fullsized":"2024-05-08T22:36:25.086617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:36:23.676082+0000","last_clean_scrub_stamp":"2024-05-08T22:36:23.676082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:03.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-05-08T22:36:25.098563+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:36:25.098563+0000","last_peered":"2024-05-08T22:36:25.098563+0000","last_clean":"2024-05-08T22:36:25.098563+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:36:25.098563+0000","last_undegraded":"2024-05-08T22:36:25.098563+0000","last_fullsized":"2024-05-08T22:36:25.098563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":827392,"data_stored":371493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63},"min":{"1min":0.44500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0529999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97099999999999997,"5min":1.101,"15min":1.101},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.51000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0089999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0649999999999999,"5min":1.21,"15min":1.21},"last":1.2170000000000001}]},{"osd":4,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.92100000000000004},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2330000000000001,"5min":1.306,"15min":1.306},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":0.94099999999999995},"min":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.234,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94999999999999996}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97799999999999998,"15min":0.96999999999999997},"min":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.008,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2649999999999999,"5min":1.476,"15min":1.476},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.68600000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3069999999999999,"5min":1.446,"15min":1.446},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2809999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.095,"15min":1.107},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4510000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.151},{"interface":"front","average":{"1min":1,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.347,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.196}]}]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":373821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.62},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.04,"15min":1.04},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.046,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.371,"15min":0.371},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.3280000000000001,"5min":1.431,"15min":1.431},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.96699999999999997},"min":{"1min":0.75,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.226,"5min":1.508,"15min":1.508},"last":1.077}]},{"osd":5,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.66200000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.268,"5min":1.327,"15min":1.327},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0569999999999999,"15min":1.093},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.268,"5min":1.726,"15min":1.726},"last":0.82799999999999996}]},{"osd":6,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.83899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.256,"5min":1.659,"15min":1.659},"last":1.0089999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.4319999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.79900000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81699999999999995}]}]},{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1437696,"data_stored":962637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.032},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.167,"15min":1.167},"last":0.91300000000000003}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90500000000000003,"15min":0.92100000000000004},"min":{"1min":0.69399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1220000000000001,"5min":1.419,"15min":1.419},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.151,"5min":1.337,"15min":1.337},"last":1.177}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0629999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.754,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1619999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.92900000000000005}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.109,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3799999999999999,"5min":1.482,"15min":1.482},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0489999999999999},"min":{"1min":0.77600000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.21,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.091},{"interface":"front","average":{"1min":1.077,"5min":1.071,"15min":1.089},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.96699999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":831488,"data_stored":371527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.141},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.88200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.997,"5min":1.758,"15min":1.758},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.216,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":0.99199999999999999},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.93100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1559999999999999,"5min":1.419,"15min":1.419},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.071,"15min":1.109},"min":{"1min":0.76000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3580000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.97099999999999997,"15min":0.999},"min":{"1min":0.68000000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.216,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.1220000000000001}]},{"osd":4,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4370000000000001,"5min":1.538,"15min":1.538},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.69499999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.407,"5min":1.458,"15min":1.458},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.119,"15min":1.1499999999999999},"min":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.528,"5min":1.681,"15min":1.681},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1519999999999999},"min":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3160000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1679999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0549999999999999,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2509999999999999,"5min":1.835,"15min":1.835},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.218,"15min":1.248},"min":{"1min":0.87,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.595,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.097}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":831488,"data_stored":371515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.53400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.014,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.123,"5min":1.52,"15min":1.52},"last":0.82599999999999996}]},{"osd":1,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81899999999999995,"15min":0.80100000000000005},"min":{"1min":0.68000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1559999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1759999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.57499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.1299999999999999}]},{"osd":3,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.86599999999999999},"min":{"1min":0.60099999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.0449999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97099999999999997,"15min":0.96399999999999997},"min":{"1min":0.82499999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.0580000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.038}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.63900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1739999999999999,"5min":1.304,"15min":1.304},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97899999999999998},"min":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.149,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1930000000000001}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93799999999999994,"15min":0.90600000000000003},"min":{"1min":0.71699999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.605,"15min":1.605},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.90100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0880000000000001,"15min":1.083},"min":{"1min":0.92400000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2330000000000001,"5min":1.538,"15min":1.538},"last":1.236},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0529999999999999,"15min":1.0169999999999999},"min":{"1min":0.90300000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.278,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.075}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":839680,"data_stored":373057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.64800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.94899999999999995}]},{"osd":1,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.872,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1819999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.163},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.67000000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.214,"5min":1.29,"15min":1.29},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.351,"5min":1.637,"15min":1.637},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0469999999999999},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.337,"5min":2.601,"15min":2.601},"last":1.038},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.073,"15min":1.119},"min":{"1min":0.81000000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4259999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.083}]},{"osd":5,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2929999999999999,"5min":1.792,"15min":1.792},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.034},"min":{"1min":0.76800000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3839999999999999,"5min":1.575,"15min":1.575},"last":1.137}]},{"osd":6,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0800000000000001,"15min":1.1000000000000001},"min":{"1min":0.71299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2849999999999999,"5min":1.831,"15min":1.831},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":1.012,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.347,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.1850000000000001}]},{"osd":7,"last update":"Wed May 8 22:36:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.099,"15min":1.1140000000000001},"min":{"1min":0.83399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5169999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1759999999999999,"15min":1.1830000000000001},"min":{"1min":0.85799999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.405,"5min":1.944,"15min":1.944},"last":1.21}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":371493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85299999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.008,"15min":1.008},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59299999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79000000000000004}]},{"osd":4,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90700000000000003},"min":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3029999999999999,"5min":1.363,"15min":1.363},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0069999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.204,"5min":1.282,"15min":1.282},"last":0.90100000000000002}]},{"osd":5,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0289999999999999,"15min":1.0389999999999999},"min":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.347,"5min":1.484,"15min":1.484},"last":0.97399999999999998},{"interface":"front","average":{"1min":1,"5min":1.0149999999999999,"15min":1.02},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2509999999999999,"5min":1.391,"15min":1.391},"last":1.2270000000000001}]},{"osd":6,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.052,"15min":1.0609999999999999},"min":{"1min":0.80200000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0820000000000001},"min":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3120000000000001,"5min":1.673,"15min":1.673},"last":0.92700000000000005}]},{"osd":7,"last update":"Wed May 8 22:36:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0700000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.997},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2849999999999999,"5min":1.409,"15min":1.409},"last":1.052}]}]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343432,"statfs":{"total":96636764160,"available":96607674368,"internally_reserved":0,"allocated":1421312,"data_stored":961861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.012,"5min":1.617,"15min":1.617},"last":0.495},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.446,"15min":1.446},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.032,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed May 8 22:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81399999999999995},"min":{"1min":0.55300000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":1.714,"15min":1.714},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.327,"15min":1.327},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.70699999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2110000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.22,"5min":1.669,"15min":1.669},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70099999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0289999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":1},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0609999999999999,"5min":1.474,"15min":1.474},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95299999999999996,"15min":0.96799999999999997},"min":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1579999999999999,"5min":1.397,"15min":1.397},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0349999999999999,"15min":1.0529999999999999},"min":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1759999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.93500000000000005}]},{"osd":7,"last update":"Wed May 8 22:36:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.85699999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.2909999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.877},{"interface":"front","average":{"1min":0.996,"5min":1.01,"15min":1.022},"min":{"1min":0.85999999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1359999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:36:32.913 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:36:32.914 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:36:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.038+0000 7fed29add700 1 Processor -- start 2024-05-08T22:36:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.038+0000 7fed29add700 1 -- start start 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fed2415d2f0 0x7fed240b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240d1aa0 0x7fed240b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed240d1fe0 con 0x7fed240b4e20 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240b52b0 con 0x7fed2415d2f0 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415d6d0 con 0x7fed240d1aa0 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed23fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fed240b4e20 0x7fed240d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54770/0 (socket says 172.21.15.179:54770) 2024-05-08T22:36:33.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed23fff700 1 -- 172.21.15.179:0/389090859 learned_addr learned my addr 172.21.15.179:0/389090859 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 --2- 172.21.15.179:0/389090859 >> v2:172.21.15.195:3300/0 conn(0x7fed2415d2f0 0x7fed240b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed292dc700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240d1aa0 0x7fed240b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed237fe700 1 -- 172.21.15.179:0/389090859 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717164907 0 0) 0x7fed240d1fe0 con 0x7fed240b4e20 2024-05-08T22:36:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed237fe700 1 -- 172.21.15.179:0/389090859 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed0c0035d0 con 0x7fed240b4e20 2024-05-08T22:36:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 -- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240d1aa0 msgr2=0x7fed240b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240d1aa0 0x7fed240b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 -- 172.21.15.179:0/389090859 >> v1:172.21.15.179:6789/0 conn(0x7fed240b4e20 legacy=0x7fed240d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 -- 172.21.15.179:0/389090859 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240d22c0 con 0x7fed2415d2f0 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed28adb700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed240b48e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed14009850 tx=0x7fed14009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=456cd20bc0f1e7a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed237fe700 1 -- 172.21.15.179:0/389090859 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14011830 con 0x7fed2415d2f0 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed237fe700 1 -- 172.21.15.179:0/389090859 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14011990 con 0x7fed2415d2f0 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed237fe700 1 -- 172.21.15.179:0/389090859 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14011d00 con 0x7fed2415d2f0 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 msgr2=0x7fed240b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed240b48e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed14009850 tx=0x7fed14009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- 172.21.15.179:0/389090859 shutdown_connections 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240d1aa0 0x7fed240b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 --2- 172.21.15.179:0/389090859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed240b48e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- 172.21.15.179:0/389090859 >> 172.21.15.179:0/389090859 conn(0x7fed240cfac0 msgr2=0x7fed240cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- 172.21.15.179:0/389090859 shutdown_connections 2024-05-08T22:36:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.042+0000 7fed29add700 1 -- 172.21.15.179:0/389090859 wait complete. 2024-05-08T22:36:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 Processor -- start 2024-05-08T22:36:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 -- start start 2024-05-08T22:36:33.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 0x7fed240c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed2414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24150330 con 0x7fed240d1aa0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241f36a0 con 0x7fed2415d2f0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed29add700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240c85f0 con 0x7fed240b4e20 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed23fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed23fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55978/0 (socket says 172.21.15.179:55978) 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.046+0000 7fed23fff700 1 -- 172.21.15.179:0/739175290 learned_addr learned my addr 172.21.15.179:0/739175290 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed28adb700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 0x7fed240c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed292dc700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed2414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 -- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 msgr2=0x7fed240c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 0x7fed240c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 -- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 msgr2=0x7fed2414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed2414ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 -- 172.21.15.179:0/739175290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140094b0 con 0x7fed240d1aa0 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed28adb700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 0x7fed240c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:33.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed23fff700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c6b0 tx=0x7fed1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180074e0 con 0x7fed240d1aa0 2024-05-08T22:36:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18007660 con 0x7fed240d1aa0 2024-05-08T22:36:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241515b0 con 0x7fed240d1aa0 2024-05-08T22:36:33.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18005a90 con 0x7fed240d1aa0 2024-05-08T22:36:33.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed24151b70 con 0x7fed240d1aa0 2024-05-08T22:36:33.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.050+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2404e700 con 0x7fed240d1aa0 2024-05-08T22:36:33.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.054+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed18005bf0 con 0x7fed240d1aa0 2024-05-08T22:36:33.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.054+0000 7fed217fa700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 0x7fed10068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.054+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fed18083630 con 0x7fed240d1aa0 2024-05-08T22:36:33.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.054+0000 7fed28adb700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 0x7fed10068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.054+0000 7fed28adb700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 0x7fed10068c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fed140090b0 tx=0x7fed1401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.062+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fed18050940 con 0x7fed240d1aa0 2024-05-08T22:36:33.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.294+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fed240b8520 con 0x7fed240d1aa0 2024-05-08T22:36:33.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.302+0000 7fed217fa700 1 -- 172.21.15.179:0/739175290 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fed18053f60 con 0x7fed240d1aa0 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 msgr2=0x7fed10068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 0x7fed10068c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fed140090b0 tx=0x7fed1401c040 comp rx=0 tx=0).stop 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 msgr2=0x7fed2414e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c6b0 tx=0x7fed1800c9c0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 shutdown_connections 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed10066780 0x7fed10068c40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2415d2f0 0x7fed2414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed240d1aa0 0x7fed2414e920 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 --2- 172.21.15.179:0/739175290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed240b4e20 0x7fed240c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.306+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 >> 172.21.15.179:0/739175290 conn(0x7fed240cfac0 msgr2=0x7fed240d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.310+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 shutdown_connections 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.310+0000 7fed29add700 1 -- 172.21.15.179:0/739175290 wait complete. 2024-05-08T22:36:33.311 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:36:33.326 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:33.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.438+0000 7f3089b05700 1 Processor -- start 2024-05-08T22:36:33.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.438+0000 7f3089b05700 1 -- start start 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.438+0000 7f3089b05700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30841434d0 0x7f308406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.438+0000 7f3089b05700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f308406dc20 0x7f308406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.438+0000 7f3089b05700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30840cd200 con 0x7f30840ce2a0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840cd4e0 con 0x7f308406dc20 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840cd7c0 con 0x7f30841434d0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f308406dc20 0x7f308406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089304700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f30840ce2a0 0x7f30840b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54772/0 (socket says 172.21.15.179:54772) 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089304700 1 -- 172.21.15.179:0/577156050 learned_addr learned my addr 172.21.15.179:0/577156050 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3088b03700 1 --2- 172.21.15.179:0/577156050 >> v2:172.21.15.195:3300/0 conn(0x7f30841434d0 0x7f308406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f30837fe700 1 -- 172.21.15.179:0/577156050 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704178617 0 0) 0x7f30840cd200 con 0x7f30840ce2a0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f30837fe700 1 -- 172.21.15.179:0/577156050 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30700035d0 con 0x7f30840ce2a0 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 -- 172.21.15.179:0/577156050 >> v2:172.21.15.195:3300/0 conn(0x7f30841434d0 msgr2=0x7f308406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 --2- 172.21.15.179:0/577156050 >> v2:172.21.15.195:3300/0 conn(0x7f30841434d0 0x7f308406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 -- 172.21.15.179:0/577156050 >> v1:172.21.15.179:6789/0 conn(0x7f30840ce2a0 legacy=0x7f30840b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 -- 172.21.15.179:0/577156050 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30840cdaa0 con 0x7f308406dc20 2024-05-08T22:36:33.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3083fff700 1 --2- 172.21.15.179:0/577156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f308406dc20 0x7f308406e640 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3084070800 tx=0x7f307400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90362a3b67840f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f30837fe700 1 -- 172.21.15.179:0/577156050 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400e070 con 0x7f308406dc20 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f30837fe700 1 -- 172.21.15.179:0/577156050 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400bb20 con 0x7f308406dc20 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f30837fe700 1 -- 172.21.15.179:0/577156050 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074012430 con 0x7f308406dc20 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- 172.21.15.179:0/577156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f308406dc20 msgr2=0x7f308406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 --2- 172.21.15.179:0/577156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f308406dc20 0x7f308406e640 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3084070800 tx=0x7f307400b330 comp rx=0 tx=0).stop 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- 172.21.15.179:0/577156050 shutdown_connections 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 --2- 172.21.15.179:0/577156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f308406dc20 0x7f308406e640 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 --2- 172.21.15.179:0/577156050 >> v2:172.21.15.195:3300/0 conn(0x7f30841434d0 0x7f308406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- 172.21.15.179:0/577156050 >> 172.21.15.179:0/577156050 conn(0x7f30840ce9d0 msgr2=0x7f30840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- 172.21.15.179:0/577156050 shutdown_connections 2024-05-08T22:36:33.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.442+0000 7f3089b05700 1 -- 172.21.15.179:0/577156050 wait complete. 2024-05-08T22:36:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 Processor -- start 2024-05-08T22:36:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- start start 2024-05-08T22:36:33.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30841434d0 0x7f3084156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3084138480 0x7f3084156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084142040 con 0x7f30840ce2a0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084159be0 con 0x7f3084138480 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3084141a80 con 0x7f30841434d0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55998/0 (socket says 172.21.15.179:55998) 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 -- 172.21.15.179:0/3448541046 learned_addr learned my addr 172.21.15.179:0/3448541046 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3083fff700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30841434d0 0x7f3084156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089304700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3084138480 0x7f3084156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 -- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30841434d0 msgr2=0x7f3084156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30841434d0 0x7f3084156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 -- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3084138480 msgr2=0x7f3084156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3084138480 0x7f3084156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 -- 172.21.15.179:0/3448541046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009a70 con 0x7f30840ce2a0 2024-05-08T22:36:33.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3088b03700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e840 tx=0x7f306c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0095a0 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c013680 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0139f0 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084141d60 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.446+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3084155ef0 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3084067740 con 0x7f30840ce2a0 2024-05-08T22:36:33.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f306c013b50 con 0x7f30840ce2a0 2024-05-08T22:36:33.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f30817fa700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 0x7f3064068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:33.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f306c0841e0 con 0x7f30840ce2a0 2024-05-08T22:36:33.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f3083fff700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 0x7f3064068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:33.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.450+0000 7f3083fff700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 0x7f3064068c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3074014040 tx=0x7f3074011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:33.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.458+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f306c051400 con 0x7f30840ce2a0 2024-05-08T22:36:33.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3084159d20 con 0x7f30840ce2a0 2024-05-08T22:36:33.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f30817fa700 1 -- 172.21.15.179:0/3448541046 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f306c054a20 con 0x7f30840ce2a0 2024-05-08T22:36:33.671 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:33.671 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:36:27.087228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 msgr2=0x7f3064068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 0x7f3064068c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3074014040 tx=0x7f3074011000 comp rx=0 tx=0).stop 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 msgr2=0x7f3084158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.670+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e840 tx=0x7f306c00eb50 comp rx=0 tx=0).stop 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 shutdown_connections 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3064066760 0x7f3064068c20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3084138480 0x7f3084156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30841434d0 0x7f3084156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 --2- 172.21.15.179:0/3448541046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30840ce2a0 0x7f3084158210 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 >> 172.21.15.179:0/3448541046 conn(0x7f30840ce9d0 msgr2=0x7f30840b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 shutdown_connections 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:33.674+0000 7f3089b05700 1 -- 172.21.15.179:0/3448541046 wait complete. 2024-05-08T22:36:33.675 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 30 2024-05-08T22:36:34.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:36:25.077203+0000', 'modified': '2024-05-08T22:36:27.087228+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55718}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55718': {'gid': 55718, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6836/3517137124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3517137124}, {'type': 'v1', 'addr': '172.21.15.195:6836', 'nonce': 3517137124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:36:34.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:36:34.096 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:36:34.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 Processor -- start 2024-05-08T22:36:34.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 -- start start 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58e815cff0 0x7f58e80d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58e80b8b40 0x7f58e80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80d1790 con 0x7f58e80b5e70 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80d1a70 con 0x7f58e815cff0 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ed803700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80d1d50 con 0x7f58e80b8b40 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58e815cff0 0x7f58e80d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58e815cff0 0x7f58e80d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56006/0 (socket says 172.21.15.179:56006) 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 -- 172.21.15.179:0/800104886 learned_addr learned my addr 172.21.15.179:0/800104886 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58e7fff700 1 --2- 172.21.15.179:0/800104886 >> v2:172.21.15.195:3300/0 conn(0x7f58e80b8b40 0x7f58e80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 -- 172.21.15.179:0/800104886 >> v2:172.21.15.195:3300/0 conn(0x7f58e80b8b40 msgr2=0x7f58e80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 --2- 172.21.15.179:0/800104886 >> v2:172.21.15.195:3300/0 conn(0x7f58e80b8b40 0x7f58e80d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 -- 172.21.15.179:0/800104886 >> v1:172.21.15.195:6789/0 conn(0x7f58e80b5e70 legacy=0x7f58e80b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 -- 172.21.15.179:0/800104886 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80d2030 con 0x7f58e815cff0 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58ec801700 1 --2- 172.21.15.179:0/800104886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e815cff0 0x7f58e80d2830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f58e80d3e40 tx=0x7f58d4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dee8322f40fa13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58e77fe700 1 -- 172.21.15.179:0/800104886 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4011830 con 0x7f58e815cff0 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58e77fe700 1 -- 172.21.15.179:0/800104886 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4011990 con 0x7f58e815cff0 2024-05-08T22:36:34.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.222+0000 7f58e77fe700 1 -- 172.21.15.179:0/800104886 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4011d20 con 0x7f58e815cff0 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- 172.21.15.179:0/800104886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e815cff0 msgr2=0x7f58e80d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- 172.21.15.179:0/800104886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e815cff0 0x7f58e80d2830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f58e80d3e40 tx=0x7f58d4009bf0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- 172.21.15.179:0/800104886 shutdown_connections 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- 172.21.15.179:0/800104886 >> v2:172.21.15.195:3300/0 conn(0x7f58e80b8b40 0x7f58e80d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- 172.21.15.179:0/800104886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e815cff0 0x7f58e80d2830 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- 172.21.15.179:0/800104886 >> 172.21.15.179:0/800104886 conn(0x7f58e80d3410 msgr2=0x7f58e80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- 172.21.15.179:0/800104886 shutdown_connections 2024-05-08T22:36:34.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- 172.21.15.179:0/800104886 wait complete. 2024-05-08T22:36:34.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 Processor -- start 2024-05-08T22:36:34.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- start start 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e80b5e70 0x7f58e8154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58e8156ca0 0x7f58e80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e8159050 con 0x7f58e80b8b40 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e815e450 con 0x7f58e80b5e70 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed803700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81592a0 con 0x7f58e8156ca0 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56014/0 (socket says 172.21.15.179:56014) 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ec801700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e80b5e70 0x7f58e8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58e7fff700 1 -- 172.21.15.179:0/1331004164 learned_addr learned my addr 172.21.15.179:0/1331004164 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.226+0000 7f58ed002700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58e8156ca0 0x7f58e80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 -- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58e8156ca0 msgr2=0x7f58e80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58e8156ca0 0x7f58e80c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 -- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e80b5e70 msgr2=0x7f58e8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e80b5e70 0x7f58e8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 -- 172.21.15.179:0/1331004164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009570 con 0x7f58e80b8b40 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e7fff700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e830 tx=0x7f58dc00eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e57fa700 1 -- 172.21.15.179:0/1331004164 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc009560 con 0x7f58e80b8b40 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e57fa700 1 -- 172.21.15.179:0/1331004164 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc013680 con 0x7f58e80b8b40 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58e57fa700 1 -- 172.21.15.179:0/1331004164 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc013a10 con 0x7f58e80b8b40 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e8159580 con 0x7f58e80b8b40 2024-05-08T22:36:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.230+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8152840 con 0x7f58e80b8b40 2024-05-08T22:36:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.234+0000 7f58e57fa700 1 -- 172.21.15.179:0/1331004164 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58dc013b70 con 0x7f58e80b8b40 2024-05-08T22:36:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.234+0000 7f58e57fa700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 0x7f58cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.234+0000 7f58e57fa700 1 -- 172.21.15.179:0/1331004164 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f58dc084200 con 0x7f58e80b8b40 2024-05-08T22:36:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.234+0000 7f58ec801700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 0x7f58cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.234+0000 7f58ec801700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 0x7f58cc068bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f58e80d3df0 tx=0x7f58d4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 Processor -- start 2024-05-08T22:36:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 -- start start 2024-05-08T22:36:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f25470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5e420 0x1f259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e84d80 con 0x1f5db00 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e82510 con 0x1f5e420 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e7de00 con 0x1f54be0 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56018/0 (socket says 172.21.15.179:56018) 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 -- 172.21.15.179:0/1204672908 learned_addr learned my addr 172.21.15.179:0/1204672908 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ed002700 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5e420 0x1f259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58e7fff700 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f25470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 -- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 msgr2=0x1f25470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f25470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 -- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5e420 msgr2=0x1f259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5e420 0x1f259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 -- 172.21.15.179:0/1204672908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cf0960 con 0x1f5db00 2024-05-08T22:36:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58ec801700 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009f40 tx=0x7f58d4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9188a94f8027004e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58d27fc700 1 -- 172.21.15.179:0/1204672908 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4020070 con 0x1f5db00 2024-05-08T22:36:34.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58d27fc700 1 -- 172.21.15.179:0/1204672908 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4011d30 con 0x1f5db00 2024-05-08T22:36:34.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.238+0000 7f58d27fc700 1 -- 172.21.15.179:0/1204672908 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40038c0 con 0x1f5db00 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- 172.21.15.179:0/1204672908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 msgr2=0x1f5dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009f40 tx=0x7f58d4010000 comp rx=0 tx=0).stop 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- 172.21.15.179:0/1204672908 shutdown_connections 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5e420 0x1f259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f25470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- 172.21.15.179:0/1204672908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5db00 0x1f5dee0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- 172.21.15.179:0/1204672908 >> 172.21.15.179:0/1204672908 conn(0x1ca54d0 msgr2=0x1f3f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- 172.21.15.179:0/1204672908 shutdown_connections 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- 172.21.15.179:0/1204672908 wait complete. 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- start start 2024-05-08T22:36:34.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 Processor -- start 2024-05-08T22:36:34.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f66ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e3b730 con 0x1f5e420 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e415b0 con 0x1f5db00 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.242+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d29aa0 con 0x1f54be0 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ed002700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 10 client.0 _ll_register_callbacks cb 0x1e77610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f66ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ed002700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56024/0 (socket says 172.21.15.179:56024) 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ed002700 1 -- 172.21.15.179:0/71644993 learned_addr learned my addr 172.21.15.179:0/71644993 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58e7fff700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 -- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 msgr2=0x1f159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 -- 172.21.15.179:0/71644993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 msgr2=0x1f15f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 -- 172.21.15.179:0/71644993 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e80d2030 con 0x1f54be0 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ed002700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58ec801700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f66ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58d4031000 tx=0x7f58d40047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec4a704c652a751 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3301/0 2024-05-08T22:36:34.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003c10 con 0x1f54be0 2024-05-08T22:36:34.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1cb21e0 con 0x1f54be0 2024-05-08T22:36:34.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mon.2 v2:172.21.15.195:3301/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f58d4003d70 con 0x1f54be0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d4032480 con 0x1f54be0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 10 client.55835 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x1e44220 con 0x1f54be0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mon.2 v2:172.21.15.195:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1cb21e0 con 0x1f54be0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mon.2 v2:172.21.15.195:3301/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f58d40325e0 con 0x1f54be0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 10 client.55835 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 10 client.55835 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 20 client.55835 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f56140 0x1f56520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 4 client.55835 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1cfc460 con 0x1f56140 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58e7fff700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f56140 0x1f56520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58e7fff700 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f56140 0x1f56520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f05873d0bcf5ad1 server_cookie=8affa2cdfe36d3ab in_seq=0 out_seq=0 2024-05-08T22:36:34.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.246+0000 7f58c77fe700 10 client.55835 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58c77fe700 1 -- 172.21.15.179:0/71644993 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1cfc460 con 0x1f56140 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58c77fe700 10 client.55835 handle_command_reply: tid=0 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 client.55835 shutdown 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 msgr2=0x1f66ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f66ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58d4031000 tx=0x7f58d40047e0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 shutdown_connections 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f56140 0x1f56520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f5e420 0x1f15f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 --2- 172.21.15.179:0/71644993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f66ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 >> 172.21.15.179:0/71644993 conn(0x1ca54d0 msgr2=0x1f3ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58c77fe700 0 client.55835 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:34.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 shutdown_connections 2024-05-08T22:36:34.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 1 -- 172.21.15.179:0/71644993 wait complete. 2024-05-08T22:36:34.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.250+0000 7f58f1d14740 20 client.55835 trim_cache size 0 max 16384 2024-05-08T22:36:34.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.270+0000 7f58f1d14740 1 Processor -- start 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.270+0000 7f58f1d14740 1 -- start start 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 0x1f62820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f5db00 0x1f62d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e484f0 con 0x1ca54d0 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eacec0 con 0x1f54be0 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e7d610 con 0x1f5db00 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56028/0 (socket says 172.21.15.179:56028) 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 0x1f62820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58ec801700 1 -- 172.21.15.179:0/3238867605 learned_addr learned my addr 172.21.15.179:0/3238867605 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58ed002700 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f5db00 0x1f62d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 -- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f5db00 msgr2=0x1f62d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f5db00 0x1f62d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 -- 172.21.15.179:0/3238867605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 msgr2=0x1f15d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 -- 172.21.15.179:0/3238867605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e2a450 con 0x1f54be0 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58ec801700 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:34.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58e7fff700 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 0x1f62820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00ee90 tx=0x7f58dc01e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d830b35d69ea201 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58d2ffd700 1 -- 172.21.15.179:0/3238867605 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc082660 con 0x1f54be0 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58d2ffd700 1 -- 172.21.15.179:0/3238867605 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc068e90 con 0x1f54be0 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58d2ffd700 1 -- 172.21.15.179:0/3238867605 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0533f0 con 0x1f54be0 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 msgr2=0x1f62820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 0x1f62820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00ee90 tx=0x7f58dc01e040 comp rx=0 tx=0).stop 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- 172.21.15.179:0/3238867605 shutdown_connections 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f54be0 0x1f62820 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f5db00 0x1f62d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 --2- 172.21.15.179:0/3238867605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- 172.21.15.179:0/3238867605 >> 172.21.15.179:0/3238867605 conn(0x1f3fe90 msgr2=0x1f56140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- 172.21.15.179:0/3238867605 shutdown_connections 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- 172.21.15.179:0/3238867605 wait complete. 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 -- start start 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.274+0000 7f58f1d14740 1 Processor -- start 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f11700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f11c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f161c0 con 0x1ca54d0 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fc9040 con 0x1f5db00 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fc92e0 con 0x1f54be0 2024-05-08T22:36:34.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 10 client.0 _ll_register_callbacks cb 0x7f58d4011630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56034/0 (socket says 172.21.15.179:56034) 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 -- 172.21.15.179:0/2005413338 learned_addr learned my addr 172.21.15.179:0/2005413338 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58e7fff700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f11700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ed002700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f11c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 -- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 msgr2=0x1f11700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f11700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 -- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 msgr2=0x1f11c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f11c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 -- 172.21.15.179:0/2005413338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e44220 con 0x1ca54d0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58ec801700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f58d40092b0 tx=0x7f58d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64df7a7f844a5712 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003e20 con 0x1ca54d0 2024-05-08T22:36:34.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58d4011d30 con 0x1ca54d0 2024-05-08T22:36:34.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f58d4045430 con 0x1ca54d0 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d40116e0 con 0x1ca54d0 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 10 client.6654 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x7f58e8152840 con 0x1ca54d0 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4011d30 con 0x1ca54d0 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f58d40455f0 con 0x1ca54d0 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 10 client.6654 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 10 client.6654 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:36:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 20 client.6654 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f5e420 0x1fd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 4 client.6654 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.278+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1d211e0 con 0x1f5e420 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58e7fff700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f5e420 0x1fd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58c77fe700 10 client.6654 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58e7fff700 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f5e420 0x1fd0dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aaf8eb24cc55698 server_cookie=30df74dff9671e2e in_seq=0 out_seq=0 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58c77fe700 1 -- 172.21.15.179:0/2005413338 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x1d211e0 con 0x1f5e420 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58c77fe700 10 client.6654 handle_command_reply: tid=0 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 client.6654 shutdown 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 msgr2=0x1f15770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f58d40092b0 tx=0x7f58d4011000 comp rx=0 tx=0).stop 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 shutdown_connections 2024-05-08T22:36:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1f5e420 0x1fd0dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f5db00 0x1f11c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58c77fe700 0 client.6654 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f54be0 0x1f11700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 --2- 172.21.15.179:0/2005413338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ca54d0 0x1f15770 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 >> 172.21.15.179:0/2005413338 conn(0x1f3fe90 msgr2=0x1f569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 shutdown_connections 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 1 -- 172.21.15.179:0/2005413338 wait complete. 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58f1d14740 20 client.6654 trim_cache size 0 max 16384 2024-05-08T22:36:34.287 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 55718, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 9.2029450609999994, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 30, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 69, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 69, 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 13.702164743000001 2024-05-08T22:36:34.288 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:34.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 msgr2=0x7f58cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 0x7f58cc068bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f58e80d3df0 tx=0x7f58d4008030 comp rx=0 tx=0).stop 2024-05-08T22:36:34.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 msgr2=0x7f58e8156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e830 tx=0x7f58dc00eb40 comp rx=0 tx=0).stop 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 shutdown_connections 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f58cc066730 0x7f58cc068bf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58e8156ca0 0x7f58e80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e80b8b40 0x7f58e8156760 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 --2- 172.21.15.179:0/1331004164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58e80b5e70 0x7f58e8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 >> 172.21.15.179:0/1331004164 conn(0x7f58e80d3410 msgr2=0x7f58e80b62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 shutdown_connections 2024-05-08T22:36:34.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.282+0000 7f58ed803700 1 -- 172.21.15.179:0/1331004164 wait complete. 2024-05-08T22:36:34.302 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:36:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.414+0000 7fddee95b700 1 Processor -- start 2024-05-08T22:36:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- start start 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdde80ce380 0x7fdde80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde806abb0 con 0x7fdde80cc700 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde806ae90 con 0x7fdde80cb510 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde80cb980 con 0x7fdde80ce380 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded959700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdde80cc700 0x7fdde815b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54776/0 (socket says 172.21.15.179:54776) 2024-05-08T22:36:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded959700 1 -- 172.21.15.179:0/3269061326 learned_addr learned my addr 172.21.15.179:0/3269061326 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:34.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded158700 1 --2- 172.21.15.179:0/3269061326 >> v2:172.21.15.195:3300/0 conn(0x7fdde80ce380 0x7fdde80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddec957700 1 -- 172.21.15.179:0/3269061326 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997572090 0 0) 0x7fdde806abb0 con 0x7fdde80cc700 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddec957700 1 -- 172.21.15.179:0/3269061326 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd40035d0 con 0x7fdde80cc700 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded158700 1 -- 172.21.15.179:0/3269061326 >> v1:172.21.15.195:6789/0 conn(0x7fdde80cb510 legacy=0x7fdde806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded158700 1 -- 172.21.15.179:0/3269061326 >> v1:172.21.15.179:6789/0 conn(0x7fdde80cc700 legacy=0x7fdde815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded158700 1 -- 172.21.15.179:0/3269061326 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80ce800 con 0x7fdde80ce380 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fdded158700 1 --2- 172.21.15.179:0/3269061326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80ce380 0x7fdde80cafd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009850 tx=0x7fdde4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13aafb7c2e1d7bab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddec957700 1 -- 172.21.15.179:0/3269061326 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4011830 con 0x7fdde80ce380 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddec957700 1 -- 172.21.15.179:0/3269061326 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4011990 con 0x7fdde80ce380 2024-05-08T22:36:34.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddec957700 1 -- 172.21.15.179:0/3269061326 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4011d40 con 0x7fdde80ce380 2024-05-08T22:36:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- 172.21.15.179:0/3269061326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80ce380 msgr2=0x7fdde80cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 --2- 172.21.15.179:0/3269061326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80ce380 0x7fdde80cafd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009850 tx=0x7fdde4009b60 comp rx=0 tx=0).stop 2024-05-08T22:36:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 -- 172.21.15.179:0/3269061326 shutdown_connections 2024-05-08T22:36:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.418+0000 7fddee95b700 1 --2- 172.21.15.179:0/3269061326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80ce380 0x7fdde80cafd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- 172.21.15.179:0/3269061326 >> 172.21.15.179:0/3269061326 conn(0x7fdde80ce9d0 msgr2=0x7fdde80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- 172.21.15.179:0/3269061326 shutdown_connections 2024-05-08T22:36:34.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- 172.21.15.179:0/3269061326 wait complete. 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 Processor -- start 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- start start 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdde80cb510 0x7fdde8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde80ce380 0x7fdde80a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8156ea0 con 0x7fdde80ce380 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8157180 con 0x7fdde80cc700 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee95b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8157460 con 0x7fdde80cb510 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fddee15a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde80ce380 0x7fdde80a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fdded158700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fdded158700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:35258/0 (socket says 172.21.15.179:35258) 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fdded158700 1 -- 172.21.15.179:0/897363404 learned_addr learned my addr 172.21.15.179:0/897363404 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.422+0000 7fdded959700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdde80cb510 0x7fdde8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 -- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdde80cb510 msgr2=0x7fdde8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdde80cb510 0x7fdde8157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 -- 172.21.15.179:0/897363404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde80ce380 msgr2=0x7fdde80a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde80ce380 0x7fdde80a8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 -- 172.21.15.179:0/897363404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde40094b0 con 0x7fdde80cc700 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fdded158700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdde4014040 tx=0x7fdde4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4019980 con 0x7fdde80cc700 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde8157740 con 0x7fdde80cc700 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde401e070 con 0x7fdde80cc700 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde81569a0 con 0x7fdde80cc700 2024-05-08T22:36:34.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4019ce0 con 0x7fdde80cc700 2024-05-08T22:36:34.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde8157c60 con 0x7fdde80cc700 2024-05-08T22:36:34.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.426+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdde401a420 con 0x7fdde80cc700 2024-05-08T22:36:34.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.430+0000 7fddde7fc700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 0x7fddcc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:34.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.430+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdde4085380 con 0x7fdde80cc700 2024-05-08T22:36:34.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.430+0000 7fdded959700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 0x7fddcc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:34.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.430+0000 7fdded959700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 0x7fddcc068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:34.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.438+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdde8157c60 con 0x7fdde80cc700 2024-05-08T22:36:34.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.646+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdde8157c60 con 0x7fdde80cc700 2024-05-08T22:36:34.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddde7fc700 1 -- 172.21.15.179:0/897363404 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdde8157c60 con 0x7fdde80cc700 2024-05-08T22:36:34.650 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:36:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 msgr2=0x7fddcc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 0x7fddcc068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fddd8007fd0 tx=0x7fddd8007de0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 msgr2=0x7fdde8151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdde4014040 tx=0x7fdde4004770 comp rx=0 tx=0).stop 2024-05-08T22:36:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 shutdown_connections 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fddcc066730 0x7fddcc068bf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdde80cc700 0x7fdde8151040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde80ce380 0x7fdde80a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 --2- 172.21.15.179:0/897363404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdde80cb510 0x7fdde8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 >> 172.21.15.179:0/897363404 conn(0x7fdde80ce9d0 msgr2=0x7fdde80aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 shutdown_connections 2024-05-08T22:36:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:34.650+0000 7fddee95b700 1 -- 172.21.15.179:0/897363404 wait complete. 2024-05-08T22:36:34.664 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:36:34.664 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:36:34.664 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:36:34.665 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:36:34.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.665 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:36:34.670 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:36:34.672 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:36:34.673 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:34.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.674 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:36:34.687 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:36:34.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.689 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:36:34.697 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:36:34.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:36:34.698 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:36:34.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.776 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:36:34.781 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:36:34.781 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:36:34.781 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:36:34.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.781 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:36:34.793 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:36:34.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:36:34.793 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:34.797 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:36:34.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:36:34.798 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:34.847 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:36:34.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:36:34.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:36:34.875 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:36:34.876 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:36:34.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.877 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:36:34.881 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:36:34.881 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:36:34.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:36:34.900 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:36:34.900 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:36:34.955 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:36:34.956 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:36:34.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:34.957 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:36:34.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:36:34.966+0000 7f2c73926640 -1 init, newargv = 0x559728a32c10 newargc=15 2024-05-08T22:36:34.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[40792]: starting ceph client 2024-05-08T22:36:34.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[40792]: starting fuse 2024-05-08T22:36:36.063 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:36:36.082 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:36:36.083 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:36:36.139 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:36:36.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:36:36.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.141 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:36:36.148 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:36:36.148 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:36:36.148 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:36:36.148 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:36:36.149 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:36:36.150 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:36:36.232 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.40792.asok 2024-05-08T22:36:36.417 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40792.asok 2024-05-08T22:36:36.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40792.asok status 2024-05-08T22:36:36.564 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:36.564 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "40792", 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 6669, 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:36:36.565 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 6669 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 4012217293 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 4012217293 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:36.566 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.6669 192.168.0.1:0/4012217293", 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/4012217293", 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 30, 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 69, 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:36:36.567 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:36.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.574 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:36:36.574 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:36.580 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:36:36.580 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:36.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.580 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:36.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.592 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-08T22:36:36.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:36.655 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-08T22:36:36.669 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:36:36.739 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:36:36.927 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:37.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 Processor -- start 2024-05-08T22:36:37.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- start start 2024-05-08T22:36:37.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 0x7ff9b415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff9b40c9230 0x7ff9b40c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b4158f60 con 0x7ff9b40c8d70 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b406b860 con 0x7ff9b40c9230 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b406ac50 con 0x7ff9b4158b80 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b9bce700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff9b40c8d70 0x7ff9b406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39604/0 (socket says 172.21.15.179:39604) 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b9bce700 1 -- 172.21.15.179:0/4166490756 learned_addr learned my addr 172.21.15.179:0/4166490756 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/4166490756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 0x7ff9b415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9babd0700 1 --2- 172.21.15.179:0/4166490756 >> v2:172.21.15.195:3300/0 conn(0x7ff9b40c9230 0x7ff9b40c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b93cd700 1 -- 172.21.15.179:0/4166490756 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788318573 0 0) 0x7ff9b4158f60 con 0x7ff9b40c8d70 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b93cd700 1 -- 172.21.15.179:0/4166490756 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99c0035d0 con 0x7ff9b40c8d70 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 -- 172.21.15.179:0/4166490756 >> v2:172.21.15.195:3300/0 conn(0x7ff9b40c9230 msgr2=0x7ff9b40c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/4166490756 >> v2:172.21.15.195:3300/0 conn(0x7ff9b40c9230 0x7ff9b40c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 -- 172.21.15.179:0/4166490756 >> v1:172.21.15.179:6789/0 conn(0x7ff9b40c8d70 legacy=0x7ff9b406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 -- 172.21.15.179:0/4166490756 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b406af30 con 0x7ff9b4158b80 2024-05-08T22:36:37.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/4166490756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 0x7ff9b415b4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009d90 tx=0x7ff9a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea8e7502b343198 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b93cd700 1 -- 172.21.15.179:0/4166490756 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400e070 con 0x7ff9b4158b80 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b93cd700 1 -- 172.21.15.179:0/4166490756 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400bb20 con 0x7ff9b4158b80 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9b93cd700 1 -- 172.21.15.179:0/4166490756 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4012430 con 0x7ff9b4158b80 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- 172.21.15.179:0/4166490756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 msgr2=0x7ff9b415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 --2- 172.21.15.179:0/4166490756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 0x7ff9b415b4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009d90 tx=0x7ff9a400b330 comp rx=0 tx=0).stop 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- 172.21.15.179:0/4166490756 shutdown_connections 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 --2- 172.21.15.179:0/4166490756 >> v2:172.21.15.195:3300/0 conn(0x7ff9b40c9230 0x7ff9b40c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 --2- 172.21.15.179:0/4166490756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b4158b80 0x7ff9b415b4d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.050+0000 7ff9bc633700 1 -- 172.21.15.179:0/4166490756 >> 172.21.15.179:0/4166490756 conn(0x7ff9b40cb660 msgr2=0x7ff9b40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- 172.21.15.179:0/4166490756 shutdown_connections 2024-05-08T22:36:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- 172.21.15.179:0/4166490756 wait complete. 2024-05-08T22:36:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 Processor -- start 2024-05-08T22:36:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- start start 2024-05-08T22:36:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9b40c8d70 0x7ff9b40c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b40c9230 0x7ff9b40c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40ce100 con 0x7ff9b4158b80 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40cd4f0 con 0x7ff9b40c8d70 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40cd7d0 con 0x7ff9b40c9230 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46282/0 (socket says 172.21.15.179:46282) 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 -- 172.21.15.179:0/2239275063 learned_addr learned my addr 172.21.15.179:0/2239275063 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9b9bce700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b40c9230 0x7ff9b40c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9b40c8d70 0x7ff9b40c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 -- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b40c9230 msgr2=0x7ff9b40c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b40c9230 0x7ff9b40c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 -- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9b40c8d70 msgr2=0x7ff9b40c1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9b40c8d70 0x7ff9b40c1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 -- 172.21.15.179:0/2239275063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4009a70 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9babd0700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00eb60 tx=0x7ff9ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac009740 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40cdab0 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac009e00 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00ca70 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.054+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b40c2410 con 0x7ff9b4158b80 2024-05-08T22:36:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b4067770 con 0x7ff9b4158b80 2024-05-08T22:36:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9ac00cbd0 con 0x7ff9b4158b80 2024-05-08T22:36:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9b2ffd700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 0x7ff9a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9ac084980 con 0x7ff9b4158b80 2024-05-08T22:36:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 0x7ff9a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.058+0000 7ff9ba3cf700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 0x7ff9a0068c20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4014040 tx=0x7ff9a4011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.066+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9ac051c40 con 0x7ff9b4158b80 2024-05-08T22:36:37.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.274+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9b406f000 con 0x7ff9b4158b80 2024-05-08T22:36:37.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9b2ffd700 1 -- 172.21.15.179:0/2239275063 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4748 (secure 0 0 0) 0x7ff9ac017090 con 0x7ff9b4158b80 2024-05-08T22:36:37.278 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:37.278 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:36:27.087228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 msgr2=0x7ff9a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 0x7ff9a0068c20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4014040 tx=0x7ff9a4011000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 msgr2=0x7ff9b40c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00eb60 tx=0x7ff9ac00ee70 comp rx=0 tx=0).stop 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 shutdown_connections 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a0066760 0x7ff9a0068c20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4158b80 0x7ff9b40c28c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9b40c9230 0x7ff9b40c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 --2- 172.21.15.179:0/2239275063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9b40c8d70 0x7ff9b40c1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 >> 172.21.15.179:0/2239275063 conn(0x7ff9b40cb660 msgr2=0x7ff9b415df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 shutdown_connections 2024-05-08T22:36:37.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.278+0000 7ff9bc633700 1 -- 172.21.15.179:0/2239275063 wait complete. 2024-05-08T22:36:37.284 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 30 2024-05-08T22:36:37.293 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-08T22:36:37.431 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:36:37.432 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:36:37.432 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T22:36:37.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 Processor -- start 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- start start 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da886780 0x5645da884ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da957620 0x5645da955980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645da8152f0 con 0x5645da884750 2024-05-08T22:36:37.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645da7d18e0 con 0x5645da886780 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645da7d1580 con 0x5645da957620 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f73655d1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da886780 0x5645da884ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46286/0 (socket says 172.21.15.179:46286) 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 -- 172.21.15.179:0/2630832689 learned_addr learned my addr 172.21.15.179:0/2630832689 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f73665d3700 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da957620 0x5645da955980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 -- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da957620 msgr2=0x5645da955980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da957620 0x5645da955980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 -- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da886780 msgr2=0x5645da884ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da886780 0x5645da884ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 -- 172.21.15.179:0/2630832689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358005d20 con 0x5645da884750 2024-05-08T22:36:37.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f7365dd2700 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f736000a840 tx=0x7f736000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df55fd8f8fa8294 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f735ffff700 1 -- 172.21.15.179:0/2630832689 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360008690 con 0x5645da884750 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f735ffff700 1 -- 172.21.15.179:0/2630832689 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360008b60 con 0x5645da884750 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f735ffff700 1 -- 172.21.15.179:0/2630832689 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736001e450 con 0x5645da884750 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- 172.21.15.179:0/2630832689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 msgr2=0x5645da8b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f736000a840 tx=0x7f736000d000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- 172.21.15.179:0/2630832689 shutdown_connections 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da886780 0x5645da884ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da957620 0x5645da955980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 --2- 172.21.15.179:0/2630832689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da884750 0x5645da8b2800 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- 172.21.15.179:0/2630832689 >> 172.21.15.179:0/2630832689 conn(0x5645da8af110 msgr2=0x5645da8cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- 172.21.15.179:0/2630832689 shutdown_connections 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.474+0000 7f736829d140 1 -- 172.21.15.179:0/2630832689 wait complete. 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 Processor -- start 2024-05-08T22:36:37.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 -- start start 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da884750 0x5645da932ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da886780 0x5645da8b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 0x5645da8b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645da7a78d0 con 0x5645da957620 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645da882790 con 0x5645da884750 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645da882a30 con 0x5645da886780 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 0x5645da8b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 0x5645da8b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46296/0 (socket says 172.21.15.179:46296) 2024-05-08T22:36:37.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 -- 172.21.15.179:0/3367744139 learned_addr learned my addr 172.21.15.179:0/3367744139 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73655d1700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da886780 0x5645da8b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f7365dd2700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da884750 0x5645da932ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 -- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da886780 msgr2=0x5645da8b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5645da886780 0x5645da8b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 -- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da884750 msgr2=0x5645da932ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645da884750 0x5645da932ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 -- 172.21.15.179:0/3367744139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645da7d9ca0 con 0x5645da957620 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f73665d3700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 0x5645da8b3ce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73580089c0 tx=0x7f7358008cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f735effd700 1 -- 172.21.15.179:0/3367744139 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800f040 con 0x5645da957620 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f735effd700 1 -- 172.21.15.179:0/3367744139 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358010820 con 0x5645da957620 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f735effd700 1 -- 172.21.15.179:0/3367744139 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358010b90 con 0x5645da957620 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.478+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7360005d20 con 0x5645da957620 2024-05-08T22:36:37.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645da7bfb00 con 0x5645da957620 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f735effd700 1 -- 172.21.15.179:0/3367744139 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7358010cf0 con 0x5645da957620 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f735effd700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7348065060 0x7f7348067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f735effd700 1 -- 172.21.15.179:0/3367744139 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7358083680 con 0x5645da957620 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f736829d140 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5645da881fd0 0x5645da93c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69) v8 -- 0x5645da933420 con 0x5645da881fd0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f7365dd2700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7348065060 0x7f7348067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f73655d1700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5645da881fd0 0x5645da93c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f7365dd2700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7348065060 0x7f7348067520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7360020000 tx=0x7f7360008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.482+0000 7f73655d1700 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5645da881fd0 0x5645da93c8a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f73655d1700 1 -- 172.21.15.179:0/3367744139 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f7350003530 con 0x5645da881fd0 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5645da881fd0 msgr2=0x5645da93c8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5645da881fd0 0x5645da93c8a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7348065060 msgr2=0x7f7348067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7348065060 0x7f7348067520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7360020000 tx=0x7f7360008000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 msgr2=0x5645da8b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 --2- 172.21.15.179:0/3367744139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645da957620 0x5645da8b3ce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73580089c0 tx=0x7f7358008cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f73665d3700 1 -- 172.21.15.179:0/3367744139 reap_dead start 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 shutdown_connections 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 >> 172.21.15.179:0/3367744139 conn(0x5645da8af110 msgr2=0x5645da884ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 shutdown_connections 2024-05-08T22:36:37.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.486+0000 7f736829d140 1 -- 172.21.15.179:0/3367744139 wait complete. 2024-05-08T22:36:37.493 INFO:teuthology.orchestra.run.smithi179.stdout:Xfiledir_test 2024-05-08T22:36:37.493 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:37.493 DEBUG:teuthology.orchestra.run.smithi179:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T22:36:37.558 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:37.558 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:36:37.558 INFO:teuthology.orchestra.run.smithi179.stdout: "ancestors": [ 2024-05-08T22:36:37.558 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:37.558 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1099511627776, 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "file", 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 2 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "dirino": 1, 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "dir_test", 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 4 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: ], 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "pool": 10, 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: "old_pools": [] 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:36:37.561 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-08T22:36:37.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 Processor -- start 2024-05-08T22:36:37.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 -- start start 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af0065f10 0x564af005ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 0x564af006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeffca2f0 con 0x564af01122c0 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeff868e0 con 0x564af0065f10 2024-05-08T22:36:37.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044fbee140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564aeff86580 con 0x564af010e6d0 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46322/0 (socket says 172.21.15.179:46322) 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 -- 172.21.15.179:0/3540649292 learned_addr learned my addr 172.21.15.179:0/3540649292 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044d723700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af0065f10 0x564af005ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044cf22700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 0x564af006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 -- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 msgr2=0x564af006aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 0x564af006aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 -- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af0065f10 msgr2=0x564af005ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af0065f10 0x564af005ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 -- 172.21.15.179:0/3540649292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440005d20 con 0x564af01122c0 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044cf22700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 0x564af006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f044df24700 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f044000a440 tx=0x7f044000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca18061ab641102 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.602+0000 7f04477fe700 1 -- 172.21.15.179:0/3540649292 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440008690 con 0x564af01122c0 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f04477fe700 1 -- 172.21.15.179:0/3540649292 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440008b80 con 0x564af01122c0 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f04477fe700 1 -- 172.21.15.179:0/3540649292 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440012410 con 0x564af01122c0 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- 172.21.15.179:0/3540649292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 msgr2=0x564af010ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f044000a440 tx=0x7f044000d000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- 172.21.15.179:0/3540649292 shutdown_connections 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af01122c0 0x564af010ecc0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af010e6d0 0x564af006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 --2- 172.21.15.179:0/3540649292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af0065f10 0x564af005ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- 172.21.15.179:0/3540649292 >> 172.21.15.179:0/3540649292 conn(0x564af00671f0 msgr2=0x564af0038ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- 172.21.15.179:0/3540649292 shutdown_connections 2024-05-08T22:36:37.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- 172.21.15.179:0/3540649292 wait complete. 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 Processor -- start 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.606+0000 7f044fbee140 1 -- start start 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 0x564af010c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af010e6d0 0x564af00fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af01122c0 0x564af010ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aeff5c8d0 con 0x564af0065f10 2024-05-08T22:36:37.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af010b5f0 con 0x564af010e6d0 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564af010b890 con 0x564af01122c0 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 0x564af010c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 0x564af010c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46328/0 (socket says 172.21.15.179:46328) 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 -- 172.21.15.179:0/35291125 learned_addr learned my addr 172.21.15.179:0/35291125 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044df24700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af01122c0 0x564af010ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044cf22700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af010e6d0 0x564af00fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 -- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af01122c0 msgr2=0x564af010ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af01122c0 0x564af010ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 -- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af010e6d0 msgr2=0x564af00fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af010e6d0 0x564af00fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 -- 172.21.15.179:0/35291125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aeff8eca0 con 0x564af0065f10 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044df24700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564af01122c0 0x564af010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044cf22700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564af010e6d0 0x564af00fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:37.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044d723700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 0x564af010c510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f04480086f0 tx=0x7f0448008ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f04467fc700 1 -- 172.21.15.179:0/35291125 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800f040 con 0x564af0065f10 2024-05-08T22:36:37.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f04467fc700 1 -- 172.21.15.179:0/35291125 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448015920 con 0x564af0065f10 2024-05-08T22:36:37.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564aeff74b00 con 0x564af0065f10 2024-05-08T22:36:37.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f04467fc700 1 -- 172.21.15.179:0/35291125 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448015cb0 con 0x564af0065f10 2024-05-08T22:36:37.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564aeff67f40 con 0x564af0065f10 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.610+0000 7f04467fc700 1 -- 172.21.15.179:0/35291125 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f044801e400 con 0x564af0065f10 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f04467fc700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0430065060 0x7f0430067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f04467fc700 1 -- 172.21.15.179:0/35291125 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0448083b30 con 0x564af0065f10 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044cf22700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0430065060 0x7f0430067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044fbee140 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x564af010b170 0x564af01b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) v8 -- 0x564af01b0be0 con 0x564af010b170 2024-05-08T22:36:37.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044df24700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x564af010b170 0x564af01b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044cf22700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0430065060 0x7f0430067520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04380044f0 tx=0x7f0438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.614+0000 7f044df24700 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x564af010b170 0x564af01b06a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044df24700 1 -- 172.21.15.179:0/35291125 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v69'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0440019040 con 0x564af010b170 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x564af010b170 msgr2=0x564af01b06a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x564af010b170 0x564af01b06a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0430065060 msgr2=0x7f0430067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0430065060 0x7f0430067520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04380044f0 tx=0x7f0438008040 comp rx=0 tx=0).stop 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 msgr2=0x564af010c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 --2- 172.21.15.179:0/35291125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564af0065f10 0x564af010c510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f04480086f0 tx=0x7f0448008ab0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044df24700 1 -- 172.21.15.179:0/35291125 reap_dead start 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 shutdown_connections 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 >> 172.21.15.179:0/35291125 conn(0x564af00671f0 msgr2=0x564af010ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 shutdown_connections 2024-05-08T22:36:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.618+0000 7f044fbee140 1 -- 172.21.15.179:0/35291125 wait complete. 2024-05-08T22:36:37.623 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-08T22:36:37.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 Processor -- start 2024-05-08T22:36:37.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 -- start start 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c0d1a10 con 0x7f6b3c15d3c0 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b8fd0 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b44688700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0d1cf0 con 0x7f6b3c0d2ab0 2024-05-08T22:36:37.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b42c25700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53900/0 (socket says 172.21.15.179:53900) 2024-05-08T22:36:37.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 -- 172.21.15.179:0/1358954817 learned_addr learned my addr 172.21.15.179:0/1358954817 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b42424700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6b3c15d3c0 0x7f6b3c0cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:55976/0 (socket says 172.21.15.179:55976) 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 -- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 msgr2=0x7f6b3c0b8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 --2- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c0b8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 -- 172.21.15.179:0/1358954817 >> v1:172.21.15.195:6789/0 conn(0x7f6b3c15d3c0 legacy=0x7f6b3c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 -- 172.21.15.179:0/1358954817 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0d1fd0 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41c23700 1 --2- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009910 tx=0x7f6b38009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc74b380b1abe9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41422700 1 -- 172.21.15.179:0/1358954817 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38011830 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41422700 1 -- 172.21.15.179:0/1358954817 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38011990 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.746+0000 7f6b41422700 1 -- 172.21.15.179:0/1358954817 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38011d40 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 msgr2=0x7f6b3c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 --2- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009910 tx=0x7f6b38009c20 comp rx=0 tx=0).stop 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- 172.21.15.179:0/1358954817 shutdown_connections 2024-05-08T22:36:37.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 --2- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c0b8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 --2- 172.21.15.179:0/1358954817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0d1310 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- 172.21.15.179:0/1358954817 >> 172.21.15.179:0/1358954817 conn(0x7f6b3c0b4ca0 msgr2=0x7f6b3c0b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- 172.21.15.179:0/1358954817 shutdown_connections 2024-05-08T22:36:37.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- 172.21.15.179:0/1358954817 wait complete. 2024-05-08T22:36:37.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 Processor -- start 2024-05-08T22:36:37.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.750+0000 7f6b44688700 1 -- start start 2024-05-08T22:36:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c06a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0c8f70 con 0x7f6b3c14d500 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0c82f0 con 0x7f6b3c0d00a0 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0c85d0 con 0x7f6b3c0d2ab0 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42424700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46340/0 (socket says 172.21.15.179:46340) 2024-05-08T22:36:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 -- 172.21.15.179:0/348555184 learned_addr learned my addr 172.21.15.179:0/348555184 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b41c23700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c06a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 -- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 msgr2=0x7f6b3c06a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c06a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 -- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 msgr2=0x7f6b3c0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 -- 172.21.15.179:0/348555184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38009570 con 0x7f6b3c14d500 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b42c25700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400efc0 tx=0x7f6b3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b2effd700 1 -- 172.21.15.179:0/348555184 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34009ea0 con 0x7f6b3c14d500 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b2effd700 1 -- 172.21.15.179:0/348555184 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3400f040 con 0x7f6b3c14d500 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c0c88b0 con 0x7f6b3c14d500 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b2effd700 1 -- 172.21.15.179:0/348555184 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400ced0 con 0x7f6b3c14d500 2024-05-08T22:36:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.754+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c0c98d0 con 0x7f6b3c14d500 2024-05-08T22:36:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b2effd700 1 -- 172.21.15.179:0/348555184 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b34014480 con 0x7f6b3c14d500 2024-05-08T22:36:37.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b2effd700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 0x7f6b20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b2effd700 1 -- 172.21.15.179:0/348555184 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b34084ab0 con 0x7f6b3c14d500 2024-05-08T22:36:37.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b42424700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 0x7f6b20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b42424700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 0x7f6b20068bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000f60 tx=0x7f6b30007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.758+0000 7f6b47937740 1 Processor -- start 2024-05-08T22:36:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- start start 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13b05b0 0x1392ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13aa1c0 0x1393000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x12d6eb0 con 0x13afc90 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12d4640 con 0x13b05b0 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12cff30 con 0x13aa1c0 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46350/0 (socket says 172.21.15.179:46350) 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 -- 172.21.15.179:0/1875700772 learned_addr learned my addr 172.21.15.179:0/1875700772 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b41c23700 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13b05b0 0x1392ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42c25700 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13aa1c0 0x1393000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 -- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13aa1c0 msgr2=0x1393000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13aa1c0 0x1393000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 -- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13b05b0 msgr2=0x1392ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13b05b0 0x1392ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 -- 172.21.15.179:0/1875700772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12f6950 con 0x13afc90 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b42424700 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6b300155e0 tx=0x7f6b300078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93a7b9078267b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b1ffff700 1 -- 172.21.15.179:0/1875700772 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30015b50 con 0x13afc90 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b1ffff700 1 -- 172.21.15.179:0/1875700772 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30015cb0 con 0x13afc90 2024-05-08T22:36:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b1ffff700 1 -- 172.21.15.179:0/1875700772 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30021650 con 0x13afc90 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- 172.21.15.179:0/1875700772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 msgr2=0x13b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6b300155e0 tx=0x7f6b300078b0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- 172.21.15.179:0/1875700772 shutdown_connections 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13b05b0 0x1392ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13aa1c0 0x1393000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 --2- 172.21.15.179:0/1875700772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13afc90 0x13b0070 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- 172.21.15.179:0/1875700772 >> 172.21.15.179:0/1875700772 conn(0x10f7700 msgr2=0x1391890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- 172.21.15.179:0/1875700772 shutdown_connections 2024-05-08T22:36:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- 172.21.15.179:0/1875700772 wait complete. 2024-05-08T22:36:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 -- start start 2024-05-08T22:36:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.762+0000 7f6b47937740 1 Processor -- start 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x138ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x13aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 0x13aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1293740 con 0x13b05b0 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12933e0 con 0x13aa1c0 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x117bcd0 con 0x13afc90 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b41c23700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x13aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 0x13aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b41c23700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x13aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:55928/0 (socket says 172.21.15.179:55928) 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42424700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x138ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b41c23700 1 -- 172.21.15.179:0/1411588077 learned_addr learned my addr 172.21.15.179:0/1411588077 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 10 client.0 _ll_register_callbacks cb 0x1321ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 -- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 msgr2=0x13aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x13aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 -- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 msgr2=0x138ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x138ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 -- 172.21.15.179:0/1411588077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c0d1a10 con 0x13b05b0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b42c25700 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 0x13aef40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6b34082e90 tx=0x7f6b34055f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8853d8d850358dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3405daf0 con 0x13b05b0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b3405dd80 con 0x13b05b0 2024-05-08T22:36:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1104410 con 0x13b05b0 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b3406d0d0 con 0x13b05b0 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 10 client.6702 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x1142b90 con 0x13b05b0 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.766+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34071070 con 0x13b05b0 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6b3406d230 con 0x13b05b0 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 10 client.6702 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 10 client.6702 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 20 client.6702 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x136af40 0x13aba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 4 client.6702 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x114e690 con 0x136af40 2024-05-08T22:36:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b42424700 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x136af40 0x13aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b1d7fa700 10 client.6702 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b42424700 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x136af40 0x13aba50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87046cefcb0861c4 server_cookie=3a7e8728a7bdcc27 in_seq=0 out_seq=0 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/1411588077 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x114e690 con 0x136af40 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b1d7fa700 10 client.6702 handle_command_reply: tid=0 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 client.6702 shutdown 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 msgr2=0x13aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 0x13aef40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6b34082e90 tx=0x7f6b34055f80 comp rx=0 tx=0).stop 2024-05-08T22:36:37.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 shutdown_connections 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x136af40 0x13aba50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13b05b0 0x13aef40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b1d7fa700 0 client.6702 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x13aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 --2- 172.21.15.179:0/1411588077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x138ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 >> 172.21.15.179:0/1411588077 conn(0x10f7700 msgr2=0x13903f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 shutdown_connections 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 1 -- 172.21.15.179:0/1411588077 wait complete. 2024-05-08T22:36:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.770+0000 7f6b47937740 20 client.6702 trim_cache size 0 max 16384 2024-05-08T22:36:37.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 Processor -- start 2024-05-08T22:36:37.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 -- start start 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x136af40 0x13a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 0x13a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x135fb70 con 0x13aa1c0 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x129ac00 con 0x13afc90 2024-05-08T22:36:37.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12cf740 con 0x136af40 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46370/0 (socket says 172.21.15.179:46370) 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 -- 172.21.15.179:0/846479096 learned_addr learned my addr 172.21.15.179:0/846479096 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b42424700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x136af40 0x13a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b42c25700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 0x13a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 -- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x136af40 msgr2=0x13a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x136af40 0x13a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 -- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 msgr2=0x13a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 0x13a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 -- 172.21.15.179:0/846479096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0c7960 con 0x13aa1c0 2024-05-08T22:36:37.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b42c25700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 0x13a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:37.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b41c23700 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6b38011870 tx=0x7f6b38010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8055c3b37bf179ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.794+0000 7f6b2cff9700 1 -- 172.21.15.179:0/846479096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38004040 con 0x13aa1c0 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b2cff9700 1 -- 172.21.15.179:0/846479096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38021050 con 0x13aa1c0 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b2cff9700 1 -- 172.21.15.179:0/846479096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380043c0 con 0x13aa1c0 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- 172.21.15.179:0/846479096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 msgr2=0x13a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6b38011870 tx=0x7f6b38010000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- 172.21.15.179:0/846479096 shutdown_connections 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13afc90 0x13a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x13aa1c0 0x13a89b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 --2- 172.21.15.179:0/846479096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x136af40 0x13a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- 172.21.15.179:0/846479096 >> 172.21.15.179:0/846479096 conn(0x10f7700 msgr2=0x138d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- 172.21.15.179:0/846479096 shutdown_connections 2024-05-08T22:36:37.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- 172.21.15.179:0/846479096 wait complete. 2024-05-08T22:36:37.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 -- start start 2024-05-08T22:36:37.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.798+0000 7f6b47937740 1 Processor -- start 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x1369a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x1368c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1369fc0 con 0x136af40 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1368660 con 0x13aa1c0 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x13ae200 con 0x13afc90 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 10 client.0 _ll_register_callbacks cb 0x1289df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46380/0 (socket says 172.21.15.179:46380) 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 -- 172.21.15.179:0/3191087595 learned_addr learned my addr 172.21.15.179:0/3191087595 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42c25700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x1368c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b41c23700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x1369a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 -- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 msgr2=0x1368c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x1368c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 -- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 msgr2=0x1369a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x1369a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 -- 172.21.15.179:0/3191087595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1142b90 con 0x136af40 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b41c23700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x1369a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b42424700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000fb0 tx=0x7f6b30000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e899e0b9fa8e8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30028030 con 0x136af40 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b3002f6f0 con 0x136af40 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13919a0 con 0x136af40 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b3002f8b0 con 0x136af40 2024-05-08T22:36:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 10 client.6708 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x7f6b3c0d1fd0 con 0x136af40 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13919a0 con 0x136af40 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6b30021450 con 0x136af40 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 10 client.6708 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 10 client.6708 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 20 client.6708 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x13b05b0 0x14113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 4 client.6708 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-08T22:36:37.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x7f6b30015cb0 con 0x13b05b0 2024-05-08T22:36:37.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b41c23700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x13b05b0 0x14113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b1d7fa700 10 client.6708 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.802+0000 7f6b41c23700 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x13b05b0 0x14113e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609f7e638bc91e9e server_cookie=a3499fd76015edd4 in_seq=0 out_seq=0 2024-05-08T22:36:37.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b1d7fa700 1 -- 172.21.15.179:0/3191087595 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f6b30015cb0 con 0x13b05b0 2024-05-08T22:36:37.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b1d7fa700 10 client.6708 handle_command_reply: tid=0 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 client.6708 shutdown 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 msgr2=0x14104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000fb0 tx=0x7f6b30000ed0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 shutdown_connections 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x13b05b0 0x14113e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x13aa1c0 0x1369a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x13afc90 0x1368c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 --2- 172.21.15.179:0/3191087595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x136af40 0x14104b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 >> 172.21.15.179:0/3191087595 conn(0x10f7700 msgr2=0x13bf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b1d7fa700 0 client.6708 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 shutdown_connections 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 1 -- 172.21.15.179:0/3191087595 wait complete. 2024-05-08T22:36:37.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b47937740 20 client.6708 trim_cache size 0 max 16384 2024-05-08T22:36:37.810 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:37.810 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:36:37.810 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "3bc95fd4-1f2b-4df6-ac96-499930be287b", 2024-05-08T22:36:37.810 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:36:37.810 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 msgr2=0x7f6b20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 0x7f6b20068bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000f60 tx=0x7f6b30007ef0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 msgr2=0x7f6b3c06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400efc0 tx=0x7f6b3400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 shutdown_connections 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b20066730 0x7f6b20068bf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3c14d500 0x7f6b3c06ab00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b3c0d2ab0 0x7f6b3c06a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 --2- 172.21.15.179:0/348555184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b3c0d00a0 0x7f6b3c0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 >> 172.21.15.179:0/348555184 conn(0x7f6b3c0b4ca0 msgr2=0x7f6b3c15c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 shutdown_connections 2024-05-08T22:36:37.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.806+0000 7f6b44688700 1 -- 172.21.15.179:0/348555184 wait complete. 2024-05-08T22:36:37.812 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-05-08T22:36:37.801+0000 7f5f0dd38700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6669=0-4194304@1} caps={6669=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6669 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56070a091b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:36:37.826 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T22:36:37.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 Processor -- start 2024-05-08T22:36:37.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 -- start start 2024-05-08T22:36:37.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d80cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 0x7fe5d806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d80cbfd0 con 0x7fe5d81448b0 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80cd970 con 0x7fe5d80cc9a0 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80cb3c0 con 0x7fe5d815c0b0 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d659c700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fe5d81448b0 0x7fe5d80ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:55990/0 (socket says 172.21.15.179:55990) 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d659c700 1 -- 172.21.15.179:0/4144682694 learned_addr learned my addr 172.21.15.179:0/4144682694 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/4144682694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d80cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.950+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/4144682694 >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 0x7fe5d806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d559a700 1 -- 172.21.15.179:0/4144682694 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275856448 0 0) 0x7fe5d80cbfd0 con 0x7fe5d81448b0 2024-05-08T22:36:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d559a700 1 -- 172.21.15.179:0/4144682694 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c00035d0 con 0x7fe5d81448b0 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/4144682694 >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 msgr2=0x7fe5d806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/4144682694 >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 0x7fe5d806bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/4144682694 >> v1:172.21.15.195:6789/0 conn(0x7fe5d81448b0 legacy=0x7fe5d80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/4144682694 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80cb6a0 con 0x7fe5d815c0b0 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/4144682694 >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 0x7fe5d806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/4144682694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d80cc460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009d90 tx=0x7fe5c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9a31cf7811ee32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d559a700 1 -- 172.21.15.179:0/4144682694 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800e070 con 0x7fe5d815c0b0 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d559a700 1 -- 172.21.15.179:0/4144682694 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c800bb20 con 0x7fe5d815c0b0 2024-05-08T22:36:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d559a700 1 -- 172.21.15.179:0/4144682694 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012430 con 0x7fe5d815c0b0 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 -- 172.21.15.179:0/4144682694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 msgr2=0x7fe5d80cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 --2- 172.21.15.179:0/4144682694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d80cc460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009d90 tx=0x7fe5c800b330 comp rx=0 tx=0).stop 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 -- 172.21.15.179:0/4144682694 shutdown_connections 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 --2- 172.21.15.179:0/4144682694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d80cc460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 --2- 172.21.15.179:0/4144682694 >> v2:172.21.15.179:3300/0 conn(0x7fe5d80cc9a0 0x7fe5d806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 -- 172.21.15.179:0/4144682694 >> 172.21.15.179:0/4144682694 conn(0x7fe5d8070d80 msgr2=0x7fe5d8071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 -- 172.21.15.179:0/4144682694 shutdown_connections 2024-05-08T22:36:37.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.954+0000 7fe5d759e700 1 -- 172.21.15.179:0/4144682694 wait complete. 2024-05-08T22:36:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 Processor -- start 2024-05-08T22:36:37.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 -- start start 2024-05-08T22:36:37.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d80cc9a0 0x7fe5d813f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d8142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8156580 con 0x7fe5d81448b0 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8142e00 con 0x7fe5d80cc9a0 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.958+0000 7fe5d759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d81430e0 con 0x7fe5d815c0b0 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46398/0 (socket says 172.21.15.179:46398) 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/2805393513 learned_addr learned my addr 172.21.15.179:0/2805393513 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d80cc9a0 0x7fe5d813f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d8142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 msgr2=0x7fe5d8142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d8142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d80cc9a0 msgr2=0x7fe5d813f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d80cc9a0 0x7fe5d813f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/2805393513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00094b0 con 0x7fe5d81448b0 2024-05-08T22:36:37.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8014040 tx=0x7fe5c8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5c6ffd700 1 -- 172.21.15.179:0/2805393513 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012430 con 0x7fe5d81448b0 2024-05-08T22:36:37.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5c6ffd700 1 -- 172.21.15.179:0/2805393513 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8020070 con 0x7fe5d81448b0 2024-05-08T22:36:37.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c8009a70 con 0x7fe5d81448b0 2024-05-08T22:36:37.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5c6ffd700 1 -- 172.21.15.179:0/2805393513 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004280 con 0x7fe5d81448b0 2024-05-08T22:36:37.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.962+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d8156a70 con 0x7fe5d81448b0 2024-05-08T22:36:37.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5c6ffd700 1 -- 172.21.15.179:0/2805393513 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5c80043e0 con 0x7fe5d81448b0 2024-05-08T22:36:37.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5c6ffd700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 0x7fe5b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5c6ffd700 1 -- 172.21.15.179:0/2805393513 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe5c8022070 con 0x7fe5d81448b0 2024-05-08T22:36:37.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 0x7fe5b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 0x7fe5b8068bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5dfec1740 1 Processor -- start 2024-05-08T22:36:37.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5dfec1740 1 -- start start 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.966+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x196a5d0 0x1941f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1942460 0x193e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18d0010 con 0x193d840 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x185df40 con 0x196a5d0 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18b0e30 con 0x1942460 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46402/0 (socket says 172.21.15.179:46402) 2024-05-08T22:36:37.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 -- 172.21.15.179:0/2479184552 learned_addr learned my addr 172.21.15.179:0/2479184552 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x196a5d0 0x1941f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1942460 0x193e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 -- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1942460 msgr2=0x193e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1942460 0x193e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 -- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x196a5d0 msgr2=0x1941f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x196a5d0 0x1941f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 -- 172.21.15.179:0/2479184552 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x171ca40 con 0x193d840 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007970 tx=0x7fe5cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6311c784bc3e27aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5b7fff700 1 -- 172.21.15.179:0/2479184552 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0062c0 con 0x193d840 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5b7fff700 1 -- 172.21.15.179:0/2479184552 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc006420 con 0x193d840 2024-05-08T22:36:37.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5b7fff700 1 -- 172.21.15.179:0/2479184552 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0067b0 con 0x193d840 2024-05-08T22:36:37.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2479184552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 msgr2=0x193dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007970 tx=0x7fe5cc012000 comp rx=0 tx=0).stop 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2479184552 shutdown_connections 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x196a5d0 0x1941f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1942460 0x193e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2479184552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x193dc20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2479184552 >> 172.21.15.179:0/2479184552 conn(0x16d15b0 msgr2=0x1969ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2479184552 shutdown_connections 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2479184552 wait complete. 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 -- start start 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.970+0000 7fe5dfec1740 1 Processor -- start 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x198a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x1963150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18ac150 con 0x193d840 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a9eb0 con 0x1961c20 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18678e0 con 0x196a5d0 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 10 client.0 _ll_register_callbacks cb 0x1846670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46406/0 (socket says 172.21.15.179:46406) 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 -- 172.21.15.179:0/2026208335 learned_addr learned my addr 172.21.15.179:0/2026208335 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x1963150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x198a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 -- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 msgr2=0x198a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x198a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 -- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 msgr2=0x1963150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x1963150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 -- 172.21.15.179:0/2026208335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c00035d0 con 0x193d840 2024-05-08T22:36:37.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006aa0 tx=0x7fe5cc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a809474d5297e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:37.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:37.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0062c0 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16de2c0 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe5cc01b050 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5cc006bf0 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 10 client.6720 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x1870290 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16de2c0 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe5cc012070 con 0x193d840 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 10 client.6720 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 10 client.6720 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 20 client.6720 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x193f950 0x193fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 4 client.6720 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:37.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1728540 con 0x193f950 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x193f950 0x193fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5b57fa700 10 client.6720 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.974+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x193f950 0x193fd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8e2f046f2a4849 server_cookie=41be2c4851a2eed8 in_seq=0 out_seq=0 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2026208335 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1728540 con 0x193f950 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5b57fa700 10 client.6720 handle_command_reply: tid=0 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 client.6720 shutdown 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 msgr2=0x1964130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006aa0 tx=0x7fe5cc000ed0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 shutdown_connections 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x193f950 0x193fd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x1963150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x198a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2026208335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x193d840 0x1964130 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:37.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5b57fa700 0 client.6720 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:37.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 >> 172.21.15.179:0/2026208335 conn(0x16d15b0 msgr2=0x193c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:37.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 shutdown_connections 2024-05-08T22:36:37.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2026208335 wait complete. 2024-05-08T22:36:37.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.978+0000 7fe5dfec1740 20 client.6720 trim_cache size 0 max 16384 2024-05-08T22:36:37.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 Processor -- start 2024-05-08T22:36:37.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- start start 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x193d840 0x193e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x19642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1755b80 con 0x1961c20 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1833a50 con 0x193d840 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x184ebd0 con 0x196a5d0 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46418/0 (socket says 172.21.15.179:46418) 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/3200348231 learned_addr learned my addr 172.21.15.179:0/3200348231 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x193d840 0x193e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x19642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 msgr2=0x19642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x19642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x193d840 msgr2=0x193e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x193d840 0x193e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 -- 172.21.15.179:0/3200348231 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18564c0 con 0x1961c20 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b680 tx=0x7fe5c801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9603f3f8e2ff5666 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5c4ff9700 1 -- 172.21.15.179:0/3200348231 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8003e30 con 0x1961c20 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5c4ff9700 1 -- 172.21.15.179:0/3200348231 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c80578b0 con 0x1961c20 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5c4ff9700 1 -- 172.21.15.179:0/3200348231 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80571b0 con 0x1961c20 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- 172.21.15.179:0/3200348231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 msgr2=0x1963d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b680 tx=0x7fe5c801e040 comp rx=0 tx=0).stop 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- 172.21.15.179:0/3200348231 shutdown_connections 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x196a5d0 0x19642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1961c20 0x1963d60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/3200348231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x193d840 0x193e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- 172.21.15.179:0/3200348231 >> 172.21.15.179:0/3200348231 conn(0x16d15b0 msgr2=0x193fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- 172.21.15.179:0/3200348231 shutdown_connections 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:37.998+0000 7fe5dfec1740 1 -- 172.21.15.179:0/3200348231 wait complete. 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 -- start start 2024-05-08T22:36:38.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 Processor -- start 2024-05-08T22:36:38.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x193d840 0x19f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x19895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x198a480 con 0x196a5d0 2024-05-08T22:36:38.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x19f08e0 con 0x1961c20 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1992ba0 con 0x193d840 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 10 client.0 _ll_register_callbacks cb 0x18516f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46430/0 (socket says 172.21.15.179:46430) 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 -- 172.21.15.179:0/2049222660 learned_addr learned my addr 172.21.15.179:0/2049222660 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d5d9b700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x19895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x193d840 0x19f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 -- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x193d840 msgr2=0x19f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x193d840 0x19f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 -- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 msgr2=0x19895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x19895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 -- 172.21.15.179:0/2049222660 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d8156a70 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5d6d9d700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0003970 tx=0x7fe5d0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84d574e5a0c3906 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d001b070 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe5d0016760 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.002+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5cc006420 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d0016920 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 10 client.6726 fetch_fsmap learned FSMap version 30 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x184a790 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc006420 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe5d0011050 con 0x196a5d0 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 10 client.6726 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T22:36:38.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 10 client.6726 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 20 client.6726 populate_metadata read hostname 'smithi179' 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x19a5c00 0x199d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 4 client.6726 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x174d2c0 con 0x19a5c00 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x19a5c00 0x199d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 10 client.6726 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d659c700 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x19a5c00 0x199d200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83db8705f38f6c88 server_cookie=da883a95615ee05d in_seq=0 out_seq=0 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 1 -- 172.21.15.179:0/2049222660 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x174d2c0 con 0x19a5c00 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 10 client.6726 handle_command_reply: tid=0 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 client.6726 shutdown 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 msgr2=0x1989b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0003970 tx=0x7fe5d0003c80 comp rx=0 tx=0).stop 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 shutdown_connections 2024-05-08T22:36:38.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x19a5c00 0x199d200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1961c20 0x19895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5b57fa700 0 client.6726 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x196a5d0 0x1989b10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 --2- 172.21.15.179:0/2049222660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x193d840 0x19f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 >> 172.21.15.179:0/2049222660 conn(0x16d15b0 msgr2=0x1963c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 shutdown_connections 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 1 -- 172.21.15.179:0/2049222660 wait complete. 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5dfec1740 20 client.6726 trim_cache size 0 max 16384 2024-05-08T22:36:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "3bc95fd4-1f2b-4df6-ac96-499930be287b": { 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/", 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "3bc95fd4-1f2b-4df6-ac96-499930be287b", 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive" 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:36:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:36:38.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 msgr2=0x7fe5b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:38.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 0x7fe5b8068bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc007de0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 msgr2=0x7fe5d8158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8014040 tx=0x7fe5c8004af0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 shutdown_connections 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe5b8066730 0x7fe5b8068bf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.006+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d815c0b0 0x7fe5d8142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.010+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5d81448b0 0x7fe5d8158670 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.010+0000 7fe5d759e700 1 --2- 172.21.15.179:0/2805393513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d80cc9a0 0x7fe5d813f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.010+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 >> 172.21.15.179:0/2805393513 conn(0x7fe5d8070d80 msgr2=0x7fe5d81439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.010+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 shutdown_connections 2024-05-08T22:36:38.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:36:38.010+0000 7fe5d759e700 1 -- 172.21.15.179:0/2805393513 wait complete. 2024-05-08T22:36:38.028 INFO:tasks.cephfs.filesystem:scrub status for tag:3bc95fd4-1f2b-4df6-ac96-499930be287b - {'path': '/', 'tag': '3bc95fd4-1f2b-4df6-ac96-499930be287b', 'options': 'recursive'} 2024-05-08T22:36:41.135 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:36:41.134+0000 7fce8f998700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T22:36:51.953 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:36:51.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:37:08.029 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T22:37:08.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 Processor -- start 2024-05-08T22:37:08.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 -- start start 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2bd80cbca0 0x7f2bd80ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2bd80afa50 0x7f2bd80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80d0a00 con 0x7f2bd815ae20 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80b02f0 con 0x7f2bd80cbca0 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bddc57700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80b0590 con 0x7f2bd80afa50 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2bd80cbca0 0x7f2bd80ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2bd80cbca0 0x7f2bd80ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52694/0 (socket says 172.21.15.179:52694) 2024-05-08T22:37:08.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 -- 172.21.15.179:0/796894512 learned_addr learned my addr 172.21.15.179:0/796894512 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bdd456700 1 --2- 172.21.15.179:0/796894512 >> v2:172.21.15.195:3300/0 conn(0x7f2bd80afa50 0x7f2bd80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 -- 172.21.15.179:0/796894512 >> v2:172.21.15.195:3300/0 conn(0x7f2bd80afa50 msgr2=0x7f2bd80b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/796894512 >> v2:172.21.15.195:3300/0 conn(0x7f2bd80afa50 0x7f2bd80b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 -- 172.21.15.179:0/796894512 >> v1:172.21.15.195:6789/0 conn(0x7f2bd815ae20 legacy=0x7f2bd80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 -- 172.21.15.179:0/796894512 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80cf480 con 0x7f2bd80cbca0 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/796894512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80ced70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80cc9d0 tx=0x7f2bc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fff3f471d2bacb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd77fe700 1 -- 172.21.15.179:0/796894512 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800b8a0 con 0x7f2bd80cbca0 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd77fe700 1 -- 172.21.15.179:0/796894512 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800ba00 con 0x7f2bd80cbca0 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.154+0000 7f2bd77fe700 1 -- 172.21.15.179:0/796894512 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800bd90 con 0x7f2bd80cbca0 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- 172.21.15.179:0/796894512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd80cbca0 msgr2=0x7f2bd80ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- 172.21.15.179:0/796894512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80ced70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80cc9d0 tx=0x7f2bc8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- 172.21.15.179:0/796894512 shutdown_connections 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- 172.21.15.179:0/796894512 >> v2:172.21.15.195:3300/0 conn(0x7f2bd80afa50 0x7f2bd80b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- 172.21.15.179:0/796894512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80ced70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- 172.21.15.179:0/796894512 >> 172.21.15.179:0/796894512 conn(0x7f2bd80b2820 msgr2=0x7f2bd80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- 172.21.15.179:0/796894512 shutdown_connections 2024-05-08T22:37:08.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- 172.21.15.179:0/796894512 wait complete. 2024-05-08T22:37:08.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 Processor -- start 2024-05-08T22:37:08.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- start start 2024-05-08T22:37:08.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80afa50 0x7f2bd80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80c70d0 con 0x7f2bd815ae20 2024-05-08T22:37:08.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8159d30 con 0x7f2bd80cbca0 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.158+0000 7f2bddc57700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80c8bf0 con 0x7f2bd80afa50 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80afa50 0x7f2bd80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52704/0 (socket says 172.21.15.179:52704) 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 -- 172.21.15.179:0/1491998364 learned_addr learned my addr 172.21.15.179:0/1491998364 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 -- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80afa50 msgr2=0x7f2bd80c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80afa50 0x7f2bd80c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 -- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd80cbca0 msgr2=0x7f2bd80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 -- 172.21.15.179:0/1491998364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8009570 con 0x7f2bd815ae20 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bdd456700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007e60 tx=0x7f2bd000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd57fa700 1 -- 172.21.15.179:0/1491998364 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000ed30 con 0x7f2bd815ae20 2024-05-08T22:37:08.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd57fa700 1 -- 172.21.15.179:0/1491998364 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0014420 con 0x7f2bd815ae20 2024-05-08T22:37:08.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80c8ed0 con 0x7f2bd815ae20 2024-05-08T22:37:08.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd57fa700 1 -- 172.21.15.179:0/1491998364 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00147d0 con 0x7f2bd815ae20 2024-05-08T22:37:08.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80c9490 con 0x7f2bd815ae20 2024-05-08T22:37:08.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd57fa700 1 -- 172.21.15.179:0/1491998364 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bd0014930 con 0x7f2bd815ae20 2024-05-08T22:37:08.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.162+0000 7f2bd57fa700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 0x7f2bc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.166+0000 7f2bd57fa700 1 -- 172.21.15.179:0/1491998364 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2bd00849b0 con 0x7f2bd815ae20 2024-05-08T22:37:08.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.166+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 0x7f2bc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.166+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 0x7f2bc4068bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000f60 tx=0x7f2bcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.166+0000 7f2be2168740 1 Processor -- start 2024-05-08T22:37:08.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.166+0000 7f2be2168740 1 -- start start 2024-05-08T22:37:08.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x2812d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x283a5d0 0x280e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x27a0010 con 0x28376f0 2024-05-08T22:37:08.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x272df40 con 0x280d840 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2780e30 con 0x283a5d0 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x283a5d0 0x280e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52714/0 (socket says 172.21.15.179:52714) 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x2812d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 -- 172.21.15.179:0/2449350102 learned_addr learned my addr 172.21.15.179:0/2449350102 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 -- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x283a5d0 msgr2=0x280e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x283a5d0 0x280e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 -- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 msgr2=0x2812d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x2812d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 -- 172.21.15.179:0/2449350102 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25eca40 con 0x28376f0 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009c30 tx=0x7f2bc8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9893d29bbf831d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bbe7fc700 1 -- 172.21.15.179:0/2449350102 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8020070 con 0x28376f0 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bbe7fc700 1 -- 172.21.15.179:0/2449350102 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800bbf0 con 0x28376f0 2024-05-08T22:37:08.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2bbe7fc700 1 -- 172.21.15.179:0/2449350102 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8004280 con 0x28376f0 2024-05-08T22:37:08.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- 172.21.15.179:0/2449350102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 msgr2=0x280dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009c30 tx=0x7f2bc8010000 comp rx=0 tx=0).stop 2024-05-08T22:37:08.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- 172.21.15.179:0/2449350102 shutdown_connections 2024-05-08T22:37:08.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x283a5d0 0x280e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x280dc20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 --2- 172.21.15.179:0/2449350102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x2812d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- 172.21.15.179:0/2449350102 >> 172.21.15.179:0/2449350102 conn(0x25a15b0 msgr2=0x2839ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- 172.21.15.179:0/2449350102 shutdown_connections 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.170+0000 7f2be2168740 1 -- 172.21.15.179:0/2449350102 wait complete. 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- start start 2024-05-08T22:37:08.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 Processor -- start 2024-05-08T22:37:08.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 0x2836550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x283dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x277c150 con 0x283a5d0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2779eb0 con 0x28376f0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x27378e0 con 0x280d840 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 10 client.0 _ll_register_callbacks cb 0x2716670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 0x2836550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x283dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52722/0 (socket says 172.21.15.179:52722) 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 -- 172.21.15.179:0/932210364 learned_addr learned my addr 172.21.15.179:0/932210364 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 0x2836550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:32904/0 (socket says 172.21.15.179:32904) 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 -- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 msgr2=0x2836550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 0x2836550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 -- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 msgr2=0x283dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x283dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 -- 172.21.15.179:0/932210364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80cf480 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bdd456700 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c040 tx=0x7f2bd0055810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654b6a645f3f9d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0084040 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2bd006d170 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25ae2c0 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd005db80 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 10 client.6738 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x2740290 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.174+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25ae2c0 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2bd0054c80 con 0x283a5d0 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 10 client.6738 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 10 client.6738 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:08.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 20 client.6738 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:08.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x28111d0 0x286cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 4 client.6738 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:08.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25f8540 con 0x28111d0 2024-05-08T22:37:08.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x28111d0 0x286cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bb77fe700 10 client.6738 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x28111d0 0x286cca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caff312fc3f2e6f1 server_cookie=7e6abcc529756b82 in_seq=0 out_seq=0 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bb77fe700 1 -- 172.21.15.179:0/932210364 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x25f8540 con 0x28111d0 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bb77fe700 10 client.6738 handle_command_reply: tid=0 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 client.6738 shutdown 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 msgr2=0x283e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c040 tx=0x7f2bd0055810 comp rx=0 tx=0).stop 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 shutdown_connections 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x28111d0 0x286cca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x283dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2bb77fe700 0 client.6738 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x283a5d0 0x283e3e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 --2- 172.21.15.179:0/932210364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x280d840 0x2836550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 >> 172.21.15.179:0/932210364 conn(0x25a15b0 msgr2=0x280bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 shutdown_connections 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 1 -- 172.21.15.179:0/932210364 wait complete. 2024-05-08T22:37:08.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.178+0000 7f2be2168740 20 client.6738 trim_cache size 0 max 16384 2024-05-08T22:37:08.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 Processor -- start 2024-05-08T22:37:08.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 -- start start 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x2811af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2625b80 con 0x280d840 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x274ac00 con 0x28376f0 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x285f670 con 0x28111d0 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52738/0 (socket says 172.21.15.179:52738) 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 -- 172.21.15.179:0/2527487832 learned_addr learned my addr 172.21.15.179:0/2527487832 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdd456700 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x2811af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bd7fff700 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 -- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 msgr2=0x28115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 -- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 msgr2=0x2811af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x2811af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 -- 172.21.15.179:0/2527487832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27264c0 con 0x280d840 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00f9f0 tx=0x7f2bcc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f2d7b43a947564 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bbeffd700 1 -- 172.21.15.179:0/2527487832 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc028040 con 0x280d840 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bbeffd700 1 -- 172.21.15.179:0/2527487832 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc029b00 con 0x280d840 2024-05-08T22:37:08.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.202+0000 7f2bbeffd700 1 -- 172.21.15.179:0/2527487832 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc029e70 con 0x280d840 2024-05-08T22:37:08.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- 172.21.15.179:0/2527487832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 msgr2=0x2812890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00f9f0 tx=0x7f2bcc00a000 comp rx=0 tx=0).stop 2024-05-08T22:37:08.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- 172.21.15.179:0/2527487832 shutdown_connections 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28376f0 0x2811af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- 172.21.15.179:0/2527487832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x280d840 0x2812890 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- 172.21.15.179:0/2527487832 >> 172.21.15.179:0/2527487832 conn(0x25a15b0 msgr2=0x283d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- 172.21.15.179:0/2527487832 shutdown_connections 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- 172.21.15.179:0/2527487832 wait complete. 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- start start 2024-05-08T22:37:08.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 Processor -- start 2024-05-08T22:37:08.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x28b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x27776c0 con 0x28376f0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x28bb2f0 con 0x280d840 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x285de20 con 0x28111d0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2be2168740 10 client.0 _ll_register_callbacks cb 0x2733d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2bdcc55700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x28b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2bdd456700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52746/0 (socket says 172.21.15.179:52746) 2024-05-08T22:37:08.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.206+0000 7f2bdd456700 1 -- 172.21.15.179:0/2429180619 learned_addr learned my addr 172.21.15.179:0/2429180619 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 -- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 msgr2=0x28ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28ba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 -- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 msgr2=0x28b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x28b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 -- 172.21.15.179:0/2429180619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80d0a00 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdd456700 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007fd0 tx=0x7f2bd0082be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61b54ff0068cce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0084040 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2bd006d170 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bc800bbf0 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd00550f0 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 10 client.6744 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x271a790 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0064080 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2bd0079070 con 0x28376f0 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 10 client.6744 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 10 client.6744 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 20 client.6744 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:08.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x283a5d0 0x285baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 4 client.6744 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:37:08.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x261d2c0 con 0x283a5d0 2024-05-08T22:37:08.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x283a5d0 0x285baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 10 client.6744 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bdcc55700 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x283a5d0 0x285baa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2240aefd030fce server_cookie=71cdda6abc47a0e9 in_seq=0 out_seq=0 2024-05-08T22:37:08.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 1 -- 172.21.15.179:0/2429180619 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x261d2c0 con 0x283a5d0 2024-05-08T22:37:08.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 10 client.6744 handle_command_reply: tid=0 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 client.6744 shutdown 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 msgr2=0x28badb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007fd0 tx=0x7f2bd0082be0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 shutdown_connections 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x283a5d0 0x285baa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28376f0 0x28badb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2bb77fe700 0 client.6744 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28111d0 0x28ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 --2- 172.21.15.179:0/2429180619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x280d840 0x28b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 >> 172.21.15.179:0/2429180619 conn(0x25a15b0 msgr2=0x28115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.210+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 shutdown_connections 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2be2168740 1 -- 172.21.15.179:0/2429180619 wait complete. 2024-05-08T22:37:08.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2be2168740 20 client.6744 trim_cache size 0 max 16384 2024-05-08T22:37:08.217 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:37:08.217 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:37:08.217 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:37:08.217 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 msgr2=0x7f2bc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 0x7f2bc4068bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000f60 tx=0x7f2bcc009040 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 msgr2=0x7f2bd81597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007e60 tx=0x7f2bd000e3c0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 shutdown_connections 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bc4066730 0x7f2bc4068bf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd80cbca0 0x7f2bd80c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bd815ae20 0x7f2bd81597f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 --2- 172.21.15.179:0/1491998364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80afa50 0x7f2bd80c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 >> 172.21.15.179:0/1491998364 conn(0x7f2bd80b2820 msgr2=0x7f2bd815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 shutdown_connections 2024-05-08T22:37:08.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.214+0000 7f2bddc57700 1 -- 172.21.15.179:0/1491998364 wait complete. 2024-05-08T22:37:08.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:37:08.233 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:08.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 Processor -- start 2024-05-08T22:37:08.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 -- start start 2024-05-08T22:37:08.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc640ce380 0x7fbc640cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6406abb0 con 0x7fbc640cc700 2024-05-08T22:37:08.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6406ae90 con 0x7fbc640cb510 2024-05-08T22:37:08.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc694f0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640cb980 con 0x7fbc640ce380 2024-05-08T22:37:08.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc63fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbc640cc700 0x7fbc6415b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43892/0 (socket says 172.21.15.179:43892) 2024-05-08T22:37:08.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc637fe700 1 --2- 172.21.15.179:0/1333930965 >> v2:172.21.15.179:3300/0 conn(0x7fbc640ce380 0x7fbc640cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc63fff700 1 -- 172.21.15.179:0/1333930965 learned_addr learned my addr 172.21.15.179:0/1333930965 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063134959 0 0) 0x7fbc6406abb0 con 0x7fbc640cc700 2024-05-08T22:37:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc500035d0 con 0x7fbc640cc700 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.350+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795112932 0 0) 0x7fbc6406ae90 con 0x7fbc640cb510 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc6406abb0 con 0x7fbc640cb510 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc637fe700 1 -- 172.21.15.179:0/1333930965 >> v1:172.21.15.195:6789/0 conn(0x7fbc640cb510 legacy=0x7fbc6406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc637fe700 1 -- 172.21.15.179:0/1333930965 >> v1:172.21.15.179:6789/0 conn(0x7fbc640cc700 legacy=0x7fbc6415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc637fe700 1 -- 172.21.15.179:0/1333930965 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640ce800 con 0x7fbc640ce380 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc637fe700 1 --2- 172.21.15.179:0/1333930965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc640cafd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009d90 tx=0x7fbc4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad56f8e61c84637a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00e070 con 0x7fbc640ce380 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c00bb20 con 0x7fbc640ce380 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc62ffd700 1 -- 172.21.15.179:0/1333930965 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c012430 con 0x7fbc640ce380 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- 172.21.15.179:0/1333930965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 msgr2=0x7fbc640cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 --2- 172.21.15.179:0/1333930965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc640cafd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009d90 tx=0x7fbc4c00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- 172.21.15.179:0/1333930965 shutdown_connections 2024-05-08T22:37:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 --2- 172.21.15.179:0/1333930965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc640cafd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- 172.21.15.179:0/1333930965 >> 172.21.15.179:0/1333930965 conn(0x7fbc640ce9d0 msgr2=0x7fbc640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- 172.21.15.179:0/1333930965 shutdown_connections 2024-05-08T22:37:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- 172.21.15.179:0/1333930965 wait complete. 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 Processor -- start 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.354+0000 7fbc694f0700 1 -- start start 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc640cb510 0x7fbc640b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc640cc700 0x7fbc64150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640b2020 con 0x7fbc640ce380 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640b22a0 con 0x7fbc640cb510 2024-05-08T22:37:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640b2580 con 0x7fbc640cc700 2024-05-08T22:37:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52772/0 (socket says 172.21.15.179:52772) 2024-05-08T22:37:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc640cb510 0x7fbc640b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc637fe700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc640cc700 0x7fbc64150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 -- 172.21.15.179:0/2707282733 learned_addr learned my addr 172.21.15.179:0/2707282733 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 -- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc640cc700 msgr2=0x7fbc64150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc640cc700 0x7fbc64150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 -- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc640cb510 msgr2=0x7fbc640b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc640cb510 0x7fbc640b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 -- 172.21.15.179:0/2707282733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c009a70 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc68cef700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c500 tx=0x7fbc5c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0052d0 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c005430 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc640b2860 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0055a0 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc640b2e20 con 0x7fbc640ce380 2024-05-08T22:37:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.358+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64067740 con 0x7fbc640ce380 2024-05-08T22:37:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.362+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc5c0038e0 con 0x7fbc640ce380 2024-05-08T22:37:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.362+0000 7fbc60ff9700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 0x7fbc44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.362+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fbc5c0836e0 con 0x7fbc640ce380 2024-05-08T22:37:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.362+0000 7fbc63fff700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 0x7fbc44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.362+0000 7fbc63fff700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 0x7fbc44068bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc54007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.370+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc5c050940 con 0x7fbc640ce380 2024-05-08T22:37:08.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.574+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc640b3010 con 0x7fbc640ce380 2024-05-08T22:37:08.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc60ff9700 1 -- 172.21.15.179:0/2707282733 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fbc5c081e60 con 0x7fbc640ce380 2024-05-08T22:37:08.579 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:08.579 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:36:41.136365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 msgr2=0x7fbc44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 0x7fbc44068bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc54007e00 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 msgr2=0x7fbc6415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c500 tx=0x7fbc5c00c810 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 shutdown_connections 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc44066730 0x7fbc44068bf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc640ce380 0x7fbc6415ec70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc640cc700 0x7fbc64150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 --2- 172.21.15.179:0/2707282733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc640cb510 0x7fbc640b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 >> 172.21.15.179:0/2707282733 conn(0x7fbc640ce9d0 msgr2=0x7fbc6415a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 shutdown_connections 2024-05-08T22:37:08.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.578+0000 7fbc694f0700 1 -- 172.21.15.179:0/2707282733 wait complete. 2024-05-08T22:37:08.583 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 31 2024-05-08T22:37:08.594 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-08T22:37:08.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.706+0000 7fe11e27f700 1 Processor -- start 2024-05-08T22:37:08.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.706+0000 7fe11e27f700 1 -- start start 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe11815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11815e050 con 0x7fe11806c2d0 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11806f0a0 con 0x7fe11815ae10 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11815b290 con 0x7fe118143580 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11da7e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe11806c2d0 0x7fe1180cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43908/0 (socket says 172.21.15.179:43908) 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11da7e700 1 -- 172.21.15.179:0/717128188 learned_addr learned my addr 172.21.15.179:0/717128188 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:08.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11d27d700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe11815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456266013 0 0) 0x7fe11815e050 con 0x7fe11806c2d0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1040035d0 con 0x7fe11806c2d0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913441270 0 0) 0x7fe11806f0a0 con 0x7fe11815ae10 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11815e050 con 0x7fe11815ae10 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1541751546 0 0) 0x7fe1040035d0 con 0x7fe11806c2d0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 msgr2=0x7fe11815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe11815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 >> v1:172.21.15.195:6789/0 conn(0x7fe11815ae10 legacy=0x7fe11806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11806edc0 con 0x7fe11806c2d0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe1100019f0 con 0x7fe11806c2d0 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 >> v1:172.21.15.179:6789/0 conn(0x7fe11806c2d0 legacy=0x7fe1180cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe104003be0 0x7fe104003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe104004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe104004db0 0x7fe104005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11815b290 con 0x7fe104004db0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400dd40 con 0x7fe104003be0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400df00 con 0x7fe118143580 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11da7e700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe104004db0 0x7fe104005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe104004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11d27d700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe104003be0 0x7fe104003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 -- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe104003be0 msgr2=0x7fe104003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe104003be0 0x7fe104003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 -- 172.21.15.179:0/717128188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe104004db0 msgr2=0x7fe104005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe104004db0 0x7fe104005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 -- 172.21.15.179:0/717128188 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040035d0 con 0x7fe118143580 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe104004870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001960 tx=0x7fe1040058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ef7dbb3c92a558 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c001450 con 0x7fe118143580 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c0052f0 con 0x7fe118143580 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe10bfff700 1 -- 172.21.15.179:0/717128188 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0036a0 con 0x7fe118143580 2024-05-08T22:37:08.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 -- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 msgr2=0x7fe104004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.710+0000 7fe11e27f700 1 --2- 172.21.15.179:0/717128188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe104004870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001960 tx=0x7fe1040058a0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11da7e700 1 -- 172.21.15.179:0/717128188 reap_dead start 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- 172.21.15.179:0/717128188 shutdown_connections 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- 172.21.15.179:0/717128188 >> 172.21.15.179:0/717128188 conn(0x7fe118070d80 msgr2=0x7fe118071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- 172.21.15.179:0/717128188 shutdown_connections 2024-05-08T22:37:08.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- 172.21.15.179:0/717128188 wait complete. 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 Processor -- start 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- start start 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe1180c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe11815ba90 0x7fe11815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11815c450 con 0x7fe1181f3890 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180c90e0 con 0x7fe11815ba90 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11e27f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11813d090 con 0x7fe118143580 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52778/0 (socket says 172.21.15.179:52778) 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11d27d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe1180c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11ca7c700 1 -- 172.21.15.179:0/2180229410 learned_addr learned my addr 172.21.15.179:0/2180229410 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.714+0000 7fe11da7e700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe11815ba90 0x7fe11815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 -- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 msgr2=0x7fe1180c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe1180c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 -- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe11815ba90 msgr2=0x7fe11815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe11815ba90 0x7fe11815bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 -- 172.21.15.179:0/2180229410 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1181f3d10 con 0x7fe1181f3890 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001190 tx=0x7fe10c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe109ffb700 1 -- 172.21.15.179:0/2180229410 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c006c90 con 0x7fe1181f3890 2024-05-08T22:37:08.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe109ffb700 1 -- 172.21.15.179:0/2180229410 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c006df0 con 0x7fe1181f3890 2024-05-08T22:37:08.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1181ed230 con 0x7fe1181f3890 2024-05-08T22:37:08.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe109ffb700 1 -- 172.21.15.179:0/2180229410 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c007280 con 0x7fe1181f3890 2024-05-08T22:37:08.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1180c87b0 con 0x7fe1181f3890 2024-05-08T22:37:08.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.718+0000 7fe109ffb700 1 -- 172.21.15.179:0/2180229410 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe10c0073e0 con 0x7fe1181f3890 2024-05-08T22:37:08.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe109ffb700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 0x7fe0fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe109ffb700 1 -- 172.21.15.179:0/2180229410 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe10c085200 con 0x7fe1181f3890 2024-05-08T22:37:08.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe11d27d700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 0x7fe0fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe11d27d700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 0x7fe0fc068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe11815c630 tx=0x7fe114009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe122790740 1 Processor -- start 2024-05-08T22:37:08.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.722+0000 7fe122790740 1 -- start start 2024-05-08T22:37:08.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x264a520 0x2701d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2629810 con 0x2702680 2024-05-08T22:37:08.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2626fa0 con 0x264a520 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2622890 con 0x27022a0 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11da7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11da7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52786/0 (socket says 172.21.15.179:52786) 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11da7e700 1 -- 172.21.15.179:0/285019004 learned_addr learned my addr 172.21.15.179:0/285019004 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11d27d700 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x264a520 0x2701d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 -- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x264a520 msgr2=0x2701d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x264a520 0x2701d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 -- 172.21.15.179:0/285019004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 msgr2=0x26cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 -- 172.21.15.179:0/285019004 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2494ba0 con 0x27022a0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11da7e700 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26caf30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0835c0 tx=0x7fe10c0011c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654cc67a91c3bf10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe102ffd700 1 -- 172.21.15.179:0/285019004 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c027ea0 con 0x27022a0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe102ffd700 1 -- 172.21.15.179:0/285019004 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c056910 con 0x27022a0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe102ffd700 1 -- 172.21.15.179:0/285019004 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0561b0 con 0x27022a0 2024-05-08T22:37:08.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 msgr2=0x26caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26caf30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0835c0 tx=0x7fe10c0011c0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- 172.21.15.179:0/285019004 shutdown_connections 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26caf30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 --2- 172.21.15.179:0/285019004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x264a520 0x2701d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- 172.21.15.179:0/285019004 >> 172.21.15.179:0/285019004 conn(0x2449710 msgr2=0x26e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- 172.21.15.179:0/285019004 shutdown_connections 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- 172.21.15.179:0/285019004 wait complete. 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 -- start start 2024-05-08T22:37:08.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.726+0000 7fe122790740 1 Processor -- start 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x26bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e5d80 con 0x264a520 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e5a20 con 0x2702680 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2515770 con 0x27022a0 2024-05-08T22:37:08.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 10 client.0 _ll_register_callbacks cb 0x261bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52802/0 (socket says 172.21.15.179:52802) 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 -- 172.21.15.179:0/3898746971 learned_addr learned my addr 172.21.15.179:0/3898746971 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11da7e700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x26bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 -- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 msgr2=0x26bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 -- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 msgr2=0x26bf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x26bf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 -- 172.21.15.179:0/3898746971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11806edc0 con 0x264a520 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe114003d60 tx=0x7fe114004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3284183c32cf5a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114003d90 con 0x264a520 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe114015420 con 0x264a520 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24a06a0 con 0x264a520 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe114005a40 con 0x264a520 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 10 client.6765 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:08.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x7fe1180c87b0 con 0x264a520 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24a06a0 con 0x264a520 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe114005ba0 con 0x264a520 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.730+0000 7fe122790740 10 client.6765 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 10 client.6765 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 20 client.6765 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26be000 0x26be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 4 client.6765 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x24c5420 con 0x26be000 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26be000 0x26be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe0f7fff700 10 client.6765 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26be000 0x26be3e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bd2da1770b29e9 server_cookie=1083ea9f4731c333 in_seq=0 out_seq=0 2024-05-08T22:37:08.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3898746971 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x24c5420 con 0x26be000 2024-05-08T22:37:08.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe0f7fff700 10 client.6765 handle_command_reply: tid=0 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 client.6765 shutdown 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 msgr2=0x2701380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe114003d60 tx=0x7fe114004770 comp rx=0 tx=0).stop 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 shutdown_connections 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26be000 0x26be3e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x26bf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe0f7fff700 0 client.6765 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x27022a0 0x26bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 --2- 172.21.15.179:0/3898746971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x264a520 0x2701380 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 >> 172.21.15.179:0/3898746971 conn(0x2449710 msgr2=0x26e11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 shutdown_connections 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 1 -- 172.21.15.179:0/3898746971 wait complete. 2024-05-08T22:37:08.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.734+0000 7fe122790740 20 client.6765 trim_cache size 0 max 16384 2024-05-08T22:37:08.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 Processor -- start 2024-05-08T22:37:08.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- start start 2024-05-08T22:37:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x26beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e00e0 con 0x2702680 2024-05-08T22:37:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2626890 con 0x27022a0 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x261fc90 con 0x264a520 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11d27d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x26beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:36346/0 (socket says 172.21.15.179:36346) 2024-05-08T22:37:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11ca7c700 1 -- 172.21.15.179:0/1911534502 learned_addr learned my addr 172.21.15.179:0/1911534502 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 -- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 msgr2=0x26beaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x26beaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 -- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 msgr2=0x26bf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 -- 172.21.15.179:0/1911534502 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25e8910 con 0x2702680 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe11da7e700 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26ff000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe11806e640 tx=0x7fe110006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a607e28707354785 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe1037fe700 1 -- 172.21.15.179:0/1911534502 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110016c80 con 0x2702680 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe1037fe700 1 -- 172.21.15.179:0/1911534502 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110016de0 con 0x2702680 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe1037fe700 1 -- 172.21.15.179:0/1911534502 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110007370 con 0x2702680 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- 172.21.15.179:0/1911534502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 msgr2=0x26ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26ff000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe11806e640 tx=0x7fe110006040 comp rx=0 tx=0).stop 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- 172.21.15.179:0/1911534502 shutdown_connections 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x27022a0 0x26bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2702680 0x26ff000 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 --2- 172.21.15.179:0/1911534502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x26beaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- 172.21.15.179:0/1911534502 >> 172.21.15.179:0/1911534502 conn(0x2449710 msgr2=0x26be3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- 172.21.15.179:0/1911534502 shutdown_connections 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.754+0000 7fe122790740 1 -- 172.21.15.179:0/1911534502 wait complete. 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- start start 2024-05-08T22:37:08.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 Processor -- start 2024-05-08T22:37:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x276a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x2727790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x26499a0 con 0x27022a0 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x262aaa0 con 0x2702680 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x25e3660 con 0x264a520 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 10 client.0 _ll_register_callbacks cb 0x2621460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52816/0 (socket says 172.21.15.179:52816) 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 -- 172.21.15.179:0/3988044855 learned_addr learned my addr 172.21.15.179:0/3988044855 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x276a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11da7e700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x2727790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 -- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 msgr2=0x276a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x276a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 -- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 msgr2=0x2727790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x2727790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 -- 172.21.15.179:0/3988044855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c079e70 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11ca7c700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001e40 tx=0x7fe10c05c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe5195f9437f5b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c05e5a0 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2456420 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe10c072040 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10c0561b0 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 10 client.6771 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x7fe110016520 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2456420 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe10c065020 con 0x27022a0 2024-05-08T22:37:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 10 client.6771 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 10 client.6771 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 20 client.6771 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26b91f0 0x26b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 4 client.6771 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x26e41f0 con 0x26b91f0 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.758+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26b91f0 0x26b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe0f7fff700 10 client.6771 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11d27d700 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26b91f0 0x26b95d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3634d5833488bd6a server_cookie=6650047b856e2f64 in_seq=0 out_seq=0 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3988044855 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x26e41f0 con 0x26b91f0 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe0f7fff700 10 client.6771 handle_command_reply: tid=0 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 client.6771 shutdown 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 msgr2=0x2727250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001e40 tx=0x7fe10c05c000 comp rx=0 tx=0).stop 2024-05-08T22:37:08.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 shutdown_connections 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26b91f0 0x26b95d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe0f7fff700 0 client.6771 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2702680 0x2727790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x27022a0 0x2727250 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 --2- 172.21.15.179:0/3988044855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x264a520 0x276a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 >> 172.21.15.179:0/3988044855 conn(0x2449710 msgr2=0x26cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 shutdown_connections 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 1 -- 172.21.15.179:0/3988044855 wait complete. 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe122790740 20 client.6771 trim_cache size 0 max 16384 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:08.767 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "backtrace", 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 2377242782, 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir_test/file" 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:08.768 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 msgr2=0x7fe0fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 0x7fe0fc068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe11815c630 tx=0x7fe114009240 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 msgr2=0x7fe1181eccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe10c001190 tx=0x7fe10c0012d0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 shutdown_connections 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0fc066730 0x7fe0fc068bf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe11815ba90 0x7fe11815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1181f3890 0x7fe1181eccf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 --2- 172.21.15.179:0/2180229410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe118143580 0x7fe1180c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 >> 172.21.15.179:0/2180229410 conn(0x7fe118070d80 msgr2=0x7fe1180ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 shutdown_connections 2024-05-08T22:37:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.762+0000 7fe11e27f700 1 -- 172.21.15.179:0/2180229410 wait complete. 2024-05-08T22:37:08.781 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:37:08.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.894+0000 7f9522531700 1 Processor -- start 2024-05-08T22:37:08.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.894+0000 7f9522531700 1 -- start start 2024-05-08T22:37:08.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f951c0c9230 0x7f951c0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c158f60 con 0x7f951c158b80 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c06b860 con 0x7f951c0c9230 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c06ac50 con 0x7f951c0c8d70 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f951c0c9230 0x7f951c0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f951c0c9230 0x7f951c0c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52824/0 (socket says 172.21.15.179:52824) 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 -- 172.21.15.179:0/3672951009 learned_addr learned my addr 172.21.15.179:0/3672951009 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9513fff700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 -- 172.21.15.179:0/3672951009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 msgr2=0x7f951c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 -- 172.21.15.179:0/3672951009 >> v1:172.21.15.179:6789/0 conn(0x7f951c158b80 legacy=0x7f951c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 -- 172.21.15.179:0/3672951009 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c06af30 con 0x7f951c0c9230 2024-05-08T22:37:08.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9520ace700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c0c96c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0735fc6ab5f98c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f951b7fe700 1 -- 172.21.15.179:0/3672951009 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400e070 con 0x7f951c0c9230 2024-05-08T22:37:08.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f951b7fe700 1 -- 172.21.15.179:0/3672951009 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951400b9c0 con 0x7f951c0c9230 2024-05-08T22:37:08.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f951b7fe700 1 -- 172.21.15.179:0/3672951009 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400bd70 con 0x7f951c0c9230 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- 172.21.15.179:0/3672951009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 msgr2=0x7f951c0c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c0c96c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951400b330 comp rx=0 tx=0).stop 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- 172.21.15.179:0/3672951009 shutdown_connections 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c0c96c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 --2- 172.21.15.179:0/3672951009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- 172.21.15.179:0/3672951009 >> 172.21.15.179:0/3672951009 conn(0x7f951c0cb660 msgr2=0x7f951c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- 172.21.15.179:0/3672951009 shutdown_connections 2024-05-08T22:37:08.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.898+0000 7f9522531700 1 -- 172.21.15.179:0/3672951009 wait complete. 2024-05-08T22:37:08.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 Processor -- start 2024-05-08T22:37:08.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- start start 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f951c158b80 0x7f951c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c7a20 con 0x7f951c0c9230 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c1572e0 con 0x7f951c158b80 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c6da0 con 0x7f951c0c8d70 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52834/0 (socket says 172.21.15.179:52834) 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 -- 172.21.15.179:0/2733274525 learned_addr learned my addr 172.21.15.179:0/2733274525 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f951bfff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9520ace700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f951c158b80 0x7f951c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 -- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 msgr2=0x7f951c156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 -- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f951c158b80 msgr2=0x7f951c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f951c158b80 0x7f951c0c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 -- 172.21.15.179:0/2733274525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009b30 con 0x7f951c0c9230 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9513fff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f950400e810 tx=0x7f950400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504013610 con 0x7f951c0c9230 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0c7080 con 0x7f951c0c9230 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0c7640 con 0x7f951c0c9230 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9504013770 con 0x7f951c0c9230 2024-05-08T22:37:08.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.902+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504013b20 con 0x7f951c0c9230 2024-05-08T22:37:08.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c067770 con 0x7f951c0c9230 2024-05-08T22:37:08.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9504013c80 con 0x7f951c0c9230 2024-05-08T22:37:08.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f95197fa700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 0x7f94fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:08.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f95040840b0 con 0x7f951c0c9230 2024-05-08T22:37:08.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f951bfff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 0x7f94fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:08.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.906+0000 7f951bfff700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 0x7f94fc068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f950c001800 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:08.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:08.914+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f95040511c0 con 0x7f951c0c9230 2024-05-08T22:37:09.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.122+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f951c073290 con 0x7f951c0c9230 2024-05-08T22:37:09.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.122+0000 7f95197fa700 1 -- 172.21.15.179:0/2733274525 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f95040547e0 con 0x7f951c0c9230 2024-05-08T22:37:09.124 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:09.124 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T22:37:09.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 msgr2=0x7f94fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 0x7f94fc068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f950c001800 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 msgr2=0x7f951c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f950400e810 tx=0x7f950400eb20 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 shutdown_connections 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94fc066730 0x7f94fc068bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f951c158b80 0x7f951c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f951c0c9230 0x7f951c15b810 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 --2- 172.21.15.179:0/2733274525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f951c0c8d70 0x7f951c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 >> 172.21.15.179:0/2733274525 conn(0x7f951c0cb660 msgr2=0x7f951c15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 shutdown_connections 2024-05-08T22:37:09.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.126+0000 7f9522531700 1 -- 172.21.15.179:0/2733274525 wait complete. 2024-05-08T22:37:09.138 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:37:09.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-08T22:37:09.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.254+0000 7f5432b39700 1 Processor -- start 2024-05-08T22:37:09.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.254+0000 7f5432b39700 1 -- start start 2024-05-08T22:37:09.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f542c15d490 0x7f542c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c06c2e0 0x7f542c0d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f542c0cde90 0x7f542c0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0d2740 con 0x7f542c15d490 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0ce840 con 0x7f542c0cde90 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15d870 con 0x7f542c06c2e0 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f542c15d490 0x7f542c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f542c15d490 0x7f542c06abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52844/0 (socket says 172.21.15.179:52844) 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 -- 172.21.15.179:0/3290104437 learned_addr learned my addr 172.21.15.179:0/3290104437 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431336700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c06c2e0 0x7f542c0d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432338700 1 --2- 172.21.15.179:0/3290104437 >> v2:172.21.15.195:3300/0 conn(0x7f542c0cde90 0x7f542c0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 -- 172.21.15.179:0/3290104437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c06c2e0 msgr2=0x7f542c0d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c06c2e0 0x7f542c0d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 -- 172.21.15.179:0/3290104437 >> v2:172.21.15.195:3300/0 conn(0x7f542c0cde90 msgr2=0x7f542c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 --2- 172.21.15.179:0/3290104437 >> v2:172.21.15.195:3300/0 conn(0x7f542c0cde90 0x7f542c0ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 -- 172.21.15.179:0/3290104437 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c06b240 con 0x7f542c15d490 2024-05-08T22:37:09.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5431b37700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15d490 0x7f542c06abb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f542c0d0de0 tx=0x7f541c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caac4fa14291c7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5430b35700 1 -- 172.21.15.179:0/3290104437 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c011830 con 0x7f542c15d490 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5430b35700 1 -- 172.21.15.179:0/3290104437 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c011990 con 0x7f542c15d490 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5430b35700 1 -- 172.21.15.179:0/3290104437 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c011d40 con 0x7f542c15d490 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- 172.21.15.179:0/3290104437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15d490 msgr2=0x7f542c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15d490 0x7f542c06abb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f542c0d0de0 tx=0x7f541c009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- 172.21.15.179:0/3290104437 shutdown_connections 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- 172.21.15.179:0/3290104437 >> v2:172.21.15.195:3300/0 conn(0x7f542c0cde90 0x7f542c0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c06c2e0 0x7f542c0d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 --2- 172.21.15.179:0/3290104437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15d490 0x7f542c06abb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- 172.21.15.179:0/3290104437 >> 172.21.15.179:0/3290104437 conn(0x7f542c0b4940 msgr2=0x7f542c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- 172.21.15.179:0/3290104437 shutdown_connections 2024-05-08T22:37:09.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.258+0000 7f5432b39700 1 -- 172.21.15.179:0/3290104437 wait complete. 2024-05-08T22:37:09.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 Processor -- start 2024-05-08T22:37:09.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 -- start start 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0cde90 0x7f542c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c15d490 0x7f542c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cb010 con 0x7f542c06c2e0 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cb2f0 con 0x7f542c15d490 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432b39700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cb5d0 con 0x7f542c0cde90 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52848/0 (socket says 172.21.15.179:52848) 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5431b37700 1 -- 172.21.15.179:0/230703180 learned_addr learned my addr 172.21.15.179:0/230703180 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5431336700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0cde90 0x7f542c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.262+0000 7f5432338700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c15d490 0x7f542c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 -- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0cde90 msgr2=0x7f542c1f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0cde90 0x7f542c1f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 -- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c15d490 msgr2=0x7f542c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c15d490 0x7f542c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 -- 172.21.15.179:0/230703180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009570 con 0x7f542c06c2e0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5431b37700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f541c000c00 tx=0x7f541c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f54227fc700 1 -- 172.21.15.179:0/230703180 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c004240 con 0x7f542c06c2e0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f54227fc700 1 -- 172.21.15.179:0/230703180 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c01f070 con 0x7f542c06c2e0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f54227fc700 1 -- 172.21.15.179:0/230703180 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0045c0 con 0x7f542c06c2e0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c0cb8b0 con 0x7f542c06c2e0 2024-05-08T22:37:09.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0c90a0 con 0x7f542c06c2e0 2024-05-08T22:37:09.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.266+0000 7f54227fc700 1 -- 172.21.15.179:0/230703180 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f541c021030 con 0x7f542c06c2e0 2024-05-08T22:37:09.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f54227fc700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f54227fc700 1 -- 172.21.15.179:0/230703180 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f541c0853e0 con 0x7f542c06c2e0 2024-05-08T22:37:09.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f5431336700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f5431336700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f542c159560 tx=0x7f5428009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f543704a740 1 Processor -- start 2024-05-08T22:37:09.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.270+0000 7f543704a740 1 -- start start 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x23626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2362be0 0x2362fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d3100 con 0x23616d0 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d0890 con 0x2362be0 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x22cc180 con 0x238e460 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52860/0 (socket says 172.21.15.179:52860) 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 -- 172.21.15.179:0/2615173008 learned_addr learned my addr 172.21.15.179:0/2615173008 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5432338700 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2362be0 0x2362fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431336700 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x23626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 -- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 msgr2=0x23626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x23626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 -- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2362be0 msgr2=0x2362fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2362be0 0x2362fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 -- 172.21.15.179:0/2615173008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x213ed00 con 0x23616d0 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f5431b37700 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b2f0 tx=0x7f541c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f725e451d84d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f540f7fe700 1 -- 172.21.15.179:0/2615173008 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c056230 con 0x23616d0 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f540f7fe700 1 -- 172.21.15.179:0/2615173008 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c05a060 con 0x23616d0 2024-05-08T22:37:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f540f7fe700 1 -- 172.21.15.179:0/2615173008 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c05e620 con 0x23616d0 2024-05-08T22:37:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- 172.21.15.179:0/2615173008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 msgr2=0x2361ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b2f0 tx=0x7f541c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- 172.21.15.179:0/2615173008 shutdown_connections 2024-05-08T22:37:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2362be0 0x2362fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x23626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 --2- 172.21.15.179:0/2615173008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x2361ab0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- 172.21.15.179:0/2615173008 >> 172.21.15.179:0/2615173008 conn(0x20f3870 msgr2=0x238dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- 172.21.15.179:0/2615173008 shutdown_connections 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- 172.21.15.179:0/2615173008 wait complete. 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 -- start start 2024-05-08T22:37:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.274+0000 7f543704a740 1 Processor -- start 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x2366160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x238b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x228f900 con 0x2362be0 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x228f5a0 con 0x23616d0 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2177e40 con 0x238e460 2024-05-08T22:37:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 10 client.0 _ll_register_callbacks cb 0x228c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52868/0 (socket says 172.21.15.179:52868) 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 -- 172.21.15.179:0/3945396967 learned_addr learned my addr 172.21.15.179:0/3945396967 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431b37700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x2366160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5432338700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x238b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 -- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 msgr2=0x238b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x238b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 -- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 msgr2=0x2366160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x2366160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 -- 172.21.15.179:0/3945396967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0c90a0 con 0x2362be0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f5431336700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5428011e10 tx=0x7f5428004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38127e54abfec063 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542801c040 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f5428011e40 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2100580 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5428014440 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 10 client.6789 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x2292540 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2100580 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f54280145a0 con 0x2362be0 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 10 client.6789 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 10 client.6789 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 20 client.6789 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x23b2b20 0x23b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 4 client.6789 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.278+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x214a800 con 0x23b2b20 2024-05-08T22:37:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f5431b37700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x23b2b20 0x23b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f5431b37700 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x23b2b20 0x23b2f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb0fb00d19bd869 server_cookie=80394ab861ce1041 in_seq=0 out_seq=0 2024-05-08T22:37:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f540cff9700 10 client.6789 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f540cff9700 1 -- 172.21.15.179:0/3945396967 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x214a800 con 0x23b2b20 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f540cff9700 10 client.6789 handle_command_reply: tid=0 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 client.6789 shutdown 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 msgr2=0x238ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5428011e10 tx=0x7f5428004770 comp rx=0 tx=0).stop 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 shutdown_connections 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x23b2b20 0x23b2f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x238e460 0x238b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2362be0 0x238ac00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 --2- 172.21.15.179:0/3945396967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x2366160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f540cff9700 0 client.6789 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 >> 172.21.15.179:0/3945396967 conn(0x20f3870 msgr2=0x238d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 shutdown_connections 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 1 -- 172.21.15.179:0/3945396967 wait complete. 2024-05-08T22:37:09.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.282+0000 7f543704a740 20 client.6789 trim_cache size 0 max 16384 2024-05-08T22:37:09.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 Processor -- start 2024-05-08T22:37:09.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- start start 2024-05-08T22:37:09.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x23b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d3100 con 0x238e460 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cb010 con 0x23616d0 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2270ac0 con 0x2362be0 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52880/0 (socket says 172.21.15.179:52880) 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 -- 172.21.15.179:0/2465535903 learned_addr learned my addr 172.21.15.179:0/2465535903 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5431b37700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x23b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5431336700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 -- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 msgr2=0x23b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 -- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 msgr2=0x23b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x23b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 -- 172.21.15.179:0/2465535903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2278770 con 0x238e460 2024-05-08T22:37:09.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5431b37700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x23b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:09.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f5432338700 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5424017850 tx=0x7f5424009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3077739d22b17765 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f540ffff700 1 -- 172.21.15.179:0/2465535903 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e040 con 0x238e460 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f540ffff700 1 -- 172.21.15.179:0/2465535903 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400f040 con 0x238e460 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f540ffff700 1 -- 172.21.15.179:0/2465535903 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424015910 con 0x238e460 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- 172.21.15.179:0/2465535903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 msgr2=0x23bbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5424017850 tx=0x7f5424009750 comp rx=0 tx=0).stop 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- 172.21.15.179:0/2465535903 shutdown_connections 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x238e460 0x23bbb70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 --2- 172.21.15.179:0/2465535903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x23616d0 0x23b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- 172.21.15.179:0/2465535903 >> 172.21.15.179:0/2465535903 conn(0x20f3870 msgr2=0x23a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- 172.21.15.179:0/2465535903 shutdown_connections 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- 172.21.15.179:0/2465535903 wait complete. 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 -- start start 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.306+0000 7f543704a740 1 Processor -- start 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x238e460 0x23c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c4b30 con 0x23616d0 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2360c90 con 0x238e460 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x23c3090 con 0x2362be0 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 10 client.0 _ll_register_callbacks cb 0x226b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52892/0 (socket says 172.21.15.179:52892) 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 -- 172.21.15.179:0/221021645 learned_addr learned my addr 172.21.15.179:0/221021645 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5432338700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x238e460 0x23c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431336700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 -- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 msgr2=0x23c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 -- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x238e460 msgr2=0x23c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x238e460 0x23c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 -- 172.21.15.179:0/221021645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54240174d0 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431b37700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f541c009f70 tx=0x7f541c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772f6c57b8281bf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c082030 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f541c05a060 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f541c06de40 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f541c055d70 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 10 client.6795 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x226ca40 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c079020 con 0x23616d0 2024-05-08T22:37:09.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f541c0740c0 con 0x23616d0 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 10 client.6795 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 10 client.6795 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 20 client.6795 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x2415330 0x2415740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 4 client.6795 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x216f580 con 0x2415330 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.310+0000 7f5431336700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x2415330 0x2415740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5431336700 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x2415330 0x2415740 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595c8223ab576cab server_cookie=cbb1ca055bc89785 in_seq=0 out_seq=0 2024-05-08T22:37:09.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f540cff9700 10 client.6795 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.317 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-05-08T22:37:09.309+0000 7f5f0dd38700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-08T22:36:36.669919+0000 1=1+0) (iversion lock) caps={6669=pAsLsXsFs/-@2},l=6669(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56070a091b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:37:09.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f540cff9700 1 -- 172.21.15.179:0/221021645 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x216f580 con 0x2415330 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f540cff9700 10 client.6795 handle_command_reply: tid=0 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 client.6795 shutdown 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 msgr2=0x23c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f541c009f70 tx=0x7f541c00b260 comp rx=0 tx=0).stop 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 shutdown_connections 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x2415330 0x2415740 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x238e460 0x23c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2362be0 0x23c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 --2- 172.21.15.179:0/221021645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x23616d0 0x23c2b50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 >> 172.21.15.179:0/221021645 conn(0x20f3870 msgr2=0x23b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f540cff9700 0 client.6795 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 shutdown_connections 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 1 -- 172.21.15.179:0/221021645 wait complete. 2024-05-08T22:37:09.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f543704a740 20 client.6795 trim_cache size 0 max 16384 2024-05-08T22:37:09.319 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:37:09.319 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:37:09.319 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "c08422f5-9e75-4f33-a0a8-0be5ec0fd062", 2024-05-08T22:37:09.319 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:37:09.319 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 msgr2=0x7f5410068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f542c159560 tx=0x7f5428009240 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 msgr2=0x7f542c0c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f541c000c00 tx=0x7f541c004770 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 shutdown_connections 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c15d490 0x7f542c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0cde90 0x7f542c1f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 --2- 172.21.15.179:0/230703180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c06c2e0 0x7f542c0c8b60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 >> 172.21.15.179:0/230703180 conn(0x7f542c0b4940 msgr2=0x7f542c15e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 shutdown_connections 2024-05-08T22:37:09.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.314+0000 7f5432b39700 1 -- 172.21.15.179:0/230703180 wait complete. 2024-05-08T22:37:09.335 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T22:37:09.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 Processor -- start 2024-05-08T22:37:09.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 -- start start 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb5406cc60 0x7fcb540ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5415d5e0 0x7fcb540ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540cfa40 con 0x7fcb5415d200 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540ce950 con 0x7fcb5406cc60 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5406df30 con 0x7fcb5415d5e0 2024-05-08T22:37:09.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb5406cc60 0x7fcb540ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb59aef700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcb5415d200 0x7fcb5406c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43920/0 (socket says 172.21.15.179:43920) 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb59aef700 1 -- 172.21.15.179:0/927217443 learned_addr learned my addr 172.21.15.179:0/927217443 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5415d5e0 0x7fcb540ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832983727 0 0) 0x7fcb540cfa40 con 0x7fcb5415d200 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb400035d0 con 0x7fcb5415d200 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 -- 172.21.15.179:0/927217443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5415d5e0 msgr2=0x7fcb540ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2811780119 0 0) 0x7fcb400035d0 con 0x7fcb5415d200 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5415d5e0 0x7fcb540ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 -- 172.21.15.179:0/927217443 >> v1:172.21.15.179:6789/0 conn(0x7fcb5415d200 legacy=0x7fcb5406c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 -- 172.21.15.179:0/927217443 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5406d970 con 0x7fcb5406cc60 2024-05-08T22:37:09.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb592ee700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5406cc60 0x7fcb540ce2e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009880 tx=0x7fcb44009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43795db69c0ac610 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50001670 con 0x7fcb5406cc60 2024-05-08T22:37:09.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb44004030 con 0x7fcb5406cc60 2024-05-08T22:37:09.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb58aed700 1 -- 172.21.15.179:0/927217443 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb440043c0 con 0x7fcb5406cc60 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 -- 172.21.15.179:0/927217443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5406cc60 msgr2=0x7fcb540ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.454+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5406cc60 0x7fcb540ce2e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009880 tx=0x7fcb44009b90 comp rx=0 tx=0).stop 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 -- 172.21.15.179:0/927217443 shutdown_connections 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5415d5e0 0x7fcb540ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/927217443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5406cc60 0x7fcb540ce2e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 -- 172.21.15.179:0/927217443 >> 172.21.15.179:0/927217443 conn(0x7fcb540b18d0 msgr2=0x7fcb540b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 -- 172.21.15.179:0/927217443 shutdown_connections 2024-05-08T22:37:09.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 -- 172.21.15.179:0/927217443 wait complete. 2024-05-08T22:37:09.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 Processor -- start 2024-05-08T22:37:09.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.458+0000 7fcb5bd53700 1 -- start start 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 0x7fcb54150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415d200 0x7fcb54155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 0x7fcb540c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54156790 con 0x7fcb5415d5e0 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541564b0 con 0x7fcb5415d200 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54152e60 con 0x7fcb5406cc60 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5a2f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 0x7fcb540c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5a2f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 0x7fcb540c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52912/0 (socket says 172.21.15.179:52912) 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5a2f0700 1 -- 172.21.15.179:0/2543469647 learned_addr learned my addr 172.21.15.179:0/2543469647 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 0x7fcb54150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415d200 0x7fcb54155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 -- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415d200 msgr2=0x7fcb54155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415d200 0x7fcb54155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 -- 172.21.15.179:0/2543469647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 msgr2=0x7fcb540c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 0x7fcb540c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 -- 172.21.15.179:0/2543469647 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440094b0 con 0x7fcb5406cc60 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 0x7fcb54150fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb50001540 tx=0x7fcb5000e340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb4a7fc700 1 -- 172.21.15.179:0/2543469647 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000ecb0 con 0x7fcb5406cc60 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb4a7fc700 1 -- 172.21.15.179:0/2543469647 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000ee10 con 0x7fcb5406cc60 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb54153140 con 0x7fcb5406cc60 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb4a7fc700 1 -- 172.21.15.179:0/2543469647 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000b310 con 0x7fcb5406cc60 2024-05-08T22:37:09.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.462+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541eef40 con 0x7fcb5406cc60 2024-05-08T22:37:09.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.466+0000 7fcb4a7fc700 1 -- 172.21.15.179:0/2543469647 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb5000b470 con 0x7fcb5406cc60 2024-05-08T22:37:09.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.466+0000 7fcb4a7fc700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 0x7fcb38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.466+0000 7fcb4a7fc700 1 -- 172.21.15.179:0/2543469647 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fcb50088110 con 0x7fcb5406cc60 2024-05-08T22:37:09.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.466+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 0x7fcb38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.466+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 0x7fcb38068bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb440090b0 tx=0x7fcb44010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 Processor -- start 2024-05-08T22:37:09.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- start start 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x261c010 con 0x2689840 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x25a9f40 con 0x26b69b0 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x25fce30 con 0x26b65d0 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52916/0 (socket says 172.21.15.179:52916) 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 -- 172.21.15.179:0/2167566856 learned_addr learned my addr 172.21.15.179:0/2167566856 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 -- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 msgr2=0x26d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 -- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 msgr2=0x26d5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26d5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 -- 172.21.15.179:0/2167566856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2468a40 con 0x2689840 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26d5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcb50086470 tx=0x7fcb50000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0b5dbe4242025f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb377fe700 1 -- 172.21.15.179:0/2167566856 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5005f070 con 0x2689840 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb377fe700 1 -- 172.21.15.179:0/2167566856 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50058f70 con 0x2689840 2024-05-08T22:37:09.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb377fe700 1 -- 172.21.15.179:0/2167566856 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5005cda0 con 0x2689840 2024-05-08T22:37:09.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- 172.21.15.179:0/2167566856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 msgr2=0x2689c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcb50086470 tx=0x7fcb50000ed0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- 172.21.15.179:0/2167566856 shutdown_connections 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26d5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2167566856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2689c20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- 172.21.15.179:0/2167566856 >> 172.21.15.179:0/2167566856 conn(0x241d5b0 msgr2=0x26b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- 172.21.15.179:0/2167566856 shutdown_connections 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.470+0000 7fcb5f002740 1 -- 172.21.15.179:0/2167566856 wait complete. 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 -- start start 2024-05-08T22:37:09.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 Processor -- start 2024-05-08T22:37:09.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2689840 0x26881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f8150 con 0x26b65d0 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f5eb0 con 0x26b69b0 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x25b38e0 con 0x2689840 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 10 client.0 _ll_register_callbacks cb 0x2592670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52920/0 (socket says 172.21.15.179:52920) 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 -- 172.21.15.179:0/730872924 learned_addr learned my addr 172.21.15.179:0/730872924 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb59aef700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2689840 0x26881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 -- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2689840 msgr2=0x26881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2689840 0x26881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 -- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 msgr2=0x26bb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26bb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 -- 172.21.15.179:0/730872924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541eef40 con 0x26b65d0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb592ee700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009ee0 tx=0x7fcb440047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80e4b7144c0516b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4402b070 con 0x26b65d0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fcb44011840 con 0x26b65d0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.474+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x242a2c0 con 0x26b65d0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb44006ea0 con 0x26b65d0 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 10 client.6810 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:09.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x25bc290 con 0x26b65d0 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x242a2c0 con 0x26b65d0 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb44011a00 con 0x26b65d0 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 10 client.6810 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 10 client.6810 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 20 client.6810 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26d38a0 0x26d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 4 client.6810 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2474540 con 0x26d38a0 2024-05-08T22:37:09.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb59aef700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26d38a0 0x26d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb59aef700 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26d38a0 0x26d3cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9cadd3a63e2cbf server_cookie=d4de73a7dd7fd167 in_seq=0 out_seq=0 2024-05-08T22:37:09.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 10 client.6810 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 1 -- 172.21.15.179:0/730872924 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2474540 con 0x26d38a0 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 10 client.6810 handle_command_reply: tid=0 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 client.6810 shutdown 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 msgr2=0x26bb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009ee0 tx=0x7fcb440047e0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 shutdown_connections 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26d38a0 0x26d3cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26bb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x26b65d0 0x26bb380 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 --2- 172.21.15.179:0/730872924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2689840 0x26881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb34ff9700 0 client.6810 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 >> 172.21.15.179:0/730872924 conn(0x241d5b0 msgr2=0x2688f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 shutdown_connections 2024-05-08T22:37:09.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 1 -- 172.21.15.179:0/730872924 wait complete. 2024-05-08T22:37:09.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.478+0000 7fcb5f002740 20 client.6810 trim_cache size 0 max 16384 2024-05-08T22:37:09.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 Processor -- start 2024-05-08T22:37:09.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 -- start start 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b65d0 0x26b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b69b0 0x26e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x24a1b80 con 0x2689840 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c6c00 con 0x26b65d0 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x257fa50 con 0x26b69b0 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.498+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52922/0 (socket says 172.21.15.179:52922) 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 -- 172.21.15.179:0/4198326506 learned_addr learned my addr 172.21.15.179:0/4198326506 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb592ee700 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b65d0 0x26b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b69b0 0x26e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 -- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b69b0 msgr2=0x26e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b69b0 0x26e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 -- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b65d0 msgr2=0x26b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b65d0 0x26b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 -- 172.21.15.179:0/4198326506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25a24c0 con 0x2689840 2024-05-08T22:37:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcb500864c0 tx=0x7fcb5005f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834c1e10676532d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb37fff700 1 -- 172.21.15.179:0/4198326506 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50061360 con 0x2689840 2024-05-08T22:37:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb37fff700 1 -- 172.21.15.179:0/4198326506 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5005cda0 con 0x2689840 2024-05-08T22:37:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb37fff700 1 -- 172.21.15.179:0/4198326506 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50058460 con 0x2689840 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- 172.21.15.179:0/4198326506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 msgr2=0x26b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcb500864c0 tx=0x7fcb5005f000 comp rx=0 tx=0).stop 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- 172.21.15.179:0/4198326506 shutdown_connections 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b65d0 0x26b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b69b0 0x26e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- 172.21.15.179:0/4198326506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x26b1d00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- 172.21.15.179:0/4198326506 >> 172.21.15.179:0/4198326506 conn(0x241d5b0 msgr2=0x26a1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- 172.21.15.179:0/4198326506 shutdown_connections 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- 172.21.15.179:0/4198326506 wait complete. 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- start start 2024-05-08T22:37:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 Processor -- start 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c4190 con 0x2689840 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f36c0 con 0x26b69b0 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x26d40e0 con 0x26b65d0 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5f002740 10 client.0 _ll_register_callbacks cb 0x268f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52934/0 (socket says 172.21.15.179:52934) 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb59aef700 1 -- 172.21.15.179:0/2627782992 learned_addr learned my addr 172.21.15.179:0/2627782992 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:09.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.502+0000 7fcb5a2f0700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 -- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 msgr2=0x26e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 -- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 msgr2=0x26e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 -- 172.21.15.179:0/2627782992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2468a40 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb59aef700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000f60 tx=0x7fcb500011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1848c8cbc3cac988 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500552b0 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb50058f70 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fcb50086170 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb50061360 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 10 client.6816 fetch_fsmap learned FSMap version 31 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x2596790 con 0x2689840 2024-05-08T22:37:09.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50058f70 con 0x2689840 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb50058460 con 0x2689840 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 10 client.6816 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 10 client.6816 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 20 client.6816 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26bb200 0x26bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 4 client.6816 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x24992c0 con 0x26bb200 2024-05-08T22:37:09.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26bb200 0x26bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:09.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 10 client.6816 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb592ee700 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26bb200 0x26bb610 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd6b059d250cd17 server_cookie=d7cb95de8e8d9ba8 in_seq=0 out_seq=0 2024-05-08T22:37:09.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 1 -- 172.21.15.179:0/2627782992 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x24992c0 con 0x26bb200 2024-05-08T22:37:09.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 10 client.6816 handle_command_reply: tid=0 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 client.6816 shutdown 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 msgr2=0x2740600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000f60 tx=0x7fcb500011a0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 shutdown_connections 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x26bb200 0x26bb610 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x26b69b0 0x26e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x26b65d0 0x26e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb34ff9700 0 client.6816 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 --2- 172.21.15.179:0/2627782992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2689840 0x2740600 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 >> 172.21.15.179:0/2627782992 conn(0x241d5b0 msgr2=0x26a1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 shutdown_connections 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 1 -- 172.21.15.179:0/2627782992 wait complete. 2024-05-08T22:37:09.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.506+0000 7fcb5f002740 20 client.6816 trim_cache size 0 max 16384 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "c08422f5-9e75-4f33-a0a8-0be5ec0fd062": { 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/", 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "c08422f5-9e75-4f33-a0a8-0be5ec0fd062", 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair,force" 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:09.513 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 msgr2=0x7fcb38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 0x7fcb38068bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb440090b0 tx=0x7fcb44010000 comp rx=0 tx=0).stop 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 msgr2=0x7fcb54150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 0x7fcb54150fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb50001540 tx=0x7fcb5000e340 comp rx=0 tx=0).stop 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 shutdown_connections 2024-05-08T22:37:09.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb38066730 0x7fcb38068bf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415d200 0x7fcb54155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb5415d5e0 0x7fcb540c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 --2- 172.21.15.179:0/2543469647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb5406cc60 0x7fcb54150fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 >> 172.21.15.179:0/2543469647 conn(0x7fcb540b18d0 msgr2=0x7fcb540b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 shutdown_connections 2024-05-08T22:37:09.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:09.510+0000 7fcb5bd53700 1 -- 172.21.15.179:0/2543469647 wait complete. 2024-05-08T22:37:09.524 INFO:tasks.cephfs.filesystem:scrub status for tag:c08422f5-9e75-4f33-a0a8-0be5ec0fd062 - {'path': '/', 'tag': 'c08422f5-9e75-4f33-a0a8-0be5ec0fd062', 'options': 'recursive,repair,force'} 2024-05-08T22:37:22.019 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:37:22.022 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:37:39.524 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T22:37:39.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 Processor -- start 2024-05-08T22:37:39.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 -- start start 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1d9806bbc0 0x7f1d9806c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980cbfd0 con 0x7f1d9815c0b0 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980cb3c0 con 0x7f1d981448b0 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9f28d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d980cd970 con 0x7f1d9806bbc0 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9da8a700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1d9815c0b0 0x7f1d980cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37816/0 (socket says 172.21.15.179:37816) 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.649+0000 7f1d9da8a700 1 -- 172.21.15.179:0/910202950 learned_addr learned my addr 172.21.15.179:0/910202950 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:39.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/910202950 >> v2:172.21.15.195:3300/0 conn(0x7f1d9806bbc0 0x7f1d9806c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9d289700 1 -- 172.21.15.179:0/910202950 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792419513 0 0) 0x7f1d980cbfd0 con 0x7f1d9815c0b0 2024-05-08T22:37:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9d289700 1 -- 172.21.15.179:0/910202950 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d840035d0 con 0x7f1d9815c0b0 2024-05-08T22:37:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/910202950 >> v1:172.21.15.195:6789/0 conn(0x7f1d981448b0 legacy=0x7f1d980ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/910202950 >> v1:172.21.15.179:6789/0 conn(0x7f1d9815c0b0 legacy=0x7f1d980cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/910202950 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980cb6a0 con 0x7f1d9806bbc0 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/910202950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9806bbc0 0x7f1d9806c050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d90 tx=0x7f1d9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205a627b1a860d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9d289700 1 -- 172.21.15.179:0/910202950 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000e070 con 0x7f1d9806bbc0 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9d289700 1 -- 172.21.15.179:0/910202950 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9000bb20 con 0x7f1d9806bbc0 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9d289700 1 -- 172.21.15.179:0/910202950 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90012430 con 0x7f1d9806bbc0 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9f28d700 1 -- 172.21.15.179:0/910202950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9806bbc0 msgr2=0x7f1d9806c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/910202950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9806bbc0 0x7f1d9806c050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d90 tx=0x7f1d9000b330 comp rx=0 tx=0).stop 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9f28d700 1 -- 172.21.15.179:0/910202950 shutdown_connections 2024-05-08T22:37:39.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/910202950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9806bbc0 0x7f1d9806c050 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.653+0000 7f1d9f28d700 1 -- 172.21.15.179:0/910202950 >> 172.21.15.179:0/910202950 conn(0x7f1d98070d80 msgr2=0x7f1d98071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- 172.21.15.179:0/910202950 shutdown_connections 2024-05-08T22:37:39.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- 172.21.15.179:0/910202950 wait complete. 2024-05-08T22:37:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 Processor -- start 2024-05-08T22:37:39.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- start start 2024-05-08T22:37:39.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9806bbc0 0x7f1d98158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d981448b0 0x7f1d9815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d98158b80 con 0x7f1d9815c0b0 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9815bd30 con 0x7f1d981448b0 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9f28d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9815ac90 con 0x7f1d9806bbc0 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9ea8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9ea8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48640/0 (socket says 172.21.15.179:48640) 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/1019519009 learned_addr learned my addr 172.21.15.179:0/1019519009 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d981448b0 0x7f1d9815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.657+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9806bbc0 0x7f1d98158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9806bbc0 msgr2=0x7f1d98158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9806bbc0 0x7f1d98158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d981448b0 msgr2=0x7f1d9815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d981448b0 0x7f1d9815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/1019519009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009a70 con 0x7f1d9815c0b0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806cd30 tx=0x7f1d90004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d8effd700 1 -- 172.21.15.179:0/1019519009 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90012430 con 0x7f1d9815c0b0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d8effd700 1 -- 172.21.15.179:0/1019519009 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90020070 con 0x7f1d9815c0b0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9815af70 con 0x7f1d9815c0b0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d8effd700 1 -- 172.21.15.179:0/1019519009 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004280 con 0x7f1d9815c0b0 2024-05-08T22:37:39.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9815b530 con 0x7f1d9815c0b0 2024-05-08T22:37:39.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.661+0000 7f1d8effd700 1 -- 172.21.15.179:0/1019519009 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d900043e0 con 0x7f1d9815c0b0 2024-05-08T22:37:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d8effd700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 0x7f1d7c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d8effd700 1 -- 172.21.15.179:0/1019519009 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1d90022070 con 0x7f1d9815c0b0 2024-05-08T22:37:39.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 0x7f1d7c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 0x7f1d7c068bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000f60 tx=0x7f1d88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 Processor -- start 2024-05-08T22:37:39.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 -- start start 2024-05-08T22:37:39.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1cb6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 0x1cb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c44010 con 0x1cdb6f0 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bd1f40 con 0x1cde5d0 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c24e30 con 0x1cb1840 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9da8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9da8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48656/0 (socket says 172.21.15.179:48656) 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9da8a700 1 -- 172.21.15.179:0/1981845110 learned_addr learned my addr 172.21.15.179:0/1981845110 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 0x1cb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.665+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1cb6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 -- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 msgr2=0x1cb6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1cb6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 -- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 msgr2=0x1cb2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 0x1cb2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 -- 172.21.15.179:0/1981845110 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a90a40 con 0x1cdb6f0 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 0x1cb2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006da0 tx=0x7f1d94007320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8689299b93f2eefe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d7bfff700 1 -- 172.21.15.179:0/1981845110 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940012f0 con 0x1cdb6f0 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d7bfff700 1 -- 172.21.15.179:0/1981845110 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94001450 con 0x1cdb6f0 2024-05-08T22:37:39.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1d7bfff700 1 -- 172.21.15.179:0/1981845110 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940017c0 con 0x1cdb6f0 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- 172.21.15.179:0/1981845110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 msgr2=0x1cb1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006da0 tx=0x7f1d94007320 comp rx=0 tx=0).stop 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- 172.21.15.179:0/1981845110 shutdown_connections 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cde5d0 0x1cb2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cdb6f0 0x1cb1c20 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 --2- 172.21.15.179:0/1981845110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1cb6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- 172.21.15.179:0/1981845110 >> 172.21.15.179:0/1981845110 conn(0x1a455b0 msgr2=0x1cddec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- 172.21.15.179:0/1981845110 shutdown_connections 2024-05-08T22:37:39.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- 172.21.15.179:0/1981845110 wait complete. 2024-05-08T22:37:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 -- start start 2024-05-08T22:37:39.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.669+0000 7f1da379e740 1 Processor -- start 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1d06590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cdb6f0 0x1ce0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c20150 con 0x1cde5d0 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c1deb0 con 0x1cdb6f0 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1bdb8e0 con 0x1cb1840 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 10 client.0 _ll_register_callbacks cb 0x1bba670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48668/0 (socket says 172.21.15.179:48668) 2024-05-08T22:37:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/652247829 learned_addr learned my addr 172.21.15.179:0/652247829 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cdb6f0 0x1ce0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1d06590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 msgr2=0x1d06590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1d06590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cdb6f0 msgr2=0x1ce0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cdb6f0 0x1ce0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 -- 172.21.15.179:0/652247829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9815b530 con 0x1cde5d0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1d90057170 tx=0x7f1d90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f5c4bde4bad473 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90012e40 con 0x1cde5d0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1d900578e0 con 0x1cde5d0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a522c0 con 0x1cde5d0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d900571e0 con 0x1cde5d0 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 10 client.6828 fetch_fsmap learned FSMap version 32 2024-05-08T22:37:39.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x7f1d840035d0 con 0x1cde5d0 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90074020 con 0x1cde5d0 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1d90084260 con 0x1cde5d0 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 10 client.6828 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 10 client.6828 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 20 client.6828 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cb51d0 0x1d10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 4 client.6828 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.673+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1a9c540 con 0x1cb51d0 2024-05-08T22:37:39.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cb51d0 0x1d10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d797fa700 10 client.6828 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:39.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cb51d0 0x1d10d80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1304af9b6537aa9c server_cookie=ccf5c0b480a97467 in_seq=0 out_seq=0 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d797fa700 1 -- 172.21.15.179:0/652247829 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1a9c540 con 0x1cb51d0 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d797fa700 10 client.6828 handle_command_reply: tid=0 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 client.6828 shutdown 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 msgr2=0x1ce0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1d90057170 tx=0x7f1d90009750 comp rx=0 tx=0).stop 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 shutdown_connections 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cb51d0 0x1d10d80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cdb6f0 0x1ce0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1d797fa700 0 client.6828 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cde5d0 0x1ce0550 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 --2- 172.21.15.179:0/652247829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cb1840 0x1d06590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 >> 172.21.15.179:0/652247829 conn(0x1a455b0 msgr2=0x1cddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 shutdown_connections 2024-05-08T22:37:39.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 1 -- 172.21.15.179:0/652247829 wait complete. 2024-05-08T22:37:39.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.677+0000 7f1da379e740 20 client.6828 trim_cache size 0 max 16384 2024-05-08T22:37:39.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 Processor -- start 2024-05-08T22:37:39.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- start start 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 0x1d0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1cb65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ac9b80 con 0x1cb1840 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bbc620 con 0x1cb51d0 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c22fa0 con 0x1cdb6f0 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48678/0 (socket says 172.21.15.179:48678) 2024-05-08T22:37:39.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 -- 172.21.15.179:0/1392591830 learned_addr learned my addr 172.21.15.179:0/1392591830 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 0x1d0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1cb65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 -- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 msgr2=0x1cb65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1cb65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 -- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 msgr2=0x1d0d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 0x1d0d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 -- 172.21.15.179:0/1392591830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1be4290 con 0x1cb1840 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 0x1d0d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800f820 tx=0x7f1d8800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b061501cc832a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d8cff9700 1 -- 172.21.15.179:0/1392591830 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88028040 con 0x1cb1840 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d8cff9700 1 -- 172.21.15.179:0/1392591830 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88029b00 con 0x1cb1840 2024-05-08T22:37:39.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1d8cff9700 1 -- 172.21.15.179:0/1392591830 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88029240 con 0x1cb1840 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- 172.21.15.179:0/1392591830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 msgr2=0x1d0cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800f820 tx=0x7f1d8800a000 comp rx=0 tx=0).stop 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- 172.21.15.179:0/1392591830 shutdown_connections 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb51d0 0x1d0d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1cb65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 --2- 172.21.15.179:0/1392591830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb1840 0x1d0cb70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.697+0000 7f1da379e740 1 -- 172.21.15.179:0/1392591830 >> 172.21.15.179:0/1392591830 conn(0x1a455b0 msgr2=0x1ce0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- 172.21.15.179:0/1392591830 shutdown_connections 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- 172.21.15.179:0/1392591830 wait complete. 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- start start 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 Processor -- start 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 0x1d62e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1d04e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c45610 con 0x1cb51d0 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ba7a50 con 0x1cb1840 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1beec00 con 0x1cdb6f0 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 10 client.0 _ll_register_callbacks cb 0x1bd7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48682/0 (socket says 172.21.15.179:48682) 2024-05-08T22:37:39.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 -- 172.21.15.179:0/3516967520 learned_addr learned my addr 172.21.15.179:0/3516967520 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9ea8c700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1d04e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 0x1d62e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 -- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 msgr2=0x1d04e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1d04e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 -- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 msgr2=0x1d62e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 0x1d62e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 -- 172.21.15.179:0/3516967520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d980cb3c0 con 0x1cb51d0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 0x1d62e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d9da8a700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x1ba87b0 tx=0x7f1d94000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdad7a3299c74fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940017c0 con 0x1cb51d0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1d94001920 con 0x1cb51d0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d94001450 con 0x1cb51d0 2024-05-08T22:37:39.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d94001b40 con 0x1cb51d0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 10 client.6834 fetch_fsmap learned FSMap version 32 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x1bca4c0 con 0x1cb51d0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94001450 con 0x1cb51d0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1d94012070 con 0x1cb51d0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 10 client.6834 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 10 client.6834 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 20 client.6834 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cde5d0 0x1cfe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 4 client.6834 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.701+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1ac12c0 con 0x1cde5d0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cde5d0 0x1cfe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d797fa700 10 client.6834 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:39.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9e28b700 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cde5d0 0x1cfe680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c76e1b84fe070bf server_cookie=c945ebe275ee75eb in_seq=0 out_seq=0 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d797fa700 1 -- 172.21.15.179:0/3516967520 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1ac12c0 con 0x1cde5d0 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d797fa700 10 client.6834 handle_command_reply: tid=0 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 client.6834 shutdown 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 msgr2=0x1d04910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x1ba87b0 tx=0x7f1d94000ed0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 shutdown_connections 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1cde5d0 0x1cfe680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1cdb6f0 0x1d04e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1cb51d0 0x1d04910 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 --2- 172.21.15.179:0/3516967520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1cb1840 0x1d62e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d797fa700 0 client.6834 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:39.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 >> 172.21.15.179:0/3516967520 conn(0x1a455b0 msgr2=0x1d15630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 shutdown_connections 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 1 -- 172.21.15.179:0/3516967520 wait complete. 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1da379e740 20 client.6834 trim_cache size 0 max 16384 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:37:39.712 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:37:39.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 msgr2=0x7f1d7c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 0x7f1d7c068bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000f60 tx=0x7f1d88009040 comp rx=0 tx=0).stop 2024-05-08T22:37:39.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 msgr2=0x7f1d9813d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806cd30 tx=0x7f1d90004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 shutdown_connections 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d7c066730 0x7f1d7c068bf0 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000f60 tx=0x7f1d88009040 comp rx=0 tx=0).stop 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d981448b0 0x7f1d9815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d9815c0b0 0x7f1d9813d8a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 --2- 172.21.15.179:0/1019519009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9806bbc0 0x7f1d98158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 >> 172.21.15.179:0/1019519009 conn(0x7f1d98070d80 msgr2=0x7f1d9815c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 shutdown_connections 2024-05-08T22:37:39.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.705+0000 7f1d9f28d700 1 -- 172.21.15.179:0/1019519009 wait complete. 2024-05-08T22:37:39.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:37:39.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:39.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 Processor -- start 2024-05-08T22:37:39.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- start start 2024-05-08T22:37:39.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe33c15ef80 0x7fe33c161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33c165d50 con 0x7fe33c164db0 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c06ad70 con 0x7fe33c15ef80 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c165260 con 0x7fe33c06de70 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe33c15ef80 0x7fe33c161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe33c15ef80 0x7fe33c161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48684/0 (socket says 172.21.15.179:48684) 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 -- 172.21.15.179:0/2131044229 learned_addr learned my addr 172.21.15.179:0/2131044229 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe340955700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 -- 172.21.15.179:0/2131044229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 msgr2=0x7fe33c06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33affd700 1 -- 172.21.15.179:0/2131044229 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202704174 0 0) 0x7fe33c165d50 con 0x7fe33c164db0 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c06b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 -- 172.21.15.179:0/2131044229 >> v1:172.21.15.195:6789/0 conn(0x7fe33c164db0 legacy=0x7fe33c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 -- 172.21.15.179:0/2131044229 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c06e4b0 con 0x7fe33c15ef80 2024-05-08T22:37:39.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33bfff700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c161790 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe324009910 tx=0x7fe324009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1961fd9b7b1a3d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33affd700 1 -- 172.21.15.179:0/2131044229 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324011830 con 0x7fe33c15ef80 2024-05-08T22:37:39.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33affd700 1 -- 172.21.15.179:0/2131044229 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe324011990 con 0x7fe33c15ef80 2024-05-08T22:37:39.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe33affd700 1 -- 172.21.15.179:0/2131044229 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324011d40 con 0x7fe33c15ef80 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- 172.21.15.179:0/2131044229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 msgr2=0x7fe33c161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c161790 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe324009910 tx=0x7fe324009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- 172.21.15.179:0/2131044229 shutdown_connections 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c06b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 --2- 172.21.15.179:0/2131044229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c161790 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- 172.21.15.179:0/2131044229 >> 172.21.15.179:0/2131044229 conn(0x7fe33c0ce9d0 msgr2=0x7fe33c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- 172.21.15.179:0/2131044229 shutdown_connections 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.845+0000 7fe341156700 1 -- 172.21.15.179:0/2131044229 wait complete. 2024-05-08T22:37:39.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 Processor -- start 2024-05-08T22:37:39.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- start start 2024-05-08T22:37:39.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c1628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33c164db0 0x7fe33c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15d890 con 0x7fe33c15ef80 2024-05-08T22:37:39.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0c8e60 con 0x7fe33c164db0 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15d5b0 con 0x7fe33c06de70 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48698/0 (socket says 172.21.15.179:48698) 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 -- 172.21.15.179:0/2000018821 learned_addr learned my addr 172.21.15.179:0/2000018821 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33bfff700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c1628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe340955700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33c164db0 0x7fe33c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 -- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 msgr2=0x7fe33c1628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c1628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 -- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33c164db0 msgr2=0x7fe33c15c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33c164db0 0x7fe33c15c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 -- 172.21.15.179:0/2000018821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324009570 con 0x7fe33c15ef80 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe33b7fe700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe33400cc50 tx=0x7fe33400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334003e50 con 0x7fe33c15ef80 2024-05-08T22:37:39.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33400a6c0 con 0x7fe33c15ef80 2024-05-08T22:37:39.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c0c9100 con 0x7fe33c15ef80 2024-05-08T22:37:39.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400aa70 con 0x7fe33c15ef80 2024-05-08T22:37:39.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.849+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c14ab60 con 0x7fe33c15ef80 2024-05-08T22:37:39.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c0c8ca0 con 0x7fe33c15ef80 2024-05-08T22:37:39.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe33400abd0 con 0x7fe33c15ef80 2024-05-08T22:37:39.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe338ff9700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 0x7fe31c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:39.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe334083e70 con 0x7fe33c15ef80 2024-05-08T22:37:39.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe33bfff700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 0x7fe31c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:39.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.853+0000 7fe33bfff700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 0x7fe31c068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe324008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:39.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:39.861+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe334087030 con 0x7fe33c15ef80 2024-05-08T22:37:40.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.073+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe33c067740 con 0x7fe33c15ef80 2024-05-08T22:37:40.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.073+0000 7fe338ff9700 1 -- 172.21.15.179:0/2000018821 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4749 (secure 0 0 0) 0x7fe334017070 con 0x7fe33c15ef80 2024-05-08T22:37:40.075 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:40.076 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:37:13.599733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.073+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 msgr2=0x7fe31c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.073+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 0x7fe31c068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe324008030 comp rx=0 tx=0).stop 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 msgr2=0x7fe33c162e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe33400cc50 tx=0x7fe33400cf60 comp rx=0 tx=0).stop 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 shutdown_connections 2024-05-08T22:37:40.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe31c066730 0x7fe31c068bf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33c164db0 0x7fe33c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe33c15ef80 0x7fe33c162e10 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 --2- 172.21.15.179:0/2000018821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe33c06de70 0x7fe33c1628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 >> 172.21.15.179:0/2000018821 conn(0x7fe33c0ce9d0 msgr2=0x7fe33c161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 shutdown_connections 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.077+0000 7fe341156700 1 -- 172.21.15.179:0/2000018821 wait complete. 2024-05-08T22:37:40.079 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 32 2024-05-08T22:37:40.090 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-08T22:37:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.201+0000 7feb09e06700 1 Processor -- start 2024-05-08T22:37:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.201+0000 7feb09e06700 1 -- start start 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.201+0000 7feb09e06700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feb041606b0 0x7feb040b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.201+0000 7feb09e06700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb0406bcd0 0x7feb04161180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0406c560 con 0x7feb04160d20 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040b6600 con 0x7feb0406bcd0 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040d1a70 con 0x7feb041606b0 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb0406bcd0 0x7feb04161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb0406bcd0 0x7feb04161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48708/0 (socket says 172.21.15.179:48708) 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb03fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7feb04160d20 0x7feb0406b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37822/0 (socket says 172.21.15.179:37822) 2024-05-08T22:37:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 -- 172.21.15.179:0/1827308577 learned_addr learned my addr 172.21.15.179:0/1827308577 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb08e04700 1 --2- 172.21.15.179:0/1827308577 >> v2:172.21.15.195:3300/0 conn(0x7feb041606b0 0x7feb040b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 -- 172.21.15.179:0/1827308577 >> v2:172.21.15.195:3300/0 conn(0x7feb041606b0 msgr2=0x7feb040b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 --2- 172.21.15.179:0/1827308577 >> v2:172.21.15.195:3300/0 conn(0x7feb041606b0 0x7feb040b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 -- 172.21.15.179:0/1827308577 >> v1:172.21.15.179:6789/0 conn(0x7feb04160d20 legacy=0x7feb0406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 -- 172.21.15.179:0/1827308577 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb041616c0 con 0x7feb0406bcd0 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb037fe700 1 -- 172.21.15.179:0/1827308577 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722489231 0 0) 0x7feb0406c560 con 0x7feb04160d20 2024-05-08T22:37:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09605700 1 --2- 172.21.15.179:0/1827308577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb0406bcd0 0x7feb04161180 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7feb040d3e10 tx=0x7feafc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0082b26f074e490 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb037fe700 1 -- 172.21.15.179:0/1827308577 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00e070 con 0x7feb0406bcd0 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb037fe700 1 -- 172.21.15.179:0/1827308577 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00b9c0 con 0x7feb0406bcd0 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb037fe700 1 -- 172.21.15.179:0/1827308577 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00bd70 con 0x7feb0406bcd0 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- 172.21.15.179:0/1827308577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb0406bcd0 msgr2=0x7feb04161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 --2- 172.21.15.179:0/1827308577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb0406bcd0 0x7feb04161180 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7feb040d3e10 tx=0x7feafc00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- 172.21.15.179:0/1827308577 shutdown_connections 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 --2- 172.21.15.179:0/1827308577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb0406bcd0 0x7feb04161180 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 --2- 172.21.15.179:0/1827308577 >> v2:172.21.15.195:3300/0 conn(0x7feb041606b0 0x7feb040b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- 172.21.15.179:0/1827308577 >> 172.21.15.179:0/1827308577 conn(0x7feb040b49a0 msgr2=0x7feb040b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- 172.21.15.179:0/1827308577 shutdown_connections 2024-05-08T22:37:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.205+0000 7feb09e06700 1 -- 172.21.15.179:0/1827308577 wait complete. 2024-05-08T22:37:40.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 Processor -- start 2024-05-08T22:37:40.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- start start 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb0406bcd0 0x7feb0415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb041606b0 0x7feb04150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415d660 con 0x7feb04160d20 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415d940 con 0x7feb0406bcd0 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415dc20 con 0x7feb041606b0 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48716/0 (socket says 172.21.15.179:48716) 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 -- 172.21.15.179:0/3170423616 learned_addr learned my addr 172.21.15.179:0/3170423616 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb08e04700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb0406bcd0 0x7feb0415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb03fff700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb041606b0 0x7feb04150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 -- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb041606b0 msgr2=0x7feb04150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb041606b0 0x7feb04150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 -- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb0406bcd0 msgr2=0x7feb0415c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb0406bcd0 0x7feb0415c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 -- 172.21.15.179:0/3170423616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009b30 con 0x7feb04160d20 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09605700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7feb0408afe0 tx=0x7feafc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb017fa700 1 -- 172.21.15.179:0/3170423616 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc004840 con 0x7feb04160d20 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb017fa700 1 -- 172.21.15.179:0/3170423616 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc0092e0 con 0x7feb04160d20 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb017fa700 1 -- 172.21.15.179:0/3170423616 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc01b650 con 0x7feb04160d20 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb04153280 con 0x7feb04160d20 2024-05-08T22:37:40.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.209+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04151800 con 0x7feb04160d20 2024-05-08T22:37:40.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.213+0000 7feb017fa700 1 -- 172.21.15.179:0/3170423616 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feafc01b7b0 con 0x7feb04160d20 2024-05-08T22:37:40.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.213+0000 7feb017fa700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 0x7feaf0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.213+0000 7feb017fa700 1 -- 172.21.15.179:0/3170423616 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7feafc085840 con 0x7feb04160d20 2024-05-08T22:37:40.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.213+0000 7feb08e04700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 0x7feaf0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb08e04700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 0x7feaf0068bf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feb041f6700 tx=0x7feaf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 Processor -- start 2024-05-08T22:37:40.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- start start 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1b35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b35a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a60810 con 0x1b35fb0 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a5dfa0 con 0x1a81520 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a59890 con 0x1b35690 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48724/0 (socket says 172.21.15.179:48724) 2024-05-08T22:37:40.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 -- 172.21.15.179:0/1690878131 learned_addr learned my addr 172.21.15.179:0/1690878131 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb08e04700 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1b35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb03fff700 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b35a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 -- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 msgr2=0x1b35a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b35a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 -- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 msgr2=0x1b35150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1b35150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 -- 172.21.15.179:0/1690878131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18cbba0 con 0x1b35fb0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb09605700 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7feafc056390 tx=0x7feafc052ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff64c3c1e350b42f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feaea7fc700 1 -- 172.21.15.179:0/1690878131 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc056d90 con 0x1b35fb0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feaea7fc700 1 -- 172.21.15.179:0/1690878131 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc056690 con 0x1b35fb0 2024-05-08T22:37:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feaea7fc700 1 -- 172.21.15.179:0/1690878131 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc05a4c0 con 0x1b35fb0 2024-05-08T22:37:40.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- 172.21.15.179:0/1690878131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 msgr2=0x1b36390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7feafc056390 tx=0x7feafc052ad0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- 172.21.15.179:0/1690878131 shutdown_connections 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b36390 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b35a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 --2- 172.21.15.179:0/1690878131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1b35150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.217+0000 7feb0e317740 1 -- 172.21.15.179:0/1690878131 >> 172.21.15.179:0/1690878131 conn(0x1880710 msgr2=0x1b1a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- 172.21.15.179:0/1690878131 shutdown_connections 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- 172.21.15.179:0/1690878131 wait complete. 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- start start 2024-05-08T22:37:40.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 Processor -- start 2024-05-08T22:37:40.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1af24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1cd80 con 0x1b35690 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1ca20 con 0x1b35fb0 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x194c770 con 0x1a81520 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 10 client.0 _ll_register_callbacks cb 0x1a52f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48738/0 (socket says 172.21.15.179:48738) 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 -- 172.21.15.179:0/3077684805 learned_addr learned my addr 172.21.15.179:0/3077684805 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb08e04700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1af24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb09605700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 -- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 msgr2=0x1af24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1af24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 -- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 msgr2=0x1b31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 -- 172.21.15.179:0/3077684805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04151800 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb09605700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb03fff700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7feaf40078c0 tx=0x7feaf4001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af89f47e5623ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40145b0 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7feaf400c950 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x188d420 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf400cb10 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 10 client.6855 fetch_fsmap learned FSMap version 32 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x1a1f910 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.221+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4024020 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7feaf4012050 con 0x1b35690 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 10 client.6855 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T22:37:40.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 10 client.6855 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 20 client.6855 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b3ed40 0x1b3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 4 client.6855 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18d76a0 con 0x1b3ed40 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb08e04700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b3ed40 0x1b3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feae37fe700 10 client.6855 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:40.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb08e04700 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b3ed40 0x1b3f120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e3ad0bf55dd3a2 server_cookie=615643e9c7b1ca15 in_seq=0 out_seq=0 2024-05-08T22:37:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feae37fe700 1 -- 172.21.15.179:0/3077684805 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x18d76a0 con 0x1b3ed40 2024-05-08T22:37:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feae37fe700 10 client.6855 handle_command_reply: tid=0 2024-05-08T22:37:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 client.6855 shutdown 2024-05-08T22:37:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 msgr2=0x1af1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7feaf40078c0 tx=0x7feaf4001700 comp rx=0 tx=0).stop 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 shutdown_connections 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b3ed40 0x1b3f120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feae37fe700 0 client.6855 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1af1400 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 --2- 172.21.15.179:0/3077684805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1af24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 >> 172.21.15.179:0/3077684805 conn(0x1880710 msgr2=0x1b18090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 shutdown_connections 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 1 -- 172.21.15.179:0/3077684805 wait complete. 2024-05-08T22:37:40.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.225+0000 7feb0e317740 20 client.6855 trim_cache size 0 max 16384 2024-05-08T22:37:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 Processor -- start 2024-05-08T22:37:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 -- start start 2024-05-08T22:37:40.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1b4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b4dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a215d0 con 0x1b35690 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a241c0 con 0x1b35fb0 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a88310 con 0x1a81520 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48744/0 (socket says 172.21.15.179:48744) 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 -- 172.21.15.179:0/3070830847 learned_addr learned my addr 172.21.15.179:0/3070830847 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb09605700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b4dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb08e04700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1b4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 -- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 msgr2=0x1b4d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1b4d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 -- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 msgr2=0x1b4dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b4dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 -- 172.21.15.179:0/3070830847 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a05b40 con 0x1b35690 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb08e04700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1b4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feb03fff700 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000fe0 tx=0x7feaf400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61591ff8bf4a9e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feaeaffd700 1 -- 172.21.15.179:0/3070830847 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400ce50 con 0x1b35690 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feaeaffd700 1 -- 172.21.15.179:0/3070830847 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4007e90 con 0x1b35690 2024-05-08T22:37:40.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.245+0000 7feaeaffd700 1 -- 172.21.15.179:0/3070830847 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4014e20 con 0x1b35690 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- 172.21.15.179:0/3070830847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 msgr2=0x1b4da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000fe0 tx=0x7feaf400f000 comp rx=0 tx=0).stop 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- 172.21.15.179:0/3070830847 shutdown_connections 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b35fb0 0x1b4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35690 0x1b4da60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 --2- 172.21.15.179:0/3070830847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a81520 0x1b4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- 172.21.15.179:0/3070830847 >> 172.21.15.179:0/3070830847 conn(0x1880710 msgr2=0x1aeefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- 172.21.15.179:0/3070830847 shutdown_connections 2024-05-08T22:37:40.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- 172.21.15.179:0/3070830847 wait complete. 2024-05-08T22:37:40.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 -- start start 2024-05-08T22:37:40.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.249+0000 7feb0e317740 1 Processor -- start 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1ba2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b50c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a2a9a0 con 0x1b35fb0 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b60760 con 0x1a81520 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b5f620 con 0x1b35690 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 10 client.0 _ll_register_callbacks cb 0x1a58460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48758/0 (socket says 172.21.15.179:48758) 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 -- 172.21.15.179:0/2190969877 learned_addr learned my addr 172.21.15.179:0/2190969877 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb08e04700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1ba2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb03fff700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b50c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 -- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 msgr2=0x1b50c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b50c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 -- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 msgr2=0x1ba2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1ba2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 -- 172.21.15.179:0/2190969877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a1f910 con 0x1b35fb0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb09605700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feafc056620 tx=0x7feafc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91be6e1496bb33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc082070 con 0x1b35fb0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7feafc056d90 con 0x1b35fb0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feafc056690 con 0x1b35fb0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feafc0099b0 con 0x1b35fb0 2024-05-08T22:37:40.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 10 client.6861 fetch_fsmap learned FSMap version 32 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x19f9c20 con 0x1b35fb0 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc056690 con 0x1b35fb0 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7feafc05a080 con 0x1b35fb0 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 10 client.6861 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 10 client.6861 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 20 client.6861 populate_metadata read hostname 'smithi179' 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b4eb20 0x1b4ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 4 client.6861 mds_command: new command op to 55718 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:37:40.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 --> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x18fc420 con 0x1b4eb20 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb08e04700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b4eb20 0x1b4ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb08e04700 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b4eb20 0x1b4ef00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd54ec6cef8b9207 server_cookie=f0282c31fa3e66a in_seq=0 out_seq=0 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 10 client.6861 ms_handle_connect on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 1 -- 172.21.15.179:0/2190969877 <== mds.0 v2:172.21.15.195:6834/3517137124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x18fc420 con 0x1b4eb20 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feae37fe700 10 client.6861 handle_command_reply: tid=0 2024-05-08T22:37:40.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.253+0000 7feb0e317740 1 client.6861 shutdown 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 msgr2=0x1b51150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feafc056620 tx=0x7feafc00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 shutdown_connections 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:6834/3517137124,v1:172.21.15.195:6836/3517137124] conn(0x1b4eb20 0x1b4ef00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b35fb0 0x1b51150 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feae37fe700 0 client.6861 ms_handle_reset on v2:172.21.15.195:6834/3517137124 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b35690 0x1b50c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 --2- 172.21.15.179:0/2190969877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a81520 0x1ba2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 >> 172.21.15.179:0/2190969877 conn(0x1880710 msgr2=0x1b30be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 shutdown_connections 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 1 -- 172.21.15.179:0/2190969877 wait complete. 2024-05-08T22:37:40.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb0e317740 20 client.6861 trim_cache size 0 max 16384 2024-05-08T22:37:40.262 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:40.262 INFO:teuthology.orchestra.run.smithi179.stdout:[] 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 msgr2=0x7feaf0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 0x7feaf0068bf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feb041f6700 tx=0x7feaf8008040 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 msgr2=0x7feb04157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7feb0408afe0 tx=0x7feafc004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 shutdown_connections 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7feaf0066730 0x7feaf0068bf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb04160d20 0x7feb04157680 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb041606b0 0x7feb04150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 --2- 172.21.15.179:0/3170423616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb0406bcd0 0x7feb0415c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 >> 172.21.15.179:0/3170423616 conn(0x7feb040b49a0 msgr2=0x7feb040b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 shutdown_connections 2024-05-08T22:37:40.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.257+0000 7feb09e06700 1 -- 172.21.15.179:0/3170423616 wait complete. 2024-05-08T22:37:40.275 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:37:40.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.389+0000 7fe2b7fff700 1 Processor -- start 2024-05-08T22:37:40.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.389+0000 7fe2b7fff700 1 -- start start 2024-05-08T22:37:40.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe2b815ef80 0x7fe2b8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b806de70 0x7fe2b806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b8165d50 con 0x7fe2b8164db0 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b806ad70 con 0x7fe2b815ef80 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8165260 con 0x7fe2b806de70 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b67fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe2b8164db0 0x7fe2b806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37836/0 (socket says 172.21.15.179:37836) 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b67fc700 1 -- 172.21.15.179:0/2274587467 learned_addr learned my addr 172.21.15.179:0/2274587467 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/2274587467 >> v2:172.21.15.195:3300/0 conn(0x7fe2b815ef80 0x7fe2b8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b77fe700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b806de70 0x7fe2b806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071967085 0 0) 0x7fe2b8165d50 con 0x7fe2b8164db0 2024-05-08T22:37:40.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a40035d0 con 0x7fe2b8164db0 2024-05-08T22:37:40.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b806de70 msgr2=0x7fe2b806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b806de70 0x7fe2b806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875123390 0 0) 0x7fe2a40035d0 con 0x7fe2b8164db0 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/2274587467 >> v1:172.21.15.179:6789/0 conn(0x7fe2b8164db0 legacy=0x7fe2b806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/2274587467 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b806e4b0 con 0x7fe2b815ef80 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b8161790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009850 tx=0x7fe2a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbffabcaacd357b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0001670 con 0x7fe2b815ef80 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8004030 con 0x7fe2b815ef80 2024-05-08T22:37:40.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b5ffb700 1 -- 172.21.15.179:0/2274587467 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80043e0 con 0x7fe2b815ef80 2024-05-08T22:37:40.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 msgr2=0x7fe2b8161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b8161790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009850 tx=0x7fe2a8009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:40.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- 172.21.15.179:0/2274587467 shutdown_connections 2024-05-08T22:37:40.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b806de70 0x7fe2b806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/2274587467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b8161790 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- 172.21.15.179:0/2274587467 >> 172.21.15.179:0/2274587467 conn(0x7fe2b80ce9d0 msgr2=0x7fe2b80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- 172.21.15.179:0/2274587467 shutdown_connections 2024-05-08T22:37:40.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.393+0000 7fe2b7fff700 1 -- 172.21.15.179:0/2274587467 wait complete. 2024-05-08T22:37:40.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 Processor -- start 2024-05-08T22:37:40.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- start start 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b8164db0 0x7fe2b8162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8155070 con 0x7fe2b806de70 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81eae40 con 0x7fe2b815ef80 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81eb120 con 0x7fe2b8164db0 2024-05-08T22:37:40.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48766/0 (socket says 172.21.15.179:48766) 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/4252874787 learned_addr learned my addr 172.21.15.179:0/4252874787 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b77fe700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b8164db0 0x7fe2b8162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b67fc700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b8164db0 msgr2=0x7fe2b8162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b8164db0 0x7fe2b8162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 msgr2=0x7fe2b815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 -- 172.21.15.179:0/4252874787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80094b0 con 0x7fe2b806de70 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b6ffd700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003d10 con 0x7fe2b806de70 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a801f070 con 0x7fe2b806de70 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81eb400 con 0x7fe2b806de70 2024-05-08T22:37:40.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8011870 con 0x7fe2b806de70 2024-05-08T22:37:40.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.397+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8154b80 con 0x7fe2b806de70 2024-05-08T22:37:40.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8067740 con 0x7fe2b806de70 2024-05-08T22:37:40.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2a80119d0 con 0x7fe2b806de70 2024-05-08T22:37:40.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2af7fe700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 0x7fe298068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe2a8021070 con 0x7fe2b806de70 2024-05-08T22:37:40.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2b67fc700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 0x7fe298068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.401+0000 7fe2b67fc700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 0x7fe298068bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe2b81fa660 tx=0x7fe2a0001e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.409+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe2a8052ce0 con 0x7fe2b806de70 2024-05-08T22:37:40.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe2b804e700 con 0x7fe2b806de70 2024-05-08T22:37:40.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2af7fe700 1 -- 172.21.15.179:0/4252874787 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe2a8056300 con 0x7fe2b806de70 2024-05-08T22:37:40.619 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:40.619 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:37:40.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 msgr2=0x7fe298068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 0x7fe298068bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe2b81fa660 tx=0x7fe2a0001e00 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 msgr2=0x7fe2b8154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8004770 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 shutdown_connections 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe298066730 0x7fe298068bf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b815ef80 0x7fe2b815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2b8164db0 0x7fe2b8162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 --2- 172.21.15.179:0/4252874787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2b806de70 0x7fe2b8154640 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 >> 172.21.15.179:0/4252874787 conn(0x7fe2b80ce9d0 msgr2=0x7fe2b8163c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 shutdown_connections 2024-05-08T22:37:40.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.617+0000 7fe2b7fff700 1 -- 172.21.15.179:0/4252874787 wait complete. 2024-05-08T22:37:40.634 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:37:40.634 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:37:40.634 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:37:40.648 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:37:40.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:37:40.668 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:37:40.668 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:40.697 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:40.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 Processor -- start 2024-05-08T22:37:40.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- start start 2024-05-08T22:37:40.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f54400d0a20 0x7f54400cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54400b01b0 con 0x7f544015b1e0 2024-05-08T22:37:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54400cc410 con 0x7f54400cbea0 2024-05-08T22:37:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400cc6f0 con 0x7f54400d0a20 2024-05-08T22:37:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f54464aa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f54400d0a20 0x7f54400cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5445ca9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f544015b1e0 0x7f54400b1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37838/0 (socket says 172.21.15.179:37838) 2024-05-08T22:37:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5445ca9700 1 -- 172.21.15.179:0/2685026845 learned_addr learned my addr 172.21.15.179:0/2685026845 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:40.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f54464aa700 1 -- 172.21.15.179:0/2685026845 >> v1:172.21.15.195:6789/0 conn(0x7f54400cbea0 legacy=0x7f54400d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f54464aa700 1 -- 172.21.15.179:0/2685026845 >> v1:172.21.15.179:6789/0 conn(0x7f544015b1e0 legacy=0x7f54400b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5444ca7700 1 -- 172.21.15.179:0/2685026845 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073541496 0 0) 0x7f54400cc410 con 0x7f54400cbea0 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f54464aa700 1 -- 172.21.15.179:0/2685026845 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400b0f60 con 0x7f54400d0a20 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f54464aa700 1 --2- 172.21.15.179:0/2685026845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400d0a20 0x7f54400cb7c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f543c009850 tx=0x7f543c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3c5574872f18b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5444ca7700 1 -- 172.21.15.179:0/2685026845 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c011830 con 0x7f54400d0a20 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5444ca7700 1 -- 172.21.15.179:0/2685026845 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c011990 con 0x7f54400d0a20 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5444ca7700 1 -- 172.21.15.179:0/2685026845 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c011d20 con 0x7f54400d0a20 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- 172.21.15.179:0/2685026845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400d0a20 msgr2=0x7f54400cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 --2- 172.21.15.179:0/2685026845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400d0a20 0x7f54400cb7c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f543c009850 tx=0x7f543c009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- 172.21.15.179:0/2685026845 shutdown_connections 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 --2- 172.21.15.179:0/2685026845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400d0a20 0x7f54400cb7c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- 172.21.15.179:0/2685026845 >> 172.21.15.179:0/2685026845 conn(0x7f54400cd8c0 msgr2=0x7f54400cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- 172.21.15.179:0/2685026845 shutdown_connections 2024-05-08T22:37:40.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.813+0000 7f5447f0d700 1 -- 172.21.15.179:0/2685026845 wait complete. 2024-05-08T22:37:40.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 Processor -- start 2024-05-08T22:37:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 -- start start 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54400d0a20 0x7f544014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f544015b1e0 0x7f54400b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544014bd60 con 0x7f54400cbea0 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544014f020 con 0x7f54400d0a20 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.817+0000 7f5447f0d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544015ce00 con 0x7f544015b1e0 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48786/0 (socket says 172.21.15.179:48786) 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 -- 172.21.15.179:0/4261592630 learned_addr learned my addr 172.21.15.179:0/4261592630 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54454a8700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54400d0a20 0x7f544014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54464aa700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f544015b1e0 0x7f54400b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 -- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f544015b1e0 msgr2=0x7f54400b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f544015b1e0 0x7f54400b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 -- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54400d0a20 msgr2=0x7f544014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54400d0a20 0x7f544014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 -- 172.21.15.179:0/4261592630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0094b0 con 0x7f54400cbea0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5445ca9700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5434001eb0 tx=0x7f543400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434005d90 con 0x7f54400cbea0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54340070d0 con 0x7f54400cbea0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544014ea60 con 0x7f54400cbea0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434007440 con 0x7f54400cbea0 2024-05-08T22:37:40.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544015ca90 con 0x7f54400cbea0 2024-05-08T22:37:40.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544015c8f0 con 0x7f54400cbea0 2024-05-08T22:37:40.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.821+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54340075a0 con 0x7f54400cbea0 2024-05-08T22:37:40.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.825+0000 7f54327fc700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 0x7f5424068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:40.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.825+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f5434083880 con 0x7f54400cbea0 2024-05-08T22:37:40.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.825+0000 7f54454a8700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 0x7f5424068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:40.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.825+0000 7f54454a8700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 0x7f5424068bf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f54400cba30 tx=0x7f5438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:40.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:40.829+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5434050bb0 con 0x7f54400cbea0 2024-05-08T22:37:41.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.001+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f544015c8f0 con 0x7f54400cbea0 2024-05-08T22:37:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f54327fc700 1 -- 172.21.15.179:0/4261592630 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16535 (secure 0 0 0) 0x7f544015c8f0 con 0x7f54400cbea0 2024-05-08T22:37:41.007 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:41.007 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":71,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:06.562633+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 msgr2=0x7f5424068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 0x7f5424068bf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f54400cba30 tx=0x7f5438009040 comp rx=0 tx=0).stop 2024-05-08T22:37:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 msgr2=0x7f544015d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5434001eb0 tx=0x7f543400c7b0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 shutdown_connections 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5424066730 0x7f5424068bf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54400d0a20 0x7f544014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f544015b1e0 0x7f54400b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 --2- 172.21.15.179:0/4261592630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54400cbea0 0x7f544015d290 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 >> 172.21.15.179:0/4261592630 conn(0x7f54400cd8c0 msgr2=0x7f54400cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 shutdown_connections 2024-05-08T22:37:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.005+0000 7f5447f0d700 1 -- 172.21.15.179:0/4261592630 wait complete. 2024-05-08T22:37:41.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:36:23.437690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:36:24.076455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:37:41.022 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:37:41.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 Processor -- start 2024-05-08T22:37:41.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 -- start start 2024-05-08T22:37:41.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06d4142890 0x7f06d40b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d40b22d0 con 0x7f06d40b2ca0 2024-05-08T22:37:41.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d40b19e0 con 0x7f06d4142890 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d8a4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d415e640 con 0x7f06d406d4b0 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d2d9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06d4142890 0x7f06d40b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:50988/0 (socket says 172.21.15.179:50988) 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 -- 172.21.15.179:0/2109114860 learned_addr learned my addr 172.21.15.179:0/2109114860 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d259c700 1 --1- 172.21.15.179:0/2109114860 >> v1:172.21.15.195:6789/0 conn(0x7f06d40b2ca0 0x7f06d406cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46118/0 (socket says 172.21.15.179:46118) 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 -- 172.21.15.179:0/2109114860 >> v2:172.21.15.195:3300/0 conn(0x7f06d4142890 msgr2=0x7f06d40b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 --2- 172.21.15.179:0/2109114860 >> v2:172.21.15.195:3300/0 conn(0x7f06d4142890 0x7f06d40b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 -- 172.21.15.179:0/2109114860 >> v1:172.21.15.195:6789/0 conn(0x7f06d40b2ca0 legacy=0x7f06d406cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.137+0000 7f06d359e700 1 -- 172.21.15.179:0/2109114860 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40b1c80 con 0x7f06d406d4b0 2024-05-08T22:37:41.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d359e700 1 --2- 172.21.15.179:0/2109114860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009e50 tx=0x7f06cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74edcfe2fe2100cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d1d9b700 1 -- 172.21.15.179:0/2109114860 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00e070 con 0x7f06d406d4b0 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d1d9b700 1 -- 172.21.15.179:0/2109114860 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc00b910 con 0x7f06d406d4b0 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d1d9b700 1 -- 172.21.15.179:0/2109114860 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00bcc0 con 0x7f06d406d4b0 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2109114860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 msgr2=0x7f06d415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2109114860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009e50 tx=0x7f06cc00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2109114860 shutdown_connections 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2109114860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d406d4b0 0x7f06d415d830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2109114860 >> v2:172.21.15.195:3300/0 conn(0x7f06d4142890 0x7f06d40b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2109114860 >> 172.21.15.179:0/2109114860 conn(0x7f06d40cea30 msgr2=0x7f06d40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2109114860 shutdown_connections 2024-05-08T22:37:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.141+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2109114860 wait complete. 2024-05-08T22:37:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 Processor -- start 2024-05-08T22:37:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- start start 2024-05-08T22:37:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d40b2ca0 0x7f06d41f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d4142890 0x7f06d41413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4139a40 con 0x7f06d406d4b0 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d41377d0 con 0x7f06d40b2ca0 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4137ab0 con 0x7f06d4142890 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48794/0 (socket says 172.21.15.179:48794) 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 -- 172.21.15.179:0/2249263946 learned_addr learned my addr 172.21.15.179:0/2249263946 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d259c700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d40b2ca0 0x7f06d41f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d359e700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d4142890 0x7f06d41413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 -- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d4142890 msgr2=0x7f06d41413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d4142890 0x7f06d41413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 -- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d40b2ca0 msgr2=0x7f06d41f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d40b2ca0 0x7f06d41f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 -- 172.21.15.179:0/2249263946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc009b30 con 0x7f06d406d4b0 2024-05-08T22:37:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d2d9d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e840 tx=0x7f06c800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80095a0 con 0x7f06d406d4b0 2024-05-08T22:37:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8013680 con 0x7f06d406d4b0 2024-05-08T22:37:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d4137d90 con 0x7f06d406d4b0 2024-05-08T22:37:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80139f0 con 0x7f06d406d4b0 2024-05-08T22:37:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.145+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d41f3be0 con 0x7f06d406d4b0 2024-05-08T22:37:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.149+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d404e700 con 0x7f06d406d4b0 2024-05-08T22:37:41.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.149+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06c8013b50 con 0x7f06d406d4b0 2024-05-08T22:37:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.149+0000 7f06bb7fe700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 0x7f06c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.149+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f06c8084200 con 0x7f06d406d4b0 2024-05-08T22:37:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.149+0000 7f06d259c700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 0x7f06c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.153+0000 7f06d259c700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 0x7f06c0068bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f06d415a590 tx=0x7f06c4007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.157+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f06c8051310 con 0x7f06d406d4b0 2024-05-08T22:37:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.325+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06d415aa70 con 0x7f06d406d4b0 2024-05-08T22:37:41.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.325+0000 7f06bb7fe700 1 -- 172.21.15.179:0/2249263946 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06c8054930 con 0x7f06d406d4b0 2024-05-08T22:37:41.328 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 msgr2=0x7f06c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 0x7f06c0068bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f06d415a590 tx=0x7f06c4007ef0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 msgr2=0x7f06d4139500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e840 tx=0x7f06c800eb50 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 shutdown_connections 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c0066730 0x7f06c0068bf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d40b2ca0 0x7f06d41f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06d4142890 0x7f06d41413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 --2- 172.21.15.179:0/2249263946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d406d4b0 0x7f06d4139500 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 >> 172.21.15.179:0/2249263946 conn(0x7f06d40cea30 msgr2=0x7f06d40afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 shutdown_connections 2024-05-08T22:37:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.329+0000 7f06d8a4d700 1 -- 172.21.15.179:0/2249263946 wait complete. 2024-05-08T22:37:41.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:37:41.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 Processor -- start 2024-05-08T22:37:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 -- start start 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc160800 0x7fa3fc0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc160be0 con 0x7fa3fc06c390 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc0c60e0 con 0x7fa3fc0afc50 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa402fa6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0b4070 con 0x7fa3fc160800 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa4017a3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa3fc06c390 0x7fa3fc0b4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37846/0 (socket says 172.21.15.179:37846) 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa4017a3700 1 -- 172.21.15.179:0/2823470032 learned_addr learned my addr 172.21.15.179:0/2823470032 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa401fa4700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc160800 0x7fa3fc0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290040222 0 0) 0x7fa3fc160be0 con 0x7fa3fc06c390 2024-05-08T22:37:41.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e80035d0 con 0x7fa3fc06c390 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474658007 0 0) 0x7fa3fc0c60e0 con 0x7fa3fc0afc50 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.457+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3fc160be0 con 0x7fa3fc0afc50 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177139461 0 0) 0x7fa3e80035d0 con 0x7fa3fc06c390 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc160800 msgr2=0x7fa3fc0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc160800 0x7fa3fc0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 >> v1:172.21.15.195:6789/0 conn(0x7fa3fc0afc50 legacy=0x7fa3fc0ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc06c910 con 0x7fa3fc06c390 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fa3f0001670 con 0x7fa3fc06c390 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 >> v1:172.21.15.179:6789/0 conn(0x7fa3fc06c390 legacy=0x7fa3fc0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8003be0 0x7fa3e8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e8004870 0x7fa3e8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e8005230 0x7fa3e800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0b4070 con 0x7fa3e8004870 2024-05-08T22:37:41.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e800c020 con 0x7fa3e8003be0 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e800c1e0 con 0x7fa3e8005230 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa401fa4700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc160800 0x7fa3fc0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e8004870 0x7fa3e8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4027a5700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e8005230 0x7fa3e800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa401fa4700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8003be0 0x7fa3e8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 -- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e8005230 msgr2=0x7fa3e800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e8005230 0x7fa3e800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 -- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8003be0 msgr2=0x7fa3e8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8003be0 0x7fa3e8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 -- 172.21.15.179:0/2823470032 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80035d0 con 0x7fa3e8004870 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4017a3700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e8004870 0x7fa3e8004cf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c210 tx=0x7fa3e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbab26c97461c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000cdb0 con 0x7fa3e8004870 2024-05-08T22:37:41.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005980 con 0x7fa3e8004870 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa400fa2700 1 -- 172.21.15.179:0/2823470032 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005d30 con 0x7fa3e8004870 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 -- 172.21.15.179:0/2823470032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e8004870 msgr2=0x7fa3e8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 --2- 172.21.15.179:0/2823470032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e8004870 0x7fa3e8004cf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c210 tx=0x7fa3e80058a0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa4027a5700 1 -- 172.21.15.179:0/2823470032 reap_dead start 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 -- 172.21.15.179:0/2823470032 shutdown_connections 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 -- 172.21.15.179:0/2823470032 >> 172.21.15.179:0/2823470032 conn(0x7fa3fc098a70 msgr2=0x7fa3fc0cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 -- 172.21.15.179:0/2823470032 shutdown_connections 2024-05-08T22:37:41.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.461+0000 7fa402fa6700 1 -- 172.21.15.179:0/2823470032 wait complete. 2024-05-08T22:37:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 Processor -- start 2024-05-08T22:37:41.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 -- start start 2024-05-08T22:37:41.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8004870 0x7fa3fc1f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc0c9da0 0x7fa3fc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0caef0 con 0x7fa3fc0cb380 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0ca2e0 con 0x7fa3e8004870 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa402fa6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0ca5c0 con 0x7fa3fc0c9da0 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48824/0 (socket says 172.21.15.179:48824) 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa401fa4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8004870 0x7fa3fc1f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 -- 172.21.15.179:0/3159983923 learned_addr learned my addr 172.21.15.179:0/3159983923 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4027a5700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc0c9da0 0x7fa3fc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 -- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc0c9da0 msgr2=0x7fa3fc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc0c9da0 0x7fa3fc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 -- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8004870 msgr2=0x7fa3fc1f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8004870 0x7fa3fc1f02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.465+0000 7fa4017a3700 1 -- 172.21.15.179:0/3159983923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0ca8a0 con 0x7fa3fc0cb380 2024-05-08T22:37:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa4017a3700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005830 tx=0x7fa3f0001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005d00 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005e60 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc0cab80 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0004ab0 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1f3150 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc0cb0a0 con 0x7fa3fc0cb380 2024-05-08T22:37:41.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3f0004c10 con 0x7fa3fc0cb380 2024-05-08T22:37:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.469+0000 7fa3ee7fc700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 0x7fa3e0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.473+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fa3f0085a50 con 0x7fa3fc0cb380 2024-05-08T22:37:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.473+0000 7fa401fa4700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 0x7fa3e0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.473+0000 7fa401fa4700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 0x7fa3e0068c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0c9780 tx=0x7fa3f8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.477+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa3fc0cb0a0 con 0x7fa3fc0cb380 2024-05-08T22:37:41.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.645+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3fc0c4990 con 0x7fa3fc0cb380 2024-05-08T22:37:41.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa3ee7fc700 1 -- 172.21.15.179:0/3159983923 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3f0052c90 con 0x7fa3fc0cb380 2024-05-08T22:37:41.650 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 msgr2=0x7fa3e0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 0x7fa3e0068c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0c9780 tx=0x7fa3f8009240 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 msgr2=0x7fa3fc0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005830 tx=0x7fa3f0001980 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 shutdown_connections 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3e0066760 0x7fa3e0068c20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3fc0c9da0 0x7fa3fc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3fc0cb380 0x7fa3fc0c9860 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 --2- 172.21.15.179:0/3159983923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8004870 0x7fa3fc1f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 >> 172.21.15.179:0/3159983923 conn(0x7fa3fc098a70 msgr2=0x7fa3fc0c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 shutdown_connections 2024-05-08T22:37:41.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.649+0000 7fa402fa6700 1 -- 172.21.15.179:0/3159983923 wait complete. 2024-05-08T22:37:41.665 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:37:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 Processor -- start 2024-05-08T22:37:41.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 -- start start 2024-05-08T22:37:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe75815d160 0x7fe7580b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580d1910 con 0x7fe7580cdb20 2024-05-08T22:37:41.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580d1bf0 con 0x7fe7580d29b0 2024-05-08T22:37:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe757fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580d1ed0 con 0x7fe75815d160 2024-05-08T22:37:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe7577fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe7580cdb20 0x7fe7580b5f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37854/0 (socket says 172.21.15.179:37854) 2024-05-08T22:37:41.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe7577fe700 1 -- 172.21.15.179:0/729184945 learned_addr learned my addr 172.21.15.179:0/729184945 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:41.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe756ffd700 1 --2- 172.21.15.179:0/729184945 >> v2:172.21.15.195:3300/0 conn(0x7fe75815d160 0x7fe7580b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243826662 0 0) 0x7fe7580d1910 con 0x7fe7580cdb20 2024-05-08T22:37:41.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7400035d0 con 0x7fe7580cdb20 2024-05-08T22:37:41.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1334245934 0 0) 0x7fe7400035d0 con 0x7fe7580cdb20 2024-05-08T22:37:41.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 >> v2:172.21.15.195:3300/0 conn(0x7fe75815d160 msgr2=0x7fe7580b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 --2- 172.21.15.179:0/729184945 >> v2:172.21.15.195:3300/0 conn(0x7fe75815d160 0x7fe7580b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 >> v1:172.21.15.195:6789/0 conn(0x7fe7580d29b0 legacy=0x7fe7580d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580d21b0 con 0x7fe7580cdb20 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe7500019f0 con 0x7fe7580cdb20 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 >> v1:172.21.15.179:6789/0 conn(0x7fe7580cdb20 legacy=0x7fe7580b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.773+0000 7fe755ffb700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740003a20 0x7fe740003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740004870 0x7fe740004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 0x7fe74000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580d1ed0 con 0x7fe740005230 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74000c0d0 con 0x7fe740004870 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe740014aa0 con 0x7fe740003a20 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 0x7fe74000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7567fc700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740004870 0x7fe740004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe756ffd700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740003a20 0x7fe740003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 -- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740003a20 msgr2=0x7fe740003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740003a20 0x7fe740003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 -- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740004870 msgr2=0x7fe740004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe740004870 0x7fe740004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 -- 172.21.15.179:0/729184945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400035d0 con 0x7fe740005230 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 -- 172.21.15.179:0/729184945 reap_dead start 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe7577fe700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 0x7fe74000ba30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a830 tx=0x7fe7400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686902ac6ababeb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000a2f0 con 0x7fe740005230 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7500072f0 con 0x7fe740005230 2024-05-08T22:37:41.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe755ffb700 1 -- 172.21.15.179:0/729184945 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500076a0 con 0x7fe740005230 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 -- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 msgr2=0x7fe74000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 0x7fe74000ba30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a830 tx=0x7fe7400058a0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 -- 172.21.15.179:0/729184945 shutdown_connections 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 --2- 172.21.15.179:0/729184945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe740005230 0x7fe74000ba30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 -- 172.21.15.179:0/729184945 >> 172.21.15.179:0/729184945 conn(0x7fe7580d3270 msgr2=0x7fe7580d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 -- 172.21.15.179:0/729184945 shutdown_connections 2024-05-08T22:37:41.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.777+0000 7fe757fff700 1 -- 172.21.15.179:0/729184945 wait complete. 2024-05-08T22:37:41.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 Processor -- start 2024-05-08T22:37:41.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 -- start start 2024-05-08T22:37:41.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740005230 0x7fe758155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7580c70d0 0x7fe7580c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75815e000 con 0x7fe758155fb0 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580c7a90 con 0x7fe7580c70d0 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe757fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe758155da0 con 0x7fe740005230 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe7567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe7567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48830/0 (socket says 172.21.15.179:48830) 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe7567fc700 1 -- 172.21.15.179:0/3831584283 learned_addr learned my addr 172.21.15.179:0/3831584283 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe7577fe700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7580c70d0 0x7fe7580c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.781+0000 7fe756ffd700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740005230 0x7fe758155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 -- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740005230 msgr2=0x7fe758155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740005230 0x7fe758155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 -- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7580c70d0 msgr2=0x7fe7580c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7580c70d0 0x7fe7580c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 -- 172.21.15.179:0/3831584283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580c6820 con 0x7fe758155fb0 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe7567fc700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe74800f820 tx=0x7fe74800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480052e0 con 0x7fe758155fb0 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe748005440 con 0x7fe758155fb0 2024-05-08T22:37:41.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7580c6b00 con 0x7fe758155fb0 2024-05-08T22:37:41.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480038c0 con 0x7fe758155fb0 2024-05-08T22:37:41.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75815dc20 con 0x7fe758155fb0 2024-05-08T22:37:41.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75815dde0 con 0x7fe758155fb0 2024-05-08T22:37:41.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.785+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe748003a20 con 0x7fe758155fb0 2024-05-08T22:37:41.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.789+0000 7fe73ffff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 0x7fe744068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:41.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.789+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe748083ff0 con 0x7fe758155fb0 2024-05-08T22:37:41.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.789+0000 7fe756ffd700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 0x7fe744068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:41.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.789+0000 7fe756ffd700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 0x7fe744068bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009bf0 tx=0x7fe74c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.793+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe748051100 con 0x7fe758155fb0 2024-05-08T22:37:41.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.957+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe75804e700 con 0x7fe758155fb0 2024-05-08T22:37:41.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe73ffff700 1 -- 172.21.15.179:0/3831584283 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe748054720 con 0x7fe758155fb0 2024-05-08T22:37:41.963 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 msgr2=0x7fe744068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 0x7fe744068bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009bf0 tx=0x7fe74c009240 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 msgr2=0x7fe7580c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe74800f820 tx=0x7fe74800fb30 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 shutdown_connections 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe744066730 0x7fe744068bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7580c70d0 0x7fe7580c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe758155fb0 0x7fe7580c9060 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 --2- 172.21.15.179:0/3831584283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe740005230 0x7fe758155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 >> 172.21.15.179:0/3831584283 conn(0x7fe7580d3270 msgr2=0x7fe7580cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 shutdown_connections 2024-05-08T22:37:41.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:41.961+0000 7fe757fff700 1 -- 172.21.15.179:0/3831584283 wait complete. 2024-05-08T22:37:41.978 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:37:42.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 Processor -- start 2024-05-08T22:37:42.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 -- start start 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 0x7ff1640cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1640b4460 con 0x7ff16406b440 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1640ccf30 con 0x7ff16415d180 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff168d10700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640b5490 con 0x7ff1640b5000 2024-05-08T22:37:42.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff16259c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 0x7ff1640cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:51058/0 (socket says 172.21.15.179:51058) 2024-05-08T22:37:42.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 -- 172.21.15.179:0/1745357364 learned_addr learned my addr 172.21.15.179:0/1745357364 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:42.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff161d9b700 1 --1- 172.21.15.179:0/1745357364 >> v1:172.21.15.195:6789/0 conn(0x7ff16406b440 0x7ff1640b48f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46144/0 (socket says 172.21.15.179:46144) 2024-05-08T22:37:42.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 -- 172.21.15.179:0/1745357364 >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 msgr2=0x7ff1640cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 --2- 172.21.15.179:0/1745357364 >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 0x7ff1640cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.089+0000 7ff162d9d700 1 -- 172.21.15.179:0/1745357364 >> v1:172.21.15.195:6789/0 conn(0x7ff16406b440 legacy=0x7ff1640b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff162d9d700 1 -- 172.21.15.179:0/1745357364 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640c4790 con 0x7ff1640b5000 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff16259c700 1 --2- 172.21.15.179:0/1745357364 >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 0x7ff1640cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff162d9d700 1 --2- 172.21.15.179:0/1745357364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00af60 tx=0x7ff15c00af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3de85d2bcdc6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff16159a700 1 -- 172.21.15.179:0/1745357364 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00f070 con 0x7ff1640b5000 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff16159a700 1 -- 172.21.15.179:0/1745357364 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c00a2e0 con 0x7ff1640b5000 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff16159a700 1 -- 172.21.15.179:0/1745357364 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00cc00 con 0x7ff1640b5000 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- 172.21.15.179:0/1745357364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 msgr2=0x7ff16415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 --2- 172.21.15.179:0/1745357364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00af60 tx=0x7ff15c00af90 comp rx=0 tx=0).stop 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- 172.21.15.179:0/1745357364 shutdown_connections 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 --2- 172.21.15.179:0/1745357364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1640b5000 0x7ff16415d560 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 --2- 172.21.15.179:0/1745357364 >> v2:172.21.15.195:3300/0 conn(0x7ff16415d180 0x7ff1640cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- 172.21.15.179:0/1745357364 >> 172.21.15.179:0/1745357364 conn(0x7ff164098a70 msgr2=0x7ff1640b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:42.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- 172.21.15.179:0/1745357364 shutdown_connections 2024-05-08T22:37:42.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- 172.21.15.179:0/1745357364 wait complete. 2024-05-08T22:37:42.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 Processor -- start 2024-05-08T22:37:42.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.093+0000 7ff168d10700 1 -- start start 2024-05-08T22:37:42.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1640b5000 0x7ff1640ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff16415d180 0x7ff1640caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164156c00 con 0x7ff16406b440 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164156ee0 con 0x7ff1640b5000 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641571c0 con 0x7ff16415d180 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48832/0 (socket says 172.21.15.179:48832) 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 -- 172.21.15.179:0/3848761806 learned_addr learned my addr 172.21.15.179:0/3848761806 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff161d9b700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1640b5000 0x7ff1640ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff162d9d700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff16415d180 0x7ff1640caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 -- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff16415d180 msgr2=0x7ff1640caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff16415d180 0x7ff1640caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 -- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1640b5000 msgr2=0x7ff1640ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1640b5000 0x7ff1640ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 -- 172.21.15.179:0/3848761806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540094b0 con 0x7ff16406b440 2024-05-08T22:37:42.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff16259c700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff15400ebe0 tx=0x7ff15400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154009770 con 0x7ff16406b440 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15400c850 con 0x7ff16406b440 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c00ac40 con 0x7ff16406b440 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400cbe0 con 0x7ff16406b440 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.097+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff164157780 con 0x7ff16406b440 2024-05-08T22:37:42.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164073290 con 0x7ff16406b440 2024-05-08T22:37:42.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff15400cd40 con 0x7ff16406b440 2024-05-08T22:37:42.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff152ffd700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 0x7ff144068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7ff154084240 con 0x7ff16406b440 2024-05-08T22:37:42.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff161d9b700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 0x7ff144068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.101+0000 7ff161d9b700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 0x7ff144068bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff164158e80 tx=0x7ff158007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.105+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff154051470 con 0x7ff16406b440 2024-05-08T22:37:42.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.289+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff164072360 con 0x7ff16406b440 2024-05-08T22:37:42.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.289+0000 7ff152ffd700 1 -- 172.21.15.179:0/3848761806 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff154054a90 con 0x7ff16406b440 2024-05-08T22:37:42.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 msgr2=0x7ff144068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 0x7ff144068bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff164158e80 tx=0x7ff158007ef0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 msgr2=0x7ff1641f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff15400ebe0 tx=0x7ff15400eef0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 shutdown_connections 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff144066730 0x7ff144068bf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1640b5000 0x7ff1640ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff16415d180 0x7ff1640caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 --2- 172.21.15.179:0/3848761806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff16406b440 0x7ff1641f5ac0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 >> 172.21.15.179:0/3848761806 conn(0x7ff164098a70 msgr2=0x7ff164157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 shutdown_connections 2024-05-08T22:37:42.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.293+0000 7ff168d10700 1 -- 172.21.15.179:0/3848761806 wait complete. 2024-05-08T22:37:42.297 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:37:42.307 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:37:42.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:37:42.308 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:37:42.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[40792]: fuse finished with error 0 and tester_r 0 2024-05-08T22:37:42.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:37:42.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:37:42.318 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:37:42.441 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:37:42.441 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:42.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:37:42.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:37:42.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:37:42.449 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:37:42.455 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:37:42.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:42.456 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:37:42.456 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:42.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:37:42.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:37:42.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:37:42.464 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:37:42.511 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:37:42.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:42.512 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:37:42.512 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:42.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:37:42.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:37:42.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:37:42.520 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:37:42.526 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:37:42.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:42.527 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:42.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 Processor -- start 2024-05-08T22:37:42.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 -- start start 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85b80ce380 0x7f85b80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b806abb0 con 0x7f85b80cb510 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b806ae90 con 0x7f85b80cc700 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85bd5ac700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b80cb980 con 0x7f85b80ce380 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85b80ce380 0x7f85b80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85b80ce380 0x7f85b80cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48848/0 (socket says 172.21.15.179:48848) 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 -- 172.21.15.179:0/866749807 learned_addr learned my addr 172.21.15.179:0/866749807 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:42.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 -- 172.21.15.179:0/866749807 >> v1:172.21.15.195:6789/0 conn(0x7f85b80cc700 legacy=0x7f85b815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 -- 172.21.15.179:0/866749807 >> v1:172.21.15.179:6789/0 conn(0x7f85b80cb510 legacy=0x7f85b806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 -- 172.21.15.179:0/866749807 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80ce800 con 0x7f85b80ce380 2024-05-08T22:37:42.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b77fe700 1 --2- 172.21.15.179:0/866749807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80ce380 0x7f85b80cafd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f953715e4525f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b6ffd700 1 -- 172.21.15.179:0/866749807 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a800e070 con 0x7f85b80ce380 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b6ffd700 1 -- 172.21.15.179:0/866749807 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a800b9c0 con 0x7f85b80ce380 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.645+0000 7f85b6ffd700 1 -- 172.21.15.179:0/866749807 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a800bd70 con 0x7f85b80ce380 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- 172.21.15.179:0/866749807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80ce380 msgr2=0x7f85b80cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/866749807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80ce380 0x7f85b80cafd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a800b330 comp rx=0 tx=0).stop 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- 172.21.15.179:0/866749807 shutdown_connections 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/866749807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80ce380 0x7f85b80cafd0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- 172.21.15.179:0/866749807 >> 172.21.15.179:0/866749807 conn(0x7f85b80ce9d0 msgr2=0x7f85b80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- 172.21.15.179:0/866749807 shutdown_connections 2024-05-08T22:37:42.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- 172.21.15.179:0/866749807 wait complete. 2024-05-08T22:37:42.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 Processor -- start 2024-05-08T22:37:42.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- start start 2024-05-08T22:37:42.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85b80cb510 0x7f85b8145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85b80ce380 0x7f85b80c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8144b20 con 0x7f85b80cc700 2024-05-08T22:37:42.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8144e00 con 0x7f85b80cb510 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bd5ac700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81450e0 con 0x7f85b80ce380 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48852/0 (socket says 172.21.15.179:48852) 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85b77fe700 1 -- 172.21.15.179:0/1437755824 learned_addr learned my addr 172.21.15.179:0/1437755824 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85b7fff700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85b80cb510 0x7f85b8145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.649+0000 7f85bcdab700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85b80ce380 0x7f85b80c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 -- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85b80ce380 msgr2=0x7f85b80c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85b80ce380 0x7f85b80c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 -- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85b80cb510 msgr2=0x7f85b8145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85b80cb510 0x7f85b8145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 -- 172.21.15.179:0/1437755824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8009b30 con 0x7f85b80cc700 2024-05-08T22:37:42.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b77fe700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f85b80b3e80 tx=0x7f85a8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8004880 con 0x7f85b80cc700 2024-05-08T22:37:42.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b81453c0 con 0x7f85b80cc700 2024-05-08T22:37:42.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a80099b0 con 0x7f85b80cc700 2024-05-08T22:37:42.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a801c680 con 0x7f85b80cc700 2024-05-08T22:37:42.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b8143db0 con 0x7f85b80cc700 2024-05-08T22:37:42.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.653+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b81458e0 con 0x7f85b80cc700 2024-05-08T22:37:42.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.657+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85a801c7e0 con 0x7f85b80cc700 2024-05-08T22:37:42.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.657+0000 7f85b4ff9700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 0x7f85a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:42.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.657+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f85a8085d70 con 0x7f85b80cc700 2024-05-08T22:37:42.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.657+0000 7f85b7fff700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 0x7f85a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:42.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.657+0000 7f85b7fff700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 0x7f85a4068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f60 tx=0x7f85ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:42.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.661+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f85b81458e0 con 0x7f85b80cc700 2024-05-08T22:37:42.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.869+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85b81458e0 con 0x7f85b80cc700 2024-05-08T22:37:42.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85b4ff9700 1 -- 172.21.15.179:0/1437755824 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f85b81458e0 con 0x7f85b80cc700 2024-05-08T22:37:42.875 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:42.875 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:37:13.599733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:42.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 msgr2=0x7f85a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 0x7f85a4068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f60 tx=0x7f85ac009040 comp rx=0 tx=0).stop 2024-05-08T22:37:42.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 msgr2=0x7f85b80c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f85b80b3e80 tx=0x7f85a8004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 shutdown_connections 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85a4066730 0x7f85a4068bf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85b80ce380 0x7f85b80c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85b80cc700 0x7f85b80c4360 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 --2- 172.21.15.179:0/1437755824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85b80cb510 0x7f85b8145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 >> 172.21.15.179:0/1437755824 conn(0x7f85b80ce9d0 msgr2=0x7f85b80aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 shutdown_connections 2024-05-08T22:37:42.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.873+0000 7f85bd5ac700 1 -- 172.21.15.179:0/1437755824 wait complete. 2024-05-08T22:37:42.879 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 32 2024-05-08T22:37:42.890 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.997+0000 7f609d1cb700 1 Processor -- start 2024-05-08T22:37:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:42.997+0000 7f609d1cb700 1 -- start start 2024-05-08T22:37:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f60980b3720 0x7f609815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098159b60 con 0x7f60981539c0 2024-05-08T22:37:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098159e40 con 0x7f60980b19e0 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609815a120 con 0x7f60980b3720 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f60980b3720 0x7f609815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f60980b3720 0x7f609815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48858/0 (socket says 172.21.15.179:48858) 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 -- 172.21.15.179:0/2213619306 learned_addr learned my addr 172.21.15.179:0/2213619306 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f6096ffd700 1 -- 172.21.15.179:0/2213619306 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375336072 0 0) 0x7f6098159b60 con 0x7f60981539c0 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f6096ffd700 1 -- 172.21.15.179:0/2213619306 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60840035d0 con 0x7f60981539c0 2024-05-08T22:37:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 -- 172.21.15.179:0/2213619306 >> v1:172.21.15.195:6789/0 conn(0x7f60980b19e0 legacy=0x7f60980b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 -- 172.21.15.179:0/2213619306 >> v1:172.21.15.179:6789/0 conn(0x7f60981539c0 legacy=0x7f609806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 -- 172.21.15.179:0/2213619306 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609815a400 con 0x7f60980b3720 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609c9ca700 1 --2- 172.21.15.179:0/2213619306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815d120 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6090009910 tx=0x7f6090009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c428e593a155103 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f6096ffd700 1 -- 172.21.15.179:0/2213619306 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090011830 con 0x7f60980b3720 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f6096ffd700 1 -- 172.21.15.179:0/2213619306 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090011990 con 0x7f60980b3720 2024-05-08T22:37:43.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f6096ffd700 1 -- 172.21.15.179:0/2213619306 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090011d40 con 0x7f60980b3720 2024-05-08T22:37:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- 172.21.15.179:0/2213619306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 msgr2=0x7f609815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 --2- 172.21.15.179:0/2213619306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815d120 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6090009910 tx=0x7f6090009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- 172.21.15.179:0/2213619306 shutdown_connections 2024-05-08T22:37:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 --2- 172.21.15.179:0/2213619306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815d120 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- 172.21.15.179:0/2213619306 >> 172.21.15.179:0/2213619306 conn(0x7f60980ce9d0 msgr2=0x7f60980cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- 172.21.15.179:0/2213619306 shutdown_connections 2024-05-08T22:37:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.001+0000 7f609d1cb700 1 -- 172.21.15.179:0/2213619306 wait complete. 2024-05-08T22:37:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 Processor -- start 2024-05-08T22:37:43.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- start start 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60980b19e0 0x7f609813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f60981539c0 0x7f609814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981470c0 con 0x7f60980b3720 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609815c120 con 0x7f60980b19e0 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609815b510 con 0x7f60981539c0 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48866/0 (socket says 172.21.15.179:48866) 2024-05-08T22:37:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 -- 172.21.15.179:0/3177038108 learned_addr learned my addr 172.21.15.179:0/3177038108 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f6097fff700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60980b19e0 0x7f609813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609c9ca700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f60981539c0 0x7f609814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 -- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f60981539c0 msgr2=0x7f609814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f60981539c0 0x7f609814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 -- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60980b19e0 msgr2=0x7f609813fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60980b19e0 0x7f609813fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 -- 172.21.15.179:0/3177038108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090009570 con 0x7f60980b3720 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f60977fe700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f608000cba0 tx=0x7f608000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080003c20 con 0x7f60980b3720 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080003d80 con 0x7f60980b3720 2024-05-08T22:37:43.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609815b7f0 con 0x7f60980b3720 2024-05-08T22:37:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000aae0 con 0x7f60980b3720 2024-05-08T22:37:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.005+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609815bdb0 con 0x7f60980b3720 2024-05-08T22:37:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098067740 con 0x7f60980b3720 2024-05-08T22:37:43.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f608000ac40 con 0x7f60980b3720 2024-05-08T22:37:43.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f6094ff9700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 0x7f6078068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f6080084300 con 0x7f60980b3720 2024-05-08T22:37:43.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f6097fff700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 0x7f6078068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.009+0000 7f6097fff700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 0x7f6078068c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60880011e0 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.017+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6080051550 con 0x7f60980b3720 2024-05-08T22:37:43.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.189+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f609815bf70 con 0x7f60980b3720 2024-05-08T22:37:43.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f6094ff9700 1 -- 172.21.15.179:0/3177038108 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16535 (secure 0 0 0) 0x7f6080054b70 con 0x7f60980b3720 2024-05-08T22:37:43.194 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:43.194 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":71,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:06.562633+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 msgr2=0x7f6078068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 0x7f6078068c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60880011e0 tx=0x7f6088009040 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 msgr2=0x7f609815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f608000cba0 tx=0x7f608000ceb0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 shutdown_connections 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6078066760 0x7f6078068c20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f60981539c0 0x7f609814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60980b3720 0x7f609815aa50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 --2- 172.21.15.179:0/3177038108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60980b19e0 0x7f609813fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 >> 172.21.15.179:0/3177038108 conn(0x7f60980ce9d0 msgr2=0x7f609815d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 shutdown_connections 2024-05-08T22:37:43.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.193+0000 7f609d1cb700 1 -- 172.21.15.179:0/3177038108 wait complete. 2024-05-08T22:37:43.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:36:23.437690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:36:24.076455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:37:43.209 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:37:43.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 Processor -- start 2024-05-08T22:37:43.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 -- start start 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 0x7fe5ac06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac0b22d0 con 0x7fe5ac142890 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0b19e0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5ab59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac15e640 con 0x7fe5ac143520 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aa59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe5ac142890 0x7fe5ac0b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37866/0 (socket says 172.21.15.179:37866) 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aa59c700 1 -- 172.21.15.179:0/805650012 learned_addr learned my addr 172.21.15.179:0/805650012 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aad9d700 1 --2- 172.21.15.179:0/805650012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5a9d9b700 1 --2- 172.21.15.179:0/805650012 >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 0x7fe5ac06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5a959a700 1 -- 172.21.15.179:0/805650012 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971447291 0 0) 0x7fe5ac0b22d0 con 0x7fe5ac142890 2024-05-08T22:37:43.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5a959a700 1 -- 172.21.15.179:0/805650012 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe58c0035d0 con 0x7fe5ac142890 2024-05-08T22:37:43.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aad9d700 1 -- 172.21.15.179:0/805650012 >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 msgr2=0x7fe5ac06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:43.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aad9d700 1 --2- 172.21.15.179:0/805650012 >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 0x7fe5ac06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aad9d700 1 -- 172.21.15.179:0/805650012 >> v1:172.21.15.179:6789/0 conn(0x7fe5ac142890 legacy=0x7fe5ac0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5aad9d700 1 -- 172.21.15.179:0/805650012 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac0b1c80 con 0x7fe5ac143520 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.325+0000 7fe5a9d9b700 1 --2- 172.21.15.179:0/805650012 >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 0x7fe5ac06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5aad9d700 1 --2- 172.21.15.179:0/805650012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac06d2e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009d90 tx=0x7fe5a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e5e13ba8b4bdcdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5a959a700 1 -- 172.21.15.179:0/805650012 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400e070 con 0x7fe5ac143520 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5a959a700 1 -- 172.21.15.179:0/805650012 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400bb20 con 0x7fe5ac143520 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5a959a700 1 -- 172.21.15.179:0/805650012 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4012430 con 0x7fe5ac143520 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- 172.21.15.179:0/805650012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 msgr2=0x7fe5ac06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/805650012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac06d2e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009d90 tx=0x7fe5a400b330 comp rx=0 tx=0).stop 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- 172.21.15.179:0/805650012 shutdown_connections 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/805650012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac06d2e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/805650012 >> v2:172.21.15.195:3300/0 conn(0x7fe5ac142fd0 0x7fe5ac06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- 172.21.15.179:0/805650012 >> 172.21.15.179:0/805650012 conn(0x7fe5ac0cea30 msgr2=0x7fe5ac0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- 172.21.15.179:0/805650012 shutdown_connections 2024-05-08T22:37:43.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- 172.21.15.179:0/805650012 wait complete. 2024-05-08T22:37:43.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 Processor -- start 2024-05-08T22:37:43.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.329+0000 7fe5ab59e700 1 -- start start 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5ac142890 0x7fe5ac1f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac13ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac138380 con 0x7fe5ac142fd0 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac157a90 con 0x7fe5ac142890 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac158560 con 0x7fe5ac143520 2024-05-08T22:37:43.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48872/0 (socket says 172.21.15.179:48872) 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 -- 172.21.15.179:0/2872558233 learned_addr learned my addr 172.21.15.179:0/2872558233 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5aa59c700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5ac142890 0x7fe5ac1f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5aad9d700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac13ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 -- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 msgr2=0x7fe5ac13ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac13ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 -- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5ac142890 msgr2=0x7fe5ac1f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5ac142890 0x7fe5ac1f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 -- 172.21.15.179:0/2872558233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0094b0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a9d9b700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00ebe0 tx=0x7fe59c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00c850 con 0x7fe5ac142fd0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c00c9b0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00cd40 con 0x7fe5ac142fd0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a4009a70 con 0x7fe5ac142fd0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac157fa0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.333+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac0cbe80 con 0x7fe5ac142fd0 2024-05-08T22:37:43.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.337+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe59c013470 con 0x7fe5ac142fd0 2024-05-08T22:37:43.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.337+0000 7fe5a2ffd700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 0x7fe590068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.337+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe59c084450 con 0x7fe5ac142fd0 2024-05-08T22:37:43.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.337+0000 7fe5aa59c700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 0x7fe590068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.337+0000 7fe5aa59c700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 0x7fe590068bf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe594007fd0 tx=0x7fe594007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.345+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe59c051550 con 0x7fe5ac142fd0 2024-05-08T22:37:43.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5ac1420f0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5a2ffd700 1 -- 172.21.15.179:0/2872558233 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe59c00cea0 con 0x7fe5ac142fd0 2024-05-08T22:37:43.519 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 msgr2=0x7fe590068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 0x7fe590068bf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe594007fd0 tx=0x7fe594007de0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 msgr2=0x7fe5ac157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00ebe0 tx=0x7fe59c00eef0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 shutdown_connections 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe590066730 0x7fe590068bf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5ac143520 0x7fe5ac13ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5ac142fd0 0x7fe5ac157550 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 --2- 172.21.15.179:0/2872558233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5ac142890 0x7fe5ac1f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.517+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 >> 172.21.15.179:0/2872558233 conn(0x7fe5ac0cea30 msgr2=0x7fe5ac159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.521+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 shutdown_connections 2024-05-08T22:37:43.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.521+0000 7fe5ab59e700 1 -- 172.21.15.179:0/2872558233 wait complete. 2024-05-08T22:37:43.534 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:37:43.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 Processor -- start 2024-05-08T22:37:43.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 -- start start 2024-05-08T22:37:43.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2f015d0c0 0x7fd2f00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 0x7fd2f006b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f015d4a0 con 0x7fd2f00d1d20 2024-05-08T22:37:43.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f006abb0 con 0x7fd2f015d0c0 2024-05-08T22:37:43.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.657+0000 7fd2f4c16700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00d2500 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 0x7fd2f006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ef7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2f015d0c0 0x7fd2f00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2eeffd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd2f00d1d20 0x7fd2f00cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46184/0 (socket says 172.21.15.179:46184) 2024-05-08T22:37:43.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2eeffd700 1 -- 172.21.15.179:0/405277737 learned_addr learned my addr 172.21.15.179:0/405277737 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ee7fc700 1 -- 172.21.15.179:0/405277737 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485781440 0 0) 0x7fd2f015d4a0 con 0x7fd2f00d1d20 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ee7fc700 1 -- 172.21.15.179:0/405277737 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2dc0035d0 con 0x7fd2f00d1d20 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 -- 172.21.15.179:0/405277737 >> v2:172.21.15.195:3300/0 conn(0x7fd2f015d0c0 msgr2=0x7fd2f00b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 --2- 172.21.15.179:0/405277737 >> v2:172.21.15.195:3300/0 conn(0x7fd2f015d0c0 0x7fd2f00b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 -- 172.21.15.179:0/405277737 >> v1:172.21.15.195:6789/0 conn(0x7fd2f00d1d20 legacy=0x7fd2f00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 -- 172.21.15.179:0/405277737 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00d27e0 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2effff700 1 --2- 172.21.15.179:0/405277737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 0x7fd2f006b370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009d90 tx=0x7fd2e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c8e81b28dc4f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ee7fc700 1 -- 172.21.15.179:0/405277737 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800e070 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ee7fc700 1 -- 172.21.15.179:0/405277737 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e800bb20 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2ee7fc700 1 -- 172.21.15.179:0/405277737 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8012430 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 -- 172.21.15.179:0/405277737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 msgr2=0x7fd2f006b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/405277737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 0x7fd2f006b370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009d90 tx=0x7fd2e800b330 comp rx=0 tx=0).stop 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 -- 172.21.15.179:0/405277737 shutdown_connections 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/405277737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00ce2b0 0x7fd2f006b370 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/405277737 >> v2:172.21.15.195:3300/0 conn(0x7fd2f015d0c0 0x7fd2f00b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 -- 172.21.15.179:0/405277737 >> 172.21.15.179:0/405277737 conn(0x7fd2f00d2ed0 msgr2=0x7fd2f00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 -- 172.21.15.179:0/405277737 shutdown_connections 2024-05-08T22:37:43.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.661+0000 7fd2f4c16700 1 -- 172.21.15.179:0/405277737 wait complete. 2024-05-08T22:37:43.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 Processor -- start 2024-05-08T22:37:43.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 -- start start 2024-05-08T22:37:43.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2f00ce2b0 0x7fd2f0153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 0x7fd2f0151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00c8720 con 0x7fd2f015d0c0 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0151c40 con 0x7fd2f00ce2b0 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2f4c16700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00c7aa0 con 0x7fd2f00d1d20 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48886/0 (socket says 172.21.15.179:48886) 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 -- 172.21.15.179:0/1144927257 learned_addr learned my addr 172.21.15.179:0/1144927257 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2eeffd700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 0x7fd2f0151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2ef7fe700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2f00ce2b0 0x7fd2f0153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 -- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 msgr2=0x7fd2f0151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 0x7fd2f0151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 -- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2f00ce2b0 msgr2=0x7fd2f0153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2f00ce2b0 0x7fd2f0153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 -- 172.21.15.179:0/1144927257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8009a70 con 0x7fd2f015d0c0 2024-05-08T22:37:43.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2eeffd700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 0x7fd2f0151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:43.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.665+0000 7fd2effff700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.669+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8012430 con 0x7fd2f015d0c0 2024-05-08T22:37:43.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.669+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8020070 con 0x7fd2f015d0c0 2024-05-08T22:37:43.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.669+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f00c7d80 con 0x7fd2f015d0c0 2024-05-08T22:37:43.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.669+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004280 con 0x7fd2f015d0c0 2024-05-08T22:37:43.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.669+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f00c8340 con 0x7fd2f015d0c0 2024-05-08T22:37:43.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f004e700 con 0x7fd2f015d0c0 2024-05-08T22:37:43.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd2e80043e0 con 0x7fd2f015d0c0 2024-05-08T22:37:43.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2e7fff700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 0x7fd2d0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fd2e8022070 con 0x7fd2f015d0c0 2024-05-08T22:37:43.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2ef7fe700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 0x7fd2d0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.673+0000 7fd2ef7fe700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 0x7fd2d0068c20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009ae0 tx=0x7fd2e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.681+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd2e8053170 con 0x7fd2f015d0c0 2024-05-08T22:37:43.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.849+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2f00676e0 con 0x7fd2f015d0c0 2024-05-08T22:37:43.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2e7fff700 1 -- 172.21.15.179:0/1144927257 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2e8056790 con 0x7fd2f015d0c0 2024-05-08T22:37:43.854 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 msgr2=0x7fd2d0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 0x7fd2d0068c20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009ae0 tx=0x7fd2e0008040 comp rx=0 tx=0).stop 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 msgr2=0x7fd2f00cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 shutdown_connections 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd2d0066760 0x7fd2d0068c20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f015d0c0 0x7fd2f00cc740 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2f00d1d20 0x7fd2f0151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 --2- 172.21.15.179:0/1144927257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2f00ce2b0 0x7fd2f0153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 >> 172.21.15.179:0/1144927257 conn(0x7fd2f00d2ed0 msgr2=0x7fd2f00b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 shutdown_connections 2024-05-08T22:37:43.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.853+0000 7fd2f4c16700 1 -- 172.21.15.179:0/1144927257 wait complete. 2024-05-08T22:37:43.869 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:37:43.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 Processor -- start 2024-05-08T22:37:43.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 -- start start 2024-05-08T22:37:43.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f36d415a300 0x7f36d415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415acc0 0x7f36d40b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d40b4650 con 0x7f36d41544e0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d40cd6e0 con 0x7f36d415a300 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d988d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d406db50 con 0x7f36d415acc0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d888b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f36d41544e0 0x7f36d406ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37868/0 (socket says 172.21.15.179:37868) 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d888b700 1 -- 172.21.15.179:0/473496779 learned_addr learned my addr 172.21.15.179:0/473496779 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 --2- 172.21.15.179:0/473496779 >> v2:172.21.15.179:3300/0 conn(0x7f36d415a300 0x7f36d415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d908c700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415acc0 0x7f36d40b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d37fe700 1 -- 172.21.15.179:0/473496779 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000591706 0 0) 0x7f36d40b4650 con 0x7f36d41544e0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d37fe700 1 -- 172.21.15.179:0/473496779 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c00035d0 con 0x7f36d41544e0 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 -- 172.21.15.179:0/473496779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415acc0 msgr2=0x7f36d40b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415acc0 0x7f36d40b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 -- 172.21.15.179:0/473496779 >> v1:172.21.15.179:6789/0 conn(0x7f36d41544e0 legacy=0x7f36d406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 -- 172.21.15.179:0/473496779 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40b4320 con 0x7f36d415a300 2024-05-08T22:37:43.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d3fff700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d415a300 0x7f36d415a780 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009850 tx=0x7f36c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9093fb1387fb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:43.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d37fe700 1 -- 172.21.15.179:0/473496779 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4011830 con 0x7f36d415a300 2024-05-08T22:37:43.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d37fe700 1 -- 172.21.15.179:0/473496779 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c4011990 con 0x7f36d415a300 2024-05-08T22:37:43.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.989+0000 7f36d37fe700 1 -- 172.21.15.179:0/473496779 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4011d40 con 0x7f36d415a300 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 -- 172.21.15.179:0/473496779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d415a300 msgr2=0x7f36d415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d415a300 0x7f36d415a780 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009850 tx=0x7f36c4009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 -- 172.21.15.179:0/473496779 shutdown_connections 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415acc0 0x7f36d40b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 --2- 172.21.15.179:0/473496779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d415a300 0x7f36d415a780 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 -- 172.21.15.179:0/473496779 >> 172.21.15.179:0/473496779 conn(0x7f36d40b18d0 msgr2=0x7f36d40b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 -- 172.21.15.179:0/473496779 shutdown_connections 2024-05-08T22:37:43.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.993+0000 7f36d988d700 1 -- 172.21.15.179:0/473496779 wait complete. 2024-05-08T22:37:43.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 Processor -- start 2024-05-08T22:37:43.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- start start 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415a300 0x7f36d413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 0x7f36d406b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f2530 con 0x7f36d41544e0 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f4f90 con 0x7f36d415acc0 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f5270 con 0x7f36d415a300 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48904/0 (socket says 172.21.15.179:48904) 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 -- 172.21.15.179:0/1451980122 learned_addr learned my addr 172.21.15.179:0/1451980122 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d908c700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 0x7f36d406b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d3fff700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415a300 0x7f36d413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 -- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415a300 msgr2=0x7f36d413b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415a300 0x7f36d413b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 -- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 msgr2=0x7f36d406b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 0x7f36d406b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 -- 172.21.15.179:0/1451980122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40094b0 con 0x7f36d41544e0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d908c700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 0x7f36d406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d888b700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00cbc0 tx=0x7f36bc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00a9e0 con 0x7f36d41544e0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc00ab40 con 0x7f36d41544e0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d41f5550 con 0x7f36d41544e0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc012400 con 0x7f36d41544e0 2024-05-08T22:37:44.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:43.997+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d413b590 con 0x7f36d41544e0 2024-05-08T22:37:44.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d404e700 con 0x7f36d41544e0 2024-05-08T22:37:44.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36bc01b3b0 con 0x7f36d41544e0 2024-05-08T22:37:44.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d17fa700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 0x7f36b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f36bc083f30 con 0x7f36d41544e0 2024-05-08T22:37:44.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d3fff700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 0x7f36b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.001+0000 7f36d3fff700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 0x7f36b4068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36c40090b0 tx=0x7f36c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.009+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f36bc051040 con 0x7f36d41544e0 2024-05-08T22:37:44.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.181+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36d406a990 con 0x7f36d41544e0 2024-05-08T22:37:44.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.181+0000 7f36d17fa700 1 -- 172.21.15.179:0/1451980122 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36bc017020 con 0x7f36d41544e0 2024-05-08T22:37:44.183 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 msgr2=0x7f36b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 0x7f36b4068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36c40090b0 tx=0x7f36c4008030 comp rx=0 tx=0).stop 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 msgr2=0x7f36d41f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00cbc0 tx=0x7f36bc00cca0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 shutdown_connections 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36b4066730 0x7f36b4068bf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d415acc0 0x7f36d406b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36d415a300 0x7f36d413b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 --2- 172.21.15.179:0/1451980122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d41544e0 0x7f36d41f1ff0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 >> 172.21.15.179:0/1451980122 conn(0x7f36d40b18d0 msgr2=0x7f36d40cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 shutdown_connections 2024-05-08T22:37:44.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.185+0000 7f36d988d700 1 -- 172.21.15.179:0/1451980122 wait complete. 2024-05-08T22:37:44.205 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:44.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 Processor -- start 2024-05-08T22:37:44.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 -- start start 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0ad0e0 0x7f936c0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f936c0b25c0 0x7f936c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0ad4c0 con 0x7f936c0ac5c0 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c15ae80 con 0x7f936c0b25c0 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.313+0000 7f9372382700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1609c0 con 0x7f936c0ad0e0 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f936c0b25c0 0x7f936c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9371b81700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f936c0ac5c0 0x7f936c160e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37880/0 (socket says 172.21.15.179:37880) 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f936c0b25c0 0x7f936c0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48914/0 (socket says 172.21.15.179:48914) 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9371b81700 1 -- 172.21.15.179:0/1051537100 learned_addr learned my addr 172.21.15.179:0/1051537100 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9371380700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0ad0e0 0x7f936c0ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 -- 172.21.15.179:0/1051537100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0ad0e0 msgr2=0x7f936c0ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0ad0e0 0x7f936c0ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 -- 172.21.15.179:0/1051537100 >> v1:172.21.15.179:6789/0 conn(0x7f936c0ac5c0 legacy=0x7f936c160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f936bfff700 1 -- 172.21.15.179:0/1051537100 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003210497 0 0) 0x7f936c0ad4c0 con 0x7f936c0ac5c0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 -- 172.21.15.179:0/1051537100 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c15d900 con 0x7f936c0b25c0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9370b7f700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0b25c0 0x7f936c0ad860 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8e92a86046018c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f936bfff700 1 -- 172.21.15.179:0/1051537100 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c011830 con 0x7f936c0b25c0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f936bfff700 1 -- 172.21.15.179:0/1051537100 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c011990 con 0x7f936c0b25c0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f936bfff700 1 -- 172.21.15.179:0/1051537100 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c011d20 con 0x7f936c0b25c0 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- 172.21.15.179:0/1051537100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0b25c0 msgr2=0x7f936c0ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0b25c0 0x7f936c0ad860 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- 172.21.15.179:0/1051537100 shutdown_connections 2024-05-08T22:37:44.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0b25c0 0x7f936c0ad860 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 --2- 172.21.15.179:0/1051537100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0ad0e0 0x7f936c0ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- 172.21.15.179:0/1051537100 >> 172.21.15.179:0/1051537100 conn(0x7f936c0ce9d0 msgr2=0x7f936c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- 172.21.15.179:0/1051537100 shutdown_connections 2024-05-08T22:37:44.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- 172.21.15.179:0/1051537100 wait complete. 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 Processor -- start 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.317+0000 7f9372382700 1 -- start start 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 0x7f936c154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 0x7f936c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0b25c0 0x7f936c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c155530 con 0x7f936c0ac5c0 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c154680 con 0x7f936c0ad0e0 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c154920 con 0x7f936c0b25c0 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9371380700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 0x7f936c154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9371380700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 0x7f936c154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48928/0 (socket says 172.21.15.179:48928) 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 0x7f936c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9371380700 1 -- 172.21.15.179:0/1432492637 learned_addr learned my addr 172.21.15.179:0/1432492637 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9371b81700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0b25c0 0x7f936c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 -- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0b25c0 msgr2=0x7f936c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0b25c0 0x7f936c1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 -- 172.21.15.179:0/1432492637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 msgr2=0x7f936c154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 0x7f936c154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 -- 172.21.15.179:0/1432492637 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0094b0 con 0x7f936c0ad0e0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9370b7f700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 0x7f936c0b3460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f935c016000 tx=0x7f935c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c004610 con 0x7f936c0ad0e0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c01e070 con 0x7f936c0ad0e0 2024-05-08T22:37:44.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c154c00 con 0x7f936c0ad0e0 2024-05-08T22:37:44.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c003900 con 0x7f936c0ad0e0 2024-05-08T22:37:44.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c1551c0 con 0x7f936c0ad0e0 2024-05-08T22:37:44.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.321+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c067740 con 0x7f936c0ad0e0 2024-05-08T22:37:44.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.325+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f935c003a60 con 0x7f936c0ad0e0 2024-05-08T22:37:44.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.325+0000 7f9369ffb700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 0x7f9358068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.325+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f935c085340 con 0x7f936c0ad0e0 2024-05-08T22:37:44.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.325+0000 7f9371380700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 0x7f9358068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.325+0000 7f9371380700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 0x7f9358068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9360009ab0 tx=0x7f9360008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.337+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f935c052570 con 0x7f936c0ad0e0 2024-05-08T22:37:44.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.545+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f936c1556a0 con 0x7f936c0ad0e0 2024-05-08T22:37:44.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.545+0000 7f9369ffb700 1 -- 172.21.15.179:0/1432492637 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f935c055b90 con 0x7f936c0ad0e0 2024-05-08T22:37:44.549 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:44.549 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:37:13.599733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 msgr2=0x7f9358068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 0x7f9358068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9360009ab0 tx=0x7f9360008040 comp rx=0 tx=0).stop 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 msgr2=0x7f936c0b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 0x7f936c0b3460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f935c016000 tx=0x7f935c004770 comp rx=0 tx=0).stop 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 shutdown_connections 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9358066730 0x7f9358068bf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f936c0ad0e0 0x7f936c0b3460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936c0b25c0 0x7f936c1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 --2- 172.21.15.179:0/1432492637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f936c0ac5c0 0x7f936c154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 >> 172.21.15.179:0/1432492637 conn(0x7f936c0ce9d0 msgr2=0x7f936c0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 shutdown_connections 2024-05-08T22:37:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.549+0000 7f9372382700 1 -- 172.21.15.179:0/1432492637 wait complete. 2024-05-08T22:37:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 32 2024-05-08T22:37:44.564 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:44.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 Processor -- start 2024-05-08T22:37:44.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- start start 2024-05-08T22:37:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f62a00ce380 0x7f62a00cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f62a00cb510 0x7f62a006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a006abb0 con 0x7f62a00cc700 2024-05-08T22:37:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a00cb980 con 0x7f62a00ce380 2024-05-08T22:37:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a006ae90 con 0x7f62a00cb510 2024-05-08T22:37:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f62a00ce380 0x7f62a00cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a6452700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f62a00cc700 0x7f62a015b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37886/0 (socket says 172.21.15.179:37886) 2024-05-08T22:37:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a6452700 1 -- 172.21.15.179:0/3344970839 learned_addr learned my addr 172.21.15.179:0/3344970839 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:44.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a6c53700 1 --2- 172.21.15.179:0/3344970839 >> v2:172.21.15.195:3300/0 conn(0x7f62a00cb510 0x7f62a006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 -- 172.21.15.179:0/3344970839 >> v2:172.21.15.195:3300/0 conn(0x7f62a00cb510 msgr2=0x7f62a006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 --2- 172.21.15.179:0/3344970839 >> v2:172.21.15.195:3300/0 conn(0x7f62a00cb510 0x7f62a006b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5450700 1 -- 172.21.15.179:0/3344970839 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563887836 0 0) 0x7f62a006abb0 con 0x7f62a00cc700 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 -- 172.21.15.179:0/3344970839 >> v1:172.21.15.179:6789/0 conn(0x7f62a00cc700 legacy=0x7f62a015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 -- 172.21.15.179:0/3344970839 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00ce800 con 0x7f62a00ce380 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5c51700 1 --2- 172.21.15.179:0/3344970839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00cafd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f62a00b3cf0 tx=0x7f6290009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3033f15083f6e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5450700 1 -- 172.21.15.179:0/3344970839 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290011830 con 0x7f62a00ce380 2024-05-08T22:37:44.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5450700 1 -- 172.21.15.179:0/3344970839 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6290011990 con 0x7f62a00ce380 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a5450700 1 -- 172.21.15.179:0/3344970839 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290011d20 con 0x7f62a00ce380 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- 172.21.15.179:0/3344970839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 msgr2=0x7f62a00cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 --2- 172.21.15.179:0/3344970839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00cafd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f62a00b3cf0 tx=0x7f6290009b30 comp rx=0 tx=0).stop 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- 172.21.15.179:0/3344970839 shutdown_connections 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 --2- 172.21.15.179:0/3344970839 >> v2:172.21.15.195:3300/0 conn(0x7f62a00cb510 0x7f62a006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 --2- 172.21.15.179:0/3344970839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00cafd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- 172.21.15.179:0/3344970839 >> 172.21.15.179:0/3344970839 conn(0x7f62a00ce9d0 msgr2=0x7f62a00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- 172.21.15.179:0/3344970839 shutdown_connections 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.673+0000 7f62a7454700 1 -- 172.21.15.179:0/3344970839 wait complete. 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 Processor -- start 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- start start 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62a00cb510 0x7f62a00c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a00cc700 0x7f62a00c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0142e10 con 0x7f62a00ce380 2024-05-08T22:37:44.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0141f90 con 0x7f62a00cc700 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0142220 con 0x7f62a00cb510 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48932/0 (socket says 172.21.15.179:48932) 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6452700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62a00cb510 0x7f62a00c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 -- 172.21.15.179:0/3105047598 learned_addr learned my addr 172.21.15.179:0/3105047598 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a5c51700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a00cc700 0x7f62a00c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 -- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62a00cb510 msgr2=0x7f62a00c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62a00cb510 0x7f62a00c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 -- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a00cc700 msgr2=0x7f62a00c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a00cc700 0x7f62a00c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 -- 172.21.15.179:0/3105047598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900094b0 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a6c53700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f629800efc0 tx=0x7f629800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298009ea0 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800f040 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a0142470 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800ced0 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.677+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a0142a30 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6298014480 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a0142f50 con 0x7f62a00ce380 2024-05-08T22:37:44.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f6296ffd700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 0x7f6284068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f6298084ae0 con 0x7f62a00ce380 2024-05-08T22:37:44.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f62a6452700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 0x7f6284068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.681+0000 7f62a6452700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 0x7f6284068bf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f629c000f60 tx=0x7f629c007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.689+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f62a0142f50 con 0x7f62a00ce380 2024-05-08T22:37:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.857+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62a0142f50 con 0x7f62a00ce380 2024-05-08T22:37:44.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f6296ffd700 1 -- 172.21.15.179:0/3105047598 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16535 (secure 0 0 0) 0x7f62a0142f50 con 0x7f62a00ce380 2024-05-08T22:37:44.863 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:44.864 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":71,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:06.562633+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:44.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 msgr2=0x7f6284068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 0x7f6284068bf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f629c000f60 tx=0x7f629c007e00 comp rx=0 tx=0).stop 2024-05-08T22:37:44.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 msgr2=0x7f62a00c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f629800efc0 tx=0x7f629800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 shutdown_connections 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6284066730 0x7f6284068bf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a00cc700 0x7f62a00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62a00ce380 0x7f62a00c6760 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 --2- 172.21.15.179:0/3105047598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62a00cb510 0x7f62a00c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 >> 172.21.15.179:0/3105047598 conn(0x7f62a00ce9d0 msgr2=0x7f62a00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.861+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 shutdown_connections 2024-05-08T22:37:44.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.865+0000 7f62a7454700 1 -- 172.21.15.179:0/3105047598 wait complete. 2024-05-08T22:37:44.879 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:37:44.879 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:44.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 Processor -- start 2024-05-08T22:37:44.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 -- start start 2024-05-08T22:37:44.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2f9c0cde70 0x7f2f9c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c0ce810 con 0x7f2f9c15d5b0 2024-05-08T22:37:44.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c06afc0 con 0x7f2f9c06b450 2024-05-08T22:37:44.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9c15d990 con 0x7f2f9c0cde70 2024-05-08T22:37:44.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9affd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2f9c15d5b0 0x7f2f9c0b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37900/0 (socket says 172.21.15.179:37900) 2024-05-08T22:37:44.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9affd700 1 -- 172.21.15.179:0/897488286 learned_addr learned my addr 172.21.15.179:0/897488286 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:44.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f927fc700 1 --2- 172.21.15.179:0/897488286 >> v2:172.21.15.195:3300/0 conn(0x7f2f9c0cde70 0x7f2f9c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9a7fc700 1 -- 172.21.15.179:0/897488286 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820216870 0 0) 0x7f2f9c0ce810 con 0x7f2f9c15d5b0 2024-05-08T22:37:44.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9a7fc700 1 -- 172.21.15.179:0/897488286 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f880035d0 con 0x7f2f9c15d5b0 2024-05-08T22:37:44.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f927fc700 1 -- 172.21.15.179:0/897488286 >> v1:172.21.15.195:6789/0 conn(0x7f2f9c06b450 legacy=0x7f2f9c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f927fc700 1 -- 172.21.15.179:0/897488286 >> v1:172.21.15.179:6789/0 conn(0x7f2f9c15d5b0 legacy=0x7f2f9c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f927fc700 1 -- 172.21.15.179:0/897488286 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0d26f0 con 0x7f2f9c0cde70 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f927fc700 1 --2- 172.21.15.179:0/897488286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c0cde70 0x7f2f9c0ce2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f84009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14565ed2137d3537 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9a7fc700 1 -- 172.21.15.179:0/897488286 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84011830 con 0x7f2f9c0cde70 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9a7fc700 1 -- 172.21.15.179:0/897488286 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84011990 con 0x7f2f9c0cde70 2024-05-08T22:37:44.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.989+0000 7f2f9a7fc700 1 -- 172.21.15.179:0/897488286 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84011d40 con 0x7f2f9c0cde70 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- 172.21.15.179:0/897488286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c0cde70 msgr2=0x7f2f9c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/897488286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c0cde70 0x7f2f9c0ce2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f84009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- 172.21.15.179:0/897488286 shutdown_connections 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/897488286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c0cde70 0x7f2f9c0ce2d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- 172.21.15.179:0/897488286 >> 172.21.15.179:0/897488286 conn(0x7f2f9c0b4940 msgr2=0x7f2f9c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- 172.21.15.179:0/897488286 shutdown_connections 2024-05-08T22:37:44.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- 172.21.15.179:0/897488286 wait complete. 2024-05-08T22:37:44.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 Processor -- start 2024-05-08T22:37:44.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- start start 2024-05-08T22:37:44.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f9c0cde70 0x7f2f9c144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 0x7f2f9c145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:44.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c153d80 con 0x7f2f9c15d5b0 2024-05-08T22:37:44.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1604f0 con 0x7f2f9c06b450 2024-05-08T22:37:44.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c15f8e0 con 0x7f2f9c0cde70 2024-05-08T22:37:44.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9affd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 0x7f2f9c145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:44.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9affd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:37984/0 (socket says 172.21.15.179:37984) 2024-05-08T22:37:44.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f9affd700 1 -- 172.21.15.179:0/2596813661 learned_addr learned my addr 172.21.15.179:0/2596813661 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:45.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.993+0000 7f2f927fc700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f9c0cde70 0x7f2f9c144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 -- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f9c0cde70 msgr2=0x7f2f9c144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f9c0cde70 0x7f2f9c144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 -- 172.21.15.179:0/2596813661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 msgr2=0x7f2f9c145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 0x7f2f9c145520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 -- 172.21.15.179:0/2596813661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840094b0 con 0x7f2f9c06b450 2024-05-08T22:37:45.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9b7fe700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 0x7f2f9c145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:45.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9affd700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00c700 tx=0x7f2f8c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c005d90 con 0x7f2f9c06b450 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0070d0 con 0x7f2f9c06b450 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c15fbc0 con 0x7f2f9c06b450 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c007480 con 0x7f2f9c06b450 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c160180 con 0x7f2f9c06b450 2024-05-08T22:37:45.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c04e700 con 0x7f2f9c06b450 2024-05-08T22:37:45.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:44.997+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f8c0075e0 con 0x7f2f9c06b450 2024-05-08T22:37:45.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.001+0000 7f2f98ff9700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 0x7f2f7c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.001+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2f8c083c20 con 0x7f2f9c06b450 2024-05-08T22:37:45.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.001+0000 7f2f927fc700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 0x7f2f7c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.001+0000 7f2f927fc700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 0x7f2f7c068bf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f840090b0 tx=0x7f2f84008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.009+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2f8c050ea0 con 0x7f2f9c06b450 2024-05-08T22:37:45.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.221+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f9c0d0e50 con 0x7f2f9c06b450 2024-05-08T22:37:45.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.221+0000 7f2f98ff9700 1 -- 172.21.15.179:0/2596813661 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f2f8c017020 con 0x7f2f9c06b450 2024-05-08T22:37:45.226 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:45.226 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:37:13.599733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 msgr2=0x7f2f7c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 0x7f2f7c068bf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2f840090b0 tx=0x7f2f84008030 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 msgr2=0x7f2f9c153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00c700 tx=0x7f2f8c00ca10 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 shutdown_connections 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f7c066730 0x7f2f7c068bf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f9c15d5b0 0x7f2f9c145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f9c0cde70 0x7f2f9c144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 --2- 172.21.15.179:0/2596813661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f9c06b450 0x7f2f9c153840 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 >> 172.21.15.179:0/2596813661 conn(0x7f2f9c0b4940 msgr2=0x7f2f9c0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:45.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 shutdown_connections 2024-05-08T22:37:45.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.225+0000 7f2f9bfff700 1 -- 172.21.15.179:0/2596813661 wait complete. 2024-05-08T22:37:45.231 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 32 2024-05-08T22:37:45.247 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:45.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 Processor -- start 2024-05-08T22:37:45.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 -- start start 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 0x7fcfe006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcfe006e700 0x7fcfe00ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe006c9b0 con 0x7fcfe0143030 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe006cc30 con 0x7fcfe006e700 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00cda90 con 0x7fcfe006c550 2024-05-08T22:37:45.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcfe006e700 0x7fcfe00ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcfe006e700 0x7fcfe00ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37254/0 (socket says 172.21.15.179:37254) 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 -- 172.21.15.179:0/1040532165 learned_addr learned my addr 172.21.15.179:0/1040532165 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe4bcf700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 0x7fcfe006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 -- 172.21.15.179:0/1040532165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 msgr2=0x7fcfe006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 0x7fcfe006e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 -- 172.21.15.179:0/1040532165 >> v1:172.21.15.195:6789/0 conn(0x7fcfe0143030 legacy=0x7fcfe015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 -- 172.21.15.179:0/1040532165 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00cdd70 con 0x7fcfe006e700 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe4bcf700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 0x7fcfe006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe5bd1700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006e700 0x7fcfe00ca140 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009940 tx=0x7fcfd0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a2f658852cd3b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfdffff700 1 -- 172.21.15.179:0/1040532165 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000ba20 con 0x7fcfe006e700 2024-05-08T22:37:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfdffff700 1 -- 172.21.15.179:0/1040532165 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0009e90 con 0x7fcfe006e700 2024-05-08T22:37:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfdffff700 1 -- 172.21.15.179:0/1040532165 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000bda0 con 0x7fcfe006e700 2024-05-08T22:37:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1040532165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006e700 msgr2=0x7fcfe00ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.361+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006e700 0x7fcfe00ca140 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009940 tx=0x7fcfd0009c50 comp rx=0 tx=0).stop 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1040532165 shutdown_connections 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006c550 0x7fcfe006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1040532165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006e700 0x7fcfe00ca140 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1040532165 >> 172.21.15.179:0/1040532165 conn(0x7fcfe00b17c0 msgr2=0x7fcfe00b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1040532165 shutdown_connections 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1040532165 wait complete. 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 Processor -- start 2024-05-08T22:37:45.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- start start 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006e700 0x7fcfe006b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe0143030 0x7fcfe01406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0128e50 con 0x7fcfe006c550 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0129130 con 0x7fcfe0143030 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe63d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0129410 con 0x7fcfe006e700 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe53d0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe53d0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37258/0 (socket says 172.21.15.179:37258) 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe53d0700 1 -- 172.21.15.179:0/1131320412 learned_addr learned my addr 172.21.15.179:0/1131320412 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe5bd1700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe0143030 0x7fcfe01406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.365+0000 7fcfe4bcf700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006e700 0x7fcfe006b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 -- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006e700 msgr2=0x7fcfe006b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006e700 0x7fcfe006b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 -- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe0143030 msgr2=0x7fcfe01406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe0143030 0x7fcfe01406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 -- 172.21.15.179:0/1131320412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0009570 con 0x7fcfe006c550 2024-05-08T22:37:45.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe53d0700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800cbd0 tx=0x7fcfc800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8003c20 con 0x7fcfe006c550 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8003d80 con 0x7fcfe006c550 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe01296f0 con 0x7fcfe006c550 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800aae0 con 0x7fcfe006c550 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe00b3800 con 0x7fcfe006c550 2024-05-08T22:37:45.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.369+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0129c10 con 0x7fcfe006c550 2024-05-08T22:37:45.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.373+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfc800ac40 con 0x7fcfe006c550 2024-05-08T22:37:45.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.373+0000 7fcfddffb700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 0x7fcfc0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.373+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fcfc8084c80 con 0x7fcfe006c550 2024-05-08T22:37:45.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.373+0000 7fcfe4bcf700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 0x7fcfc0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.373+0000 7fcfe4bcf700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 0x7fcfc0068bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80044b0 tx=0x7fcfd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.377+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcfc8051e00 con 0x7fcfe006c550 2024-05-08T22:37:45.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.585+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfe0129c10 con 0x7fcfe006c550 2024-05-08T22:37:45.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.585+0000 7fcfddffb700 1 -- 172.21.15.179:0/1131320412 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4749 (secure 0 0 0) 0x7fcfe0129c10 con 0x7fcfe006c550 2024-05-08T22:37:45.589 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:45.589 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:36:25.077203+0000","modified":"2024-05-08T22:37:13.599733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55718":{"gid":55718,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6836/3517137124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3517137124},{"type":"v1","addr":"172.21.15.195:6836","nonce":3517137124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 msgr2=0x7fcfc0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 0x7fcfc0068bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80044b0 tx=0x7fcfd8008040 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 msgr2=0x7fcfe00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800cbd0 tx=0x7fcfc800cee0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 shutdown_connections 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfc0066730 0x7fcfc0068bf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe0143030 0x7fcfe01406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe006e700 0x7fcfe006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 --2- 172.21.15.179:0/1131320412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfe006c550 0x7fcfe00b32c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 >> 172.21.15.179:0/1131320412 conn(0x7fcfe00b17c0 msgr2=0x7fcfe015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:45.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 shutdown_connections 2024-05-08T22:37:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.589+0000 7fcfe63d2700 1 -- 172.21.15.179:0/1131320412 wait complete. 2024-05-08T22:37:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 32 2024-05-08T22:37:45.608 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:45.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.717+0000 7f0cf16e5700 1 Processor -- start 2024-05-08T22:37:45.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.717+0000 7f0cf16e5700 1 -- start start 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cec0cc700 0x7f0cec0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0cec0cb6e0 0x7f0cec06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cec06abb0 con 0x7f0cec15b1b0 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec06ae90 con 0x7f0cec0cc700 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec15b9c0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cec0cc700 0x7f0cec0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cec0cc700 0x7f0cec0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37274/0 (socket says 172.21.15.179:37274) 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 -- 172.21.15.179:0/3693530319 learned_addr learned my addr 172.21.15.179:0/3693530319 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf0ee4700 1 --2- 172.21.15.179:0/3693530319 >> v2:172.21.15.195:3300/0 conn(0x7f0cec0cb6e0 0x7f0cec06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 -- 172.21.15.179:0/3693530319 >> v2:172.21.15.195:3300/0 conn(0x7f0cec0cb6e0 msgr2=0x7f0cec06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 --2- 172.21.15.179:0/3693530319 >> v2:172.21.15.195:3300/0 conn(0x7f0cec0cb6e0 0x7f0cec06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 -- 172.21.15.179:0/3693530319 >> v1:172.21.15.195:6789/0 conn(0x7f0cec15b1b0 legacy=0x7f0cec0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 -- 172.21.15.179:0/3693530319 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec15bca0 con 0x7f0cec0cc700 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cebfff700 1 --2- 172.21.15.179:0/3693530319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cc700 0x7f0cec0ce380 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3cf0 tx=0x7f0ce400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dc0005fd9cd0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0ceaffd700 1 -- 172.21.15.179:0/3693530319 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400e070 con 0x7f0cec0cc700 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0ceaffd700 1 -- 172.21.15.179:0/3693530319 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400b9c0 con 0x7f0cec0cc700 2024-05-08T22:37:45.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0ceaffd700 1 -- 172.21.15.179:0/3693530319 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400bd70 con 0x7f0cec0cc700 2024-05-08T22:37:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3693530319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cc700 msgr2=0x7f0cec0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3693530319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cc700 0x7f0cec0ce380 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3cf0 tx=0x7f0ce400b330 comp rx=0 tx=0).stop 2024-05-08T22:37:45.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3693530319 shutdown_connections 2024-05-08T22:37:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3693530319 >> v2:172.21.15.195:3300/0 conn(0x7f0cec0cb6e0 0x7f0cec06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3693530319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cc700 0x7f0cec0ce380 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3693530319 >> 172.21.15.179:0/3693530319 conn(0x7f0cec0ce9d0 msgr2=0x7f0cec0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3693530319 shutdown_connections 2024-05-08T22:37:45.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.721+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3693530319 wait complete. 2024-05-08T22:37:45.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 Processor -- start 2024-05-08T22:37:45.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 -- start start 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec0cc700 0x7f0cec0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cec15b1b0 0x7f0cec0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec1401e0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec13ceb0 con 0x7f0cec0cc700 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.725+0000 7f0cf16e5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec13f5d0 con 0x7f0cec15b1b0 2024-05-08T22:37:45.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37278/0 (socket says 172.21.15.179:37278) 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 -- 172.21.15.179:0/3083532268 learned_addr learned my addr 172.21.15.179:0/3083532268 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cf0ee4700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cec15b1b0 0x7f0cec0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ceb7fe700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec0cc700 0x7f0cec0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 -- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cec15b1b0 msgr2=0x7f0cec0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cec15b1b0 0x7f0cec0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 -- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec0cc700 msgr2=0x7f0cec0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec0cc700 0x7f0cec0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 -- 172.21.15.179:0/3083532268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009b30 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cebfff700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3e80 tx=0x7f0ce4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40048c0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce40099b0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec13f8b0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401c680 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec13fe70 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cec067740 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ce401c7e0 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.729+0000 7f0ce8ff9700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 0x7f0cd8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:45.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.733+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f0ce4085e20 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.733+0000 7f0ceb7fe700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 0x7f0cd8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:45.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.733+0000 7f0ceb7fe700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 0x7f0cd8068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0cec15ab90 tx=0x7f0ce0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:45.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.737+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0ce4053050 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.909+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cec04e700 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.913+0000 7f0ce8ff9700 1 -- 172.21.15.179:0/3083532268 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16535 (secure 0 0 0) 0x7f0ce4056670 con 0x7f0cec0cb6e0 2024-05-08T22:37:45.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:45.916 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":71,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:06.562633+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:36:23.437690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T22:36:24.076455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.35","mappings":[{"from":4,"to":3}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 msgr2=0x7f0cd8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 0x7f0cd8068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0cec15ab90 tx=0x7f0ce0009040 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 msgr2=0x7f0cec0ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3e80 tx=0x7f0ce4004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 shutdown_connections 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cd8066730 0x7f0cd8068bf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec0cc700 0x7f0cec0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cec15b1b0 0x7f0cec0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 --2- 172.21.15.179:0/3083532268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cec0cb6e0 0x7f0cec0ab2e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 >> 172.21.15.179:0/3083532268 conn(0x7f0cec0ce9d0 msgr2=0x7f0cec0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:45.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 shutdown_connections 2024-05-08T22:37:45.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:45.917+0000 7f0cf16e5700 1 -- 172.21.15.179:0/3083532268 wait complete. 2024-05-08T22:37:45.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:37:46.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 Processor -- start 2024-05-08T22:37:46.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 -- start start 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 0x7fae4c0eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4c04fe20 con 0x7fae4c048210 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4c0ebe70 con 0x7fae4c033fb0 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae5aba3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c04fb80 con 0x7fae4c047cc0 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae593a0700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fae4c048210 0x7fae4c052f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60734/0 (socket says 172.21.15.179:60734) 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.053+0000 7fae593a0700 1 -- 172.21.15.179:0/2615765306 learned_addr learned my addr 172.21.15.179:0/2615765306 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:46.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5a3a2700 1 --2- 172.21.15.179:0/2615765306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 0x7fae4c0eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae58b9f700 1 -- 172.21.15.179:0/2615765306 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782727860 0 0) 0x7fae4c04fe20 con 0x7fae4c048210 2024-05-08T22:37:46.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae58b9f700 1 -- 172.21.15.179:0/2615765306 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae400035d0 con 0x7fae4c048210 2024-05-08T22:37:46.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2615765306 >> v1:172.21.15.195:6789/0 conn(0x7fae4c033fb0 legacy=0x7fae4c0eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2615765306 >> v1:172.21.15.179:6789/0 conn(0x7fae4c048210 legacy=0x7fae4c052f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2615765306 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c0ec0c0 con 0x7fae4c047cc0 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5a3a2700 1 --2- 172.21.15.179:0/2615765306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 0x7fae4c0eb930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae5404f480 tx=0x7fae5404f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8ceef0dd0525ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae58b9f700 1 -- 172.21.15.179:0/2615765306 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54067250 con 0x7fae4c047cc0 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae58b9f700 1 -- 172.21.15.179:0/2615765306 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae540673b0 con 0x7fae4c047cc0 2024-05-08T22:37:46.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae58b9f700 1 -- 172.21.15.179:0/2615765306 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54067960 con 0x7fae4c047cc0 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 -- 172.21.15.179:0/2615765306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 msgr2=0x7fae4c0eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2615765306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 0x7fae4c0eb930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae5404f480 tx=0x7fae5404f710 comp rx=0 tx=0).stop 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 -- 172.21.15.179:0/2615765306 shutdown_connections 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2615765306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c047cc0 0x7fae4c0eb930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 -- 172.21.15.179:0/2615765306 >> 172.21.15.179:0/2615765306 conn(0x7fae4c054b70 msgr2=0x7fae4c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 -- 172.21.15.179:0/2615765306 shutdown_connections 2024-05-08T22:37:46.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.057+0000 7fae5aba3700 1 -- 172.21.15.179:0/2615765306 wait complete. 2024-05-08T22:37:46.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 Processor -- start 2024-05-08T22:37:46.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 -- start start 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c033fb0 0x7fae4c0d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae4c047cc0 0x7fae4c17d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c0d5800 con 0x7fae4c048210 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c17e7e0 con 0x7fae4c047cc0 2024-05-08T22:37:46.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.061+0000 7fae5aba3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c17df40 con 0x7fae4c033fb0 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37282/0 (socket says 172.21.15.179:37282) 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2872208166 learned_addr learned my addr 172.21.15.179:0/2872208166 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae593a0700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae4c047cc0 0x7fae4c17d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c033fb0 msgr2=0x7fae4c0d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c033fb0 0x7fae4c0d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae4c047cc0 msgr2=0x7fae4c17d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae4c047cc0 0x7fae4c17d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 -- 172.21.15.179:0/2872208166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c17e220 con 0x7fae4c048210 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5a3a2700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae54074ec0 tx=0x7fae540739d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54066eb0 con 0x7fae4c048210 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54066150 con 0x7fae4c048210 2024-05-08T22:37:46.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c17e500 con 0x7fae4c048210 2024-05-08T22:37:46.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54066520 con 0x7fae4c048210 2024-05-08T22:37:46.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c17dbc0 con 0x7fae4c048210 2024-05-08T22:37:46.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c049f60 con 0x7fae4c048210 2024-05-08T22:37:46.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.065+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae54066680 con 0x7fae4c048210 2024-05-08T22:37:46.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.069+0000 7fae467fc700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 0x7fae38068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.069+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fae54099070 con 0x7fae4c048210 2024-05-08T22:37:46.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.069+0000 7fae59ba1700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 0x7fae38068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.069+0000 7fae59ba1700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 0x7fae38068c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae50000f60 tx=0x7fae50007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.077+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae540cc3d0 con 0x7fae4c048210 2024-05-08T22:37:46.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.285+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fae4c049f60 con 0x7fae4c048210 2024-05-08T22:37:46.289 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:37:46.285+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:37:46.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.301+0000 7fae467fc700 1 -- 172.21.15.179:0/2872208166 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fae4c049f60 con 0x7fae4c048210 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 msgr2=0x7fae38068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 0x7fae38068c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae50000f60 tx=0x7fae50007a80 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 msgr2=0x7fae4c039470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae54074ec0 tx=0x7fae540739d0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 shutdown_connections 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae38066780 0x7fae38068c40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae4c047cc0 0x7fae4c17d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c048210 0x7fae4c039470 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 --2- 172.21.15.179:0/2872208166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae4c033fb0 0x7fae4c0d52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 >> 172.21.15.179:0/2872208166 conn(0x7fae4c054b70 msgr2=0x7fae4c005e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:46.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 shutdown_connections 2024-05-08T22:37:46.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.305+0000 7fae5aba3700 1 -- 172.21.15.179:0/2872208166 wait complete. 2024-05-08T22:37:46.308 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:37:46.310 INFO:tasks.ceph.mds.b.smithi195.stderr: -4541> 2024-05-08T22:36:37.801+0000 7f5f0dd38700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6669=0-4194304@1} caps={6669=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6669 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56070a091b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:37:46.311 INFO:tasks.ceph.mds.b.smithi195.stderr: -2729> 2024-05-08T22:37:09.309+0000 7f5f0dd38700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-08T22:36:36.669919+0000 1=1+0) (iversion lock) caps={6669=pAsLsXsFs/-@2},l=6669(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56070a091b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:37:46.322 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:37:46.341 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:37:46.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 Processor -- start 2024-05-08T22:37:46.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 -- start start 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f16900d29a0 0x7f16900b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16900cdb10 0x7f16900cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900ce460 con 0x7f169015d0e0 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169015d4c0 con 0x7f16900cdb10 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16900b6f80 con 0x7f16900d29a0 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16900cdb10 0x7f16900cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16900cdb10 0x7f16900cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37288/0 (socket says 172.21.15.179:37288) 2024-05-08T22:37:46.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 -- 172.21.15.179:0/89295093 learned_addr learned my addr 172.21.15.179:0/89295093 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168e59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f169015d0e0 0x7f16900d1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60736/0 (socket says 172.21.15.179:60736) 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168dd9b700 1 --2- 172.21.15.179:0/89295093 >> v2:172.21.15.195:3300/0 conn(0x7f16900d29a0 0x7f16900b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 -- 172.21.15.179:0/89295093 >> v2:172.21.15.195:3300/0 conn(0x7f16900d29a0 msgr2=0x7f16900b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 --2- 172.21.15.179:0/89295093 >> v2:172.21.15.195:3300/0 conn(0x7f16900d29a0 0x7f16900b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 -- 172.21.15.179:0/89295093 >> v1:172.21.15.179:6789/0 conn(0x7f169015d0e0 legacy=0x7f16900d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168d59a700 1 -- 172.21.15.179:0/89295093 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388685473 0 0) 0x7f16900ce460 con 0x7f169015d0e0 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 -- 172.21.15.179:0/89295093 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900d1900 con 0x7f16900cdb10 2024-05-08T22:37:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168ed9d700 1 --2- 172.21.15.179:0/89295093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900cdb10 0x7f16900cdf20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f16900b48e0 tx=0x7f1688009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7425601832fc0ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168d59a700 1 -- 172.21.15.179:0/89295093 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011830 con 0x7f16900cdb10 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168d59a700 1 -- 172.21.15.179:0/89295093 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688011990 con 0x7f16900cdb10 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.433+0000 7f168d59a700 1 -- 172.21.15.179:0/89295093 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011d20 con 0x7f16900cdb10 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- 172.21.15.179:0/89295093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900cdb10 msgr2=0x7f16900cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- 172.21.15.179:0/89295093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900cdb10 0x7f16900cdf20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f16900b48e0 tx=0x7f1688009bf0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- 172.21.15.179:0/89295093 shutdown_connections 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- 172.21.15.179:0/89295093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900cdb10 0x7f16900cdf20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- 172.21.15.179:0/89295093 >> v2:172.21.15.195:3300/0 conn(0x7f16900d29a0 0x7f16900b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- 172.21.15.179:0/89295093 >> 172.21.15.179:0/89295093 conn(0x7f16900d3260 msgr2=0x7f16900d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- 172.21.15.179:0/89295093 shutdown_connections 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- 172.21.15.179:0/89295093 wait complete. 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 Processor -- start 2024-05-08T22:37:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- start start 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16900cdb10 0x7f16900ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 0x7f169015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900cc6e0 con 0x7f16900d29a0 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900cba60 con 0x7f169015d0e0 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900cbd40 con 0x7f16900cdb10 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.437+0000 7f168dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37304/0 (socket says 172.21.15.179:37304) 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168dd9b700 1 -- 172.21.15.179:0/4210457880 learned_addr learned my addr 172.21.15.179:0/4210457880 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:46.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 0x7f169015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168e59c700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16900cdb10 0x7f16900ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 -- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16900cdb10 msgr2=0x7f16900ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16900cdb10 0x7f16900ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 -- 172.21.15.179:0/4210457880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 msgr2=0x7f16900c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 -- 172.21.15.179:0/4210457880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688009570 con 0x7f169015d0e0 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168dd9b700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168ed9d700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 0x7f169015ac10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1688016000 tx=0x7f1688004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688004450 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16880045b0 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688003900 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16900cc020 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1690156bb0 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16900cc820 con 0x7f169015d0e0 2024-05-08T22:37:46.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.441+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1688003a60 con 0x7f169015d0e0 2024-05-08T22:37:46.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.445+0000 7f1686ffd700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 0x7f1674068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:46.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.445+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f1688021030 con 0x7f169015d0e0 2024-05-08T22:37:46.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.445+0000 7f168e59c700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 0x7f1674068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:46.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.445+0000 7f168e59c700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 0x7f1674068c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f16800012e0 tx=0x7f1680001aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:46.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.453+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1688052920 con 0x7f169015d0e0 2024-05-08T22:37:46.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:46.661+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f16900cc820 con 0x7f169015d0e0 2024-05-08T22:37:47.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.305+0000 7f1686ffd700 1 -- 172.21.15.179:0/4210457880 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16900cc820 con 0x7f169015d0e0 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 msgr2=0x7f1674068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 0x7f1674068c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f16800012e0 tx=0x7f1680001aa0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 msgr2=0x7f169015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 0x7f169015ac10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1688016000 tx=0x7f1688004770 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 shutdown_connections 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066760 0x7f1674068c20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169015d0e0 0x7f169015ac10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16900d29a0 0x7f16900c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 --2- 172.21.15.179:0/4210457880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16900cdb10 0x7f16900ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 >> 172.21.15.179:0/4210457880 conn(0x7f16900d3260 msgr2=0x7f16900d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 shutdown_connections 2024-05-08T22:37:47.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.309+0000 7f168f59e700 1 -- 172.21.15.179:0/4210457880 wait complete. 2024-05-08T22:37:47.330 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:37:47.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.453+0000 7fbc662ab700 1 Processor -- start 2024-05-08T22:37:47.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.453+0000 7fbc662ab700 1 -- start start 2024-05-08T22:37:47.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc60158bc0 0x7fbc600b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc600b3c40 0x7fbc600cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 0x7fbc600cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc60158fa0 con 0x7fbc600b3c40 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc600b2e90 con 0x7fbc600cbf10 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600cc870 con 0x7fbc60158bc0 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc600b3c40 0x7fbc600cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc600b3c40 0x7fbc600cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37318/0 (socket says 172.21.15.179:37318) 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 -- 172.21.15.179:0/3995603975 learned_addr learned my addr 172.21.15.179:0/3995603975 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc65aaa700 1 --2- 172.21.15.179:0/3995603975 >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 0x7fbc600cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc652a9700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc60158bc0 0x7fbc600b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 -- 172.21.15.179:0/3995603975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc60158bc0 msgr2=0x7fbc600b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc60158bc0 0x7fbc600b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 -- 172.21.15.179:0/3995603975 >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 msgr2=0x7fbc600cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:47.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 --2- 172.21.15.179:0/3995603975 >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 0x7fbc600cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 -- 172.21.15.179:0/3995603975 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600ce4f0 con 0x7fbc600b3c40 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc65aaa700 1 --2- 172.21.15.179:0/3995603975 >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 0x7fbc600cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc64aa8700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc600b3c40 0x7fbc600cb3d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006f190 tx=0x7fbc5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1769b4e3dc1f4f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc5ffff700 1 -- 172.21.15.179:0/3995603975 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5000e070 con 0x7fbc600b3c40 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc5ffff700 1 -- 172.21.15.179:0/3995603975 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5000b9c0 con 0x7fbc600b3c40 2024-05-08T22:37:47.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc5ffff700 1 -- 172.21.15.179:0/3995603975 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5000bd70 con 0x7fbc600b3c40 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- 172.21.15.179:0/3995603975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc600b3c40 msgr2=0x7fbc600cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc600b3c40 0x7fbc600cb3d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006f190 tx=0x7fbc5000b330 comp rx=0 tx=0).stop 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- 172.21.15.179:0/3995603975 shutdown_connections 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3995603975 >> v2:172.21.15.195:3300/0 conn(0x7fbc600cbf10 0x7fbc600cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc600b3c40 0x7fbc600cb3d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3995603975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc60158bc0 0x7fbc600b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- 172.21.15.179:0/3995603975 >> 172.21.15.179:0/3995603975 conn(0x7fbc600ce9a0 msgr2=0x7fbc600cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:47.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- 172.21.15.179:0/3995603975 shutdown_connections 2024-05-08T22:37:47.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.457+0000 7fbc662ab700 1 -- 172.21.15.179:0/3995603975 wait complete. 2024-05-08T22:37:47.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.461+0000 7fbc662ab700 1 Processor -- start 2024-05-08T22:37:47.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.461+0000 7fbc662ab700 1 -- start start 2024-05-08T22:37:47.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 0x7fbc6014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc600cbf10 0x7fbc6006b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60154540 con 0x7fbc60158bc0 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc601573d0 con 0x7fbc600b3c40 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60154320 con 0x7fbc600cbf10 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc652a9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 0x7fbc6014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc64aa8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc600cbf10 0x7fbc6006b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37324/0 (socket says 172.21.15.179:37324) 2024-05-08T22:37:47.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 -- 172.21.15.179:0/3970241820 learned_addr learned my addr 172.21.15.179:0/3970241820 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:47.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 -- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc600cbf10 msgr2=0x7fbc6006b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:47.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc600cbf10 0x7fbc6006b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 -- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 msgr2=0x7fbc6014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 0x7fbc6014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 -- 172.21.15.179:0/3970241820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50009b30 con 0x7fbc60158bc0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc652a9700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 0x7fbc6014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc65aaa700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800d800 tx=0x7fbc5800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58004030 con 0x7fbc60158bc0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc58004190 con 0x7fbc60158bc0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6006cd80 con 0x7fbc60158bc0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58004520 con 0x7fbc60158bc0 2024-05-08T22:37:47.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6006d340 con 0x7fbc60158bc0 2024-05-08T22:37:47.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.465+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6004e770 con 0x7fbc60158bc0 2024-05-08T22:37:47.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.469+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc580036c0 con 0x7fbc60158bc0 2024-05-08T22:37:47.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.469+0000 7fbc5dffb700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 0x7fbc4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:47.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.469+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fbc58083bf0 con 0x7fbc60158bc0 2024-05-08T22:37:47.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.469+0000 7fbc652a9700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 0x7fbc4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:47.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.469+0000 7fbc652a9700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 0x7fbc4c068bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007e50 tx=0x7fbc54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:47.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.477+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc58050dc0 con 0x7fbc60158bc0 2024-05-08T22:37:47.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:47.649+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc600ccb60 con 0x7fbc60158bc0 2024-05-08T22:37:48.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.317+0000 7fbc5dffb700 1 -- 172.21.15.179:0/3970241820 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbc58019020 con 0x7fbc60158bc0 2024-05-08T22:37:48.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 msgr2=0x7fbc4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 0x7fbc4c068bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007e50 tx=0x7fbc54008040 comp rx=0 tx=0).stop 2024-05-08T22:37:48.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 msgr2=0x7fbc6006bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800d800 tx=0x7fbc5800db10 comp rx=0 tx=0).stop 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 shutdown_connections 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc4c066730 0x7fbc4c068bf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc60158bc0 0x7fbc6006bb10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc600cbf10 0x7fbc6006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 --2- 172.21.15.179:0/3970241820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc600b3c40 0x7fbc6014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 >> 172.21.15.179:0/3970241820 conn(0x7fbc600ce9a0 msgr2=0x7fbc600ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.321+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 shutdown_connections 2024-05-08T22:37:48.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.325+0000 7fbc662ab700 1 -- 172.21.15.179:0/3970241820 wait complete. 2024-05-08T22:37:48.327 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:37:48.339 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:37:48.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.457+0000 7f6e17fff700 1 Processor -- start 2024-05-08T22:37:48.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.457+0000 7f6e17fff700 1 -- start start 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e180d0dc0 0x7f6e180d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e1806abb0 0x7f6e1806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e180b8d70 con 0x7f6e180d2290 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1806c560 con 0x7f6e180d0dc0 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1806bfa0 con 0x7f6e1806abb0 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e180d0dc0 0x7f6e180d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e167fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6e180d2290 0x7f6e180b8660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60746/0 (socket says 172.21.15.179:60746) 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e167fc700 1 -- 172.21.15.179:0/3436529834 learned_addr learned my addr 172.21.15.179:0/3436529834 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e177fe700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e1806abb0 0x7f6e1806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e15ffb700 1 -- 172.21.15.179:0/3436529834 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750996306 0 0) 0x7f6e180b8d70 con 0x7f6e180d2290 2024-05-08T22:37:48.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e15ffb700 1 -- 172.21.15.179:0/3436529834 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e000035d0 con 0x7f6e180d2290 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 -- 172.21.15.179:0/3436529834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e1806abb0 msgr2=0x7f6e1806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e1806abb0 0x7f6e1806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 -- 172.21.15.179:0/3436529834 >> v1:172.21.15.179:6789/0 conn(0x7f6e180d2290 legacy=0x7f6e180b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 -- 172.21.15.179:0/3436529834 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1806c280 con 0x7f6e180d0dc0 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e16ffd700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e180d11a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009850 tx=0x7f6e0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4055aa1a6ef8c9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e15ffb700 1 -- 172.21.15.179:0/3436529834 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c011830 con 0x7f6e180d0dc0 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e15ffb700 1 -- 172.21.15.179:0/3436529834 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c011990 con 0x7f6e180d0dc0 2024-05-08T22:37:48.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e15ffb700 1 -- 172.21.15.179:0/3436529834 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c011d00 con 0x7f6e180d0dc0 2024-05-08T22:37:48.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- 172.21.15.179:0/3436529834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 msgr2=0x7f6e180d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e180d11a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009850 tx=0x7f6e0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:48.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- 172.21.15.179:0/3436529834 shutdown_connections 2024-05-08T22:37:48.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e1806abb0 0x7f6e1806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 --2- 172.21.15.179:0/3436529834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e180d11a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- 172.21.15.179:0/3436529834 >> 172.21.15.179:0/3436529834 conn(0x7f6e180b48d0 msgr2=0x7f6e180b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:48.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- 172.21.15.179:0/3436529834 shutdown_connections 2024-05-08T22:37:48.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.461+0000 7f6e17fff700 1 -- 172.21.15.179:0/3436529834 wait complete. 2024-05-08T22:37:48.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 Processor -- start 2024-05-08T22:37:48.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- start start 2024-05-08T22:37:48.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e1806abb0 0x7f6e181e2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 0x7f6e181e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18157e90 con 0x7f6e180d0dc0 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18156f30 con 0x7f6e1806abb0 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18157210 con 0x7f6e180d2290 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37342/0 (socket says 172.21.15.179:37342) 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 -- 172.21.15.179:0/2328389544 learned_addr learned my addr 172.21.15.179:0/2328389544 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e16ffd700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e1806abb0 0x7f6e181e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e177fe700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 0x7f6e181e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 -- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 msgr2=0x7f6e181e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 0x7f6e181e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 -- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e1806abb0 msgr2=0x7f6e181e2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e1806abb0 0x7f6e181e2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:48.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 -- 172.21.15.179:0/2328389544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0094b0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e177fe700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 0x7f6e181e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e167fc700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c700 tx=0x7f6e0800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08005d90 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e080070d0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e180d1860 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08007480 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.465+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e181577d0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e080075e0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e18157fd0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6dff7fe700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 0x7f6e04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f6e080837b0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6e16ffd700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 0x7f6e04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:48.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.469+0000 7f6e16ffd700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 0x7f6e04068bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0090b0 tx=0x7f6e0c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:48.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.477+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6e18157fd0 con 0x7f6e180d0dc0 2024-05-08T22:37:48.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:48.649+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e18157fd0 con 0x7f6e180d0dc0 2024-05-08T22:37:49.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.317+0000 7f6dff7fe700 1 -- 172.21.15.179:0/2328389544 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e18157fd0 con 0x7f6e180d0dc0 2024-05-08T22:37:49.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 msgr2=0x7f6e04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 0x7f6e04068bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0090b0 tx=0x7f6e0c008030 comp rx=0 tx=0).stop 2024-05-08T22:37:49.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 msgr2=0x7f6e18145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c700 tx=0x7f6e0800cac0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 shutdown_connections 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e04066730 0x7f6e04068bf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e180d2290 0x7f6e181e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e180d0dc0 0x7f6e18145eb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 --2- 172.21.15.179:0/2328389544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e1806abb0 0x7f6e181e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 >> 172.21.15.179:0/2328389544 conn(0x7f6e180b48d0 msgr2=0x7f6e180b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:49.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 shutdown_connections 2024-05-08T22:37:49.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.325+0000 7f6e17fff700 1 -- 172.21.15.179:0/2328389544 wait complete. 2024-05-08T22:37:49.329 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:37:49.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:37:49.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 Processor -- start 2024-05-08T22:37:49.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 -- start start 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7c8143580 0x7fe7c815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c815e050 con 0x7fe7c806c2d0 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c815b2c0 con 0x7fe7c815ae10 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7ccbf9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c806f0a0 con 0x7fe7c8143580 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7c7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe7c806c2d0 0x7fe7c80cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60758/0 (socket says 172.21.15.179:60758) 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.465+0000 7fe7c7fff700 1 -- 172.21.15.179:0/1549312049 learned_addr learned my addr 172.21.15.179:0/1549312049 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:49.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c77fe700 1 --2- 172.21.15.179:0/1549312049 >> v2:172.21.15.195:3300/0 conn(0x7fe7c8143580 0x7fe7c815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155628339 0 0) 0x7fe7c815e050 con 0x7fe7c806c2d0 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b40035d0 con 0x7fe7c806c2d0 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452024510 0 0) 0x7fe7c815b2c0 con 0x7fe7c815ae10 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c815e050 con 0x7fe7c815ae10 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997234376 0 0) 0x7fe7b40035d0 con 0x7fe7c806c2d0 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c77fe700 1 -- 172.21.15.179:0/1549312049 >> v1:172.21.15.195:6789/0 conn(0x7fe7c815ae10 legacy=0x7fe7c806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c77fe700 1 -- 172.21.15.179:0/1549312049 >> v1:172.21.15.179:6789/0 conn(0x7fe7c806c2d0 legacy=0x7fe7c80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c77fe700 1 -- 172.21.15.179:0/1549312049 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c806edc0 con 0x7fe7c8143580 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c77fe700 1 --2- 172.21.15.179:0/1549312049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c8143580 0x7fe7c815e4e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009850 tx=0x7fe7b8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc7c8e9cd93a711 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00019f0 con 0x7fe7c8143580 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8004030 con 0x7fe7c8143580 2024-05-08T22:37:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7c6ffd700 1 -- 172.21.15.179:0/1549312049 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80043a0 con 0x7fe7c8143580 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/1549312049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c8143580 msgr2=0x7fe7c815e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/1549312049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c8143580 0x7fe7c815e4e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009850 tx=0x7fe7b8009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/1549312049 shutdown_connections 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/1549312049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c8143580 0x7fe7c815e4e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/1549312049 >> 172.21.15.179:0/1549312049 conn(0x7fe7c8070d80 msgr2=0x7fe7c8071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/1549312049 shutdown_connections 2024-05-08T22:37:49.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.469+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/1549312049 wait complete. 2024-05-08T22:37:49.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 Processor -- start 2024-05-08T22:37:49.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- start start 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c806c2d0 0x7fe7c80ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 0x7fe7c80adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80ac490 con 0x7fe7c8143580 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80ac770 con 0x7fe7c806c2d0 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80aca50 con 0x7fe7c815ae10 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37346/0 (socket says 172.21.15.179:37346) 2024-05-08T22:37:49.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 -- 172.21.15.179:0/305850897 learned_addr learned my addr 172.21.15.179:0/305850897 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c77fe700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c806c2d0 0x7fe7c80ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c7fff700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 0x7fe7c80adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 -- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 msgr2=0x7fe7c80adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 0x7fe7c80adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 -- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c806c2d0 msgr2=0x7fe7c80ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c806c2d0 0x7fe7c80ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 -- 172.21.15.179:0/305850897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80094b0 con 0x7fe7c8143580 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c7fff700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 0x7fe7c80adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7beffd700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c490 tx=0x7fe7b000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00052d0 con 0x7fe7c8143580 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0005450 con 0x7fe7c8143580 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c80acd30 con 0x7fe7c8143580 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00038e0 con 0x7fe7c8143580 2024-05-08T22:37:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.473+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c80ab270 con 0x7fe7c8143580 2024-05-08T22:37:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.477+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c80ab430 con 0x7fe7c8143580 2024-05-08T22:37:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.477+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7b0003a40 con 0x7fe7c8143580 2024-05-08T22:37:49.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.477+0000 7fe7c4ff9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 0x7fe7a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.477+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe7b0083460 con 0x7fe7c8143580 2024-05-08T22:37:49.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.477+0000 7fe7c77fe700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 0x7fe7a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.481+0000 7fe7c77fe700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 0x7fe7a8068bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80090b0 tx=0x7fe7b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.485+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe7b0050910 con 0x7fe7c8143580 2024-05-08T22:37:49.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.657+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7c804e700 con 0x7fe7c8143580 2024-05-08T22:37:49.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.661+0000 7fe7c4ff9700 1 -- 172.21.15.179:0/305850897 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7b0053f30 con 0x7fe7c8143580 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.665+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 msgr2=0x7fe7a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.665+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 0x7fe7a8068bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80090b0 tx=0x7fe7b8008030 comp rx=0 tx=0).stop 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.665+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 msgr2=0x7fe7c80abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.665+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c490 tx=0x7fe7b000c7a0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.665+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 shutdown_connections 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7a8066730 0x7fe7a8068bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c815ae10 0x7fe7c80adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c8143580 0x7fe7c80abab0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 --2- 172.21.15.179:0/305850897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c806c2d0 0x7fe7c80ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 >> 172.21.15.179:0/305850897 conn(0x7fe7c8070d80 msgr2=0x7fe7c806d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:49.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 shutdown_connections 2024-05-08T22:37:49.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.669+0000 7fe7ccbf9700 1 -- 172.21.15.179:0/305850897 wait complete. 2024-05-08T22:37:49.685 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-08T22:37:49.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.793+0000 7fce8b444700 1 Processor -- start 2024-05-08T22:37:49.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.793+0000 7fce8b444700 1 -- start start 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce8406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fce8406ed50 0x7fce840b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840cd200 con 0x7fce8406dc20 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce840cd4e0 con 0x7fce8406ed50 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce840cd7c0 con 0x7fce841434d0 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fce8406ed50 0x7fce840b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fce8406ed50 0x7fce840b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37362/0 (socket says 172.21.15.179:37362) 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 -- 172.21.15.179:0/2328930564 learned_addr learned my addr 172.21.15.179:0/2328930564 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:49.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8a442700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce8406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 -- 172.21.15.179:0/2328930564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 msgr2=0x7fce8406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce8406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 -- 172.21.15.179:0/2328930564 >> v1:172.21.15.195:6789/0 conn(0x7fce8406dc20 legacy=0x7fce8406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 -- 172.21.15.179:0/2328930564 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840cdaa0 con 0x7fce8406ed50 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8a442700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce8406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8ac43700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce8406ed50 0x7fce840b19e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fce74009910 tx=0x7fce74009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5647370d652c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce89440700 1 -- 172.21.15.179:0/2328930564 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400ba60 con 0x7fce8406ed50 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce89440700 1 -- 172.21.15.179:0/2328930564 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400bbc0 con 0x7fce8406ed50 2024-05-08T22:37:49.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce89440700 1 -- 172.21.15.179:0/2328930564 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74009e60 con 0x7fce8406ed50 2024-05-08T22:37:49.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- 172.21.15.179:0/2328930564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce8406ed50 msgr2=0x7fce840b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce8406ed50 0x7fce840b19e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fce74009910 tx=0x7fce74009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:49.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- 172.21.15.179:0/2328930564 shutdown_connections 2024-05-08T22:37:49.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce8406ed50 0x7fce840b19e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 --2- 172.21.15.179:0/2328930564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce8406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- 172.21.15.179:0/2328930564 >> 172.21.15.179:0/2328930564 conn(0x7fce840ce9d0 msgr2=0x7fce840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- 172.21.15.179:0/2328930564 shutdown_connections 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.797+0000 7fce8b444700 1 -- 172.21.15.179:0/2328930564 wait complete. 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 Processor -- start 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- start start 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce8406ed50 0x7fce8413c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce840c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840b2510 con 0x7fce840b1b40 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce84159ba0 con 0x7fce8406ed50 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce841588b0 con 0x7fce841434d0 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37374/0 (socket says 172.21.15.179:37374) 2024-05-08T22:37:49.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 -- 172.21.15.179:0/1294086411 learned_addr learned my addr 172.21.15.179:0/1294086411 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8a442700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce8406ed50 0x7fce8413c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce89c41700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce840c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 -- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 msgr2=0x7fce840c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce840c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 -- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce8406ed50 msgr2=0x7fce8413c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce8406ed50 0x7fce8413c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 -- 172.21.15.179:0/1294086411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009570 con 0x7fce840b1b40 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8ac43700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fce74013000 tx=0x7fce74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74004550 con 0x7fce840b1b40 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7401f070 con 0x7fce840b1b40 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74003900 con 0x7fce840b1b40 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8413cba0 con 0x7fce840b1b40 2024-05-08T22:37:49.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce84158de0 con 0x7fce840b1b40 2024-05-08T22:37:49.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.801+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84067740 con 0x7fce840b1b40 2024-05-08T22:37:49.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.805+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce74003a60 con 0x7fce840b1b40 2024-05-08T22:37:49.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.805+0000 7fce7affd700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 0x7fce70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:49.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.805+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fce74021070 con 0x7fce840b1b40 2024-05-08T22:37:49.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.805+0000 7fce8a442700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 0x7fce70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:49.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.805+0000 7fce8a442700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 0x7fce70068bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c920 tx=0x7fce7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:49.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:49.813+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fce74052a80 con 0x7fce840b1b40 2024-05-08T22:37:50.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.053+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fce840b2b90 con 0x7fce840b1b40 2024-05-08T22:37:50.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.325+0000 7fce7affd700 1 -- 172.21.15.179:0/1294086411 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v231) v1 ==== 169+0+0 (secure 0 0 0) 0x7fce740560a0 con 0x7fce840b1b40 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 msgr2=0x7fce70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 0x7fce70068bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c920 tx=0x7fce7c008040 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 msgr2=0x7fce840b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fce74013000 tx=0x7fce74004770 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 shutdown_connections 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce70066730 0x7fce70068bf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce840b1b40 0x7fce840b1fd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce841434d0 0x7fce840c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 --2- 172.21.15.179:0/1294086411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce8406ed50 0x7fce8413c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 >> 172.21.15.179:0/1294086411 conn(0x7fce840ce9d0 msgr2=0x7fce84140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 shutdown_connections 2024-05-08T22:37:50.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.329+0000 7fce8b444700 1 -- 172.21.15.179:0/1294086411 wait complete. 2024-05-08T22:37:50.348 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T22:37:50.349 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T22:37:50.349 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:37:50.349 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:50.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 Processor -- start 2024-05-08T22:37:50.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 -- start start 2024-05-08T22:37:50.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff05815b000 0x7ff0580cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff0580b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0580cf630 con 0x7ff0580d06d0 2024-05-08T22:37:50.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05815b3e0 con 0x7ff05815b000 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d8b2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0580cf910 con 0x7ff0580cc040 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff057fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff0580d06d0 0x7ff0580cb930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60762/0 (socket says 172.21.15.179:60762) 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff057fff700 1 -- 172.21.15.179:0/2235586102 learned_addr learned my addr 172.21.15.179:0/2235586102 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff0580b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff05c8b0700 1 --2- 172.21.15.179:0/2235586102 >> v2:172.21.15.195:3300/0 conn(0x7ff05815b000 0x7ff0580cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106978622 0 0) 0x7ff0580cf630 con 0x7ff0580d06d0 2024-05-08T22:37:50.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.465+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0380035d0 con 0x7ff0580d06d0 2024-05-08T22:37:50.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247385727 0 0) 0x7ff0380035d0 con 0x7ff0580d06d0 2024-05-08T22:37:50.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 msgr2=0x7ff0580b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff0580b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 >> v2:172.21.15.195:3300/0 conn(0x7ff05815b000 msgr2=0x7ff0580cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 --2- 172.21.15.179:0/2235586102 >> v2:172.21.15.195:3300/0 conn(0x7ff05815b000 0x7ff0580cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580cfbf0 con 0x7ff0580d06d0 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff040001670 con 0x7ff0580d06d0 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 >> v1:172.21.15.179:6789/0 conn(0x7ff0580d06d0 legacy=0x7ff0580cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff038003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff038003fa0 0x7ff038004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff038004db0 0x7ff038005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815b3e0 con 0x7ff038004db0 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0580cf910 con 0x7ff038003fa0 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03800bb10 con 0x7ff0580cc040 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff038004db0 0x7ff038005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05c8b0700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff038003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff057fff700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff038003fa0 0x7ff038004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 -- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 msgr2=0x7ff038003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0580cc040 0x7ff038003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 -- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff038003fa0 msgr2=0x7ff038004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff038003fa0 0x7ff038004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 -- 172.21.15.179:0/2235586102 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380035d0 con 0x7ff038004db0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff038004db0 0x7ff038005230 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff050009fc0 tx=0x7ff0380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7b5d27ba56d4b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05000ec30 con 0x7ff038004db0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05000f040 con 0x7ff038004db0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff0577fe700 1 -- 172.21.15.179:0/2235586102 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050014480 con 0x7ff038004db0 2024-05-08T22:37:50.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2235586102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff038004db0 msgr2=0x7ff038005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2235586102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff038004db0 0x7ff038005230 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff050009fc0 tx=0x7ff0380058a0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d0b1700 1 -- 172.21.15.179:0/2235586102 reap_dead start 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2235586102 shutdown_connections 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2235586102 >> 172.21.15.179:0/2235586102 conn(0x7ff0580b2780 msgr2=0x7ff0580b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2235586102 shutdown_connections 2024-05-08T22:37:50.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.469+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2235586102 wait complete. 2024-05-08T22:37:50.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 Processor -- start 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- start start 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05814d9e0 0x7ff05814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05815e100 0x7ff058152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815c110 con 0x7ff0581518c0 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815b560 con 0x7ff05815e100 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815b770 con 0x7ff05814d9e0 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37388/0 (socket says 172.21.15.179:37388) 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 -- 172.21.15.179:0/2338480424 learned_addr learned my addr 172.21.15.179:0/2338480424 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05c8b0700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05814d9e0 0x7ff05814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d0b1700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05815e100 0x7ff058152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 -- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05814d9e0 msgr2=0x7ff05814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05814d9e0 0x7ff05814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 -- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05815e100 msgr2=0x7ff058152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05815e100 0x7ff058152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 -- 172.21.15.179:0/2338480424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05815ba50 con 0x7ff0581518c0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff057fff700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff0400048f0 tx=0x7ff0400075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000f6c0 con 0x7ff0581518c0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000f820 con 0x7ff0581518c0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05815bd30 con 0x7ff0581518c0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000fbb0 con 0x7ff0581518c0 2024-05-08T22:37:50.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.473+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05814e5b0 con 0x7ff0581518c0 2024-05-08T22:37:50.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.477+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05815c250 con 0x7ff0581518c0 2024-05-08T22:37:50.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.477+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff04000fd10 con 0x7ff0581518c0 2024-05-08T22:37:50.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.481+0000 7ff0557fa700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 0x7ff03c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.481+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff040084860 con 0x7ff0581518c0 2024-05-08T22:37:50.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.481+0000 7ff05c8b0700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 0x7ff03c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.481+0000 7ff05c8b0700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 0x7ff03c068bf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff048009cf0 tx=0x7ff048009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.485+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff040051d70 con 0x7ff0581518c0 2024-05-08T22:37:50.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.661+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff05815c250 con 0x7ff0581518c0 2024-05-08T22:37:50.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.661+0000 7ff0557fa700 1 -- 172.21.15.179:0/2338480424 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+12774 (secure 0 0 0) 0x7ff05815c250 con 0x7ff0581518c0 2024-05-08T22:37:50.665 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:50.666 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":74,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:49.313146+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6836/3517137124":"2024-05-09T22:37:46.289871+0000","172.21.15.195:6834/3517137124":"2024-05-09T22:37:46.289871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 msgr2=0x7ff03c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 0x7ff03c068bf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff048009cf0 tx=0x7ff048009240 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 msgr2=0x7ff05815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff0400048f0 tx=0x7ff0400075f0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 shutdown_connections 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff03c066730 0x7ff03c068bf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff05815e100 0x7ff058152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0581518c0 0x7ff05815dbc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 --2- 172.21.15.179:0/2338480424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff05814d9e0 0x7ff05814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 >> 172.21.15.179:0/2338480424 conn(0x7ff0580b2780 msgr2=0x7ff05806e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 shutdown_connections 2024-05-08T22:37:50.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.665+0000 7ff05d8b2700 1 -- 172.21.15.179:0/2338480424 wait complete. 2024-05-08T22:37:50.680 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:37:50.681 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:37:50.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 Processor -- start 2024-05-08T22:37:50.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 -- start start 2024-05-08T22:37:50.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f31cc15d0c0 0x7f31cc0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc15d4a0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc06abb0 con 0x7f31cc0ce2b0 2024-05-08T22:37:50.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d2dd1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0d2500 con 0x7f31cc15d0c0 2024-05-08T22:37:50.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d15ce700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f31cc0d1d20 0x7f31cc0cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60770/0 (socket says 172.21.15.179:60770) 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d15ce700 1 -- 172.21.15.179:0/1829992042 learned_addr learned my addr 172.21.15.179:0/1829992042 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.793+0000 7f31d1dcf700 1 --2- 172.21.15.179:0/1829992042 >> v2:172.21.15.179:3300/0 conn(0x7f31cc15d0c0 0x7f31cc0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929622461 0 0) 0x7f31cc15d4a0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b80035d0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d1dcf700 1 -- 172.21.15.179:0/1829992042 >> v1:172.21.15.195:6789/0 conn(0x7f31cc0ce2b0 legacy=0x7f31cc06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055104326 0 0) 0x7f31cc06abb0 con 0x7f31cc0ce2b0 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d1dcf700 1 -- 172.21.15.179:0/1829992042 >> v1:172.21.15.179:6789/0 conn(0x7f31cc0d1d20 legacy=0x7f31cc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d1dcf700 1 -- 172.21.15.179:0/1829992042 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0d27e0 con 0x7f31cc15d0c0 2024-05-08T22:37:50.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d1dcf700 1 --2- 172.21.15.179:0/1829992042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc15d0c0 0x7f31cc0b8a80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009880 tx=0x7f31c0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d6c936b90ead8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0011830 con 0x7f31cc15d0c0 2024-05-08T22:37:50.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0011990 con 0x7f31cc15d0c0 2024-05-08T22:37:50.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d0dcd700 1 -- 172.21.15.179:0/1829992042 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0011d20 con 0x7f31cc15d0c0 2024-05-08T22:37:50.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1829992042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc15d0c0 msgr2=0x7f31cc0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1829992042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc15d0c0 0x7f31cc0b8a80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009880 tx=0x7f31c0009b90 comp rx=0 tx=0).stop 2024-05-08T22:37:50.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1829992042 shutdown_connections 2024-05-08T22:37:50.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1829992042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc15d0c0 0x7f31cc0b8a80 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1829992042 >> 172.21.15.179:0/1829992042 conn(0x7f31cc0d2ed0 msgr2=0x7f31cc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1829992042 shutdown_connections 2024-05-08T22:37:50.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1829992042 wait complete. 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.797+0000 7f31d2dd1700 1 Processor -- start 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- start start 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31cc0ce2b0 0x7f31cc1f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 0x7f31cc1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1f3790 con 0x7f31cc0d1d20 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1f3f30 con 0x7f31cc15d0c0 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1f4210 con 0x7f31cc0ce2b0 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37412/0 (socket says 172.21.15.179:37412) 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 -- 172.21.15.179:0/1722954949 learned_addr learned my addr 172.21.15.179:0/1722954949 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:50.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d1dcf700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31cc0ce2b0 0x7f31cc1f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d25d0700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 0x7f31cc1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 -- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31cc0ce2b0 msgr2=0x7f31cc1f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31cc0ce2b0 0x7f31cc1f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 -- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 msgr2=0x7f31cc1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 0x7f31cc1f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 -- 172.21.15.179:0/1722954949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00094b0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d25d0700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 0x7f31cc1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:50.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d15ce700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c4f0 tx=0x7f31c800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005d90 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80070d0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc1f44f0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8007440 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.801+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1f4ab0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc1f4ca0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31c80075a0 con 0x7f31cc0d1d20 2024-05-08T22:37:50.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31be7fc700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 0x7f31b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f31c8083310 con 0x7f31cc0d1d20 2024-05-08T22:37:50.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31d1dcf700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 0x7f31b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.805+0000 7f31d1dcf700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 0x7f31b0068bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009850 tx=0x7f31c001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.809+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f31c8050900 con 0x7f31cc0d1d20 2024-05-08T22:37:50.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.977+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31cc04e700 con 0x7f31cc0d1d20 2024-05-08T22:37:50.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.977+0000 7f31be7fc700 1 -- 172.21.15.179:0/1722954949 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31c8053f20 con 0x7f31cc0d1d20 2024-05-08T22:37:50.982 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 msgr2=0x7f31b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 0x7f31b0068bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009850 tx=0x7f31c001c040 comp rx=0 tx=0).stop 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 msgr2=0x7f31cc1f0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c4f0 tx=0x7f31c800c800 comp rx=0 tx=0).stop 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 shutdown_connections 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31b0066730 0x7f31b0068bf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc15d0c0 0x7f31cc1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31cc0d1d20 0x7f31cc1f0410 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 --2- 172.21.15.179:0/1722954949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31cc0ce2b0 0x7f31cc1f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 >> 172.21.15.179:0/1722954949 conn(0x7f31cc0d2ed0 msgr2=0x7f31cc0c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 shutdown_connections 2024-05-08T22:37:50.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:50.981+0000 7f31d2dd1700 1 -- 172.21.15.179:0/1722954949 wait complete. 2024-05-08T22:37:50.996 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-08T22:37:51.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 Processor -- start 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- start start 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef815ef80 0x7f2ef8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef806de70 0x7f2ef806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef8165d50 con 0x7f2ef8164db0 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef806ad70 con 0x7f2ef815ef80 2024-05-08T22:37:51.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8165260 con 0x7f2ef806de70 2024-05-08T22:37:51.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef815ef80 0x7f2ef8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ef815ef80 0x7f2ef8161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37418/0 (socket says 172.21.15.179:37418) 2024-05-08T22:37:51.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 -- 172.21.15.179:0/3332439133 learned_addr learned my addr 172.21.15.179:0/3332439133 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef6d9d700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef806de70 0x7f2ef806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 -- 172.21.15.179:0/3332439133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef806de70 msgr2=0x7f2ef806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef806de70 0x7f2ef806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 -- 172.21.15.179:0/3332439133 >> v1:172.21.15.195:6789/0 conn(0x7f2ef8164db0 legacy=0x7f2ef806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 -- 172.21.15.179:0/3332439133 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef806e4b0 con 0x7f2ef815ef80 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef559a700 1 -- 172.21.15.179:0/3332439133 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073500252 0 0) 0x7f2ef8165d50 con 0x7f2ef8164db0 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef659c700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef815ef80 0x7f2ef8161790 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009910 tx=0x7f2ee0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9e7aff6ac3088f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef559a700 1 -- 172.21.15.179:0/3332439133 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0011830 con 0x7f2ef815ef80 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef559a700 1 -- 172.21.15.179:0/3332439133 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0011990 con 0x7f2ef815ef80 2024-05-08T22:37:51.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef559a700 1 -- 172.21.15.179:0/3332439133 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0011d40 con 0x7f2ef815ef80 2024-05-08T22:37:51.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- 172.21.15.179:0/3332439133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef815ef80 msgr2=0x7f2ef8161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:51.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef815ef80 0x7f2ef8161790 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009910 tx=0x7f2ee0009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- 172.21.15.179:0/3332439133 shutdown_connections 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef806de70 0x7f2ef806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3332439133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef815ef80 0x7f2ef8161790 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- 172.21.15.179:0/3332439133 >> 172.21.15.179:0/3332439133 conn(0x7f2ef80ce9d0 msgr2=0x7f2ef80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- 172.21.15.179:0/3332439133 shutdown_connections 2024-05-08T22:37:51.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.105+0000 7f2ef759e700 1 -- 172.21.15.179:0/3332439133 wait complete. 2024-05-08T22:37:51.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 Processor -- start 2024-05-08T22:37:51.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- start start 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef815ef80 0x7f2ef8162e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 0x7f2ef815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef815d890 con 0x7f2ef806de70 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80c8e60 con 0x7f2ef8164db0 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef815d5b0 con 0x7f2ef815ef80 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37430/0 (socket says 172.21.15.179:37430) 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 -- 172.21.15.179:0/3117550024 learned_addr learned my addr 172.21.15.179:0/3117550024 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef5d9b700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef815ef80 0x7f2ef8162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef6d9d700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 0x7f2ef815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 -- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef815ef80 msgr2=0x7f2ef8162e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef815ef80 0x7f2ef8162e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 -- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 msgr2=0x7f2ef815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 0x7f2ef815c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 -- 172.21.15.179:0/3117550024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0009570 con 0x7f2ef806de70 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef6d9d700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 0x7f2ef815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef659c700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8072020 tx=0x7f2ee0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0004480 con 0x7f2ef806de70 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee001f070 con 0x7f2ef806de70 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef80c9100 con 0x7f2ef806de70 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0003900 con 0x7f2ef806de70 2024-05-08T22:37:51.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.109+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef814ab60 con 0x7f2ef806de70 2024-05-08T22:37:51.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.113+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef80c8ca0 con 0x7f2ef806de70 2024-05-08T22:37:51.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.113+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ee0021030 con 0x7f2ef806de70 2024-05-08T22:37:51.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.113+0000 7f2eeeffd700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 0x7f2ed8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:51.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.113+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2ee0085470 con 0x7f2ef806de70 2024-05-08T22:37:51.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.113+0000 7f2ef5d9b700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 0x7f2ed8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:51.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.117+0000 7f2ef5d9b700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 0x7f2ed8068bf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ef815b130 tx=0x7f2ef0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:51.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.121+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2ee0052990 con 0x7f2ef806de70 2024-05-08T22:37:51.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:51.357+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f2ef8067740 con 0x7f2ef806de70 2024-05-08T22:37:52.082 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:37:52.085 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:37:52.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.333+0000 7f2eeeffd700 1 -- 172.21.15.179:0/3117550024 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v233) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2ee0055fb0 con 0x7f2ef806de70 2024-05-08T22:37:52.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 msgr2=0x7f2ed8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 0x7f2ed8068bf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ef815b130 tx=0x7f2ef0009040 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 msgr2=0x7f2ef81628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8072020 tx=0x7f2ee0004770 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 shutdown_connections 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ed8066730 0x7f2ed8068bf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8164db0 0x7f2ef815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ef815ef80 0x7f2ef8162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 --2- 172.21.15.179:0/3117550024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ef806de70 0x7f2ef81628d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 >> 172.21.15.179:0/3117550024 conn(0x7f2ef80ce9d0 msgr2=0x7f2ef8161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 shutdown_connections 2024-05-08T22:37:52.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.341+0000 7f2ef759e700 1 -- 172.21.15.179:0/3117550024 wait complete. 2024-05-08T22:37:52.381 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:37:52.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 Processor -- start 2024-05-08T22:37:52.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 -- start start 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b2c0cdb20 0x7f2b2c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0d1910 con 0x7f2b2c15d160 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0ce4f0 con 0x7f2b2c0cdb20 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b328fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0b6f90 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b320fd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b2c0cdb20 0x7f2b2c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b310fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b318fc700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2b2c15d160 0x7f2b2c0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:57448/0 (socket says 172.21.15.179:57448) 2024-05-08T22:37:52.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b318fc700 1 -- 172.21.15.179:0/2665014905 learned_addr learned my addr 172.21.15.179:0/2665014905 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b308fa700 1 -- 172.21.15.179:0/2665014905 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845740029 0 0) 0x7f2b2c0d1910 con 0x7f2b2c15d160 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b308fa700 1 -- 172.21.15.179:0/2665014905 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b180035d0 con 0x7f2b2c15d160 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b310fb700 1 -- 172.21.15.179:0/2665014905 >> v2:172.21.15.195:3300/0 conn(0x7f2b2c0cdb20 msgr2=0x7f2b2c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b310fb700 1 --2- 172.21.15.179:0/2665014905 >> v2:172.21.15.195:3300/0 conn(0x7f2b2c0cdb20 0x7f2b2c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.509+0000 7f2b310fb700 1 -- 172.21.15.179:0/2665014905 >> v1:172.21.15.195:6789/0 conn(0x7f2b2c15d160 legacy=0x7f2b2c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b310fb700 1 -- 172.21.15.179:0/2665014905 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0d1bf0 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b310fb700 1 --2- 172.21.15.179:0/2665014905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0b8a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009850 tx=0x7f2b20009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793372d35ca33034 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b308fa700 1 -- 172.21.15.179:0/2665014905 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20011830 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b308fa700 1 -- 172.21.15.179:0/2665014905 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20011990 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b308fa700 1 -- 172.21.15.179:0/2665014905 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20011d40 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- 172.21.15.179:0/2665014905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 msgr2=0x7f2b2c0b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2665014905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0b8a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009850 tx=0x7f2b20009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- 172.21.15.179:0/2665014905 shutdown_connections 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2665014905 >> v2:172.21.15.195:3300/0 conn(0x7f2b2c0cdb20 0x7f2b2c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2665014905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0b8a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- 172.21.15.179:0/2665014905 >> 172.21.15.179:0/2665014905 conn(0x7f2b2c0d3270 msgr2=0x7f2b2c0d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- 172.21.15.179:0/2665014905 shutdown_connections 2024-05-08T22:37:52.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- 172.21.15.179:0/2665014905 wait complete. 2024-05-08T22:37:52.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 Processor -- start 2024-05-08T22:37:52.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.513+0000 7f2b328fe700 1 -- start start 2024-05-08T22:37:52.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 0x7f2b2c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0cc5d0 con 0x7f2b2c0cdb20 2024-05-08T22:37:52.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c14bbb0 con 0x7f2b2c15d160 2024-05-08T22:37:52.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8d20 con 0x7f2b2c0d29b0 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b318fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 0x7f2b2c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b318fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37434/0 (socket says 172.21.15.179:37434) 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b318fc700 1 -- 172.21.15.179:0/2939081255 learned_addr learned my addr 172.21.15.179:0/2939081255 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b310fb700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 -- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 msgr2=0x7f2b2c0c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 -- 172.21.15.179:0/2939081255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 msgr2=0x7f2b2c14de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 -- 172.21.15.179:0/2939081255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200094b0 con 0x7f2b2c15d160 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b318fc700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b320fd700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 0x7f2b2c0caf70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400efc0 tx=0x7f2b2400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400cc90 con 0x7f2b2c15d160 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2400f040 con 0x7f2b2c15d160 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c0cb9c0 con 0x7f2b2c15d160 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b240146c0 con 0x7f2b2c15d160 2024-05-08T22:37:52.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.517+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c0cbf80 con 0x7f2b2c15d160 2024-05-08T22:37:52.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0cc740 con 0x7f2b2c15d160 2024-05-08T22:37:52.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b24014820 con 0x7f2b2c15d160 2024-05-08T22:37:52.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b1e7fc700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 0x7f2b10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2b24084a50 con 0x7f2b2c15d160 2024-05-08T22:37:52.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b318fc700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 0x7f2b10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.521+0000 7f2b318fc700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 0x7f2b10068bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2b280011e0 tx=0x7f2b28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.533+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2b24051f90 con 0x7f2b2c15d160 2024-05-08T22:37:52.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.713+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2b2c04e700 con 0x7f2b2c15d160 2024-05-08T22:37:52.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.721+0000 7f2b1e7fc700 1 -- 172.21.15.179:0/2939081255 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b240180c0 con 0x7f2b2c15d160 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 msgr2=0x7f2b10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 0x7f2b10068bf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2b280011e0 tx=0x7f2b28009040 comp rx=0 tx=0).stop 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 msgr2=0x7f2b2c0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 0x7f2b2c0caf70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400efc0 tx=0x7f2b2400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 shutdown_connections 2024-05-08T22:37:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2b10066730 0x7f2b10068bf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b2c15d160 0x7f2b2c0caf70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b2c0d29b0 0x7f2b2c0c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 --2- 172.21.15.179:0/2939081255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b2c0cdb20 0x7f2b2c14de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 >> 172.21.15.179:0/2939081255 conn(0x7f2b2c0d3270 msgr2=0x7f2b2c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 shutdown_connections 2024-05-08T22:37:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.725+0000 7f2b328fe700 1 -- 172.21.15.179:0/2939081255 wait complete. 2024-05-08T22:37:52.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:37:52.740 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:37:52.754 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:37:52.774 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:37:52.774 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:37:52.775 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:52.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:37:52.782 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:37:52.782 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:52.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:37:52.789 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:37:52.789 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:52.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:37:52.833 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:37:52.833 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:37:52.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:37:52.840 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:37:52.840 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:37:52.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.953+0000 7f1144a74700 1 Processor -- start 2024-05-08T22:37:52.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.953+0000 7f1144a74700 1 -- start start 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11400cc700 0x7f11400ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114015b1b0 0x7f11400cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114006abb0 con 0x7f11400cb510 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11400cb9a0 con 0x7f11400cc700 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114006ae90 con 0x7f114015b1b0 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11400cc700 0x7f11400ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11400cc700 0x7f11400ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37438/0 (socket says 172.21.15.179:37438) 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113ffff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f11400cb510 0x7f114015e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:57456/0 (socket says 172.21.15.179:57456) 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 -- 172.21.15.179:0/4020940402 learned_addr learned my addr 172.21.15.179:0/4020940402 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:52.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113effd700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114015b1b0 0x7f11400cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113e7fc700 1 -- 172.21.15.179:0/4020940402 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046691038 0 0) 0x7f114006abb0 con 0x7f11400cb510 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 -- 172.21.15.179:0/4020940402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114015b1b0 msgr2=0x7f11400cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114015b1b0 0x7f11400cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 -- 172.21.15.179:0/4020940402 >> v1:172.21.15.195:6789/0 conn(0x7f11400cb510 legacy=0x7f114015e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 -- 172.21.15.179:0/4020940402 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114015b9c0 con 0x7f11400cc700 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113f7fe700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11400cc700 0x7f11400ce380 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1134009910 tx=0x7f1134009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ac4acdae17d117 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113e7fc700 1 -- 172.21.15.179:0/4020940402 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134011830 con 0x7f11400cc700 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113e7fc700 1 -- 172.21.15.179:0/4020940402 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134011990 con 0x7f11400cc700 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f113e7fc700 1 -- 172.21.15.179:0/4020940402 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134011d40 con 0x7f11400cc700 2024-05-08T22:37:52.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- 172.21.15.179:0/4020940402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11400cc700 msgr2=0x7f11400ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11400cc700 0x7f11400ce380 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1134009910 tx=0x7f1134009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- 172.21.15.179:0/4020940402 shutdown_connections 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f114015b1b0 0x7f11400cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 --2- 172.21.15.179:0/4020940402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11400cc700 0x7f11400ce380 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- 172.21.15.179:0/4020940402 >> 172.21.15.179:0/4020940402 conn(0x7f11400ce9d0 msgr2=0x7f11400cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- 172.21.15.179:0/4020940402 shutdown_connections 2024-05-08T22:37:52.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.957+0000 7f1144a74700 1 -- 172.21.15.179:0/4020940402 wait complete. 2024-05-08T22:37:52.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 Processor -- start 2024-05-08T22:37:52.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 -- start start 2024-05-08T22:37:52.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 0x7f11400ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11400cc700 0x7f11400afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11400ace70 con 0x7f114015b1b0 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401511d0 con 0x7f11400cc700 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f1144a74700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11401e4690 con 0x7f11400cb510 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f113ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f113ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37444/0 (socket says 172.21.15.179:37444) 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f113ffff700 1 -- 172.21.15.179:0/374124796 learned_addr learned my addr 172.21.15.179:0/374124796 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.961+0000 7f113effd700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11400cc700 0x7f11400afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113f7fe700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 0x7f11400ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 -- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 msgr2=0x7f11400ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 0x7f11400ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 -- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11400cc700 msgr2=0x7f11400afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11400cc700 0x7f11400afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 -- 172.21.15.179:0/374124796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134009570 con 0x7f114015b1b0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113f7fe700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 0x7f11400ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f113ffff700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f113800cbd0 tx=0x7f113800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800ab70 con 0x7f114015b1b0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800acf0 con 0x7f114015b1b0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11400ad060 con 0x7f114015b1b0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138007310 con 0x7f114015b1b0 2024-05-08T22:37:52.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11400ad620 con 0x7f114015b1b0 2024-05-08T22:37:52.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1140067740 con 0x7f114015b1b0 2024-05-08T22:37:52.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1138007470 con 0x7f114015b1b0 2024-05-08T22:37:52.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 0x7f112c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:52.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.965+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1138083d80 con 0x7f114015b1b0 2024-05-08T22:37:52.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.969+0000 7f113f7fe700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 0x7f112c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:52.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.969+0000 7f113f7fe700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 0x7f112c068bf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1134009c60 tx=0x7f1134008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:52.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:52.977+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11380512b0 con 0x7f114015b1b0 2024-05-08T22:37:53.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.184+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f114004e700 con 0x7f114015b1b0 2024-05-08T22:37:53.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.184+0000 7f1127fff700 1 -- 172.21.15.179:0/374124796 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3540 (secure 0 0 0) 0x7f1138018090 con 0x7f114015b1b0 2024-05-08T22:37:53.190 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:53.190 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1753396824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1753396824},{"type":"v1","addr":"172.21.15.179:6833","nonce":1753396824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/224453291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":224453291},{"type":"v1","addr":"172.21.15.179:6835","nonce":224453291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":55715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1568423595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1568423595},{"type":"v1","addr":"172.21.15.195:6837","nonce":1568423595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":56096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2394679749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2394679749},{"type":"v1","addr":"172.21.15.195:6836","nonce":2394679749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2024-05-08T22:37:53.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 msgr2=0x7f112c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 0x7f112c068bf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1134009c60 tx=0x7f1134008030 comp rx=0 tx=0).stop 2024-05-08T22:37:53.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 msgr2=0x7f1140159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f113800cbd0 tx=0x7f113800cf90 comp rx=0 tx=0).stop 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 shutdown_connections 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f112c066730 0x7f112c068bf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11400cc700 0x7f11400afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114015b1b0 0x7f1140159ed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 --2- 172.21.15.179:0/374124796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11400cb510 0x7f11400ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 >> 172.21.15.179:0/374124796 conn(0x7f11400ce9d0 msgr2=0x7f114006e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 shutdown_connections 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.188+0000 7f1144a74700 1 -- 172.21.15.179:0/374124796 wait complete. 2024-05-08T22:37:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 34 2024-05-08T22:37:53.205 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:37:53.205 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:37:53.205 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:37:53.205 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:37:53.206 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:37:53.206 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:37:53.206 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:37:53.206 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:37:53.206 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:37:53.206 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:37:53.207 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:37:53.207 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:37:53.207 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:37:53.207 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:37:53.207 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:37:53.207 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:37:53.217 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:37:53.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:37:53.219 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:37:53.220 DEBUG:teuthology.parallel:result is None 2024-05-08T22:37:53.221 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:37:53.221 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:37:53.223 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:37:53.223 DEBUG:teuthology.parallel:result is None 2024-05-08T22:37:53.224 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:37:53.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:37:53.225 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:37:53.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:37:53.270 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:37:53.270 DEBUG:teuthology.parallel:result is None 2024-05-08T22:37:53.271 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:37:53.272 DEBUG:teuthology.parallel:result is None 2024-05-08T22:37:53.272 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:37:53.296 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:37:53.296 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:37:53.355 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:37:53.355 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:37:53.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.372+0000 7f5613fff700 1 Processor -- start 2024-05-08T22:37:53.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.372+0000 7f5613fff700 1 -- start start 2024-05-08T22:37:53.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5614159270 0x7f56140b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f561415cf70 0x7f561406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561415d950 con 0x7f561415d3f0 2024-05-08T22:37:53.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561415dc30 con 0x7f561415cf70 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56140b1af0 con 0x7f5614159270 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f561415cf70 0x7f561406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56137fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f561415d3f0 0x7f561406cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60786/0 (socket says 172.21.15.179:60786) 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56137fe700 1 -- 172.21.15.179:0/3163249005 learned_addr learned my addr 172.21.15.179:0/3163249005 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5612ffd700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5614159270 0x7f56140b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 -- 172.21.15.179:0/3163249005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5614159270 msgr2=0x7f56140b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5614159270 0x7f56140b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 -- 172.21.15.179:0/3163249005 >> v1:172.21.15.179:6789/0 conn(0x7f561415d3f0 legacy=0x7f561406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 -- 172.21.15.179:0/3163249005 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140b1dd0 con 0x7f561415cf70 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5611ffb700 1 -- 172.21.15.179:0/3163249005 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700938449 0 0) 0x7f561415d950 con 0x7f561415d3f0 2024-05-08T22:37:53.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f56127fc700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f561406b440 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5608009850 tx=0x7f5608009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4bb0fbd374dbcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5611ffb700 1 -- 172.21.15.179:0/3163249005 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608011830 con 0x7f561415cf70 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5611ffb700 1 -- 172.21.15.179:0/3163249005 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608011990 con 0x7f561415cf70 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5611ffb700 1 -- 172.21.15.179:0/3163249005 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608011d40 con 0x7f561415cf70 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- 172.21.15.179:0/3163249005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 msgr2=0x7f561406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f561406b440 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5608009850 tx=0x7f5608009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- 172.21.15.179:0/3163249005 shutdown_connections 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f561406b440 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 --2- 172.21.15.179:0/3163249005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5614159270 0x7f56140b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- 172.21.15.179:0/3163249005 >> 172.21.15.179:0/3163249005 conn(0x7f56140ce960 msgr2=0x7f56140ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- 172.21.15.179:0/3163249005 shutdown_connections 2024-05-08T22:37:53.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.376+0000 7f5613fff700 1 -- 172.21.15.179:0/3163249005 wait complete. 2024-05-08T22:37:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 Processor -- start 2024-05-08T22:37:53.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 -- start start 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 0x7f561414ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f5614144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561415d3f0 0x7f5614151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56141446d0 con 0x7f561415cf70 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56141f4280 con 0x7f5614159270 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5613fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56141f4560 con 0x7f561415d3f0 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5612ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 0x7f561414ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5612ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 0x7f561414ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:50712/0 (socket says 172.21.15.179:50712) 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f5612ffd700 1 -- 172.21.15.179:0/2876183493 learned_addr learned my addr 172.21.15.179:0/2876183493 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f5614144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56137fe700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561415d3f0 0x7f5614151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 -- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561415d3f0 msgr2=0x7f5614151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561415d3f0 0x7f5614151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 -- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 msgr2=0x7f561414ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 0x7f561414ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 -- 172.21.15.179:0/2876183493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080094b0 con 0x7f561415cf70 2024-05-08T22:37:53.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.380+0000 7f56127fc700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f5614144190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5608016000 tx=0x7f5608004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:53.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.384+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560801a990 con 0x7f561415cf70 2024-05-08T22:37:53.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.384+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560801f070 con 0x7f561415cf70 2024-05-08T22:37:53.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.384+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560801ad30 con 0x7f561415cf70 2024-05-08T22:37:53.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.384+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56141f4840 con 0x7f561415cf70 2024-05-08T22:37:53.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.384+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56141502b0 con 0x7f561415cf70 2024-05-08T22:37:53.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561404e700 con 0x7f561415cf70 2024-05-08T22:37:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f560802e450 con 0x7f561415cf70 2024-05-08T22:37:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f55fbfff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 0x7f5600068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:53.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5608084de0 con 0x7f561415cf70 2024-05-08T22:37:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f5612ffd700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 0x7f5600068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:53.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.388+0000 7f5612ffd700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 0x7f5600068c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f560c009a40 tx=0x7f560c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.392+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5608052480 con 0x7f561415cf70 2024-05-08T22:37:53.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:53.572+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f561414caf0 con 0x7f561415cf70 2024-05-08T22:37:54.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.396+0000 7f55fbfff700 1 -- 172.21.15.179:0/2876183493 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5608015080 con 0x7f561415cf70 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 msgr2=0x7f5600068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 0x7f5600068c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f560c009a40 tx=0x7f560c00b040 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 msgr2=0x7f5614144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f5614144190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5608016000 tx=0x7f5608004770 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 shutdown_connections 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5600066780 0x7f5600068c40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561415d3f0 0x7f5614151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561415cf70 0x7f5614144190 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 --2- 172.21.15.179:0/2876183493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5614159270 0x7f561414ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 >> 172.21.15.179:0/2876183493 conn(0x7f56140ce960 msgr2=0x7f56140cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 shutdown_connections 2024-05-08T22:37:54.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.400+0000 7f5613fff700 1 -- 172.21.15.179:0/2876183493 wait complete. 2024-05-08T22:37:54.406 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:37:54.423 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:37:54.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.584+0000 7f2db359e700 1 Processor -- start 2024-05-08T22:37:54.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.584+0000 7f2db359e700 1 -- start start 2024-05-08T22:37:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2db40b25c0 0x7f2db40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db40ad4c0 con 0x7f2db40ad0e0 2024-05-08T22:37:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db41609c0 con 0x7f2db415d3d0 2024-05-08T22:37:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db415ae80 con 0x7f2db40b25c0 2024-05-08T22:37:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2db40b25c0 0x7f2db40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db259c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2db40ad0e0 0x7f2db40ac080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60794/0 (socket says 172.21.15.179:60794) 2024-05-08T22:37:54.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2db40b25c0 0x7f2db40ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37482/0 (socket says 172.21.15.179:37482) 2024-05-08T22:37:54.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db259c700 1 -- 172.21.15.179:0/65322449 learned_addr learned my addr 172.21.15.179:0/65322449 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 -- 172.21.15.179:0/65322449 >> v1:172.21.15.195:6789/0 conn(0x7f2db415d3d0 legacy=0x7f2db4160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 -- 172.21.15.179:0/65322449 >> v1:172.21.15.179:6789/0 conn(0x7f2db40ad0e0 legacy=0x7f2db40ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 -- 172.21.15.179:0/65322449 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db415d900 con 0x7f2db40b25c0 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db1d9b700 1 --2- 172.21.15.179:0/65322449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db40b25c0 0x7f2db40ad860 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370f329cb14bb429 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db159a700 1 -- 172.21.15.179:0/65322449 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00e070 con 0x7f2db40b25c0 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db159a700 1 -- 172.21.15.179:0/65322449 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac00bbd0 con 0x7f2db40b25c0 2024-05-08T22:37:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db159a700 1 -- 172.21.15.179:0/65322449 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac012430 con 0x7f2db40b25c0 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- 172.21.15.179:0/65322449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db40b25c0 msgr2=0x7f2db40ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 --2- 172.21.15.179:0/65322449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db40b25c0 0x7f2db40ad860 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- 172.21.15.179:0/65322449 shutdown_connections 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 --2- 172.21.15.179:0/65322449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db40b25c0 0x7f2db40ad860 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- 172.21.15.179:0/65322449 >> 172.21.15.179:0/65322449 conn(0x7f2db40ce9d0 msgr2=0x7f2db40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- 172.21.15.179:0/65322449 shutdown_connections 2024-05-08T22:37:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- 172.21.15.179:0/65322449 wait complete. 2024-05-08T22:37:54.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 Processor -- start 2024-05-08T22:37:54.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.588+0000 7f2db359e700 1 -- start start 2024-05-08T22:37:54.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2db40ad0e0 0x7f2db414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2db40b25c0 0x7f2db40cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41f30f0 con 0x7f2db415d3d0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40cd0d0 con 0x7f2db40b25c0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40cd3b0 con 0x7f2db40ad0e0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db259c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2db40ad0e0 0x7f2db414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37486/0 (socket says 172.21.15.179:37486) 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 -- 172.21.15.179:0/3365451355 learned_addr learned my addr 172.21.15.179:0/3365451355 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db1d9b700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2db40b25c0 0x7f2db40cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 -- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2db40ad0e0 msgr2=0x7f2db414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2db40ad0e0 0x7f2db414b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 -- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2db40b25c0 msgr2=0x7f2db40cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2db40b25c0 0x7f2db40cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 -- 172.21.15.179:0/3365451355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009a70 con 0x7f2db415d3d0 2024-05-08T22:37:54.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db2d9d700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2da400c480 tx=0x7f2da400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40052d0 con 0x7f2db415d3d0 2024-05-08T22:37:54.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da400ce90 con 0x7f2db415d3d0 2024-05-08T22:37:54.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db40cd690 con 0x7f2db415d3d0 2024-05-08T22:37:54.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40036a0 con 0x7f2db415d3d0 2024-05-08T22:37:54.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db40cbfb0 con 0x7f2db415d3d0 2024-05-08T22:37:54.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.592+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d98005310 con 0x7f2db415d3d0 2024-05-08T22:37:54.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.596+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2da4003800 con 0x7f2db415d3d0 2024-05-08T22:37:54.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.596+0000 7f2daaffd700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 0x7f2d94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.596+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2da4083260 con 0x7f2db415d3d0 2024-05-08T22:37:54.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.596+0000 7f2db259c700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 0x7f2d94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.596+0000 7f2db259c700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 0x7f2d94068bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c005930 tx=0x7f2d9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.600+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2da40507d0 con 0x7f2db415d3d0 2024-05-08T22:37:54.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.832+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2d980055c0 con 0x7f2db415d3d0 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.832+0000 7f2daaffd700 1 -- 172.21.15.179:0/3365451355 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2da40011c0 con 0x7f2db415d3d0 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.838 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.839 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:37:54.840 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.841 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.842 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:37:54.843 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.844 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:37:54.845 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.846 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.847 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.848 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:37:54.849 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:37:54.850 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:37:54.851 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:37:54.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 msgr2=0x7f2d94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 0x7f2d94068bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c005930 tx=0x7f2d9c009040 comp rx=0 tx=0).stop 2024-05-08T22:37:54.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 msgr2=0x7f2db415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2da400c480 tx=0x7f2da400c790 comp rx=0 tx=0).stop 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 shutdown_connections 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d94066730 0x7f2d94068bf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2db40b25c0 0x7f2db40cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2db415d3d0 0x7f2db415fb80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 --2- 172.21.15.179:0/3365451355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2db40ad0e0 0x7f2db414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 >> 172.21.15.179:0/3365451355 conn(0x7f2db40ce9d0 msgr2=0x7f2db40b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 shutdown_connections 2024-05-08T22:37:54.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.836+0000 7f2db359e700 1 -- 172.21.15.179:0/3365451355 wait complete. 2024-05-08T22:37:54.855 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:54.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 Processor -- start 2024-05-08T22:37:54.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 -- start start 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fda280b3720 0x7fda2815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda28159b60 con 0x7fda281539c0 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda28159e40 con 0x7fda280b19e0 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2f1ed700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2815a120 con 0x7fda280b3720 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fda280b3720 0x7fda2815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fda280b3720 0x7fda2815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37500/0 (socket says 172.21.15.179:37500) 2024-05-08T22:37:54.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 -- 172.21.15.179:0/500816564 learned_addr learned my addr 172.21.15.179:0/500816564 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 -- 172.21.15.179:0/500816564 >> v1:172.21.15.195:6789/0 conn(0x7fda280b19e0 legacy=0x7fda280b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 -- 172.21.15.179:0/500816564 >> v1:172.21.15.179:6789/0 conn(0x7fda281539c0 legacy=0x7fda2806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.964+0000 7fda2e9ec700 1 -- 172.21.15.179:0/500816564 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2815a400 con 0x7fda280b3720 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2e9ec700 1 --2- 172.21.15.179:0/500816564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda2815d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fda20009e50 tx=0x7fda2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb478be7c2bf38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2d1e9700 1 -- 172.21.15.179:0/500816564 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2000e070 con 0x7fda280b3720 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2d1e9700 1 -- 172.21.15.179:0/500816564 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20003a40 con 0x7fda280b3720 2024-05-08T22:37:54.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2d1e9700 1 -- 172.21.15.179:0/500816564 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20003df0 con 0x7fda280b3720 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- 172.21.15.179:0/500816564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 msgr2=0x7fda2815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/500816564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda2815d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fda20009e50 tx=0x7fda2000b330 comp rx=0 tx=0).stop 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- 172.21.15.179:0/500816564 shutdown_connections 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/500816564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda2815d120 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- 172.21.15.179:0/500816564 >> 172.21.15.179:0/500816564 conn(0x7fda280ce9d0 msgr2=0x7fda280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- 172.21.15.179:0/500816564 shutdown_connections 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- 172.21.15.179:0/500816564 wait complete. 2024-05-08T22:37:54.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 Processor -- start 2024-05-08T22:37:54.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- start start 2024-05-08T22:37:54.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 0x7fda2815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 0x7fda281472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281647d0 con 0x7fda280b3720 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281644c0 con 0x7fda281539c0 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.968+0000 7fda2f1ed700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2815b9a0 con 0x7fda280b19e0 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37504/0 (socket says 172.21.15.179:37504) 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 -- 172.21.15.179:0/1338644607 learned_addr learned my addr 172.21.15.179:0/1338644607 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2e9ec700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 0x7fda281472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2e1eb700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 0x7fda2815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 -- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 msgr2=0x7fda2815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 0x7fda2815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 -- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 msgr2=0x7fda281472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 0x7fda281472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 -- 172.21.15.179:0/1338644607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009b30 con 0x7fda280b3720 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2e9ec700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 0x7fda281472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2e1eb700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 0x7fda2815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2d9ea700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b450 tx=0x7fda2400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24004020 con 0x7fda280b3720 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24004180 con 0x7fda280b3720 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24004530 con 0x7fda280b3720 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2815bc20 con 0x7fda280b3720 2024-05-08T22:37:54.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2815c1e0 con 0x7fda280b3720 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28164910 con 0x7fda280b3720 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda240036a0 con 0x7fda280b3720 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 0x7fda0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fda24083a90 con 0x7fda280b3720 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.972+0000 7fda2e1eb700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 0x7fda0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:54.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.976+0000 7fda2e1eb700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 0x7fda0c068c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda18007fd0 tx=0x7fda18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:54.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:54.980+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda28164910 con 0x7fda280b3720 2024-05-08T22:37:55.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.148+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda28164910 con 0x7fda280b3720 2024-05-08T22:37:55.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.148+0000 7fda1effd700 1 -- 172.21.15.179:0/1338644607 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+12643 (secure 0 0 0) 0x7fda28164910 con 0x7fda280b3720 2024-05-08T22:37:55.154 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:55.154 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":79,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:54.391756+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:55.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 msgr2=0x7fda0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 0x7fda0c068c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda18007fd0 tx=0x7fda18009040 comp rx=0 tx=0).stop 2024-05-08T22:37:55.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 msgr2=0x7fda28146d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b450 tx=0x7fda2400b810 comp rx=0 tx=0).stop 2024-05-08T22:37:55.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 shutdown_connections 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda0c066760 0x7fda0c068c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda281539c0 0x7fda281472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda280b3720 0x7fda28146d70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 --2- 172.21.15.179:0/1338644607 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda280b19e0 0x7fda2815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 >> 172.21.15.179:0/1338644607 conn(0x7fda280ce9d0 msgr2=0x7fda280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 shutdown_connections 2024-05-08T22:37:55.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.152+0000 7fda2f1ed700 1 -- 172.21.15.179:0/1338644607 wait complete. 2024-05-08T22:37:55.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:37:55.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:37:55.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 Processor -- start 2024-05-08T22:37:55.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 -- start start 2024-05-08T22:37:55.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f926c161fd0 0x7f926c1623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c06d230 con 0x7f926c0b2640 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c0b2f40 con 0x7f926c161fd0 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927309c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0b3220 con 0x7f926c0b2aa0 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f9271899700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f926c0b2640 0x7f926c06bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56958/0 (socket says 172.21.15.179:56958) 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f9271899700 1 -- 172.21.15.179:0/148843741 learned_addr learned my addr 172.21.15.179:0/148843741 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.280+0000 7f927209a700 1 --2- 172.21.15.179:0/148843741 >> v2:172.21.15.179:3300/0 conn(0x7f926c161fd0 0x7f926c1623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927289b700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193673386 0 0) 0x7f926c06d230 con 0x7f926c0b2640 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92580035d0 con 0x7f926c0b2640 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927209a700 1 -- 172.21.15.179:0/148843741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 msgr2=0x7f926c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927209a700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c06d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927209a700 1 -- 172.21.15.179:0/148843741 >> v1:172.21.15.179:6789/0 conn(0x7f926c0b2640 legacy=0x7f926c06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1333614526 0 0) 0x7f92580035d0 con 0x7f926c0b2640 2024-05-08T22:37:55.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927209a700 1 -- 172.21.15.179:0/148843741 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0b3500 con 0x7f926c161fd0 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927209a700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c161fd0 0x7f926c1623b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9264009850 tx=0x7f9264009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5cd79aeb2d921 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540019f0 con 0x7f926c161fd0 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264004030 con 0x7f926c161fd0 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f9271098700 1 -- 172.21.15.179:0/148843741 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640043a0 con 0x7f926c161fd0 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 -- 172.21.15.179:0/148843741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c161fd0 msgr2=0x7f926c1623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c161fd0 0x7f926c1623b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9264009850 tx=0x7f9264009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 -- 172.21.15.179:0/148843741 shutdown_connections 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 --2- 172.21.15.179:0/148843741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c161fd0 0x7f926c1623b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 -- 172.21.15.179:0/148843741 >> 172.21.15.179:0/148843741 conn(0x7f926c070dc0 msgr2=0x7f926c0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 -- 172.21.15.179:0/148843741 shutdown_connections 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.284+0000 7f927309c700 1 -- 172.21.15.179:0/148843741 wait complete. 2024-05-08T22:37:55.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 Processor -- start 2024-05-08T22:37:55.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- start start 2024-05-08T22:37:55.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c1f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 0x7f926c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1f7750 con 0x7f926c0b2640 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1f7a30 con 0x7f926c161fd0 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1f7d10 con 0x7f926c0b2aa0 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58572/0 (socket says 172.21.15.179:58572) 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 -- 172.21.15.179:0/3947553444 learned_addr learned my addr 172.21.15.179:0/3947553444 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f9271899700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c1f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927289b700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 0x7f926c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 -- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 msgr2=0x7f926c1f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c1f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 -- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 msgr2=0x7f926c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 0x7f926c157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 -- 172.21.15.179:0/3947553444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92640094b0 con 0x7f926c0b2640 2024-05-08T22:37:55.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927289b700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 0x7f926c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927209a700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f926c067bb0 tx=0x7f9264004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264003b70 con 0x7f926c0b2640 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926401f070 con 0x7f926c0b2640 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c1591c0 con 0x7f926c0b2640 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264011630 con 0x7f926c0b2640 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.288+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c15db20 con 0x7f926c0b2640 2024-05-08T22:37:55.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c04e700 con 0x7f926c0b2640 2024-05-08T22:37:55.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9264021030 con 0x7f926c0b2640 2024-05-08T22:37:55.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f926affd700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9264085190 con 0x7f926c0b2640 2024-05-08T22:37:55.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f9271899700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.292+0000 7f9271899700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 0x7f924c068bf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f926c0cdfb0 tx=0x7f9254001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.300+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f92640527a0 con 0x7f926c0b2640 2024-05-08T22:37:55.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.468+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f926c071a20 con 0x7f926c0b2640 2024-05-08T22:37:55.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.468+0000 7f926affd700 1 -- 172.21.15.179:0/3947553444 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9264055dc0 con 0x7f926c0b2640 2024-05-08T22:37:55.472 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 msgr2=0x7f924c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 0x7f924c068bf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f926c0cdfb0 tx=0x7f9254001a90 comp rx=0 tx=0).stop 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 msgr2=0x7f926c1516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f926c067bb0 tx=0x7f9264004770 comp rx=0 tx=0).stop 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 shutdown_connections 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c161fd0 0x7f926c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0b2aa0 0x7f926c1f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 --2- 172.21.15.179:0/3947553444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926c0b2640 0x7f926c1516d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 >> 172.21.15.179:0/3947553444 conn(0x7f926c070dc0 msgr2=0x7f926c0ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 shutdown_connections 2024-05-08T22:37:55.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.472+0000 7f927309c700 1 -- 172.21.15.179:0/3947553444 wait complete. 2024-05-08T22:37:55.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:37:55.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 Processor -- start 2024-05-08T22:37:55.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 -- start start 2024-05-08T22:37:55.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f106006ed50 0x7f10600b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106006ddf0 con 0x7f10601434d0 2024-05-08T22:37:55.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600cd200 con 0x7f10600ce2a0 2024-05-08T22:37:55.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10668d7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10600cd4e0 con 0x7f106006ed50 2024-05-08T22:37:55.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10660d6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f106006ed50 0x7f10600b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10658d5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f10601434d0 0x7f106006d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56960/0 (socket says 172.21.15.179:56960) 2024-05-08T22:37:55.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10658d5700 1 -- 172.21.15.179:0/1841077537 learned_addr learned my addr 172.21.15.179:0/1841077537 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:55.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10660d6700 1 -- 172.21.15.179:0/1841077537 >> v1:172.21.15.195:6789/0 conn(0x7f10600ce2a0 legacy=0x7f106006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10660d6700 1 -- 172.21.15.179:0/1841077537 >> v1:172.21.15.179:6789/0 conn(0x7f10601434d0 legacy=0x7f106006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10648d3700 1 -- 172.21.15.179:0/1841077537 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408989493 0 0) 0x7f106006ddf0 con 0x7f10601434d0 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10660d6700 1 -- 172.21.15.179:0/1841077537 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600cd7c0 con 0x7f106006ed50 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10660d6700 1 --2- 172.21.15.179:0/1841077537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106006ed50 0x7f10600b19e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2ad47199198797 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10648d3700 1 -- 172.21.15.179:0/1841077537 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800e070 con 0x7f106006ed50 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10648d3700 1 -- 172.21.15.179:0/1841077537 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800bbd0 con 0x7f106006ed50 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.592+0000 7f10648d3700 1 -- 172.21.15.179:0/1841077537 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012430 con 0x7f106006ed50 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- 172.21.15.179:0/1841077537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106006ed50 msgr2=0x7f10600b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 --2- 172.21.15.179:0/1841077537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106006ed50 0x7f10600b19e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105800b330 comp rx=0 tx=0).stop 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- 172.21.15.179:0/1841077537 shutdown_connections 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 --2- 172.21.15.179:0/1841077537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106006ed50 0x7f10600b19e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- 172.21.15.179:0/1841077537 >> 172.21.15.179:0/1841077537 conn(0x7f10600ce9d0 msgr2=0x7f10600cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- 172.21.15.179:0/1841077537 shutdown_connections 2024-05-08T22:37:55.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- 172.21.15.179:0/1841077537 wait complete. 2024-05-08T22:37:55.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 Processor -- start 2024-05-08T22:37:55.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- start start 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 0x7f1060137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10600ce2a0 0x7f1060158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106013ac20 con 0x7f10601434d0 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106013a8f0 con 0x7f10600ce2a0 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10668d7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10601596b0 con 0x7f106006ed50 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58594/0 (socket says 172.21.15.179:58594) 2024-05-08T22:37:55.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 -- 172.21.15.179:0/2189816060 learned_addr learned my addr 172.21.15.179:0/2189816060 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10650d4700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10600ce2a0 0x7f1060158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10658d5700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 0x7f1060137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 -- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 msgr2=0x7f1060137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 0x7f1060137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 -- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10600ce2a0 msgr2=0x7f1060158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10600ce2a0 0x7f1060158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.596+0000 7f10660d6700 1 -- 172.21.15.179:0/2189816060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009a70 con 0x7f10601434d0 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10658d5700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 0x7f1060137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:55.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10660d6700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1058014040 tx=0x7f1058004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105801b820 con 0x7f10601434d0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058020070 con 0x7f10601434d0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1060159990 con 0x7f10601434d0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105801bc00 con 0x7f10601434d0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10601387b0 con 0x7f10601434d0 2024-05-08T22:37:55.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106013add0 con 0x7f10601434d0 2024-05-08T22:37:55.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.600+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1058026400 con 0x7f10601434d0 2024-05-08T22:37:55.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.604+0000 7f10567fc700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 0x7f1044068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.604+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1058085d60 con 0x7f10601434d0 2024-05-08T22:37:55.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.604+0000 7f10658d5700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 0x7f1044068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.604+0000 7f10658d5700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 0x7f1044068bf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f105c000f60 tx=0x7f105c007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.612+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f106013add0 con 0x7f10601434d0 2024-05-08T22:37:55.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.784+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f106006af20 con 0x7f10601434d0 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.784+0000 7f10567fc700 1 -- 172.21.15.179:0/2189816060 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1058022060 con 0x7f10601434d0 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:37:55.789 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:37:55.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 msgr2=0x7f1044068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 0x7f1044068bf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f105c000f60 tx=0x7f105c007e90 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 msgr2=0x7f10600c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1058014040 tx=0x7f1058004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 shutdown_connections 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1044066730 0x7f1044068bf0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10600ce2a0 0x7f1060158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10601434d0 0x7f10600c9fe0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 --2- 172.21.15.179:0/2189816060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106006ed50 0x7f1060137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 >> 172.21.15.179:0/2189816060 conn(0x7f10600ce9d0 msgr2=0x7f106015a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 shutdown_connections 2024-05-08T22:37:55.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.788+0000 7f10668d7700 1 -- 172.21.15.179:0/2189816060 wait complete. 2024-05-08T22:37:55.804 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:37:55.804 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:37:55.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.924+0000 7fe7e69d6700 1 Processor -- start 2024-05-08T22:37:55.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.924+0000 7fe7e69d6700 1 -- start start 2024-05-08T22:37:55.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7e00ccdd0 0x7fe7e00b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7e00b3df0 0x7fe7e00b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e00b6650 con 0x7fe7e00b01c0 2024-05-08T22:37:55.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e00b6ae0 con 0x7fe7e00b3df0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e0161c60 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7e00b3df0 0x7fe7e00b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7e00b3df0 0x7fe7e00b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58602/0 (socket says 172.21.15.179:58602) 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e51d3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe7e00b01c0 0x7fe7e00cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56966/0 (socket says 172.21.15.179:56966) 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 -- 172.21.15.179:0/55244668 learned_addr learned my addr 172.21.15.179:0/55244668 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e59d4700 1 --2- 172.21.15.179:0/55244668 >> v2:172.21.15.195:3300/0 conn(0x7fe7e00ccdd0 0x7fe7e00b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 -- 172.21.15.179:0/55244668 >> v2:172.21.15.195:3300/0 conn(0x7fe7e00ccdd0 msgr2=0x7fe7e00b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 --2- 172.21.15.179:0/55244668 >> v2:172.21.15.195:3300/0 conn(0x7fe7e00ccdd0 0x7fe7e00b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 -- 172.21.15.179:0/55244668 >> v1:172.21.15.179:6789/0 conn(0x7fe7e00b01c0 legacy=0x7fe7e00cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e49d2700 1 -- 172.21.15.179:0/55244668 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192774821 0 0) 0x7fe7e00b6650 con 0x7fe7e00b01c0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 -- 172.21.15.179:0/55244668 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00caac0 con 0x7fe7e00b3df0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e61d5700 1 --2- 172.21.15.179:0/55244668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00b3df0 0x7fe7e00b4880 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00745e0 tx=0x7fe7d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658c6bb0b5b422ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e49d2700 1 -- 172.21.15.179:0/55244668 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d800e070 con 0x7fe7e00b3df0 2024-05-08T22:37:55.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e49d2700 1 -- 172.21.15.179:0/55244668 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d800b910 con 0x7fe7e00b3df0 2024-05-08T22:37:55.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e49d2700 1 -- 172.21.15.179:0/55244668 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d800bcc0 con 0x7fe7e00b3df0 2024-05-08T22:37:55.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 -- 172.21.15.179:0/55244668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00b3df0 msgr2=0x7fe7e00b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/55244668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00b3df0 0x7fe7e00b4880 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00745e0 tx=0x7fe7d800b330 comp rx=0 tx=0).stop 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 -- 172.21.15.179:0/55244668 shutdown_connections 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/55244668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00b3df0 0x7fe7e00b4880 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.928+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/55244668 >> v2:172.21.15.195:3300/0 conn(0x7fe7e00ccdd0 0x7fe7e00b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.932+0000 7fe7e69d6700 1 -- 172.21.15.179:0/55244668 >> 172.21.15.179:0/55244668 conn(0x7fe7e00b2670 msgr2=0x7fe7e00b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.932+0000 7fe7e69d6700 1 -- 172.21.15.179:0/55244668 shutdown_connections 2024-05-08T22:37:55.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.932+0000 7fe7e69d6700 1 -- 172.21.15.179:0/55244668 wait complete. 2024-05-08T22:37:55.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.932+0000 7fe7e69d6700 1 Processor -- start 2024-05-08T22:37:55.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.932+0000 7fe7e69d6700 1 -- start start 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 0x7fe7e0152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7e00b3df0 0x7fe7e015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e015d400 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0154df0 con 0x7fe7e00b3df0 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e69d6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01550d0 con 0x7fe7e00b01c0 2024-05-08T22:37:55.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58606/0 (socket says 172.21.15.179:58606) 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 -- 172.21.15.179:0/3500539464 learned_addr learned my addr 172.21.15.179:0/3500539464 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e51d3700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7e00b3df0 0x7fe7e015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e59d4700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 0x7fe7e0152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 -- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 msgr2=0x7fe7e0152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 0x7fe7e0152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 -- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7e00b3df0 msgr2=0x7fe7e015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7e00b3df0 0x7fe7e015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 -- 172.21.15.179:0/3500539464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8009b30 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e59d4700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 0x7fe7e0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:55.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7e61d5700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0072b70 tx=0x7fe7d8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80048f0 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80099b0 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.936+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d801c680 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01553b0 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e0152670 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e015d540 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7d801c7e0 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7d27fc700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 0x7fe7c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:55.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.940+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe7d8085b60 con 0x7fe7e00ccdd0 2024-05-08T22:37:55.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.944+0000 7fe7e59d4700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 0x7fe7c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:55.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.944+0000 7fe7e59d4700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 0x7fe7c4068bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009ba0 tx=0x7fe7dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:55.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:55.944+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe7e015d540 con 0x7fe7e00ccdd0 2024-05-08T22:37:56.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.116+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7e015d540 con 0x7fe7e00ccdd0 2024-05-08T22:37:56.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.408+0000 7fe7d27fc700 1 -- 172.21.15.179:0/3500539464 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe7e015d540 con 0x7fe7e00ccdd0 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 msgr2=0x7fe7c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 0x7fe7c4068bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009ba0 tx=0x7fe7dc008040 comp rx=0 tx=0).stop 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 msgr2=0x7fe7e0152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0072b70 tx=0x7fe7d8004af0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 shutdown_connections 2024-05-08T22:37:56.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe7c4066730 0x7fe7c4068bf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7e00b3df0 0x7fe7e015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7e00ccdd0 0x7fe7e0152d30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 --2- 172.21.15.179:0/3500539464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7e00b01c0 0x7fe7e0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 >> 172.21.15.179:0/3500539464 conn(0x7fe7e00b2670 msgr2=0x7fe7e00b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 shutdown_connections 2024-05-08T22:37:56.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.416+0000 7fe7e69d6700 1 -- 172.21.15.179:0/3500539464 wait complete. 2024-05-08T22:37:56.425 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:37:56.437 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:37:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.556+0000 7f8ee4c77700 1 Processor -- start 2024-05-08T22:37:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.556+0000 7f8ee4c77700 1 -- start start 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ee015d0c0 0x7f8ee00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 0x7f8ee00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee015d4a0 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee00ce570 con 0x7f8ee015d0c0 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee006abb0 con 0x7f8ee00d1d20 2024-05-08T22:37:56.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ee015d0c0 0x7f8ee00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ee015d0c0 0x7f8ee00b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58614/0 (socket says 172.21.15.179:58614) 2024-05-08T22:37:56.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8ee00ce0e0 0x7f8ee0160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56978/0 (socket says 172.21.15.179:56978) 2024-05-08T22:37:56.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edeffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 0x7f8ee00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 -- 172.21.15.179:0/2228376880 learned_addr learned my addr 172.21.15.179:0/2228376880 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 -- 172.21.15.179:0/2228376880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 msgr2=0x7f8ee00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 0x7f8ee00cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 -- 172.21.15.179:0/2228376880 >> v1:172.21.15.179:6789/0 conn(0x7f8ee00ce0e0 legacy=0x7f8ee0160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 -- 172.21.15.179:0/2228376880 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00d2500 con 0x7f8ee015d0c0 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edeffd700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 0x7f8ee00cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ede7fc700 1 -- 172.21.15.179:0/2228376880 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649125442 0 0) 0x7f8ee015d4a0 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8edf7fe700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee015d0c0 0x7f8ee00b8a80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009910 tx=0x7f8ec8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0702fcc48eb496 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ede7fc700 1 -- 172.21.15.179:0/2228376880 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011830 con 0x7f8ee015d0c0 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ede7fc700 1 -- 172.21.15.179:0/2228376880 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011990 con 0x7f8ee015d0c0 2024-05-08T22:37:56.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ede7fc700 1 -- 172.21.15.179:0/2228376880 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011d40 con 0x7f8ee015d0c0 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- 172.21.15.179:0/2228376880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee015d0c0 msgr2=0x7f8ee00b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee015d0c0 0x7f8ee00b8a80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009910 tx=0x7f8ec8009c20 comp rx=0 tx=0).stop 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- 172.21.15.179:0/2228376880 shutdown_connections 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee00d1d20 0x7f8ee00cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/2228376880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee015d0c0 0x7f8ee00b8a80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- 172.21.15.179:0/2228376880 >> 172.21.15.179:0/2228376880 conn(0x7f8ee00d2ed0 msgr2=0x7f8ee00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.560+0000 7f8ee4c77700 1 -- 172.21.15.179:0/2228376880 shutdown_connections 2024-05-08T22:37:56.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.564+0000 7f8ee4c77700 1 -- 172.21.15.179:0/2228376880 wait complete. 2024-05-08T22:37:56.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.564+0000 7f8ee4c77700 1 Processor -- start 2024-05-08T22:37:56.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.564+0000 7f8ee4c77700 1 -- start start 2024-05-08T22:37:56.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 0x7f8ee014d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee015d0c0 0x7f8ee014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00c9fa0 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00ca1f0 con 0x7f8ee00d1d20 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00ca4d0 con 0x7f8ee015d0c0 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58620/0 (socket says 172.21.15.179:58620) 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 -- 172.21.15.179:0/204280760 learned_addr learned my addr 172.21.15.179:0/204280760 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edeffd700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 0x7f8ee014d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edffff700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee015d0c0 0x7f8ee014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 -- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee015d0c0 msgr2=0x7f8ee014de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee015d0c0 0x7f8ee014de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 -- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 msgr2=0x7f8ee014d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 0x7f8ee014d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 -- 172.21.15.179:0/204280760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00094b0 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edeffd700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 0x7f8ee014d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:56.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8edf7fe700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8014040 tx=0x7f8ec8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:56.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800ba70 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec801e070 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800be50 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec8009570 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee00caa90 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.568+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee00cef20 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.572+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ec802d450 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.572+0000 7f8ed7fff700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 0x7f8ec0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:56.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.572+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f8ec8085170 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.572+0000 7f8edeffd700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 0x7f8ec0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:56.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.572+0000 7f8edeffd700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 0x7f8ec0068bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ee01f5f00 tx=0x7f8ed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:56.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.580+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ec80524f0 con 0x7f8ee00ce0e0 2024-05-08T22:37:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:56.752+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8ee004e700 con 0x7f8ee00ce0e0 2024-05-08T22:37:57.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.420+0000 7f8ed7fff700 1 -- 172.21.15.179:0/204280760 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8ec8055b10 con 0x7f8ee00ce0e0 2024-05-08T22:37:57.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 msgr2=0x7f8ec0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 0x7f8ec0068bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ee01f5f00 tx=0x7f8ed0008040 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 msgr2=0x7f8ee00cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8014040 tx=0x7f8ec8004770 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 shutdown_connections 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ec0066730 0x7f8ec0068bf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ee00d1d20 0x7f8ee014d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ee015d0c0 0x7f8ee014de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 --2- 172.21.15.179:0/204280760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ee00ce0e0 0x7f8ee00cc4c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 >> 172.21.15.179:0/204280760 conn(0x7f8ee00d2ed0 msgr2=0x7f8ee00b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 shutdown_connections 2024-05-08T22:37:57.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.432+0000 7f8ee4c77700 1 -- 172.21.15.179:0/204280760 wait complete. 2024-05-08T22:37:57.440 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:37:57.454 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.580+0000 7f18e3fff700 1 Processor -- start 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.580+0000 7f18e3fff700 1 -- start start 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.580+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e415d0c0 0x7f18e40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.580+0000 7f18e3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f18e40d1d20 0x7f18e40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e415d4a0 con 0x7f18e40ce0e0 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e40ce570 con 0x7f18e40d1d20 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e406abb0 con 0x7f18e415d0c0 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f18e40ce0e0 0x7f18e4160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56990/0 (socket says 172.21.15.179:56990) 2024-05-08T22:37:57.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f18e40d1d20 0x7f18e40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e37fe700 1 -- 172.21.15.179:0/265897452 learned_addr learned my addr 172.21.15.179:0/265897452 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e2ffd700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e415d0c0 0x7f18e40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e1ffb700 1 -- 172.21.15.179:0/265897452 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826881957 0 0) 0x7f18e415d4a0 con 0x7f18e40ce0e0 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e1ffb700 1 -- 172.21.15.179:0/265897452 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d00035d0 con 0x7f18e40ce0e0 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 -- 172.21.15.179:0/265897452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e415d0c0 msgr2=0x7f18e40b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e415d0c0 0x7f18e40b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 -- 172.21.15.179:0/265897452 >> v1:172.21.15.179:6789/0 conn(0x7f18e40ce0e0 legacy=0x7f18e4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 -- 172.21.15.179:0/265897452 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e40d2500 con 0x7f18e40d1d20 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e27fc700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e40d1d20 0x7f18e40cdba0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009850 tx=0x7f18d4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545735d901c7896 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e1ffb700 1 -- 172.21.15.179:0/265897452 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18e40d1d20 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e1ffb700 1 -- 172.21.15.179:0/265897452 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4011990 con 0x7f18e40d1d20 2024-05-08T22:37:57.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e1ffb700 1 -- 172.21.15.179:0/265897452 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011d40 con 0x7f18e40d1d20 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- 172.21.15.179:0/265897452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e40d1d20 msgr2=0x7f18e40cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e40d1d20 0x7f18e40cdba0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009850 tx=0x7f18d4009b60 comp rx=0 tx=0).stop 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- 172.21.15.179:0/265897452 shutdown_connections 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e40d1d20 0x7f18e40cdba0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 --2- 172.21.15.179:0/265897452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e415d0c0 0x7f18e40b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- 172.21.15.179:0/265897452 >> 172.21.15.179:0/265897452 conn(0x7f18e40d2ed0 msgr2=0x7f18e40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- 172.21.15.179:0/265897452 shutdown_connections 2024-05-08T22:37:57.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.584+0000 7f18e3fff700 1 -- 172.21.15.179:0/265897452 wait complete. 2024-05-08T22:37:57.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 Processor -- start 2024-05-08T22:37:57.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- start start 2024-05-08T22:37:57.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e40ce0e0 0x7f18e4152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18e40d1d20 0x7f18e40c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4153180 con 0x7f18e415d0c0 2024-05-08T22:37:57.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40caa60 con 0x7f18e40d1d20 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40cad40 con 0x7f18e40ce0e0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58636/0 (socket says 172.21.15.179:58636) 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 -- 172.21.15.179:0/868029291 learned_addr learned my addr 172.21.15.179:0/868029291 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e2ffd700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e40ce0e0 0x7f18e4152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e27fc700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18e40d1d20 0x7f18e40c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 -- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e40ce0e0 msgr2=0x7f18e4152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e40ce0e0 0x7f18e4152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 -- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18e40d1d20 msgr2=0x7f18e40c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18e40d1d20 0x7f18e40c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 -- 172.21.15.179:0/868029291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40094b0 con 0x7f18e415d0c0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e37fe700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00cb40 tx=0x7f18dc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007500 con 0x7f18e415d0c0 2024-05-08T22:37:57.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc007660 con 0x7f18e415d0c0 2024-05-08T22:37:57.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e40cb020 con 0x7f18e415d0c0 2024-05-08T22:37:57.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e40cb5e0 con 0x7f18e415d0c0 2024-05-08T22:37:57.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.588+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0079f0 con 0x7f18e415d0c0 2024-05-08T22:37:57.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.592+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e40cb820 con 0x7f18e415d0c0 2024-05-08T22:37:57.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.592+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18dc01b3b0 con 0x7f18e415d0c0 2024-05-08T22:37:57.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.592+0000 7f18db7fe700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 0x7f18c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.592+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f18dc083be0 con 0x7f18e415d0c0 2024-05-08T22:37:57.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.592+0000 7f18e2ffd700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 0x7f18c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.596+0000 7f18e2ffd700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 0x7f18c4068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18e404ed90 tx=0x7f18cc009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.604+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f18e40cb820 con 0x7f18e415d0c0 2024-05-08T22:37:57.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.804+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f18e40cb820 con 0x7f18e415d0c0 2024-05-08T22:37:57.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.836+0000 7f18db7fe700 1 -- 172.21.15.179:0/868029291 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f18e40cb820 con 0x7f18e415d0c0 2024-05-08T22:37:57.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 msgr2=0x7f18c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 0x7f18c4068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18e404ed90 tx=0x7f18cc009240 comp rx=0 tx=0).stop 2024-05-08T22:37:57.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 msgr2=0x7f18e40cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00cb40 tx=0x7f18dc00cb70 comp rx=0 tx=0).stop 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 shutdown_connections 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18c4066730 0x7f18c4068bf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18e40d1d20 0x7f18e40c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e415d0c0 0x7f18e40cc240 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 --2- 172.21.15.179:0/868029291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18e40ce0e0 0x7f18e4152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 >> 172.21.15.179:0/868029291 conn(0x7f18e40d2ed0 msgr2=0x7f18e40b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 shutdown_connections 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.844+0000 7f18e3fff700 1 -- 172.21.15.179:0/868029291 wait complete. 2024-05-08T22:37:57.849 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 11 and data pool 12 2024-05-08T22:37:57.859 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:57.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.960+0000 7fc90ee01700 1 Processor -- start 2024-05-08T22:37:57.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.960+0000 7fc90ee01700 1 -- start start 2024-05-08T22:37:57.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc90813de70 0x7fc90815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 0x7fc90806c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90815c360 con 0x7fc90815b900 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90806cda0 con 0x7fc90813de70 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90815bda0 con 0x7fc90806c450 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc90813de70 0x7fc90815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc90813de70 0x7fc90815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58652/0 (socket says 172.21.15.179:58652) 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 -- 172.21.15.179:0/366959236 learned_addr learned my addr 172.21.15.179:0/366959236 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90e600700 1 --2- 172.21.15.179:0/366959236 >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 0x7fc90806c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 -- 172.21.15.179:0/366959236 >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 msgr2=0x7fc90806c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 --2- 172.21.15.179:0/366959236 >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 0x7fc90806c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 -- 172.21.15.179:0/366959236 >> v1:172.21.15.179:6789/0 conn(0x7fc90815b900 legacy=0x7fc90813e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 -- 172.21.15.179:0/366959236 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90815c080 con 0x7fc90813de70 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90e600700 1 --2- 172.21.15.179:0/366959236 >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 0x7fc90806c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:37:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ddff700 1 --2- 172.21.15.179:0/366959236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90813de70 0x7fc90815b3c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc90806d8c0 tx=0x7fc904009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49121c09639648cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90cdfd700 1 -- 172.21.15.179:0/366959236 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90400b8a0 con 0x7fc90813de70 2024-05-08T22:37:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90cdfd700 1 -- 172.21.15.179:0/366959236 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90400ba00 con 0x7fc90813de70 2024-05-08T22:37:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90cdfd700 1 -- 172.21.15.179:0/366959236 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90400bd90 con 0x7fc90813de70 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- 172.21.15.179:0/366959236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90813de70 msgr2=0x7fc90815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 --2- 172.21.15.179:0/366959236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90813de70 0x7fc90815b3c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc90806d8c0 tx=0x7fc904009bf0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- 172.21.15.179:0/366959236 shutdown_connections 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 --2- 172.21.15.179:0/366959236 >> v2:172.21.15.195:3300/0 conn(0x7fc90806c450 0x7fc90806c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 --2- 172.21.15.179:0/366959236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90813de70 0x7fc90815b3c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- 172.21.15.179:0/366959236 >> 172.21.15.179:0/366959236 conn(0x7fc9080cfae0 msgr2=0x7fc9080cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- 172.21.15.179:0/366959236 shutdown_connections 2024-05-08T22:37:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.964+0000 7fc90ee01700 1 -- 172.21.15.179:0/366959236 wait complete. 2024-05-08T22:37:57.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 Processor -- start 2024-05-08T22:37:57.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 -- start start 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 0x7fc90813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc90813de70 0x7fc908153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081ee5c0 con 0x7fc90815b900 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908154490 con 0x7fc90806c450 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ee01700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9080cdab0 con 0x7fc90813de70 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58662/0 (socket says 172.21.15.179:58662) 2024-05-08T22:37:57.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 -- 172.21.15.179:0/2564460587 learned_addr learned my addr 172.21.15.179:0/2564460587 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ddff700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 0x7fc90813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90d5fe700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc90813de70 0x7fc908153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 -- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc90813de70 msgr2=0x7fc908153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc90813de70 0x7fc908153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 -- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 msgr2=0x7fc90813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 0x7fc90813fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 -- 172.21.15.179:0/2564460587 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904009570 con 0x7fc90815b900 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90ddff700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 0x7fc90813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.968+0000 7fc90e600700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc90000bfb0 tx=0x7fc90000e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900016420 con 0x7fc90815b900 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9081ee200 con 0x7fc90815b900 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000165a0 con 0x7fc90815b900 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900016950 con 0x7fc90815b900 2024-05-08T22:37:57.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9081405d0 con 0x7fc90815b900 2024-05-08T22:37:57.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90806ef10 con 0x7fc90815b900 2024-05-08T22:37:57.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc900016ab0 con 0x7fc90815b900 2024-05-08T22:37:57.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:57.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc900088e00 con 0x7fc90815b900 2024-05-08T22:37:57.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc90ddff700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:57.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.972+0000 7fc90ddff700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc904000c00 tx=0x7fc904008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:57.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:57.980+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc900056040 con 0x7fc90815b900 2024-05-08T22:37:58.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.156+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9081ee700 con 0x7fc90815b900 2024-05-08T22:37:58.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.160+0000 7fc8fe7fc700 1 -- 172.21.15.179:0/2564460587 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16370 (secure 0 0 0) 0x7fc9081ee700 con 0x7fc90815b900 2024-05-08T22:37:58.164 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:58.165 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":82,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:57.811883+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:58.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 msgr2=0x7fc8ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc904000c00 tx=0x7fc904008030 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 msgr2=0x7fc9081ee9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc90000bfb0 tx=0x7fc90000e3f0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 shutdown_connections 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc8ec066730 0x7fc8ec068bf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90815b900 0x7fc9081ee9e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc90813de70 0x7fc908153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 --2- 172.21.15.179:0/2564460587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc90806c450 0x7fc90813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 >> 172.21.15.179:0/2564460587 conn(0x7fc9080cfae0 msgr2=0x7fc9080b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 shutdown_connections 2024-05-08T22:37:58.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.164+0000 7fc90ee01700 1 -- 172.21.15.179:0/2564460587 wait complete. 2024-05-08T22:37:58.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:37:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.288+0000 7ff39daac700 1 Processor -- start 2024-05-08T22:37:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.288+0000 7ff39daac700 1 -- start start 2024-05-08T22:37:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff39815ef80 0x7ff398161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff39806de70 0x7ff39806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff398165d50 con 0x7ff398164db0 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39806ad70 con 0x7ff39806de70 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff398165260 con 0x7ff39815ef80 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff39806de70 0x7ff39806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff39806de70 0x7ff39806b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58668/0 (socket says 172.21.15.179:58668) 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff397fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff398164db0 0x7ff39806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57004/0 (socket says 172.21.15.179:57004) 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 -- 172.21.15.179:0/1179148724 learned_addr learned my addr 172.21.15.179:0/1179148724 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39caaa700 1 --2- 172.21.15.179:0/1179148724 >> v2:172.21.15.195:3300/0 conn(0x7ff39815ef80 0x7ff398161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 -- 172.21.15.179:0/1179148724 >> v2:172.21.15.195:3300/0 conn(0x7ff39815ef80 msgr2=0x7ff398161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 --2- 172.21.15.179:0/1179148724 >> v2:172.21.15.195:3300/0 conn(0x7ff39815ef80 0x7ff398161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 -- 172.21.15.179:0/1179148724 >> v1:172.21.15.179:6789/0 conn(0x7ff398164db0 legacy=0x7ff39806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 -- 172.21.15.179:0/1179148724 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39806e4b0 con 0x7ff39806de70 2024-05-08T22:37:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39d2ab700 1 --2- 172.21.15.179:0/1179148724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39806de70 0x7ff39806b200 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff398071ff0 tx=0x7ff39000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545eab0cd4e675e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff3977fe700 1 -- 172.21.15.179:0/1179148724 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000e070 con 0x7ff39806de70 2024-05-08T22:37:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff3977fe700 1 -- 172.21.15.179:0/1179148724 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39000b9c0 con 0x7ff39806de70 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff3977fe700 1 -- 172.21.15.179:0/1179148724 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000bd70 con 0x7ff39806de70 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- 172.21.15.179:0/1179148724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39806de70 msgr2=0x7ff39806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 --2- 172.21.15.179:0/1179148724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39806de70 0x7ff39806b200 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff398071ff0 tx=0x7ff39000b330 comp rx=0 tx=0).stop 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- 172.21.15.179:0/1179148724 shutdown_connections 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 --2- 172.21.15.179:0/1179148724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39806de70 0x7ff39806b200 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 --2- 172.21.15.179:0/1179148724 >> v2:172.21.15.195:3300/0 conn(0x7ff39815ef80 0x7ff398161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- 172.21.15.179:0/1179148724 >> 172.21.15.179:0/1179148724 conn(0x7ff3980ce9d0 msgr2=0x7ff3980cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- 172.21.15.179:0/1179148724 shutdown_connections 2024-05-08T22:37:58.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.292+0000 7ff39daac700 1 -- 172.21.15.179:0/1179148724 wait complete. 2024-05-08T22:37:58.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 Processor -- start 2024-05-08T22:37:58.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- start start 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff39806de70 0x7ff398157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff398164db0 0x7ff3981f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398159580 con 0x7ff39815ef80 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398159860 con 0x7ff39806de70 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff398159b40 con 0x7ff398164db0 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58676/0 (socket says 172.21.15.179:58676) 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 -- 172.21.15.179:0/2153221017 learned_addr learned my addr 172.21.15.179:0/2153221017 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:58.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39caaa700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff39806de70 0x7ff398157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39d2ab700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff398164db0 0x7ff3981f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 -- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff398164db0 msgr2=0x7ff3981f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff398164db0 0x7ff3981f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 -- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff39806de70 msgr2=0x7ff398157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff39806de70 0x7ff398157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 -- 172.21.15.179:0/2153221017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390009b30 con 0x7ff39815ef80 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff397fff700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c4d0 tx=0x7ff38800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800cea0 con 0x7ff39815ef80 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388005cd0 con 0x7ff39815ef80 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880072d0 con 0x7ff39815ef80 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff398159e20 con 0x7ff39815ef80 2024-05-08T22:37:58.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.296+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3981583e0 con 0x7ff39815ef80 2024-05-08T22:37:58.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff398067740 con 0x7ff39815ef80 2024-05-08T22:37:58.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff388007430 con 0x7ff39815ef80 2024-05-08T22:37:58.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff3957fa700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 0x7ff378068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff388083480 con 0x7ff39815ef80 2024-05-08T22:37:58.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff39caaa700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 0x7ff378068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.300+0000 7ff39caaa700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 0x7ff378068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff380009a00 tx=0x7ff380008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.304+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff388050760 con 0x7ff39815ef80 2024-05-08T22:37:58.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.476+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff39804e700 con 0x7ff39815ef80 2024-05-08T22:37:58.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.476+0000 7ff3957fa700 1 -- 172.21.15.179:0/2153221017 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16370 (secure 0 0 0) 0x7ff388053d80 con 0x7ff39815ef80 2024-05-08T22:37:58.481 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:37:58.481 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":82,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:57.811883+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 msgr2=0x7ff378068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 0x7ff378068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff380009a00 tx=0x7ff380008040 comp rx=0 tx=0).stop 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 msgr2=0x7ff3980ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c4d0 tx=0x7ff38800c7e0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 shutdown_connections 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff378066730 0x7ff378068bf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff398164db0 0x7ff3981f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39815ef80 0x7ff3980ccfb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 --2- 172.21.15.179:0/2153221017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff39806de70 0x7ff398157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 >> 172.21.15.179:0/2153221017 conn(0x7ff3980ce9d0 msgr2=0x7ff3981607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:58.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 shutdown_connections 2024-05-08T22:37:58.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.480+0000 7ff39daac700 1 -- 172.21.15.179:0/2153221017 wait complete. 2024-05-08T22:37:58.501 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:37:58.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.616+0000 7f113aa3c700 1 Processor -- start 2024-05-08T22:37:58.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.616+0000 7f113aa3c700 1 -- start start 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 0x7f113406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 0x7f11340c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11340add60 con 0x7f1134159ce0 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11340aece0 con 0x7f11340acc00 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340ccf10 con 0x7f113406ded0 2024-05-08T22:37:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1139a3a700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1134159ce0 0x7f11340ae5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57014/0 (socket says 172.21.15.179:57014) 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1139a3a700 1 -- 172.21.15.179:0/4100894297 learned_addr learned my addr 172.21.15.179:0/4100894297 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1139239700 1 --2- 172.21.15.179:0/4100894297 >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 0x7f113406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 --2- 172.21.15.179:0/4100894297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 0x7f11340c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1138a38700 1 -- 172.21.15.179:0/4100894297 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931572220 0 0) 0x7f11340add60 con 0x7f1134159ce0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1138a38700 1 -- 172.21.15.179:0/4100894297 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11200057e0 con 0x7f1134159ce0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 -- 172.21.15.179:0/4100894297 >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 msgr2=0x7f113406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 --2- 172.21.15.179:0/4100894297 >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 0x7f113406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 -- 172.21.15.179:0/4100894297 >> v1:172.21.15.179:6789/0 conn(0x7f1134159ce0 legacy=0x7f11340ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 -- 172.21.15.179:0/4100894297 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340ad150 con 0x7f113406ded0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1139239700 1 --2- 172.21.15.179:0/4100894297 >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 0x7f113406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113a23b700 1 --2- 172.21.15.179:0/4100894297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 0x7f11340c4180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f4554dfe47b8741 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1138a38700 1 -- 172.21.15.179:0/4100894297 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00e070 con 0x7f113406ded0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1138a38700 1 -- 172.21.15.179:0/4100894297 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00bb20 con 0x7f113406ded0 2024-05-08T22:37:58.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f1138a38700 1 -- 172.21.15.179:0/4100894297 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00be90 con 0x7f113406ded0 2024-05-08T22:37:58.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- 172.21.15.179:0/4100894297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 msgr2=0x7f11340c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 --2- 172.21.15.179:0/4100894297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 0x7f11340c4180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c00b330 comp rx=0 tx=0).stop 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- 172.21.15.179:0/4100894297 shutdown_connections 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 --2- 172.21.15.179:0/4100894297 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f113406ded0 0x7f11340c4180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 --2- 172.21.15.179:0/4100894297 >> v2:172.21.15.195:3300/0 conn(0x7f11340acc00 0x7f113406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- 172.21.15.179:0/4100894297 >> 172.21.15.179:0/4100894297 conn(0x7f1134098a70 msgr2=0x7f11340cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- 172.21.15.179:0/4100894297 shutdown_connections 2024-05-08T22:37:58.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.620+0000 7f113aa3c700 1 -- 172.21.15.179:0/4100894297 wait complete. 2024-05-08T22:37:58.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.624+0000 7f113aa3c700 1 Processor -- start 2024-05-08T22:37:58.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.624+0000 7f113aa3c700 1 -- start start 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f113406ded0 0x7f11340ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1134159ce0 0x7f113414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134151c10 con 0x7f11340acc00 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113415c5f0 con 0x7f113406ded0 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113415c8d0 con 0x7f1134159ce0 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58688/0 (socket says 172.21.15.179:58688) 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 -- 172.21.15.179:0/2659399628 learned_addr learned my addr 172.21.15.179:0/2659399628 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139a3a700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f113406ded0 0x7f11340ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113a23b700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1134159ce0 0x7f113414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 -- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1134159ce0 msgr2=0x7f113414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1134159ce0 0x7f113414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 -- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f113406ded0 msgr2=0x7f11340ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:58.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f113406ded0 0x7f11340ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:58.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 -- 172.21.15.179:0/2659399628 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240094b0 con 0x7f11340acc00 2024-05-08T22:37:58.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f1139239700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f112400ebe0 tx=0x7f112400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400c850 con 0x7f11340acc00 2024-05-08T22:37:58.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400c9b0 con 0x7f11340acc00 2024-05-08T22:37:58.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c009a70 con 0x7f11340acc00 2024-05-08T22:37:58.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400cd40 con 0x7f11340acc00 2024-05-08T22:37:58.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.628+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113415ce90 con 0x7f11340acc00 2024-05-08T22:37:58.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1124013470 con 0x7f11340acc00 2024-05-08T22:37:58.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113415d0b0 con 0x7f11340acc00 2024-05-08T22:37:58.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f112a7fc700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 0x7f1118068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:58.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11240846c0 con 0x7f11340acc00 2024-05-08T22:37:58.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f1139a3a700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 0x7f1118068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:58.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.632+0000 7f1139a3a700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 0x7f1118068bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1130007fd0 tx=0x7f1130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:58.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.636+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11240518f0 con 0x7f11340acc00 2024-05-08T22:37:58.737 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:37:58.732+0000 7fb366198700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:37:58.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:58.840+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f113404e700 con 0x7f11340acc00 2024-05-08T22:37:59.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.820+0000 7f112a7fc700 1 -- 172.21.15.179:0/2659399628 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f112400cea0 con 0x7f11340acc00 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 msgr2=0x7f1118068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 0x7f1118068bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1130007fd0 tx=0x7f1130009040 comp rx=0 tx=0).stop 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 msgr2=0x7f113414da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f112400ebe0 tx=0x7f112400eef0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 shutdown_connections 2024-05-08T22:37:59.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1118066730 0x7f1118068bf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1134159ce0 0x7f113414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11340acc00 0x7f113414da20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 --2- 172.21.15.179:0/2659399628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f113406ded0 0x7f11340ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 >> 172.21.15.179:0/2659399628 conn(0x7f1134098a70 msgr2=0x7f113406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 shutdown_connections 2024-05-08T22:37:59.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.824+0000 7f113aa3c700 1 -- 172.21.15.179:0/2659399628 wait complete. 2024-05-08T22:37:59.845 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:37:59.846 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:37:59.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 Processor -- start 2024-05-08T22:37:59.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 -- start start 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4afc15d160 0x7f4afc0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0d1910 con 0x7f4afc0d29b0 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0d1bf0 con 0x7f4afc0cdb20 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.972+0000 7f4b017f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0d1ed0 con 0x7f4afc15d160 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afb7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4afc0d29b0 0x7f4afc0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57016/0 (socket says 172.21.15.179:57016) 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afb7fe700 1 -- 172.21.15.179:0/3063466724 learned_addr learned my addr 172.21.15.179:0/3063466724 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afbfff700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4afc15d160 0x7f4afc0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759214322 0 0) 0x7f4afc0d1910 con 0x7f4afc0d29b0 2024-05-08T22:37:59.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae80035d0 con 0x7f4afc0d29b0 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316103794 0 0) 0x7f4afc0d1bf0 con 0x7f4afc0cdb20 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc0d1910 con 0x7f4afc0cdb20 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939434818 0 0) 0x7f4ae80035d0 con 0x7f4afc0d29b0 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4afc15d160 msgr2=0x7f4afc0b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4afc15d160 0x7f4afc0b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 >> v1:172.21.15.195:6789/0 conn(0x7f4afc0cdb20 legacy=0x7f4afc0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0d21b0 con 0x7f4afc0d29b0 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f4aec001670 con 0x7f4afc0d29b0 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 >> v1:172.21.15.179:6789/0 conn(0x7f4afc0d29b0 legacy=0x7f4afc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae8003be0 0x7f4ae8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8004870 0x7f4ae8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae8005230 0x7f4ae800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0d1ed0 con 0x7f4ae8005230 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae800bfa0 con 0x7f4ae8003be0 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae800c160 con 0x7f4ae8004870 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae8005230 0x7f4ae800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afbfff700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4afc15d160 0x7f4afc0b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afb7fe700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8004870 0x7f4ae8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afbfff700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae8003be0 0x7f4ae8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 -- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8004870 msgr2=0x7f4ae8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8004870 0x7f4ae8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 -- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae8003be0 msgr2=0x7f4ae8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae8003be0 0x7f4ae8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 -- 172.21.15.179:0/3063466724 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae80035d0 con 0x7f4ae8005230 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afbfff700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae8003be0 0x7f4ae8003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae8005230 0x7f4ae800ba30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c320 tx=0x7f4ae80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe970086dcf80d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4001f20 con 0x7f4ae8005230 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af400ce80 con 0x7f4ae8005230 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4afaffd700 1 -- 172.21.15.179:0/3063466724 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003c60 con 0x7f4ae8005230 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b017f6700 1 -- 172.21.15.179:0/3063466724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae8005230 msgr2=0x7f4ae800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b017f6700 1 --2- 172.21.15.179:0/3063466724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae8005230 0x7f4ae800ba30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c320 tx=0x7f4ae80058a0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b00ff5700 1 -- 172.21.15.179:0/3063466724 reap_dead start 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b017f6700 1 -- 172.21.15.179:0/3063466724 shutdown_connections 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b017f6700 1 -- 172.21.15.179:0/3063466724 >> 172.21.15.179:0/3063466724 conn(0x7f4afc0d3270 msgr2=0x7f4afc0d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.976+0000 7f4b017f6700 1 -- 172.21.15.179:0/3063466724 shutdown_connections 2024-05-08T22:37:59.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 -- 172.21.15.179:0/3063466724 wait complete. 2024-05-08T22:37:59.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 Processor -- start 2024-05-08T22:37:59.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 -- start start 2024-05-08T22:37:59.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8005230 0x7f4afc157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 0x7f4afc0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc158be0 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc157990 con 0x7f4afc0c8a70 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b017f6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc157f80 con 0x7f4ae8005230 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58716/0 (socket says 172.21.15.179:58716) 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 -- 172.21.15.179:0/1000146934 learned_addr learned my addr 172.21.15.179:0/1000146934 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4afbfff700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8005230 0x7f4afc157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4afb7fe700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 0x7f4afc0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 -- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8005230 msgr2=0x7f4afc157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8005230 0x7f4afc157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 -- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 msgr2=0x7f4afc0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 0x7f4afc0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 -- 172.21.15.179:0/1000146934 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc158240 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4afb7fe700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 0x7f4afc0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.980+0000 7f4b00ff5700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800bf70 tx=0x7f4af4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400cc80 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4003e00 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4005520 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc158520 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc157c00 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc04e700 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4af4019070 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 0x7f4adc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:59.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4af4084b90 con 0x7f4afc0c8ed0 2024-05-08T22:37:59.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4afbfff700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 0x7f4adc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:59.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.984+0000 7f4afbfff700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 0x7f4adc068c20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0d3f20 tx=0x7f4ae4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:59.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:37:59.992+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4af4051e50 con 0x7f4afc0c8ed0 2024-05-08T22:38:00.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.204+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4afc0b8790 con 0x7f4afc0c8ed0 2024-05-08T22:38:00.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.824+0000 7f4af8ff9700 1 -- 172.21.15.179:0/1000146934 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4af40220c0 con 0x7f4afc0c8ed0 2024-05-08T22:38:00.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 msgr2=0x7f4adc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 0x7f4adc068c20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0d3f20 tx=0x7f4ae4008040 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 msgr2=0x7f4afc0b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800bf70 tx=0x7f4af4001650 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 shutdown_connections 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4adc066760 0x7f4adc068c20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4afc0c8a70 0x7f4afc0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc0c8ed0 0x7f4afc0b3720 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 --2- 172.21.15.179:0/1000146934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae8005230 0x7f4afc157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 >> 172.21.15.179:0/1000146934 conn(0x7f4afc0d3270 msgr2=0x7f4afc0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 shutdown_connections 2024-05-08T22:38:00.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.828+0000 7f4b017f6700 1 -- 172.21.15.179:0/1000146934 wait complete. 2024-05-08T22:38:00.845 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:38:00.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:00.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 Processor -- start 2024-05-08T22:38:00.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 -- start start 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c04f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0540b0 con 0x7f828c0e0250 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0532f0 con 0x7f828c0e3b40 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829aca2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0535d0 con 0x7f828c006150 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829949f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f828c0e0250 0x7f828c04f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57018/0 (socket says 172.21.15.179:57018) 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829949f700 1 -- 172.21.15.179:0/3525173479 learned_addr learned my addr 172.21.15.179:0/3525173479 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.956+0000 7f829a4a1700 1 --2- 172.21.15.179:0/3525173479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c04f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:00.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f8298c9e700 1 -- 172.21.15.179:0/3525173479 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694050967 0 0) 0x7f828c0540b0 con 0x7f828c0e0250 2024-05-08T22:38:00.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829a4a1700 1 -- 172.21.15.179:0/3525173479 >> v1:172.21.15.195:6789/0 conn(0x7f828c0e3b40 legacy=0x7f828c0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829a4a1700 1 -- 172.21.15.179:0/3525173479 >> v1:172.21.15.179:6789/0 conn(0x7f828c0e0250 legacy=0x7f828c04f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829a4a1700 1 -- 172.21.15.179:0/3525173479 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0538b0 con 0x7f828c006150 2024-05-08T22:38:00.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829a4a1700 1 --2- 172.21.15.179:0/3525173479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c04f8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8290009880 tx=0x7f8290009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a61ba2162aac65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:00.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f8298c9e700 1 -- 172.21.15.179:0/3525173479 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000f8a0 con 0x7f828c006150 2024-05-08T22:38:00.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f8298c9e700 1 -- 172.21.15.179:0/3525173479 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829000fa00 con 0x7f828c006150 2024-05-08T22:38:00.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f8298c9e700 1 -- 172.21.15.179:0/3525173479 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000fd90 con 0x7f828c006150 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 -- 172.21.15.179:0/3525173479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 msgr2=0x7f828c04f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 --2- 172.21.15.179:0/3525173479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c04f8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8290009880 tx=0x7f8290009b90 comp rx=0 tx=0).stop 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 -- 172.21.15.179:0/3525173479 shutdown_connections 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 --2- 172.21.15.179:0/3525173479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c04f8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 -- 172.21.15.179:0/3525173479 >> 172.21.15.179:0/3525173479 conn(0x7f828c054b70 msgr2=0x7f828c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 -- 172.21.15.179:0/3525173479 shutdown_connections 2024-05-08T22:38:00.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.960+0000 7f829aca2700 1 -- 172.21.15.179:0/3525173479 wait complete. 2024-05-08T22:38:00.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 Processor -- start 2024-05-08T22:38:00.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 -- start start 2024-05-08T22:38:00.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:00.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0e3b40 0x7f828c0e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:00.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c04a2a0 con 0x7f828c0e3190 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0cc390 con 0x7f828c0e3b40 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829aca2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c049970 con 0x7f828c006150 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829a4a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829a4a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58724/0 (socket says 172.21.15.179:58724) 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829a4a1700 1 -- 172.21.15.179:0/444902395 learned_addr learned my addr 172.21.15.179:0/444902395 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f8299ca0700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.964+0000 7f829949f700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0e3b40 0x7f828c0e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 -- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 msgr2=0x7f828c0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 -- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0e3b40 msgr2=0x7f828c0e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0e3b40 0x7f828c0e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:00.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 -- 172.21.15.179:0/444902395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900094b0 con 0x7f828c0e3190 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829a4a1700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8290009bd0 tx=0x7f8290004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829001d040 con 0x7f828c0e3190 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290003ad0 con 0x7f828c0e3190 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c049c50 con 0x7f828c0e3190 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290003e80 con 0x7f828c0e3190 2024-05-08T22:38:00.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c0d7220 con 0x7f828c0e3190 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c04a410 con 0x7f828c0e3190 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f829002c400 con 0x7f828c0e3190 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 0x7f8278068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8290087d80 con 0x7f828c0e3190 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f8299ca0700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 0x7f8278068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:00.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.968+0000 7f8299ca0700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 0x7f8278068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8294063cf0 tx=0x7f8294073b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:00.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:00.976+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8290055040 con 0x7f828c0e3190 2024-05-08T22:38:01.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.164+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f828c0d73e0 con 0x7f828c0e3190 2024-05-08T22:38:01.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.164+0000 7f82867fc700 1 -- 172.21.15.179:0/444902395 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f8290058660 con 0x7f828c0e3190 2024-05-08T22:38:01.170 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:01.170 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:00.821878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56162":{"gid":56162,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/123759130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":123759130},{"type":"v1","addr":"172.21.15.195:6837","nonce":123759130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:01.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 msgr2=0x7f8278068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 0x7f8278068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8294063cf0 tx=0x7f8294073b90 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 msgr2=0x7f828c036b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8290009bd0 tx=0x7f8290004770 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 shutdown_connections 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8278066730 0x7f8278068bf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0e3b40 0x7f828c0e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f828c0e3190 0x7f828c036b80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 --2- 172.21.15.179:0/444902395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f828c006150 0x7f828c0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 >> 172.21.15.179:0/444902395 conn(0x7f828c054b70 msgr2=0x7f828c052720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 shutdown_connections 2024-05-08T22:38:01.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.168+0000 7f829aca2700 1 -- 172.21.15.179:0/444902395 wait complete. 2024-05-08T22:38:01.174 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 39 2024-05-08T22:38:01.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:01.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 Processor -- start 2024-05-08T22:38:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 -- start start 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24f00ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f006abb0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f006ae90 con 0x7f24f015b1b0 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ef59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015b9c0 con 0x7f24f00cc700 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24eed9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f24f00cb6e0 0x7f24f006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57024/0 (socket says 172.21.15.179:57024) 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24eed9d700 1 -- 172.21.15.179:0/2570972623 learned_addr learned my addr 172.21.15.179:0/2570972623 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:01.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.284+0000 7f24ee59c700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24f00ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873266252 0 0) 0x7f24f006abb0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d00035d0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347975002 0 0) 0x7f24f006ae90 con 0x7f24f015b1b0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f006abb0 con 0x7f24f015b1b0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831467137 0 0) 0x7f24d00035d0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 msgr2=0x7f24f00ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24f00ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 >> v1:172.21.15.195:6789/0 conn(0x7f24f015b1b0 legacy=0x7f24f00cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f015bca0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f24e80019f0 con 0x7f24f00cb6e0 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 >> v1:172.21.15.179:6789/0 conn(0x7f24f00cb6e0 legacy=0x7f24f006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24d0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24d0004870 0x7f24d0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24d0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015b9c0 con 0x7f24d0003be0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d000bb40 con 0x7f24d0004870 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d000bd00 con 0x7f24f00cc700 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24d0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24edd9b700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24d0004870 0x7f24d0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24eed9d700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24d0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 -- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 msgr2=0x7f24d0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f00cc700 0x7f24d0005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 -- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24d0004870 msgr2=0x7f24d0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24d0004870 0x7f24d0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 -- 172.21.15.179:0/2570972623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00035d0 con 0x7f24d0003be0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ee59c700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24d0003fe0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009b50 tx=0x7f24d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f1c8bd5bc5dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000e6c0 con 0x7f24d0003be0 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000e820 con 0x7f24d0003be0 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ed59a700 1 -- 172.21.15.179:0/2570972623 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000ebd0 con 0x7f24d0003be0 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 -- 172.21.15.179:0/2570972623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 msgr2=0x7f24d0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 --2- 172.21.15.179:0/2570972623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24d0003fe0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009b50 tx=0x7f24d00058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24eed9d700 1 -- 172.21.15.179:0/2570972623 reap_dead start 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 -- 172.21.15.179:0/2570972623 shutdown_connections 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 -- 172.21.15.179:0/2570972623 >> 172.21.15.179:0/2570972623 conn(0x7f24f00ce9d0 msgr2=0x7f24f00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 -- 172.21.15.179:0/2570972623 shutdown_connections 2024-05-08T22:38:01.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 -- 172.21.15.179:0/2570972623 wait complete. 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.288+0000 7f24ef59e700 1 Processor -- start 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- start start 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f01409a0 0x7f24f0142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f006ba50 0x7f24f013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f00ac900 con 0x7f24d0003be0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0141e30 con 0x7f24f006ba50 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0142790 con 0x7f24f01409a0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58736/0 (socket says 172.21.15.179:58736) 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 -- 172.21.15.179:0/4058118604 learned_addr learned my addr 172.21.15.179:0/4058118604 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24edd9b700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f01409a0 0x7f24f0142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24eed9d700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f006ba50 0x7f24f013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 -- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f01409a0 msgr2=0x7f24f0142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f01409a0 0x7f24f0142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 -- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f006ba50 msgr2=0x7f24f013fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f006ba50 0x7f24f013fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 -- 172.21.15.179:0/4058118604 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f01411b0 con 0x7f24d0003be0 2024-05-08T22:38:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ee59c700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bb10 tx=0x7f24e00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000e6c0 con 0x7f24d0003be0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0004030 con 0x7f24d0003be0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00043e0 con 0x7f24d0003be0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f0141490 con 0x7f24d0003be0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.292+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0141a50 con 0x7f24d0003be0 2024-05-08T22:38:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.296+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f0067740 con 0x7f24d0003be0 2024-05-08T22:38:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.296+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24e00036c0 con 0x7f24d0003be0 2024-05-08T22:38:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.296+0000 7f24e6ffd700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 0x7f24d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.296+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f24e0086230 con 0x7f24d0003be0 2024-05-08T22:38:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.296+0000 7f24edd9b700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 0x7f24d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.300+0000 7f24edd9b700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 0x7f24d4068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f24f0158cf0 tx=0x7f24d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.304+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f24e00535f0 con 0x7f24d0003be0 2024-05-08T22:38:01.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.472+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f004e700 con 0x7f24d0003be0 2024-05-08T22:38:01.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.476+0000 7f24e6ffd700 1 -- 172.21.15.179:0/4058118604 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16357 (secure 0 0 0) 0x7f24e0056c10 con 0x7f24d0003be0 2024-05-08T22:38:01.480 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:01.480 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:58.814700+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 msgr2=0x7f24d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 0x7f24d4068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f24f0158cf0 tx=0x7f24d8009040 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 msgr2=0x7f24f0150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f24d000bb10 tx=0x7f24e00047e0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 shutdown_connections 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f24d4066730 0x7f24d4068bf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f006ba50 0x7f24f013fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f01409a0 0x7f24f0142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 --2- 172.21.15.179:0/4058118604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0003be0 0x7f24f0150a70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 >> 172.21.15.179:0/4058118604 conn(0x7f24f00ce9d0 msgr2=0x7f24f00ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 shutdown_connections 2024-05-08T22:38:01.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.480+0000 7f24ef59e700 1 -- 172.21.15.179:0/4058118604 wait complete. 2024-05-08T22:38:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:38:01.498 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:38:01.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 Processor -- start 2024-05-08T22:38:01.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 -- start start 2024-05-08T22:38:01.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff5f006eb80 0x7ff5f00b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f006efd0 con 0x7ff5f01434d0 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f006ddf0 con 0x7ff5f006eb80 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f7354700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00cd200 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f6352700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5f01434d0 0x7ff5f006d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57036/0 (socket says 172.21.15.179:57036) 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f6352700 1 -- 172.21.15.179:0/1824036662 learned_addr learned my addr 172.21.15.179:0/1824036662 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f5b51700 1 --2- 172.21.15.179:0/1824036662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.604+0000 7ff5f6b53700 1 --2- 172.21.15.179:0/1824036662 >> v2:172.21.15.195:3300/0 conn(0x7ff5f006eb80 0x7ff5f00b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5350700 1 -- 172.21.15.179:0/1824036662 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966723834 0 0) 0x7ff5e4001110 con 0x7ff5f01434d0 2024-05-08T22:38:01.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5350700 1 -- 172.21.15.179:0/1824036662 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5dc0035d0 con 0x7ff5f01434d0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5b51700 1 -- 172.21.15.179:0/1824036662 >> v2:172.21.15.195:3300/0 conn(0x7ff5f006eb80 msgr2=0x7ff5f00b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5b51700 1 --2- 172.21.15.179:0/1824036662 >> v2:172.21.15.195:3300/0 conn(0x7ff5f006eb80 0x7ff5f00b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5b51700 1 -- 172.21.15.179:0/1824036662 >> v1:172.21.15.179:6789/0 conn(0x7ff5f01434d0 legacy=0x7ff5f006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5b51700 1 -- 172.21.15.179:0/1824036662 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00cd4e0 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5b51700 1 --2- 172.21.15.179:0/1824036662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f006e640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009d90 tx=0x7ff5ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdf2b8a01ce0862 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5350700 1 -- 172.21.15.179:0/1824036662 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00e070 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5350700 1 -- 172.21.15.179:0/1824036662 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00bb20 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f5350700 1 -- 172.21.15.179:0/1824036662 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00be90 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- 172.21.15.179:0/1824036662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 msgr2=0x7ff5f006e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 --2- 172.21.15.179:0/1824036662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f006e640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009d90 tx=0x7ff5ec00b330 comp rx=0 tx=0).stop 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- 172.21.15.179:0/1824036662 shutdown_connections 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 --2- 172.21.15.179:0/1824036662 >> v2:172.21.15.195:3300/0 conn(0x7ff5f006eb80 0x7ff5f00b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 --2- 172.21.15.179:0/1824036662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f006e640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- 172.21.15.179:0/1824036662 >> 172.21.15.179:0/1824036662 conn(0x7ff5f00ce9d0 msgr2=0x7ff5f00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- 172.21.15.179:0/1824036662 shutdown_connections 2024-05-08T22:38:01.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- 172.21.15.179:0/1824036662 wait complete. 2024-05-08T22:38:01.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 Processor -- start 2024-05-08T22:38:01.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.608+0000 7ff5f7354700 1 -- start start 2024-05-08T22:38:01.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f006eb80 0x7ff5f013b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f00a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00c9d10 con 0x7ff5f01434d0 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00c9100 con 0x7ff5f006eb80 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00c93e0 con 0x7ff5f00ce2a0 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58752/0 (socket says 172.21.15.179:58752) 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 -- 172.21.15.179:0/327343060 learned_addr learned my addr 172.21.15.179:0/327343060 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6352700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f006eb80 0x7ff5f013b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f5b51700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f00a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 -- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 msgr2=0x7ff5f00a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f00a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 -- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f006eb80 msgr2=0x7ff5f013b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f006eb80 0x7ff5f013b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 -- 172.21.15.179:0/327343060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009a70 con 0x7ff5f01434d0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f5b51700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f00a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f6b53700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e7f0 tx=0x7ff5e800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80095c0 con 0x7ff5f01434d0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e8013680 con 0x7ff5f01434d0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f00c96c0 con 0x7ff5f01434d0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8013a10 con 0x7ff5f01434d0 2024-05-08T22:38:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.612+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f013bcb0 con 0x7ff5f01434d0 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f0067740 con 0x7ff5f01434d0 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5e8013b70 con 0x7ff5f01434d0 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5e2ffd700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 0x7ff5d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff5e8084250 con 0x7ff5f01434d0 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5f6352700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 0x7ff5d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.616+0000 7ff5f6352700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 0x7ff5d4068bf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4005930 tx=0x7ff5e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.620+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5e80513f0 con 0x7ff5f01434d0 2024-05-08T22:38:01.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.784+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5f015d080 con 0x7ff5d4066730 2024-05-08T22:38:01.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.800+0000 7ff5e2ffd700 1 -- 172.21.15.179:0/327343060 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287288 (secure 0 0 0) 0x7ff5f015d080 con 0x7ff5d4066730 2024-05-08T22:38:01.809 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 msgr2=0x7ff5d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 0x7ff5d4068bf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4005930 tx=0x7ff5e4009040 comp rx=0 tx=0).stop 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 msgr2=0x7ff5f00ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e7f0 tx=0x7ff5e800eb00 comp rx=0 tx=0).stop 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 shutdown_connections 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff5d4066730 0x7ff5d4068bf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5f01434d0 0x7ff5f00ca1a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f00ce2a0 0x7ff5f00a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 --2- 172.21.15.179:0/327343060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5f006eb80 0x7ff5f013b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 >> 172.21.15.179:0/327343060 conn(0x7ff5f00ce9d0 msgr2=0x7ff5f006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 shutdown_connections 2024-05-08T22:38:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.808+0000 7ff5f7354700 1 -- 172.21.15.179:0/327343060 wait complete. 2024-05-08T22:38:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:38:01.825 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-05-08T22:38:01.478708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253804,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754720916,"statfs":{"total":773094113280,"available":772834217984,"internally_reserved":0,"allocated":9134080,"data_stored":4759162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.067987"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848306+0000","last_change":"2024-05-08T22:37:57.844883+0000","last_active":"2024-05-08T22:37:57.848306+0000","last_peered":"2024-05-08T22:37:57.848306+0000","last_clean":"2024-05-08T22:37:57.848306+0000","last_became_active":"2024-05-08T22:37:57.844614+0000","last_became_peered":"2024-05-08T22:37:57.844614+0000","last_unstale":"2024-05-08T22:37:57.848306+0000","last_undegraded":"2024-05-08T22:37:57.848306+0000","last_fullsized":"2024-05-08T22:37:57.848306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:56:36.890701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848623+0000","last_change":"2024-05-08T22:37:57.844447+0000","last_active":"2024-05-08T22:37:57.848623+0000","last_peered":"2024-05-08T22:37:57.848623+0000","last_clean":"2024-05-08T22:37:57.848623+0000","last_became_active":"2024-05-08T22:37:57.844201+0000","last_became_peered":"2024-05-08T22:37:57.844201+0000","last_unstale":"2024-05-08T22:37:57.848623+0000","last_undegraded":"2024-05-08T22:37:57.848623+0000","last_fullsized":"2024-05-08T22:37:57.848623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:10.013238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838916+0000","last_change":"2024-05-08T22:37:57.835351+0000","last_active":"2024-05-08T22:37:57.838916+0000","last_peered":"2024-05-08T22:37:57.838916+0000","last_clean":"2024-05-08T22:37:57.838916+0000","last_became_active":"2024-05-08T22:37:57.835104+0000","last_became_peered":"2024-05-08T22:37:57.835104+0000","last_unstale":"2024-05-08T22:37:57.838916+0000","last_undegraded":"2024-05-08T22:37:57.838916+0000","last_fullsized":"2024-05-08T22:37:57.838916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844279+0000","last_change":"2024-05-08T22:37:57.842727+0000","last_active":"2024-05-08T22:37:57.844279+0000","last_peered":"2024-05-08T22:37:57.844279+0000","last_clean":"2024-05-08T22:37:57.844279+0000","last_became_active":"2024-05-08T22:37:57.842350+0000","last_became_peered":"2024-05-08T22:37:57.842350+0000","last_unstale":"2024-05-08T22:37:57.844279+0000","last_undegraded":"2024-05-08T22:37:57.844279+0000","last_fullsized":"2024-05-08T22:37:57.844279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:32.252275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848269+0000","last_change":"2024-05-08T22:37:57.846399+0000","last_active":"2024-05-08T22:37:57.848269+0000","last_peered":"2024-05-08T22:37:57.848269+0000","last_clean":"2024-05-08T22:37:57.848269+0000","last_became_active":"2024-05-08T22:37:57.845308+0000","last_became_peered":"2024-05-08T22:37:57.845308+0000","last_unstale":"2024-05-08T22:37:57.848269+0000","last_undegraded":"2024-05-08T22:37:57.848269+0000","last_fullsized":"2024-05-08T22:37:57.848269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:57.759733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850942+0000","last_change":"2024-05-08T22:37:57.847700+0000","last_active":"2024-05-08T22:37:57.850942+0000","last_peered":"2024-05-08T22:37:57.850942+0000","last_clean":"2024-05-08T22:37:57.850942+0000","last_became_active":"2024-05-08T22:37:57.847333+0000","last_became_peered":"2024-05-08T22:37:57.847333+0000","last_unstale":"2024-05-08T22:37:57.850942+0000","last_undegraded":"2024-05-08T22:37:57.850942+0000","last_fullsized":"2024-05-08T22:37:57.850942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:59.881785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839403+0000","last_change":"2024-05-08T22:37:57.832432+0000","last_active":"2024-05-08T22:37:57.839403+0000","last_peered":"2024-05-08T22:37:57.839403+0000","last_clean":"2024-05-08T22:37:57.839403+0000","last_became_active":"2024-05-08T22:37:57.832048+0000","last_became_peered":"2024-05-08T22:37:57.832048+0000","last_unstale":"2024-05-08T22:37:57.839403+0000","last_undegraded":"2024-05-08T22:37:57.839403+0000","last_fullsized":"2024-05-08T22:37:57.839403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:26.697556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848302+0000","last_change":"2024-05-08T22:37:57.846519+0000","last_active":"2024-05-08T22:37:57.848302+0000","last_peered":"2024-05-08T22:37:57.848302+0000","last_clean":"2024-05-08T22:37:57.848302+0000","last_became_active":"2024-05-08T22:37:57.845361+0000","last_became_peered":"2024-05-08T22:37:57.845361+0000","last_unstale":"2024-05-08T22:37:57.848302+0000","last_undegraded":"2024-05-08T22:37:57.848302+0000","last_fullsized":"2024-05-08T22:37:57.848302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:34.786251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.430120+0000","last_change":"2024-05-08T22:37:57.430120+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.430120+0000","last_undegraded":"2024-05-08T22:37:57.430120+0000","last_fullsized":"2024-05-08T22:37:57.430120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839006+0000","last_change":"2024-05-08T22:37:57.825872+0000","last_active":"2024-05-08T22:37:57.839006+0000","last_peered":"2024-05-08T22:37:57.839006+0000","last_clean":"2024-05-08T22:37:57.839006+0000","last_became_active":"2024-05-08T22:37:57.825591+0000","last_became_peered":"2024-05-08T22:37:57.825591+0000","last_unstale":"2024-05-08T22:37:57.839006+0000","last_undegraded":"2024-05-08T22:37:57.839006+0000","last_fullsized":"2024-05-08T22:37:57.839006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:07.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848675+0000","last_change":"2024-05-08T22:37:57.844985+0000","last_active":"2024-05-08T22:37:57.848675+0000","last_peered":"2024-05-08T22:37:57.848675+0000","last_clean":"2024-05-08T22:37:57.848675+0000","last_became_active":"2024-05-08T22:37:57.844748+0000","last_became_peered":"2024-05-08T22:37:57.844748+0000","last_unstale":"2024-05-08T22:37:57.848675+0000","last_undegraded":"2024-05-08T22:37:57.848675+0000","last_fullsized":"2024-05-08T22:37:57.848675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:02.244593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838791+0000","last_change":"2024-05-08T22:37:57.830338+0000","last_active":"2024-05-08T22:37:57.838791+0000","last_peered":"2024-05-08T22:37:57.838791+0000","last_clean":"2024-05-08T22:37:57.838791+0000","last_became_active":"2024-05-08T22:37:57.830069+0000","last_became_peered":"2024-05-08T22:37:57.830069+0000","last_unstale":"2024-05-08T22:37:57.838791+0000","last_undegraded":"2024-05-08T22:37:57.838791+0000","last_fullsized":"2024-05-08T22:37:57.838791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:53.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841042+0000","last_change":"2024-05-08T22:37:57.835141+0000","last_active":"2024-05-08T22:37:57.841042+0000","last_peered":"2024-05-08T22:37:57.841042+0000","last_clean":"2024-05-08T22:37:57.841042+0000","last_became_active":"2024-05-08T22:37:57.832615+0000","last_became_peered":"2024-05-08T22:37:57.832615+0000","last_unstale":"2024-05-08T22:37:57.841042+0000","last_undegraded":"2024-05-08T22:37:57.841042+0000","last_fullsized":"2024-05-08T22:37:57.841042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:40.282177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850914+0000","last_change":"2024-05-08T22:37:57.847622+0000","last_active":"2024-05-08T22:37:57.850914+0000","last_peered":"2024-05-08T22:37:57.850914+0000","last_clean":"2024-05-08T22:37:57.850914+0000","last_became_active":"2024-05-08T22:37:57.847294+0000","last_became_peered":"2024-05-08T22:37:57.847294+0000","last_unstale":"2024-05-08T22:37:57.850914+0000","last_undegraded":"2024-05-08T22:37:57.850914+0000","last_fullsized":"2024-05-08T22:37:57.850914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:14.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843497+0000","last_change":"2024-05-08T22:37:57.836871+0000","last_active":"2024-05-08T22:37:57.843497+0000","last_peered":"2024-05-08T22:37:57.843497+0000","last_clean":"2024-05-08T22:37:57.843497+0000","last_became_active":"2024-05-08T22:37:57.836626+0000","last_became_peered":"2024-05-08T22:37:57.836626+0000","last_unstale":"2024-05-08T22:37:57.843497+0000","last_undegraded":"2024-05-08T22:37:57.843497+0000","last_fullsized":"2024-05-08T22:37:57.843497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:22.008737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844329+0000","last_change":"2024-05-08T22:37:57.840135+0000","last_active":"2024-05-08T22:37:57.844329+0000","last_peered":"2024-05-08T22:37:57.844329+0000","last_clean":"2024-05-08T22:37:57.844329+0000","last_became_active":"2024-05-08T22:37:57.839806+0000","last_became_peered":"2024-05-08T22:37:57.839806+0000","last_unstale":"2024-05-08T22:37:57.844329+0000","last_undegraded":"2024-05-08T22:37:57.844329+0000","last_fullsized":"2024-05-08T22:37:57.844329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:53.757689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841087+0000","last_change":"2024-05-08T22:37:57.832054+0000","last_active":"2024-05-08T22:37:57.841087+0000","last_peered":"2024-05-08T22:37:57.841087+0000","last_clean":"2024-05-08T22:37:57.841087+0000","last_became_active":"2024-05-08T22:37:57.831311+0000","last_became_peered":"2024-05-08T22:37:57.831311+0000","last_unstale":"2024-05-08T22:37:57.841087+0000","last_undegraded":"2024-05-08T22:37:57.841087+0000","last_fullsized":"2024-05-08T22:37:57.841087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:27.877407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850784+0000","last_change":"2024-05-08T22:37:57.847959+0000","last_active":"2024-05-08T22:37:57.850784+0000","last_peered":"2024-05-08T22:37:57.850784+0000","last_clean":"2024-05-08T22:37:57.850784+0000","last_became_active":"2024-05-08T22:37:57.847531+0000","last_became_peered":"2024-05-08T22:37:57.847531+0000","last_unstale":"2024-05-08T22:37:57.850784+0000","last_undegraded":"2024-05-08T22:37:57.850784+0000","last_fullsized":"2024-05-08T22:37:57.850784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:15.951130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833932+0000","last_change":"2024-05-08T22:37:57.832828+0000","last_active":"2024-05-08T22:37:57.833932+0000","last_peered":"2024-05-08T22:37:57.833932+0000","last_clean":"2024-05-08T22:37:57.833932+0000","last_became_active":"2024-05-08T22:37:57.832544+0000","last_became_peered":"2024-05-08T22:37:57.832544+0000","last_unstale":"2024-05-08T22:37:57.833932+0000","last_undegraded":"2024-05-08T22:37:57.833932+0000","last_fullsized":"2024-05-08T22:37:57.833932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:57.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843557+0000","last_change":"2024-05-08T22:37:57.831045+0000","last_active":"2024-05-08T22:37:57.843557+0000","last_peered":"2024-05-08T22:37:57.843557+0000","last_clean":"2024-05-08T22:37:57.843557+0000","last_became_active":"2024-05-08T22:37:57.830745+0000","last_became_peered":"2024-05-08T22:37:57.830745+0000","last_unstale":"2024-05-08T22:37:57.843557+0000","last_undegraded":"2024-05-08T22:37:57.843557+0000","last_fullsized":"2024-05-08T22:37:57.843557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:31.214851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429934+0000","last_change":"2024-05-08T22:37:57.429934+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429934+0000","last_undegraded":"2024-05-08T22:37:57.429934+0000","last_fullsized":"2024-05-08T22:37:57.429934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846622+0000","last_change":"2024-05-08T22:37:57.844194+0000","last_active":"2024-05-08T22:37:57.846622+0000","last_peered":"2024-05-08T22:37:57.846622+0000","last_clean":"2024-05-08T22:37:57.846622+0000","last_became_active":"2024-05-08T22:37:57.841102+0000","last_became_peered":"2024-05-08T22:37:57.841102+0000","last_unstale":"2024-05-08T22:37:57.846622+0000","last_undegraded":"2024-05-08T22:37:57.846622+0000","last_fullsized":"2024-05-08T22:37:57.846622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:15.481984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424461+0000","last_change":"2024-05-08T22:37:57.424461+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424461+0000","last_undegraded":"2024-05-08T22:37:57.424461+0000","last_fullsized":"2024-05-08T22:37:57.424461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847805+0000","last_change":"2024-05-08T22:37:57.840565+0000","last_active":"2024-05-08T22:37:57.847805+0000","last_peered":"2024-05-08T22:37:57.847805+0000","last_clean":"2024-05-08T22:37:57.847805+0000","last_became_active":"2024-05-08T22:37:57.834465+0000","last_became_peered":"2024-05-08T22:37:57.834465+0000","last_unstale":"2024-05-08T22:37:57.847805+0000","last_undegraded":"2024-05-08T22:37:57.847805+0000","last_fullsized":"2024-05-08T22:37:57.847805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:23.426774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840471+0000","last_change":"2024-05-08T22:37:57.834804+0000","last_active":"2024-05-08T22:37:57.840471+0000","last_peered":"2024-05-08T22:37:57.840471+0000","last_clean":"2024-05-08T22:37:57.840471+0000","last_became_active":"2024-05-08T22:37:57.834268+0000","last_became_peered":"2024-05-08T22:37:57.834268+0000","last_unstale":"2024-05-08T22:37:57.840471+0000","last_undegraded":"2024-05-08T22:37:57.840471+0000","last_fullsized":"2024-05-08T22:37:57.840471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:10.524450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839026+0000","last_change":"2024-05-08T22:37:57.831011+0000","last_active":"2024-05-08T22:37:57.839026+0000","last_peered":"2024-05-08T22:37:57.839026+0000","last_clean":"2024-05-08T22:37:57.839026+0000","last_became_active":"2024-05-08T22:37:57.830662+0000","last_became_peered":"2024-05-08T22:37:57.830662+0000","last_unstale":"2024-05-08T22:37:57.839026+0000","last_undegraded":"2024-05-08T22:37:57.839026+0000","last_fullsized":"2024-05-08T22:37:57.839026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:33.816353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844342+0000","last_change":"2024-05-08T22:37:57.840718+0000","last_active":"2024-05-08T22:37:57.844342+0000","last_peered":"2024-05-08T22:37:57.844342+0000","last_clean":"2024-05-08T22:37:57.844342+0000","last_became_active":"2024-05-08T22:37:57.840400+0000","last_became_peered":"2024-05-08T22:37:57.840400+0000","last_unstale":"2024-05-08T22:37:57.844342+0000","last_undegraded":"2024-05-08T22:37:57.844342+0000","last_fullsized":"2024-05-08T22:37:57.844342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.153666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846441+0000","last_change":"2024-05-08T22:37:57.844109+0000","last_active":"2024-05-08T22:37:57.846441+0000","last_peered":"2024-05-08T22:37:57.846441+0000","last_clean":"2024-05-08T22:37:57.846441+0000","last_became_active":"2024-05-08T22:37:57.843582+0000","last_became_peered":"2024-05-08T22:37:57.843582+0000","last_unstale":"2024-05-08T22:37:57.846441+0000","last_undegraded":"2024-05-08T22:37:57.846441+0000","last_fullsized":"2024-05-08T22:37:57.846441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:58.974111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847790+0000","last_change":"2024-05-08T22:37:57.841963+0000","last_active":"2024-05-08T22:37:57.847790+0000","last_peered":"2024-05-08T22:37:57.847790+0000","last_clean":"2024-05-08T22:37:57.847790+0000","last_became_active":"2024-05-08T22:37:57.841575+0000","last_became_peered":"2024-05-08T22:37:57.841575+0000","last_unstale":"2024-05-08T22:37:57.847790+0000","last_undegraded":"2024-05-08T22:37:57.847790+0000","last_fullsized":"2024-05-08T22:37:57.847790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:33.257259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845819+0000","last_change":"2024-05-08T22:37:57.840274+0000","last_active":"2024-05-08T22:37:57.845819+0000","last_peered":"2024-05-08T22:37:57.845819+0000","last_clean":"2024-05-08T22:37:57.845819+0000","last_became_active":"2024-05-08T22:37:57.840011+0000","last_became_peered":"2024-05-08T22:37:57.840011+0000","last_unstale":"2024-05-08T22:37:57.845819+0000","last_undegraded":"2024-05-08T22:37:57.845819+0000","last_fullsized":"2024-05-08T22:37:57.845819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:45.626643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.832439+0000","last_change":"2024-05-08T22:37:57.827795+0000","last_active":"2024-05-08T22:37:57.832439+0000","last_peered":"2024-05-08T22:37:57.832439+0000","last_clean":"2024-05-08T22:37:57.832439+0000","last_became_active":"2024-05-08T22:37:57.827400+0000","last_became_peered":"2024-05-08T22:37:57.827400+0000","last_unstale":"2024-05-08T22:37:57.832439+0000","last_undegraded":"2024-05-08T22:37:57.832439+0000","last_fullsized":"2024-05-08T22:37:57.832439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:31.579840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848125+0000","last_change":"2024-05-08T22:37:57.844868+0000","last_active":"2024-05-08T22:37:57.848125+0000","last_peered":"2024-05-08T22:37:57.848125+0000","last_clean":"2024-05-08T22:37:57.848125+0000","last_became_active":"2024-05-08T22:37:57.844575+0000","last_became_peered":"2024-05-08T22:37:57.844575+0000","last_unstale":"2024-05-08T22:37:57.848125+0000","last_undegraded":"2024-05-08T22:37:57.848125+0000","last_fullsized":"2024-05-08T22:37:57.848125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850774+0000","last_change":"2024-05-08T22:37:57.847858+0000","last_active":"2024-05-08T22:37:57.850774+0000","last_peered":"2024-05-08T22:37:57.850774+0000","last_clean":"2024-05-08T22:37:57.850774+0000","last_became_active":"2024-05-08T22:37:57.847609+0000","last_became_peered":"2024-05-08T22:37:57.847609+0000","last_unstale":"2024-05-08T22:37:57.850774+0000","last_undegraded":"2024-05-08T22:37:57.850774+0000","last_fullsized":"2024-05-08T22:37:57.850774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:00.575919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844834+0000","last_change":"2024-05-08T22:37:57.446359+0000","last_active":"2024-05-08T22:37:57.844834+0000","last_peered":"2024-05-08T22:37:57.844834+0000","last_clean":"2024-05-08T22:37:57.844834+0000","last_became_active":"2024-05-08T22:37:57.446036+0000","last_became_peered":"2024-05-08T22:37:57.446036+0000","last_unstale":"2024-05-08T22:37:57.844834+0000","last_undegraded":"2024-05-08T22:37:57.844834+0000","last_fullsized":"2024-05-08T22:37:57.844834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:00.419625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847776+0000","last_change":"2024-05-08T22:37:57.842007+0000","last_active":"2024-05-08T22:37:57.847776+0000","last_peered":"2024-05-08T22:37:57.847776+0000","last_clean":"2024-05-08T22:37:57.847776+0000","last_became_active":"2024-05-08T22:37:57.841764+0000","last_became_peered":"2024-05-08T22:37:57.841764+0000","last_unstale":"2024-05-08T22:37:57.847776+0000","last_undegraded":"2024-05-08T22:37:57.847776+0000","last_fullsized":"2024-05-08T22:37:57.847776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:58.771407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842515+0000","last_change":"2024-05-08T22:37:57.440281+0000","last_active":"2024-05-08T22:37:57.842515+0000","last_peered":"2024-05-08T22:37:57.842515+0000","last_clean":"2024-05-08T22:37:57.842515+0000","last_became_active":"2024-05-08T22:37:57.439987+0000","last_became_peered":"2024-05-08T22:37:57.439987+0000","last_unstale":"2024-05-08T22:37:57.842515+0000","last_undegraded":"2024-05-08T22:37:57.842515+0000","last_fullsized":"2024-05-08T22:37:57.842515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:32.804508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.425803+0000","last_change":"2024-05-08T22:37:57.425803+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.425803+0000","last_undegraded":"2024-05-08T22:37:57.425803+0000","last_fullsized":"2024-05-08T22:37:57.425803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846799+0000","last_change":"2024-05-08T22:37:57.448745+0000","last_active":"2024-05-08T22:37:57.846799+0000","last_peered":"2024-05-08T22:37:57.846799+0000","last_clean":"2024-05-08T22:37:57.846799+0000","last_became_active":"2024-05-08T22:37:57.448495+0000","last_became_peered":"2024-05-08T22:37:57.448495+0000","last_unstale":"2024-05-08T22:37:57.846799+0000","last_undegraded":"2024-05-08T22:37:57.846799+0000","last_fullsized":"2024-05-08T22:37:57.846799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:52.343924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429477+0000","last_change":"2024-05-08T22:37:57.429477+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429477+0000","last_undegraded":"2024-05-08T22:37:57.429477+0000","last_fullsized":"2024-05-08T22:37:57.429477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848978+0000","last_change":"2024-05-08T22:37:57.442124+0000","last_active":"2024-05-08T22:37:57.848978+0000","last_peered":"2024-05-08T22:37:57.848978+0000","last_clean":"2024-05-08T22:37:57.848978+0000","last_became_active":"2024-05-08T22:37:57.441844+0000","last_became_peered":"2024-05-08T22:37:57.441844+0000","last_unstale":"2024-05-08T22:37:57.848978+0000","last_undegraded":"2024-05-08T22:37:57.848978+0000","last_fullsized":"2024-05-08T22:37:57.848978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:12.294211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848681+0000","last_change":"2024-05-08T22:37:57.845013+0000","last_active":"2024-05-08T22:37:57.848681+0000","last_peered":"2024-05-08T22:37:57.848681+0000","last_clean":"2024-05-08T22:37:57.848681+0000","last_became_active":"2024-05-08T22:37:57.844721+0000","last_became_peered":"2024-05-08T22:37:57.844721+0000","last_unstale":"2024-05-08T22:37:57.848681+0000","last_undegraded":"2024-05-08T22:37:57.848681+0000","last_fullsized":"2024-05-08T22:37:57.848681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.527397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439079+0000","last_change":"2024-05-08T22:37:57.438281+0000","last_active":"2024-05-08T22:37:57.439079+0000","last_peered":"2024-05-08T22:37:57.439079+0000","last_clean":"2024-05-08T22:37:57.439079+0000","last_became_active":"2024-05-08T22:37:57.438016+0000","last_became_peered":"2024-05-08T22:37:57.438016+0000","last_unstale":"2024-05-08T22:37:57.439079+0000","last_undegraded":"2024-05-08T22:37:57.439079+0000","last_fullsized":"2024-05-08T22:37:57.439079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:22.432349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848718+0000","last_change":"2024-05-08T22:37:57.845395+0000","last_active":"2024-05-08T22:37:57.848718+0000","last_peered":"2024-05-08T22:37:57.848718+0000","last_clean":"2024-05-08T22:37:57.848718+0000","last_became_active":"2024-05-08T22:37:57.845129+0000","last_became_peered":"2024-05-08T22:37:57.845129+0000","last_unstale":"2024-05-08T22:37:57.848718+0000","last_undegraded":"2024-05-08T22:37:57.848718+0000","last_fullsized":"2024-05-08T22:37:57.848718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:26.794424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838895+0000","last_change":"2024-05-08T22:37:57.436502+0000","last_active":"2024-05-08T22:37:57.838895+0000","last_peered":"2024-05-08T22:37:57.838895+0000","last_clean":"2024-05-08T22:37:57.838895+0000","last_became_active":"2024-05-08T22:37:57.436246+0000","last_became_peered":"2024-05-08T22:37:57.436246+0000","last_unstale":"2024-05-08T22:37:57.838895+0000","last_undegraded":"2024-05-08T22:37:57.838895+0000","last_fullsized":"2024-05-08T22:37:57.838895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:07.871185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.426045+0000","last_change":"2024-05-08T22:37:57.426045+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.426045+0000","last_undegraded":"2024-05-08T22:37:57.426045+0000","last_fullsized":"2024-05-08T22:37:57.426045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826353+0000","last_change":"2024-05-08T22:37:57.443165+0000","last_active":"2024-05-08T22:37:57.826353+0000","last_peered":"2024-05-08T22:37:57.826353+0000","last_clean":"2024-05-08T22:37:57.826353+0000","last_became_active":"2024-05-08T22:37:57.442883+0000","last_became_peered":"2024-05-08T22:37:57.442883+0000","last_unstale":"2024-05-08T22:37:57.826353+0000","last_undegraded":"2024-05-08T22:37:57.826353+0000","last_fullsized":"2024-05-08T22:37:57.826353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:43.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849717+0000","last_change":"2024-05-08T22:37:57.840713+0000","last_active":"2024-05-08T22:37:57.849717+0000","last_peered":"2024-05-08T22:37:57.849717+0000","last_clean":"2024-05-08T22:37:57.849717+0000","last_became_active":"2024-05-08T22:37:57.839777+0000","last_became_peered":"2024-05-08T22:37:57.839777+0000","last_unstale":"2024-05-08T22:37:57.849717+0000","last_undegraded":"2024-05-08T22:37:57.849717+0000","last_fullsized":"2024-05-08T22:37:57.849717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:09.133783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.828973+0000","last_change":"2024-05-08T22:37:57.445418+0000","last_active":"2024-05-08T22:37:57.828973+0000","last_peered":"2024-05-08T22:37:57.828973+0000","last_clean":"2024-05-08T22:37:57.828973+0000","last_became_active":"2024-05-08T22:37:57.445005+0000","last_became_peered":"2024-05-08T22:37:57.445005+0000","last_unstale":"2024-05-08T22:37:57.828973+0000","last_undegraded":"2024-05-08T22:37:57.828973+0000","last_fullsized":"2024-05-08T22:37:57.828973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:01.056718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848795+0000","last_change":"2024-05-08T22:37:57.846487+0000","last_active":"2024-05-08T22:37:57.848795+0000","last_peered":"2024-05-08T22:37:57.848795+0000","last_clean":"2024-05-08T22:37:57.848795+0000","last_became_active":"2024-05-08T22:37:57.845715+0000","last_became_peered":"2024-05-08T22:37:57.845715+0000","last_unstale":"2024-05-08T22:37:57.848795+0000","last_undegraded":"2024-05-08T22:37:57.848795+0000","last_fullsized":"2024-05-08T22:37:57.848795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:56.850979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829513+0000","last_change":"2024-05-08T22:37:57.442931+0000","last_active":"2024-05-08T22:37:57.829513+0000","last_peered":"2024-05-08T22:37:57.829513+0000","last_clean":"2024-05-08T22:37:57.829513+0000","last_became_active":"2024-05-08T22:37:57.441198+0000","last_became_peered":"2024-05-08T22:37:57.441198+0000","last_unstale":"2024-05-08T22:37:57.829513+0000","last_undegraded":"2024-05-08T22:37:57.829513+0000","last_fullsized":"2024-05-08T22:37:57.829513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:35.216047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844949+0000","last_change":"2024-05-08T22:37:57.844158+0000","last_active":"2024-05-08T22:37:57.844949+0000","last_peered":"2024-05-08T22:37:57.844949+0000","last_clean":"2024-05-08T22:37:57.844949+0000","last_became_active":"2024-05-08T22:37:57.843855+0000","last_became_peered":"2024-05-08T22:37:57.843855+0000","last_unstale":"2024-05-08T22:37:57.844949+0000","last_undegraded":"2024-05-08T22:37:57.844949+0000","last_fullsized":"2024-05-08T22:37:57.844949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:36.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822311+0000","last_change":"2024-05-08T22:37:57.439889+0000","last_active":"2024-05-08T22:37:57.822311+0000","last_peered":"2024-05-08T22:37:57.822311+0000","last_clean":"2024-05-08T22:37:57.822311+0000","last_became_active":"2024-05-08T22:37:57.439633+0000","last_became_peered":"2024-05-08T22:37:57.439633+0000","last_unstale":"2024-05-08T22:37:57.822311+0000","last_undegraded":"2024-05-08T22:37:57.822311+0000","last_fullsized":"2024-05-08T22:37:57.822311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:01.221867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839271+0000","last_change":"2024-05-08T22:37:57.833440+0000","last_active":"2024-05-08T22:37:57.839271+0000","last_peered":"2024-05-08T22:37:57.839271+0000","last_clean":"2024-05-08T22:37:57.839271+0000","last_became_active":"2024-05-08T22:37:57.833212+0000","last_became_peered":"2024-05-08T22:37:57.833212+0000","last_unstale":"2024-05-08T22:37:57.839271+0000","last_undegraded":"2024-05-08T22:37:57.839271+0000","last_fullsized":"2024-05-08T22:37:57.839271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:08.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846625+0000","last_change":"2024-05-08T22:37:57.444957+0000","last_active":"2024-05-08T22:37:57.846625+0000","last_peered":"2024-05-08T22:37:57.846625+0000","last_clean":"2024-05-08T22:37:57.846625+0000","last_became_active":"2024-05-08T22:37:57.444484+0000","last_became_peered":"2024-05-08T22:37:57.444484+0000","last_unstale":"2024-05-08T22:37:57.846625+0000","last_undegraded":"2024-05-08T22:37:57.846625+0000","last_fullsized":"2024-05-08T22:37:57.846625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:46.229790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836275+0000","last_change":"2024-05-08T22:37:57.830761+0000","last_active":"2024-05-08T22:37:57.836275+0000","last_peered":"2024-05-08T22:37:57.836275+0000","last_clean":"2024-05-08T22:37:57.836275+0000","last_became_active":"2024-05-08T22:37:57.830491+0000","last_became_peered":"2024-05-08T22:37:57.830491+0000","last_unstale":"2024-05-08T22:37:57.836275+0000","last_undegraded":"2024-05-08T22:37:57.836275+0000","last_fullsized":"2024-05-08T22:37:57.836275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:12.683143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849871+0000","last_change":"2024-05-08T22:37:57.439248+0000","last_active":"2024-05-08T22:37:57.849871+0000","last_peered":"2024-05-08T22:37:57.849871+0000","last_clean":"2024-05-08T22:37:57.849871+0000","last_became_active":"2024-05-08T22:37:57.438927+0000","last_became_peered":"2024-05-08T22:37:57.438927+0000","last_unstale":"2024-05-08T22:37:57.849871+0000","last_undegraded":"2024-05-08T22:37:57.849871+0000","last_fullsized":"2024-05-08T22:37:57.849871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:11.645499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838819+0000","last_change":"2024-05-08T22:37:57.835299+0000","last_active":"2024-05-08T22:37:57.838819+0000","last_peered":"2024-05-08T22:37:57.838819+0000","last_clean":"2024-05-08T22:37:57.838819+0000","last_became_active":"2024-05-08T22:37:57.834959+0000","last_became_peered":"2024-05-08T22:37:57.834959+0000","last_unstale":"2024-05-08T22:37:57.838819+0000","last_undegraded":"2024-05-08T22:37:57.838819+0000","last_fullsized":"2024-05-08T22:37:57.838819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:19.550971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848959+0000","last_change":"2024-05-08T22:37:57.436168+0000","last_active":"2024-05-08T22:37:57.848959+0000","last_peered":"2024-05-08T22:37:57.848959+0000","last_clean":"2024-05-08T22:37:57.848959+0000","last_became_active":"2024-05-08T22:37:57.435911+0000","last_became_peered":"2024-05-08T22:37:57.435911+0000","last_unstale":"2024-05-08T22:37:57.848959+0000","last_undegraded":"2024-05-08T22:37:57.848959+0000","last_fullsized":"2024-05-08T22:37:57.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:39.531241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839382+0000","last_change":"2024-05-08T22:37:57.834557+0000","last_active":"2024-05-08T22:37:57.839382+0000","last_peered":"2024-05-08T22:37:57.839382+0000","last_clean":"2024-05-08T22:37:57.839382+0000","last_became_active":"2024-05-08T22:37:57.834336+0000","last_became_peered":"2024-05-08T22:37:57.834336+0000","last_unstale":"2024-05-08T22:37:57.839382+0000","last_undegraded":"2024-05-08T22:37:57.839382+0000","last_fullsized":"2024-05-08T22:37:57.839382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:07.977190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826416+0000","last_change":"2024-05-08T22:37:57.442582+0000","last_active":"2024-05-08T22:37:57.826416+0000","last_peered":"2024-05-08T22:37:57.826416+0000","last_clean":"2024-05-08T22:37:57.826416+0000","last_became_active":"2024-05-08T22:37:57.440956+0000","last_became_peered":"2024-05-08T22:37:57.440956+0000","last_unstale":"2024-05-08T22:37:57.826416+0000","last_undegraded":"2024-05-08T22:37:57.826416+0000","last_fullsized":"2024-05-08T22:37:57.826416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:40.041454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849792+0000","last_change":"2024-05-08T22:37:57.846329+0000","last_active":"2024-05-08T22:37:57.849792+0000","last_peered":"2024-05-08T22:37:57.849792+0000","last_clean":"2024-05-08T22:37:57.849792+0000","last_became_active":"2024-05-08T22:37:57.846056+0000","last_became_peered":"2024-05-08T22:37:57.846056+0000","last_unstale":"2024-05-08T22:37:57.849792+0000","last_undegraded":"2024-05-08T22:37:57.849792+0000","last_fullsized":"2024-05-08T22:37:57.849792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829433+0000","last_change":"2024-05-08T22:37:57.448219+0000","last_active":"2024-05-08T22:37:57.829433+0000","last_peered":"2024-05-08T22:37:57.829433+0000","last_clean":"2024-05-08T22:37:57.829433+0000","last_became_active":"2024-05-08T22:37:57.447923+0000","last_became_peered":"2024-05-08T22:37:57.447923+0000","last_unstale":"2024-05-08T22:37:57.829433+0000","last_undegraded":"2024-05-08T22:37:57.829433+0000","last_fullsized":"2024-05-08T22:37:57.829433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:30.907553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839082+0000","last_change":"2024-05-08T22:37:57.829299+0000","last_active":"2024-05-08T22:37:57.839082+0000","last_peered":"2024-05-08T22:37:57.839082+0000","last_clean":"2024-05-08T22:37:57.839082+0000","last_became_active":"2024-05-08T22:37:57.828970+0000","last_became_peered":"2024-05-08T22:37:57.828970+0000","last_unstale":"2024-05-08T22:37:57.839082+0000","last_undegraded":"2024-05-08T22:37:57.839082+0000","last_fullsized":"2024-05-08T22:37:57.839082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:01.133413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.851908+0000","last_change":"2024-05-08T22:37:57.444179+0000","last_active":"2024-05-08T22:37:57.851908+0000","last_peered":"2024-05-08T22:37:57.851908+0000","last_clean":"2024-05-08T22:37:57.851908+0000","last_became_active":"2024-05-08T22:37:57.443673+0000","last_became_peered":"2024-05-08T22:37:57.443673+0000","last_unstale":"2024-05-08T22:37:57.851908+0000","last_undegraded":"2024-05-08T22:37:57.851908+0000","last_fullsized":"2024-05-08T22:37:57.851908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:53.287958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845235+0000","last_change":"2024-05-08T22:37:57.841025+0000","last_active":"2024-05-08T22:37:57.845235+0000","last_peered":"2024-05-08T22:37:57.845235+0000","last_clean":"2024-05-08T22:37:57.845235+0000","last_became_active":"2024-05-08T22:37:57.840591+0000","last_became_peered":"2024-05-08T22:37:57.840591+0000","last_unstale":"2024-05-08T22:37:57.845235+0000","last_undegraded":"2024-05-08T22:37:57.845235+0000","last_fullsized":"2024-05-08T22:37:57.845235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.910062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842909+0000","last_change":"2024-05-08T22:37:57.440960+0000","last_active":"2024-05-08T22:37:57.842909+0000","last_peered":"2024-05-08T22:37:57.842909+0000","last_clean":"2024-05-08T22:37:57.842909+0000","last_became_active":"2024-05-08T22:37:57.440504+0000","last_became_peered":"2024-05-08T22:37:57.440504+0000","last_unstale":"2024-05-08T22:37:57.842909+0000","last_undegraded":"2024-05-08T22:37:57.842909+0000","last_fullsized":"2024-05-08T22:37:57.842909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:06.401155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841075+0000","last_change":"2024-05-08T22:37:57.831846+0000","last_active":"2024-05-08T22:37:57.841075+0000","last_peered":"2024-05-08T22:37:57.841075+0000","last_clean":"2024-05-08T22:37:57.841075+0000","last_became_active":"2024-05-08T22:37:57.829692+0000","last_became_peered":"2024-05-08T22:37:57.829692+0000","last_unstale":"2024-05-08T22:37:57.841075+0000","last_undegraded":"2024-05-08T22:37:57.841075+0000","last_fullsized":"2024-05-08T22:37:57.841075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.674154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439143+0000","last_change":"2024-05-08T22:37:57.437527+0000","last_active":"2024-05-08T22:37:57.439143+0000","last_peered":"2024-05-08T22:37:57.439143+0000","last_clean":"2024-05-08T22:37:57.439143+0000","last_became_active":"2024-05-08T22:37:57.437261+0000","last_became_peered":"2024-05-08T22:37:57.437261+0000","last_unstale":"2024-05-08T22:37:57.439143+0000","last_undegraded":"2024-05-08T22:37:57.439143+0000","last_fullsized":"2024-05-08T22:37:57.439143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:19.394053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844277+0000","last_change":"2024-05-08T22:37:57.832513+0000","last_active":"2024-05-08T22:37:57.844277+0000","last_peered":"2024-05-08T22:37:57.844277+0000","last_clean":"2024-05-08T22:37:57.844277+0000","last_became_active":"2024-05-08T22:37:57.832275+0000","last_became_peered":"2024-05-08T22:37:57.832275+0000","last_unstale":"2024-05-08T22:37:57.844277+0000","last_undegraded":"2024-05-08T22:37:57.844277+0000","last_fullsized":"2024-05-08T22:37:57.844277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:08.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847277+0000","last_change":"2024-05-08T22:37:57.444668+0000","last_active":"2024-05-08T22:37:57.847277+0000","last_peered":"2024-05-08T22:37:57.847277+0000","last_clean":"2024-05-08T22:37:57.847277+0000","last_became_active":"2024-05-08T22:37:57.441598+0000","last_became_peered":"2024-05-08T22:37:57.441598+0000","last_unstale":"2024-05-08T22:37:57.847277+0000","last_undegraded":"2024-05-08T22:37:57.847277+0000","last_fullsized":"2024-05-08T22:37:57.847277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:33.716602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833929+0000","last_change":"2024-05-08T22:37:57.832325+0000","last_active":"2024-05-08T22:37:57.833929+0000","last_peered":"2024-05-08T22:37:57.833929+0000","last_clean":"2024-05-08T22:37:57.833929+0000","last_became_active":"2024-05-08T22:37:57.832067+0000","last_became_peered":"2024-05-08T22:37:57.832067+0000","last_unstale":"2024-05-08T22:37:57.833929+0000","last_undegraded":"2024-05-08T22:37:57.833929+0000","last_fullsized":"2024-05-08T22:37:57.833929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:42.187631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817518+0000","last_change":"2024-05-08T22:37:57.442255+0000","last_active":"2024-05-08T22:37:57.817518+0000","last_peered":"2024-05-08T22:37:57.817518+0000","last_clean":"2024-05-08T22:37:57.817518+0000","last_became_active":"2024-05-08T22:37:57.441960+0000","last_became_peered":"2024-05-08T22:37:57.441960+0000","last_unstale":"2024-05-08T22:37:57.817518+0000","last_undegraded":"2024-05-08T22:37:57.817518+0000","last_fullsized":"2024-05-08T22:37:57.817518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:59.020957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848551+0000","last_change":"2024-05-08T22:37:57.842118+0000","last_active":"2024-05-08T22:37:57.848551+0000","last_peered":"2024-05-08T22:37:57.848551+0000","last_clean":"2024-05-08T22:37:57.848551+0000","last_became_active":"2024-05-08T22:37:57.841826+0000","last_became_peered":"2024-05-08T22:37:57.841826+0000","last_unstale":"2024-05-08T22:37:57.848551+0000","last_undegraded":"2024-05-08T22:37:57.848551+0000","last_fullsized":"2024-05-08T22:37:57.848551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:55.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840125+0000","last_change":"2024-05-08T22:37:57.439634+0000","last_active":"2024-05-08T22:37:57.840125+0000","last_peered":"2024-05-08T22:37:57.840125+0000","last_clean":"2024-05-08T22:37:57.840125+0000","last_became_active":"2024-05-08T22:37:57.439387+0000","last_became_peered":"2024-05-08T22:37:57.439387+0000","last_unstale":"2024-05-08T22:37:57.840125+0000","last_undegraded":"2024-05-08T22:37:57.840125+0000","last_fullsized":"2024-05-08T22:37:57.840125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:49.158578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848777+0000","last_change":"2024-05-08T22:37:57.846904+0000","last_active":"2024-05-08T22:37:57.848777+0000","last_peered":"2024-05-08T22:37:57.848777+0000","last_clean":"2024-05-08T22:37:57.848777+0000","last_became_active":"2024-05-08T22:37:57.846651+0000","last_became_peered":"2024-05-08T22:37:57.846651+0000","last_unstale":"2024-05-08T22:37:57.848777+0000","last_undegraded":"2024-05-08T22:37:57.848777+0000","last_fullsized":"2024-05-08T22:37:57.848777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:05.340141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.821917+0000","last_change":"2024-05-08T22:37:57.443724+0000","last_active":"2024-05-08T22:37:57.821917+0000","last_peered":"2024-05-08T22:37:57.821917+0000","last_clean":"2024-05-08T22:37:57.821917+0000","last_became_active":"2024-05-08T22:37:57.443460+0000","last_became_peered":"2024-05-08T22:37:57.443460+0000","last_unstale":"2024-05-08T22:37:57.821917+0000","last_undegraded":"2024-05-08T22:37:57.821917+0000","last_fullsized":"2024-05-08T22:37:57.821917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:08.118921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.837256+0000","last_change":"2024-05-08T22:37:57.826029+0000","last_active":"2024-05-08T22:37:57.837256+0000","last_peered":"2024-05-08T22:37:57.837256+0000","last_clean":"2024-05-08T22:37:57.837256+0000","last_became_active":"2024-05-08T22:37:57.825743+0000","last_became_peered":"2024-05-08T22:37:57.825743+0000","last_unstale":"2024-05-08T22:37:57.837256+0000","last_undegraded":"2024-05-08T22:37:57.837256+0000","last_fullsized":"2024-05-08T22:37:57.837256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.587376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822152+0000","last_change":"2024-05-08T22:37:57.444844+0000","last_active":"2024-05-08T22:37:57.822152+0000","last_peered":"2024-05-08T22:37:57.822152+0000","last_clean":"2024-05-08T22:37:57.822152+0000","last_became_active":"2024-05-08T22:37:57.444488+0000","last_became_peered":"2024-05-08T22:37:57.444488+0000","last_unstale":"2024-05-08T22:37:57.822152+0000","last_undegraded":"2024-05-08T22:37:57.822152+0000","last_fullsized":"2024-05-08T22:37:57.822152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:36.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424456+0000","last_change":"2024-05-08T22:37:57.424456+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424456+0000","last_undegraded":"2024-05-08T22:37:57.424456+0000","last_fullsized":"2024-05-08T22:37:57.424456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838790+0000","last_change":"2024-05-08T22:37:57.438922+0000","last_active":"2024-05-08T22:37:57.838790+0000","last_peered":"2024-05-08T22:37:57.838790+0000","last_clean":"2024-05-08T22:37:57.838790+0000","last_became_active":"2024-05-08T22:37:57.438655+0000","last_became_peered":"2024-05-08T22:37:57.438655+0000","last_unstale":"2024-05-08T22:37:57.838790+0000","last_undegraded":"2024-05-08T22:37:57.838790+0000","last_fullsized":"2024-05-08T22:37:57.838790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:09:11.390285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838438+0000","last_change":"2024-05-08T22:37:57.828420+0000","last_active":"2024-05-08T22:37:57.838438+0000","last_peered":"2024-05-08T22:37:57.838438+0000","last_clean":"2024-05-08T22:37:57.838438+0000","last_became_active":"2024-05-08T22:37:57.828132+0000","last_became_peered":"2024-05-08T22:37:57.828132+0000","last_unstale":"2024-05-08T22:37:57.838438+0000","last_undegraded":"2024-05-08T22:37:57.838438+0000","last_fullsized":"2024-05-08T22:37:57.838438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:23.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.364530+0000","last_change":"2024-05-08T22:37:57.447723+0000","last_active":"2024-05-08T22:37:58.364530+0000","last_peered":"2024-05-08T22:37:58.364530+0000","last_clean":"2024-05-08T22:37:58.364530+0000","last_became_active":"2024-05-08T22:37:57.447478+0000","last_became_peered":"2024-05-08T22:37:57.447478+0000","last_unstale":"2024-05-08T22:37:58.364530+0000","last_undegraded":"2024-05-08T22:37:58.364530+0000","last_fullsized":"2024-05-08T22:37:58.364530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:09.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424687+0000","last_change":"2024-05-08T22:37:57.424687+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424687+0000","last_undegraded":"2024-05-08T22:37:57.424687+0000","last_fullsized":"2024-05-08T22:37:57.424687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.827351+0000","last_change":"2024-05-08T22:37:57.437575+0000","last_active":"2024-05-08T22:37:57.827351+0000","last_peered":"2024-05-08T22:37:57.827351+0000","last_clean":"2024-05-08T22:37:57.827351+0000","last_became_active":"2024-05-08T22:37:57.437308+0000","last_became_peered":"2024-05-08T22:37:57.437308+0000","last_unstale":"2024-05-08T22:37:57.827351+0000","last_undegraded":"2024-05-08T22:37:57.827351+0000","last_fullsized":"2024-05-08T22:37:57.827351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:33.154187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848732+0000","last_change":"2024-05-08T22:37:57.845583+0000","last_active":"2024-05-08T22:37:57.848732+0000","last_peered":"2024-05-08T22:37:57.848732+0000","last_clean":"2024-05-08T22:37:57.848732+0000","last_became_active":"2024-05-08T22:37:57.845321+0000","last_became_peered":"2024-05-08T22:37:57.845321+0000","last_unstale":"2024-05-08T22:37:57.848732+0000","last_undegraded":"2024-05-08T22:37:57.848732+0000","last_fullsized":"2024-05-08T22:37:57.848732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:27.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825845+0000","last_change":"2024-05-08T22:37:57.440921+0000","last_active":"2024-05-08T22:37:57.825845+0000","last_peered":"2024-05-08T22:37:57.825845+0000","last_clean":"2024-05-08T22:37:57.825845+0000","last_became_active":"2024-05-08T22:37:57.440652+0000","last_became_peered":"2024-05-08T22:37:57.440652+0000","last_unstale":"2024-05-08T22:37:57.825845+0000","last_undegraded":"2024-05-08T22:37:57.825845+0000","last_fullsized":"2024-05-08T22:37:57.825845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:02.665256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838442+0000","last_change":"2024-05-08T22:37:57.828599+0000","last_active":"2024-05-08T22:37:57.838442+0000","last_peered":"2024-05-08T22:37:57.838442+0000","last_clean":"2024-05-08T22:37:57.838442+0000","last_became_active":"2024-05-08T22:37:57.828254+0000","last_became_peered":"2024-05-08T22:37:57.828254+0000","last_unstale":"2024-05-08T22:37:57.838442+0000","last_undegraded":"2024-05-08T22:37:57.838442+0000","last_fullsized":"2024-05-08T22:37:57.838442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:30.500016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822000+0000","last_change":"2024-05-08T22:37:57.442291+0000","last_active":"2024-05-08T22:37:57.822000+0000","last_peered":"2024-05-08T22:37:57.822000+0000","last_clean":"2024-05-08T22:37:57.822000+0000","last_became_active":"2024-05-08T22:37:57.441935+0000","last_became_peered":"2024-05-08T22:37:57.441935+0000","last_unstale":"2024-05-08T22:37:57.822000+0000","last_undegraded":"2024-05-08T22:37:57.822000+0000","last_fullsized":"2024-05-08T22:37:57.822000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:55.703307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839361+0000","last_change":"2024-05-08T22:37:57.832660+0000","last_active":"2024-05-08T22:37:57.839361+0000","last_peered":"2024-05-08T22:37:57.839361+0000","last_clean":"2024-05-08T22:37:57.839361+0000","last_became_active":"2024-05-08T22:37:57.832291+0000","last_became_peered":"2024-05-08T22:37:57.832291+0000","last_unstale":"2024-05-08T22:37:57.839361+0000","last_undegraded":"2024-05-08T22:37:57.839361+0000","last_fullsized":"2024-05-08T22:37:57.839361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:40.277738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.830600+0000","last_change":"2024-05-08T22:37:57.448069+0000","last_active":"2024-05-08T22:37:57.830600+0000","last_peered":"2024-05-08T22:37:57.830600+0000","last_clean":"2024-05-08T22:37:57.830600+0000","last_became_active":"2024-05-08T22:37:57.446523+0000","last_became_peered":"2024-05-08T22:37:57.446523+0000","last_unstale":"2024-05-08T22:37:57.830600+0000","last_undegraded":"2024-05-08T22:37:57.830600+0000","last_fullsized":"2024-05-08T22:37:57.830600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:44.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836038+0000","last_change":"2024-05-08T22:37:57.830871+0000","last_active":"2024-05-08T22:37:57.836038+0000","last_peered":"2024-05-08T22:37:57.836038+0000","last_clean":"2024-05-08T22:37:57.836038+0000","last_became_active":"2024-05-08T22:37:57.830594+0000","last_became_peered":"2024-05-08T22:37:57.830594+0000","last_unstale":"2024-05-08T22:37:57.836038+0000","last_undegraded":"2024-05-08T22:37:57.836038+0000","last_fullsized":"2024-05-08T22:37:57.836038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:02.234727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.823044+0000","last_change":"2024-05-08T22:37:57.438169+0000","last_active":"2024-05-08T22:37:57.823044+0000","last_peered":"2024-05-08T22:37:57.823044+0000","last_clean":"2024-05-08T22:37:57.823044+0000","last_became_active":"2024-05-08T22:37:57.437880+0000","last_became_peered":"2024-05-08T22:37:57.437880+0000","last_unstale":"2024-05-08T22:37:57.823044+0000","last_undegraded":"2024-05-08T22:37:57.823044+0000","last_fullsized":"2024-05-08T22:37:57.823044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:58.133761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850749+0000","last_change":"2024-05-08T22:37:57.846104+0000","last_active":"2024-05-08T22:37:57.850749+0000","last_peered":"2024-05-08T22:37:57.850749+0000","last_clean":"2024-05-08T22:37:57.850749+0000","last_became_active":"2024-05-08T22:37:57.845848+0000","last_became_peered":"2024-05-08T22:37:57.845848+0000","last_unstale":"2024-05-08T22:37:57.850749+0000","last_undegraded":"2024-05-08T22:37:57.850749+0000","last_fullsized":"2024-05-08T22:37:57.850749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:06:32.854193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817725+0000","last_change":"2024-05-08T22:37:57.437069+0000","last_active":"2024-05-08T22:37:57.817725+0000","last_peered":"2024-05-08T22:37:57.817725+0000","last_clean":"2024-05-08T22:37:57.817725+0000","last_became_active":"2024-05-08T22:37:57.436461+0000","last_became_peered":"2024-05-08T22:37:57.436461+0000","last_unstale":"2024-05-08T22:37:57.817725+0000","last_undegraded":"2024-05-08T22:37:57.817725+0000","last_fullsized":"2024-05-08T22:37:57.817725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:31.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845596+0000","last_change":"2024-05-08T22:37:57.843298+0000","last_active":"2024-05-08T22:37:57.845596+0000","last_peered":"2024-05-08T22:37:57.845596+0000","last_clean":"2024-05-08T22:37:57.845596+0000","last_became_active":"2024-05-08T22:37:57.842988+0000","last_became_peered":"2024-05-08T22:37:57.842988+0000","last_unstale":"2024-05-08T22:37:57.845596+0000","last_undegraded":"2024-05-08T22:37:57.845596+0000","last_fullsized":"2024-05-08T22:37:57.845596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:17.302357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825542+0000","last_change":"2024-05-08T22:37:57.444385+0000","last_active":"2024-05-08T22:37:57.825542+0000","last_peered":"2024-05-08T22:37:57.825542+0000","last_clean":"2024-05-08T22:37:57.825542+0000","last_became_active":"2024-05-08T22:37:57.444124+0000","last_became_peered":"2024-05-08T22:37:57.444124+0000","last_unstale":"2024-05-08T22:37:57.825542+0000","last_undegraded":"2024-05-08T22:37:57.825542+0000","last_fullsized":"2024-05-08T22:37:57.825542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:35.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.429399+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:37:57.429399+0000","last_peered":"2024-05-08T22:37:57.429399+0000","last_clean":"2024-05-08T22:37:57.429399+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:37:57.429399+0000","last_undegraded":"2024-05-08T22:37:57.429399+0000","last_fullsized":"2024-05-08T22:37:57.429399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":999424,"data_stored":451081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.309}]},{"osd":1,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.47599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89100000000000001,"5min":1.101,"15min":1.101},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.627}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.018,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.57899999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1140000000000001,"5min":1.306,"15min":1.306},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2310000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.274,"5min":1.339,"15min":1.339},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.0089999999999999,"15min":1.0269999999999999},"min":{"1min":0.76000000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2749999999999999,"5min":1.476,"15min":1.476},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98199999999999998,"15min":0.97199999999999998},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2370000000000001,"5min":1.446,"15min":1.446},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.753,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.341,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.105},"min":{"1min":0.82199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.298,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.056},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.036,"15min":1.034},"min":{"1min":0.81000000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4570000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.88400000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":450611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":1.04,"15min":1.04},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.58399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2130000000000001,"5min":1.583,"15min":1.583},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2050000000000001,"5min":1.431,"15min":1.431},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.81399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2470000000000001,"5min":1.508,"15min":1.508},"last":0.747}]},{"osd":5,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.028,"15min":1.083},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.397,"5min":1.726,"15min":1.726},"last":0.96999999999999997}]},{"osd":6,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.77200000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.446,"5min":1.659,"15min":1.659},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.071},"min":{"1min":0.85399999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3129999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0569999999999999},"min":{"1min":0.81699999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.409,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85599999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1593344,"data_stored":1040889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85499999999999998,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2749999999999999,"5min":1.282,"15min":1.282},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.84099999999999997,"15min":0.85999999999999999},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.121}]},{"osd":1,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91800000000000004,"15min":0.91000000000000003},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.099,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1140000000000001,"5min":1.419,"15min":1.419},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95099999999999996,"15min":0.96199999999999997},"min":{"1min":0.69199999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.337,"15min":1.337},"last":1.022}]},{"osd":3,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.79400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5269999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.72099999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.177},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.89300000000000002},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.103,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95999999999999996,"15min":0.98199999999999998},"min":{"1min":0.67000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.482,"15min":1.482},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.85399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.264}]},{"osd":6,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.76500000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4179999999999999,"5min":1.599,"15min":1.599},"last":1.0800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":1007616,"data_stored":452063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.141,"15min":1.141},"last":0.997}]},{"osd":1,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3300000000000001,"5min":1.758,"15min":1.758},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91800000000000004,"15min":0.93000000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1850000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.115}]},{"osd":2,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.92900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.397,"5min":1.419,"15min":1.419},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.034,"15min":1.0960000000000001},"min":{"1min":0.68000000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2110000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.99099999999999999},"min":{"1min":0.625,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2869999999999999,"5min":1.538,"15min":1.538},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":1.0469999999999999},"min":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.149,"5min":1.458,"15min":1.458},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.28,"5min":1.681,"15min":1.681},"last":1.194},{"interface":"front","average":{"1min":0.998,"5min":1.0840000000000001,"15min":1.1399999999999999},"min":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2050000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.014,"15min":1.085},"min":{"1min":0.71899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0780000000000001,"5min":1.835,"15min":1.835},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.179,"15min":1.2350000000000001},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3300000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1559999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":999424,"data_stored":450533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82399999999999995,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.145,"5min":1.52,"15min":1.52},"last":0.79000000000000004}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.016,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.266,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90400000000000003,"15min":0.88},"min":{"1min":0.71899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.83999999999999997},"min":{"1min":0.67600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.204}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":0.96499999999999997},"min":{"1min":0.71599999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.70099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.016,"15min":0.98599999999999999},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96799999999999997,"15min":0.91600000000000004},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2589999999999999,"5min":1.605,"15min":1.605},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.069,"15min":1.044},"min":{"1min":0.89200000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3080000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.3340000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.0880000000000001},"min":{"1min":0.98299999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.411,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.133,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.91500000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3089999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2909999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":970752,"data_stored":436809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.113,"5min":1.28,"15min":1.28},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.294,"15min":1.294},"last":0.72299999999999998}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.871},"min":{"1min":0.65800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.163,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90700000000000003,"15min":0.88600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.173,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0329999999999999,"5min":1.29,"15min":1.29},"last":1.224},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91900000000000004,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.135,"5min":1.637,"15min":1.637},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1619999999999999,"5min":2.601,"15min":2.601},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.032,"15min":1.105},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.998}]},{"osd":5,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.0349999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.121,"5min":1.792,"15min":1.792},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.026},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.137,"5min":1.575,"15min":1.575},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0489999999999999,"15min":1.0900000000000001},"min":{"1min":0.69399999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1100000000000001,"5min":1.831,"15min":1.831},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.093,"15min":1.1020000000000001},"min":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2210000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0609999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.071,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.23,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.159,"15min":1.177},"min":{"1min":0.878,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3,"5min":1.944,"15min":1.944},"last":1.085}]}]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":443404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.133,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":1.008,"15min":1.008},"last":0.879},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.58799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.333,"5min":1.363,"15min":1.363},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.006,"15min":1.028},"min":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.27,"5min":1.282,"15min":1.282},"last":1.2609999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.407,"5min":1.484,"15min":1.484},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.3}]},{"osd":6,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0600000000000001},"min":{"1min":0.84499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0700000000000001,"15min":1.0840000000000001},"min":{"1min":0.871,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4450000000000001,"5min":1.673,"15min":1.673},"last":1.101}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.119,"15min":1.1299999999999999},"min":{"1min":0.878,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.056}]}]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343276,"statfs":{"total":96636764160,"available":96607514624,"internally_reserved":0,"allocated":1576960,"data_stored":1033772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.617,"15min":1.617},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":1.446,"15min":1.446},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0129999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3089999999999999,"5min":1.714,"15min":1.714},"last":1.609},{"interface":"front","average":{"1min":0.755,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2569999999999999,"5min":1.327,"15min":1.327},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2709999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":4.391},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.73499999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3,"5min":1.669,"15min":1.669},"last":2.0590000000000002}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2210000000000001,"5min":1.474,"15min":1.474},"last":1.0760000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.96499999999999997},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2869999999999999,"5min":1.397,"15min":1.397},"last":1.8660000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":1.0469999999999999},"min":{"1min":0.78600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.177,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":4.4859999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0649999999999999,"15min":1.089},"min":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.234,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.532},{"interface":"front","average":{"1min":1.018,"5min":1.008,"15min":1.022},"min":{"1min":0.77500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.304}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:38:01.831 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:38:01.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 Processor -- start 2024-05-08T22:38:01.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- start start 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb98c14ae40 0x7fb98c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb98c165710 0x7fb98c162b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c0b3fb0 con 0x7fb98c165c80 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c14b220 con 0x7fb98c165710 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c0b33a0 con 0x7fb98c14ae40 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb98c165710 0x7fb98c162b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb98c165710 0x7fb98c162b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58762/0 (socket says 172.21.15.179:58762) 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 -- 172.21.15.179:0/3197637962 learned_addr learned my addr 172.21.15.179:0/3197637962 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98ad9d700 1 --2- 172.21.15.179:0/3197637962 >> v2:172.21.15.195:3300/0 conn(0x7fb98c14ae40 0x7fb98c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 -- 172.21.15.179:0/3197637962 >> v2:172.21.15.195:3300/0 conn(0x7fb98c14ae40 msgr2=0x7fb98c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:01.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 --2- 172.21.15.179:0/3197637962 >> v2:172.21.15.195:3300/0 conn(0x7fb98c14ae40 0x7fb98c0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 -- 172.21.15.179:0/3197637962 >> v1:172.21.15.195:6789/0 conn(0x7fb98c165c80 legacy=0x7fb98c165000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 -- 172.21.15.179:0/3197637962 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0b3680 con 0x7fb98c165710 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb98b59e700 1 --2- 172.21.15.179:0/3197637962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c165710 0x7fb98c162b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb98c071a60 tx=0x7fb97c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99b9950269524c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb989d9b700 1 -- 172.21.15.179:0/3197637962 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c011830 con 0x7fb98c165710 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb989d9b700 1 -- 172.21.15.179:0/3197637962 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c011990 con 0x7fb98c165710 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb989d9b700 1 -- 172.21.15.179:0/3197637962 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c011d20 con 0x7fb98c165710 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- 172.21.15.179:0/3197637962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c165710 msgr2=0x7fb98c162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 --2- 172.21.15.179:0/3197637962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c165710 0x7fb98c162b20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb98c071a60 tx=0x7fb97c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- 172.21.15.179:0/3197637962 shutdown_connections 2024-05-08T22:38:01.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 --2- 172.21.15.179:0/3197637962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c165710 0x7fb98c162b20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 --2- 172.21.15.179:0/3197637962 >> v2:172.21.15.195:3300/0 conn(0x7fb98c14ae40 0x7fb98c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- 172.21.15.179:0/3197637962 >> 172.21.15.179:0/3197637962 conn(0x7fb98c0cfae0 msgr2=0x7fb98c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:01.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- 172.21.15.179:0/3197637962 shutdown_connections 2024-05-08T22:38:01.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.936+0000 7fb990884700 1 -- 172.21.15.179:0/3197637962 wait complete. 2024-05-08T22:38:01.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 Processor -- start 2024-05-08T22:38:01.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 -- start start 2024-05-08T22:38:01.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb98c165710 0x7fb98c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb98c165c80 0x7fb98c13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c13e770 con 0x7fb98c14ae40 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c13d190 con 0x7fb98c165710 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb990884700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c13d470 con 0x7fb98c165c80 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb98ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb98ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58768/0 (socket says 172.21.15.179:58768) 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb98ad9d700 1 -- 172.21.15.179:0/574474171 learned_addr learned my addr 172.21.15.179:0/574474171 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb98b59e700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb98c165c80 0x7fb98c13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.940+0000 7fb98a59c700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb98c165710 0x7fb98c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 -- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb98c165c80 msgr2=0x7fb98c13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb98c165c80 0x7fb98c13c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 -- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb98c165710 msgr2=0x7fb98c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb98c165710 0x7fb98c15c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 -- 172.21.15.179:0/574474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009570 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb98ad9d700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9d0 tx=0x7fb97400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974009720 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400ee70 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c13d750 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400ca50 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c13dd10 con 0x7fb98c14ae40 2024-05-08T22:38:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c13ded0 con 0x7fb98c14ae40 2024-05-08T22:38:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.944+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb97400cbb0 con 0x7fb98c14ae40 2024-05-08T22:38:01.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.948+0000 7fb9837fe700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 0x7fb96c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:01.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.948+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb974084240 con 0x7fb98c14ae40 2024-05-08T22:38:01.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.948+0000 7fb98a59c700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 0x7fb96c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:01.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.948+0000 7fb98a59c700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 0x7fb96c068c20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb98c1499b0 tx=0x7fb984009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:01.952+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb974051470 con 0x7fb98c14ae40 2024-05-08T22:38:02.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.124+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb98c0b2ed0 con 0x7fb96c066760 2024-05-08T22:38:02.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.144+0000 7fb9837fe700 1 -- 172.21.15.179:0/574474171 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287288 (secure 0 0 0) 0x7fb98c0b2ed0 con 0x7fb96c066760 2024-05-08T22:38:02.149 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 msgr2=0x7fb96c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 0x7fb96c068c20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb98c1499b0 tx=0x7fb984009040 comp rx=0 tx=0).stop 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 msgr2=0x7fb98c13e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9d0 tx=0x7fb97400ece0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 shutdown_connections 2024-05-08T22:38:02.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb96c066760 0x7fb96c068c20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb98c165710 0x7fb98c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb98c165c80 0x7fb98c13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 --2- 172.21.15.179:0/574474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb98c14ae40 0x7fb98c13e230 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 >> 172.21.15.179:0/574474171 conn(0x7fb98c0cfae0 msgr2=0x7fb98c161910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 shutdown_connections 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.148+0000 7fb990884700 1 -- 172.21.15.179:0/574474171 wait complete. 2024-05-08T22:38:02.152 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:38:02.165 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-05-08T22:38:01.478708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253804,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754720916,"statfs":{"total":773094113280,"available":772834217984,"internally_reserved":0,"allocated":9134080,"data_stored":4759162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.067987"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848306+0000","last_change":"2024-05-08T22:37:57.844883+0000","last_active":"2024-05-08T22:37:57.848306+0000","last_peered":"2024-05-08T22:37:57.848306+0000","last_clean":"2024-05-08T22:37:57.848306+0000","last_became_active":"2024-05-08T22:37:57.844614+0000","last_became_peered":"2024-05-08T22:37:57.844614+0000","last_unstale":"2024-05-08T22:37:57.848306+0000","last_undegraded":"2024-05-08T22:37:57.848306+0000","last_fullsized":"2024-05-08T22:37:57.848306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:56:36.890701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848623+0000","last_change":"2024-05-08T22:37:57.844447+0000","last_active":"2024-05-08T22:37:57.848623+0000","last_peered":"2024-05-08T22:37:57.848623+0000","last_clean":"2024-05-08T22:37:57.848623+0000","last_became_active":"2024-05-08T22:37:57.844201+0000","last_became_peered":"2024-05-08T22:37:57.844201+0000","last_unstale":"2024-05-08T22:37:57.848623+0000","last_undegraded":"2024-05-08T22:37:57.848623+0000","last_fullsized":"2024-05-08T22:37:57.848623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:10.013238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838916+0000","last_change":"2024-05-08T22:37:57.835351+0000","last_active":"2024-05-08T22:37:57.838916+0000","last_peered":"2024-05-08T22:37:57.838916+0000","last_clean":"2024-05-08T22:37:57.838916+0000","last_became_active":"2024-05-08T22:37:57.835104+0000","last_became_peered":"2024-05-08T22:37:57.835104+0000","last_unstale":"2024-05-08T22:37:57.838916+0000","last_undegraded":"2024-05-08T22:37:57.838916+0000","last_fullsized":"2024-05-08T22:37:57.838916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844279+0000","last_change":"2024-05-08T22:37:57.842727+0000","last_active":"2024-05-08T22:37:57.844279+0000","last_peered":"2024-05-08T22:37:57.844279+0000","last_clean":"2024-05-08T22:37:57.844279+0000","last_became_active":"2024-05-08T22:37:57.842350+0000","last_became_peered":"2024-05-08T22:37:57.842350+0000","last_unstale":"2024-05-08T22:37:57.844279+0000","last_undegraded":"2024-05-08T22:37:57.844279+0000","last_fullsized":"2024-05-08T22:37:57.844279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:32.252275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848269+0000","last_change":"2024-05-08T22:37:57.846399+0000","last_active":"2024-05-08T22:37:57.848269+0000","last_peered":"2024-05-08T22:37:57.848269+0000","last_clean":"2024-05-08T22:37:57.848269+0000","last_became_active":"2024-05-08T22:37:57.845308+0000","last_became_peered":"2024-05-08T22:37:57.845308+0000","last_unstale":"2024-05-08T22:37:57.848269+0000","last_undegraded":"2024-05-08T22:37:57.848269+0000","last_fullsized":"2024-05-08T22:37:57.848269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:57.759733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850942+0000","last_change":"2024-05-08T22:37:57.847700+0000","last_active":"2024-05-08T22:37:57.850942+0000","last_peered":"2024-05-08T22:37:57.850942+0000","last_clean":"2024-05-08T22:37:57.850942+0000","last_became_active":"2024-05-08T22:37:57.847333+0000","last_became_peered":"2024-05-08T22:37:57.847333+0000","last_unstale":"2024-05-08T22:37:57.850942+0000","last_undegraded":"2024-05-08T22:37:57.850942+0000","last_fullsized":"2024-05-08T22:37:57.850942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:59.881785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839403+0000","last_change":"2024-05-08T22:37:57.832432+0000","last_active":"2024-05-08T22:37:57.839403+0000","last_peered":"2024-05-08T22:37:57.839403+0000","last_clean":"2024-05-08T22:37:57.839403+0000","last_became_active":"2024-05-08T22:37:57.832048+0000","last_became_peered":"2024-05-08T22:37:57.832048+0000","last_unstale":"2024-05-08T22:37:57.839403+0000","last_undegraded":"2024-05-08T22:37:57.839403+0000","last_fullsized":"2024-05-08T22:37:57.839403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:26.697556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848302+0000","last_change":"2024-05-08T22:37:57.846519+0000","last_active":"2024-05-08T22:37:57.848302+0000","last_peered":"2024-05-08T22:37:57.848302+0000","last_clean":"2024-05-08T22:37:57.848302+0000","last_became_active":"2024-05-08T22:37:57.845361+0000","last_became_peered":"2024-05-08T22:37:57.845361+0000","last_unstale":"2024-05-08T22:37:57.848302+0000","last_undegraded":"2024-05-08T22:37:57.848302+0000","last_fullsized":"2024-05-08T22:37:57.848302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:34.786251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.430120+0000","last_change":"2024-05-08T22:37:57.430120+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.430120+0000","last_undegraded":"2024-05-08T22:37:57.430120+0000","last_fullsized":"2024-05-08T22:37:57.430120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839006+0000","last_change":"2024-05-08T22:37:57.825872+0000","last_active":"2024-05-08T22:37:57.839006+0000","last_peered":"2024-05-08T22:37:57.839006+0000","last_clean":"2024-05-08T22:37:57.839006+0000","last_became_active":"2024-05-08T22:37:57.825591+0000","last_became_peered":"2024-05-08T22:37:57.825591+0000","last_unstale":"2024-05-08T22:37:57.839006+0000","last_undegraded":"2024-05-08T22:37:57.839006+0000","last_fullsized":"2024-05-08T22:37:57.839006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:07.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848675+0000","last_change":"2024-05-08T22:37:57.844985+0000","last_active":"2024-05-08T22:37:57.848675+0000","last_peered":"2024-05-08T22:37:57.848675+0000","last_clean":"2024-05-08T22:37:57.848675+0000","last_became_active":"2024-05-08T22:37:57.844748+0000","last_became_peered":"2024-05-08T22:37:57.844748+0000","last_unstale":"2024-05-08T22:37:57.848675+0000","last_undegraded":"2024-05-08T22:37:57.848675+0000","last_fullsized":"2024-05-08T22:37:57.848675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:02.244593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838791+0000","last_change":"2024-05-08T22:37:57.830338+0000","last_active":"2024-05-08T22:37:57.838791+0000","last_peered":"2024-05-08T22:37:57.838791+0000","last_clean":"2024-05-08T22:37:57.838791+0000","last_became_active":"2024-05-08T22:37:57.830069+0000","last_became_peered":"2024-05-08T22:37:57.830069+0000","last_unstale":"2024-05-08T22:37:57.838791+0000","last_undegraded":"2024-05-08T22:37:57.838791+0000","last_fullsized":"2024-05-08T22:37:57.838791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:53.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841042+0000","last_change":"2024-05-08T22:37:57.835141+0000","last_active":"2024-05-08T22:37:57.841042+0000","last_peered":"2024-05-08T22:37:57.841042+0000","last_clean":"2024-05-08T22:37:57.841042+0000","last_became_active":"2024-05-08T22:37:57.832615+0000","last_became_peered":"2024-05-08T22:37:57.832615+0000","last_unstale":"2024-05-08T22:37:57.841042+0000","last_undegraded":"2024-05-08T22:37:57.841042+0000","last_fullsized":"2024-05-08T22:37:57.841042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:40.282177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850914+0000","last_change":"2024-05-08T22:37:57.847622+0000","last_active":"2024-05-08T22:37:57.850914+0000","last_peered":"2024-05-08T22:37:57.850914+0000","last_clean":"2024-05-08T22:37:57.850914+0000","last_became_active":"2024-05-08T22:37:57.847294+0000","last_became_peered":"2024-05-08T22:37:57.847294+0000","last_unstale":"2024-05-08T22:37:57.850914+0000","last_undegraded":"2024-05-08T22:37:57.850914+0000","last_fullsized":"2024-05-08T22:37:57.850914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:14.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843497+0000","last_change":"2024-05-08T22:37:57.836871+0000","last_active":"2024-05-08T22:37:57.843497+0000","last_peered":"2024-05-08T22:37:57.843497+0000","last_clean":"2024-05-08T22:37:57.843497+0000","last_became_active":"2024-05-08T22:37:57.836626+0000","last_became_peered":"2024-05-08T22:37:57.836626+0000","last_unstale":"2024-05-08T22:37:57.843497+0000","last_undegraded":"2024-05-08T22:37:57.843497+0000","last_fullsized":"2024-05-08T22:37:57.843497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:22.008737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844329+0000","last_change":"2024-05-08T22:37:57.840135+0000","last_active":"2024-05-08T22:37:57.844329+0000","last_peered":"2024-05-08T22:37:57.844329+0000","last_clean":"2024-05-08T22:37:57.844329+0000","last_became_active":"2024-05-08T22:37:57.839806+0000","last_became_peered":"2024-05-08T22:37:57.839806+0000","last_unstale":"2024-05-08T22:37:57.844329+0000","last_undegraded":"2024-05-08T22:37:57.844329+0000","last_fullsized":"2024-05-08T22:37:57.844329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:53.757689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841087+0000","last_change":"2024-05-08T22:37:57.832054+0000","last_active":"2024-05-08T22:37:57.841087+0000","last_peered":"2024-05-08T22:37:57.841087+0000","last_clean":"2024-05-08T22:37:57.841087+0000","last_became_active":"2024-05-08T22:37:57.831311+0000","last_became_peered":"2024-05-08T22:37:57.831311+0000","last_unstale":"2024-05-08T22:37:57.841087+0000","last_undegraded":"2024-05-08T22:37:57.841087+0000","last_fullsized":"2024-05-08T22:37:57.841087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:27.877407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850784+0000","last_change":"2024-05-08T22:37:57.847959+0000","last_active":"2024-05-08T22:37:57.850784+0000","last_peered":"2024-05-08T22:37:57.850784+0000","last_clean":"2024-05-08T22:37:57.850784+0000","last_became_active":"2024-05-08T22:37:57.847531+0000","last_became_peered":"2024-05-08T22:37:57.847531+0000","last_unstale":"2024-05-08T22:37:57.850784+0000","last_undegraded":"2024-05-08T22:37:57.850784+0000","last_fullsized":"2024-05-08T22:37:57.850784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:15.951130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833932+0000","last_change":"2024-05-08T22:37:57.832828+0000","last_active":"2024-05-08T22:37:57.833932+0000","last_peered":"2024-05-08T22:37:57.833932+0000","last_clean":"2024-05-08T22:37:57.833932+0000","last_became_active":"2024-05-08T22:37:57.832544+0000","last_became_peered":"2024-05-08T22:37:57.832544+0000","last_unstale":"2024-05-08T22:37:57.833932+0000","last_undegraded":"2024-05-08T22:37:57.833932+0000","last_fullsized":"2024-05-08T22:37:57.833932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:57.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843557+0000","last_change":"2024-05-08T22:37:57.831045+0000","last_active":"2024-05-08T22:37:57.843557+0000","last_peered":"2024-05-08T22:37:57.843557+0000","last_clean":"2024-05-08T22:37:57.843557+0000","last_became_active":"2024-05-08T22:37:57.830745+0000","last_became_peered":"2024-05-08T22:37:57.830745+0000","last_unstale":"2024-05-08T22:37:57.843557+0000","last_undegraded":"2024-05-08T22:37:57.843557+0000","last_fullsized":"2024-05-08T22:37:57.843557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:31.214851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429934+0000","last_change":"2024-05-08T22:37:57.429934+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429934+0000","last_undegraded":"2024-05-08T22:37:57.429934+0000","last_fullsized":"2024-05-08T22:37:57.429934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846622+0000","last_change":"2024-05-08T22:37:57.844194+0000","last_active":"2024-05-08T22:37:57.846622+0000","last_peered":"2024-05-08T22:37:57.846622+0000","last_clean":"2024-05-08T22:37:57.846622+0000","last_became_active":"2024-05-08T22:37:57.841102+0000","last_became_peered":"2024-05-08T22:37:57.841102+0000","last_unstale":"2024-05-08T22:37:57.846622+0000","last_undegraded":"2024-05-08T22:37:57.846622+0000","last_fullsized":"2024-05-08T22:37:57.846622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:15.481984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424461+0000","last_change":"2024-05-08T22:37:57.424461+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424461+0000","last_undegraded":"2024-05-08T22:37:57.424461+0000","last_fullsized":"2024-05-08T22:37:57.424461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847805+0000","last_change":"2024-05-08T22:37:57.840565+0000","last_active":"2024-05-08T22:37:57.847805+0000","last_peered":"2024-05-08T22:37:57.847805+0000","last_clean":"2024-05-08T22:37:57.847805+0000","last_became_active":"2024-05-08T22:37:57.834465+0000","last_became_peered":"2024-05-08T22:37:57.834465+0000","last_unstale":"2024-05-08T22:37:57.847805+0000","last_undegraded":"2024-05-08T22:37:57.847805+0000","last_fullsized":"2024-05-08T22:37:57.847805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:23.426774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840471+0000","last_change":"2024-05-08T22:37:57.834804+0000","last_active":"2024-05-08T22:37:57.840471+0000","last_peered":"2024-05-08T22:37:57.840471+0000","last_clean":"2024-05-08T22:37:57.840471+0000","last_became_active":"2024-05-08T22:37:57.834268+0000","last_became_peered":"2024-05-08T22:37:57.834268+0000","last_unstale":"2024-05-08T22:37:57.840471+0000","last_undegraded":"2024-05-08T22:37:57.840471+0000","last_fullsized":"2024-05-08T22:37:57.840471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:10.524450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839026+0000","last_change":"2024-05-08T22:37:57.831011+0000","last_active":"2024-05-08T22:37:57.839026+0000","last_peered":"2024-05-08T22:37:57.839026+0000","last_clean":"2024-05-08T22:37:57.839026+0000","last_became_active":"2024-05-08T22:37:57.830662+0000","last_became_peered":"2024-05-08T22:37:57.830662+0000","last_unstale":"2024-05-08T22:37:57.839026+0000","last_undegraded":"2024-05-08T22:37:57.839026+0000","last_fullsized":"2024-05-08T22:37:57.839026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:33.816353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844342+0000","last_change":"2024-05-08T22:37:57.840718+0000","last_active":"2024-05-08T22:37:57.844342+0000","last_peered":"2024-05-08T22:37:57.844342+0000","last_clean":"2024-05-08T22:37:57.844342+0000","last_became_active":"2024-05-08T22:37:57.840400+0000","last_became_peered":"2024-05-08T22:37:57.840400+0000","last_unstale":"2024-05-08T22:37:57.844342+0000","last_undegraded":"2024-05-08T22:37:57.844342+0000","last_fullsized":"2024-05-08T22:37:57.844342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.153666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846441+0000","last_change":"2024-05-08T22:37:57.844109+0000","last_active":"2024-05-08T22:37:57.846441+0000","last_peered":"2024-05-08T22:37:57.846441+0000","last_clean":"2024-05-08T22:37:57.846441+0000","last_became_active":"2024-05-08T22:37:57.843582+0000","last_became_peered":"2024-05-08T22:37:57.843582+0000","last_unstale":"2024-05-08T22:37:57.846441+0000","last_undegraded":"2024-05-08T22:37:57.846441+0000","last_fullsized":"2024-05-08T22:37:57.846441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:58.974111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847790+0000","last_change":"2024-05-08T22:37:57.841963+0000","last_active":"2024-05-08T22:37:57.847790+0000","last_peered":"2024-05-08T22:37:57.847790+0000","last_clean":"2024-05-08T22:37:57.847790+0000","last_became_active":"2024-05-08T22:37:57.841575+0000","last_became_peered":"2024-05-08T22:37:57.841575+0000","last_unstale":"2024-05-08T22:37:57.847790+0000","last_undegraded":"2024-05-08T22:37:57.847790+0000","last_fullsized":"2024-05-08T22:37:57.847790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:33.257259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845819+0000","last_change":"2024-05-08T22:37:57.840274+0000","last_active":"2024-05-08T22:37:57.845819+0000","last_peered":"2024-05-08T22:37:57.845819+0000","last_clean":"2024-05-08T22:37:57.845819+0000","last_became_active":"2024-05-08T22:37:57.840011+0000","last_became_peered":"2024-05-08T22:37:57.840011+0000","last_unstale":"2024-05-08T22:37:57.845819+0000","last_undegraded":"2024-05-08T22:37:57.845819+0000","last_fullsized":"2024-05-08T22:37:57.845819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:45.626643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.832439+0000","last_change":"2024-05-08T22:37:57.827795+0000","last_active":"2024-05-08T22:37:57.832439+0000","last_peered":"2024-05-08T22:37:57.832439+0000","last_clean":"2024-05-08T22:37:57.832439+0000","last_became_active":"2024-05-08T22:37:57.827400+0000","last_became_peered":"2024-05-08T22:37:57.827400+0000","last_unstale":"2024-05-08T22:37:57.832439+0000","last_undegraded":"2024-05-08T22:37:57.832439+0000","last_fullsized":"2024-05-08T22:37:57.832439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:31.579840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848125+0000","last_change":"2024-05-08T22:37:57.844868+0000","last_active":"2024-05-08T22:37:57.848125+0000","last_peered":"2024-05-08T22:37:57.848125+0000","last_clean":"2024-05-08T22:37:57.848125+0000","last_became_active":"2024-05-08T22:37:57.844575+0000","last_became_peered":"2024-05-08T22:37:57.844575+0000","last_unstale":"2024-05-08T22:37:57.848125+0000","last_undegraded":"2024-05-08T22:37:57.848125+0000","last_fullsized":"2024-05-08T22:37:57.848125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850774+0000","last_change":"2024-05-08T22:37:57.847858+0000","last_active":"2024-05-08T22:37:57.850774+0000","last_peered":"2024-05-08T22:37:57.850774+0000","last_clean":"2024-05-08T22:37:57.850774+0000","last_became_active":"2024-05-08T22:37:57.847609+0000","last_became_peered":"2024-05-08T22:37:57.847609+0000","last_unstale":"2024-05-08T22:37:57.850774+0000","last_undegraded":"2024-05-08T22:37:57.850774+0000","last_fullsized":"2024-05-08T22:37:57.850774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:00.575919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844834+0000","last_change":"2024-05-08T22:37:57.446359+0000","last_active":"2024-05-08T22:37:57.844834+0000","last_peered":"2024-05-08T22:37:57.844834+0000","last_clean":"2024-05-08T22:37:57.844834+0000","last_became_active":"2024-05-08T22:37:57.446036+0000","last_became_peered":"2024-05-08T22:37:57.446036+0000","last_unstale":"2024-05-08T22:37:57.844834+0000","last_undegraded":"2024-05-08T22:37:57.844834+0000","last_fullsized":"2024-05-08T22:37:57.844834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:00.419625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847776+0000","last_change":"2024-05-08T22:37:57.842007+0000","last_active":"2024-05-08T22:37:57.847776+0000","last_peered":"2024-05-08T22:37:57.847776+0000","last_clean":"2024-05-08T22:37:57.847776+0000","last_became_active":"2024-05-08T22:37:57.841764+0000","last_became_peered":"2024-05-08T22:37:57.841764+0000","last_unstale":"2024-05-08T22:37:57.847776+0000","last_undegraded":"2024-05-08T22:37:57.847776+0000","last_fullsized":"2024-05-08T22:37:57.847776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:58.771407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842515+0000","last_change":"2024-05-08T22:37:57.440281+0000","last_active":"2024-05-08T22:37:57.842515+0000","last_peered":"2024-05-08T22:37:57.842515+0000","last_clean":"2024-05-08T22:37:57.842515+0000","last_became_active":"2024-05-08T22:37:57.439987+0000","last_became_peered":"2024-05-08T22:37:57.439987+0000","last_unstale":"2024-05-08T22:37:57.842515+0000","last_undegraded":"2024-05-08T22:37:57.842515+0000","last_fullsized":"2024-05-08T22:37:57.842515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:32.804508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.425803+0000","last_change":"2024-05-08T22:37:57.425803+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.425803+0000","last_undegraded":"2024-05-08T22:37:57.425803+0000","last_fullsized":"2024-05-08T22:37:57.425803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846799+0000","last_change":"2024-05-08T22:37:57.448745+0000","last_active":"2024-05-08T22:37:57.846799+0000","last_peered":"2024-05-08T22:37:57.846799+0000","last_clean":"2024-05-08T22:37:57.846799+0000","last_became_active":"2024-05-08T22:37:57.448495+0000","last_became_peered":"2024-05-08T22:37:57.448495+0000","last_unstale":"2024-05-08T22:37:57.846799+0000","last_undegraded":"2024-05-08T22:37:57.846799+0000","last_fullsized":"2024-05-08T22:37:57.846799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:52.343924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429477+0000","last_change":"2024-05-08T22:37:57.429477+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429477+0000","last_undegraded":"2024-05-08T22:37:57.429477+0000","last_fullsized":"2024-05-08T22:37:57.429477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848978+0000","last_change":"2024-05-08T22:37:57.442124+0000","last_active":"2024-05-08T22:37:57.848978+0000","last_peered":"2024-05-08T22:37:57.848978+0000","last_clean":"2024-05-08T22:37:57.848978+0000","last_became_active":"2024-05-08T22:37:57.441844+0000","last_became_peered":"2024-05-08T22:37:57.441844+0000","last_unstale":"2024-05-08T22:37:57.848978+0000","last_undegraded":"2024-05-08T22:37:57.848978+0000","last_fullsized":"2024-05-08T22:37:57.848978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:12.294211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848681+0000","last_change":"2024-05-08T22:37:57.845013+0000","last_active":"2024-05-08T22:37:57.848681+0000","last_peered":"2024-05-08T22:37:57.848681+0000","last_clean":"2024-05-08T22:37:57.848681+0000","last_became_active":"2024-05-08T22:37:57.844721+0000","last_became_peered":"2024-05-08T22:37:57.844721+0000","last_unstale":"2024-05-08T22:37:57.848681+0000","last_undegraded":"2024-05-08T22:37:57.848681+0000","last_fullsized":"2024-05-08T22:37:57.848681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.527397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439079+0000","last_change":"2024-05-08T22:37:57.438281+0000","last_active":"2024-05-08T22:37:57.439079+0000","last_peered":"2024-05-08T22:37:57.439079+0000","last_clean":"2024-05-08T22:37:57.439079+0000","last_became_active":"2024-05-08T22:37:57.438016+0000","last_became_peered":"2024-05-08T22:37:57.438016+0000","last_unstale":"2024-05-08T22:37:57.439079+0000","last_undegraded":"2024-05-08T22:37:57.439079+0000","last_fullsized":"2024-05-08T22:37:57.439079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:22.432349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848718+0000","last_change":"2024-05-08T22:37:57.845395+0000","last_active":"2024-05-08T22:37:57.848718+0000","last_peered":"2024-05-08T22:37:57.848718+0000","last_clean":"2024-05-08T22:37:57.848718+0000","last_became_active":"2024-05-08T22:37:57.845129+0000","last_became_peered":"2024-05-08T22:37:57.845129+0000","last_unstale":"2024-05-08T22:37:57.848718+0000","last_undegraded":"2024-05-08T22:37:57.848718+0000","last_fullsized":"2024-05-08T22:37:57.848718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:26.794424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838895+0000","last_change":"2024-05-08T22:37:57.436502+0000","last_active":"2024-05-08T22:37:57.838895+0000","last_peered":"2024-05-08T22:37:57.838895+0000","last_clean":"2024-05-08T22:37:57.838895+0000","last_became_active":"2024-05-08T22:37:57.436246+0000","last_became_peered":"2024-05-08T22:37:57.436246+0000","last_unstale":"2024-05-08T22:37:57.838895+0000","last_undegraded":"2024-05-08T22:37:57.838895+0000","last_fullsized":"2024-05-08T22:37:57.838895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:07.871185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.426045+0000","last_change":"2024-05-08T22:37:57.426045+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.426045+0000","last_undegraded":"2024-05-08T22:37:57.426045+0000","last_fullsized":"2024-05-08T22:37:57.426045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826353+0000","last_change":"2024-05-08T22:37:57.443165+0000","last_active":"2024-05-08T22:37:57.826353+0000","last_peered":"2024-05-08T22:37:57.826353+0000","last_clean":"2024-05-08T22:37:57.826353+0000","last_became_active":"2024-05-08T22:37:57.442883+0000","last_became_peered":"2024-05-08T22:37:57.442883+0000","last_unstale":"2024-05-08T22:37:57.826353+0000","last_undegraded":"2024-05-08T22:37:57.826353+0000","last_fullsized":"2024-05-08T22:37:57.826353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:43.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849717+0000","last_change":"2024-05-08T22:37:57.840713+0000","last_active":"2024-05-08T22:37:57.849717+0000","last_peered":"2024-05-08T22:37:57.849717+0000","last_clean":"2024-05-08T22:37:57.849717+0000","last_became_active":"2024-05-08T22:37:57.839777+0000","last_became_peered":"2024-05-08T22:37:57.839777+0000","last_unstale":"2024-05-08T22:37:57.849717+0000","last_undegraded":"2024-05-08T22:37:57.849717+0000","last_fullsized":"2024-05-08T22:37:57.849717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:09.133783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.828973+0000","last_change":"2024-05-08T22:37:57.445418+0000","last_active":"2024-05-08T22:37:57.828973+0000","last_peered":"2024-05-08T22:37:57.828973+0000","last_clean":"2024-05-08T22:37:57.828973+0000","last_became_active":"2024-05-08T22:37:57.445005+0000","last_became_peered":"2024-05-08T22:37:57.445005+0000","last_unstale":"2024-05-08T22:37:57.828973+0000","last_undegraded":"2024-05-08T22:37:57.828973+0000","last_fullsized":"2024-05-08T22:37:57.828973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:01.056718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848795+0000","last_change":"2024-05-08T22:37:57.846487+0000","last_active":"2024-05-08T22:37:57.848795+0000","last_peered":"2024-05-08T22:37:57.848795+0000","last_clean":"2024-05-08T22:37:57.848795+0000","last_became_active":"2024-05-08T22:37:57.845715+0000","last_became_peered":"2024-05-08T22:37:57.845715+0000","last_unstale":"2024-05-08T22:37:57.848795+0000","last_undegraded":"2024-05-08T22:37:57.848795+0000","last_fullsized":"2024-05-08T22:37:57.848795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:56.850979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829513+0000","last_change":"2024-05-08T22:37:57.442931+0000","last_active":"2024-05-08T22:37:57.829513+0000","last_peered":"2024-05-08T22:37:57.829513+0000","last_clean":"2024-05-08T22:37:57.829513+0000","last_became_active":"2024-05-08T22:37:57.441198+0000","last_became_peered":"2024-05-08T22:37:57.441198+0000","last_unstale":"2024-05-08T22:37:57.829513+0000","last_undegraded":"2024-05-08T22:37:57.829513+0000","last_fullsized":"2024-05-08T22:37:57.829513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:35.216047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844949+0000","last_change":"2024-05-08T22:37:57.844158+0000","last_active":"2024-05-08T22:37:57.844949+0000","last_peered":"2024-05-08T22:37:57.844949+0000","last_clean":"2024-05-08T22:37:57.844949+0000","last_became_active":"2024-05-08T22:37:57.843855+0000","last_became_peered":"2024-05-08T22:37:57.843855+0000","last_unstale":"2024-05-08T22:37:57.844949+0000","last_undegraded":"2024-05-08T22:37:57.844949+0000","last_fullsized":"2024-05-08T22:37:57.844949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:36.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822311+0000","last_change":"2024-05-08T22:37:57.439889+0000","last_active":"2024-05-08T22:37:57.822311+0000","last_peered":"2024-05-08T22:37:57.822311+0000","last_clean":"2024-05-08T22:37:57.822311+0000","last_became_active":"2024-05-08T22:37:57.439633+0000","last_became_peered":"2024-05-08T22:37:57.439633+0000","last_unstale":"2024-05-08T22:37:57.822311+0000","last_undegraded":"2024-05-08T22:37:57.822311+0000","last_fullsized":"2024-05-08T22:37:57.822311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:01.221867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839271+0000","last_change":"2024-05-08T22:37:57.833440+0000","last_active":"2024-05-08T22:37:57.839271+0000","last_peered":"2024-05-08T22:37:57.839271+0000","last_clean":"2024-05-08T22:37:57.839271+0000","last_became_active":"2024-05-08T22:37:57.833212+0000","last_became_peered":"2024-05-08T22:37:57.833212+0000","last_unstale":"2024-05-08T22:37:57.839271+0000","last_undegraded":"2024-05-08T22:37:57.839271+0000","last_fullsized":"2024-05-08T22:37:57.839271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:08.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846625+0000","last_change":"2024-05-08T22:37:57.444957+0000","last_active":"2024-05-08T22:37:57.846625+0000","last_peered":"2024-05-08T22:37:57.846625+0000","last_clean":"2024-05-08T22:37:57.846625+0000","last_became_active":"2024-05-08T22:37:57.444484+0000","last_became_peered":"2024-05-08T22:37:57.444484+0000","last_unstale":"2024-05-08T22:37:57.846625+0000","last_undegraded":"2024-05-08T22:37:57.846625+0000","last_fullsized":"2024-05-08T22:37:57.846625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:46.229790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836275+0000","last_change":"2024-05-08T22:37:57.830761+0000","last_active":"2024-05-08T22:37:57.836275+0000","last_peered":"2024-05-08T22:37:57.836275+0000","last_clean":"2024-05-08T22:37:57.836275+0000","last_became_active":"2024-05-08T22:37:57.830491+0000","last_became_peered":"2024-05-08T22:37:57.830491+0000","last_unstale":"2024-05-08T22:37:57.836275+0000","last_undegraded":"2024-05-08T22:37:57.836275+0000","last_fullsized":"2024-05-08T22:37:57.836275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:12.683143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849871+0000","last_change":"2024-05-08T22:37:57.439248+0000","last_active":"2024-05-08T22:37:57.849871+0000","last_peered":"2024-05-08T22:37:57.849871+0000","last_clean":"2024-05-08T22:37:57.849871+0000","last_became_active":"2024-05-08T22:37:57.438927+0000","last_became_peered":"2024-05-08T22:37:57.438927+0000","last_unstale":"2024-05-08T22:37:57.849871+0000","last_undegraded":"2024-05-08T22:37:57.849871+0000","last_fullsized":"2024-05-08T22:37:57.849871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:11.645499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838819+0000","last_change":"2024-05-08T22:37:57.835299+0000","last_active":"2024-05-08T22:37:57.838819+0000","last_peered":"2024-05-08T22:37:57.838819+0000","last_clean":"2024-05-08T22:37:57.838819+0000","last_became_active":"2024-05-08T22:37:57.834959+0000","last_became_peered":"2024-05-08T22:37:57.834959+0000","last_unstale":"2024-05-08T22:37:57.838819+0000","last_undegraded":"2024-05-08T22:37:57.838819+0000","last_fullsized":"2024-05-08T22:37:57.838819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:19.550971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848959+0000","last_change":"2024-05-08T22:37:57.436168+0000","last_active":"2024-05-08T22:37:57.848959+0000","last_peered":"2024-05-08T22:37:57.848959+0000","last_clean":"2024-05-08T22:37:57.848959+0000","last_became_active":"2024-05-08T22:37:57.435911+0000","last_became_peered":"2024-05-08T22:37:57.435911+0000","last_unstale":"2024-05-08T22:37:57.848959+0000","last_undegraded":"2024-05-08T22:37:57.848959+0000","last_fullsized":"2024-05-08T22:37:57.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:39.531241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839382+0000","last_change":"2024-05-08T22:37:57.834557+0000","last_active":"2024-05-08T22:37:57.839382+0000","last_peered":"2024-05-08T22:37:57.839382+0000","last_clean":"2024-05-08T22:37:57.839382+0000","last_became_active":"2024-05-08T22:37:57.834336+0000","last_became_peered":"2024-05-08T22:37:57.834336+0000","last_unstale":"2024-05-08T22:37:57.839382+0000","last_undegraded":"2024-05-08T22:37:57.839382+0000","last_fullsized":"2024-05-08T22:37:57.839382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:07.977190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826416+0000","last_change":"2024-05-08T22:37:57.442582+0000","last_active":"2024-05-08T22:37:57.826416+0000","last_peered":"2024-05-08T22:37:57.826416+0000","last_clean":"2024-05-08T22:37:57.826416+0000","last_became_active":"2024-05-08T22:37:57.440956+0000","last_became_peered":"2024-05-08T22:37:57.440956+0000","last_unstale":"2024-05-08T22:37:57.826416+0000","last_undegraded":"2024-05-08T22:37:57.826416+0000","last_fullsized":"2024-05-08T22:37:57.826416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:40.041454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849792+0000","last_change":"2024-05-08T22:37:57.846329+0000","last_active":"2024-05-08T22:37:57.849792+0000","last_peered":"2024-05-08T22:37:57.849792+0000","last_clean":"2024-05-08T22:37:57.849792+0000","last_became_active":"2024-05-08T22:37:57.846056+0000","last_became_peered":"2024-05-08T22:37:57.846056+0000","last_unstale":"2024-05-08T22:37:57.849792+0000","last_undegraded":"2024-05-08T22:37:57.849792+0000","last_fullsized":"2024-05-08T22:37:57.849792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829433+0000","last_change":"2024-05-08T22:37:57.448219+0000","last_active":"2024-05-08T22:37:57.829433+0000","last_peered":"2024-05-08T22:37:57.829433+0000","last_clean":"2024-05-08T22:37:57.829433+0000","last_became_active":"2024-05-08T22:37:57.447923+0000","last_became_peered":"2024-05-08T22:37:57.447923+0000","last_unstale":"2024-05-08T22:37:57.829433+0000","last_undegraded":"2024-05-08T22:37:57.829433+0000","last_fullsized":"2024-05-08T22:37:57.829433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:30.907553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839082+0000","last_change":"2024-05-08T22:37:57.829299+0000","last_active":"2024-05-08T22:37:57.839082+0000","last_peered":"2024-05-08T22:37:57.839082+0000","last_clean":"2024-05-08T22:37:57.839082+0000","last_became_active":"2024-05-08T22:37:57.828970+0000","last_became_peered":"2024-05-08T22:37:57.828970+0000","last_unstale":"2024-05-08T22:37:57.839082+0000","last_undegraded":"2024-05-08T22:37:57.839082+0000","last_fullsized":"2024-05-08T22:37:57.839082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:01.133413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.851908+0000","last_change":"2024-05-08T22:37:57.444179+0000","last_active":"2024-05-08T22:37:57.851908+0000","last_peered":"2024-05-08T22:37:57.851908+0000","last_clean":"2024-05-08T22:37:57.851908+0000","last_became_active":"2024-05-08T22:37:57.443673+0000","last_became_peered":"2024-05-08T22:37:57.443673+0000","last_unstale":"2024-05-08T22:37:57.851908+0000","last_undegraded":"2024-05-08T22:37:57.851908+0000","last_fullsized":"2024-05-08T22:37:57.851908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:53.287958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845235+0000","last_change":"2024-05-08T22:37:57.841025+0000","last_active":"2024-05-08T22:37:57.845235+0000","last_peered":"2024-05-08T22:37:57.845235+0000","last_clean":"2024-05-08T22:37:57.845235+0000","last_became_active":"2024-05-08T22:37:57.840591+0000","last_became_peered":"2024-05-08T22:37:57.840591+0000","last_unstale":"2024-05-08T22:37:57.845235+0000","last_undegraded":"2024-05-08T22:37:57.845235+0000","last_fullsized":"2024-05-08T22:37:57.845235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.910062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842909+0000","last_change":"2024-05-08T22:37:57.440960+0000","last_active":"2024-05-08T22:37:57.842909+0000","last_peered":"2024-05-08T22:37:57.842909+0000","last_clean":"2024-05-08T22:37:57.842909+0000","last_became_active":"2024-05-08T22:37:57.440504+0000","last_became_peered":"2024-05-08T22:37:57.440504+0000","last_unstale":"2024-05-08T22:37:57.842909+0000","last_undegraded":"2024-05-08T22:37:57.842909+0000","last_fullsized":"2024-05-08T22:37:57.842909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:06.401155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841075+0000","last_change":"2024-05-08T22:37:57.831846+0000","last_active":"2024-05-08T22:37:57.841075+0000","last_peered":"2024-05-08T22:37:57.841075+0000","last_clean":"2024-05-08T22:37:57.841075+0000","last_became_active":"2024-05-08T22:37:57.829692+0000","last_became_peered":"2024-05-08T22:37:57.829692+0000","last_unstale":"2024-05-08T22:37:57.841075+0000","last_undegraded":"2024-05-08T22:37:57.841075+0000","last_fullsized":"2024-05-08T22:37:57.841075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.674154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439143+0000","last_change":"2024-05-08T22:37:57.437527+0000","last_active":"2024-05-08T22:37:57.439143+0000","last_peered":"2024-05-08T22:37:57.439143+0000","last_clean":"2024-05-08T22:37:57.439143+0000","last_became_active":"2024-05-08T22:37:57.437261+0000","last_became_peered":"2024-05-08T22:37:57.437261+0000","last_unstale":"2024-05-08T22:37:57.439143+0000","last_undegraded":"2024-05-08T22:37:57.439143+0000","last_fullsized":"2024-05-08T22:37:57.439143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:19.394053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844277+0000","last_change":"2024-05-08T22:37:57.832513+0000","last_active":"2024-05-08T22:37:57.844277+0000","last_peered":"2024-05-08T22:37:57.844277+0000","last_clean":"2024-05-08T22:37:57.844277+0000","last_became_active":"2024-05-08T22:37:57.832275+0000","last_became_peered":"2024-05-08T22:37:57.832275+0000","last_unstale":"2024-05-08T22:37:57.844277+0000","last_undegraded":"2024-05-08T22:37:57.844277+0000","last_fullsized":"2024-05-08T22:37:57.844277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:08.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847277+0000","last_change":"2024-05-08T22:37:57.444668+0000","last_active":"2024-05-08T22:37:57.847277+0000","last_peered":"2024-05-08T22:37:57.847277+0000","last_clean":"2024-05-08T22:37:57.847277+0000","last_became_active":"2024-05-08T22:37:57.441598+0000","last_became_peered":"2024-05-08T22:37:57.441598+0000","last_unstale":"2024-05-08T22:37:57.847277+0000","last_undegraded":"2024-05-08T22:37:57.847277+0000","last_fullsized":"2024-05-08T22:37:57.847277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:33.716602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833929+0000","last_change":"2024-05-08T22:37:57.832325+0000","last_active":"2024-05-08T22:37:57.833929+0000","last_peered":"2024-05-08T22:37:57.833929+0000","last_clean":"2024-05-08T22:37:57.833929+0000","last_became_active":"2024-05-08T22:37:57.832067+0000","last_became_peered":"2024-05-08T22:37:57.832067+0000","last_unstale":"2024-05-08T22:37:57.833929+0000","last_undegraded":"2024-05-08T22:37:57.833929+0000","last_fullsized":"2024-05-08T22:37:57.833929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:42.187631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817518+0000","last_change":"2024-05-08T22:37:57.442255+0000","last_active":"2024-05-08T22:37:57.817518+0000","last_peered":"2024-05-08T22:37:57.817518+0000","last_clean":"2024-05-08T22:37:57.817518+0000","last_became_active":"2024-05-08T22:37:57.441960+0000","last_became_peered":"2024-05-08T22:37:57.441960+0000","last_unstale":"2024-05-08T22:37:57.817518+0000","last_undegraded":"2024-05-08T22:37:57.817518+0000","last_fullsized":"2024-05-08T22:37:57.817518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:59.020957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848551+0000","last_change":"2024-05-08T22:37:57.842118+0000","last_active":"2024-05-08T22:37:57.848551+0000","last_peered":"2024-05-08T22:37:57.848551+0000","last_clean":"2024-05-08T22:37:57.848551+0000","last_became_active":"2024-05-08T22:37:57.841826+0000","last_became_peered":"2024-05-08T22:37:57.841826+0000","last_unstale":"2024-05-08T22:37:57.848551+0000","last_undegraded":"2024-05-08T22:37:57.848551+0000","last_fullsized":"2024-05-08T22:37:57.848551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:55.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840125+0000","last_change":"2024-05-08T22:37:57.439634+0000","last_active":"2024-05-08T22:37:57.840125+0000","last_peered":"2024-05-08T22:37:57.840125+0000","last_clean":"2024-05-08T22:37:57.840125+0000","last_became_active":"2024-05-08T22:37:57.439387+0000","last_became_peered":"2024-05-08T22:37:57.439387+0000","last_unstale":"2024-05-08T22:37:57.840125+0000","last_undegraded":"2024-05-08T22:37:57.840125+0000","last_fullsized":"2024-05-08T22:37:57.840125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:49.158578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848777+0000","last_change":"2024-05-08T22:37:57.846904+0000","last_active":"2024-05-08T22:37:57.848777+0000","last_peered":"2024-05-08T22:37:57.848777+0000","last_clean":"2024-05-08T22:37:57.848777+0000","last_became_active":"2024-05-08T22:37:57.846651+0000","last_became_peered":"2024-05-08T22:37:57.846651+0000","last_unstale":"2024-05-08T22:37:57.848777+0000","last_undegraded":"2024-05-08T22:37:57.848777+0000","last_fullsized":"2024-05-08T22:37:57.848777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:05.340141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.821917+0000","last_change":"2024-05-08T22:37:57.443724+0000","last_active":"2024-05-08T22:37:57.821917+0000","last_peered":"2024-05-08T22:37:57.821917+0000","last_clean":"2024-05-08T22:37:57.821917+0000","last_became_active":"2024-05-08T22:37:57.443460+0000","last_became_peered":"2024-05-08T22:37:57.443460+0000","last_unstale":"2024-05-08T22:37:57.821917+0000","last_undegraded":"2024-05-08T22:37:57.821917+0000","last_fullsized":"2024-05-08T22:37:57.821917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:08.118921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.837256+0000","last_change":"2024-05-08T22:37:57.826029+0000","last_active":"2024-05-08T22:37:57.837256+0000","last_peered":"2024-05-08T22:37:57.837256+0000","last_clean":"2024-05-08T22:37:57.837256+0000","last_became_active":"2024-05-08T22:37:57.825743+0000","last_became_peered":"2024-05-08T22:37:57.825743+0000","last_unstale":"2024-05-08T22:37:57.837256+0000","last_undegraded":"2024-05-08T22:37:57.837256+0000","last_fullsized":"2024-05-08T22:37:57.837256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.587376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822152+0000","last_change":"2024-05-08T22:37:57.444844+0000","last_active":"2024-05-08T22:37:57.822152+0000","last_peered":"2024-05-08T22:37:57.822152+0000","last_clean":"2024-05-08T22:37:57.822152+0000","last_became_active":"2024-05-08T22:37:57.444488+0000","last_became_peered":"2024-05-08T22:37:57.444488+0000","last_unstale":"2024-05-08T22:37:57.822152+0000","last_undegraded":"2024-05-08T22:37:57.822152+0000","last_fullsized":"2024-05-08T22:37:57.822152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:36.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424456+0000","last_change":"2024-05-08T22:37:57.424456+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424456+0000","last_undegraded":"2024-05-08T22:37:57.424456+0000","last_fullsized":"2024-05-08T22:37:57.424456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838790+0000","last_change":"2024-05-08T22:37:57.438922+0000","last_active":"2024-05-08T22:37:57.838790+0000","last_peered":"2024-05-08T22:37:57.838790+0000","last_clean":"2024-05-08T22:37:57.838790+0000","last_became_active":"2024-05-08T22:37:57.438655+0000","last_became_peered":"2024-05-08T22:37:57.438655+0000","last_unstale":"2024-05-08T22:37:57.838790+0000","last_undegraded":"2024-05-08T22:37:57.838790+0000","last_fullsized":"2024-05-08T22:37:57.838790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:09:11.390285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838438+0000","last_change":"2024-05-08T22:37:57.828420+0000","last_active":"2024-05-08T22:37:57.838438+0000","last_peered":"2024-05-08T22:37:57.838438+0000","last_clean":"2024-05-08T22:37:57.838438+0000","last_became_active":"2024-05-08T22:37:57.828132+0000","last_became_peered":"2024-05-08T22:37:57.828132+0000","last_unstale":"2024-05-08T22:37:57.838438+0000","last_undegraded":"2024-05-08T22:37:57.838438+0000","last_fullsized":"2024-05-08T22:37:57.838438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:23.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.364530+0000","last_change":"2024-05-08T22:37:57.447723+0000","last_active":"2024-05-08T22:37:58.364530+0000","last_peered":"2024-05-08T22:37:58.364530+0000","last_clean":"2024-05-08T22:37:58.364530+0000","last_became_active":"2024-05-08T22:37:57.447478+0000","last_became_peered":"2024-05-08T22:37:57.447478+0000","last_unstale":"2024-05-08T22:37:58.364530+0000","last_undegraded":"2024-05-08T22:37:58.364530+0000","last_fullsized":"2024-05-08T22:37:58.364530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:09.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424687+0000","last_change":"2024-05-08T22:37:57.424687+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424687+0000","last_undegraded":"2024-05-08T22:37:57.424687+0000","last_fullsized":"2024-05-08T22:37:57.424687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.827351+0000","last_change":"2024-05-08T22:37:57.437575+0000","last_active":"2024-05-08T22:37:57.827351+0000","last_peered":"2024-05-08T22:37:57.827351+0000","last_clean":"2024-05-08T22:37:57.827351+0000","last_became_active":"2024-05-08T22:37:57.437308+0000","last_became_peered":"2024-05-08T22:37:57.437308+0000","last_unstale":"2024-05-08T22:37:57.827351+0000","last_undegraded":"2024-05-08T22:37:57.827351+0000","last_fullsized":"2024-05-08T22:37:57.827351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:33.154187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848732+0000","last_change":"2024-05-08T22:37:57.845583+0000","last_active":"2024-05-08T22:37:57.848732+0000","last_peered":"2024-05-08T22:37:57.848732+0000","last_clean":"2024-05-08T22:37:57.848732+0000","last_became_active":"2024-05-08T22:37:57.845321+0000","last_became_peered":"2024-05-08T22:37:57.845321+0000","last_unstale":"2024-05-08T22:37:57.848732+0000","last_undegraded":"2024-05-08T22:37:57.848732+0000","last_fullsized":"2024-05-08T22:37:57.848732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:27.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825845+0000","last_change":"2024-05-08T22:37:57.440921+0000","last_active":"2024-05-08T22:37:57.825845+0000","last_peered":"2024-05-08T22:37:57.825845+0000","last_clean":"2024-05-08T22:37:57.825845+0000","last_became_active":"2024-05-08T22:37:57.440652+0000","last_became_peered":"2024-05-08T22:37:57.440652+0000","last_unstale":"2024-05-08T22:37:57.825845+0000","last_undegraded":"2024-05-08T22:37:57.825845+0000","last_fullsized":"2024-05-08T22:37:57.825845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:02.665256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838442+0000","last_change":"2024-05-08T22:37:57.828599+0000","last_active":"2024-05-08T22:37:57.838442+0000","last_peered":"2024-05-08T22:37:57.838442+0000","last_clean":"2024-05-08T22:37:57.838442+0000","last_became_active":"2024-05-08T22:37:57.828254+0000","last_became_peered":"2024-05-08T22:37:57.828254+0000","last_unstale":"2024-05-08T22:37:57.838442+0000","last_undegraded":"2024-05-08T22:37:57.838442+0000","last_fullsized":"2024-05-08T22:37:57.838442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:30.500016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822000+0000","last_change":"2024-05-08T22:37:57.442291+0000","last_active":"2024-05-08T22:37:57.822000+0000","last_peered":"2024-05-08T22:37:57.822000+0000","last_clean":"2024-05-08T22:37:57.822000+0000","last_became_active":"2024-05-08T22:37:57.441935+0000","last_became_peered":"2024-05-08T22:37:57.441935+0000","last_unstale":"2024-05-08T22:37:57.822000+0000","last_undegraded":"2024-05-08T22:37:57.822000+0000","last_fullsized":"2024-05-08T22:37:57.822000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:55.703307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839361+0000","last_change":"2024-05-08T22:37:57.832660+0000","last_active":"2024-05-08T22:37:57.839361+0000","last_peered":"2024-05-08T22:37:57.839361+0000","last_clean":"2024-05-08T22:37:57.839361+0000","last_became_active":"2024-05-08T22:37:57.832291+0000","last_became_peered":"2024-05-08T22:37:57.832291+0000","last_unstale":"2024-05-08T22:37:57.839361+0000","last_undegraded":"2024-05-08T22:37:57.839361+0000","last_fullsized":"2024-05-08T22:37:57.839361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:40.277738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.830600+0000","last_change":"2024-05-08T22:37:57.448069+0000","last_active":"2024-05-08T22:37:57.830600+0000","last_peered":"2024-05-08T22:37:57.830600+0000","last_clean":"2024-05-08T22:37:57.830600+0000","last_became_active":"2024-05-08T22:37:57.446523+0000","last_became_peered":"2024-05-08T22:37:57.446523+0000","last_unstale":"2024-05-08T22:37:57.830600+0000","last_undegraded":"2024-05-08T22:37:57.830600+0000","last_fullsized":"2024-05-08T22:37:57.830600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:44.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836038+0000","last_change":"2024-05-08T22:37:57.830871+0000","last_active":"2024-05-08T22:37:57.836038+0000","last_peered":"2024-05-08T22:37:57.836038+0000","last_clean":"2024-05-08T22:37:57.836038+0000","last_became_active":"2024-05-08T22:37:57.830594+0000","last_became_peered":"2024-05-08T22:37:57.830594+0000","last_unstale":"2024-05-08T22:37:57.836038+0000","last_undegraded":"2024-05-08T22:37:57.836038+0000","last_fullsized":"2024-05-08T22:37:57.836038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:02.234727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.823044+0000","last_change":"2024-05-08T22:37:57.438169+0000","last_active":"2024-05-08T22:37:57.823044+0000","last_peered":"2024-05-08T22:37:57.823044+0000","last_clean":"2024-05-08T22:37:57.823044+0000","last_became_active":"2024-05-08T22:37:57.437880+0000","last_became_peered":"2024-05-08T22:37:57.437880+0000","last_unstale":"2024-05-08T22:37:57.823044+0000","last_undegraded":"2024-05-08T22:37:57.823044+0000","last_fullsized":"2024-05-08T22:37:57.823044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:58.133761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850749+0000","last_change":"2024-05-08T22:37:57.846104+0000","last_active":"2024-05-08T22:37:57.850749+0000","last_peered":"2024-05-08T22:37:57.850749+0000","last_clean":"2024-05-08T22:37:57.850749+0000","last_became_active":"2024-05-08T22:37:57.845848+0000","last_became_peered":"2024-05-08T22:37:57.845848+0000","last_unstale":"2024-05-08T22:37:57.850749+0000","last_undegraded":"2024-05-08T22:37:57.850749+0000","last_fullsized":"2024-05-08T22:37:57.850749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:06:32.854193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817725+0000","last_change":"2024-05-08T22:37:57.437069+0000","last_active":"2024-05-08T22:37:57.817725+0000","last_peered":"2024-05-08T22:37:57.817725+0000","last_clean":"2024-05-08T22:37:57.817725+0000","last_became_active":"2024-05-08T22:37:57.436461+0000","last_became_peered":"2024-05-08T22:37:57.436461+0000","last_unstale":"2024-05-08T22:37:57.817725+0000","last_undegraded":"2024-05-08T22:37:57.817725+0000","last_fullsized":"2024-05-08T22:37:57.817725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:31.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845596+0000","last_change":"2024-05-08T22:37:57.843298+0000","last_active":"2024-05-08T22:37:57.845596+0000","last_peered":"2024-05-08T22:37:57.845596+0000","last_clean":"2024-05-08T22:37:57.845596+0000","last_became_active":"2024-05-08T22:37:57.842988+0000","last_became_peered":"2024-05-08T22:37:57.842988+0000","last_unstale":"2024-05-08T22:37:57.845596+0000","last_undegraded":"2024-05-08T22:37:57.845596+0000","last_fullsized":"2024-05-08T22:37:57.845596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:17.302357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825542+0000","last_change":"2024-05-08T22:37:57.444385+0000","last_active":"2024-05-08T22:37:57.825542+0000","last_peered":"2024-05-08T22:37:57.825542+0000","last_clean":"2024-05-08T22:37:57.825542+0000","last_became_active":"2024-05-08T22:37:57.444124+0000","last_became_peered":"2024-05-08T22:37:57.444124+0000","last_unstale":"2024-05-08T22:37:57.825542+0000","last_undegraded":"2024-05-08T22:37:57.825542+0000","last_fullsized":"2024-05-08T22:37:57.825542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:35.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.429399+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:37:57.429399+0000","last_peered":"2024-05-08T22:37:57.429399+0000","last_clean":"2024-05-08T22:37:57.429399+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:37:57.429399+0000","last_undegraded":"2024-05-08T22:37:57.429399+0000","last_fullsized":"2024-05-08T22:37:57.429399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":999424,"data_stored":451081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.309}]},{"osd":1,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.47599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89100000000000001,"5min":1.101,"15min":1.101},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.627}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.018,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.57899999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1140000000000001,"5min":1.306,"15min":1.306},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2310000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.274,"5min":1.339,"15min":1.339},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.0089999999999999,"15min":1.0269999999999999},"min":{"1min":0.76000000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2749999999999999,"5min":1.476,"15min":1.476},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98199999999999998,"15min":0.97199999999999998},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2370000000000001,"5min":1.446,"15min":1.446},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.753,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.341,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.105},"min":{"1min":0.82199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.298,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.056},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.036,"15min":1.034},"min":{"1min":0.81000000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4570000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.88400000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":450611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":1.04,"15min":1.04},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.58399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2130000000000001,"5min":1.583,"15min":1.583},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2050000000000001,"5min":1.431,"15min":1.431},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.81399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2470000000000001,"5min":1.508,"15min":1.508},"last":0.747}]},{"osd":5,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.028,"15min":1.083},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.397,"5min":1.726,"15min":1.726},"last":0.96999999999999997}]},{"osd":6,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.77200000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.446,"5min":1.659,"15min":1.659},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.071},"min":{"1min":0.85399999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3129999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0569999999999999},"min":{"1min":0.81699999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.409,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85599999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1593344,"data_stored":1040889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85499999999999998,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2749999999999999,"5min":1.282,"15min":1.282},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.84099999999999997,"15min":0.85999999999999999},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.121}]},{"osd":1,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91800000000000004,"15min":0.91000000000000003},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.099,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1140000000000001,"5min":1.419,"15min":1.419},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95099999999999996,"15min":0.96199999999999997},"min":{"1min":0.69199999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.337,"15min":1.337},"last":1.022}]},{"osd":3,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.79400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5269999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.72099999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.177},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.89300000000000002},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.103,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95999999999999996,"15min":0.98199999999999998},"min":{"1min":0.67000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.482,"15min":1.482},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.85399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.264}]},{"osd":6,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.76500000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4179999999999999,"5min":1.599,"15min":1.599},"last":1.0800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":1007616,"data_stored":452063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.141,"15min":1.141},"last":0.997}]},{"osd":1,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3300000000000001,"5min":1.758,"15min":1.758},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91800000000000004,"15min":0.93000000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1850000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.115}]},{"osd":2,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.92900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.397,"5min":1.419,"15min":1.419},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.034,"15min":1.0960000000000001},"min":{"1min":0.68000000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2110000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.99099999999999999},"min":{"1min":0.625,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2869999999999999,"5min":1.538,"15min":1.538},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":1.0469999999999999},"min":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.149,"5min":1.458,"15min":1.458},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.28,"5min":1.681,"15min":1.681},"last":1.194},{"interface":"front","average":{"1min":0.998,"5min":1.0840000000000001,"15min":1.1399999999999999},"min":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2050000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.014,"15min":1.085},"min":{"1min":0.71899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0780000000000001,"5min":1.835,"15min":1.835},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.179,"15min":1.2350000000000001},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3300000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1559999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":999424,"data_stored":450533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82399999999999995,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.145,"5min":1.52,"15min":1.52},"last":0.79000000000000004}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.016,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.266,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90400000000000003,"15min":0.88},"min":{"1min":0.71899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.83999999999999997},"min":{"1min":0.67600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.204}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":0.96499999999999997},"min":{"1min":0.71599999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.70099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.016,"15min":0.98599999999999999},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96799999999999997,"15min":0.91600000000000004},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2589999999999999,"5min":1.605,"15min":1.605},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.069,"15min":1.044},"min":{"1min":0.89200000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3080000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.3340000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.0880000000000001},"min":{"1min":0.98299999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.411,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.133,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.91500000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3089999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2909999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":970752,"data_stored":436809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.113,"5min":1.28,"15min":1.28},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.294,"15min":1.294},"last":0.72299999999999998}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.871},"min":{"1min":0.65800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.163,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90700000000000003,"15min":0.88600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.173,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0329999999999999,"5min":1.29,"15min":1.29},"last":1.224},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91900000000000004,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.135,"5min":1.637,"15min":1.637},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1619999999999999,"5min":2.601,"15min":2.601},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.032,"15min":1.105},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.998}]},{"osd":5,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.0349999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.121,"5min":1.792,"15min":1.792},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.026},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.137,"5min":1.575,"15min":1.575},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0489999999999999,"15min":1.0900000000000001},"min":{"1min":0.69399999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1100000000000001,"5min":1.831,"15min":1.831},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.093,"15min":1.1020000000000001},"min":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2210000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0609999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.071,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.23,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.159,"15min":1.177},"min":{"1min":0.878,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3,"5min":1.944,"15min":1.944},"last":1.085}]}]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":443404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.133,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":1.008,"15min":1.008},"last":0.879},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.58799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.333,"5min":1.363,"15min":1.363},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.006,"15min":1.028},"min":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.27,"5min":1.282,"15min":1.282},"last":1.2609999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.407,"5min":1.484,"15min":1.484},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.3}]},{"osd":6,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0600000000000001},"min":{"1min":0.84499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0700000000000001,"15min":1.0840000000000001},"min":{"1min":0.871,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4450000000000001,"5min":1.673,"15min":1.673},"last":1.101}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.119,"15min":1.1299999999999999},"min":{"1min":0.878,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.056}]}]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343276,"statfs":{"total":96636764160,"available":96607514624,"internally_reserved":0,"allocated":1576960,"data_stored":1033772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.617,"15min":1.617},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":1.446,"15min":1.446},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0129999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3089999999999999,"5min":1.714,"15min":1.714},"last":1.609},{"interface":"front","average":{"1min":0.755,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2569999999999999,"5min":1.327,"15min":1.327},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2709999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":4.391},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.73499999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3,"5min":1.669,"15min":1.669},"last":2.0590000000000002}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2210000000000001,"5min":1.474,"15min":1.474},"last":1.0760000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.96499999999999997},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2869999999999999,"5min":1.397,"15min":1.397},"last":1.8660000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":1.0469999999999999},"min":{"1min":0.78600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.177,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":4.4859999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0649999999999999,"15min":1.089},"min":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.234,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.532},{"interface":"front","average":{"1min":1.018,"5min":1.008,"15min":1.022},"min":{"1min":0.77500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.304}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:38:02.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:38:02.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 Processor -- start 2024-05-08T22:38:02.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- start start 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f524415ef80 0x7f5244161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5244164db0 0x7f524406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5244165d50 con 0x7f524406dca0 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524406e130 con 0x7f524415ef80 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524406ad70 con 0x7f5244164db0 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f524415ef80 0x7f5244161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f524415ef80 0x7f5244161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58780/0 (socket says 172.21.15.179:58780) 2024-05-08T22:38:02.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 -- 172.21.15.179:0/489801329 learned_addr learned my addr 172.21.15.179:0/489801329 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f52492f9700 1 --1- 172.21.15.179:0/489801329 >> v1:172.21.15.179:6789/0 conn(0x7f524406dca0 0x7f52440b2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57052/0 (socket says 172.21.15.179:57052) 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5243fff700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5244164db0 0x7f524406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 -- 172.21.15.179:0/489801329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5244164db0 msgr2=0x7f524406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5244164db0 0x7f524406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 -- 172.21.15.179:0/489801329 >> v1:172.21.15.179:6789/0 conn(0x7f524406dca0 legacy=0x7f52440b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 -- 172.21.15.179:0/489801329 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244165230 con 0x7f524415ef80 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5248af8700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f5244161790 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f522c009910 tx=0x7f522c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c6a95acc19ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f52437fe700 1 -- 172.21.15.179:0/489801329 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00ba60 con 0x7f524415ef80 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f52437fe700 1 -- 172.21.15.179:0/489801329 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00bbc0 con 0x7f524415ef80 2024-05-08T22:38:02.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f52437fe700 1 -- 172.21.15.179:0/489801329 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c009e60 con 0x7f524415ef80 2024-05-08T22:38:02.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- 172.21.15.179:0/489801329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 msgr2=0x7f5244161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f5244161790 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f522c009910 tx=0x7f522c009c20 comp rx=0 tx=0).stop 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- 172.21.15.179:0/489801329 shutdown_connections 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5244164db0 0x7f524406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 --2- 172.21.15.179:0/489801329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f5244161790 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- 172.21.15.179:0/489801329 >> 172.21.15.179:0/489801329 conn(0x7f52440ce9d0 msgr2=0x7f52440cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- 172.21.15.179:0/489801329 shutdown_connections 2024-05-08T22:38:02.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.284+0000 7f5249afa700 1 -- 172.21.15.179:0/489801329 wait complete. 2024-05-08T22:38:02.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.288+0000 7f5249afa700 1 Processor -- start 2024-05-08T22:38:02.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.288+0000 7f5249afa700 1 -- start start 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f524406dca0 0x7f524415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5244164db0 0x7f52440c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440cc680 con 0x7f524415ef80 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52441602b0 con 0x7f5244164db0 2024-05-08T22:38:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52440cbce0 con 0x7f524406dca0 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58784/0 (socket says 172.21.15.179:58784) 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 -- 172.21.15.179:0/3648450022 learned_addr learned my addr 172.21.15.179:0/3648450022 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5248af8700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f524406dca0 0x7f524415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f52492f9700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5244164db0 0x7f52440c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 -- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f524406dca0 msgr2=0x7f524415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f524406dca0 0x7f524415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 -- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5244164db0 msgr2=0x7f52440c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5244164db0 0x7f52440c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 -- 172.21.15.179:0/3648450022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c009570 con 0x7f524415ef80 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5243fff700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5234007e40 tx=0x7f523400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400eea0 con 0x7f524415ef80 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234014420 con 0x7f524415ef80 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52440cbfc0 con 0x7f524415ef80 2024-05-08T22:38:02.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340147d0 con 0x7f524415ef80 2024-05-08T22:38:02.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.292+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524415c3b0 con 0x7f524415ef80 2024-05-08T22:38:02.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5234014930 con 0x7f524415ef80 2024-05-08T22:38:02.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5244067740 con 0x7f524415ef80 2024-05-08T22:38:02.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f52417fa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 0x7f5224068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f52340849c0 con 0x7f524415ef80 2024-05-08T22:38:02.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f5248af8700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 0x7f5224068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.296+0000 7f5248af8700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 0x7f5224068bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.300+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5234051c80 con 0x7f524415ef80 2024-05-08T22:38:02.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.468+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5244165790 con 0x7f5224066730 2024-05-08T22:38:02.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.488+0000 7f52417fa700 1 -- 172.21.15.179:0/3648450022 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287288 (secure 0 0 0) 0x7f522c00b860 con 0x7f5224066730 2024-05-08T22:38:02.492 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 msgr2=0x7f5224068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 0x7f5224068bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c008030 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 msgr2=0x7f524415f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5234007e40 tx=0x7f523400e3c0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 shutdown_connections 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5224066730 0x7f5224068bf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5244164db0 0x7f52440c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f524415ef80 0x7f524415f960 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 --2- 172.21.15.179:0/3648450022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f524406dca0 0x7f524415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 >> 172.21.15.179:0/3648450022 conn(0x7f52440ce9d0 msgr2=0x7f5244070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 shutdown_connections 2024-05-08T22:38:02.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.492+0000 7f5249afa700 1 -- 172.21.15.179:0/3648450022 wait complete. 2024-05-08T22:38:02.496 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:38:02.508 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-05-08T22:38:01.478708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253804,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754720916,"statfs":{"total":773094113280,"available":772834217984,"internally_reserved":0,"allocated":9134080,"data_stored":4759162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.067987"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848306+0000","last_change":"2024-05-08T22:37:57.844883+0000","last_active":"2024-05-08T22:37:57.848306+0000","last_peered":"2024-05-08T22:37:57.848306+0000","last_clean":"2024-05-08T22:37:57.848306+0000","last_became_active":"2024-05-08T22:37:57.844614+0000","last_became_peered":"2024-05-08T22:37:57.844614+0000","last_unstale":"2024-05-08T22:37:57.848306+0000","last_undegraded":"2024-05-08T22:37:57.848306+0000","last_fullsized":"2024-05-08T22:37:57.848306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:56:36.890701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848623+0000","last_change":"2024-05-08T22:37:57.844447+0000","last_active":"2024-05-08T22:37:57.848623+0000","last_peered":"2024-05-08T22:37:57.848623+0000","last_clean":"2024-05-08T22:37:57.848623+0000","last_became_active":"2024-05-08T22:37:57.844201+0000","last_became_peered":"2024-05-08T22:37:57.844201+0000","last_unstale":"2024-05-08T22:37:57.848623+0000","last_undegraded":"2024-05-08T22:37:57.848623+0000","last_fullsized":"2024-05-08T22:37:57.848623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:10.013238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838916+0000","last_change":"2024-05-08T22:37:57.835351+0000","last_active":"2024-05-08T22:37:57.838916+0000","last_peered":"2024-05-08T22:37:57.838916+0000","last_clean":"2024-05-08T22:37:57.838916+0000","last_became_active":"2024-05-08T22:37:57.835104+0000","last_became_peered":"2024-05-08T22:37:57.835104+0000","last_unstale":"2024-05-08T22:37:57.838916+0000","last_undegraded":"2024-05-08T22:37:57.838916+0000","last_fullsized":"2024-05-08T22:37:57.838916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844279+0000","last_change":"2024-05-08T22:37:57.842727+0000","last_active":"2024-05-08T22:37:57.844279+0000","last_peered":"2024-05-08T22:37:57.844279+0000","last_clean":"2024-05-08T22:37:57.844279+0000","last_became_active":"2024-05-08T22:37:57.842350+0000","last_became_peered":"2024-05-08T22:37:57.842350+0000","last_unstale":"2024-05-08T22:37:57.844279+0000","last_undegraded":"2024-05-08T22:37:57.844279+0000","last_fullsized":"2024-05-08T22:37:57.844279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:32.252275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848269+0000","last_change":"2024-05-08T22:37:57.846399+0000","last_active":"2024-05-08T22:37:57.848269+0000","last_peered":"2024-05-08T22:37:57.848269+0000","last_clean":"2024-05-08T22:37:57.848269+0000","last_became_active":"2024-05-08T22:37:57.845308+0000","last_became_peered":"2024-05-08T22:37:57.845308+0000","last_unstale":"2024-05-08T22:37:57.848269+0000","last_undegraded":"2024-05-08T22:37:57.848269+0000","last_fullsized":"2024-05-08T22:37:57.848269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:57.759733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850942+0000","last_change":"2024-05-08T22:37:57.847700+0000","last_active":"2024-05-08T22:37:57.850942+0000","last_peered":"2024-05-08T22:37:57.850942+0000","last_clean":"2024-05-08T22:37:57.850942+0000","last_became_active":"2024-05-08T22:37:57.847333+0000","last_became_peered":"2024-05-08T22:37:57.847333+0000","last_unstale":"2024-05-08T22:37:57.850942+0000","last_undegraded":"2024-05-08T22:37:57.850942+0000","last_fullsized":"2024-05-08T22:37:57.850942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:59.881785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839403+0000","last_change":"2024-05-08T22:37:57.832432+0000","last_active":"2024-05-08T22:37:57.839403+0000","last_peered":"2024-05-08T22:37:57.839403+0000","last_clean":"2024-05-08T22:37:57.839403+0000","last_became_active":"2024-05-08T22:37:57.832048+0000","last_became_peered":"2024-05-08T22:37:57.832048+0000","last_unstale":"2024-05-08T22:37:57.839403+0000","last_undegraded":"2024-05-08T22:37:57.839403+0000","last_fullsized":"2024-05-08T22:37:57.839403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:26.697556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848302+0000","last_change":"2024-05-08T22:37:57.846519+0000","last_active":"2024-05-08T22:37:57.848302+0000","last_peered":"2024-05-08T22:37:57.848302+0000","last_clean":"2024-05-08T22:37:57.848302+0000","last_became_active":"2024-05-08T22:37:57.845361+0000","last_became_peered":"2024-05-08T22:37:57.845361+0000","last_unstale":"2024-05-08T22:37:57.848302+0000","last_undegraded":"2024-05-08T22:37:57.848302+0000","last_fullsized":"2024-05-08T22:37:57.848302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:34.786251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.430120+0000","last_change":"2024-05-08T22:37:57.430120+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.430120+0000","last_undegraded":"2024-05-08T22:37:57.430120+0000","last_fullsized":"2024-05-08T22:37:57.430120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839006+0000","last_change":"2024-05-08T22:37:57.825872+0000","last_active":"2024-05-08T22:37:57.839006+0000","last_peered":"2024-05-08T22:37:57.839006+0000","last_clean":"2024-05-08T22:37:57.839006+0000","last_became_active":"2024-05-08T22:37:57.825591+0000","last_became_peered":"2024-05-08T22:37:57.825591+0000","last_unstale":"2024-05-08T22:37:57.839006+0000","last_undegraded":"2024-05-08T22:37:57.839006+0000","last_fullsized":"2024-05-08T22:37:57.839006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:07.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848675+0000","last_change":"2024-05-08T22:37:57.844985+0000","last_active":"2024-05-08T22:37:57.848675+0000","last_peered":"2024-05-08T22:37:57.848675+0000","last_clean":"2024-05-08T22:37:57.848675+0000","last_became_active":"2024-05-08T22:37:57.844748+0000","last_became_peered":"2024-05-08T22:37:57.844748+0000","last_unstale":"2024-05-08T22:37:57.848675+0000","last_undegraded":"2024-05-08T22:37:57.848675+0000","last_fullsized":"2024-05-08T22:37:57.848675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:02.244593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838791+0000","last_change":"2024-05-08T22:37:57.830338+0000","last_active":"2024-05-08T22:37:57.838791+0000","last_peered":"2024-05-08T22:37:57.838791+0000","last_clean":"2024-05-08T22:37:57.838791+0000","last_became_active":"2024-05-08T22:37:57.830069+0000","last_became_peered":"2024-05-08T22:37:57.830069+0000","last_unstale":"2024-05-08T22:37:57.838791+0000","last_undegraded":"2024-05-08T22:37:57.838791+0000","last_fullsized":"2024-05-08T22:37:57.838791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:53.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841042+0000","last_change":"2024-05-08T22:37:57.835141+0000","last_active":"2024-05-08T22:37:57.841042+0000","last_peered":"2024-05-08T22:37:57.841042+0000","last_clean":"2024-05-08T22:37:57.841042+0000","last_became_active":"2024-05-08T22:37:57.832615+0000","last_became_peered":"2024-05-08T22:37:57.832615+0000","last_unstale":"2024-05-08T22:37:57.841042+0000","last_undegraded":"2024-05-08T22:37:57.841042+0000","last_fullsized":"2024-05-08T22:37:57.841042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:40.282177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850914+0000","last_change":"2024-05-08T22:37:57.847622+0000","last_active":"2024-05-08T22:37:57.850914+0000","last_peered":"2024-05-08T22:37:57.850914+0000","last_clean":"2024-05-08T22:37:57.850914+0000","last_became_active":"2024-05-08T22:37:57.847294+0000","last_became_peered":"2024-05-08T22:37:57.847294+0000","last_unstale":"2024-05-08T22:37:57.850914+0000","last_undegraded":"2024-05-08T22:37:57.850914+0000","last_fullsized":"2024-05-08T22:37:57.850914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:14.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843497+0000","last_change":"2024-05-08T22:37:57.836871+0000","last_active":"2024-05-08T22:37:57.843497+0000","last_peered":"2024-05-08T22:37:57.843497+0000","last_clean":"2024-05-08T22:37:57.843497+0000","last_became_active":"2024-05-08T22:37:57.836626+0000","last_became_peered":"2024-05-08T22:37:57.836626+0000","last_unstale":"2024-05-08T22:37:57.843497+0000","last_undegraded":"2024-05-08T22:37:57.843497+0000","last_fullsized":"2024-05-08T22:37:57.843497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:22.008737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844329+0000","last_change":"2024-05-08T22:37:57.840135+0000","last_active":"2024-05-08T22:37:57.844329+0000","last_peered":"2024-05-08T22:37:57.844329+0000","last_clean":"2024-05-08T22:37:57.844329+0000","last_became_active":"2024-05-08T22:37:57.839806+0000","last_became_peered":"2024-05-08T22:37:57.839806+0000","last_unstale":"2024-05-08T22:37:57.844329+0000","last_undegraded":"2024-05-08T22:37:57.844329+0000","last_fullsized":"2024-05-08T22:37:57.844329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:53.757689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841087+0000","last_change":"2024-05-08T22:37:57.832054+0000","last_active":"2024-05-08T22:37:57.841087+0000","last_peered":"2024-05-08T22:37:57.841087+0000","last_clean":"2024-05-08T22:37:57.841087+0000","last_became_active":"2024-05-08T22:37:57.831311+0000","last_became_peered":"2024-05-08T22:37:57.831311+0000","last_unstale":"2024-05-08T22:37:57.841087+0000","last_undegraded":"2024-05-08T22:37:57.841087+0000","last_fullsized":"2024-05-08T22:37:57.841087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:27.877407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850784+0000","last_change":"2024-05-08T22:37:57.847959+0000","last_active":"2024-05-08T22:37:57.850784+0000","last_peered":"2024-05-08T22:37:57.850784+0000","last_clean":"2024-05-08T22:37:57.850784+0000","last_became_active":"2024-05-08T22:37:57.847531+0000","last_became_peered":"2024-05-08T22:37:57.847531+0000","last_unstale":"2024-05-08T22:37:57.850784+0000","last_undegraded":"2024-05-08T22:37:57.850784+0000","last_fullsized":"2024-05-08T22:37:57.850784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:15.951130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833932+0000","last_change":"2024-05-08T22:37:57.832828+0000","last_active":"2024-05-08T22:37:57.833932+0000","last_peered":"2024-05-08T22:37:57.833932+0000","last_clean":"2024-05-08T22:37:57.833932+0000","last_became_active":"2024-05-08T22:37:57.832544+0000","last_became_peered":"2024-05-08T22:37:57.832544+0000","last_unstale":"2024-05-08T22:37:57.833932+0000","last_undegraded":"2024-05-08T22:37:57.833932+0000","last_fullsized":"2024-05-08T22:37:57.833932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:57.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843557+0000","last_change":"2024-05-08T22:37:57.831045+0000","last_active":"2024-05-08T22:37:57.843557+0000","last_peered":"2024-05-08T22:37:57.843557+0000","last_clean":"2024-05-08T22:37:57.843557+0000","last_became_active":"2024-05-08T22:37:57.830745+0000","last_became_peered":"2024-05-08T22:37:57.830745+0000","last_unstale":"2024-05-08T22:37:57.843557+0000","last_undegraded":"2024-05-08T22:37:57.843557+0000","last_fullsized":"2024-05-08T22:37:57.843557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:31.214851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429934+0000","last_change":"2024-05-08T22:37:57.429934+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429934+0000","last_undegraded":"2024-05-08T22:37:57.429934+0000","last_fullsized":"2024-05-08T22:37:57.429934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846622+0000","last_change":"2024-05-08T22:37:57.844194+0000","last_active":"2024-05-08T22:37:57.846622+0000","last_peered":"2024-05-08T22:37:57.846622+0000","last_clean":"2024-05-08T22:37:57.846622+0000","last_became_active":"2024-05-08T22:37:57.841102+0000","last_became_peered":"2024-05-08T22:37:57.841102+0000","last_unstale":"2024-05-08T22:37:57.846622+0000","last_undegraded":"2024-05-08T22:37:57.846622+0000","last_fullsized":"2024-05-08T22:37:57.846622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:15.481984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424461+0000","last_change":"2024-05-08T22:37:57.424461+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424461+0000","last_undegraded":"2024-05-08T22:37:57.424461+0000","last_fullsized":"2024-05-08T22:37:57.424461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847805+0000","last_change":"2024-05-08T22:37:57.840565+0000","last_active":"2024-05-08T22:37:57.847805+0000","last_peered":"2024-05-08T22:37:57.847805+0000","last_clean":"2024-05-08T22:37:57.847805+0000","last_became_active":"2024-05-08T22:37:57.834465+0000","last_became_peered":"2024-05-08T22:37:57.834465+0000","last_unstale":"2024-05-08T22:37:57.847805+0000","last_undegraded":"2024-05-08T22:37:57.847805+0000","last_fullsized":"2024-05-08T22:37:57.847805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:23.426774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840471+0000","last_change":"2024-05-08T22:37:57.834804+0000","last_active":"2024-05-08T22:37:57.840471+0000","last_peered":"2024-05-08T22:37:57.840471+0000","last_clean":"2024-05-08T22:37:57.840471+0000","last_became_active":"2024-05-08T22:37:57.834268+0000","last_became_peered":"2024-05-08T22:37:57.834268+0000","last_unstale":"2024-05-08T22:37:57.840471+0000","last_undegraded":"2024-05-08T22:37:57.840471+0000","last_fullsized":"2024-05-08T22:37:57.840471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:10.524450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839026+0000","last_change":"2024-05-08T22:37:57.831011+0000","last_active":"2024-05-08T22:37:57.839026+0000","last_peered":"2024-05-08T22:37:57.839026+0000","last_clean":"2024-05-08T22:37:57.839026+0000","last_became_active":"2024-05-08T22:37:57.830662+0000","last_became_peered":"2024-05-08T22:37:57.830662+0000","last_unstale":"2024-05-08T22:37:57.839026+0000","last_undegraded":"2024-05-08T22:37:57.839026+0000","last_fullsized":"2024-05-08T22:37:57.839026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:33.816353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844342+0000","last_change":"2024-05-08T22:37:57.840718+0000","last_active":"2024-05-08T22:37:57.844342+0000","last_peered":"2024-05-08T22:37:57.844342+0000","last_clean":"2024-05-08T22:37:57.844342+0000","last_became_active":"2024-05-08T22:37:57.840400+0000","last_became_peered":"2024-05-08T22:37:57.840400+0000","last_unstale":"2024-05-08T22:37:57.844342+0000","last_undegraded":"2024-05-08T22:37:57.844342+0000","last_fullsized":"2024-05-08T22:37:57.844342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.153666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846441+0000","last_change":"2024-05-08T22:37:57.844109+0000","last_active":"2024-05-08T22:37:57.846441+0000","last_peered":"2024-05-08T22:37:57.846441+0000","last_clean":"2024-05-08T22:37:57.846441+0000","last_became_active":"2024-05-08T22:37:57.843582+0000","last_became_peered":"2024-05-08T22:37:57.843582+0000","last_unstale":"2024-05-08T22:37:57.846441+0000","last_undegraded":"2024-05-08T22:37:57.846441+0000","last_fullsized":"2024-05-08T22:37:57.846441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:58.974111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847790+0000","last_change":"2024-05-08T22:37:57.841963+0000","last_active":"2024-05-08T22:37:57.847790+0000","last_peered":"2024-05-08T22:37:57.847790+0000","last_clean":"2024-05-08T22:37:57.847790+0000","last_became_active":"2024-05-08T22:37:57.841575+0000","last_became_peered":"2024-05-08T22:37:57.841575+0000","last_unstale":"2024-05-08T22:37:57.847790+0000","last_undegraded":"2024-05-08T22:37:57.847790+0000","last_fullsized":"2024-05-08T22:37:57.847790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:33.257259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845819+0000","last_change":"2024-05-08T22:37:57.840274+0000","last_active":"2024-05-08T22:37:57.845819+0000","last_peered":"2024-05-08T22:37:57.845819+0000","last_clean":"2024-05-08T22:37:57.845819+0000","last_became_active":"2024-05-08T22:37:57.840011+0000","last_became_peered":"2024-05-08T22:37:57.840011+0000","last_unstale":"2024-05-08T22:37:57.845819+0000","last_undegraded":"2024-05-08T22:37:57.845819+0000","last_fullsized":"2024-05-08T22:37:57.845819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:45.626643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.832439+0000","last_change":"2024-05-08T22:37:57.827795+0000","last_active":"2024-05-08T22:37:57.832439+0000","last_peered":"2024-05-08T22:37:57.832439+0000","last_clean":"2024-05-08T22:37:57.832439+0000","last_became_active":"2024-05-08T22:37:57.827400+0000","last_became_peered":"2024-05-08T22:37:57.827400+0000","last_unstale":"2024-05-08T22:37:57.832439+0000","last_undegraded":"2024-05-08T22:37:57.832439+0000","last_fullsized":"2024-05-08T22:37:57.832439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:31.579840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848125+0000","last_change":"2024-05-08T22:37:57.844868+0000","last_active":"2024-05-08T22:37:57.848125+0000","last_peered":"2024-05-08T22:37:57.848125+0000","last_clean":"2024-05-08T22:37:57.848125+0000","last_became_active":"2024-05-08T22:37:57.844575+0000","last_became_peered":"2024-05-08T22:37:57.844575+0000","last_unstale":"2024-05-08T22:37:57.848125+0000","last_undegraded":"2024-05-08T22:37:57.848125+0000","last_fullsized":"2024-05-08T22:37:57.848125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850774+0000","last_change":"2024-05-08T22:37:57.847858+0000","last_active":"2024-05-08T22:37:57.850774+0000","last_peered":"2024-05-08T22:37:57.850774+0000","last_clean":"2024-05-08T22:37:57.850774+0000","last_became_active":"2024-05-08T22:37:57.847609+0000","last_became_peered":"2024-05-08T22:37:57.847609+0000","last_unstale":"2024-05-08T22:37:57.850774+0000","last_undegraded":"2024-05-08T22:37:57.850774+0000","last_fullsized":"2024-05-08T22:37:57.850774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:00.575919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844834+0000","last_change":"2024-05-08T22:37:57.446359+0000","last_active":"2024-05-08T22:37:57.844834+0000","last_peered":"2024-05-08T22:37:57.844834+0000","last_clean":"2024-05-08T22:37:57.844834+0000","last_became_active":"2024-05-08T22:37:57.446036+0000","last_became_peered":"2024-05-08T22:37:57.446036+0000","last_unstale":"2024-05-08T22:37:57.844834+0000","last_undegraded":"2024-05-08T22:37:57.844834+0000","last_fullsized":"2024-05-08T22:37:57.844834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:00.419625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847776+0000","last_change":"2024-05-08T22:37:57.842007+0000","last_active":"2024-05-08T22:37:57.847776+0000","last_peered":"2024-05-08T22:37:57.847776+0000","last_clean":"2024-05-08T22:37:57.847776+0000","last_became_active":"2024-05-08T22:37:57.841764+0000","last_became_peered":"2024-05-08T22:37:57.841764+0000","last_unstale":"2024-05-08T22:37:57.847776+0000","last_undegraded":"2024-05-08T22:37:57.847776+0000","last_fullsized":"2024-05-08T22:37:57.847776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:58.771407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842515+0000","last_change":"2024-05-08T22:37:57.440281+0000","last_active":"2024-05-08T22:37:57.842515+0000","last_peered":"2024-05-08T22:37:57.842515+0000","last_clean":"2024-05-08T22:37:57.842515+0000","last_became_active":"2024-05-08T22:37:57.439987+0000","last_became_peered":"2024-05-08T22:37:57.439987+0000","last_unstale":"2024-05-08T22:37:57.842515+0000","last_undegraded":"2024-05-08T22:37:57.842515+0000","last_fullsized":"2024-05-08T22:37:57.842515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:32.804508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.425803+0000","last_change":"2024-05-08T22:37:57.425803+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.425803+0000","last_undegraded":"2024-05-08T22:37:57.425803+0000","last_fullsized":"2024-05-08T22:37:57.425803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846799+0000","last_change":"2024-05-08T22:37:57.448745+0000","last_active":"2024-05-08T22:37:57.846799+0000","last_peered":"2024-05-08T22:37:57.846799+0000","last_clean":"2024-05-08T22:37:57.846799+0000","last_became_active":"2024-05-08T22:37:57.448495+0000","last_became_peered":"2024-05-08T22:37:57.448495+0000","last_unstale":"2024-05-08T22:37:57.846799+0000","last_undegraded":"2024-05-08T22:37:57.846799+0000","last_fullsized":"2024-05-08T22:37:57.846799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:52.343924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429477+0000","last_change":"2024-05-08T22:37:57.429477+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429477+0000","last_undegraded":"2024-05-08T22:37:57.429477+0000","last_fullsized":"2024-05-08T22:37:57.429477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848978+0000","last_change":"2024-05-08T22:37:57.442124+0000","last_active":"2024-05-08T22:37:57.848978+0000","last_peered":"2024-05-08T22:37:57.848978+0000","last_clean":"2024-05-08T22:37:57.848978+0000","last_became_active":"2024-05-08T22:37:57.441844+0000","last_became_peered":"2024-05-08T22:37:57.441844+0000","last_unstale":"2024-05-08T22:37:57.848978+0000","last_undegraded":"2024-05-08T22:37:57.848978+0000","last_fullsized":"2024-05-08T22:37:57.848978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:12.294211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848681+0000","last_change":"2024-05-08T22:37:57.845013+0000","last_active":"2024-05-08T22:37:57.848681+0000","last_peered":"2024-05-08T22:37:57.848681+0000","last_clean":"2024-05-08T22:37:57.848681+0000","last_became_active":"2024-05-08T22:37:57.844721+0000","last_became_peered":"2024-05-08T22:37:57.844721+0000","last_unstale":"2024-05-08T22:37:57.848681+0000","last_undegraded":"2024-05-08T22:37:57.848681+0000","last_fullsized":"2024-05-08T22:37:57.848681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.527397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439079+0000","last_change":"2024-05-08T22:37:57.438281+0000","last_active":"2024-05-08T22:37:57.439079+0000","last_peered":"2024-05-08T22:37:57.439079+0000","last_clean":"2024-05-08T22:37:57.439079+0000","last_became_active":"2024-05-08T22:37:57.438016+0000","last_became_peered":"2024-05-08T22:37:57.438016+0000","last_unstale":"2024-05-08T22:37:57.439079+0000","last_undegraded":"2024-05-08T22:37:57.439079+0000","last_fullsized":"2024-05-08T22:37:57.439079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:22.432349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848718+0000","last_change":"2024-05-08T22:37:57.845395+0000","last_active":"2024-05-08T22:37:57.848718+0000","last_peered":"2024-05-08T22:37:57.848718+0000","last_clean":"2024-05-08T22:37:57.848718+0000","last_became_active":"2024-05-08T22:37:57.845129+0000","last_became_peered":"2024-05-08T22:37:57.845129+0000","last_unstale":"2024-05-08T22:37:57.848718+0000","last_undegraded":"2024-05-08T22:37:57.848718+0000","last_fullsized":"2024-05-08T22:37:57.848718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:26.794424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838895+0000","last_change":"2024-05-08T22:37:57.436502+0000","last_active":"2024-05-08T22:37:57.838895+0000","last_peered":"2024-05-08T22:37:57.838895+0000","last_clean":"2024-05-08T22:37:57.838895+0000","last_became_active":"2024-05-08T22:37:57.436246+0000","last_became_peered":"2024-05-08T22:37:57.436246+0000","last_unstale":"2024-05-08T22:37:57.838895+0000","last_undegraded":"2024-05-08T22:37:57.838895+0000","last_fullsized":"2024-05-08T22:37:57.838895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:07.871185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.426045+0000","last_change":"2024-05-08T22:37:57.426045+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.426045+0000","last_undegraded":"2024-05-08T22:37:57.426045+0000","last_fullsized":"2024-05-08T22:37:57.426045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826353+0000","last_change":"2024-05-08T22:37:57.443165+0000","last_active":"2024-05-08T22:37:57.826353+0000","last_peered":"2024-05-08T22:37:57.826353+0000","last_clean":"2024-05-08T22:37:57.826353+0000","last_became_active":"2024-05-08T22:37:57.442883+0000","last_became_peered":"2024-05-08T22:37:57.442883+0000","last_unstale":"2024-05-08T22:37:57.826353+0000","last_undegraded":"2024-05-08T22:37:57.826353+0000","last_fullsized":"2024-05-08T22:37:57.826353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:43.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849717+0000","last_change":"2024-05-08T22:37:57.840713+0000","last_active":"2024-05-08T22:37:57.849717+0000","last_peered":"2024-05-08T22:37:57.849717+0000","last_clean":"2024-05-08T22:37:57.849717+0000","last_became_active":"2024-05-08T22:37:57.839777+0000","last_became_peered":"2024-05-08T22:37:57.839777+0000","last_unstale":"2024-05-08T22:37:57.849717+0000","last_undegraded":"2024-05-08T22:37:57.849717+0000","last_fullsized":"2024-05-08T22:37:57.849717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:09.133783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.828973+0000","last_change":"2024-05-08T22:37:57.445418+0000","last_active":"2024-05-08T22:37:57.828973+0000","last_peered":"2024-05-08T22:37:57.828973+0000","last_clean":"2024-05-08T22:37:57.828973+0000","last_became_active":"2024-05-08T22:37:57.445005+0000","last_became_peered":"2024-05-08T22:37:57.445005+0000","last_unstale":"2024-05-08T22:37:57.828973+0000","last_undegraded":"2024-05-08T22:37:57.828973+0000","last_fullsized":"2024-05-08T22:37:57.828973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:01.056718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848795+0000","last_change":"2024-05-08T22:37:57.846487+0000","last_active":"2024-05-08T22:37:57.848795+0000","last_peered":"2024-05-08T22:37:57.848795+0000","last_clean":"2024-05-08T22:37:57.848795+0000","last_became_active":"2024-05-08T22:37:57.845715+0000","last_became_peered":"2024-05-08T22:37:57.845715+0000","last_unstale":"2024-05-08T22:37:57.848795+0000","last_undegraded":"2024-05-08T22:37:57.848795+0000","last_fullsized":"2024-05-08T22:37:57.848795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:56.850979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829513+0000","last_change":"2024-05-08T22:37:57.442931+0000","last_active":"2024-05-08T22:37:57.829513+0000","last_peered":"2024-05-08T22:37:57.829513+0000","last_clean":"2024-05-08T22:37:57.829513+0000","last_became_active":"2024-05-08T22:37:57.441198+0000","last_became_peered":"2024-05-08T22:37:57.441198+0000","last_unstale":"2024-05-08T22:37:57.829513+0000","last_undegraded":"2024-05-08T22:37:57.829513+0000","last_fullsized":"2024-05-08T22:37:57.829513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:35.216047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844949+0000","last_change":"2024-05-08T22:37:57.844158+0000","last_active":"2024-05-08T22:37:57.844949+0000","last_peered":"2024-05-08T22:37:57.844949+0000","last_clean":"2024-05-08T22:37:57.844949+0000","last_became_active":"2024-05-08T22:37:57.843855+0000","last_became_peered":"2024-05-08T22:37:57.843855+0000","last_unstale":"2024-05-08T22:37:57.844949+0000","last_undegraded":"2024-05-08T22:37:57.844949+0000","last_fullsized":"2024-05-08T22:37:57.844949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:36.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822311+0000","last_change":"2024-05-08T22:37:57.439889+0000","last_active":"2024-05-08T22:37:57.822311+0000","last_peered":"2024-05-08T22:37:57.822311+0000","last_clean":"2024-05-08T22:37:57.822311+0000","last_became_active":"2024-05-08T22:37:57.439633+0000","last_became_peered":"2024-05-08T22:37:57.439633+0000","last_unstale":"2024-05-08T22:37:57.822311+0000","last_undegraded":"2024-05-08T22:37:57.822311+0000","last_fullsized":"2024-05-08T22:37:57.822311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:01.221867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839271+0000","last_change":"2024-05-08T22:37:57.833440+0000","last_active":"2024-05-08T22:37:57.839271+0000","last_peered":"2024-05-08T22:37:57.839271+0000","last_clean":"2024-05-08T22:37:57.839271+0000","last_became_active":"2024-05-08T22:37:57.833212+0000","last_became_peered":"2024-05-08T22:37:57.833212+0000","last_unstale":"2024-05-08T22:37:57.839271+0000","last_undegraded":"2024-05-08T22:37:57.839271+0000","last_fullsized":"2024-05-08T22:37:57.839271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:08.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846625+0000","last_change":"2024-05-08T22:37:57.444957+0000","last_active":"2024-05-08T22:37:57.846625+0000","last_peered":"2024-05-08T22:37:57.846625+0000","last_clean":"2024-05-08T22:37:57.846625+0000","last_became_active":"2024-05-08T22:37:57.444484+0000","last_became_peered":"2024-05-08T22:37:57.444484+0000","last_unstale":"2024-05-08T22:37:57.846625+0000","last_undegraded":"2024-05-08T22:37:57.846625+0000","last_fullsized":"2024-05-08T22:37:57.846625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:46.229790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836275+0000","last_change":"2024-05-08T22:37:57.830761+0000","last_active":"2024-05-08T22:37:57.836275+0000","last_peered":"2024-05-08T22:37:57.836275+0000","last_clean":"2024-05-08T22:37:57.836275+0000","last_became_active":"2024-05-08T22:37:57.830491+0000","last_became_peered":"2024-05-08T22:37:57.830491+0000","last_unstale":"2024-05-08T22:37:57.836275+0000","last_undegraded":"2024-05-08T22:37:57.836275+0000","last_fullsized":"2024-05-08T22:37:57.836275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:12.683143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849871+0000","last_change":"2024-05-08T22:37:57.439248+0000","last_active":"2024-05-08T22:37:57.849871+0000","last_peered":"2024-05-08T22:37:57.849871+0000","last_clean":"2024-05-08T22:37:57.849871+0000","last_became_active":"2024-05-08T22:37:57.438927+0000","last_became_peered":"2024-05-08T22:37:57.438927+0000","last_unstale":"2024-05-08T22:37:57.849871+0000","last_undegraded":"2024-05-08T22:37:57.849871+0000","last_fullsized":"2024-05-08T22:37:57.849871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:11.645499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838819+0000","last_change":"2024-05-08T22:37:57.835299+0000","last_active":"2024-05-08T22:37:57.838819+0000","last_peered":"2024-05-08T22:37:57.838819+0000","last_clean":"2024-05-08T22:37:57.838819+0000","last_became_active":"2024-05-08T22:37:57.834959+0000","last_became_peered":"2024-05-08T22:37:57.834959+0000","last_unstale":"2024-05-08T22:37:57.838819+0000","last_undegraded":"2024-05-08T22:37:57.838819+0000","last_fullsized":"2024-05-08T22:37:57.838819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:19.550971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848959+0000","last_change":"2024-05-08T22:37:57.436168+0000","last_active":"2024-05-08T22:37:57.848959+0000","last_peered":"2024-05-08T22:37:57.848959+0000","last_clean":"2024-05-08T22:37:57.848959+0000","last_became_active":"2024-05-08T22:37:57.435911+0000","last_became_peered":"2024-05-08T22:37:57.435911+0000","last_unstale":"2024-05-08T22:37:57.848959+0000","last_undegraded":"2024-05-08T22:37:57.848959+0000","last_fullsized":"2024-05-08T22:37:57.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:39.531241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839382+0000","last_change":"2024-05-08T22:37:57.834557+0000","last_active":"2024-05-08T22:37:57.839382+0000","last_peered":"2024-05-08T22:37:57.839382+0000","last_clean":"2024-05-08T22:37:57.839382+0000","last_became_active":"2024-05-08T22:37:57.834336+0000","last_became_peered":"2024-05-08T22:37:57.834336+0000","last_unstale":"2024-05-08T22:37:57.839382+0000","last_undegraded":"2024-05-08T22:37:57.839382+0000","last_fullsized":"2024-05-08T22:37:57.839382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:07.977190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826416+0000","last_change":"2024-05-08T22:37:57.442582+0000","last_active":"2024-05-08T22:37:57.826416+0000","last_peered":"2024-05-08T22:37:57.826416+0000","last_clean":"2024-05-08T22:37:57.826416+0000","last_became_active":"2024-05-08T22:37:57.440956+0000","last_became_peered":"2024-05-08T22:37:57.440956+0000","last_unstale":"2024-05-08T22:37:57.826416+0000","last_undegraded":"2024-05-08T22:37:57.826416+0000","last_fullsized":"2024-05-08T22:37:57.826416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:40.041454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849792+0000","last_change":"2024-05-08T22:37:57.846329+0000","last_active":"2024-05-08T22:37:57.849792+0000","last_peered":"2024-05-08T22:37:57.849792+0000","last_clean":"2024-05-08T22:37:57.849792+0000","last_became_active":"2024-05-08T22:37:57.846056+0000","last_became_peered":"2024-05-08T22:37:57.846056+0000","last_unstale":"2024-05-08T22:37:57.849792+0000","last_undegraded":"2024-05-08T22:37:57.849792+0000","last_fullsized":"2024-05-08T22:37:57.849792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829433+0000","last_change":"2024-05-08T22:37:57.448219+0000","last_active":"2024-05-08T22:37:57.829433+0000","last_peered":"2024-05-08T22:37:57.829433+0000","last_clean":"2024-05-08T22:37:57.829433+0000","last_became_active":"2024-05-08T22:37:57.447923+0000","last_became_peered":"2024-05-08T22:37:57.447923+0000","last_unstale":"2024-05-08T22:37:57.829433+0000","last_undegraded":"2024-05-08T22:37:57.829433+0000","last_fullsized":"2024-05-08T22:37:57.829433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:30.907553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839082+0000","last_change":"2024-05-08T22:37:57.829299+0000","last_active":"2024-05-08T22:37:57.839082+0000","last_peered":"2024-05-08T22:37:57.839082+0000","last_clean":"2024-05-08T22:37:57.839082+0000","last_became_active":"2024-05-08T22:37:57.828970+0000","last_became_peered":"2024-05-08T22:37:57.828970+0000","last_unstale":"2024-05-08T22:37:57.839082+0000","last_undegraded":"2024-05-08T22:37:57.839082+0000","last_fullsized":"2024-05-08T22:37:57.839082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:01.133413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.851908+0000","last_change":"2024-05-08T22:37:57.444179+0000","last_active":"2024-05-08T22:37:57.851908+0000","last_peered":"2024-05-08T22:37:57.851908+0000","last_clean":"2024-05-08T22:37:57.851908+0000","last_became_active":"2024-05-08T22:37:57.443673+0000","last_became_peered":"2024-05-08T22:37:57.443673+0000","last_unstale":"2024-05-08T22:37:57.851908+0000","last_undegraded":"2024-05-08T22:37:57.851908+0000","last_fullsized":"2024-05-08T22:37:57.851908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:53.287958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845235+0000","last_change":"2024-05-08T22:37:57.841025+0000","last_active":"2024-05-08T22:37:57.845235+0000","last_peered":"2024-05-08T22:37:57.845235+0000","last_clean":"2024-05-08T22:37:57.845235+0000","last_became_active":"2024-05-08T22:37:57.840591+0000","last_became_peered":"2024-05-08T22:37:57.840591+0000","last_unstale":"2024-05-08T22:37:57.845235+0000","last_undegraded":"2024-05-08T22:37:57.845235+0000","last_fullsized":"2024-05-08T22:37:57.845235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.910062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842909+0000","last_change":"2024-05-08T22:37:57.440960+0000","last_active":"2024-05-08T22:37:57.842909+0000","last_peered":"2024-05-08T22:37:57.842909+0000","last_clean":"2024-05-08T22:37:57.842909+0000","last_became_active":"2024-05-08T22:37:57.440504+0000","last_became_peered":"2024-05-08T22:37:57.440504+0000","last_unstale":"2024-05-08T22:37:57.842909+0000","last_undegraded":"2024-05-08T22:37:57.842909+0000","last_fullsized":"2024-05-08T22:37:57.842909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:06.401155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841075+0000","last_change":"2024-05-08T22:37:57.831846+0000","last_active":"2024-05-08T22:37:57.841075+0000","last_peered":"2024-05-08T22:37:57.841075+0000","last_clean":"2024-05-08T22:37:57.841075+0000","last_became_active":"2024-05-08T22:37:57.829692+0000","last_became_peered":"2024-05-08T22:37:57.829692+0000","last_unstale":"2024-05-08T22:37:57.841075+0000","last_undegraded":"2024-05-08T22:37:57.841075+0000","last_fullsized":"2024-05-08T22:37:57.841075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.674154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439143+0000","last_change":"2024-05-08T22:37:57.437527+0000","last_active":"2024-05-08T22:37:57.439143+0000","last_peered":"2024-05-08T22:37:57.439143+0000","last_clean":"2024-05-08T22:37:57.439143+0000","last_became_active":"2024-05-08T22:37:57.437261+0000","last_became_peered":"2024-05-08T22:37:57.437261+0000","last_unstale":"2024-05-08T22:37:57.439143+0000","last_undegraded":"2024-05-08T22:37:57.439143+0000","last_fullsized":"2024-05-08T22:37:57.439143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:19.394053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844277+0000","last_change":"2024-05-08T22:37:57.832513+0000","last_active":"2024-05-08T22:37:57.844277+0000","last_peered":"2024-05-08T22:37:57.844277+0000","last_clean":"2024-05-08T22:37:57.844277+0000","last_became_active":"2024-05-08T22:37:57.832275+0000","last_became_peered":"2024-05-08T22:37:57.832275+0000","last_unstale":"2024-05-08T22:37:57.844277+0000","last_undegraded":"2024-05-08T22:37:57.844277+0000","last_fullsized":"2024-05-08T22:37:57.844277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:08.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847277+0000","last_change":"2024-05-08T22:37:57.444668+0000","last_active":"2024-05-08T22:37:57.847277+0000","last_peered":"2024-05-08T22:37:57.847277+0000","last_clean":"2024-05-08T22:37:57.847277+0000","last_became_active":"2024-05-08T22:37:57.441598+0000","last_became_peered":"2024-05-08T22:37:57.441598+0000","last_unstale":"2024-05-08T22:37:57.847277+0000","last_undegraded":"2024-05-08T22:37:57.847277+0000","last_fullsized":"2024-05-08T22:37:57.847277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:33.716602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833929+0000","last_change":"2024-05-08T22:37:57.832325+0000","last_active":"2024-05-08T22:37:57.833929+0000","last_peered":"2024-05-08T22:37:57.833929+0000","last_clean":"2024-05-08T22:37:57.833929+0000","last_became_active":"2024-05-08T22:37:57.832067+0000","last_became_peered":"2024-05-08T22:37:57.832067+0000","last_unstale":"2024-05-08T22:37:57.833929+0000","last_undegraded":"2024-05-08T22:37:57.833929+0000","last_fullsized":"2024-05-08T22:37:57.833929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:42.187631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817518+0000","last_change":"2024-05-08T22:37:57.442255+0000","last_active":"2024-05-08T22:37:57.817518+0000","last_peered":"2024-05-08T22:37:57.817518+0000","last_clean":"2024-05-08T22:37:57.817518+0000","last_became_active":"2024-05-08T22:37:57.441960+0000","last_became_peered":"2024-05-08T22:37:57.441960+0000","last_unstale":"2024-05-08T22:37:57.817518+0000","last_undegraded":"2024-05-08T22:37:57.817518+0000","last_fullsized":"2024-05-08T22:37:57.817518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:59.020957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848551+0000","last_change":"2024-05-08T22:37:57.842118+0000","last_active":"2024-05-08T22:37:57.848551+0000","last_peered":"2024-05-08T22:37:57.848551+0000","last_clean":"2024-05-08T22:37:57.848551+0000","last_became_active":"2024-05-08T22:37:57.841826+0000","last_became_peered":"2024-05-08T22:37:57.841826+0000","last_unstale":"2024-05-08T22:37:57.848551+0000","last_undegraded":"2024-05-08T22:37:57.848551+0000","last_fullsized":"2024-05-08T22:37:57.848551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:55.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840125+0000","last_change":"2024-05-08T22:37:57.439634+0000","last_active":"2024-05-08T22:37:57.840125+0000","last_peered":"2024-05-08T22:37:57.840125+0000","last_clean":"2024-05-08T22:37:57.840125+0000","last_became_active":"2024-05-08T22:37:57.439387+0000","last_became_peered":"2024-05-08T22:37:57.439387+0000","last_unstale":"2024-05-08T22:37:57.840125+0000","last_undegraded":"2024-05-08T22:37:57.840125+0000","last_fullsized":"2024-05-08T22:37:57.840125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:49.158578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848777+0000","last_change":"2024-05-08T22:37:57.846904+0000","last_active":"2024-05-08T22:37:57.848777+0000","last_peered":"2024-05-08T22:37:57.848777+0000","last_clean":"2024-05-08T22:37:57.848777+0000","last_became_active":"2024-05-08T22:37:57.846651+0000","last_became_peered":"2024-05-08T22:37:57.846651+0000","last_unstale":"2024-05-08T22:37:57.848777+0000","last_undegraded":"2024-05-08T22:37:57.848777+0000","last_fullsized":"2024-05-08T22:37:57.848777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:05.340141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.821917+0000","last_change":"2024-05-08T22:37:57.443724+0000","last_active":"2024-05-08T22:37:57.821917+0000","last_peered":"2024-05-08T22:37:57.821917+0000","last_clean":"2024-05-08T22:37:57.821917+0000","last_became_active":"2024-05-08T22:37:57.443460+0000","last_became_peered":"2024-05-08T22:37:57.443460+0000","last_unstale":"2024-05-08T22:37:57.821917+0000","last_undegraded":"2024-05-08T22:37:57.821917+0000","last_fullsized":"2024-05-08T22:37:57.821917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:08.118921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.837256+0000","last_change":"2024-05-08T22:37:57.826029+0000","last_active":"2024-05-08T22:37:57.837256+0000","last_peered":"2024-05-08T22:37:57.837256+0000","last_clean":"2024-05-08T22:37:57.837256+0000","last_became_active":"2024-05-08T22:37:57.825743+0000","last_became_peered":"2024-05-08T22:37:57.825743+0000","last_unstale":"2024-05-08T22:37:57.837256+0000","last_undegraded":"2024-05-08T22:37:57.837256+0000","last_fullsized":"2024-05-08T22:37:57.837256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.587376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822152+0000","last_change":"2024-05-08T22:37:57.444844+0000","last_active":"2024-05-08T22:37:57.822152+0000","last_peered":"2024-05-08T22:37:57.822152+0000","last_clean":"2024-05-08T22:37:57.822152+0000","last_became_active":"2024-05-08T22:37:57.444488+0000","last_became_peered":"2024-05-08T22:37:57.444488+0000","last_unstale":"2024-05-08T22:37:57.822152+0000","last_undegraded":"2024-05-08T22:37:57.822152+0000","last_fullsized":"2024-05-08T22:37:57.822152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:36.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424456+0000","last_change":"2024-05-08T22:37:57.424456+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424456+0000","last_undegraded":"2024-05-08T22:37:57.424456+0000","last_fullsized":"2024-05-08T22:37:57.424456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838790+0000","last_change":"2024-05-08T22:37:57.438922+0000","last_active":"2024-05-08T22:37:57.838790+0000","last_peered":"2024-05-08T22:37:57.838790+0000","last_clean":"2024-05-08T22:37:57.838790+0000","last_became_active":"2024-05-08T22:37:57.438655+0000","last_became_peered":"2024-05-08T22:37:57.438655+0000","last_unstale":"2024-05-08T22:37:57.838790+0000","last_undegraded":"2024-05-08T22:37:57.838790+0000","last_fullsized":"2024-05-08T22:37:57.838790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:09:11.390285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838438+0000","last_change":"2024-05-08T22:37:57.828420+0000","last_active":"2024-05-08T22:37:57.838438+0000","last_peered":"2024-05-08T22:37:57.838438+0000","last_clean":"2024-05-08T22:37:57.838438+0000","last_became_active":"2024-05-08T22:37:57.828132+0000","last_became_peered":"2024-05-08T22:37:57.828132+0000","last_unstale":"2024-05-08T22:37:57.838438+0000","last_undegraded":"2024-05-08T22:37:57.838438+0000","last_fullsized":"2024-05-08T22:37:57.838438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:23.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.364530+0000","last_change":"2024-05-08T22:37:57.447723+0000","last_active":"2024-05-08T22:37:58.364530+0000","last_peered":"2024-05-08T22:37:58.364530+0000","last_clean":"2024-05-08T22:37:58.364530+0000","last_became_active":"2024-05-08T22:37:57.447478+0000","last_became_peered":"2024-05-08T22:37:57.447478+0000","last_unstale":"2024-05-08T22:37:58.364530+0000","last_undegraded":"2024-05-08T22:37:58.364530+0000","last_fullsized":"2024-05-08T22:37:58.364530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:09.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424687+0000","last_change":"2024-05-08T22:37:57.424687+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424687+0000","last_undegraded":"2024-05-08T22:37:57.424687+0000","last_fullsized":"2024-05-08T22:37:57.424687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.827351+0000","last_change":"2024-05-08T22:37:57.437575+0000","last_active":"2024-05-08T22:37:57.827351+0000","last_peered":"2024-05-08T22:37:57.827351+0000","last_clean":"2024-05-08T22:37:57.827351+0000","last_became_active":"2024-05-08T22:37:57.437308+0000","last_became_peered":"2024-05-08T22:37:57.437308+0000","last_unstale":"2024-05-08T22:37:57.827351+0000","last_undegraded":"2024-05-08T22:37:57.827351+0000","last_fullsized":"2024-05-08T22:37:57.827351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:33.154187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848732+0000","last_change":"2024-05-08T22:37:57.845583+0000","last_active":"2024-05-08T22:37:57.848732+0000","last_peered":"2024-05-08T22:37:57.848732+0000","last_clean":"2024-05-08T22:37:57.848732+0000","last_became_active":"2024-05-08T22:37:57.845321+0000","last_became_peered":"2024-05-08T22:37:57.845321+0000","last_unstale":"2024-05-08T22:37:57.848732+0000","last_undegraded":"2024-05-08T22:37:57.848732+0000","last_fullsized":"2024-05-08T22:37:57.848732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:27.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825845+0000","last_change":"2024-05-08T22:37:57.440921+0000","last_active":"2024-05-08T22:37:57.825845+0000","last_peered":"2024-05-08T22:37:57.825845+0000","last_clean":"2024-05-08T22:37:57.825845+0000","last_became_active":"2024-05-08T22:37:57.440652+0000","last_became_peered":"2024-05-08T22:37:57.440652+0000","last_unstale":"2024-05-08T22:37:57.825845+0000","last_undegraded":"2024-05-08T22:37:57.825845+0000","last_fullsized":"2024-05-08T22:37:57.825845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:02.665256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838442+0000","last_change":"2024-05-08T22:37:57.828599+0000","last_active":"2024-05-08T22:37:57.838442+0000","last_peered":"2024-05-08T22:37:57.838442+0000","last_clean":"2024-05-08T22:37:57.838442+0000","last_became_active":"2024-05-08T22:37:57.828254+0000","last_became_peered":"2024-05-08T22:37:57.828254+0000","last_unstale":"2024-05-08T22:37:57.838442+0000","last_undegraded":"2024-05-08T22:37:57.838442+0000","last_fullsized":"2024-05-08T22:37:57.838442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:30.500016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822000+0000","last_change":"2024-05-08T22:37:57.442291+0000","last_active":"2024-05-08T22:37:57.822000+0000","last_peered":"2024-05-08T22:37:57.822000+0000","last_clean":"2024-05-08T22:37:57.822000+0000","last_became_active":"2024-05-08T22:37:57.441935+0000","last_became_peered":"2024-05-08T22:37:57.441935+0000","last_unstale":"2024-05-08T22:37:57.822000+0000","last_undegraded":"2024-05-08T22:37:57.822000+0000","last_fullsized":"2024-05-08T22:37:57.822000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:55.703307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839361+0000","last_change":"2024-05-08T22:37:57.832660+0000","last_active":"2024-05-08T22:37:57.839361+0000","last_peered":"2024-05-08T22:37:57.839361+0000","last_clean":"2024-05-08T22:37:57.839361+0000","last_became_active":"2024-05-08T22:37:57.832291+0000","last_became_peered":"2024-05-08T22:37:57.832291+0000","last_unstale":"2024-05-08T22:37:57.839361+0000","last_undegraded":"2024-05-08T22:37:57.839361+0000","last_fullsized":"2024-05-08T22:37:57.839361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:40.277738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.830600+0000","last_change":"2024-05-08T22:37:57.448069+0000","last_active":"2024-05-08T22:37:57.830600+0000","last_peered":"2024-05-08T22:37:57.830600+0000","last_clean":"2024-05-08T22:37:57.830600+0000","last_became_active":"2024-05-08T22:37:57.446523+0000","last_became_peered":"2024-05-08T22:37:57.446523+0000","last_unstale":"2024-05-08T22:37:57.830600+0000","last_undegraded":"2024-05-08T22:37:57.830600+0000","last_fullsized":"2024-05-08T22:37:57.830600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:44.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836038+0000","last_change":"2024-05-08T22:37:57.830871+0000","last_active":"2024-05-08T22:37:57.836038+0000","last_peered":"2024-05-08T22:37:57.836038+0000","last_clean":"2024-05-08T22:37:57.836038+0000","last_became_active":"2024-05-08T22:37:57.830594+0000","last_became_peered":"2024-05-08T22:37:57.830594+0000","last_unstale":"2024-05-08T22:37:57.836038+0000","last_undegraded":"2024-05-08T22:37:57.836038+0000","last_fullsized":"2024-05-08T22:37:57.836038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:02.234727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.823044+0000","last_change":"2024-05-08T22:37:57.438169+0000","last_active":"2024-05-08T22:37:57.823044+0000","last_peered":"2024-05-08T22:37:57.823044+0000","last_clean":"2024-05-08T22:37:57.823044+0000","last_became_active":"2024-05-08T22:37:57.437880+0000","last_became_peered":"2024-05-08T22:37:57.437880+0000","last_unstale":"2024-05-08T22:37:57.823044+0000","last_undegraded":"2024-05-08T22:37:57.823044+0000","last_fullsized":"2024-05-08T22:37:57.823044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:58.133761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850749+0000","last_change":"2024-05-08T22:37:57.846104+0000","last_active":"2024-05-08T22:37:57.850749+0000","last_peered":"2024-05-08T22:37:57.850749+0000","last_clean":"2024-05-08T22:37:57.850749+0000","last_became_active":"2024-05-08T22:37:57.845848+0000","last_became_peered":"2024-05-08T22:37:57.845848+0000","last_unstale":"2024-05-08T22:37:57.850749+0000","last_undegraded":"2024-05-08T22:37:57.850749+0000","last_fullsized":"2024-05-08T22:37:57.850749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:06:32.854193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817725+0000","last_change":"2024-05-08T22:37:57.437069+0000","last_active":"2024-05-08T22:37:57.817725+0000","last_peered":"2024-05-08T22:37:57.817725+0000","last_clean":"2024-05-08T22:37:57.817725+0000","last_became_active":"2024-05-08T22:37:57.436461+0000","last_became_peered":"2024-05-08T22:37:57.436461+0000","last_unstale":"2024-05-08T22:37:57.817725+0000","last_undegraded":"2024-05-08T22:37:57.817725+0000","last_fullsized":"2024-05-08T22:37:57.817725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:31.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845596+0000","last_change":"2024-05-08T22:37:57.843298+0000","last_active":"2024-05-08T22:37:57.845596+0000","last_peered":"2024-05-08T22:37:57.845596+0000","last_clean":"2024-05-08T22:37:57.845596+0000","last_became_active":"2024-05-08T22:37:57.842988+0000","last_became_peered":"2024-05-08T22:37:57.842988+0000","last_unstale":"2024-05-08T22:37:57.845596+0000","last_undegraded":"2024-05-08T22:37:57.845596+0000","last_fullsized":"2024-05-08T22:37:57.845596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:17.302357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825542+0000","last_change":"2024-05-08T22:37:57.444385+0000","last_active":"2024-05-08T22:37:57.825542+0000","last_peered":"2024-05-08T22:37:57.825542+0000","last_clean":"2024-05-08T22:37:57.825542+0000","last_became_active":"2024-05-08T22:37:57.444124+0000","last_became_peered":"2024-05-08T22:37:57.444124+0000","last_unstale":"2024-05-08T22:37:57.825542+0000","last_undegraded":"2024-05-08T22:37:57.825542+0000","last_fullsized":"2024-05-08T22:37:57.825542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:35.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.429399+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:37:57.429399+0000","last_peered":"2024-05-08T22:37:57.429399+0000","last_clean":"2024-05-08T22:37:57.429399+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:37:57.429399+0000","last_undegraded":"2024-05-08T22:37:57.429399+0000","last_fullsized":"2024-05-08T22:37:57.429399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":999424,"data_stored":451081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.309}]},{"osd":1,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.47599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89100000000000001,"5min":1.101,"15min":1.101},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.627}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.018,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.57899999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1140000000000001,"5min":1.306,"15min":1.306},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2310000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.274,"5min":1.339,"15min":1.339},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.0089999999999999,"15min":1.0269999999999999},"min":{"1min":0.76000000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2749999999999999,"5min":1.476,"15min":1.476},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98199999999999998,"15min":0.97199999999999998},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2370000000000001,"5min":1.446,"15min":1.446},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.753,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.341,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.105},"min":{"1min":0.82199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.298,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.056},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.036,"15min":1.034},"min":{"1min":0.81000000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4570000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.88400000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":450611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":1.04,"15min":1.04},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.58399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2130000000000001,"5min":1.583,"15min":1.583},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2050000000000001,"5min":1.431,"15min":1.431},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.81399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2470000000000001,"5min":1.508,"15min":1.508},"last":0.747}]},{"osd":5,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.028,"15min":1.083},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.397,"5min":1.726,"15min":1.726},"last":0.96999999999999997}]},{"osd":6,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.77200000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.446,"5min":1.659,"15min":1.659},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.071},"min":{"1min":0.85399999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3129999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0569999999999999},"min":{"1min":0.81699999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.409,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85599999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1593344,"data_stored":1040889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85499999999999998,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2749999999999999,"5min":1.282,"15min":1.282},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.84099999999999997,"15min":0.85999999999999999},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.121}]},{"osd":1,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91800000000000004,"15min":0.91000000000000003},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.099,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1140000000000001,"5min":1.419,"15min":1.419},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95099999999999996,"15min":0.96199999999999997},"min":{"1min":0.69199999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.337,"15min":1.337},"last":1.022}]},{"osd":3,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.79400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5269999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.72099999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.177},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.89300000000000002},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.103,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95999999999999996,"15min":0.98199999999999998},"min":{"1min":0.67000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.482,"15min":1.482},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.85399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.264}]},{"osd":6,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.76500000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4179999999999999,"5min":1.599,"15min":1.599},"last":1.0800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":1007616,"data_stored":452063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.141,"15min":1.141},"last":0.997}]},{"osd":1,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3300000000000001,"5min":1.758,"15min":1.758},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91800000000000004,"15min":0.93000000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1850000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.115}]},{"osd":2,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.92900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.397,"5min":1.419,"15min":1.419},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.034,"15min":1.0960000000000001},"min":{"1min":0.68000000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2110000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.99099999999999999},"min":{"1min":0.625,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2869999999999999,"5min":1.538,"15min":1.538},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":1.0469999999999999},"min":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.149,"5min":1.458,"15min":1.458},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.28,"5min":1.681,"15min":1.681},"last":1.194},{"interface":"front","average":{"1min":0.998,"5min":1.0840000000000001,"15min":1.1399999999999999},"min":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2050000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.014,"15min":1.085},"min":{"1min":0.71899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0780000000000001,"5min":1.835,"15min":1.835},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.179,"15min":1.2350000000000001},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3300000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1559999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":999424,"data_stored":450533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82399999999999995,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.145,"5min":1.52,"15min":1.52},"last":0.79000000000000004}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.016,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.266,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90400000000000003,"15min":0.88},"min":{"1min":0.71899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.83999999999999997},"min":{"1min":0.67600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.204}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":0.96499999999999997},"min":{"1min":0.71599999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.70099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.016,"15min":0.98599999999999999},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96799999999999997,"15min":0.91600000000000004},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2589999999999999,"5min":1.605,"15min":1.605},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.069,"15min":1.044},"min":{"1min":0.89200000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3080000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.3340000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.0880000000000001},"min":{"1min":0.98299999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.411,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.133,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.91500000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3089999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2909999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":970752,"data_stored":436809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.113,"5min":1.28,"15min":1.28},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.294,"15min":1.294},"last":0.72299999999999998}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.871},"min":{"1min":0.65800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.163,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90700000000000003,"15min":0.88600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.173,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0329999999999999,"5min":1.29,"15min":1.29},"last":1.224},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91900000000000004,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.135,"5min":1.637,"15min":1.637},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1619999999999999,"5min":2.601,"15min":2.601},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.032,"15min":1.105},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.998}]},{"osd":5,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.0349999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.121,"5min":1.792,"15min":1.792},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.026},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.137,"5min":1.575,"15min":1.575},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0489999999999999,"15min":1.0900000000000001},"min":{"1min":0.69399999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1100000000000001,"5min":1.831,"15min":1.831},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.093,"15min":1.1020000000000001},"min":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2210000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0609999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.071,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.23,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.159,"15min":1.177},"min":{"1min":0.878,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3,"5min":1.944,"15min":1.944},"last":1.085}]}]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":443404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.133,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":1.008,"15min":1.008},"last":0.879},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.58799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.333,"5min":1.363,"15min":1.363},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.006,"15min":1.028},"min":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.27,"5min":1.282,"15min":1.282},"last":1.2609999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.407,"5min":1.484,"15min":1.484},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.3}]},{"osd":6,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0600000000000001},"min":{"1min":0.84499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0700000000000001,"15min":1.0840000000000001},"min":{"1min":0.871,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4450000000000001,"5min":1.673,"15min":1.673},"last":1.101}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.119,"15min":1.1299999999999999},"min":{"1min":0.878,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.056}]}]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343276,"statfs":{"total":96636764160,"available":96607514624,"internally_reserved":0,"allocated":1576960,"data_stored":1033772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.617,"15min":1.617},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":1.446,"15min":1.446},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0129999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3089999999999999,"5min":1.714,"15min":1.714},"last":1.609},{"interface":"front","average":{"1min":0.755,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2569999999999999,"5min":1.327,"15min":1.327},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2709999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":4.391},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.73499999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3,"5min":1.669,"15min":1.669},"last":2.0590000000000002}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2210000000000001,"5min":1.474,"15min":1.474},"last":1.0760000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.96499999999999997},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2869999999999999,"5min":1.397,"15min":1.397},"last":1.8660000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":1.0469999999999999},"min":{"1min":0.78600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.177,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":4.4859999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0649999999999999,"15min":1.089},"min":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.234,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.532},{"interface":"front","average":{"1min":1.018,"5min":1.008,"15min":1.022},"min":{"1min":0.77500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.304}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.430120+0000', 'last_change': '2024-05-08T22:37:57.430120+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.430120+0000', 'last_undegraded': '2024-05-08T22:37:57.430120+0000', 'last_fullsized': '2024-05-08T22:37:57.430120+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.429934+0000', 'last_change': '2024-05-08T22:37:57.429934+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.429934+0000', 'last_undegraded': '2024-05-08T22:37:57.429934+0000', 'last_fullsized': '2024-05-08T22:37:57.429934+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.424461+0000', 'last_change': '2024-05-08T22:37:57.424461+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.424461+0000', 'last_undegraded': '2024-05-08T22:37:57.424461+0000', 'last_fullsized': '2024-05-08T22:37:57.424461+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.425803+0000', 'last_change': '2024-05-08T22:37:57.425803+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.425803+0000', 'last_undegraded': '2024-05-08T22:37:57.425803+0000', 'last_fullsized': '2024-05-08T22:37:57.425803+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.429477+0000', 'last_change': '2024-05-08T22:37:57.429477+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.429477+0000', 'last_undegraded': '2024-05-08T22:37:57.429477+0000', 'last_fullsized': '2024-05-08T22:37:57.429477+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.426045+0000', 'last_change': '2024-05-08T22:37:57.426045+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.426045+0000', 'last_undegraded': '2024-05-08T22:37:57.426045+0000', 'last_fullsized': '2024-05-08T22:37:57.426045+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.424456+0000', 'last_change': '2024-05-08T22:37:57.424456+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.424456+0000', 'last_undegraded': '2024-05-08T22:37:57.424456+0000', 'last_fullsized': '2024-05-08T22:37:57.424456+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-05-08T22:38:02.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2024-05-08T22:37:57.424687+0000', 'last_change': '2024-05-08T22:37:57.424687+0000', 'last_active': '2024-05-08T22:37:57.410053+0000', 'last_peered': '2024-05-08T22:37:57.410053+0000', 'last_clean': '2024-05-08T22:37:57.410053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T22:37:57.424687+0000', 'last_undegraded': '2024-05-08T22:37:57.424687+0000', 'last_fullsized': '2024-05-08T22:37:57.424687+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'last_clean_scrub_stamp': '2024-05-08T22:37:57.410053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T22:38:02.515 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T22:38:02.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 Processor -- start 2024-05-08T22:38:02.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 -- start start 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7da00e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7da0006150 0x7f7da00e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00e49a0 con 0x7f7da00e0250 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da00540b0 con 0x7f7da0006150 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daecb6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00532f0 con 0x7f7da00e3b40 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7da00e0250 0x7f7da0038ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57064/0 (socket says 172.21.15.179:57064) 2024-05-08T22:38:02.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 -- 172.21.15.179:0/2041016091 learned_addr learned my addr 172.21.15.179:0/2041016091 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7da00e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dae4b5700 1 --2- 172.21.15.179:0/2041016091 >> v2:172.21.15.195:3300/0 conn(0x7f7da0006150 0x7f7da00e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144040793 0 0) 0x7f7da00e49a0 con 0x7f7da00e0250 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d940035d0 con 0x7f7da00e0250 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109561915 0 0) 0x7f7d940035d0 con 0x7f7da00e0250 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 msgr2=0x7f7da00e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7da00e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 >> v2:172.21.15.195:3300/0 conn(0x7f7da0006150 msgr2=0x7f7da00e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 --2- 172.21.15.179:0/2041016091 >> v2:172.21.15.195:3300/0 conn(0x7f7da0006150 0x7f7da00e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00535d0 con 0x7f7da00e0250 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f7d9c001670 con 0x7f7da00e0250 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 >> v1:172.21.15.179:6789/0 conn(0x7f7da00e0250 legacy=0x7f7da0038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94003a20 0x7f7d94003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d94004870 0x7f7d94004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7d94005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00540b0 con 0x7f7d94004870 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00532f0 con 0x7f7d94003a20 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9400bbb0 con 0x7f7da00e3b40 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d94004870 0x7f7d94004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94003a20 0x7f7d94003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dae4b5700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7d94005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 -- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 msgr2=0x7f7d94005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da00e3b40 0x7f7d94005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 -- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94003a20 msgr2=0x7f7d94003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94003a20 0x7f7d94003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.624+0000 7f7dad4b3700 1 -- 172.21.15.179:0/2041016091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940035d0 con 0x7f7d94004870 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94003a20 0x7f7d94003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:02.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d94004870 0x7f7d94004cf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c320 tx=0x7f7d940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258644690dd22b24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00ce60 con 0x7f7d94004870 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c005980 con 0x7f7d94004870 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daccb2700 1 -- 172.21.15.179:0/2041016091 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c005d10 con 0x7f7d94004870 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 -- 172.21.15.179:0/2041016091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d94004870 msgr2=0x7f7d94004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 --2- 172.21.15.179:0/2041016091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d94004870 0x7f7d94004cf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c320 tx=0x7f7d940058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7dae4b5700 1 -- 172.21.15.179:0/2041016091 reap_dead start 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 -- 172.21.15.179:0/2041016091 shutdown_connections 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 -- 172.21.15.179:0/2041016091 >> 172.21.15.179:0/2041016091 conn(0x7f7da0054b70 msgr2=0x7f7da0054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 -- 172.21.15.179:0/2041016091 shutdown_connections 2024-05-08T22:38:02.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.628+0000 7f7daecb6700 1 -- 172.21.15.179:0/2041016091 wait complete. 2024-05-08T22:38:02.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 Processor -- start 2024-05-08T22:38:02.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 -- start start 2024-05-08T22:38:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 0x7f7da00d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 0x7f7da00bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da0049690 con 0x7f7da00e2c90 2024-05-08T22:38:02.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00d4fd0 con 0x7f7d94004870 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00d4600 con 0x7f7da0049210 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58804/0 (socket says 172.21.15.179:58804) 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 -- 172.21.15.179:0/3722971527 learned_addr learned my addr 172.21.15.179:0/3722971527 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 0x7f7da00d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dae4b5700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 0x7f7da00bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 -- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 msgr2=0x7f7da00bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 0x7f7da00bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 -- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 msgr2=0x7f7da00d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 0x7f7da00d4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 -- 172.21.15.179:0/3722971527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00e3550 con 0x7f7da00e2c90 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 0x7f7da00d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dae4b5700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 0x7f7da00bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7dad4b3700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c910 tx=0x7f7d9c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c001660 con 0x7f7da00e2c90 2024-05-08T22:38:02.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c004aa0 con 0x7f7da00e2c90 2024-05-08T22:38:02.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da00d3cd0 con 0x7f7da00e2c90 2024-05-08T22:38:02.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.632+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004e50 con 0x7f7da00e2c90 2024-05-08T22:38:02.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da00d4290 con 0x7f7da00e2c90 2024-05-08T22:38:02.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da00d4420 con 0x7f7da00e2c90 2024-05-08T22:38:02.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d9c0070d0 con 0x7f7da00e2c90 2024-05-08T22:38:02.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7d9a7fc700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 0x7f7d8c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:02.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d9c085800 con 0x7f7da00e2c90 2024-05-08T22:38:02.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 0x7f7d8c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:02.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.636+0000 7f7dadcb4700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 0x7f7d8c068c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009b60 tx=0x7f7da4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:02.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.648+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7d9c052be0 con 0x7f7da00e2c90 2024-05-08T22:38:02.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.876+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7da0005440 con 0x7f7da00e2c90 2024-05-08T22:38:02.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.876+0000 7f7d9a7fc700 1 -- 172.21.15.179:0/3722971527 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1261 (secure 0 0 0) 0x7f7d9c007dc0 con 0x7f7da00e2c90 2024-05-08T22:38:02.880 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:02.881 INFO:teuthology.orchestra.run.smithi179.stdout:{"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":303,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1715207588,"num_in_osds":8,"osd_in_since":1715207580,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":259895296,"bytes_avail":772834217984,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2828,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":39,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":56162}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-05-08T22:37:21.465090+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T22:38:02.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 msgr2=0x7f7d8c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 0x7f7d8c068c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009b60 tx=0x7f7da4008040 comp rx=0 tx=0).stop 2024-05-08T22:38:02.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 msgr2=0x7f7da0048cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:02.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c910 tx=0x7f7d9c001a00 comp rx=0 tx=0).stop 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 shutdown_connections 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d8c066760 0x7f7d8c068c20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7da0049210 0x7f7da00bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7da00e2c90 0x7f7da0048cd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 --2- 172.21.15.179:0/3722971527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d94004870 0x7f7da00d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 >> 172.21.15.179:0/3722971527 conn(0x7f7da0054b70 msgr2=0x7f7da00df580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 shutdown_connections 2024-05-08T22:38:02.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:02.880+0000 7f7daecb6700 1 -- 172.21.15.179:0/3722971527 wait complete. 2024-05-08T22:38:02.895 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T22:38:02.895 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:38:03.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 Processor -- start 2024-05-08T22:38:03.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 -- start start 2024-05-08T22:38:03.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94e015d5b0 0x7f94e00b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e006afc0 con 0x7f94e00ce3b0 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e00ce840 con 0x7f94e015d5b0 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94dffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e015d990 con 0x7f94e00b2120 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94e015d5b0 0x7f94e00b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f94e015d5b0 0x7f94e00b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58818/0 (socket says 172.21.15.179:58818) 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 -- 172.21.15.179:0/1030131612 learned_addr learned my addr 172.21.15.179:0/1030131612 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:03.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94de7fc700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94ddffb700 1 -- 172.21.15.179:0/1030131612 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924165945 0 0) 0x7f94e006afc0 con 0x7f94e00ce3b0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94ddffb700 1 -- 172.21.15.179:0/1030131612 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c80035d0 con 0x7f94e00ce3b0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 -- 172.21.15.179:0/1030131612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 msgr2=0x7f94e00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e00cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 -- 172.21.15.179:0/1030131612 >> v1:172.21.15.195:6789/0 conn(0x7f94e00ce3b0 legacy=0x7f94e0161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 -- 172.21.15.179:0/1030131612 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e006c460 con 0x7f94e015d5b0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94deffd700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e00b1be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009910 tx=0x7f94d0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5937c4fb53ed0550 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94ddffb700 1 -- 172.21.15.179:0/1030131612 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0011830 con 0x7f94e015d5b0 2024-05-08T22:38:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94ddffb700 1 -- 172.21.15.179:0/1030131612 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d0011990 con 0x7f94e015d5b0 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.000+0000 7f94ddffb700 1 -- 172.21.15.179:0/1030131612 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0011d00 con 0x7f94e015d5b0 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- 172.21.15.179:0/1030131612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 msgr2=0x7f94e00b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e00b1be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009910 tx=0x7f94d0009c20 comp rx=0 tx=0).stop 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- 172.21.15.179:0/1030131612 shutdown_connections 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e00cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- 172.21.15.179:0/1030131612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e00b1be0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- 172.21.15.179:0/1030131612 >> 172.21.15.179:0/1030131612 conn(0x7f94e00b4940 msgr2=0x7f94e00b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- 172.21.15.179:0/1030131612 shutdown_connections 2024-05-08T22:38:03.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- 172.21.15.179:0/1030131612 wait complete. 2024-05-08T22:38:03.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 Processor -- start 2024-05-08T22:38:03.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- start start 2024-05-08T22:38:03.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94e00ce3b0 0x7f94e01586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e014b680 con 0x7f94e015d5b0 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0155890 con 0x7f94e00ce3b0 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94dffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0155b70 con 0x7f94e00b2120 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94df7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94df7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58830/0 (socket says 172.21.15.179:58830) 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94df7fe700 1 -- 172.21.15.179:0/2372273716 learned_addr learned my addr 172.21.15.179:0/2372273716 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94de7fc700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94e00ce3b0 0x7f94e01586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94deffd700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.004+0000 7f94df7fe700 1 -- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 msgr2=0x7f94e014b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94df7fe700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e014b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94df7fe700 1 -- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94e00ce3b0 msgr2=0x7f94e01586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94df7fe700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94e00ce3b0 0x7f94e01586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94df7fe700 1 -- 172.21.15.179:0/2372273716 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009570 con 0x7f94e015d5b0 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94df7fe700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f94d800cbc0 tx=0x7f94d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8003c20 con 0x7f94e015d5b0 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8003d80 con 0x7f94e015d5b0 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e0155e50 con 0x7f94e015d5b0 2024-05-08T22:38:03.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800aaa0 con 0x7f94e015d5b0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e0156410 con 0x7f94e015d5b0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e0156630 con 0x7f94e015d5b0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94d800ac00 con 0x7f94e015d5b0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 0x7f94cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f94d8084270 con 0x7f94e015d5b0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.008+0000 7f94deffd700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 0x7f94cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.012+0000 7f94deffd700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 0x7f94cc068bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94d0014040 tx=0x7f94d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.016+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f94d8051400 con 0x7f94e015d5b0 2024-05-08T22:38:03.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.184+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94e00b25f0 con 0x7f94cc066730 2024-05-08T22:38:03.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.200+0000 7f94c7fff700 1 -- 172.21.15.179:0/2372273716 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287288 (secure 0 0 0) 0x7f94e00b25f0 con 0x7f94cc066730 2024-05-08T22:38:03.210 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:03.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 msgr2=0x7f94cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 0x7f94cc068bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94d0014040 tx=0x7f94d0008030 comp rx=0 tx=0).stop 2024-05-08T22:38:03.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 msgr2=0x7f94e0158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f94d800cbc0 tx=0x7f94d800ced0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 shutdown_connections 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94cc066730 0x7f94cc068bf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94e00ce3b0 0x7f94e01586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e015d5b0 0x7f94e0158c30 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 --2- 172.21.15.179:0/2372273716 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94e00b2120 0x7f94e014b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 >> 172.21.15.179:0/2372273716 conn(0x7f94e00b4940 msgr2=0x7f94e00b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 shutdown_connections 2024-05-08T22:38:03.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:03.204+0000 7f94dffff700 1 -- 172.21.15.179:0/2372273716 wait complete. 2024-05-08T22:38:03.214 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:38:03.224 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-05-08T22:38:01.478708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253804,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754720916,"statfs":{"total":773094113280,"available":772834217984,"internally_reserved":0,"allocated":9134080,"data_stored":4759162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.067987"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848306+0000","last_change":"2024-05-08T22:37:57.844883+0000","last_active":"2024-05-08T22:37:57.848306+0000","last_peered":"2024-05-08T22:37:57.848306+0000","last_clean":"2024-05-08T22:37:57.848306+0000","last_became_active":"2024-05-08T22:37:57.844614+0000","last_became_peered":"2024-05-08T22:37:57.844614+0000","last_unstale":"2024-05-08T22:37:57.848306+0000","last_undegraded":"2024-05-08T22:37:57.848306+0000","last_fullsized":"2024-05-08T22:37:57.848306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:56:36.890701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848623+0000","last_change":"2024-05-08T22:37:57.844447+0000","last_active":"2024-05-08T22:37:57.848623+0000","last_peered":"2024-05-08T22:37:57.848623+0000","last_clean":"2024-05-08T22:37:57.848623+0000","last_became_active":"2024-05-08T22:37:57.844201+0000","last_became_peered":"2024-05-08T22:37:57.844201+0000","last_unstale":"2024-05-08T22:37:57.848623+0000","last_undegraded":"2024-05-08T22:37:57.848623+0000","last_fullsized":"2024-05-08T22:37:57.848623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:10.013238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838916+0000","last_change":"2024-05-08T22:37:57.835351+0000","last_active":"2024-05-08T22:37:57.838916+0000","last_peered":"2024-05-08T22:37:57.838916+0000","last_clean":"2024-05-08T22:37:57.838916+0000","last_became_active":"2024-05-08T22:37:57.835104+0000","last_became_peered":"2024-05-08T22:37:57.835104+0000","last_unstale":"2024-05-08T22:37:57.838916+0000","last_undegraded":"2024-05-08T22:37:57.838916+0000","last_fullsized":"2024-05-08T22:37:57.838916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844279+0000","last_change":"2024-05-08T22:37:57.842727+0000","last_active":"2024-05-08T22:37:57.844279+0000","last_peered":"2024-05-08T22:37:57.844279+0000","last_clean":"2024-05-08T22:37:57.844279+0000","last_became_active":"2024-05-08T22:37:57.842350+0000","last_became_peered":"2024-05-08T22:37:57.842350+0000","last_unstale":"2024-05-08T22:37:57.844279+0000","last_undegraded":"2024-05-08T22:37:57.844279+0000","last_fullsized":"2024-05-08T22:37:57.844279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:32.252275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848269+0000","last_change":"2024-05-08T22:37:57.846399+0000","last_active":"2024-05-08T22:37:57.848269+0000","last_peered":"2024-05-08T22:37:57.848269+0000","last_clean":"2024-05-08T22:37:57.848269+0000","last_became_active":"2024-05-08T22:37:57.845308+0000","last_became_peered":"2024-05-08T22:37:57.845308+0000","last_unstale":"2024-05-08T22:37:57.848269+0000","last_undegraded":"2024-05-08T22:37:57.848269+0000","last_fullsized":"2024-05-08T22:37:57.848269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:57.759733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850942+0000","last_change":"2024-05-08T22:37:57.847700+0000","last_active":"2024-05-08T22:37:57.850942+0000","last_peered":"2024-05-08T22:37:57.850942+0000","last_clean":"2024-05-08T22:37:57.850942+0000","last_became_active":"2024-05-08T22:37:57.847333+0000","last_became_peered":"2024-05-08T22:37:57.847333+0000","last_unstale":"2024-05-08T22:37:57.850942+0000","last_undegraded":"2024-05-08T22:37:57.850942+0000","last_fullsized":"2024-05-08T22:37:57.850942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:59.881785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839403+0000","last_change":"2024-05-08T22:37:57.832432+0000","last_active":"2024-05-08T22:37:57.839403+0000","last_peered":"2024-05-08T22:37:57.839403+0000","last_clean":"2024-05-08T22:37:57.839403+0000","last_became_active":"2024-05-08T22:37:57.832048+0000","last_became_peered":"2024-05-08T22:37:57.832048+0000","last_unstale":"2024-05-08T22:37:57.839403+0000","last_undegraded":"2024-05-08T22:37:57.839403+0000","last_fullsized":"2024-05-08T22:37:57.839403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:26.697556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848302+0000","last_change":"2024-05-08T22:37:57.846519+0000","last_active":"2024-05-08T22:37:57.848302+0000","last_peered":"2024-05-08T22:37:57.848302+0000","last_clean":"2024-05-08T22:37:57.848302+0000","last_became_active":"2024-05-08T22:37:57.845361+0000","last_became_peered":"2024-05-08T22:37:57.845361+0000","last_unstale":"2024-05-08T22:37:57.848302+0000","last_undegraded":"2024-05-08T22:37:57.848302+0000","last_fullsized":"2024-05-08T22:37:57.848302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:34.786251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.430120+0000","last_change":"2024-05-08T22:37:57.430120+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.430120+0000","last_undegraded":"2024-05-08T22:37:57.430120+0000","last_fullsized":"2024-05-08T22:37:57.430120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839006+0000","last_change":"2024-05-08T22:37:57.825872+0000","last_active":"2024-05-08T22:37:57.839006+0000","last_peered":"2024-05-08T22:37:57.839006+0000","last_clean":"2024-05-08T22:37:57.839006+0000","last_became_active":"2024-05-08T22:37:57.825591+0000","last_became_peered":"2024-05-08T22:37:57.825591+0000","last_unstale":"2024-05-08T22:37:57.839006+0000","last_undegraded":"2024-05-08T22:37:57.839006+0000","last_fullsized":"2024-05-08T22:37:57.839006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:07.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848675+0000","last_change":"2024-05-08T22:37:57.844985+0000","last_active":"2024-05-08T22:37:57.848675+0000","last_peered":"2024-05-08T22:37:57.848675+0000","last_clean":"2024-05-08T22:37:57.848675+0000","last_became_active":"2024-05-08T22:37:57.844748+0000","last_became_peered":"2024-05-08T22:37:57.844748+0000","last_unstale":"2024-05-08T22:37:57.848675+0000","last_undegraded":"2024-05-08T22:37:57.848675+0000","last_fullsized":"2024-05-08T22:37:57.848675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:02.244593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838791+0000","last_change":"2024-05-08T22:37:57.830338+0000","last_active":"2024-05-08T22:37:57.838791+0000","last_peered":"2024-05-08T22:37:57.838791+0000","last_clean":"2024-05-08T22:37:57.838791+0000","last_became_active":"2024-05-08T22:37:57.830069+0000","last_became_peered":"2024-05-08T22:37:57.830069+0000","last_unstale":"2024-05-08T22:37:57.838791+0000","last_undegraded":"2024-05-08T22:37:57.838791+0000","last_fullsized":"2024-05-08T22:37:57.838791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:53.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841042+0000","last_change":"2024-05-08T22:37:57.835141+0000","last_active":"2024-05-08T22:37:57.841042+0000","last_peered":"2024-05-08T22:37:57.841042+0000","last_clean":"2024-05-08T22:37:57.841042+0000","last_became_active":"2024-05-08T22:37:57.832615+0000","last_became_peered":"2024-05-08T22:37:57.832615+0000","last_unstale":"2024-05-08T22:37:57.841042+0000","last_undegraded":"2024-05-08T22:37:57.841042+0000","last_fullsized":"2024-05-08T22:37:57.841042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:40.282177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850914+0000","last_change":"2024-05-08T22:37:57.847622+0000","last_active":"2024-05-08T22:37:57.850914+0000","last_peered":"2024-05-08T22:37:57.850914+0000","last_clean":"2024-05-08T22:37:57.850914+0000","last_became_active":"2024-05-08T22:37:57.847294+0000","last_became_peered":"2024-05-08T22:37:57.847294+0000","last_unstale":"2024-05-08T22:37:57.850914+0000","last_undegraded":"2024-05-08T22:37:57.850914+0000","last_fullsized":"2024-05-08T22:37:57.850914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:14.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843497+0000","last_change":"2024-05-08T22:37:57.836871+0000","last_active":"2024-05-08T22:37:57.843497+0000","last_peered":"2024-05-08T22:37:57.843497+0000","last_clean":"2024-05-08T22:37:57.843497+0000","last_became_active":"2024-05-08T22:37:57.836626+0000","last_became_peered":"2024-05-08T22:37:57.836626+0000","last_unstale":"2024-05-08T22:37:57.843497+0000","last_undegraded":"2024-05-08T22:37:57.843497+0000","last_fullsized":"2024-05-08T22:37:57.843497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:22.008737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844329+0000","last_change":"2024-05-08T22:37:57.840135+0000","last_active":"2024-05-08T22:37:57.844329+0000","last_peered":"2024-05-08T22:37:57.844329+0000","last_clean":"2024-05-08T22:37:57.844329+0000","last_became_active":"2024-05-08T22:37:57.839806+0000","last_became_peered":"2024-05-08T22:37:57.839806+0000","last_unstale":"2024-05-08T22:37:57.844329+0000","last_undegraded":"2024-05-08T22:37:57.844329+0000","last_fullsized":"2024-05-08T22:37:57.844329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:53.757689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841087+0000","last_change":"2024-05-08T22:37:57.832054+0000","last_active":"2024-05-08T22:37:57.841087+0000","last_peered":"2024-05-08T22:37:57.841087+0000","last_clean":"2024-05-08T22:37:57.841087+0000","last_became_active":"2024-05-08T22:37:57.831311+0000","last_became_peered":"2024-05-08T22:37:57.831311+0000","last_unstale":"2024-05-08T22:37:57.841087+0000","last_undegraded":"2024-05-08T22:37:57.841087+0000","last_fullsized":"2024-05-08T22:37:57.841087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:27.877407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850784+0000","last_change":"2024-05-08T22:37:57.847959+0000","last_active":"2024-05-08T22:37:57.850784+0000","last_peered":"2024-05-08T22:37:57.850784+0000","last_clean":"2024-05-08T22:37:57.850784+0000","last_became_active":"2024-05-08T22:37:57.847531+0000","last_became_peered":"2024-05-08T22:37:57.847531+0000","last_unstale":"2024-05-08T22:37:57.850784+0000","last_undegraded":"2024-05-08T22:37:57.850784+0000","last_fullsized":"2024-05-08T22:37:57.850784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:15.951130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833932+0000","last_change":"2024-05-08T22:37:57.832828+0000","last_active":"2024-05-08T22:37:57.833932+0000","last_peered":"2024-05-08T22:37:57.833932+0000","last_clean":"2024-05-08T22:37:57.833932+0000","last_became_active":"2024-05-08T22:37:57.832544+0000","last_became_peered":"2024-05-08T22:37:57.832544+0000","last_unstale":"2024-05-08T22:37:57.833932+0000","last_undegraded":"2024-05-08T22:37:57.833932+0000","last_fullsized":"2024-05-08T22:37:57.833932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:57.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843557+0000","last_change":"2024-05-08T22:37:57.831045+0000","last_active":"2024-05-08T22:37:57.843557+0000","last_peered":"2024-05-08T22:37:57.843557+0000","last_clean":"2024-05-08T22:37:57.843557+0000","last_became_active":"2024-05-08T22:37:57.830745+0000","last_became_peered":"2024-05-08T22:37:57.830745+0000","last_unstale":"2024-05-08T22:37:57.843557+0000","last_undegraded":"2024-05-08T22:37:57.843557+0000","last_fullsized":"2024-05-08T22:37:57.843557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:31.214851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429934+0000","last_change":"2024-05-08T22:37:57.429934+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429934+0000","last_undegraded":"2024-05-08T22:37:57.429934+0000","last_fullsized":"2024-05-08T22:37:57.429934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846622+0000","last_change":"2024-05-08T22:37:57.844194+0000","last_active":"2024-05-08T22:37:57.846622+0000","last_peered":"2024-05-08T22:37:57.846622+0000","last_clean":"2024-05-08T22:37:57.846622+0000","last_became_active":"2024-05-08T22:37:57.841102+0000","last_became_peered":"2024-05-08T22:37:57.841102+0000","last_unstale":"2024-05-08T22:37:57.846622+0000","last_undegraded":"2024-05-08T22:37:57.846622+0000","last_fullsized":"2024-05-08T22:37:57.846622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:15.481984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424461+0000","last_change":"2024-05-08T22:37:57.424461+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424461+0000","last_undegraded":"2024-05-08T22:37:57.424461+0000","last_fullsized":"2024-05-08T22:37:57.424461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847805+0000","last_change":"2024-05-08T22:37:57.840565+0000","last_active":"2024-05-08T22:37:57.847805+0000","last_peered":"2024-05-08T22:37:57.847805+0000","last_clean":"2024-05-08T22:37:57.847805+0000","last_became_active":"2024-05-08T22:37:57.834465+0000","last_became_peered":"2024-05-08T22:37:57.834465+0000","last_unstale":"2024-05-08T22:37:57.847805+0000","last_undegraded":"2024-05-08T22:37:57.847805+0000","last_fullsized":"2024-05-08T22:37:57.847805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:23.426774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840471+0000","last_change":"2024-05-08T22:37:57.834804+0000","last_active":"2024-05-08T22:37:57.840471+0000","last_peered":"2024-05-08T22:37:57.840471+0000","last_clean":"2024-05-08T22:37:57.840471+0000","last_became_active":"2024-05-08T22:37:57.834268+0000","last_became_peered":"2024-05-08T22:37:57.834268+0000","last_unstale":"2024-05-08T22:37:57.840471+0000","last_undegraded":"2024-05-08T22:37:57.840471+0000","last_fullsized":"2024-05-08T22:37:57.840471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:10.524450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839026+0000","last_change":"2024-05-08T22:37:57.831011+0000","last_active":"2024-05-08T22:37:57.839026+0000","last_peered":"2024-05-08T22:37:57.839026+0000","last_clean":"2024-05-08T22:37:57.839026+0000","last_became_active":"2024-05-08T22:37:57.830662+0000","last_became_peered":"2024-05-08T22:37:57.830662+0000","last_unstale":"2024-05-08T22:37:57.839026+0000","last_undegraded":"2024-05-08T22:37:57.839026+0000","last_fullsized":"2024-05-08T22:37:57.839026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:33.816353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844342+0000","last_change":"2024-05-08T22:37:57.840718+0000","last_active":"2024-05-08T22:37:57.844342+0000","last_peered":"2024-05-08T22:37:57.844342+0000","last_clean":"2024-05-08T22:37:57.844342+0000","last_became_active":"2024-05-08T22:37:57.840400+0000","last_became_peered":"2024-05-08T22:37:57.840400+0000","last_unstale":"2024-05-08T22:37:57.844342+0000","last_undegraded":"2024-05-08T22:37:57.844342+0000","last_fullsized":"2024-05-08T22:37:57.844342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.153666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846441+0000","last_change":"2024-05-08T22:37:57.844109+0000","last_active":"2024-05-08T22:37:57.846441+0000","last_peered":"2024-05-08T22:37:57.846441+0000","last_clean":"2024-05-08T22:37:57.846441+0000","last_became_active":"2024-05-08T22:37:57.843582+0000","last_became_peered":"2024-05-08T22:37:57.843582+0000","last_unstale":"2024-05-08T22:37:57.846441+0000","last_undegraded":"2024-05-08T22:37:57.846441+0000","last_fullsized":"2024-05-08T22:37:57.846441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:58.974111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847790+0000","last_change":"2024-05-08T22:37:57.841963+0000","last_active":"2024-05-08T22:37:57.847790+0000","last_peered":"2024-05-08T22:37:57.847790+0000","last_clean":"2024-05-08T22:37:57.847790+0000","last_became_active":"2024-05-08T22:37:57.841575+0000","last_became_peered":"2024-05-08T22:37:57.841575+0000","last_unstale":"2024-05-08T22:37:57.847790+0000","last_undegraded":"2024-05-08T22:37:57.847790+0000","last_fullsized":"2024-05-08T22:37:57.847790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:33.257259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845819+0000","last_change":"2024-05-08T22:37:57.840274+0000","last_active":"2024-05-08T22:37:57.845819+0000","last_peered":"2024-05-08T22:37:57.845819+0000","last_clean":"2024-05-08T22:37:57.845819+0000","last_became_active":"2024-05-08T22:37:57.840011+0000","last_became_peered":"2024-05-08T22:37:57.840011+0000","last_unstale":"2024-05-08T22:37:57.845819+0000","last_undegraded":"2024-05-08T22:37:57.845819+0000","last_fullsized":"2024-05-08T22:37:57.845819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:45.626643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.832439+0000","last_change":"2024-05-08T22:37:57.827795+0000","last_active":"2024-05-08T22:37:57.832439+0000","last_peered":"2024-05-08T22:37:57.832439+0000","last_clean":"2024-05-08T22:37:57.832439+0000","last_became_active":"2024-05-08T22:37:57.827400+0000","last_became_peered":"2024-05-08T22:37:57.827400+0000","last_unstale":"2024-05-08T22:37:57.832439+0000","last_undegraded":"2024-05-08T22:37:57.832439+0000","last_fullsized":"2024-05-08T22:37:57.832439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:31.579840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848125+0000","last_change":"2024-05-08T22:37:57.844868+0000","last_active":"2024-05-08T22:37:57.848125+0000","last_peered":"2024-05-08T22:37:57.848125+0000","last_clean":"2024-05-08T22:37:57.848125+0000","last_became_active":"2024-05-08T22:37:57.844575+0000","last_became_peered":"2024-05-08T22:37:57.844575+0000","last_unstale":"2024-05-08T22:37:57.848125+0000","last_undegraded":"2024-05-08T22:37:57.848125+0000","last_fullsized":"2024-05-08T22:37:57.848125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850774+0000","last_change":"2024-05-08T22:37:57.847858+0000","last_active":"2024-05-08T22:37:57.850774+0000","last_peered":"2024-05-08T22:37:57.850774+0000","last_clean":"2024-05-08T22:37:57.850774+0000","last_became_active":"2024-05-08T22:37:57.847609+0000","last_became_peered":"2024-05-08T22:37:57.847609+0000","last_unstale":"2024-05-08T22:37:57.850774+0000","last_undegraded":"2024-05-08T22:37:57.850774+0000","last_fullsized":"2024-05-08T22:37:57.850774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:00.575919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844834+0000","last_change":"2024-05-08T22:37:57.446359+0000","last_active":"2024-05-08T22:37:57.844834+0000","last_peered":"2024-05-08T22:37:57.844834+0000","last_clean":"2024-05-08T22:37:57.844834+0000","last_became_active":"2024-05-08T22:37:57.446036+0000","last_became_peered":"2024-05-08T22:37:57.446036+0000","last_unstale":"2024-05-08T22:37:57.844834+0000","last_undegraded":"2024-05-08T22:37:57.844834+0000","last_fullsized":"2024-05-08T22:37:57.844834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:00.419625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847776+0000","last_change":"2024-05-08T22:37:57.842007+0000","last_active":"2024-05-08T22:37:57.847776+0000","last_peered":"2024-05-08T22:37:57.847776+0000","last_clean":"2024-05-08T22:37:57.847776+0000","last_became_active":"2024-05-08T22:37:57.841764+0000","last_became_peered":"2024-05-08T22:37:57.841764+0000","last_unstale":"2024-05-08T22:37:57.847776+0000","last_undegraded":"2024-05-08T22:37:57.847776+0000","last_fullsized":"2024-05-08T22:37:57.847776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:58.771407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842515+0000","last_change":"2024-05-08T22:37:57.440281+0000","last_active":"2024-05-08T22:37:57.842515+0000","last_peered":"2024-05-08T22:37:57.842515+0000","last_clean":"2024-05-08T22:37:57.842515+0000","last_became_active":"2024-05-08T22:37:57.439987+0000","last_became_peered":"2024-05-08T22:37:57.439987+0000","last_unstale":"2024-05-08T22:37:57.842515+0000","last_undegraded":"2024-05-08T22:37:57.842515+0000","last_fullsized":"2024-05-08T22:37:57.842515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:32.804508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.425803+0000","last_change":"2024-05-08T22:37:57.425803+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.425803+0000","last_undegraded":"2024-05-08T22:37:57.425803+0000","last_fullsized":"2024-05-08T22:37:57.425803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846799+0000","last_change":"2024-05-08T22:37:57.448745+0000","last_active":"2024-05-08T22:37:57.846799+0000","last_peered":"2024-05-08T22:37:57.846799+0000","last_clean":"2024-05-08T22:37:57.846799+0000","last_became_active":"2024-05-08T22:37:57.448495+0000","last_became_peered":"2024-05-08T22:37:57.448495+0000","last_unstale":"2024-05-08T22:37:57.846799+0000","last_undegraded":"2024-05-08T22:37:57.846799+0000","last_fullsized":"2024-05-08T22:37:57.846799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:52.343924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.429477+0000","last_change":"2024-05-08T22:37:57.429477+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.429477+0000","last_undegraded":"2024-05-08T22:37:57.429477+0000","last_fullsized":"2024-05-08T22:37:57.429477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848978+0000","last_change":"2024-05-08T22:37:57.442124+0000","last_active":"2024-05-08T22:37:57.848978+0000","last_peered":"2024-05-08T22:37:57.848978+0000","last_clean":"2024-05-08T22:37:57.848978+0000","last_became_active":"2024-05-08T22:37:57.441844+0000","last_became_peered":"2024-05-08T22:37:57.441844+0000","last_unstale":"2024-05-08T22:37:57.848978+0000","last_undegraded":"2024-05-08T22:37:57.848978+0000","last_fullsized":"2024-05-08T22:37:57.848978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:12.294211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848681+0000","last_change":"2024-05-08T22:37:57.845013+0000","last_active":"2024-05-08T22:37:57.848681+0000","last_peered":"2024-05-08T22:37:57.848681+0000","last_clean":"2024-05-08T22:37:57.848681+0000","last_became_active":"2024-05-08T22:37:57.844721+0000","last_became_peered":"2024-05-08T22:37:57.844721+0000","last_unstale":"2024-05-08T22:37:57.848681+0000","last_undegraded":"2024-05-08T22:37:57.848681+0000","last_fullsized":"2024-05-08T22:37:57.848681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.527397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439079+0000","last_change":"2024-05-08T22:37:57.438281+0000","last_active":"2024-05-08T22:37:57.439079+0000","last_peered":"2024-05-08T22:37:57.439079+0000","last_clean":"2024-05-08T22:37:57.439079+0000","last_became_active":"2024-05-08T22:37:57.438016+0000","last_became_peered":"2024-05-08T22:37:57.438016+0000","last_unstale":"2024-05-08T22:37:57.439079+0000","last_undegraded":"2024-05-08T22:37:57.439079+0000","last_fullsized":"2024-05-08T22:37:57.439079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:22.432349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848718+0000","last_change":"2024-05-08T22:37:57.845395+0000","last_active":"2024-05-08T22:37:57.848718+0000","last_peered":"2024-05-08T22:37:57.848718+0000","last_clean":"2024-05-08T22:37:57.848718+0000","last_became_active":"2024-05-08T22:37:57.845129+0000","last_became_peered":"2024-05-08T22:37:57.845129+0000","last_unstale":"2024-05-08T22:37:57.848718+0000","last_undegraded":"2024-05-08T22:37:57.848718+0000","last_fullsized":"2024-05-08T22:37:57.848718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:26.794424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838895+0000","last_change":"2024-05-08T22:37:57.436502+0000","last_active":"2024-05-08T22:37:57.838895+0000","last_peered":"2024-05-08T22:37:57.838895+0000","last_clean":"2024-05-08T22:37:57.838895+0000","last_became_active":"2024-05-08T22:37:57.436246+0000","last_became_peered":"2024-05-08T22:37:57.436246+0000","last_unstale":"2024-05-08T22:37:57.838895+0000","last_undegraded":"2024-05-08T22:37:57.838895+0000","last_fullsized":"2024-05-08T22:37:57.838895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:07.871185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.426045+0000","last_change":"2024-05-08T22:37:57.426045+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.426045+0000","last_undegraded":"2024-05-08T22:37:57.426045+0000","last_fullsized":"2024-05-08T22:37:57.426045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826353+0000","last_change":"2024-05-08T22:37:57.443165+0000","last_active":"2024-05-08T22:37:57.826353+0000","last_peered":"2024-05-08T22:37:57.826353+0000","last_clean":"2024-05-08T22:37:57.826353+0000","last_became_active":"2024-05-08T22:37:57.442883+0000","last_became_peered":"2024-05-08T22:37:57.442883+0000","last_unstale":"2024-05-08T22:37:57.826353+0000","last_undegraded":"2024-05-08T22:37:57.826353+0000","last_fullsized":"2024-05-08T22:37:57.826353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:43.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849717+0000","last_change":"2024-05-08T22:37:57.840713+0000","last_active":"2024-05-08T22:37:57.849717+0000","last_peered":"2024-05-08T22:37:57.849717+0000","last_clean":"2024-05-08T22:37:57.849717+0000","last_became_active":"2024-05-08T22:37:57.839777+0000","last_became_peered":"2024-05-08T22:37:57.839777+0000","last_unstale":"2024-05-08T22:37:57.849717+0000","last_undegraded":"2024-05-08T22:37:57.849717+0000","last_fullsized":"2024-05-08T22:37:57.849717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:09.133783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.828973+0000","last_change":"2024-05-08T22:37:57.445418+0000","last_active":"2024-05-08T22:37:57.828973+0000","last_peered":"2024-05-08T22:37:57.828973+0000","last_clean":"2024-05-08T22:37:57.828973+0000","last_became_active":"2024-05-08T22:37:57.445005+0000","last_became_peered":"2024-05-08T22:37:57.445005+0000","last_unstale":"2024-05-08T22:37:57.828973+0000","last_undegraded":"2024-05-08T22:37:57.828973+0000","last_fullsized":"2024-05-08T22:37:57.828973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:01.056718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848795+0000","last_change":"2024-05-08T22:37:57.846487+0000","last_active":"2024-05-08T22:37:57.848795+0000","last_peered":"2024-05-08T22:37:57.848795+0000","last_clean":"2024-05-08T22:37:57.848795+0000","last_became_active":"2024-05-08T22:37:57.845715+0000","last_became_peered":"2024-05-08T22:37:57.845715+0000","last_unstale":"2024-05-08T22:37:57.848795+0000","last_undegraded":"2024-05-08T22:37:57.848795+0000","last_fullsized":"2024-05-08T22:37:57.848795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:56.850979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829513+0000","last_change":"2024-05-08T22:37:57.442931+0000","last_active":"2024-05-08T22:37:57.829513+0000","last_peered":"2024-05-08T22:37:57.829513+0000","last_clean":"2024-05-08T22:37:57.829513+0000","last_became_active":"2024-05-08T22:37:57.441198+0000","last_became_peered":"2024-05-08T22:37:57.441198+0000","last_unstale":"2024-05-08T22:37:57.829513+0000","last_undegraded":"2024-05-08T22:37:57.829513+0000","last_fullsized":"2024-05-08T22:37:57.829513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:35.216047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844949+0000","last_change":"2024-05-08T22:37:57.844158+0000","last_active":"2024-05-08T22:37:57.844949+0000","last_peered":"2024-05-08T22:37:57.844949+0000","last_clean":"2024-05-08T22:37:57.844949+0000","last_became_active":"2024-05-08T22:37:57.843855+0000","last_became_peered":"2024-05-08T22:37:57.843855+0000","last_unstale":"2024-05-08T22:37:57.844949+0000","last_undegraded":"2024-05-08T22:37:57.844949+0000","last_fullsized":"2024-05-08T22:37:57.844949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:36.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822311+0000","last_change":"2024-05-08T22:37:57.439889+0000","last_active":"2024-05-08T22:37:57.822311+0000","last_peered":"2024-05-08T22:37:57.822311+0000","last_clean":"2024-05-08T22:37:57.822311+0000","last_became_active":"2024-05-08T22:37:57.439633+0000","last_became_peered":"2024-05-08T22:37:57.439633+0000","last_unstale":"2024-05-08T22:37:57.822311+0000","last_undegraded":"2024-05-08T22:37:57.822311+0000","last_fullsized":"2024-05-08T22:37:57.822311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:01.221867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839271+0000","last_change":"2024-05-08T22:37:57.833440+0000","last_active":"2024-05-08T22:37:57.839271+0000","last_peered":"2024-05-08T22:37:57.839271+0000","last_clean":"2024-05-08T22:37:57.839271+0000","last_became_active":"2024-05-08T22:37:57.833212+0000","last_became_peered":"2024-05-08T22:37:57.833212+0000","last_unstale":"2024-05-08T22:37:57.839271+0000","last_undegraded":"2024-05-08T22:37:57.839271+0000","last_fullsized":"2024-05-08T22:37:57.839271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:08.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846625+0000","last_change":"2024-05-08T22:37:57.444957+0000","last_active":"2024-05-08T22:37:57.846625+0000","last_peered":"2024-05-08T22:37:57.846625+0000","last_clean":"2024-05-08T22:37:57.846625+0000","last_became_active":"2024-05-08T22:37:57.444484+0000","last_became_peered":"2024-05-08T22:37:57.444484+0000","last_unstale":"2024-05-08T22:37:57.846625+0000","last_undegraded":"2024-05-08T22:37:57.846625+0000","last_fullsized":"2024-05-08T22:37:57.846625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:46.229790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836275+0000","last_change":"2024-05-08T22:37:57.830761+0000","last_active":"2024-05-08T22:37:57.836275+0000","last_peered":"2024-05-08T22:37:57.836275+0000","last_clean":"2024-05-08T22:37:57.836275+0000","last_became_active":"2024-05-08T22:37:57.830491+0000","last_became_peered":"2024-05-08T22:37:57.830491+0000","last_unstale":"2024-05-08T22:37:57.836275+0000","last_undegraded":"2024-05-08T22:37:57.836275+0000","last_fullsized":"2024-05-08T22:37:57.836275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:12.683143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849871+0000","last_change":"2024-05-08T22:37:57.439248+0000","last_active":"2024-05-08T22:37:57.849871+0000","last_peered":"2024-05-08T22:37:57.849871+0000","last_clean":"2024-05-08T22:37:57.849871+0000","last_became_active":"2024-05-08T22:37:57.438927+0000","last_became_peered":"2024-05-08T22:37:57.438927+0000","last_unstale":"2024-05-08T22:37:57.849871+0000","last_undegraded":"2024-05-08T22:37:57.849871+0000","last_fullsized":"2024-05-08T22:37:57.849871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:11.645499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838819+0000","last_change":"2024-05-08T22:37:57.835299+0000","last_active":"2024-05-08T22:37:57.838819+0000","last_peered":"2024-05-08T22:37:57.838819+0000","last_clean":"2024-05-08T22:37:57.838819+0000","last_became_active":"2024-05-08T22:37:57.834959+0000","last_became_peered":"2024-05-08T22:37:57.834959+0000","last_unstale":"2024-05-08T22:37:57.838819+0000","last_undegraded":"2024-05-08T22:37:57.838819+0000","last_fullsized":"2024-05-08T22:37:57.838819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:19.550971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848959+0000","last_change":"2024-05-08T22:37:57.436168+0000","last_active":"2024-05-08T22:37:57.848959+0000","last_peered":"2024-05-08T22:37:57.848959+0000","last_clean":"2024-05-08T22:37:57.848959+0000","last_became_active":"2024-05-08T22:37:57.435911+0000","last_became_peered":"2024-05-08T22:37:57.435911+0000","last_unstale":"2024-05-08T22:37:57.848959+0000","last_undegraded":"2024-05-08T22:37:57.848959+0000","last_fullsized":"2024-05-08T22:37:57.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:39.531241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839382+0000","last_change":"2024-05-08T22:37:57.834557+0000","last_active":"2024-05-08T22:37:57.839382+0000","last_peered":"2024-05-08T22:37:57.839382+0000","last_clean":"2024-05-08T22:37:57.839382+0000","last_became_active":"2024-05-08T22:37:57.834336+0000","last_became_peered":"2024-05-08T22:37:57.834336+0000","last_unstale":"2024-05-08T22:37:57.839382+0000","last_undegraded":"2024-05-08T22:37:57.839382+0000","last_fullsized":"2024-05-08T22:37:57.839382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:07.977190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826416+0000","last_change":"2024-05-08T22:37:57.442582+0000","last_active":"2024-05-08T22:37:57.826416+0000","last_peered":"2024-05-08T22:37:57.826416+0000","last_clean":"2024-05-08T22:37:57.826416+0000","last_became_active":"2024-05-08T22:37:57.440956+0000","last_became_peered":"2024-05-08T22:37:57.440956+0000","last_unstale":"2024-05-08T22:37:57.826416+0000","last_undegraded":"2024-05-08T22:37:57.826416+0000","last_fullsized":"2024-05-08T22:37:57.826416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:40.041454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.849792+0000","last_change":"2024-05-08T22:37:57.846329+0000","last_active":"2024-05-08T22:37:57.849792+0000","last_peered":"2024-05-08T22:37:57.849792+0000","last_clean":"2024-05-08T22:37:57.849792+0000","last_became_active":"2024-05-08T22:37:57.846056+0000","last_became_peered":"2024-05-08T22:37:57.846056+0000","last_unstale":"2024-05-08T22:37:57.849792+0000","last_undegraded":"2024-05-08T22:37:57.849792+0000","last_fullsized":"2024-05-08T22:37:57.849792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.829433+0000","last_change":"2024-05-08T22:37:57.448219+0000","last_active":"2024-05-08T22:37:57.829433+0000","last_peered":"2024-05-08T22:37:57.829433+0000","last_clean":"2024-05-08T22:37:57.829433+0000","last_became_active":"2024-05-08T22:37:57.447923+0000","last_became_peered":"2024-05-08T22:37:57.447923+0000","last_unstale":"2024-05-08T22:37:57.829433+0000","last_undegraded":"2024-05-08T22:37:57.829433+0000","last_fullsized":"2024-05-08T22:37:57.829433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:30.907553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839082+0000","last_change":"2024-05-08T22:37:57.829299+0000","last_active":"2024-05-08T22:37:57.839082+0000","last_peered":"2024-05-08T22:37:57.839082+0000","last_clean":"2024-05-08T22:37:57.839082+0000","last_became_active":"2024-05-08T22:37:57.828970+0000","last_became_peered":"2024-05-08T22:37:57.828970+0000","last_unstale":"2024-05-08T22:37:57.839082+0000","last_undegraded":"2024-05-08T22:37:57.839082+0000","last_fullsized":"2024-05-08T22:37:57.839082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:01.133413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.851908+0000","last_change":"2024-05-08T22:37:57.444179+0000","last_active":"2024-05-08T22:37:57.851908+0000","last_peered":"2024-05-08T22:37:57.851908+0000","last_clean":"2024-05-08T22:37:57.851908+0000","last_became_active":"2024-05-08T22:37:57.443673+0000","last_became_peered":"2024-05-08T22:37:57.443673+0000","last_unstale":"2024-05-08T22:37:57.851908+0000","last_undegraded":"2024-05-08T22:37:57.851908+0000","last_fullsized":"2024-05-08T22:37:57.851908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:53.287958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845235+0000","last_change":"2024-05-08T22:37:57.841025+0000","last_active":"2024-05-08T22:37:57.845235+0000","last_peered":"2024-05-08T22:37:57.845235+0000","last_clean":"2024-05-08T22:37:57.845235+0000","last_became_active":"2024-05-08T22:37:57.840591+0000","last_became_peered":"2024-05-08T22:37:57.840591+0000","last_unstale":"2024-05-08T22:37:57.845235+0000","last_undegraded":"2024-05-08T22:37:57.845235+0000","last_fullsized":"2024-05-08T22:37:57.845235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.910062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.842909+0000","last_change":"2024-05-08T22:37:57.440960+0000","last_active":"2024-05-08T22:37:57.842909+0000","last_peered":"2024-05-08T22:37:57.842909+0000","last_clean":"2024-05-08T22:37:57.842909+0000","last_became_active":"2024-05-08T22:37:57.440504+0000","last_became_peered":"2024-05-08T22:37:57.440504+0000","last_unstale":"2024-05-08T22:37:57.842909+0000","last_undegraded":"2024-05-08T22:37:57.842909+0000","last_fullsized":"2024-05-08T22:37:57.842909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:06.401155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.841075+0000","last_change":"2024-05-08T22:37:57.831846+0000","last_active":"2024-05-08T22:37:57.841075+0000","last_peered":"2024-05-08T22:37:57.841075+0000","last_clean":"2024-05-08T22:37:57.841075+0000","last_became_active":"2024-05-08T22:37:57.829692+0000","last_became_peered":"2024-05-08T22:37:57.829692+0000","last_unstale":"2024-05-08T22:37:57.841075+0000","last_undegraded":"2024-05-08T22:37:57.841075+0000","last_fullsized":"2024-05-08T22:37:57.841075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.674154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.439143+0000","last_change":"2024-05-08T22:37:57.437527+0000","last_active":"2024-05-08T22:37:57.439143+0000","last_peered":"2024-05-08T22:37:57.439143+0000","last_clean":"2024-05-08T22:37:57.439143+0000","last_became_active":"2024-05-08T22:37:57.437261+0000","last_became_peered":"2024-05-08T22:37:57.437261+0000","last_unstale":"2024-05-08T22:37:57.439143+0000","last_undegraded":"2024-05-08T22:37:57.439143+0000","last_fullsized":"2024-05-08T22:37:57.439143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:19.394053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844277+0000","last_change":"2024-05-08T22:37:57.832513+0000","last_active":"2024-05-08T22:37:57.844277+0000","last_peered":"2024-05-08T22:37:57.844277+0000","last_clean":"2024-05-08T22:37:57.844277+0000","last_became_active":"2024-05-08T22:37:57.832275+0000","last_became_peered":"2024-05-08T22:37:57.832275+0000","last_unstale":"2024-05-08T22:37:57.844277+0000","last_undegraded":"2024-05-08T22:37:57.844277+0000","last_fullsized":"2024-05-08T22:37:57.844277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:08.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847277+0000","last_change":"2024-05-08T22:37:57.444668+0000","last_active":"2024-05-08T22:37:57.847277+0000","last_peered":"2024-05-08T22:37:57.847277+0000","last_clean":"2024-05-08T22:37:57.847277+0000","last_became_active":"2024-05-08T22:37:57.441598+0000","last_became_peered":"2024-05-08T22:37:57.441598+0000","last_unstale":"2024-05-08T22:37:57.847277+0000","last_undegraded":"2024-05-08T22:37:57.847277+0000","last_fullsized":"2024-05-08T22:37:57.847277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:33.716602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.833929+0000","last_change":"2024-05-08T22:37:57.832325+0000","last_active":"2024-05-08T22:37:57.833929+0000","last_peered":"2024-05-08T22:37:57.833929+0000","last_clean":"2024-05-08T22:37:57.833929+0000","last_became_active":"2024-05-08T22:37:57.832067+0000","last_became_peered":"2024-05-08T22:37:57.832067+0000","last_unstale":"2024-05-08T22:37:57.833929+0000","last_undegraded":"2024-05-08T22:37:57.833929+0000","last_fullsized":"2024-05-08T22:37:57.833929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:42.187631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817518+0000","last_change":"2024-05-08T22:37:57.442255+0000","last_active":"2024-05-08T22:37:57.817518+0000","last_peered":"2024-05-08T22:37:57.817518+0000","last_clean":"2024-05-08T22:37:57.817518+0000","last_became_active":"2024-05-08T22:37:57.441960+0000","last_became_peered":"2024-05-08T22:37:57.441960+0000","last_unstale":"2024-05-08T22:37:57.817518+0000","last_undegraded":"2024-05-08T22:37:57.817518+0000","last_fullsized":"2024-05-08T22:37:57.817518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:59.020957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848551+0000","last_change":"2024-05-08T22:37:57.842118+0000","last_active":"2024-05-08T22:37:57.848551+0000","last_peered":"2024-05-08T22:37:57.848551+0000","last_clean":"2024-05-08T22:37:57.848551+0000","last_became_active":"2024-05-08T22:37:57.841826+0000","last_became_peered":"2024-05-08T22:37:57.841826+0000","last_unstale":"2024-05-08T22:37:57.848551+0000","last_undegraded":"2024-05-08T22:37:57.848551+0000","last_fullsized":"2024-05-08T22:37:57.848551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:55.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.840125+0000","last_change":"2024-05-08T22:37:57.439634+0000","last_active":"2024-05-08T22:37:57.840125+0000","last_peered":"2024-05-08T22:37:57.840125+0000","last_clean":"2024-05-08T22:37:57.840125+0000","last_became_active":"2024-05-08T22:37:57.439387+0000","last_became_peered":"2024-05-08T22:37:57.439387+0000","last_unstale":"2024-05-08T22:37:57.840125+0000","last_undegraded":"2024-05-08T22:37:57.840125+0000","last_fullsized":"2024-05-08T22:37:57.840125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:49.158578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848777+0000","last_change":"2024-05-08T22:37:57.846904+0000","last_active":"2024-05-08T22:37:57.848777+0000","last_peered":"2024-05-08T22:37:57.848777+0000","last_clean":"2024-05-08T22:37:57.848777+0000","last_became_active":"2024-05-08T22:37:57.846651+0000","last_became_peered":"2024-05-08T22:37:57.846651+0000","last_unstale":"2024-05-08T22:37:57.848777+0000","last_undegraded":"2024-05-08T22:37:57.848777+0000","last_fullsized":"2024-05-08T22:37:57.848777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:05.340141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.821917+0000","last_change":"2024-05-08T22:37:57.443724+0000","last_active":"2024-05-08T22:37:57.821917+0000","last_peered":"2024-05-08T22:37:57.821917+0000","last_clean":"2024-05-08T22:37:57.821917+0000","last_became_active":"2024-05-08T22:37:57.443460+0000","last_became_peered":"2024-05-08T22:37:57.443460+0000","last_unstale":"2024-05-08T22:37:57.821917+0000","last_undegraded":"2024-05-08T22:37:57.821917+0000","last_fullsized":"2024-05-08T22:37:57.821917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:08.118921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.837256+0000","last_change":"2024-05-08T22:37:57.826029+0000","last_active":"2024-05-08T22:37:57.837256+0000","last_peered":"2024-05-08T22:37:57.837256+0000","last_clean":"2024-05-08T22:37:57.837256+0000","last_became_active":"2024-05-08T22:37:57.825743+0000","last_became_peered":"2024-05-08T22:37:57.825743+0000","last_unstale":"2024-05-08T22:37:57.837256+0000","last_undegraded":"2024-05-08T22:37:57.837256+0000","last_fullsized":"2024-05-08T22:37:57.837256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.587376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822152+0000","last_change":"2024-05-08T22:37:57.444844+0000","last_active":"2024-05-08T22:37:57.822152+0000","last_peered":"2024-05-08T22:37:57.822152+0000","last_clean":"2024-05-08T22:37:57.822152+0000","last_became_active":"2024-05-08T22:37:57.444488+0000","last_became_peered":"2024-05-08T22:37:57.444488+0000","last_unstale":"2024-05-08T22:37:57.822152+0000","last_undegraded":"2024-05-08T22:37:57.822152+0000","last_fullsized":"2024-05-08T22:37:57.822152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:36.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424456+0000","last_change":"2024-05-08T22:37:57.424456+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424456+0000","last_undegraded":"2024-05-08T22:37:57.424456+0000","last_fullsized":"2024-05-08T22:37:57.424456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838790+0000","last_change":"2024-05-08T22:37:57.438922+0000","last_active":"2024-05-08T22:37:57.838790+0000","last_peered":"2024-05-08T22:37:57.838790+0000","last_clean":"2024-05-08T22:37:57.838790+0000","last_became_active":"2024-05-08T22:37:57.438655+0000","last_became_peered":"2024-05-08T22:37:57.438655+0000","last_unstale":"2024-05-08T22:37:57.838790+0000","last_undegraded":"2024-05-08T22:37:57.838790+0000","last_fullsized":"2024-05-08T22:37:57.838790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:09:11.390285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838438+0000","last_change":"2024-05-08T22:37:57.828420+0000","last_active":"2024-05-08T22:37:57.838438+0000","last_peered":"2024-05-08T22:37:57.838438+0000","last_clean":"2024-05-08T22:37:57.838438+0000","last_became_active":"2024-05-08T22:37:57.828132+0000","last_became_peered":"2024-05-08T22:37:57.828132+0000","last_unstale":"2024-05-08T22:37:57.838438+0000","last_undegraded":"2024-05-08T22:37:57.838438+0000","last_fullsized":"2024-05-08T22:37:57.838438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:23.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.364530+0000","last_change":"2024-05-08T22:37:57.447723+0000","last_active":"2024-05-08T22:37:58.364530+0000","last_peered":"2024-05-08T22:37:58.364530+0000","last_clean":"2024-05-08T22:37:58.364530+0000","last_became_active":"2024-05-08T22:37:57.447478+0000","last_became_peered":"2024-05-08T22:37:57.447478+0000","last_unstale":"2024-05-08T22:37:58.364530+0000","last_undegraded":"2024-05-08T22:37:58.364530+0000","last_fullsized":"2024-05-08T22:37:58.364530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:09.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-05-08T22:37:57.424687+0000","last_change":"2024-05-08T22:37:57.424687+0000","last_active":"2024-05-08T22:37:57.410053+0000","last_peered":"2024-05-08T22:37:57.410053+0000","last_clean":"2024-05-08T22:37:57.410053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:37:57.424687+0000","last_undegraded":"2024-05-08T22:37:57.424687+0000","last_fullsized":"2024-05-08T22:37:57.424687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.827351+0000","last_change":"2024-05-08T22:37:57.437575+0000","last_active":"2024-05-08T22:37:57.827351+0000","last_peered":"2024-05-08T22:37:57.827351+0000","last_clean":"2024-05-08T22:37:57.827351+0000","last_became_active":"2024-05-08T22:37:57.437308+0000","last_became_peered":"2024-05-08T22:37:57.437308+0000","last_unstale":"2024-05-08T22:37:57.827351+0000","last_undegraded":"2024-05-08T22:37:57.827351+0000","last_fullsized":"2024-05-08T22:37:57.827351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:33.154187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848732+0000","last_change":"2024-05-08T22:37:57.845583+0000","last_active":"2024-05-08T22:37:57.848732+0000","last_peered":"2024-05-08T22:37:57.848732+0000","last_clean":"2024-05-08T22:37:57.848732+0000","last_became_active":"2024-05-08T22:37:57.845321+0000","last_became_peered":"2024-05-08T22:37:57.845321+0000","last_unstale":"2024-05-08T22:37:57.848732+0000","last_undegraded":"2024-05-08T22:37:57.848732+0000","last_fullsized":"2024-05-08T22:37:57.848732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:27.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825845+0000","last_change":"2024-05-08T22:37:57.440921+0000","last_active":"2024-05-08T22:37:57.825845+0000","last_peered":"2024-05-08T22:37:57.825845+0000","last_clean":"2024-05-08T22:37:57.825845+0000","last_became_active":"2024-05-08T22:37:57.440652+0000","last_became_peered":"2024-05-08T22:37:57.440652+0000","last_unstale":"2024-05-08T22:37:57.825845+0000","last_undegraded":"2024-05-08T22:37:57.825845+0000","last_fullsized":"2024-05-08T22:37:57.825845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:02.665256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.838442+0000","last_change":"2024-05-08T22:37:57.828599+0000","last_active":"2024-05-08T22:37:57.838442+0000","last_peered":"2024-05-08T22:37:57.838442+0000","last_clean":"2024-05-08T22:37:57.838442+0000","last_became_active":"2024-05-08T22:37:57.828254+0000","last_became_peered":"2024-05-08T22:37:57.828254+0000","last_unstale":"2024-05-08T22:37:57.838442+0000","last_undegraded":"2024-05-08T22:37:57.838442+0000","last_fullsized":"2024-05-08T22:37:57.838442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:30.500016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.822000+0000","last_change":"2024-05-08T22:37:57.442291+0000","last_active":"2024-05-08T22:37:57.822000+0000","last_peered":"2024-05-08T22:37:57.822000+0000","last_clean":"2024-05-08T22:37:57.822000+0000","last_became_active":"2024-05-08T22:37:57.441935+0000","last_became_peered":"2024-05-08T22:37:57.441935+0000","last_unstale":"2024-05-08T22:37:57.822000+0000","last_undegraded":"2024-05-08T22:37:57.822000+0000","last_fullsized":"2024-05-08T22:37:57.822000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:55.703307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.839361+0000","last_change":"2024-05-08T22:37:57.832660+0000","last_active":"2024-05-08T22:37:57.839361+0000","last_peered":"2024-05-08T22:37:57.839361+0000","last_clean":"2024-05-08T22:37:57.839361+0000","last_became_active":"2024-05-08T22:37:57.832291+0000","last_became_peered":"2024-05-08T22:37:57.832291+0000","last_unstale":"2024-05-08T22:37:57.839361+0000","last_undegraded":"2024-05-08T22:37:57.839361+0000","last_fullsized":"2024-05-08T22:37:57.839361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:40.277738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.830600+0000","last_change":"2024-05-08T22:37:57.448069+0000","last_active":"2024-05-08T22:37:57.830600+0000","last_peered":"2024-05-08T22:37:57.830600+0000","last_clean":"2024-05-08T22:37:57.830600+0000","last_became_active":"2024-05-08T22:37:57.446523+0000","last_became_peered":"2024-05-08T22:37:57.446523+0000","last_unstale":"2024-05-08T22:37:57.830600+0000","last_undegraded":"2024-05-08T22:37:57.830600+0000","last_fullsized":"2024-05-08T22:37:57.830600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:44.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.836038+0000","last_change":"2024-05-08T22:37:57.830871+0000","last_active":"2024-05-08T22:37:57.836038+0000","last_peered":"2024-05-08T22:37:57.836038+0000","last_clean":"2024-05-08T22:37:57.836038+0000","last_became_active":"2024-05-08T22:37:57.830594+0000","last_became_peered":"2024-05-08T22:37:57.830594+0000","last_unstale":"2024-05-08T22:37:57.836038+0000","last_undegraded":"2024-05-08T22:37:57.836038+0000","last_fullsized":"2024-05-08T22:37:57.836038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:02.234727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.823044+0000","last_change":"2024-05-08T22:37:57.438169+0000","last_active":"2024-05-08T22:37:57.823044+0000","last_peered":"2024-05-08T22:37:57.823044+0000","last_clean":"2024-05-08T22:37:57.823044+0000","last_became_active":"2024-05-08T22:37:57.437880+0000","last_became_peered":"2024-05-08T22:37:57.437880+0000","last_unstale":"2024-05-08T22:37:57.823044+0000","last_undegraded":"2024-05-08T22:37:57.823044+0000","last_fullsized":"2024-05-08T22:37:57.823044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:58.133761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.850749+0000","last_change":"2024-05-08T22:37:57.846104+0000","last_active":"2024-05-08T22:37:57.850749+0000","last_peered":"2024-05-08T22:37:57.850749+0000","last_clean":"2024-05-08T22:37:57.850749+0000","last_became_active":"2024-05-08T22:37:57.845848+0000","last_became_peered":"2024-05-08T22:37:57.845848+0000","last_unstale":"2024-05-08T22:37:57.850749+0000","last_undegraded":"2024-05-08T22:37:57.850749+0000","last_fullsized":"2024-05-08T22:37:57.850749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:06:32.854193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.817725+0000","last_change":"2024-05-08T22:37:57.437069+0000","last_active":"2024-05-08T22:37:57.817725+0000","last_peered":"2024-05-08T22:37:57.817725+0000","last_clean":"2024-05-08T22:37:57.817725+0000","last_became_active":"2024-05-08T22:37:57.436461+0000","last_became_peered":"2024-05-08T22:37:57.436461+0000","last_unstale":"2024-05-08T22:37:57.817725+0000","last_undegraded":"2024-05-08T22:37:57.817725+0000","last_fullsized":"2024-05-08T22:37:57.817725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:31.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.845596+0000","last_change":"2024-05-08T22:37:57.843298+0000","last_active":"2024-05-08T22:37:57.845596+0000","last_peered":"2024-05-08T22:37:57.845596+0000","last_clean":"2024-05-08T22:37:57.845596+0000","last_became_active":"2024-05-08T22:37:57.842988+0000","last_became_peered":"2024-05-08T22:37:57.842988+0000","last_unstale":"2024-05-08T22:37:57.845596+0000","last_undegraded":"2024-05-08T22:37:57.845596+0000","last_fullsized":"2024-05-08T22:37:57.845596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:17.302357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.825542+0000","last_change":"2024-05-08T22:37:57.444385+0000","last_active":"2024-05-08T22:37:57.825542+0000","last_peered":"2024-05-08T22:37:57.825542+0000","last_clean":"2024-05-08T22:37:57.825542+0000","last_became_active":"2024-05-08T22:37:57.444124+0000","last_became_peered":"2024-05-08T22:37:57.444124+0000","last_unstale":"2024-05-08T22:37:57.825542+0000","last_undegraded":"2024-05-08T22:37:57.825542+0000","last_fullsized":"2024-05-08T22:37:57.825542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:35.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.429399+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:37:57.429399+0000","last_peered":"2024-05-08T22:37:57.429399+0000","last_clean":"2024-05-08T22:37:57.429399+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:37:57.429399+0000","last_undegraded":"2024-05-08T22:37:57.429399+0000","last_fullsized":"2024-05-08T22:37:57.429399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":999424,"data_stored":451081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.309}]},{"osd":1,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.47599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89100000000000001,"5min":1.101,"15min":1.101},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.627}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.018,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.57899999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1140000000000001,"5min":1.306,"15min":1.306},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2310000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.274,"5min":1.339,"15min":1.339},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.0089999999999999,"15min":1.0269999999999999},"min":{"1min":0.76000000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2749999999999999,"5min":1.476,"15min":1.476},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98199999999999998,"15min":0.97199999999999998},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2370000000000001,"5min":1.446,"15min":1.446},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.753,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.341,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.105},"min":{"1min":0.82199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.298,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.056},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.036,"15min":1.034},"min":{"1min":0.81000000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4570000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.88400000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":450611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":1.04,"15min":1.04},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.58399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2130000000000001,"5min":1.583,"15min":1.583},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2050000000000001,"5min":1.431,"15min":1.431},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.81399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2470000000000001,"5min":1.508,"15min":1.508},"last":0.747}]},{"osd":5,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.028,"15min":1.083},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.397,"5min":1.726,"15min":1.726},"last":0.96999999999999997}]},{"osd":6,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.77200000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.446,"5min":1.659,"15min":1.659},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.071},"min":{"1min":0.85399999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3129999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0569999999999999},"min":{"1min":0.81699999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.409,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85599999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1593344,"data_stored":1040889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85499999999999998,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2749999999999999,"5min":1.282,"15min":1.282},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.84099999999999997,"15min":0.85999999999999999},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.121}]},{"osd":1,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91800000000000004,"15min":0.91000000000000003},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.099,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1140000000000001,"5min":1.419,"15min":1.419},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95099999999999996,"15min":0.96199999999999997},"min":{"1min":0.69199999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.337,"15min":1.337},"last":1.022}]},{"osd":3,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.79400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5269999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.72099999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.177},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.89300000000000002},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.103,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95999999999999996,"15min":0.98199999999999998},"min":{"1min":0.67000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.482,"15min":1.482},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.85399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.264}]},{"osd":6,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.76500000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4179999999999999,"5min":1.599,"15min":1.599},"last":1.0800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":1007616,"data_stored":452063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.141,"15min":1.141},"last":0.997}]},{"osd":1,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3300000000000001,"5min":1.758,"15min":1.758},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91800000000000004,"15min":0.93000000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1850000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.115}]},{"osd":2,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.92900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.397,"5min":1.419,"15min":1.419},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.034,"15min":1.0960000000000001},"min":{"1min":0.68000000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2110000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.99099999999999999},"min":{"1min":0.625,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2869999999999999,"5min":1.538,"15min":1.538},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":1.0469999999999999},"min":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.149,"5min":1.458,"15min":1.458},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.28,"5min":1.681,"15min":1.681},"last":1.194},{"interface":"front","average":{"1min":0.998,"5min":1.0840000000000001,"15min":1.1399999999999999},"min":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2050000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.014,"15min":1.085},"min":{"1min":0.71899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0780000000000001,"5min":1.835,"15min":1.835},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.179,"15min":1.2350000000000001},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3300000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.1559999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":999424,"data_stored":450533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82399999999999995,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.145,"5min":1.52,"15min":1.52},"last":0.79000000000000004}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.016,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.266,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90400000000000003,"15min":0.88},"min":{"1min":0.71899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.83999999999999997},"min":{"1min":0.67600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.204}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":0.96499999999999997},"min":{"1min":0.71599999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.70099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.016,"15min":0.98599999999999999},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96799999999999997,"15min":0.91600000000000004},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2589999999999999,"5min":1.605,"15min":1.605},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.069,"15min":1.044},"min":{"1min":0.89200000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3080000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.3340000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.0880000000000001},"min":{"1min":0.98299999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.411,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.133,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.91500000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3089999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2909999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":970752,"data_stored":436809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.113,"5min":1.28,"15min":1.28},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.294,"15min":1.294},"last":0.72299999999999998}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.871},"min":{"1min":0.65800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.163,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90700000000000003,"15min":0.88600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.173,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0329999999999999,"5min":1.29,"15min":1.29},"last":1.224},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91900000000000004,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.135,"5min":1.637,"15min":1.637},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1619999999999999,"5min":2.601,"15min":2.601},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.032,"15min":1.105},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.998}]},{"osd":5,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.0349999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.121,"5min":1.792,"15min":1.792},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.026},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.137,"5min":1.575,"15min":1.575},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0489999999999999,"15min":1.0900000000000001},"min":{"1min":0.69399999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1100000000000001,"5min":1.831,"15min":1.831},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.093,"15min":1.1020000000000001},"min":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2210000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0609999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.071,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.23,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.268},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.159,"15min":1.177},"min":{"1min":0.878,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3,"5min":1.944,"15min":1.944},"last":1.085}]}]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":443404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.133,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":1.008,"15min":1.008},"last":0.879},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.58799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.333,"5min":1.363,"15min":1.363},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.006,"15min":1.028},"min":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.27,"5min":1.282,"15min":1.282},"last":1.2609999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.407,"5min":1.484,"15min":1.484},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.3}]},{"osd":6,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0600000000000001},"min":{"1min":0.84499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0700000000000001,"15min":1.0840000000000001},"min":{"1min":0.871,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4450000000000001,"5min":1.673,"15min":1.673},"last":1.101}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.119,"15min":1.1299999999999999},"min":{"1min":0.878,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.056}]}]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343276,"statfs":{"total":96636764160,"available":96607514624,"internally_reserved":0,"allocated":1576960,"data_stored":1033772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.617,"15min":1.617},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":1.446,"15min":1.446},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0129999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.4410000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3089999999999999,"5min":1.714,"15min":1.714},"last":1.609},{"interface":"front","average":{"1min":0.755,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2569999999999999,"5min":1.327,"15min":1.327},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2709999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":4.391},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.73499999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3,"5min":1.669,"15min":1.669},"last":2.0590000000000002}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2210000000000001,"5min":1.474,"15min":1.474},"last":1.0760000000000001}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.96499999999999997},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2869999999999999,"5min":1.397,"15min":1.397},"last":1.8660000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":1.0469999999999999},"min":{"1min":0.78600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.177,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":4.4859999999999998}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0649999999999999,"15min":1.089},"min":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.234,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.532},{"interface":"front","average":{"1min":1.018,"5min":1.008,"15min":1.022},"min":{"1min":0.77500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.304}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:38:06.230 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:38:06.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 Processor -- start 2024-05-08T22:38:06.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 -- start start 2024-05-08T22:38:06.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f11240ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f11240cb6e0 0x7f112406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112406abb0 con 0x7f112415b1b0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112406ae90 con 0x7f11240cb6e0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1129cab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112415b9c0 con 0x7f11240cc700 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1123fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f112415b1b0 0x7f11240cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48642/0 (socket says 172.21.15.179:48642) 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.348+0000 7f1123fff700 1 -- 172.21.15.179:0/1748222728 learned_addr learned my addr 172.21.15.179:0/1748222728 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f11240ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11294aa700 1 --2- 172.21.15.179:0/1748222728 >> v2:172.21.15.195:3300/0 conn(0x7f11240cb6e0 0x7f112406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395868448 0 0) 0x7f112406abb0 con 0x7f112415b1b0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11040035d0 con 0x7f112415b1b0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089492471 0 0) 0x7f11040035d0 con 0x7f112415b1b0 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 msgr2=0x7f11240ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f11240ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 >> v2:172.21.15.195:3300/0 conn(0x7f11240cb6e0 msgr2=0x7f112406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:06.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 --2- 172.21.15.179:0/1748222728 >> v2:172.21.15.195:3300/0 conn(0x7f11240cb6e0 0x7f112406b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112415bca0 con 0x7f112415b1b0 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f110c001670 con 0x7f112415b1b0 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 >> v1:172.21.15.179:6789/0 conn(0x7f112415b1b0 legacy=0x7f11240cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1104003a20 0x7f1104003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f1104004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104004db0 0x7f1104005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112406ae90 con 0x7f1104003a20 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112415b9c0 con 0x7f1104004db0 2024-05-08T22:38:06.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110400ba30 con 0x7f11240cc700 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1104003a20 0x7f1104003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1123fff700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f1104004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11294aa700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104004db0 0x7f1104005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 -- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 msgr2=0x7f1104004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11240cc700 0x7f1104004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 -- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104004db0 msgr2=0x7f1104005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104004db0 0x7f1104005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 -- 172.21.15.179:0/1748222728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040035d0 con 0x7f1104003a20 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1128ca9700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1104003a20 0x7f1104003e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1114009b70 tx=0x7f11040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446a225c96449103 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400e6c0 con 0x7f1104003a20 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400e820 con 0x7f1104003a20 2024-05-08T22:38:06.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11237fe700 1 -- 172.21.15.179:0/1748222728 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400eb90 con 0x7f1104003a20 2024-05-08T22:38:06.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 -- 172.21.15.179:0/1748222728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1104003a20 msgr2=0x7f1104003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 --2- 172.21.15.179:0/1748222728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1104003a20 0x7f1104003e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1114009b70 tx=0x7f11040058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f11294aa700 1 -- 172.21.15.179:0/1748222728 reap_dead start 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 -- 172.21.15.179:0/1748222728 shutdown_connections 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 -- 172.21.15.179:0/1748222728 >> 172.21.15.179:0/1748222728 conn(0x7f11240ce9d0 msgr2=0x7f11240cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 -- 172.21.15.179:0/1748222728 shutdown_connections 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.352+0000 7f1129cab700 1 -- 172.21.15.179:0/1748222728 wait complete. 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 Processor -- start 2024-05-08T22:38:06.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 -- start start 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104003a20 0x7f11240a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1124145050 0x7f11241454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1124157e20 con 0x7f1124156590 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1124145a10 con 0x7f1104003a20 2024-05-08T22:38:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.356+0000 7f1129cab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1124156f30 con 0x7f1124145050 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39262/0 (socket says 172.21.15.179:39262) 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 -- 172.21.15.179:0/2320668394 learned_addr learned my addr 172.21.15.179:0/2320668394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f1123fff700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1124145050 0x7f11241454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f1128ca9700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104003a20 0x7f11240a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 -- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1124145050 msgr2=0x7f11241454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1124145050 0x7f11241454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 -- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104003a20 msgr2=0x7f11240a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104003a20 0x7f11240a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 -- 172.21.15.179:0/2320668394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240b3660 con 0x7f1124156590 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11294aa700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f112406d060 tx=0x7f111c0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00e070 con 0x7f1124156590 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c003d10 con 0x7f1124156590 2024-05-08T22:38:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11241571a0 con 0x7f1124156590 2024-05-08T22:38:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0117a0 con 0x7f1124156590 2024-05-08T22:38:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1124157760 con 0x7f1124156590 2024-05-08T22:38:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.360+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1124067740 con 0x7f1124156590 2024-05-08T22:38:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.364+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f111c011900 con 0x7f1124156590 2024-05-08T22:38:06.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.364+0000 7f11217fa700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 0x7f1108068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.364+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f111c084460 con 0x7f1124156590 2024-05-08T22:38:06.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.364+0000 7f1128ca9700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 0x7f1108068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.364+0000 7f1128ca9700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 0x7f1108068bf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11140090b0 tx=0x7f111401e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.372+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f111c051690 con 0x7f1124156590 2024-05-08T22:38:06.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.536+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1124157f60 con 0x7f1108066730 2024-05-08T22:38:06.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.556+0000 7f11217fa700 1 -- 172.21.15.179:0/2320668394 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287804 (secure 0 0 0) 0x7f1124157f60 con 0x7f1108066730 2024-05-08T22:38:06.561 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 msgr2=0x7f1108068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 0x7f1108068bf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11140090b0 tx=0x7f111401e040 comp rx=0 tx=0).stop 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 msgr2=0x7f11241569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f112406d060 tx=0x7f111c0111d0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 shutdown_connections 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1108066730 0x7f1108068bf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1124156590 0x7f11241569f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1124145050 0x7f11241454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 --2- 172.21.15.179:0/2320668394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1104003a20 0x7f11240a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 >> 172.21.15.179:0/2320668394 conn(0x7f11240ce9d0 msgr2=0x7f112406de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:06.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 shutdown_connections 2024-05-08T22:38:06.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.560+0000 7f1129cab700 1 -- 172.21.15.179:0/2320668394 wait complete. 2024-05-08T22:38:06.564 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:38:06.576 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-05-08T22:38:05.480054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258024,"kb_used_data":9096,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754716696,"statfs":{"total":773094113280,"available":772829896704,"internally_reserved":0,"allocated":9314304,"data_stored":4842412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002701"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493611+0000","last_change":"2024-05-08T22:37:57.844883+0000","last_active":"2024-05-08T22:37:58.493611+0000","last_peered":"2024-05-08T22:37:58.493611+0000","last_clean":"2024-05-08T22:37:58.493611+0000","last_became_active":"2024-05-08T22:37:57.844614+0000","last_became_peered":"2024-05-08T22:37:57.844614+0000","last_unstale":"2024-05-08T22:37:58.493611+0000","last_undegraded":"2024-05-08T22:37:58.493611+0000","last_fullsized":"2024-05-08T22:37:58.493611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:56:36.890701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490388+0000","last_change":"2024-05-08T22:37:57.844447+0000","last_active":"2024-05-08T22:37:58.490388+0000","last_peered":"2024-05-08T22:37:58.490388+0000","last_clean":"2024-05-08T22:37:58.490388+0000","last_became_active":"2024-05-08T22:37:57.844201+0000","last_became_peered":"2024-05-08T22:37:57.844201+0000","last_unstale":"2024-05-08T22:37:58.490388+0000","last_undegraded":"2024-05-08T22:37:58.490388+0000","last_fullsized":"2024-05-08T22:37:58.490388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:10.013238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495766+0000","last_change":"2024-05-08T22:37:57.835351+0000","last_active":"2024-05-08T22:37:59.495766+0000","last_peered":"2024-05-08T22:37:59.495766+0000","last_clean":"2024-05-08T22:37:59.495766+0000","last_became_active":"2024-05-08T22:37:57.835104+0000","last_became_peered":"2024-05-08T22:37:57.835104+0000","last_unstale":"2024-05-08T22:37:59.495766+0000","last_undegraded":"2024-05-08T22:37:59.495766+0000","last_fullsized":"2024-05-08T22:37:59.495766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493936+0000","last_change":"2024-05-08T22:37:57.842727+0000","last_active":"2024-05-08T22:37:58.493936+0000","last_peered":"2024-05-08T22:37:58.493936+0000","last_clean":"2024-05-08T22:37:58.493936+0000","last_became_active":"2024-05-08T22:37:57.842350+0000","last_became_peered":"2024-05-08T22:37:57.842350+0000","last_unstale":"2024-05-08T22:37:58.493936+0000","last_undegraded":"2024-05-08T22:37:58.493936+0000","last_fullsized":"2024-05-08T22:37:58.493936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:32.252275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481495+0000","last_change":"2024-05-08T22:37:57.846399+0000","last_active":"2024-05-08T22:37:58.481495+0000","last_peered":"2024-05-08T22:37:58.481495+0000","last_clean":"2024-05-08T22:37:58.481495+0000","last_became_active":"2024-05-08T22:37:57.845308+0000","last_became_peered":"2024-05-08T22:37:57.845308+0000","last_unstale":"2024-05-08T22:37:58.481495+0000","last_undegraded":"2024-05-08T22:37:58.481495+0000","last_fullsized":"2024-05-08T22:37:58.481495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:57.759733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494952+0000","last_change":"2024-05-08T22:37:57.847700+0000","last_active":"2024-05-08T22:37:59.494952+0000","last_peered":"2024-05-08T22:37:59.494952+0000","last_clean":"2024-05-08T22:37:59.494952+0000","last_became_active":"2024-05-08T22:37:57.847333+0000","last_became_peered":"2024-05-08T22:37:57.847333+0000","last_unstale":"2024-05-08T22:37:59.494952+0000","last_undegraded":"2024-05-08T22:37:59.494952+0000","last_fullsized":"2024-05-08T22:37:59.494952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:59.881785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489315+0000","last_change":"2024-05-08T22:37:57.832432+0000","last_active":"2024-05-08T22:37:58.489315+0000","last_peered":"2024-05-08T22:37:58.489315+0000","last_clean":"2024-05-08T22:37:58.489315+0000","last_became_active":"2024-05-08T22:37:57.832048+0000","last_became_peered":"2024-05-08T22:37:57.832048+0000","last_unstale":"2024-05-08T22:37:58.489315+0000","last_undegraded":"2024-05-08T22:37:58.489315+0000","last_fullsized":"2024-05-08T22:37:58.489315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:26.697556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481165+0000","last_change":"2024-05-08T22:37:57.846519+0000","last_active":"2024-05-08T22:37:58.481165+0000","last_peered":"2024-05-08T22:37:58.481165+0000","last_clean":"2024-05-08T22:37:58.481165+0000","last_became_active":"2024-05-08T22:37:57.845361+0000","last_became_peered":"2024-05-08T22:37:57.845361+0000","last_unstale":"2024-05-08T22:37:58.481165+0000","last_undegraded":"2024-05-08T22:37:58.481165+0000","last_fullsized":"2024-05-08T22:37:58.481165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:34.786251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846922+0000","last_change":"2024-05-08T22:37:57.844001+0000","last_active":"2024-05-08T22:37:57.846922+0000","last_peered":"2024-05-08T22:37:57.846922+0000","last_clean":"2024-05-08T22:37:57.846922+0000","last_became_active":"2024-05-08T22:37:57.842269+0000","last_became_peered":"2024-05-08T22:37:57.842269+0000","last_unstale":"2024-05-08T22:37:57.846922+0000","last_undegraded":"2024-05-08T22:37:57.846922+0000","last_fullsized":"2024-05-08T22:37:57.846922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:25.678715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.494004+0000","last_change":"2024-05-08T22:37:57.825872+0000","last_active":"2024-05-08T22:37:58.494004+0000","last_peered":"2024-05-08T22:37:58.494004+0000","last_clean":"2024-05-08T22:37:58.494004+0000","last_became_active":"2024-05-08T22:37:57.825591+0000","last_became_peered":"2024-05-08T22:37:57.825591+0000","last_unstale":"2024-05-08T22:37:58.494004+0000","last_undegraded":"2024-05-08T22:37:58.494004+0000","last_fullsized":"2024-05-08T22:37:58.494004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:07.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490347+0000","last_change":"2024-05-08T22:37:57.844985+0000","last_active":"2024-05-08T22:37:58.490347+0000","last_peered":"2024-05-08T22:37:58.490347+0000","last_clean":"2024-05-08T22:37:58.490347+0000","last_became_active":"2024-05-08T22:37:57.844748+0000","last_became_peered":"2024-05-08T22:37:57.844748+0000","last_unstale":"2024-05-08T22:37:58.490347+0000","last_undegraded":"2024-05-08T22:37:58.490347+0000","last_fullsized":"2024-05-08T22:37:58.490347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:02.244593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493957+0000","last_change":"2024-05-08T22:37:57.830338+0000","last_active":"2024-05-08T22:37:58.493957+0000","last_peered":"2024-05-08T22:37:58.493957+0000","last_clean":"2024-05-08T22:37:58.493957+0000","last_became_active":"2024-05-08T22:37:57.830069+0000","last_became_peered":"2024-05-08T22:37:57.830069+0000","last_unstale":"2024-05-08T22:37:58.493957+0000","last_undegraded":"2024-05-08T22:37:58.493957+0000","last_fullsized":"2024-05-08T22:37:58.493957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:53.355769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481539+0000","last_change":"2024-05-08T22:37:57.835141+0000","last_active":"2024-05-08T22:37:58.481539+0000","last_peered":"2024-05-08T22:37:58.481539+0000","last_clean":"2024-05-08T22:37:58.481539+0000","last_became_active":"2024-05-08T22:37:57.832615+0000","last_became_peered":"2024-05-08T22:37:57.832615+0000","last_unstale":"2024-05-08T22:37:58.481539+0000","last_undegraded":"2024-05-08T22:37:58.481539+0000","last_fullsized":"2024-05-08T22:37:58.481539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:40.282177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494974+0000","last_change":"2024-05-08T22:37:57.847622+0000","last_active":"2024-05-08T22:37:59.494974+0000","last_peered":"2024-05-08T22:37:59.494974+0000","last_clean":"2024-05-08T22:37:59.494974+0000","last_became_active":"2024-05-08T22:37:57.847294+0000","last_became_peered":"2024-05-08T22:37:57.847294+0000","last_unstale":"2024-05-08T22:37:59.494974+0000","last_undegraded":"2024-05-08T22:37:59.494974+0000","last_fullsized":"2024-05-08T22:37:59.494974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:14.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490200+0000","last_change":"2024-05-08T22:37:57.836871+0000","last_active":"2024-05-08T22:37:58.490200+0000","last_peered":"2024-05-08T22:37:58.490200+0000","last_clean":"2024-05-08T22:37:58.490200+0000","last_became_active":"2024-05-08T22:37:57.836626+0000","last_became_peered":"2024-05-08T22:37:57.836626+0000","last_unstale":"2024-05-08T22:37:58.490200+0000","last_undegraded":"2024-05-08T22:37:58.490200+0000","last_fullsized":"2024-05-08T22:37:58.490200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:22.008737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493267+0000","last_change":"2024-05-08T22:37:57.840135+0000","last_active":"2024-05-08T22:37:58.493267+0000","last_peered":"2024-05-08T22:37:58.493267+0000","last_clean":"2024-05-08T22:37:58.493267+0000","last_became_active":"2024-05-08T22:37:57.839806+0000","last_became_peered":"2024-05-08T22:37:57.839806+0000","last_unstale":"2024-05-08T22:37:58.493267+0000","last_undegraded":"2024-05-08T22:37:58.493267+0000","last_fullsized":"2024-05-08T22:37:58.493267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:53.757689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481766+0000","last_change":"2024-05-08T22:37:57.832054+0000","last_active":"2024-05-08T22:37:58.481766+0000","last_peered":"2024-05-08T22:37:58.481766+0000","last_clean":"2024-05-08T22:37:58.481766+0000","last_became_active":"2024-05-08T22:37:57.831311+0000","last_became_peered":"2024-05-08T22:37:57.831311+0000","last_unstale":"2024-05-08T22:37:58.481766+0000","last_undegraded":"2024-05-08T22:37:58.481766+0000","last_fullsized":"2024-05-08T22:37:58.481766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:27.877407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495812+0000","last_change":"2024-05-08T22:37:57.847959+0000","last_active":"2024-05-08T22:37:59.495812+0000","last_peered":"2024-05-08T22:37:59.495812+0000","last_clean":"2024-05-08T22:37:59.495812+0000","last_became_active":"2024-05-08T22:37:57.847531+0000","last_became_peered":"2024-05-08T22:37:57.847531+0000","last_unstale":"2024-05-08T22:37:59.495812+0000","last_undegraded":"2024-05-08T22:37:59.495812+0000","last_fullsized":"2024-05-08T22:37:59.495812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:15.951130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495084+0000","last_change":"2024-05-08T22:37:57.832828+0000","last_active":"2024-05-08T22:37:59.495084+0000","last_peered":"2024-05-08T22:37:59.495084+0000","last_clean":"2024-05-08T22:37:59.495084+0000","last_became_active":"2024-05-08T22:37:57.832544+0000","last_became_peered":"2024-05-08T22:37:57.832544+0000","last_unstale":"2024-05-08T22:37:59.495084+0000","last_undegraded":"2024-05-08T22:37:59.495084+0000","last_fullsized":"2024-05-08T22:37:59.495084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:57.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481581+0000","last_change":"2024-05-08T22:37:57.831045+0000","last_active":"2024-05-08T22:37:58.481581+0000","last_peered":"2024-05-08T22:37:58.481581+0000","last_clean":"2024-05-08T22:37:58.481581+0000","last_became_active":"2024-05-08T22:37:57.830745+0000","last_became_peered":"2024-05-08T22:37:57.830745+0000","last_unstale":"2024-05-08T22:37:58.481581+0000","last_undegraded":"2024-05-08T22:37:58.481581+0000","last_fullsized":"2024-05-08T22:37:58.481581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:46:31.214851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.847149+0000","last_change":"2024-05-08T22:37:57.844790+0000","last_active":"2024-05-08T22:37:57.847149+0000","last_peered":"2024-05-08T22:37:57.847149+0000","last_clean":"2024-05-08T22:37:57.847149+0000","last_became_active":"2024-05-08T22:37:57.844328+0000","last_became_peered":"2024-05-08T22:37:57.844328+0000","last_unstale":"2024-05-08T22:37:57.847149+0000","last_undegraded":"2024-05-08T22:37:57.847149+0000","last_fullsized":"2024-05-08T22:37:57.847149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:14.646073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481400+0000","last_change":"2024-05-08T22:37:57.844194+0000","last_active":"2024-05-08T22:37:58.481400+0000","last_peered":"2024-05-08T22:37:58.481400+0000","last_clean":"2024-05-08T22:37:58.481400+0000","last_became_active":"2024-05-08T22:37:57.841102+0000","last_became_peered":"2024-05-08T22:37:57.841102+0000","last_unstale":"2024-05-08T22:37:58.481400+0000","last_undegraded":"2024-05-08T22:37:58.481400+0000","last_fullsized":"2024-05-08T22:37:58.481400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:15.481984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846894+0000","last_change":"2024-05-08T22:37:57.843856+0000","last_active":"2024-05-08T22:37:57.846894+0000","last_peered":"2024-05-08T22:37:57.846894+0000","last_clean":"2024-05-08T22:37:57.846894+0000","last_became_active":"2024-05-08T22:37:57.842176+0000","last_became_peered":"2024-05-08T22:37:57.842176+0000","last_unstale":"2024-05-08T22:37:57.846894+0000","last_undegraded":"2024-05-08T22:37:57.846894+0000","last_fullsized":"2024-05-08T22:37:57.846894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.895994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490120+0000","last_change":"2024-05-08T22:37:57.840565+0000","last_active":"2024-05-08T22:37:58.490120+0000","last_peered":"2024-05-08T22:37:58.490120+0000","last_clean":"2024-05-08T22:37:58.490120+0000","last_became_active":"2024-05-08T22:37:57.834465+0000","last_became_peered":"2024-05-08T22:37:57.834465+0000","last_unstale":"2024-05-08T22:37:58.490120+0000","last_undegraded":"2024-05-08T22:37:58.490120+0000","last_fullsized":"2024-05-08T22:37:58.490120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:23.426774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490457+0000","last_change":"2024-05-08T22:37:57.834804+0000","last_active":"2024-05-08T22:37:58.490457+0000","last_peered":"2024-05-08T22:37:58.490457+0000","last_clean":"2024-05-08T22:37:58.490457+0000","last_became_active":"2024-05-08T22:37:57.834268+0000","last_became_peered":"2024-05-08T22:37:57.834268+0000","last_unstale":"2024-05-08T22:37:58.490457+0000","last_undegraded":"2024-05-08T22:37:58.490457+0000","last_fullsized":"2024-05-08T22:37:58.490457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:10.524450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.494029+0000","last_change":"2024-05-08T22:37:57.831011+0000","last_active":"2024-05-08T22:37:58.494029+0000","last_peered":"2024-05-08T22:37:58.494029+0000","last_clean":"2024-05-08T22:37:58.494029+0000","last_became_active":"2024-05-08T22:37:57.830662+0000","last_became_peered":"2024-05-08T22:37:57.830662+0000","last_unstale":"2024-05-08T22:37:58.494029+0000","last_undegraded":"2024-05-08T22:37:58.494029+0000","last_fullsized":"2024-05-08T22:37:58.494029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:33.816353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493477+0000","last_change":"2024-05-08T22:37:57.840718+0000","last_active":"2024-05-08T22:37:58.493477+0000","last_peered":"2024-05-08T22:37:58.493477+0000","last_clean":"2024-05-08T22:37:58.493477+0000","last_became_active":"2024-05-08T22:37:57.840400+0000","last_became_peered":"2024-05-08T22:37:57.840400+0000","last_unstale":"2024-05-08T22:37:58.493477+0000","last_undegraded":"2024-05-08T22:37:58.493477+0000","last_fullsized":"2024-05-08T22:37:58.493477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.153666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481619+0000","last_change":"2024-05-08T22:37:57.844109+0000","last_active":"2024-05-08T22:37:58.481619+0000","last_peered":"2024-05-08T22:37:58.481619+0000","last_clean":"2024-05-08T22:37:58.481619+0000","last_became_active":"2024-05-08T22:37:57.843582+0000","last_became_peered":"2024-05-08T22:37:57.843582+0000","last_unstale":"2024-05-08T22:37:58.481619+0000","last_undegraded":"2024-05-08T22:37:58.481619+0000","last_fullsized":"2024-05-08T22:37:58.481619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:57:58.974111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490261+0000","last_change":"2024-05-08T22:37:57.841963+0000","last_active":"2024-05-08T22:37:58.490261+0000","last_peered":"2024-05-08T22:37:58.490261+0000","last_clean":"2024-05-08T22:37:58.490261+0000","last_became_active":"2024-05-08T22:37:57.841575+0000","last_became_peered":"2024-05-08T22:37:57.841575+0000","last_unstale":"2024-05-08T22:37:58.490261+0000","last_undegraded":"2024-05-08T22:37:58.490261+0000","last_fullsized":"2024-05-08T22:37:58.490261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:33.257259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493365+0000","last_change":"2024-05-08T22:37:57.840274+0000","last_active":"2024-05-08T22:37:58.493365+0000","last_peered":"2024-05-08T22:37:58.493365+0000","last_clean":"2024-05-08T22:37:58.493365+0000","last_became_active":"2024-05-08T22:37:57.840011+0000","last_became_peered":"2024-05-08T22:37:57.840011+0000","last_unstale":"2024-05-08T22:37:58.493365+0000","last_undegraded":"2024-05-08T22:37:58.493365+0000","last_fullsized":"2024-05-08T22:37:58.493365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:45.626643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495580+0000","last_change":"2024-05-08T22:37:57.827795+0000","last_active":"2024-05-08T22:37:59.495580+0000","last_peered":"2024-05-08T22:37:59.495580+0000","last_clean":"2024-05-08T22:37:59.495580+0000","last_became_active":"2024-05-08T22:37:57.827400+0000","last_became_peered":"2024-05-08T22:37:57.827400+0000","last_unstale":"2024-05-08T22:37:59.495580+0000","last_undegraded":"2024-05-08T22:37:59.495580+0000","last_fullsized":"2024-05-08T22:37:59.495580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:31.579840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493302+0000","last_change":"2024-05-08T22:37:57.844868+0000","last_active":"2024-05-08T22:37:58.493302+0000","last_peered":"2024-05-08T22:37:58.493302+0000","last_clean":"2024-05-08T22:37:58.493302+0000","last_became_active":"2024-05-08T22:37:57.844575+0000","last_became_peered":"2024-05-08T22:37:57.844575+0000","last_unstale":"2024-05-08T22:37:58.493302+0000","last_undegraded":"2024-05-08T22:37:58.493302+0000","last_fullsized":"2024-05-08T22:37:58.493302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495475+0000","last_change":"2024-05-08T22:37:57.847858+0000","last_active":"2024-05-08T22:37:59.495475+0000","last_peered":"2024-05-08T22:37:59.495475+0000","last_clean":"2024-05-08T22:37:59.495475+0000","last_became_active":"2024-05-08T22:37:57.847609+0000","last_became_peered":"2024-05-08T22:37:57.847609+0000","last_unstale":"2024-05-08T22:37:59.495475+0000","last_undegraded":"2024-05-08T22:37:59.495475+0000","last_fullsized":"2024-05-08T22:37:59.495475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:00.575919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481813+0000","last_change":"2024-05-08T22:37:57.446359+0000","last_active":"2024-05-08T22:37:58.481813+0000","last_peered":"2024-05-08T22:37:58.481813+0000","last_clean":"2024-05-08T22:37:58.481813+0000","last_became_active":"2024-05-08T22:37:57.446036+0000","last_became_peered":"2024-05-08T22:37:57.446036+0000","last_unstale":"2024-05-08T22:37:58.481813+0000","last_undegraded":"2024-05-08T22:37:58.481813+0000","last_fullsized":"2024-05-08T22:37:58.481813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:00.419625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490220+0000","last_change":"2024-05-08T22:37:57.842007+0000","last_active":"2024-05-08T22:37:58.490220+0000","last_peered":"2024-05-08T22:37:58.490220+0000","last_clean":"2024-05-08T22:37:58.490220+0000","last_became_active":"2024-05-08T22:37:57.841764+0000","last_became_peered":"2024-05-08T22:37:57.841764+0000","last_unstale":"2024-05-08T22:37:58.490220+0000","last_undegraded":"2024-05-08T22:37:58.490220+0000","last_fullsized":"2024-05-08T22:37:58.490220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:58.771407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493583+0000","last_change":"2024-05-08T22:37:57.440281+0000","last_active":"2024-05-08T22:37:58.493583+0000","last_peered":"2024-05-08T22:37:58.493583+0000","last_clean":"2024-05-08T22:37:58.493583+0000","last_became_active":"2024-05-08T22:37:57.439987+0000","last_became_peered":"2024-05-08T22:37:57.439987+0000","last_unstale":"2024-05-08T22:37:58.493583+0000","last_undegraded":"2024-05-08T22:37:58.493583+0000","last_fullsized":"2024-05-08T22:37:58.493583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:32.804508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848002+0000","last_change":"2024-05-08T22:37:57.845393+0000","last_active":"2024-05-08T22:37:57.848002+0000","last_peered":"2024-05-08T22:37:57.848002+0000","last_clean":"2024-05-08T22:37:57.848002+0000","last_became_active":"2024-05-08T22:37:57.845010+0000","last_became_peered":"2024-05-08T22:37:57.845010+0000","last_unstale":"2024-05-08T22:37:57.848002+0000","last_undegraded":"2024-05-08T22:37:57.848002+0000","last_fullsized":"2024-05-08T22:37:57.848002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:57.486642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493507+0000","last_change":"2024-05-08T22:37:57.448745+0000","last_active":"2024-05-08T22:37:58.493507+0000","last_peered":"2024-05-08T22:37:58.493507+0000","last_clean":"2024-05-08T22:37:58.493507+0000","last_became_active":"2024-05-08T22:37:57.448495+0000","last_became_peered":"2024-05-08T22:37:57.448495+0000","last_unstale":"2024-05-08T22:37:58.493507+0000","last_undegraded":"2024-05-08T22:37:58.493507+0000","last_fullsized":"2024-05-08T22:37:58.493507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:52.343924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848245+0000","last_change":"2024-05-08T22:37:57.846392+0000","last_active":"2024-05-08T22:37:57.848245+0000","last_peered":"2024-05-08T22:37:57.848245+0000","last_clean":"2024-05-08T22:37:57.848245+0000","last_became_active":"2024-05-08T22:37:57.846087+0000","last_became_peered":"2024-05-08T22:37:57.846087+0000","last_unstale":"2024-05-08T22:37:57.848245+0000","last_undegraded":"2024-05-08T22:37:57.848245+0000","last_fullsized":"2024-05-08T22:37:57.848245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:47.153054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489456+0000","last_change":"2024-05-08T22:37:57.442124+0000","last_active":"2024-05-08T22:37:58.489456+0000","last_peered":"2024-05-08T22:37:58.489456+0000","last_clean":"2024-05-08T22:37:58.489456+0000","last_became_active":"2024-05-08T22:37:57.441844+0000","last_became_peered":"2024-05-08T22:37:57.441844+0000","last_unstale":"2024-05-08T22:37:58.489456+0000","last_undegraded":"2024-05-08T22:37:58.489456+0000","last_fullsized":"2024-05-08T22:37:58.489456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:12.294211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490301+0000","last_change":"2024-05-08T22:37:57.845013+0000","last_active":"2024-05-08T22:37:58.490301+0000","last_peered":"2024-05-08T22:37:58.490301+0000","last_clean":"2024-05-08T22:37:58.490301+0000","last_became_active":"2024-05-08T22:37:57.844721+0000","last_became_peered":"2024-05-08T22:37:57.844721+0000","last_unstale":"2024-05-08T22:37:58.490301+0000","last_undegraded":"2024-05-08T22:37:58.490301+0000","last_fullsized":"2024-05-08T22:37:58.490301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.527397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.844545+0000","last_change":"2024-05-08T22:37:57.438281+0000","last_active":"2024-05-08T22:37:57.844545+0000","last_peered":"2024-05-08T22:37:57.844545+0000","last_clean":"2024-05-08T22:37:57.844545+0000","last_became_active":"2024-05-08T22:37:57.438016+0000","last_became_peered":"2024-05-08T22:37:57.438016+0000","last_unstale":"2024-05-08T22:37:57.844545+0000","last_undegraded":"2024-05-08T22:37:57.844545+0000","last_fullsized":"2024-05-08T22:37:57.844545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:22.432349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490368+0000","last_change":"2024-05-08T22:37:57.845395+0000","last_active":"2024-05-08T22:37:58.490368+0000","last_peered":"2024-05-08T22:37:58.490368+0000","last_clean":"2024-05-08T22:37:58.490368+0000","last_became_active":"2024-05-08T22:37:57.845129+0000","last_became_peered":"2024-05-08T22:37:57.845129+0000","last_unstale":"2024-05-08T22:37:58.490368+0000","last_undegraded":"2024-05-08T22:37:58.490368+0000","last_fullsized":"2024-05-08T22:37:58.490368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:26.794424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493887+0000","last_change":"2024-05-08T22:37:57.436502+0000","last_active":"2024-05-08T22:37:58.493887+0000","last_peered":"2024-05-08T22:37:58.493887+0000","last_clean":"2024-05-08T22:37:58.493887+0000","last_became_active":"2024-05-08T22:37:57.436246+0000","last_became_peered":"2024-05-08T22:37:57.436246+0000","last_unstale":"2024-05-08T22:37:58.493887+0000","last_undegraded":"2024-05-08T22:37:58.493887+0000","last_fullsized":"2024-05-08T22:37:58.493887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:07.871185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846988+0000","last_change":"2024-05-08T22:37:57.844103+0000","last_active":"2024-05-08T22:37:57.846988+0000","last_peered":"2024-05-08T22:37:57.846988+0000","last_clean":"2024-05-08T22:37:57.846988+0000","last_became_active":"2024-05-08T22:37:57.843664+0000","last_became_peered":"2024-05-08T22:37:57.843664+0000","last_unstale":"2024-05-08T22:37:57.846988+0000","last_undegraded":"2024-05-08T22:37:57.846988+0000","last_fullsized":"2024-05-08T22:37:57.846988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:10.043782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490240+0000","last_change":"2024-05-08T22:37:57.443165+0000","last_active":"2024-05-08T22:37:58.490240+0000","last_peered":"2024-05-08T22:37:58.490240+0000","last_clean":"2024-05-08T22:37:58.490240+0000","last_became_active":"2024-05-08T22:37:57.442883+0000","last_became_peered":"2024-05-08T22:37:57.442883+0000","last_unstale":"2024-05-08T22:37:58.490240+0000","last_undegraded":"2024-05-08T22:37:58.490240+0000","last_fullsized":"2024-05-08T22:37:58.490240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:43.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489572+0000","last_change":"2024-05-08T22:37:57.840713+0000","last_active":"2024-05-08T22:37:58.489572+0000","last_peered":"2024-05-08T22:37:58.489572+0000","last_clean":"2024-05-08T22:37:58.489572+0000","last_became_active":"2024-05-08T22:37:57.839777+0000","last_became_peered":"2024-05-08T22:37:57.839777+0000","last_unstale":"2024-05-08T22:37:58.489572+0000","last_undegraded":"2024-05-08T22:37:58.489572+0000","last_fullsized":"2024-05-08T22:37:58.489572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:09.133783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481222+0000","last_change":"2024-05-08T22:37:57.445418+0000","last_active":"2024-05-08T22:37:58.481222+0000","last_peered":"2024-05-08T22:37:58.481222+0000","last_clean":"2024-05-08T22:37:58.481222+0000","last_became_active":"2024-05-08T22:37:57.445005+0000","last_became_peered":"2024-05-08T22:37:57.445005+0000","last_unstale":"2024-05-08T22:37:58.481222+0000","last_undegraded":"2024-05-08T22:37:58.481222+0000","last_fullsized":"2024-05-08T22:37:58.481222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:01.056718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490150+0000","last_change":"2024-05-08T22:37:57.846487+0000","last_active":"2024-05-08T22:37:58.490150+0000","last_peered":"2024-05-08T22:37:58.490150+0000","last_clean":"2024-05-08T22:37:58.490150+0000","last_became_active":"2024-05-08T22:37:57.845715+0000","last_became_peered":"2024-05-08T22:37:57.845715+0000","last_unstale":"2024-05-08T22:37:58.490150+0000","last_undegraded":"2024-05-08T22:37:58.490150+0000","last_fullsized":"2024-05-08T22:37:58.490150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:56.850979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493638+0000","last_change":"2024-05-08T22:37:57.442931+0000","last_active":"2024-05-08T22:37:58.493638+0000","last_peered":"2024-05-08T22:37:58.493638+0000","last_clean":"2024-05-08T22:37:58.493638+0000","last_became_active":"2024-05-08T22:37:57.441198+0000","last_became_peered":"2024-05-08T22:37:57.441198+0000","last_unstale":"2024-05-08T22:37:58.493638+0000","last_undegraded":"2024-05-08T22:37:58.493638+0000","last_fullsized":"2024-05-08T22:37:58.493638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:35.216047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493856+0000","last_change":"2024-05-08T22:37:57.844158+0000","last_active":"2024-05-08T22:37:58.493856+0000","last_peered":"2024-05-08T22:37:58.493856+0000","last_clean":"2024-05-08T22:37:58.493856+0000","last_became_active":"2024-05-08T22:37:57.843855+0000","last_became_peered":"2024-05-08T22:37:57.843855+0000","last_unstale":"2024-05-08T22:37:58.493856+0000","last_undegraded":"2024-05-08T22:37:58.493856+0000","last_fullsized":"2024-05-08T22:37:58.493856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:36.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.494051+0000","last_change":"2024-05-08T22:37:57.439889+0000","last_active":"2024-05-08T22:37:58.494051+0000","last_peered":"2024-05-08T22:37:58.494051+0000","last_clean":"2024-05-08T22:37:58.494051+0000","last_became_active":"2024-05-08T22:37:57.439633+0000","last_became_peered":"2024-05-08T22:37:57.439633+0000","last_unstale":"2024-05-08T22:37:58.494051+0000","last_undegraded":"2024-05-08T22:37:58.494051+0000","last_fullsized":"2024-05-08T22:37:58.494051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:01.221867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494994+0000","last_change":"2024-05-08T22:37:57.833440+0000","last_active":"2024-05-08T22:37:59.494994+0000","last_peered":"2024-05-08T22:37:59.494994+0000","last_clean":"2024-05-08T22:37:59.494994+0000","last_became_active":"2024-05-08T22:37:57.833212+0000","last_became_peered":"2024-05-08T22:37:57.833212+0000","last_unstale":"2024-05-08T22:37:59.494994+0000","last_undegraded":"2024-05-08T22:37:59.494994+0000","last_fullsized":"2024-05-08T22:37:59.494994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:08.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481667+0000","last_change":"2024-05-08T22:37:57.444957+0000","last_active":"2024-05-08T22:37:58.481667+0000","last_peered":"2024-05-08T22:37:58.481667+0000","last_clean":"2024-05-08T22:37:58.481667+0000","last_became_active":"2024-05-08T22:37:57.444484+0000","last_became_peered":"2024-05-08T22:37:57.444484+0000","last_unstale":"2024-05-08T22:37:58.481667+0000","last_undegraded":"2024-05-08T22:37:58.481667+0000","last_fullsized":"2024-05-08T22:37:58.481667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:46.229790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495629+0000","last_change":"2024-05-08T22:37:57.830761+0000","last_active":"2024-05-08T22:37:59.495629+0000","last_peered":"2024-05-08T22:37:59.495629+0000","last_clean":"2024-05-08T22:37:59.495629+0000","last_became_active":"2024-05-08T22:37:57.830491+0000","last_became_peered":"2024-05-08T22:37:57.830491+0000","last_unstale":"2024-05-08T22:37:59.495629+0000","last_undegraded":"2024-05-08T22:37:59.495629+0000","last_fullsized":"2024-05-08T22:37:59.495629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:04:12.683143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489481+0000","last_change":"2024-05-08T22:37:57.439248+0000","last_active":"2024-05-08T22:37:58.489481+0000","last_peered":"2024-05-08T22:37:58.489481+0000","last_clean":"2024-05-08T22:37:58.489481+0000","last_became_active":"2024-05-08T22:37:57.438927+0000","last_became_peered":"2024-05-08T22:37:57.438927+0000","last_unstale":"2024-05-08T22:37:58.489481+0000","last_undegraded":"2024-05-08T22:37:58.489481+0000","last_fullsized":"2024-05-08T22:37:58.489481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:11.645499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493977+0000","last_change":"2024-05-08T22:37:57.835299+0000","last_active":"2024-05-08T22:37:58.493977+0000","last_peered":"2024-05-08T22:37:58.493977+0000","last_clean":"2024-05-08T22:37:58.493977+0000","last_became_active":"2024-05-08T22:37:57.834959+0000","last_became_peered":"2024-05-08T22:37:57.834959+0000","last_unstale":"2024-05-08T22:37:58.493977+0000","last_undegraded":"2024-05-08T22:37:58.493977+0000","last_fullsized":"2024-05-08T22:37:58.493977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:19.550971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493912+0000","last_change":"2024-05-08T22:37:57.436168+0000","last_active":"2024-05-08T22:37:58.493912+0000","last_peered":"2024-05-08T22:37:58.493912+0000","last_clean":"2024-05-08T22:37:58.493912+0000","last_became_active":"2024-05-08T22:37:57.435911+0000","last_became_peered":"2024-05-08T22:37:57.435911+0000","last_unstale":"2024-05-08T22:37:58.493912+0000","last_undegraded":"2024-05-08T22:37:58.493912+0000","last_fullsized":"2024-05-08T22:37:58.493912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:39.531241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489506+0000","last_change":"2024-05-08T22:37:57.834557+0000","last_active":"2024-05-08T22:37:58.489506+0000","last_peered":"2024-05-08T22:37:58.489506+0000","last_clean":"2024-05-08T22:37:58.489506+0000","last_became_active":"2024-05-08T22:37:57.834336+0000","last_became_peered":"2024-05-08T22:37:57.834336+0000","last_unstale":"2024-05-08T22:37:58.489506+0000","last_undegraded":"2024-05-08T22:37:58.489506+0000","last_fullsized":"2024-05-08T22:37:58.489506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:07.977190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493414+0000","last_change":"2024-05-08T22:37:57.442582+0000","last_active":"2024-05-08T22:37:58.493414+0000","last_peered":"2024-05-08T22:37:58.493414+0000","last_clean":"2024-05-08T22:37:58.493414+0000","last_became_active":"2024-05-08T22:37:57.440956+0000","last_became_peered":"2024-05-08T22:37:57.440956+0000","last_unstale":"2024-05-08T22:37:58.493414+0000","last_undegraded":"2024-05-08T22:37:58.493414+0000","last_fullsized":"2024-05-08T22:37:58.493414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:40.041454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489549+0000","last_change":"2024-05-08T22:37:57.846329+0000","last_active":"2024-05-08T22:37:58.489549+0000","last_peered":"2024-05-08T22:37:58.489549+0000","last_clean":"2024-05-08T22:37:58.489549+0000","last_became_active":"2024-05-08T22:37:57.846056+0000","last_became_peered":"2024-05-08T22:37:57.846056+0000","last_unstale":"2024-05-08T22:37:58.489549+0000","last_undegraded":"2024-05-08T22:37:58.489549+0000","last_fullsized":"2024-05-08T22:37:58.489549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:07.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481309+0000","last_change":"2024-05-08T22:37:57.448219+0000","last_active":"2024-05-08T22:37:58.481309+0000","last_peered":"2024-05-08T22:37:58.481309+0000","last_clean":"2024-05-08T22:37:58.481309+0000","last_became_active":"2024-05-08T22:37:57.447923+0000","last_became_peered":"2024-05-08T22:37:57.447923+0000","last_unstale":"2024-05-08T22:37:58.481309+0000","last_undegraded":"2024-05-08T22:37:58.481309+0000","last_fullsized":"2024-05-08T22:37:58.481309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:30.907553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489593+0000","last_change":"2024-05-08T22:37:57.829299+0000","last_active":"2024-05-08T22:37:58.489593+0000","last_peered":"2024-05-08T22:37:58.489593+0000","last_clean":"2024-05-08T22:37:58.489593+0000","last_became_active":"2024-05-08T22:37:57.828970+0000","last_became_peered":"2024-05-08T22:37:57.828970+0000","last_unstale":"2024-05-08T22:37:58.489593+0000","last_undegraded":"2024-05-08T22:37:58.489593+0000","last_fullsized":"2024-05-08T22:37:58.489593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:01.133413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490177+0000","last_change":"2024-05-08T22:37:57.444179+0000","last_active":"2024-05-08T22:37:58.490177+0000","last_peered":"2024-05-08T22:37:58.490177+0000","last_clean":"2024-05-08T22:37:58.490177+0000","last_became_active":"2024-05-08T22:37:57.443673+0000","last_became_peered":"2024-05-08T22:37:57.443673+0000","last_unstale":"2024-05-08T22:37:58.490177+0000","last_undegraded":"2024-05-08T22:37:58.490177+0000","last_fullsized":"2024-05-08T22:37:58.490177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:53.287958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494877+0000","last_change":"2024-05-08T22:37:57.841025+0000","last_active":"2024-05-08T22:37:59.494877+0000","last_peered":"2024-05-08T22:37:59.494877+0000","last_clean":"2024-05-08T22:37:59.494877+0000","last_became_active":"2024-05-08T22:37:57.840591+0000","last_became_peered":"2024-05-08T22:37:57.840591+0000","last_unstale":"2024-05-08T22:37:59.494877+0000","last_undegraded":"2024-05-08T22:37:59.494877+0000","last_fullsized":"2024-05-08T22:37:59.494877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.910062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493660+0000","last_change":"2024-05-08T22:37:57.440960+0000","last_active":"2024-05-08T22:37:58.493660+0000","last_peered":"2024-05-08T22:37:58.493660+0000","last_clean":"2024-05-08T22:37:58.493660+0000","last_became_active":"2024-05-08T22:37:57.440504+0000","last_became_peered":"2024-05-08T22:37:57.440504+0000","last_unstale":"2024-05-08T22:37:58.493660+0000","last_undegraded":"2024-05-08T22:37:58.493660+0000","last_fullsized":"2024-05-08T22:37:58.493660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:06.401155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481269+0000","last_change":"2024-05-08T22:37:57.831846+0000","last_active":"2024-05-08T22:37:58.481269+0000","last_peered":"2024-05-08T22:37:58.481269+0000","last_clean":"2024-05-08T22:37:58.481269+0000","last_became_active":"2024-05-08T22:37:57.829692+0000","last_became_peered":"2024-05-08T22:37:57.829692+0000","last_unstale":"2024-05-08T22:37:58.481269+0000","last_undegraded":"2024-05-08T22:37:58.481269+0000","last_fullsized":"2024-05-08T22:37:58.481269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:53.674154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.843506+0000","last_change":"2024-05-08T22:37:57.437527+0000","last_active":"2024-05-08T22:37:57.843506+0000","last_peered":"2024-05-08T22:37:57.843506+0000","last_clean":"2024-05-08T22:37:57.843506+0000","last_became_active":"2024-05-08T22:37:57.437261+0000","last_became_peered":"2024-05-08T22:37:57.437261+0000","last_unstale":"2024-05-08T22:37:57.843506+0000","last_undegraded":"2024-05-08T22:37:57.843506+0000","last_fullsized":"2024-05-08T22:37:57.843506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:19.394053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481362+0000","last_change":"2024-05-08T22:37:57.832513+0000","last_active":"2024-05-08T22:37:58.481362+0000","last_peered":"2024-05-08T22:37:58.481362+0000","last_clean":"2024-05-08T22:37:58.481362+0000","last_became_active":"2024-05-08T22:37:57.832275+0000","last_became_peered":"2024-05-08T22:37:57.832275+0000","last_unstale":"2024-05-08T22:37:58.481362+0000","last_undegraded":"2024-05-08T22:37:58.481362+0000","last_fullsized":"2024-05-08T22:37:58.481362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:08.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481718+0000","last_change":"2024-05-08T22:37:57.444668+0000","last_active":"2024-05-08T22:37:58.481718+0000","last_peered":"2024-05-08T22:37:58.481718+0000","last_clean":"2024-05-08T22:37:58.481718+0000","last_became_active":"2024-05-08T22:37:57.441598+0000","last_became_peered":"2024-05-08T22:37:57.441598+0000","last_unstale":"2024-05-08T22:37:58.481718+0000","last_undegraded":"2024-05-08T22:37:58.481718+0000","last_fullsized":"2024-05-08T22:37:58.481718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:33.716602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495036+0000","last_change":"2024-05-08T22:37:57.832325+0000","last_active":"2024-05-08T22:37:59.495036+0000","last_peered":"2024-05-08T22:37:59.495036+0000","last_clean":"2024-05-08T22:37:59.495036+0000","last_became_active":"2024-05-08T22:37:57.832067+0000","last_became_peered":"2024-05-08T22:37:57.832067+0000","last_unstale":"2024-05-08T22:37:59.495036+0000","last_undegraded":"2024-05-08T22:37:59.495036+0000","last_fullsized":"2024-05-08T22:37:59.495036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:42.187631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495716+0000","last_change":"2024-05-08T22:37:57.442255+0000","last_active":"2024-05-08T22:37:59.495716+0000","last_peered":"2024-05-08T22:37:59.495716+0000","last_clean":"2024-05-08T22:37:59.495716+0000","last_became_active":"2024-05-08T22:37:57.441960+0000","last_became_peered":"2024-05-08T22:37:57.441960+0000","last_unstale":"2024-05-08T22:37:59.495716+0000","last_undegraded":"2024-05-08T22:37:59.495716+0000","last_fullsized":"2024-05-08T22:37:59.495716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:59.020957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490326+0000","last_change":"2024-05-08T22:37:57.842118+0000","last_active":"2024-05-08T22:37:58.490326+0000","last_peered":"2024-05-08T22:37:58.490326+0000","last_clean":"2024-05-08T22:37:58.490326+0000","last_became_active":"2024-05-08T22:37:57.841826+0000","last_became_peered":"2024-05-08T22:37:57.841826+0000","last_unstale":"2024-05-08T22:37:58.490326+0000","last_undegraded":"2024-05-08T22:37:58.490326+0000","last_fullsized":"2024-05-08T22:37:58.490326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:55.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495061+0000","last_change":"2024-05-08T22:37:57.439634+0000","last_active":"2024-05-08T22:37:59.495061+0000","last_peered":"2024-05-08T22:37:59.495061+0000","last_clean":"2024-05-08T22:37:59.495061+0000","last_became_active":"2024-05-08T22:37:57.439387+0000","last_became_peered":"2024-05-08T22:37:57.439387+0000","last_unstale":"2024-05-08T22:37:59.495061+0000","last_undegraded":"2024-05-08T22:37:59.495061+0000","last_fullsized":"2024-05-08T22:37:59.495061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:49.158578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490407+0000","last_change":"2024-05-08T22:37:57.846904+0000","last_active":"2024-05-08T22:37:58.490407+0000","last_peered":"2024-05-08T22:37:58.490407+0000","last_clean":"2024-05-08T22:37:58.490407+0000","last_became_active":"2024-05-08T22:37:57.846651+0000","last_became_peered":"2024-05-08T22:37:57.846651+0000","last_unstale":"2024-05-08T22:37:58.490407+0000","last_undegraded":"2024-05-08T22:37:58.490407+0000","last_fullsized":"2024-05-08T22:37:58.490407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:05.340141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494931+0000","last_change":"2024-05-08T22:37:57.443724+0000","last_active":"2024-05-08T22:37:59.494931+0000","last_peered":"2024-05-08T22:37:59.494931+0000","last_clean":"2024-05-08T22:37:59.494931+0000","last_became_active":"2024-05-08T22:37:57.443460+0000","last_became_peered":"2024-05-08T22:37:57.443460+0000","last_unstale":"2024-05-08T22:37:59.494931+0000","last_undegraded":"2024-05-08T22:37:59.494931+0000","last_fullsized":"2024-05-08T22:37:59.494931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:08.118921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489618+0000","last_change":"2024-05-08T22:37:57.826029+0000","last_active":"2024-05-08T22:37:58.489618+0000","last_peered":"2024-05-08T22:37:58.489618+0000","last_clean":"2024-05-08T22:37:58.489618+0000","last_became_active":"2024-05-08T22:37:57.825743+0000","last_became_peered":"2024-05-08T22:37:57.825743+0000","last_unstale":"2024-05-08T22:37:58.489618+0000","last_undegraded":"2024-05-08T22:37:58.489618+0000","last_fullsized":"2024-05-08T22:37:58.489618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.587376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.494908+0000","last_change":"2024-05-08T22:37:57.444844+0000","last_active":"2024-05-08T22:37:59.494908+0000","last_peered":"2024-05-08T22:37:59.494908+0000","last_clean":"2024-05-08T22:37:59.494908+0000","last_became_active":"2024-05-08T22:37:57.444488+0000","last_became_peered":"2024-05-08T22:37:57.444488+0000","last_unstale":"2024-05-08T22:37:59.494908+0000","last_undegraded":"2024-05-08T22:37:59.494908+0000","last_fullsized":"2024-05-08T22:37:59.494908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:36.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.846950+0000","last_change":"2024-05-08T22:37:57.843798+0000","last_active":"2024-05-08T22:37:57.846950+0000","last_peered":"2024-05-08T22:37:57.846950+0000","last_clean":"2024-05-08T22:37:57.846950+0000","last_became_active":"2024-05-08T22:37:57.842276+0000","last_became_peered":"2024-05-08T22:37:57.842276+0000","last_unstale":"2024-05-08T22:37:57.846950+0000","last_undegraded":"2024-05-08T22:37:57.846950+0000","last_fullsized":"2024-05-08T22:37:57.846950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:34.166032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.494078+0000","last_change":"2024-05-08T22:37:57.438922+0000","last_active":"2024-05-08T22:37:58.494078+0000","last_peered":"2024-05-08T22:37:58.494078+0000","last_clean":"2024-05-08T22:37:58.494078+0000","last_became_active":"2024-05-08T22:37:57.438655+0000","last_became_peered":"2024-05-08T22:37:57.438655+0000","last_unstale":"2024-05-08T22:37:58.494078+0000","last_undegraded":"2024-05-08T22:37:58.494078+0000","last_fullsized":"2024-05-08T22:37:58.494078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:09:11.390285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489349+0000","last_change":"2024-05-08T22:37:57.828420+0000","last_active":"2024-05-08T22:37:58.489349+0000","last_peered":"2024-05-08T22:37:58.489349+0000","last_clean":"2024-05-08T22:37:58.489349+0000","last_became_active":"2024-05-08T22:37:57.828132+0000","last_became_peered":"2024-05-08T22:37:57.828132+0000","last_unstale":"2024-05-08T22:37:58.489349+0000","last_undegraded":"2024-05-08T22:37:58.489349+0000","last_fullsized":"2024-05-08T22:37:58.489349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:23.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490435+0000","last_change":"2024-05-08T22:37:57.447723+0000","last_active":"2024-05-08T22:37:58.490435+0000","last_peered":"2024-05-08T22:37:58.490435+0000","last_clean":"2024-05-08T22:37:58.490435+0000","last_became_active":"2024-05-08T22:37:57.447478+0000","last_became_peered":"2024-05-08T22:37:57.447478+0000","last_unstale":"2024-05-08T22:37:58.490435+0000","last_undegraded":"2024-05-08T22:37:58.490435+0000","last_fullsized":"2024-05-08T22:37:58.490435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:09.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.848541+0000","last_change":"2024-05-08T22:37:57.847595+0000","last_active":"2024-05-08T22:37:57.848541+0000","last_peered":"2024-05-08T22:37:57.848541+0000","last_clean":"2024-05-08T22:37:57.848541+0000","last_became_active":"2024-05-08T22:37:57.847254+0000","last_became_peered":"2024-05-08T22:37:57.847254+0000","last_unstale":"2024-05-08T22:37:57.848541+0000","last_undegraded":"2024-05-08T22:37:57.848541+0000","last_fullsized":"2024-05-08T22:37:57.848541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:11.256993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493554+0000","last_change":"2024-05-08T22:37:57.437575+0000","last_active":"2024-05-08T22:37:58.493554+0000","last_peered":"2024-05-08T22:37:58.493554+0000","last_clean":"2024-05-08T22:37:58.493554+0000","last_became_active":"2024-05-08T22:37:57.437308+0000","last_became_peered":"2024-05-08T22:37:57.437308+0000","last_unstale":"2024-05-08T22:37:58.493554+0000","last_undegraded":"2024-05-08T22:37:58.493554+0000","last_fullsized":"2024-05-08T22:37:58.493554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:33.154187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.490280+0000","last_change":"2024-05-08T22:37:57.845583+0000","last_active":"2024-05-08T22:37:58.490280+0000","last_peered":"2024-05-08T22:37:58.490280+0000","last_clean":"2024-05-08T22:37:58.490280+0000","last_became_active":"2024-05-08T22:37:57.845321+0000","last_became_peered":"2024-05-08T22:37:57.845321+0000","last_unstale":"2024-05-08T22:37:58.490280+0000","last_undegraded":"2024-05-08T22:37:58.490280+0000","last_fullsized":"2024-05-08T22:37:58.490280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:27.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493454+0000","last_change":"2024-05-08T22:37:57.440921+0000","last_active":"2024-05-08T22:37:58.493454+0000","last_peered":"2024-05-08T22:37:58.493454+0000","last_clean":"2024-05-08T22:37:58.493454+0000","last_became_active":"2024-05-08T22:37:57.440652+0000","last_became_peered":"2024-05-08T22:37:57.440652+0000","last_unstale":"2024-05-08T22:37:58.493454+0000","last_undegraded":"2024-05-08T22:37:58.493454+0000","last_fullsized":"2024-05-08T22:37:58.493454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:02.665256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489427+0000","last_change":"2024-05-08T22:37:57.828599+0000","last_active":"2024-05-08T22:37:58.489427+0000","last_peered":"2024-05-08T22:37:58.489427+0000","last_clean":"2024-05-08T22:37:58.489427+0000","last_became_active":"2024-05-08T22:37:57.828254+0000","last_became_peered":"2024-05-08T22:37:57.828254+0000","last_unstale":"2024-05-08T22:37:58.489427+0000","last_undegraded":"2024-05-08T22:37:58.489427+0000","last_fullsized":"2024-05-08T22:37:58.489427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:30.500016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489396+0000","last_change":"2024-05-08T22:37:57.442291+0000","last_active":"2024-05-08T22:37:58.489396+0000","last_peered":"2024-05-08T22:37:58.489396+0000","last_clean":"2024-05-08T22:37:58.489396+0000","last_became_active":"2024-05-08T22:37:57.441935+0000","last_became_peered":"2024-05-08T22:37:57.441935+0000","last_unstale":"2024-05-08T22:37:58.489396+0000","last_undegraded":"2024-05-08T22:37:58.489396+0000","last_fullsized":"2024-05-08T22:37:58.489396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:55.703307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489528+0000","last_change":"2024-05-08T22:37:57.832660+0000","last_active":"2024-05-08T22:37:58.489528+0000","last_peered":"2024-05-08T22:37:58.489528+0000","last_clean":"2024-05-08T22:37:58.489528+0000","last_became_active":"2024-05-08T22:37:57.832291+0000","last_became_peered":"2024-05-08T22:37:57.832291+0000","last_unstale":"2024-05-08T22:37:58.489528+0000","last_undegraded":"2024-05-08T22:37:58.489528+0000","last_fullsized":"2024-05-08T22:37:58.489528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:40.277738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.481450+0000","last_change":"2024-05-08T22:37:57.448069+0000","last_active":"2024-05-08T22:37:58.481450+0000","last_peered":"2024-05-08T22:37:58.481450+0000","last_clean":"2024-05-08T22:37:58.481450+0000","last_became_active":"2024-05-08T22:37:57.446523+0000","last_became_peered":"2024-05-08T22:37:57.446523+0000","last_unstale":"2024-05-08T22:37:58.481450+0000","last_undegraded":"2024-05-08T22:37:58.481450+0000","last_fullsized":"2024-05-08T22:37:58.481450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:44.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495858+0000","last_change":"2024-05-08T22:37:57.830871+0000","last_active":"2024-05-08T22:37:59.495858+0000","last_peered":"2024-05-08T22:37:59.495858+0000","last_clean":"2024-05-08T22:37:59.495858+0000","last_became_active":"2024-05-08T22:37:57.830594+0000","last_became_peered":"2024-05-08T22:37:57.830594+0000","last_unstale":"2024-05-08T22:37:59.495858+0000","last_undegraded":"2024-05-08T22:37:59.495858+0000","last_fullsized":"2024-05-08T22:37:59.495858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:02.234727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.489371+0000","last_change":"2024-05-08T22:37:57.438169+0000","last_active":"2024-05-08T22:37:58.489371+0000","last_peered":"2024-05-08T22:37:58.489371+0000","last_clean":"2024-05-08T22:37:58.489371+0000","last_became_active":"2024-05-08T22:37:57.437880+0000","last_became_peered":"2024-05-08T22:37:57.437880+0000","last_unstale":"2024-05-08T22:37:58.489371+0000","last_undegraded":"2024-05-08T22:37:58.489371+0000","last_fullsized":"2024-05-08T22:37:58.489371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:58.133761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495906+0000","last_change":"2024-05-08T22:37:57.846104+0000","last_active":"2024-05-08T22:37:59.495906+0000","last_peered":"2024-05-08T22:37:59.495906+0000","last_clean":"2024-05-08T22:37:59.495906+0000","last_became_active":"2024-05-08T22:37:57.845848+0000","last_became_peered":"2024-05-08T22:37:57.845848+0000","last_unstale":"2024-05-08T22:37:59.495906+0000","last_undegraded":"2024-05-08T22:37:59.495906+0000","last_fullsized":"2024-05-08T22:37:59.495906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:06:32.854193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495531+0000","last_change":"2024-05-08T22:37:57.437069+0000","last_active":"2024-05-08T22:37:59.495531+0000","last_peered":"2024-05-08T22:37:59.495531+0000","last_clean":"2024-05-08T22:37:59.495531+0000","last_became_active":"2024-05-08T22:37:57.436461+0000","last_became_peered":"2024-05-08T22:37:57.436461+0000","last_unstale":"2024-05-08T22:37:59.495531+0000","last_undegraded":"2024-05-08T22:37:59.495531+0000","last_fullsized":"2024-05-08T22:37:59.495531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:31.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:58.493530+0000","last_change":"2024-05-08T22:37:57.843298+0000","last_active":"2024-05-08T22:37:58.493530+0000","last_peered":"2024-05-08T22:37:58.493530+0000","last_clean":"2024-05-08T22:37:58.493530+0000","last_became_active":"2024-05-08T22:37:57.842988+0000","last_became_peered":"2024-05-08T22:37:57.842988+0000","last_unstale":"2024-05-08T22:37:58.493530+0000","last_undegraded":"2024-05-08T22:37:58.493530+0000","last_fullsized":"2024-05-08T22:37:58.493530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:57.410053+0000","last_clean_scrub_stamp":"2024-05-08T22:37:57.410053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:17.302357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:59.495015+0000","last_change":"2024-05-08T22:37:57.444385+0000","last_active":"2024-05-08T22:37:59.495015+0000","last_peered":"2024-05-08T22:37:59.495015+0000","last_clean":"2024-05-08T22:37:59.495015+0000","last_became_active":"2024-05-08T22:37:57.444124+0000","last_became_peered":"2024-05-08T22:37:57.444124+0000","last_unstale":"2024-05-08T22:37:59.495015+0000","last_undegraded":"2024-05-08T22:37:59.495015+0000","last_fullsized":"2024-05-08T22:37:59.495015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:37:56.398208+0000","last_clean_scrub_stamp":"2024-05-08T22:37:56.398208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:35.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":220,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T22:37:57.826502+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:37:57.826502+0000","last_peered":"2024-05-08T22:37:57.826502+0000","last_clean":"2024-05-08T22:37:57.826502+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:37:57.826502+0000","last_undegraded":"2024-05-08T22:37:57.826502+0000","last_fullsized":"2024-05-08T22:37:57.826502+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":1011712,"data_stored":457701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.47599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89100000000000001,"5min":1.101,"15min":1.101},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.018,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.57899999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.746}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1140000000000001,"5min":1.306,"15min":1.306},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2310000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.028}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.274,"5min":1.339,"15min":1.339},"last":1.137},{"interface":"front","average":{"1min":1.04,"5min":1.0089999999999999,"15min":1.0269999999999999},"min":{"1min":0.76000000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2749999999999999,"5min":1.476,"15min":1.476},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98199999999999998,"15min":0.97199999999999998},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2370000000000001,"5min":1.446,"15min":1.446},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.753,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.341,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.105},"min":{"1min":0.82199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.298,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.036,"15min":1.034},"min":{"1min":0.81000000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4570000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.91800000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":457231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.84099999999999997}]},{"osd":2,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":1.04,"15min":1.04},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.58399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.98399999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65900000000000003}]},{"osd":3,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.022},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2130000000000001,"5min":1.583,"15min":1.583},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.97499999999999998},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2050000000000001,"5min":1.431,"15min":1.431},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.81399999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2470000000000001,"5min":1.508,"15min":1.508},"last":0.93799999999999994}]},{"osd":5,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.028,"15min":1.083},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.397,"5min":1.726,"15min":1.726},"last":0.90400000000000003}]},{"osd":6,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.77200000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.873},{"interface":"front","average":{"1min":1.02,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.446,"5min":1.659,"15min":1.659},"last":0.82199999999999995}]},{"osd":7,"last update":"Wed May 8 22:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.071},"min":{"1min":0.85399999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3129999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.145},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0569999999999999},"min":{"1min":0.81699999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.409,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.998}]}]},{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1605632,"data_stored":1047509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85499999999999998,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2749999999999999,"5min":1.282,"15min":1.282},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.84099999999999997,"15min":0.85999999999999999},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.91300000000000003}]},{"osd":1,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91800000000000004,"15min":0.91000000000000003},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.099,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.077},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.85799999999999998},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1140000000000001,"5min":1.419,"15min":1.419},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95099999999999996,"15min":0.96199999999999997},"min":{"1min":0.69199999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.177,"5min":1.337,"15min":1.337},"last":1.054}]},{"osd":3,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.79400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5269999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.72099999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88500000000000001,"15min":0.89300000000000002},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.103,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95999999999999996,"15min":0.98199999999999998},"min":{"1min":0.67000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.482,"15min":1.482},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.85399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.139}]},{"osd":6,"last update":"Wed May 8 22:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.76500000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4179999999999999,"5min":1.599,"15min":1.599},"last":0.95099999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36208,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335632,"statfs":{"total":96636764160,"available":96599687168,"internally_reserved":0,"allocated":1019904,"data_stored":458683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.141,"15min":1.141},"last":0.76000000000000001}]},{"osd":1,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3300000000000001,"5min":1.758,"15min":1.758},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91800000000000004,"15min":0.93000000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1850000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94599999999999995,"15min":0.98599999999999999},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.92900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.397,"5min":1.419,"15min":1.419},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.034,"15min":1.0960000000000001},"min":{"1min":0.68000000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2110000000000001,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.99099999999999999},"min":{"1min":0.625,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2869999999999999,"5min":1.538,"15min":1.538},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":1.0469999999999999},"min":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.149,"5min":1.458,"15min":1.458},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.83699999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.28,"5min":1.681,"15min":1.681},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.998,"5min":1.0840000000000001,"15min":1.1399999999999999},"min":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2050000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.018}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.014,"15min":1.085},"min":{"1min":0.71899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0780000000000001,"5min":1.835,"15min":1.835},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.179,"15min":1.2350000000000001},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3300000000000001,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.0329999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":1011712,"data_stored":457153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82399999999999995,"15min":0.79800000000000004},"min":{"1min":0.64700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.145,"5min":1.52,"15min":1.52},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.016,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.266,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90400000000000003,"15min":0.88},"min":{"1min":0.71899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.83999999999999997},"min":{"1min":0.67600000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.02},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97499999999999998,"15min":0.96499999999999997},"min":{"1min":0.71599999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.70099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.016,"15min":0.98599999999999999},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.327}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96799999999999997,"15min":0.91600000000000004},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2589999999999999,"5min":1.605,"15min":1.605},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.069,"15min":1.044},"min":{"1min":0.89200000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3080000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1040000000000001,"15min":1.0880000000000001},"min":{"1min":0.98299999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.411,"5min":1.538,"15min":1.538},"last":1.224},{"interface":"front","average":{"1min":1.133,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.91500000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3089999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.385}]}]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":1028096,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.113,"5min":1.28,"15min":1.28},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.294,"15min":1.294},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.871},"min":{"1min":0.65800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.163,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90700000000000003,"15min":0.88600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.173,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85899999999999999,"15min":0.88700000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0329999999999999,"5min":1.29,"15min":1.29},"last":0.751},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91900000000000004,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.135,"5min":1.637,"15min":1.637},"last":1.0529999999999999}]},{"osd":3,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1619999999999999,"5min":2.601,"15min":2.601},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.032,"15min":1.105},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.0349999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.121,"5min":1.792,"15min":1.792},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.026},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.137,"5min":1.575,"15min":1.575},"last":0.89600000000000002}]},{"osd":6,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0489999999999999,"15min":1.0900000000000001},"min":{"1min":0.69399999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1100000000000001,"5min":1.831,"15min":1.831},"last":1.103},{"interface":"front","average":{"1min":1.014,"5min":1.093,"15min":1.1020000000000001},"min":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2210000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0049999999999999}]},{"osd":7,"last update":"Wed May 8 22:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.071,"15min":1.105},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.23,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.159,"15min":1.177},"min":{"1min":0.878,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3,"5min":1.944,"15min":1.944},"last":1.1910000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":457861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.133,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.98499999999999999}]},{"osd":1,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed May 8 22:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":1.008,"15min":1.008},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.58799999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.752,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.333,"5min":1.363,"15min":1.363},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.006,"15min":1.028},"min":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.27,"5min":1.282,"15min":1.282},"last":1.02}]},{"osd":5,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.407,"5min":1.484,"15min":1.484},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.87}]},{"osd":6,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0600000000000001},"min":{"1min":0.84499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0700000000000001,"15min":1.0840000000000001},"min":{"1min":0.871,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4450000000000001,"5min":1.673,"15min":1.673},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed May 8 22:37:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.048},{"interface":"front","average":{"1min":1.133,"5min":1.119,"15min":1.1299999999999999},"min":{"1min":0.878,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0820000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542205,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1601536,"data_stored":1047487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.617,"15min":1.617},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":1.446,"15min":1.446},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0129999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed May 8 22:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.56799999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3089999999999999,"5min":1.714,"15min":1.714},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2569999999999999,"5min":1.327,"15min":1.327},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2709999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.73499999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3,"5min":1.669,"15min":1.669},"last":1.0349999999999999}]},{"osd":5,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.89900000000000002,"15min":0.91100000000000003},"min":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2210000000000001,"5min":1.474,"15min":1.474},"last":0.999}]},{"osd":6,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.96499999999999997},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2869999999999999,"5min":1.397,"15min":1.397},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":1.0469999999999999},"min":{"1min":0.78600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.177,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Wed May 8 22:37:15 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0649999999999999,"15min":1.089},"min":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.234,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.117},{"interface":"front","average":{"1min":1.018,"5min":1.008,"15min":1.022},"min":{"1min":0.77500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:38:06.581 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:38:06.581 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:38:06.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 Processor -- start 2024-05-08T22:38:06.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 -- start start 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 0x7f0bf80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0bf80b2ca0 0x7f0bf806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf80b22d0 con 0x7f0bf8142fd0 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf80b19e0 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf815e640 con 0x7f0bf8142890 2024-05-08T22:38:06.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf5d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0bf80b2ca0 0x7f0bf806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf6d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0bf8142fd0 0x7f0bf815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48646/0 (socket says 172.21.15.179:48646) 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf6d9d700 1 -- 172.21.15.179:0/2014058004 learned_addr learned my addr 172.21.15.179:0/2014058004 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf659c700 1 --2- 172.21.15.179:0/2014058004 >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 0x7f0bf80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf559a700 1 -- 172.21.15.179:0/2014058004 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964598475 0 0) 0x7f0bf80b22d0 con 0x7f0bf8142fd0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf559a700 1 -- 172.21.15.179:0/2014058004 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be40035d0 con 0x7f0bf8142fd0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/2014058004 >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 msgr2=0x7f0bf80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf5d9b700 1 --2- 172.21.15.179:0/2014058004 >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 0x7f0bf80b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/2014058004 >> v1:172.21.15.179:6789/0 conn(0x7f0bf8142fd0 legacy=0x7f0bf815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/2014058004 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf80b1c80 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.708+0000 7f0bf659c700 1 --2- 172.21.15.179:0/2014058004 >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 0x7f0bf80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf5d9b700 1 --2- 172.21.15.179:0/2014058004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf80b2ca0 0x7f0bf806cda0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80720b0 tx=0x7f0be0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b391b65f78977d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf559a700 1 -- 172.21.15.179:0/2014058004 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0011830 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf559a700 1 -- 172.21.15.179:0/2014058004 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0011990 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf559a700 1 -- 172.21.15.179:0/2014058004 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0011d00 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- 172.21.15.179:0/2014058004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf80b2ca0 msgr2=0x7f0bf806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 --2- 172.21.15.179:0/2014058004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf80b2ca0 0x7f0bf806cda0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80720b0 tx=0x7f0be0009b30 comp rx=0 tx=0).stop 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- 172.21.15.179:0/2014058004 shutdown_connections 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 --2- 172.21.15.179:0/2014058004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf80b2ca0 0x7f0bf806cda0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 --2- 172.21.15.179:0/2014058004 >> v2:172.21.15.195:3300/0 conn(0x7f0bf8142890 0x7f0bf80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- 172.21.15.179:0/2014058004 >> 172.21.15.179:0/2014058004 conn(0x7f0bf80cea30 msgr2=0x7f0bf80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- 172.21.15.179:0/2014058004 shutdown_connections 2024-05-08T22:38:06.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- 172.21.15.179:0/2014058004 wait complete. 2024-05-08T22:38:06.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 Processor -- start 2024-05-08T22:38:06.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.712+0000 7f0bf759e700 1 -- start start 2024-05-08T22:38:06.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bf80b2ca0 0x7f0bf813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bf8142fd0 0x7f0bf8140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81f29d0 con 0x7f0bf8142890 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf813d910 con 0x7f0bf8142fd0 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf813a6d0 con 0x7f0bf80b2ca0 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39288/0 (socket says 172.21.15.179:39288) 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/1079285882 learned_addr learned my addr 172.21.15.179:0/1079285882 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf659c700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bf80b2ca0 0x7f0bf813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf6d9d700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bf8142fd0 0x7f0bf8140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bf80b2ca0 msgr2=0x7f0bf813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bf80b2ca0 0x7f0bf813fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bf8142fd0 msgr2=0x7f0bf8140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bf8142fd0 0x7f0bf8140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 -- 172.21.15.179:0/1079285882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00094b0 con 0x7f0bf8142890 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf5d9b700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0be0000c00 tx=0x7f0be0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004610 con 0x7f0bf8142890 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be001f070 con 0x7f0bf8142890 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be00094b0 con 0x7f0bf8142890 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0003900 con 0x7f0bf8142890 2024-05-08T22:38:06.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf813a970 con 0x7f0bf8142890 2024-05-08T22:38:06.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.716+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf804e700 con 0x7f0bf8142890 2024-05-08T22:38:06.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.720+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0be0003a60 con 0x7f0bf8142890 2024-05-08T22:38:06.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.720+0000 7f0beeffd700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 0x7f0bd8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:06.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.720+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0be0021070 con 0x7f0bf8142890 2024-05-08T22:38:06.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.720+0000 7f0bf659c700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 0x7f0bd8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:06.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.720+0000 7f0bf659c700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 0x7f0bd8068bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009d80 tx=0x7f0bf0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:06.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.724+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0be0052c70 con 0x7f0bf8142890 2024-05-08T22:38:06.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.952+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0bf80aead0 con 0x7f0bf8142890 2024-05-08T22:38:06.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0beeffd700 1 -- 172.21.15.179:0/1079285882 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0be0015020 con 0x7f0bf8142890 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 msgr2=0x7f0bd8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 0x7f0bd8068bf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009d80 tx=0x7f0bf0009240 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 msgr2=0x7f0bf814c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0be0000c00 tx=0x7f0be0004770 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 shutdown_connections 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0bd8066730 0x7f0bd8068bf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bf8142fd0 0x7f0bf8140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf8142890 0x7f0bf814c0d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 --2- 172.21.15.179:0/1079285882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0bf80b2ca0 0x7f0bf813fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 >> 172.21.15.179:0/1079285882 conn(0x7f0bf80cea30 msgr2=0x7f0bf806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 shutdown_connections 2024-05-08T22:38:06.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:06.960+0000 7f0bf759e700 1 -- 172.21.15.179:0/1079285882 wait complete. 2024-05-08T22:38:06.967 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:38:06.978 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:07.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 Processor -- start 2024-05-08T22:38:07.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 -- start start 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4018161790 0x7f4018164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4018165d50 con 0x7f401815ef80 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401806ad70 con 0x7f401806de70 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401806e4b0 con 0x7f4018161790 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f401815ef80 0x7f401806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48648/0 (socket says 172.21.15.179:48648) 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401659c700 1 -- 172.21.15.179:0/2460800028 learned_addr learned my addr 172.21.15.179:0/2460800028 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 --2- 172.21.15.179:0/2460800028 >> v2:172.21.15.179:3300/0 conn(0x7f4018161790 0x7f4018164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876505527 0 0) 0x7f4018165d50 con 0x7f401815ef80 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40040035d0 con 0x7f401815ef80 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301832949 0 0) 0x7f40040035d0 con 0x7f401815ef80 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 >> v2:172.21.15.179:3300/0 conn(0x7f4018161790 msgr2=0x7f4018164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 --2- 172.21.15.179:0/2460800028 >> v2:172.21.15.179:3300/0 conn(0x7f4018161790 0x7f4018164db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 >> v1:172.21.15.195:6789/0 conn(0x7f401806de70 legacy=0x7f401806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40181654b0 con 0x7f401815ef80 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f4008001670 con 0x7f401815ef80 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 >> v1:172.21.15.179:6789/0 conn(0x7f401815ef80 legacy=0x7f401806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004003a20 0x7f4004003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4004004870 0x7f4004004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 0x7f400400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401806e4b0 con 0x7f4004005230 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400400c1d0 con 0x7f4004004870 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400400c390 con 0x7f4004003a20 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 0x7f400400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f401659c700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004003a20 0x7f4004003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 -- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004003a20 msgr2=0x7f4004003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004003a20 0x7f4004003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 -- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4004004870 msgr2=0x7f4004004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4004004870 0x7f4004004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 -- 172.21.15.179:0/2460800028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40040035d0 con 0x7f4004005230 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 -- 172.21.15.179:0/2460800028 reap_dead start 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4016d9d700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 0x7f400400ba30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4010009fc0 tx=0x7f40040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ec8a050300c982 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010003c40 con 0x7f4004005230 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401000f040 con 0x7f4004005230 2024-05-08T22:38:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.092+0000 7f4015d9b700 1 -- 172.21.15.179:0/2460800028 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000ea30 con 0x7f4004005230 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 msgr2=0x7f400400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 0x7f400400ba30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4010009fc0 tx=0x7f40040058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- 172.21.15.179:0/2460800028 shutdown_connections 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 --2- 172.21.15.179:0/2460800028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4004005230 0x7f400400ba30 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- 172.21.15.179:0/2460800028 >> 172.21.15.179:0/2460800028 conn(0x7f40180ce9d0 msgr2=0x7f40180cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- 172.21.15.179:0/2460800028 shutdown_connections 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- 172.21.15.179:0/2460800028 wait complete. 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 Processor -- start 2024-05-08T22:38:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- start start 2024-05-08T22:38:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004005230 0x7f40180cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40181f6d40 0x7f401815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180cd500 con 0x7f40181f71a0 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180ccb40 con 0x7f40181f6d40 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.096+0000 7f401759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018156c60 con 0x7f4004005230 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39306/0 (socket says 172.21.15.179:39306) 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 -- 172.21.15.179:0/3656795820 learned_addr learned my addr 172.21.15.179:0/3656795820 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f401659c700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004005230 0x7f40180cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400dd9b700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40181f6d40 0x7f401815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 -- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004005230 msgr2=0x7f40180cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004005230 0x7f40180cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 -- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40181f6d40 msgr2=0x7f401815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40181f6d40 0x7f401815b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 -- 172.21.15.179:0/3656795820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018156f40 con 0x7f40181f71a0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f4016d9d700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4018071e90 tx=0x7f4010009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004450 con 0x7f40181f71a0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40100045b0 con 0x7f40181f71a0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4018157220 con 0x7f40181f71a0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004960 con 0x7f40181f71a0 2024-05-08T22:38:07.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40181577e0 con 0x7f40181f71a0 2024-05-08T22:38:07.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40181579a0 con 0x7f40181f71a0 2024-05-08T22:38:07.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4010014480 con 0x7f40181f71a0 2024-05-08T22:38:07.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.100+0000 7f400f7fe700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 0x7f3ff8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.104+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4010085990 con 0x7f40181f71a0 2024-05-08T22:38:07.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.104+0000 7f401659c700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 0x7f3ff8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.104+0000 7f401659c700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 0x7f3ff8068c20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40080015f0 tx=0x7f4008007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.108+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4010052c50 con 0x7f40181f71a0 2024-05-08T22:38:07.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.312+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4018067740 con 0x7f40181f71a0 2024-05-08T22:38:07.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f400f7fe700 1 -- 172.21.15.179:0/3656795820 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f4010018090 con 0x7f40181f71a0 2024-05-08T22:38:07.318 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:07.319 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:00.821878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56162":{"gid":56162,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/123759130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":123759130},{"type":"v1","addr":"172.21.15.195:6837","nonce":123759130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 msgr2=0x7f3ff8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 0x7f3ff8068c20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40080015f0 tx=0x7f4008007d50 comp rx=0 tx=0).stop 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 msgr2=0x7f401814b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4018071e90 tx=0x7f4010009750 comp rx=0 tx=0).stop 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 shutdown_connections 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3ff8066760 0x7f3ff8068c20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40181f6d40 0x7f401815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40181f71a0 0x7f401814b600 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 --2- 172.21.15.179:0/3656795820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4004005230 0x7f40180cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 >> 172.21.15.179:0/3656795820 conn(0x7f40180ce9d0 msgr2=0x7f401815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 shutdown_connections 2024-05-08T22:38:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.316+0000 7f401759e700 1 -- 172.21.15.179:0/3656795820 wait complete. 2024-05-08T22:38:07.323 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 39 2024-05-08T22:38:07.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:37:57.812267+0000', 'modified': '2024-05-08T22:38:00.821878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56162': {'gid': 56162, 'name': 'd', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/123759130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 123759130}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 123759130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:38:07.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:38:07.737 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:38:07.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 Processor -- start 2024-05-08T22:38:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 -- start start 2024-05-08T22:38:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f77b40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b406b540 con 0x7f77b40b25d0 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b406ac50 con 0x7f77b406e180 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b406aef0 con 0x7f77b406dda0 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f77b40b25d0 0x7f77b40b2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48664/0 (socket says 172.21.15.179:48664) 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b37fe700 1 -- 172.21.15.179:0/4230479272 learned_addr learned my addr 172.21.15.179:0/4230479272 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f77b40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949597403 0 0) 0x7f77b406b540 con 0x7f77b40b25d0 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f779c0035d0 con 0x7f77b40b25d0 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013263969 0 0) 0x7f77b406ac50 con 0x7f77b406e180 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b406b540 con 0x7f77b406e180 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1026920505 0 0) 0x7f779c0035d0 con 0x7f77b40b25d0 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 msgr2=0x7f77b40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f77b40cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 >> v1:172.21.15.195:6789/0 conn(0x7f77b406e180 legacy=0x7f77b406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b406b1d0 con 0x7f77b40b25d0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f77ac001970 con 0x7f77b40b25d0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 >> v1:172.21.15.179:6789/0 conn(0x7f77b40b25d0 legacy=0x7f77b40b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c003be0 0x7f779c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f779c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f779c004db0 0x7f779c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b406aef0 con 0x7f779c004db0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c00bae0 con 0x7f779c003be0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.860+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c00bca0 con 0x7f77b406dda0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f779c004db0 0x7f779c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b27fc700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f779c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c003be0 0x7f779c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 -- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 msgr2=0x7f779c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b406dda0 0x7f779c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 -- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c003be0 msgr2=0x7f779c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c003be0 0x7f779c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 -- 172.21.15.179:0/4230479272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c0035d0 con 0x7f779c004db0 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f779c004db0 0x7f779c005230 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00a680 tx=0x7f779c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bdec5920da879e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00a140 con 0x7f779c004db0 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac0070d0 con 0x7f779c004db0 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b1ffb700 1 -- 172.21.15.179:0/4230479272 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007480 con 0x7f779c004db0 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- 172.21.15.179:0/4230479272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f779c004db0 msgr2=0x7f779c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 --2- 172.21.15.179:0/4230479272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f779c004db0 0x7f779c005230 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00a680 tx=0x7f779c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b37fe700 1 -- 172.21.15.179:0/4230479272 reap_dead start 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- 172.21.15.179:0/4230479272 shutdown_connections 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- 172.21.15.179:0/4230479272 >> 172.21.15.179:0/4230479272 conn(0x7f77b40ce960 msgr2=0x7f77b40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- 172.21.15.179:0/4230479272 shutdown_connections 2024-05-08T22:38:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- 172.21.15.179:0/4230479272 wait complete. 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 Processor -- start 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.864+0000 7f77b3fff700 1 -- start start 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 0x7f77b40afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77b415f940 0x7f77b415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b02d0 con 0x7f77b4151cd0 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40a75e0 con 0x7f77b415f940 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40a78c0 con 0x7f77b40af910 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39312/0 (socket says 172.21.15.179:39312) 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 -- 172.21.15.179:0/101599751 learned_addr learned my addr 172.21.15.179:0/101599751 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b37fe700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77b415f940 0x7f77b415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b27fc700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 0x7f77b40afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 -- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 msgr2=0x7f77b40afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 0x7f77b40afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 -- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77b415f940 msgr2=0x7f77b415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77b415f940 0x7f77b415fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 -- 172.21.15.179:0/101599751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40a7ba0 con 0x7f77b4151cd0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b27fc700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 0x7f77b40afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f77a4011a20 tx=0x7f77a4011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f779b7fe700 1 -- 172.21.15.179:0/101599751 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a400f940 con 0x7f77b4151cd0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f779b7fe700 1 -- 172.21.15.179:0/101599751 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a400faa0 con 0x7f77b4151cd0 2024-05-08T22:38:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b40a7e80 con 0x7f77b4151cd0 2024-05-08T22:38:07.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f779b7fe700 1 -- 172.21.15.179:0/101599751 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a400fe50 con 0x7f77b4151cd0 2024-05-08T22:38:07.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.868+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b4160300 con 0x7f77b4151cd0 2024-05-08T22:38:07.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f779b7fe700 1 -- 172.21.15.179:0/101599751 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77a4014440 con 0x7f77b4151cd0 2024-05-08T22:38:07.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f779b7fe700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 0x7f77a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f779b7fe700 1 -- 172.21.15.179:0/101599751 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f77a4084a00 con 0x7f77b4151cd0 2024-05-08T22:38:07.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f77b27fc700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 0x7f77a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f77b27fc700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 0x7f77a0068bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77b4160780 tx=0x7f77a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f77bca36740 1 Processor -- start 2024-05-08T22:38:07.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.872+0000 7f77bca36740 1 -- start start 2024-05-08T22:38:07.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdd8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdd89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd37d80 con 0xe10b00 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xd35510 con 0xe11420 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xd30e00 con 0xe07be0 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39326/0 (socket says 172.21.15.179:39326) 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 -- 172.21.15.179:0/953476646 learned_addr learned my addr 172.21.15.179:0/953476646 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b27fc700 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdd8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b37fe700 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdd89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 -- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 msgr2=0xdd8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdd8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 -- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 msgr2=0xdd89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdd89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 -- 172.21.15.179:0/953476646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xba3960 con 0xe10b00 2024-05-08T22:38:07.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f77a4082d60 tx=0x7f77a4051e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a74f786808f47b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f7798ff9700 1 -- 172.21.15.179:0/953476646 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a405dda0 con 0xe10b00 2024-05-08T22:38:07.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f7798ff9700 1 -- 172.21.15.179:0/953476646 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4053d50 con 0xe10b00 2024-05-08T22:38:07.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f7798ff9700 1 -- 172.21.15.179:0/953476646 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4055e50 con 0xe10b00 2024-05-08T22:38:07.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- 172.21.15.179:0/953476646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 msgr2=0xe10ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f77a4082d60 tx=0x7f77a4051e20 comp rx=0 tx=0).stop 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- 172.21.15.179:0/953476646 shutdown_connections 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdd89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdd8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 --2- 172.21.15.179:0/953476646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe10ee0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- 172.21.15.179:0/953476646 >> 172.21.15.179:0/953476646 conn(0xb584d0 msgr2=0xdf2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- 172.21.15.179:0/953476646 shutdown_connections 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.876+0000 7f77bca36740 1 -- 172.21.15.179:0/953476646 wait complete. 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- start start 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 Processor -- start 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe08a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xcee730 con 0xe10b00 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xcf45b0 con 0xe11420 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xbdcaa0 con 0xe07be0 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 10 client.0 _ll_register_callbacks cb 0xd2a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe08a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39332/0 (socket says 172.21.15.179:39332) 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 -- 172.21.15.179:0/3758561137 learned_addr learned my addr 172.21.15.179:0/3758561137 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 -- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 msgr2=0xdca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 -- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 msgr2=0xe08a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe08a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 -- 172.21.15.179:0/3758561137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b4160300 con 0xe10b00 2024-05-08T22:38:07.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b27fc700 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f77a80154d0 tx=0x7f77a8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd4b1e15683261e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8028070 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f77a801d070 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xb651e0 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a8015e80 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 10 client.7251 fetch_fsmap learned FSMap version 39 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0xcf7220 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a8032020 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f77a802b020 con 0xe10b00 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 10 client.7251 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 10 client.7251 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 20 client.7251 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xddbaf0 0xe241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 4 client.7251 mds_command: new command op to 56162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:38:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 --> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xbaf460 con 0xddbaf0 2024-05-08T22:38:07.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.880+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xddbaf0 0xe241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f7791ffb700 10 client.7251 ms_handle_connect on v2:172.21.15.195:6835/123759130 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xddbaf0 0xe241a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d763f3cbe8a94a65 server_cookie=e47d1ac090ddf9a7 in_seq=0 out_seq=0 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f7791ffb700 1 -- 172.21.15.179:0/3758561137 <== mds.0 v2:172.21.15.195:6835/123759130 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xbaf460 con 0xddbaf0 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f7791ffb700 10 client.7251 handle_command_reply: tid=0 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 client.7251 shutdown 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 msgr2=0xe08530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f77a80154d0 tx=0x7f77a8007d70 comp rx=0 tx=0).stop 2024-05-08T22:38:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 shutdown_connections 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xddbaf0 0xe241a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe08a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f7791ffb700 0 client.7251 ms_handle_reset on v2:172.21.15.195:6835/123759130 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe10b00 0xe08530 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 --2- 172.21.15.179:0/3758561137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe07be0 0xdca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 >> 172.21.15.179:0/3758561137 conn(0xb584d0 msgr2=0xdf1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 shutdown_connections 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 1 -- 172.21.15.179:0/3758561137 wait complete. 2024-05-08T22:38:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.884+0000 7f77bca36740 20 client.7251 trim_cache size 0 max 16384 2024-05-08T22:38:07.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 Processor -- start 2024-05-08T22:38:07.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 -- start start 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xdf1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdf1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xcfb4f0 con 0xe07be0 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xccf470 con 0xe11420 2024-05-08T22:38:07.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xd30610 con 0xe10b00 2024-05-08T22:38:07.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39334/0 (socket says 172.21.15.179:39334) 2024-05-08T22:38:07.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 -- 172.21.15.179:0/862010210 learned_addr learned my addr 172.21.15.179:0/862010210 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b37fe700 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdf1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b27fc700 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xdf1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 -- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 msgr2=0xdf1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xdf1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 -- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 msgr2=0xdf1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdf1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.904+0000 7f77b2ffd700 1 -- 172.21.15.179:0/862010210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe089d0 con 0xe07be0 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f77a4051eb0 tx=0x7f77a407e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aebd3cd758fcc5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77997fa700 1 -- 172.21.15.179:0/862010210 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4046070 con 0xe07be0 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77997fa700 1 -- 172.21.15.179:0/862010210 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4053af0 con 0xe07be0 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77997fa700 1 -- 172.21.15.179:0/862010210 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4054ea0 con 0xe07be0 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- 172.21.15.179:0/862010210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 msgr2=0xe265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f77a4051eb0 tx=0x7f77a407e000 comp rx=0 tx=0).stop 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- 172.21.15.179:0/862010210 shutdown_connections 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xdf1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xdf1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 --2- 172.21.15.179:0/862010210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe265e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- 172.21.15.179:0/862010210 >> 172.21.15.179:0/862010210 conn(0xb584d0 msgr2=0xdd9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- 172.21.15.179:0/862010210 shutdown_connections 2024-05-08T22:38:07.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- 172.21.15.179:0/862010210 wait complete. 2024-05-08T22:38:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 -- start start 2024-05-08T22:38:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.908+0000 7f77bca36740 1 Processor -- start 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xe71230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xdf2f30 con 0xe07be0 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xdcb8f0 con 0xe11420 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xe0fcf0 con 0xe10b00 2024-05-08T22:38:07.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 10 client.0 _ll_register_callbacks cb 0x7f77a4051d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39338/0 (socket says 172.21.15.179:39338) 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 -- 172.21.15.179:0/4045824289 learned_addr learned my addr 172.21.15.179:0/4045824289 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b37fe700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b27fc700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xe71230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 -- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 msgr2=0xe71230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xe71230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 -- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 msgr2=0xe258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 -- 172.21.15.179:0/4045824289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a4071040 con 0xe07be0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b27fc700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xe71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b2ffd700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f77a40556b0 tx=0x7f77a4057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598f2aad263a811e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:07.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4046050 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f77a4055e20 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77a4053d50 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a4054ea0 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 10 client.7257 fetch_fsmap learned FSMap version 39 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0xcdd450 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4053d50 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f77a405d910 con 0xe07be0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 10 client.7257 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 10 client.7257 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 20 client.7257 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xe722c0 0xe726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 4 client.7257 mds_command: new command op to 56162 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 --> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] -- command(tid 0: {"prefix": "status"}) v1 -- 0xbd41e0 con 0xe722c0 2024-05-08T22:38:07.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b27fc700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xe722c0 0xe726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f7791ffb700 10 client.7257 ms_handle_connect on v2:172.21.15.195:6835/123759130 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.912+0000 7f77b27fc700 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xe722c0 0xe726a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabe87e97aa39955 server_cookie=4c386d50e8416132 in_seq=0 out_seq=0 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f7791ffb700 1 -- 172.21.15.179:0/4045824289 <== mds.0 v2:172.21.15.195:6835/123759130 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0xbd41e0 con 0xe722c0 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f7791ffb700 10 client.7257 handle_command_reply: tid=0 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 client.7257 shutdown 2024-05-08T22:38:07.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 msgr2=0xe262e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f77a40556b0 tx=0x7f77a4057000 comp rx=0 tx=0).stop 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 shutdown_connections 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:6835/123759130,v1:172.21.15.195:6837/123759130] conn(0xe722c0 0xe726a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe11420 0xe258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f7791ffb700 0 client.7257 ms_handle_reset on v2:172.21.15.195:6835/123759130 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe10b00 0xe71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 --2- 172.21.15.179:0/4045824289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xe07be0 0xe262e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 >> 172.21.15.179:0/4045824289 conn(0xb584d0 msgr2=0xdf26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 shutdown_connections 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 1 -- 172.21.15.179:0/4045824289 wait complete. 2024-05-08T22:38:07.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77bca36740 20 client.7257 trim_cache size 0 max 16384 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 56162, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 10.103053514999999, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 39, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 83, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 83, 2024-05-08T22:38:07.923 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 14.560482463 2024-05-08T22:38:07.924 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:07.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 msgr2=0x7f77a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 0x7f77a0068bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77b4160780 tx=0x7f77a8009040 comp rx=0 tx=0).stop 2024-05-08T22:38:07.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 msgr2=0x7f77b41f7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f77a4011a20 tx=0x7f77a4011d30 comp rx=0 tx=0).stop 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 shutdown_connections 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f77a0066730 0x7f77a0068bf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77b415f940 0x7f77b415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77b40af910 0x7f77b40afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 --2- 172.21.15.179:0/101599751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b4151cd0 0x7f77b41f7f10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 >> 172.21.15.179:0/101599751 conn(0x7f77b40ce960 msgr2=0x7f77b4163870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 shutdown_connections 2024-05-08T22:38:07.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:07.916+0000 7f77b3fff700 1 -- 172.21.15.179:0/101599751 wait complete. 2024-05-08T22:38:07.938 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:38:08.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.052+0000 7f06ac8fa700 1 Processor -- start 2024-05-08T22:38:08.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.052+0000 7f06ac8fa700 1 -- start start 2024-05-08T22:38:08.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 0x7f06a80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06a80cb510 0x7f06a815e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80cb9a0 con 0x7f06a80cb510 2024-05-08T22:38:08.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a806abb0 con 0x7f06a80cc700 2024-05-08T22:38:08.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a806ae90 con 0x7f06a815b1b0 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06a80cb510 0x7f06a815e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06a80cb510 0x7f06a815e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39354/0 (socket says 172.21.15.179:39354) 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 -- 172.21.15.179:0/1026359274 learned_addr learned my addr 172.21.15.179:0/1026359274 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a77fe700 1 --2- 172.21.15.179:0/1026359274 >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 0x7f06a80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a6ffd700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 -- 172.21.15.179:0/1026359274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 msgr2=0x7f06a80cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a80cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 -- 172.21.15.179:0/1026359274 >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 msgr2=0x7f06a80ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 --2- 172.21.15.179:0/1026359274 >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 0x7f06a80ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 -- 172.21.15.179:0/1026359274 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a815b9c0 con 0x7f06a80cb510 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a77fe700 1 --2- 172.21.15.179:0/1026359274 >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 0x7f06a80ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:08.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a7fff700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a815e7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f06a8070800 tx=0x7f06a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518ec46ba1ca6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:08.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a67fc700 1 -- 172.21.15.179:0/1026359274 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000e070 con 0x7f06a80cb510 2024-05-08T22:38:08.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a67fc700 1 -- 172.21.15.179:0/1026359274 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000b910 con 0x7f06a80cb510 2024-05-08T22:38:08.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06a67fc700 1 -- 172.21.15.179:0/1026359274 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000bcc0 con 0x7f06a80cb510 2024-05-08T22:38:08.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- 172.21.15.179:0/1026359274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 msgr2=0x7f06a815e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a815e7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f06a8070800 tx=0x7f06a000b330 comp rx=0 tx=0).stop 2024-05-08T22:38:08.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- 172.21.15.179:0/1026359274 shutdown_connections 2024-05-08T22:38:08.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a815e7d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/1026359274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a80cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/1026359274 >> v2:172.21.15.195:3300/0 conn(0x7f06a80cc700 0x7f06a80ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.056+0000 7f06ac8fa700 1 -- 172.21.15.179:0/1026359274 >> 172.21.15.179:0/1026359274 conn(0x7f06a80ce9d0 msgr2=0x7f06a80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:08.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.060+0000 7f06ac8fa700 1 -- 172.21.15.179:0/1026359274 shutdown_connections 2024-05-08T22:38:08.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.060+0000 7f06ac8fa700 1 -- 172.21.15.179:0/1026359274 wait complete. 2024-05-08T22:38:08.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.060+0000 7f06ac8fa700 1 Processor -- start 2024-05-08T22:38:08.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.060+0000 7f06ac8fa700 1 -- start start 2024-05-08T22:38:08.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a80cc700 0x7f06a8140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a813df50 con 0x7f06a80cb510 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8156f10 con 0x7f06a80cc700 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8140b50 con 0x7f06a815b1b0 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39366/0 (socket says 172.21.15.179:39366) 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 -- 172.21.15.179:0/2877962845 learned_addr learned my addr 172.21.15.179:0/2877962845 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a6ffd700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a80cc700 0x7f06a8140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a7fff700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 -- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 msgr2=0x7f06a806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a806db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 -- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a80cc700 msgr2=0x7f06a8140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a80cc700 0x7f06a8140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 -- 172.21.15.179:0/2877962845 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980094b0 con 0x7f06a80cb510 2024-05-08T22:38:08.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06a77fe700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f069800ebe0 tx=0x7f069800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:08.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800c7f0 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069800c950 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0009b30 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800cce0 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a8156ba0 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.064+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a8156a00 con 0x7f06a80cb510 2024-05-08T22:38:08.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.068+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0698013470 con 0x7f06a80cb510 2024-05-08T22:38:08.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.068+0000 7f068ffff700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 0x7f0694068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.068+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f06980842a0 con 0x7f06a80cb510 2024-05-08T22:38:08.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.068+0000 7f06a6ffd700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 0x7f0694068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.068+0000 7f06a6ffd700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 0x7f0694068bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b0270 tx=0x7f069c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:08.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.076+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f06a8156a00 con 0x7f06a80cb510 2024-05-08T22:38:08.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.284+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06a8156a00 con 0x7f06a80cb510 2024-05-08T22:38:08.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.284+0000 7f068ffff700 1 -- 172.21.15.179:0/2877962845 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06a8156a00 con 0x7f06a80cb510 2024-05-08T22:38:08.289 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 msgr2=0x7f0694068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 0x7f0694068bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b0270 tx=0x7f069c009240 comp rx=0 tx=0).stop 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 msgr2=0x7f06a813e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f069800ebe0 tx=0x7f069800eef0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 shutdown_connections 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0694066730 0x7f0694068bf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06a80cc700 0x7f06a8140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06a815b1b0 0x7f06a806db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 --2- 172.21.15.179:0/2877962845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a80cb510 0x7f06a813e1e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 >> 172.21.15.179:0/2877962845 conn(0x7f06a80ce9d0 msgr2=0x7f06a81584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 shutdown_connections 2024-05-08T22:38:08.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:08.288+0000 7f06ac8fa700 1 -- 172.21.15.179:0/2877962845 wait complete. 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:38:08.304 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:08.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.305 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:38:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42283sec preferred_lft 42283sec 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:08.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.315 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:38:08.327 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:38:08.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.328 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:38:08.378 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:38:08.378 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:38:08.378 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:38:08.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.379 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:38:08.379 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:38:08.379 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:38:08.379 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:38:08.380 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:38:08.380 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:38:08.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.416 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:38:08.424 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:38:08.424 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:38:08.425 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:08.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.425 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:38:08.437 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:38:08.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:38:08.437 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:08.488 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:08.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:38:08.489 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:08.495 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:08.513 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:08.513 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:08.526 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:08.526 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:08.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.527 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:08.576 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:38:08.576 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:38:08.579 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:08.595 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:08.596 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:08.605 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:08.606 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:08.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:08.607 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:08.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:38:08.664+0000 7f3a654c0640 -1 init, newargv = 0x559ecc544e30 newargc=15 2024-05-08T22:38:08.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[42948]: starting ceph client 2024-05-08T22:38:08.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[42948]: starting fuse 2024-05-08T22:38:09.761 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:09.780 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:09.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:09.792 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:09.793 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:09.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:09.794 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:09.844 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:38:09.844 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:38:09.845 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:38:09.846 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:38:09.847 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:09.847 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:38:09.847 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:38:09.933 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.42948.asok 2024-05-08T22:38:10.117 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42948.asok 2024-05-08T22:38:10.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.117 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42948.asok status 2024-05-08T22:38:10.215 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:10.215 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "42948", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 7272, 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 7272 2024-05-08T22:38:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2513576936 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2513576936 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.7272 192.168.0.1:0/2513576936", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/2513576936", 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 39, 2024-05-08T22:38:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 83, 2024-05-08T22:38:10.218 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:38:10.218 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:38:10.218 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:38:10.218 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:10.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.225 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:38:10.225 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:10.231 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:38:10.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:10.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.232 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:10.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.248 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-08T22:38:10.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.268 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-08T22:38:10.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:10.281 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-08T22:38:10.333 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:38:10.399 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:38:10.591 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:38:10.591 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:10.599 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 Processor -- start 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 -- start start 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f307c0b25c0 0x7f307c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c15d3d0 0x7f307c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0ad4c0 con 0x7f307c0ad0e0 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c15ae80 con 0x7f307c0b25c0 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3081f8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1609c0 con 0x7f307c15d3d0 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f307c0b25c0 0x7f307c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3080f8c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f307c0ad0e0 0x7f307c0ac080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48676/0 (socket says 172.21.15.179:48676) 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f3080f8c700 1 -- 172.21.15.179:0/3152572257 learned_addr learned my addr 172.21.15.179:0/3152572257 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f308178d700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c15d3d0 0x7f307c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 -- 172.21.15.179:0/3152572257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c15d3d0 msgr2=0x7f307c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c15d3d0 0x7f307c0adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 -- 172.21.15.179:0/3152572257 >> v1:172.21.15.179:6789/0 conn(0x7f307c0ad0e0 legacy=0x7f307c0ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 -- 172.21.15.179:0/3152572257 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c15d900 con 0x7f307c0b25c0 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307bfff700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c0b25c0 0x7f307c0ad860 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3064009d90 tx=0x7f306400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16a9f4cd859366f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307b7fe700 1 -- 172.21.15.179:0/3152572257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306400e070 con 0x7f307c0b25c0 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307b7fe700 1 -- 172.21.15.179:0/3152572257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306400bbd0 con 0x7f307c0b25c0 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.760+0000 7f307b7fe700 1 -- 172.21.15.179:0/3152572257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064012430 con 0x7f307c0b25c0 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- 172.21.15.179:0/3152572257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c0b25c0 msgr2=0x7f307c0ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c0b25c0 0x7f307c0ad860 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3064009d90 tx=0x7f306400b330 comp rx=0 tx=0).stop 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- 172.21.15.179:0/3152572257 shutdown_connections 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c15d3d0 0x7f307c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- 172.21.15.179:0/3152572257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c0b25c0 0x7f307c0ad860 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- 172.21.15.179:0/3152572257 >> 172.21.15.179:0/3152572257 conn(0x7f307c0ce9d0 msgr2=0x7f307c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- 172.21.15.179:0/3152572257 shutdown_connections 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- 172.21.15.179:0/3152572257 wait complete. 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 Processor -- start 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- start start 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f307c0ad0e0 0x7f307c06add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c0b25c0 0x7f307c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1e6820 con 0x7f307c15d3d0 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0cc2f0 con 0x7f307c0ad0e0 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.764+0000 7f3081f8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0cc5d0 con 0x7f307c0b25c0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39394/0 (socket says 172.21.15.179:39394) 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 -- 172.21.15.179:0/1667445429 learned_addr learned my addr 172.21.15.179:0/1667445429 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f307bfff700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c0b25c0 0x7f307c156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f3080f8c700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f307c0ad0e0 0x7f307c06add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 -- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c0b25c0 msgr2=0x7f307c156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c0b25c0 0x7f307c156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 -- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f307c0ad0e0 msgr2=0x7f307c06add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f307c0ad0e0 0x7f307c06add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 -- 172.21.15.179:0/1667445429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009a70 con 0x7f307c15d3d0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f308178d700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f307400e810 tx=0x7f307400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400c070 con 0x7f307c15d3d0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074013650 con 0x7f307c15d3d0 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0cc8b0 con 0x7f307c15d3d0 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074013a00 con 0x7f307c15d3d0 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c06b630 con 0x7f307c15d3d0 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.768+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c0cd0b0 con 0x7f307c15d3d0 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.772+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3074013b60 con 0x7f307c15d3d0 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.772+0000 7f30797fa700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 0x7f305c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.772+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3074084070 con 0x7f307c15d3d0 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.772+0000 7f3080f8c700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 0x7f305c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.772+0000 7f3080f8c700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 0x7f305c068c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f60 tx=0x7f306c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.776+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f30740512a0 con 0x7f307c15d3d0 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.948+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f307c067740 con 0x7f307c15d3d0 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f30797fa700 1 -- 172.21.15.179:0/1667445429 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16357 (secure 0 0 0) 0x7f30740548c0 con 0x7f307c15d3d0 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:58.814700+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 msgr2=0x7f305c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 0x7f305c068c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f60 tx=0x7f306c007f30 comp rx=0 tx=0).stop 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 msgr2=0x7f307c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f307400e810 tx=0x7f307400eb20 comp rx=0 tx=0).stop 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 shutdown_connections 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f305c066760 0x7f305c068c20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f307c15d3d0 0x7f307c15bcd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f307c0b25c0 0x7f307c156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 --2- 172.21.15.179:0/1667445429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f307c0ad0e0 0x7f307c06add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 >> 172.21.15.179:0/1667445429 conn(0x7f307c0ce9d0 msgr2=0x7f307c0ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 shutdown_connections 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:10.952+0000 7f3081f8e700 1 -- 172.21.15.179:0/1667445429 wait complete. 2024-05-08T22:38:10.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:10.971 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 Processor -- start 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 -- start start 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47dc15d0c0 0x7f47dc0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc0ce2b0 0x7f47dc06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc15d4a0 con 0x7f47dc0d1d20 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc06abb0 con 0x7f47dc15d0c0 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e2537700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0d2500 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e0d34700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f47dc0d1d20 0x7f47dc0cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:51080/0 (socket says 172.21.15.179:51080) 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e1d36700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc0ce2b0 0x7f47dc06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e0d34700 1 -- 172.21.15.179:0/1449072532 learned_addr learned my addr 172.21.15.179:0/1449072532 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47e1535700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47dc15d0c0 0x7f47dc0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.092+0000 7f47dbfff700 1 -- 172.21.15.179:0/1449072532 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938125934 0 0) 0x7f47dc15d4a0 con 0x7f47dc0d1d20 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47dbfff700 1 -- 172.21.15.179:0/1449072532 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c40035d0 con 0x7f47dc0d1d20 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e1535700 1 -- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc0ce2b0 msgr2=0x7f47dc06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e1535700 1 --2- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc0ce2b0 0x7f47dc06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e1535700 1 -- 172.21.15.179:0/1449072532 >> v1:172.21.15.195:6789/0 conn(0x7f47dc0d1d20 legacy=0x7f47dc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e1535700 1 -- 172.21.15.179:0/1449072532 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0d27e0 con 0x7f47dc15d0c0 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e1535700 1 --2- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc15d0c0 0x7f47dc0b8a80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009850 tx=0x7f47cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2026fe3a3f5dcf0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47dbfff700 1 -- 172.21.15.179:0/1449072532 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc011830 con 0x7f47dc15d0c0 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47dbfff700 1 -- 172.21.15.179:0/1449072532 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc011990 con 0x7f47dc15d0c0 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47dbfff700 1 -- 172.21.15.179:0/1449072532 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc011d00 con 0x7f47dc15d0c0 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc15d0c0 msgr2=0x7f47dc0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 --2- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc15d0c0 0x7f47dc0b8a80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009850 tx=0x7f47cc009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- 172.21.15.179:0/1449072532 shutdown_connections 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 --2- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc0ce2b0 0x7f47dc06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 --2- 172.21.15.179:0/1449072532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc15d0c0 0x7f47dc0b8a80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- 172.21.15.179:0/1449072532 >> 172.21.15.179:0/1449072532 conn(0x7f47dc0d2ed0 msgr2=0x7f47dc0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- 172.21.15.179:0/1449072532 shutdown_connections 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- 172.21.15.179:0/1449072532 wait complete. 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 Processor -- start 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.096+0000 7f47e2537700 1 -- start start 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc0d1d20 0x7f47dc15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc15d0c0 0x7f47dc0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc15c250 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc15b5d0 con 0x7f47dc0d1d20 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc15b8b0 con 0x7f47dc15d0c0 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39410/0 (socket says 172.21.15.179:39410) 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e0d34700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc0d1d20 0x7f47dc15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 -- 172.21.15.179:0/4015320637 learned_addr learned my addr 172.21.15.179:0/4015320637 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1d36700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc15d0c0 0x7f47dc0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 -- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc15d0c0 msgr2=0x7f47dc0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc15d0c0 0x7f47dc0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 -- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc0d1d20 msgr2=0x7f47dc15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc0d1d20 0x7f47dc15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 -- 172.21.15.179:0/4015320637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0094b0 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e1535700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009ba0 tx=0x7f47cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004590 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc01e070 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc15bb90 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc003900 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.100+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc15b1f0 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc15c390 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47cc003a60 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47d9ffb700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 0x7f47c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47cc0850f0 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47e0d34700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 0x7f47c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.104+0000 7f47e0d34700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 0x7f47c8068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007fd0 tx=0x7f47d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.112+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47cc052310 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47dc0cd750 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47d9ffb700 1 -- 172.21.15.179:0/4015320637 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47cc055930 con 0x7f47dc0ce2b0 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 msgr2=0x7f47c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 0x7f47c8068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007fd0 tx=0x7f47d0009040 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 msgr2=0x7f47dc152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009ba0 tx=0x7f47cc004770 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 shutdown_connections 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c8066730 0x7f47c8068bf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47dc0d1d20 0x7f47dc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47dc15d0c0 0x7f47dc0b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 --2- 172.21.15.179:0/4015320637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47dc0ce2b0 0x7f47dc152920 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 >> 172.21.15.179:0/4015320637 conn(0x7f47dc0d2ed0 msgr2=0x7f47dc0c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 shutdown_connections 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.284+0000 7f47e2537700 1 -- 172.21.15.179:0/4015320637 wait complete. 2024-05-08T22:38:11.302 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 Processor -- start 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 -- start start 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41bc1614c0 0x7f41bc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0d1040 con 0x7f41bc0d2b10 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc06c330 con 0x7f41bc15d160 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41c1fab700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc06c610 con 0x7f41bc1614c0 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41b2ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41bc1614c0 0x7f41bc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.416+0000 7f41bbfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f41bc0d2b10 0x7f41bc06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48686/0 (socket says 172.21.15.179:48686) 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41bbfff700 1 -- 172.21.15.179:0/149025217 learned_addr learned my addr 172.21.15.179:0/149025217 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41b2ffd700 1 -- 172.21.15.179:0/149025217 >> v1:172.21.15.195:6789/0 conn(0x7f41bc15d160 legacy=0x7f41bc160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41b2ffd700 1 -- 172.21.15.179:0/149025217 >> v1:172.21.15.179:6789/0 conn(0x7f41bc0d2b10 legacy=0x7f41bc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41b2ffd700 1 -- 172.21.15.179:0/149025217 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0d1300 con 0x7f41bc1614c0 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41baffd700 1 -- 172.21.15.179:0/149025217 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998108537 0 0) 0x7f41bc0d1040 con 0x7f41bc0d2b10 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41b2ffd700 1 --2- 172.21.15.179:0/149025217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc1614c0 0x7f41bc0b8b20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009850 tx=0x7f41a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cacea89c601cf8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41baffd700 1 -- 172.21.15.179:0/149025217 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011830 con 0x7f41bc1614c0 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41baffd700 1 -- 172.21.15.179:0/149025217 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a4011990 con 0x7f41bc1614c0 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41baffd700 1 -- 172.21.15.179:0/149025217 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011d20 con 0x7f41bc1614c0 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 -- 172.21.15.179:0/149025217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc1614c0 msgr2=0x7f41bc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 --2- 172.21.15.179:0/149025217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc1614c0 0x7f41bc0b8b20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009850 tx=0x7f41a4009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 -- 172.21.15.179:0/149025217 shutdown_connections 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 --2- 172.21.15.179:0/149025217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc1614c0 0x7f41bc0b8b20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 -- 172.21.15.179:0/149025217 >> 172.21.15.179:0/149025217 conn(0x7f41bc0b4af0 msgr2=0x7f41bc0b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 -- 172.21.15.179:0/149025217 shutdown_connections 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.420+0000 7f41c1fab700 1 -- 172.21.15.179:0/149025217 wait complete. 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 Processor -- start 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 -- start start 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 0x7f41bc15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 0x7f41bc1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc160180 con 0x7f41bc0d2b10 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc160420 con 0x7f41bc15d160 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41c1fab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc153cf0 con 0x7f41bc1614c0 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39424/0 (socket says 172.21.15.179:39424) 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41bb7fe700 1 -- 172.21.15.179:0/1803142761 learned_addr learned my addr 172.21.15.179:0/1803142761 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41b2ffd700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 0x7f41bc15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.424+0000 7f41bbfff700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 0x7f41bc1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 -- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 msgr2=0x7f41bc1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 0x7f41bc1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 -- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 msgr2=0x7f41bc15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 0x7f41bc15fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 -- 172.21.15.179:0/1803142761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40094b0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41b2ffd700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 0x7f41bc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bbfff700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 0x7f41bc1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41bb7fe700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c500 tx=0x7f41ac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0052d0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac005430 con 0x7f41bc0d2b10 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc153fd0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0055a0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc160700 con 0x7f41bc0d2b10 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc073290 con 0x7f41bc0d2b10 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.428+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41ac0038e0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.432+0000 7f41b8ff9700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 0x7f419c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.432+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f41ac0835b0 con 0x7f41bc0d2b10 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.432+0000 7f41b2ffd700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 0x7f419c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.432+0000 7f41b2ffd700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 0x7f419c068bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f41a4014040 tx=0x7f41a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.440+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f41ac050870 con 0x7f41bc0d2b10 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.608+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41bc0b8730 con 0x7f41bc0d2b10 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.608+0000 7f41b8ff9700 1 -- 172.21.15.179:0/1803142761 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41ac053e90 con 0x7f41bc0d2b10 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 msgr2=0x7f419c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 0x7f419c068bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f41a4014040 tx=0x7f41a4008030 comp rx=0 tx=0).stop 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 msgr2=0x7f41bc150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c500 tx=0x7f41ac00c810 comp rx=0 tx=0).stop 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 shutdown_connections 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f419c066730 0x7f419c068bf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41bc15d160 0x7f41bc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41bc1614c0 0x7f41bc1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 --2- 172.21.15.179:0/1803142761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41bc0d2b10 0x7f41bc150ea0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 >> 172.21.15.179:0/1803142761 conn(0x7f41bc0b4af0 msgr2=0x7f41bc159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 shutdown_connections 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.612+0000 7f41c1fab700 1 -- 172.21.15.179:0/1803142761 wait complete. 2024-05-08T22:38:11.626 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.740+0000 7f94f2a8e700 1 Processor -- start 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.740+0000 7f94f2a8e700 1 -- start start 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 0x7f94ec06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec06cda0 con 0x7f94ec159320 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec0cb4c0 con 0x7f94ec159700 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b1f10 con 0x7f94ec0b19e0 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f1a8c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f94ec159320 0x7f94ec0cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48688/0 (socket says 172.21.15.179:48688) 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f1a8c700 1 -- 172.21.15.179:0/925421170 learned_addr learned my addr 172.21.15.179:0/925421170 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f228d700 1 --2- 172.21.15.179:0/925421170 >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 0x7f94ec06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 --2- 172.21.15.179:0/925421170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048696002 0 0) 0x7f94ec06cda0 con 0x7f94ec159320 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d80035d0 con 0x7f94ec159320 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858633119 0 0) 0x7f94d80035d0 con 0x7f94ec159320 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 -- 172.21.15.179:0/925421170 >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 msgr2=0x7f94ec06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 --2- 172.21.15.179:0/925421170 >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 0x7f94ec06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 -- 172.21.15.179:0/925421170 >> v1:172.21.15.179:6789/0 conn(0x7f94ec159320 legacy=0x7f94ec0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 -- 172.21.15.179:0/925421170 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0b21f0 con 0x7f94ec0b19e0 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f228d700 1 --2- 172.21.15.179:0/925421170 >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 0x7f94ec06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f128b700 1 --2- 172.21.15.179:0/925421170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec0cc630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009d90 tx=0x7f94e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa7e1ca0f145359 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800e070 con 0x7f94ec0b19e0 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8003a40 con 0x7f94ec0b19e0 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f0a8a700 1 -- 172.21.15.179:0/925421170 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0001670 con 0x7f94ec0b19e0 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- 172.21.15.179:0/925421170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 msgr2=0x7f94ec0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/925421170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec0cc630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009d90 tx=0x7f94e800b330 comp rx=0 tx=0).stop 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- 172.21.15.179:0/925421170 shutdown_connections 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/925421170 >> v2:172.21.15.195:3300/0 conn(0x7f94ec159700 0x7f94ec06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/925421170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec0cc630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- 172.21.15.179:0/925421170 >> 172.21.15.179:0/925421170 conn(0x7f94ec0cea30 msgr2=0x7f94ec0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- 172.21.15.179:0/925421170 shutdown_connections 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.744+0000 7f94f2a8e700 1 -- 172.21.15.179:0/925421170 wait complete. 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 Processor -- start 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- start start 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 0x7f94ec0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec150f20 con 0x7f94ec159320 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec150330 con 0x7f94ec159700 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec1505f0 con 0x7f94ec0b19e0 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39438/0 (socket says 172.21.15.179:39438) 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 -- 172.21.15.179:0/3844688697 learned_addr learned my addr 172.21.15.179:0/3844688697 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f1a8c700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f228d700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 0x7f94ec0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 -- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 msgr2=0x7f94ec149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 -- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 msgr2=0x7f94ec0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 0x7f94ec0c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 -- 172.21.15.179:0/3844688697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40094b0 con 0x7f94ec159320 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f228d700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 0x7f94ec0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f128b700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f80 tx=0x7f94e8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8004680 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e80047e0 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e8009a70 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8012670 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.748+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec150bb0 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4005310 con 0x7f94ec159320 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94e80127d0 con 0x7f94ec159320 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94de7fc700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 0x7f94d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f94e8085dc0 con 0x7f94ec159320 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94f1a8c700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 0x7f94d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.752+0000 7f94f1a8c700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 0x7f94d0068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f94e0001eb0 tx=0x7f94e0001e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.756+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f94e8052fe0 con 0x7f94ec159320 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.924+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94d4005b40 con 0x7f94ec159320 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.924+0000 7f94de7fc700 1 -- 172.21.15.179:0/3844688697 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94e8004950 con 0x7f94ec159320 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 msgr2=0x7f94d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 0x7f94d0068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f94e0001eb0 tx=0x7f94e0001e40 comp rx=0 tx=0).stop 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 msgr2=0x7f94ec144420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f80 tx=0x7f94e8004af0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 shutdown_connections 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f94d0066730 0x7f94d0068bf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ec159700 0x7f94ec0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ec159320 0x7f94ec144420 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 --2- 172.21.15.179:0/3844688697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94ec0b19e0 0x7f94ec149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 >> 172.21.15.179:0/3844688697 conn(0x7f94ec0cea30 msgr2=0x7f94ec0ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 shutdown_connections 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:11.928+0000 7f94f2a8e700 1 -- 172.21.15.179:0/3844688697 wait complete. 2024-05-08T22:38:11.944 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:38:12.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.052+0000 7fa641ce2700 1 Processor -- start 2024-05-08T22:38:12.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.052+0000 7fa641ce2700 1 -- start start 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.052+0000 7fa641ce2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63c15cf70 0x7fa63c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.052+0000 7fa641ce2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 0x7fa63c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.052+0000 7fa641ce2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c15d950 con 0x7fa63c159270 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c15dc30 con 0x7fa63c15cf70 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0b1af0 con 0x7fa63c15d420 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63c15cf70 0x7fa63c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa63c15cf70 0x7fa63c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39454/0 (socket says 172.21.15.179:39454) 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 -- 172.21.15.179:0/1208406106 learned_addr learned my addr 172.21.15.179:0/1208406106 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa6414e1700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 0x7fa63c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 -- 172.21.15.179:0/1208406106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 msgr2=0x7fa63c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 0x7fa63c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 -- 172.21.15.179:0/1208406106 >> v1:172.21.15.195:6789/0 conn(0x7fa63c159270 legacy=0x7fa63c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 -- 172.21.15.179:0/1208406106 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0b1dd0 con 0x7fa63c15cf70 2024-05-08T22:38:12.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa6414e1700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 0x7fa63c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:12.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63bfff700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15cf70 0x7fa63c06b440 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009e50 tx=0x7fa62c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1766fe0e7a8c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63b7fe700 1 -- 172.21.15.179:0/1208406106 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00e070 con 0x7fa63c15cf70 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63b7fe700 1 -- 172.21.15.179:0/1208406106 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c00b9c0 con 0x7fa63c15cf70 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa63b7fe700 1 -- 172.21.15.179:0/1208406106 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00bd70 con 0x7fa63c15cf70 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- 172.21.15.179:0/1208406106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15cf70 msgr2=0x7fa63c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15cf70 0x7fa63c06b440 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009e50 tx=0x7fa62c00b330 comp rx=0 tx=0).stop 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- 172.21.15.179:0/1208406106 shutdown_connections 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c15d420 0x7fa63c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1208406106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15cf70 0x7fa63c06b440 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- 172.21.15.179:0/1208406106 >> 172.21.15.179:0/1208406106 conn(0x7fa63c0ce960 msgr2=0x7fa63c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- 172.21.15.179:0/1208406106 shutdown_connections 2024-05-08T22:38:12.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.056+0000 7fa641ce2700 1 -- 172.21.15.179:0/1208406106 wait complete. 2024-05-08T22:38:12.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 Processor -- start 2024-05-08T22:38:12.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- start start 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 0x7fa63c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63c15cf70 0x7fa63c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c152140 con 0x7fa63c15d420 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1525d0 con 0x7fa63c15cf70 2024-05-08T22:38:12.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c151530 con 0x7fa63c159270 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39464/0 (socket says 172.21.15.179:39464) 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 -- 172.21.15.179:0/1717470476 learned_addr learned my addr 172.21.15.179:0/1717470476 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa63bfff700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63c15cf70 0x7fa63c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa640ce0700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 0x7fa63c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 -- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 msgr2=0x7fa63c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 0x7fa63c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 -- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63c15cf70 msgr2=0x7fa63c150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63c15cf70 0x7fa63c150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 -- 172.21.15.179:0/1717470476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c009b30 con 0x7fa63c15d420 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa640ce0700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 0x7fa63c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:12.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6414e1700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e810 tx=0x7fa63400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400c070 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634004020 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c151810 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340043f0 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.060+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c151dd0 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.064+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c1522b0 con 0x7fa63c15d420 2024-05-08T22:38:12.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.064+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6340036c0 con 0x7fa63c15d420 2024-05-08T22:38:12.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.064+0000 7fa6397fa700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 0x7fa61c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.064+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa634083b60 con 0x7fa63c15d420 2024-05-08T22:38:12.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.064+0000 7fa640ce0700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 0x7fa61c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.068+0000 7fa640ce0700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 0x7fa61c068c20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa624000f60 tx=0x7fa624009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.072+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa634050d90 con 0x7fa63c15d420 2024-05-08T22:38:12.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.256+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa63c0cb310 con 0x7fa63c15d420 2024-05-08T22:38:12.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.256+0000 7fa6397fa700 1 -- 172.21.15.179:0/1717470476 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa634017080 con 0x7fa63c15d420 2024-05-08T22:38:12.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 msgr2=0x7fa61c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 0x7fa61c068c20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa624000f60 tx=0x7fa624009040 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 msgr2=0x7fa63c14c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e810 tx=0x7fa63400eb20 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 shutdown_connections 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa61c066760 0x7fa61c068c20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63c15cf70 0x7fa63c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa63c15d420 0x7fa63c14c540 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 --2- 172.21.15.179:0/1717470476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63c159270 0x7fa63c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 >> 172.21.15.179:0/1717470476 conn(0x7fa63c0ce960 msgr2=0x7fa63c159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 shutdown_connections 2024-05-08T22:38:12.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.260+0000 7fa641ce2700 1 -- 172.21.15.179:0/1717470476 wait complete. 2024-05-08T22:38:12.264 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:38:12.274 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:38:12.274 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:12.282 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:12.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 Processor -- start 2024-05-08T22:38:12.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 -- start start 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3f41434d0 0x7ff3f406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f406dc20 0x7ff3f406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f40cd200 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f40cd4e0 con 0x7ff3f41434d0 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40cd7c0 con 0x7ff3f406dc20 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff3f40ce2a0 0x7ff3f40b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48700/0 (socket says 172.21.15.179:48700) 2024-05-08T22:38:12.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f37fe700 1 -- 172.21.15.179:0/2011583453 learned_addr learned my addr 172.21.15.179:0/2011583453 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:12.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/2011583453 >> v2:172.21.15.195:3300/0 conn(0x7ff3f41434d0 0x7ff3f406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f406dc20 0x7ff3f406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.392+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070615555 0 0) 0x7ff3f40cd200 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d40057e0 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3466122909 0 0) 0x7ff3d40057e0 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f406dc20 msgr2=0x7ff3f406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f406dc20 0x7ff3f406e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 >> v2:172.21.15.195:3300/0 conn(0x7ff3f41434d0 msgr2=0x7ff3f406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 --2- 172.21.15.179:0/2011583453 >> v2:172.21.15.195:3300/0 conn(0x7ff3f41434d0 0x7ff3f406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40cdaa0 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff3ec0019f0 con 0x7ff3f40ce2a0 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 >> v1:172.21.15.179:6789/0 conn(0x7ff3f40ce2a0 legacy=0x7ff3f40b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3d4005c30 0x7ff3d4006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3d4006f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3d4007440 0x7ff3d400dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40cd4e0 con 0x7ff3d4006a80 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40cd7c0 con 0x7ff3d4005c30 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4006850 con 0x7ff3d4007440 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/2011583453 >> v2:172.21.15.195:3300/0 conn(0x7ff3f41434d0 0x7ff3f406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3d4006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f37fe700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3d4007440 0x7ff3d400dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3d4005c30 0x7ff3d4006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 -- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3d4007440 msgr2=0x7ff3d400dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3d4007440 0x7ff3d400dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 -- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3d4005c30 msgr2=0x7ff3d4006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3d4005c30 0x7ff3d4006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 -- 172.21.15.179:0/2011583453 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40057e0 con 0x7ff3d4006a80 2024-05-08T22:38:12.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3d4005c30 0x7ff3d4006050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3d4006f00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b40 tx=0x7ff3d4007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b071519ffd893a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00e6c0 con 0x7ff3d4006a80 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc00e820 con 0x7ff3d4006a80 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f1ffb700 1 -- 172.21.15.179:0/2011583453 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00eb90 con 0x7ff3d4006a80 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 -- 172.21.15.179:0/2011583453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 msgr2=0x7ff3d4006f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/2011583453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3d4006f00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b40 tx=0x7ff3d4007ab0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f37fe700 1 -- 172.21.15.179:0/2011583453 reap_dead start 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 -- 172.21.15.179:0/2011583453 shutdown_connections 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 -- 172.21.15.179:0/2011583453 >> 172.21.15.179:0/2011583453 conn(0x7ff3f40ce9d0 msgr2=0x7ff3f40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 -- 172.21.15.179:0/2011583453 shutdown_connections 2024-05-08T22:38:12.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.396+0000 7ff3f3fff700 1 -- 172.21.15.179:0/2011583453 wait complete. 2024-05-08T22:38:12.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 Processor -- start 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- start start 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f40b2950 0x7ff3f40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 0x7ff3f40c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40b3310 con 0x7ff3d4006a80 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f414d0a0 con 0x7ff3f40c8770 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f414d380 con 0x7ff3f40b2950 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39482/0 (socket says 172.21.15.179:39482) 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 -- 172.21.15.179:0/1797497989 learned_addr learned my addr 172.21.15.179:0/1797497989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f37fe700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 0x7ff3f40c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f40b2950 0x7ff3f40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 -- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f40b2950 msgr2=0x7ff3f40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f40b2950 0x7ff3f40b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 -- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 msgr2=0x7ff3f40c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 0x7ff3f40c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 -- 172.21.15.179:0/1797497989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f413b010 con 0x7ff3d4006a80 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f37fe700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 0x7ff3f40c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f2ffd700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4010c90 tx=0x7ff3e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400eac0 con 0x7ff3d4006a80 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e400ec20 con 0x7ff3d4006a80 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f40c9410 con 0x7ff3d4006a80 2024-05-08T22:38:12.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4016460 con 0x7ff3d4006a80 2024-05-08T22:38:12.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.400+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f40c99d0 con 0x7ff3d4006a80 2024-05-08T22:38:12.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f4067740 con 0x7ff3d4006a80 2024-05-08T22:38:12.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3e40165c0 con 0x7ff3d4006a80 2024-05-08T22:38:12.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff3e4084860 con 0x7ff3d4006a80 2024-05-08T22:38:12.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.404+0000 7ff3f27fc700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0090b0 tx=0x7ff3dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.412+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff3e4051a90 con 0x7ff3d4006a80 2024-05-08T22:38:12.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.584+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3f40c9bf0 con 0x7ff3d4006a80 2024-05-08T22:38:12.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3ebfff700 1 -- 172.21.15.179:0/1797497989 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16357 (secure 0 0 0) 0x7ff3e40550b0 con 0x7ff3d4006a80 2024-05-08T22:38:12.590 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:12.591 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:37:58.814700+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 msgr2=0x7ff3d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0090b0 tx=0x7ff3dc008030 comp rx=0 tx=0).stop 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 msgr2=0x7ff3f413aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4010c90 tx=0x7ff3e400e4b0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 shutdown_connections 2024-05-08T22:38:12.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3f40c8770 0x7ff3f40c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3f40b2950 0x7ff3f40b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 --2- 172.21.15.179:0/1797497989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3d4006a80 0x7ff3f413aad0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 >> 172.21.15.179:0/1797497989 conn(0x7ff3f40ce9d0 msgr2=0x7ff3f415b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 shutdown_connections 2024-05-08T22:38:12.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.588+0000 7ff3f3fff700 1 -- 172.21.15.179:0/1797497989 wait complete. 2024-05-08T22:38:12.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:12.606 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:12.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 Processor -- start 2024-05-08T22:38:12.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 -- start start 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87b40ccd90 0x7f87b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40cd1f0 0x7f87b406f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40cc900 con 0x7f87b415b020 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b406f860 con 0x7f87b40ccd90 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87ba482700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b406e930 con 0x7f87b40cd1f0 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87b40ccd90 0x7f87b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87b40ccd90 0x7f87b406ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39490/0 (socket says 172.21.15.179:39490) 2024-05-08T22:38:12.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 -- 172.21.15.179:0/1044165884 learned_addr learned my addr 172.21.15.179:0/1044165884 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b9c81700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40cd1f0 0x7f87b406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 -- 172.21.15.179:0/1044165884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40cd1f0 msgr2=0x7f87b406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40cd1f0 0x7f87b406f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 -- 172.21.15.179:0/1044165884 >> v1:172.21.15.195:6789/0 conn(0x7f87b415b020 legacy=0x7f87b40b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87a7fff700 1 -- 172.21.15.179:0/1044165884 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967961684 0 0) 0x7f87b40cc900 con 0x7f87b415b020 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 -- 172.21.15.179:0/1044165884 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40b6ba0 con 0x7f87b40ccd90 2024-05-08T22:38:12.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87b8c7f700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40ccd90 0x7f87b406ede0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009940 tx=0x7f87b0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cb657d97bbec5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87a7fff700 1 -- 172.21.15.179:0/1044165884 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0011830 con 0x7f87b40ccd90 2024-05-08T22:38:12.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87a7fff700 1 -- 172.21.15.179:0/1044165884 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b0011990 con 0x7f87b40ccd90 2024-05-08T22:38:12.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.720+0000 7f87a7fff700 1 -- 172.21.15.179:0/1044165884 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0011d20 con 0x7f87b40ccd90 2024-05-08T22:38:12.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- 172.21.15.179:0/1044165884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40ccd90 msgr2=0x7f87b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40ccd90 0x7f87b406ede0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009940 tx=0x7f87b0009c50 comp rx=0 tx=0).stop 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- 172.21.15.179:0/1044165884 shutdown_connections 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40cd1f0 0x7f87b406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- 172.21.15.179:0/1044165884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40ccd90 0x7f87b406ede0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- 172.21.15.179:0/1044165884 >> 172.21.15.179:0/1044165884 conn(0x7f87b40d0ca0 msgr2=0x7f87b40d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- 172.21.15.179:0/1044165884 shutdown_connections 2024-05-08T22:38:12.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- 172.21.15.179:0/1044165884 wait complete. 2024-05-08T22:38:12.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 Processor -- start 2024-05-08T22:38:12.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- start start 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 0x7f87b41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b415b020 0x7f87b40ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c8280 con 0x7f87b40cd1f0 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4152720 con 0x7f87b415b020 2024-05-08T22:38:12.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.724+0000 7f87ba482700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40acc70 con 0x7f87b40ccd90 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39494/0 (socket says 172.21.15.179:39494) 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 -- 172.21.15.179:0/3112297312 learned_addr learned my addr 172.21.15.179:0/3112297312 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b9c81700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b415b020 0x7f87b40ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b9480700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 0x7f87b41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 -- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 msgr2=0x7f87b41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 0x7f87b41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 -- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b415b020 msgr2=0x7f87b40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b415b020 0x7f87b40ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 -- 172.21.15.179:0/3112297312 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009570 con 0x7f87b40cd1f0 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b9480700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 0x7f87b41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87b8c7f700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f87b0000c00 tx=0x7f87b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00044c0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b001f070 con 0x7f87b40cd1f0 2024-05-08T22:38:12.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00038e0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40ac6b0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40c7ea0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.728+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b40c83c0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.732+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f87b0021030 con 0x7f87b40cd1f0 2024-05-08T22:38:12.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.732+0000 7f87a5ffb700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:12.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.732+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f87b0085630 con 0x7f87b40cd1f0 2024-05-08T22:38:12.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.732+0000 7f87b9480700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:12.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.732+0000 7f87b9480700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000f60 tx=0x7f87a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:12.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.736+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f87b40c83c0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.904+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87b40c83c0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.904+0000 7f87a5ffb700 1 -- 172.21.15.179:0/3112297312 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b40c83c0 con 0x7f87b40cd1f0 2024-05-08T22:38:12.909 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:38:12.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 msgr2=0x7f8798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000f60 tx=0x7f87a8009040 comp rx=0 tx=0).stop 2024-05-08T22:38:12.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 msgr2=0x7f87b40abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f87b0000c00 tx=0x7f87b0004770 comp rx=0 tx=0).stop 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 shutdown_connections 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8798066730 0x7f8798068bf0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b415b020 0x7f87b40ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b40cd1f0 0x7f87b40abc30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 --2- 172.21.15.179:0/3112297312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87b40ccd90 0x7f87b41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 >> 172.21.15.179:0/3112297312 conn(0x7f87b40d0ca0 msgr2=0x7f87b406ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 shutdown_connections 2024-05-08T22:38:12.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:12.908+0000 7f87ba482700 1 -- 172.21.15.179:0/3112297312 wait complete. 2024-05-08T22:38:12.924 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:13.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 Processor -- start 2024-05-08T22:38:13.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 -- start start 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 0x7faa1c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faa1c0ce2b0 0x7faa1c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15d4a0 con 0x7faa1c0d1d20 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c06abb0 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa21762700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0d2500 con 0x7faa1c15d0c0 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.036+0000 7faa20f61700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faa1c0ce2b0 0x7faa1c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faa1c0ce2b0 0x7faa1c06b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39500/0 (socket says 172.21.15.179:39500) 2024-05-08T22:38:13.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 -- 172.21.15.179:0/4229600563 learned_addr learned my addr 172.21.15.179:0/4229600563 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa1bfff700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 0x7faa1c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 -- 172.21.15.179:0/4229600563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 msgr2=0x7faa1c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 0x7faa1c0b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 -- 172.21.15.179:0/4229600563 >> v1:172.21.15.195:6789/0 conn(0x7faa1c0d1d20 legacy=0x7faa1c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 -- 172.21.15.179:0/4229600563 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0d27e0 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa1bfff700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 0x7faa1c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa20f61700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c0ce2b0 0x7faa1c06b370 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf354fee2e0f84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa1affd700 1 -- 172.21.15.179:0/4229600563 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400e070 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa1affd700 1 -- 172.21.15.179:0/4229600563 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14003a40 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa1affd700 1 -- 172.21.15.179:0/4229600563 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14003df0 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- 172.21.15.179:0/4229600563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c0ce2b0 msgr2=0x7faa1c06b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c0ce2b0 0x7faa1c06b370 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400b330 comp rx=0 tx=0).stop 2024-05-08T22:38:13.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- 172.21.15.179:0/4229600563 shutdown_connections 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c0ce2b0 0x7faa1c06b370 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 --2- 172.21.15.179:0/4229600563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c15d0c0 0x7faa1c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- 172.21.15.179:0/4229600563 >> 172.21.15.179:0/4229600563 conn(0x7faa1c0d2ed0 msgr2=0x7faa1c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- 172.21.15.179:0/4229600563 shutdown_connections 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- 172.21.15.179:0/4229600563 wait complete. 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 Processor -- start 2024-05-08T22:38:13.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.040+0000 7faa21762700 1 -- start start 2024-05-08T22:38:13.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c0ce2b0 0x7faa1c1f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa1c0d1d20 0x7faa1c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15b730 con 0x7faa1c15d0c0 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c154aa0 con 0x7faa1c0d1d20 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c154770 con 0x7faa1c0ce2b0 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39508/0 (socket says 172.21.15.179:39508) 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 -- 172.21.15.179:0/1825898817 learned_addr learned my addr 172.21.15.179:0/1825898817 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa1b7fe700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa1c0d1d20 0x7faa1c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa1bfff700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c0ce2b0 0x7faa1c1f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 -- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c0ce2b0 msgr2=0x7faa1c1f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c0ce2b0 0x7faa1c1f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 -- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa1c0d1d20 msgr2=0x7faa1c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa1c0d1d20 0x7faa1c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 -- 172.21.15.179:0/1825898817 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009b30 con 0x7faa1c15d0c0 2024-05-08T22:38:13.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa20f61700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7faa1c08b290 tx=0x7faa14004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14004440 con 0x7faa1c15d0c0 2024-05-08T22:38:13.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa140045c0 con 0x7faa1c15d0c0 2024-05-08T22:38:13.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c15abf0 con 0x7faa1c15d0c0 2024-05-08T22:38:13.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14004970 con 0x7faa1c15d0c0 2024-05-08T22:38:13.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c15b170 con 0x7faa1c15d0c0 2024-05-08T22:38:13.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.044+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c0d2160 con 0x7faa1c15d0c0 2024-05-08T22:38:13.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.048+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa1402f480 con 0x7faa1c15d0c0 2024-05-08T22:38:13.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.048+0000 7faa18ff9700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 0x7faa08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.048+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faa14085c80 con 0x7faa1c15d0c0 2024-05-08T22:38:13.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.048+0000 7faa1bfff700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 0x7faa08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.052+0000 7faa1bfff700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 0x7faa08068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faa1000ab60 tx=0x7faa10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.052+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa14052f20 con 0x7faa1c15d0c0 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.220+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa1c0d2160 con 0x7faa1c15d0c0 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.220+0000 7faa18ff9700 1 -- 172.21.15.179:0/1825898817 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa1c0d2160 con 0x7faa1c15d0c0 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 msgr2=0x7faa08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 0x7faa08068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faa1000ab60 tx=0x7faa10008040 comp rx=0 tx=0).stop 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 msgr2=0x7faa1c151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7faa1c08b290 tx=0x7faa14004af0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 shutdown_connections 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faa08066730 0x7faa08068bf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa1c0d1d20 0x7faa1c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c15d0c0 0x7faa1c151cd0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 --2- 172.21.15.179:0/1825898817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa1c0ce2b0 0x7faa1c1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 >> 172.21.15.179:0/1825898817 conn(0x7faa1c0d2ed0 msgr2=0x7faa1c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 shutdown_connections 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.224+0000 7faa21762700 1 -- 172.21.15.179:0/1825898817 wait complete. 2024-05-08T22:38:13.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:13.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.352+0000 7f8fcf59e700 1 Processor -- start 2024-05-08T22:38:13.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.352+0000 7f8fcf59e700 1 -- start start 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcf59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8fd00b8a80 0x7f8fd00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcf59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd00b8ee0 con 0x7f8fd015d0c0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcf59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd015d4a0 con 0x7f8fd00ce0e0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8fd015d0c0 0x7f8fd00d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48708/0 (socket says 172.21.15.179:48708) 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 -- 172.21.15.179:0/1077437731 learned_addr learned my addr 172.21.15.179:0/1077437731 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd00ce570 con 0x7f8fd00b8a80 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/1077437731 >> v2:172.21.15.195:3300/0 conn(0x7f8fd00b8a80 0x7f8fd00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625567644 0 0) 0x7f8fd00b8ee0 con 0x7f8fd015d0c0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fbc0035d0 con 0x7f8fd015d0c0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975901192 0 0) 0x7f8fbc0035d0 con 0x7f8fd015d0c0 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 >> v2:172.21.15.195:3300/0 conn(0x7f8fd00b8a80 msgr2=0x7f8fd00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:13.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 --2- 172.21.15.179:0/1077437731 >> v2:172.21.15.195:3300/0 conn(0x7f8fd00b8a80 0x7f8fd00cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 >> v1:172.21.15.195:6789/0 conn(0x7f8fd00ce0e0 legacy=0x7f8fd006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd006abb0 con 0x7f8fd015d0c0 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f8fc0001670 con 0x7f8fd015d0c0 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 >> v1:172.21.15.179:6789/0 conn(0x7f8fd015d0c0 legacy=0x7f8fd00d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fbc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc004870 0x7f8fbc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc005230 0x7f8fbc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd00ce570 con 0x7f8fbc003a20 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc00c1d0 con 0x7f8fbc004870 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc00c390 con 0x7f8fbc005230 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fbc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc004870 0x7f8fbc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fced9d700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc005230 0x7f8fbc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 -- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc005230 msgr2=0x7f8fbc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fbc005230 0x7f8fbc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 -- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc004870 msgr2=0x7f8fbc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fbc004870 0x7f8fbc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 -- 172.21.15.179:0/1077437731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0035d0 con 0x7f8fbc003a20 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fced9d700 1 -- 172.21.15.179:0/1077437731 reap_dead start 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fce59c700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fbc003e40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c2f0 tx=0x7f8fbc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec73eb99ab21376 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0007c30 con 0x7f8fbc003a20 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000cdf0 con 0x7f8fbc003a20 2024-05-08T22:38:13.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.356+0000 7f8fcd59a700 1 -- 172.21.15.179:0/1077437731 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0005bc0 con 0x7f8fbc003a20 2024-05-08T22:38:13.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 msgr2=0x7f8fbc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fbc003e40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c2f0 tx=0x7f8fbc0058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 shutdown_connections 2024-05-08T22:38:13.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/1077437731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fbc003e40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 >> 172.21.15.179:0/1077437731 conn(0x7f8fd00d2ed0 msgr2=0x7f8fd00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 shutdown_connections 2024-05-08T22:38:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- 172.21.15.179:0/1077437731 wait complete. 2024-05-08T22:38:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 Processor -- start 2024-05-08T22:38:13.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- start start 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 0x7f8fd0144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd0144970 0x7f8fd00c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd015e040 con 0x7f8fbc003a20 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0144df0 con 0x7f8fd0144970 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcf59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd00cb360 con 0x7f8fd015dbe0 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39534/0 (socket says 172.21.15.179:39534) 2024-05-08T22:38:13.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 -- 172.21.15.179:0/3367396759 learned_addr learned my addr 172.21.15.179:0/3367396759 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 0x7f8fd0144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fced9d700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd0144970 0x7f8fd00c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 -- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 msgr2=0x7f8fd0144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 0x7f8fd0144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 -- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd0144970 msgr2=0x7f8fd00c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd0144970 0x7f8fd00c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fce59c700 1 -- 172.21.15.179:0/3367396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00cb640 con 0x7f8fbc003a20 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.360+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 0x7f8fd0144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fce59c700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c8e0 tx=0x7f8fd006ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0001640 con 0x7f8fbc003a20 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0000ed0 con 0x7f8fbc003a20 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd00cb920 con 0x7f8fbc003a20 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0007330 con 0x7f8fbc003a20 2024-05-08T22:38:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd00cbee0 con 0x7f8fbc003a20 2024-05-08T22:38:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd015dfc0 con 0x7f8fbc003a20 2024-05-08T22:38:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fc0007490 con 0x7f8fbc003a20 2024-05-08T22:38:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 0x7f8fb0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8fc0085390 con 0x7f8fbc003a20 2024-05-08T22:38:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.364+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 0x7f8fb0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.368+0000 7f8fcdd9b700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 0x7f8fb0068c20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00cd010 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.372+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8fc0052650 con 0x7f8fbc003a20 2024-05-08T22:38:13.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.540+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fd004e700 con 0x7f8fbc003a20 2024-05-08T22:38:13.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.540+0000 7f8fc6ffd700 1 -- 172.21.15.179:0/3367396759 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fc0055c70 con 0x7f8fbc003a20 2024-05-08T22:38:13.546 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 msgr2=0x7f8fb0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 0x7f8fb0068c20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00cd010 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 msgr2=0x7f8fd015d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c8e0 tx=0x7f8fd006ad40 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 shutdown_connections 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8fb0066760 0x7f8fb0068c20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fd0144970 0x7f8fd00c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fd015dbe0 0x7f8fd0144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 --2- 172.21.15.179:0/3367396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fbc003a20 0x7f8fd015d6a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 >> 172.21.15.179:0/3367396759 conn(0x7f8fd00d2ed0 msgr2=0x7f8fd00b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 shutdown_connections 2024-05-08T22:38:13.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.544+0000 7f8fcf59e700 1 -- 172.21.15.179:0/3367396759 wait complete. 2024-05-08T22:38:13.560 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:38:13.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 Processor -- start 2024-05-08T22:38:13.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- start start 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f04c40ca860 0x7f04c40cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c415be90 con 0x7f04c415bab0 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c40cee70 con 0x7f04c40cde20 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c40c4af0 con 0x7f04c40ca860 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c2ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f04c415bab0 0x7f04c40ca150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:48720/0 (socket says 172.21.15.179:48720) 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c2ffd700 1 -- 172.21.15.179:0/2574971386 learned_addr learned my addr 172.21.15.179:0/2574971386 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c27fc700 1 --2- 172.21.15.179:0/2574971386 >> v2:172.21.15.179:3300/0 conn(0x7f04c40ca860 0x7f04c40cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069901756 0 0) 0x7f04c415be90 con 0x7f04c415bab0 2024-05-08T22:38:13.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04ac0035d0 con 0x7f04c415bab0 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c27fc700 1 -- 172.21.15.179:0/2574971386 >> v1:172.21.15.195:6789/0 conn(0x7f04c40cde20 legacy=0x7f04c40c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c27fc700 1 -- 172.21.15.179:0/2574971386 >> v1:172.21.15.179:6789/0 conn(0x7f04c415bab0 legacy=0x7f04c40ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536869228 0 0) 0x7f04ac0035d0 con 0x7f04c415bab0 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c27fc700 1 -- 172.21.15.179:0/2574971386 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c40ce5d0 con 0x7f04c40ca860 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c27fc700 1 --2- 172.21.15.179:0/2574971386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40cd8e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009850 tx=0x7f04b8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b69416148f1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4001670 con 0x7f04c40ca860 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8004030 con 0x7f04c40ca860 2024-05-08T22:38:13.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c1ffb700 1 -- 172.21.15.179:0/2574971386 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80043c0 con 0x7f04c40ca860 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- 172.21.15.179:0/2574971386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 msgr2=0x7f04c40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2574971386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40cd8e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009850 tx=0x7f04b8009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 -- 172.21.15.179:0/2574971386 shutdown_connections 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.668+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2574971386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40cd8e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- 172.21.15.179:0/2574971386 >> 172.21.15.179:0/2574971386 conn(0x7f04c40cfb00 msgr2=0x7f04c40cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- 172.21.15.179:0/2574971386 shutdown_connections 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- 172.21.15.179:0/2574971386 wait complete. 2024-05-08T22:38:13.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 Processor -- start 2024-05-08T22:38:13.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- start start 2024-05-08T22:38:13.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40cde20 0x7f04c4152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415bab0 0x7f04c40c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41534b0 con 0x7f04c40ca860 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41637d0 con 0x7f04c40cde20 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40c7c80 con 0x7f04c415bab0 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40cde20 0x7f04c4152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39544/0 (socket says 172.21.15.179:39544) 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415bab0 0x7f04c40c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 -- 172.21.15.179:0/2685021640 learned_addr learned my addr 172.21.15.179:0/2685021640 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 -- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415bab0 msgr2=0x7f04c40c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415bab0 0x7f04c40c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 -- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40cde20 msgr2=0x7f04c4152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40cde20 0x7f04c4152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 -- 172.21.15.179:0/2685021640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80094b0 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c2ffd700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f04b40011e0 tx=0x7f04b400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004ca0 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4004e00 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c40c7f60 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.672+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40072f0 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c41633f0 con 0x7f04c40ca860 2024-05-08T22:38:13.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c404e700 con 0x7f04c40ca860 2024-05-08T22:38:13.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04b4007450 con 0x7f04c40ca860 2024-05-08T22:38:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04ab7fe700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 0x7f04b0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f04b4084040 con 0x7f04c40ca860 2024-05-08T22:38:13.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.676+0000 7f04c27fc700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 0x7f04b0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:13.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.680+0000 7f04c27fc700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 0x7f04b0068c20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f04b80090b0 tx=0x7f04b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:13.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.684+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f04b40511f0 con 0x7f04c40ca860 2024-05-08T22:38:13.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.868+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f04c40c6110 con 0x7f04c40ca860 2024-05-08T22:38:13.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.868+0000 7f04ab7fe700 1 -- 172.21.15.179:0/2685021640 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f04b4054810 con 0x7f04c40ca860 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 msgr2=0x7f04b0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 0x7f04b0068c20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f04b80090b0 tx=0x7f04b8008030 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 msgr2=0x7f04c40c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f04b40011e0 tx=0x7f04b400ca50 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 shutdown_connections 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04b0066760 0x7f04b0068c20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40cde20 0x7f04c4152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415bab0 0x7f04c40c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 --2- 172.21.15.179:0/2685021640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04c40ca860 0x7f04c40c8760 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 >> 172.21.15.179:0/2685021640 conn(0x7f04c40cfb00 msgr2=0x7f04c40ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 shutdown_connections 2024-05-08T22:38:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:13.872+0000 7f04c3fff700 1 -- 172.21.15.179:0/2685021640 wait complete. 2024-05-08T22:38:13.876 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:38:13.886 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:38:13.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:13.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:13.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[42948]: fuse finished with error 0 and tester_r 0 2024-05-08T22:38:13.902 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:38:20.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:38:20.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:20.006 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:38:20.014 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:20.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 Processor -- start 2024-05-08T22:38:20.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 -- start start 2024-05-08T22:38:20.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f35d8158b80 0x7f35d815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d8158f60 con 0x7f35d806c230 2024-05-08T22:38:20.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d806b860 con 0x7f35d8158b80 2024-05-08T22:38:20.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35de22b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806ac50 con 0x7f35d80c8d70 2024-05-08T22:38:20.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f35d8158b80 0x7f35d815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f35d8158b80 0x7f35d815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39202/0 (socket says 172.21.15.179:39202) 2024-05-08T22:38:20.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d7fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f35d806c230 0x7f35d80c9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46364/0 (socket says 172.21.15.179:46364) 2024-05-08T22:38:20.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 -- 172.21.15.179:0/751570184 learned_addr learned my addr 172.21.15.179:0/751570184 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d6ffd700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 -- 172.21.15.179:0/751570184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 msgr2=0x7f35d806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 -- 172.21.15.179:0/751570184 >> v1:172.21.15.195:6789/0 conn(0x7f35d806c230 legacy=0x7f35d80c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 -- 172.21.15.179:0/751570184 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d806af30 con 0x7f35d8158b80 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d77fe700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d8158b80 0x7f35d815b4d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009910 tx=0x7f35cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0e3fe8bcd635f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d67fc700 1 -- 172.21.15.179:0/751570184 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc011830 con 0x7f35d8158b80 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d67fc700 1 -- 172.21.15.179:0/751570184 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc011990 con 0x7f35d8158b80 2024-05-08T22:38:20.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.136+0000 7f35d67fc700 1 -- 172.21.15.179:0/751570184 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc011d40 con 0x7f35d8158b80 2024-05-08T22:38:20.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 -- 172.21.15.179:0/751570184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d8158b80 msgr2=0x7f35d815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d8158b80 0x7f35d815b4d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009910 tx=0x7f35cc009c20 comp rx=0 tx=0).stop 2024-05-08T22:38:20.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 -- 172.21.15.179:0/751570184 shutdown_connections 2024-05-08T22:38:20.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 --2- 172.21.15.179:0/751570184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d8158b80 0x7f35d815b4d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 -- 172.21.15.179:0/751570184 >> 172.21.15.179:0/751570184 conn(0x7f35d80cb660 msgr2=0x7f35d80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 -- 172.21.15.179:0/751570184 shutdown_connections 2024-05-08T22:38:20.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.140+0000 7f35de22b700 1 -- 172.21.15.179:0/751570184 wait complete. 2024-05-08T22:38:20.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 Processor -- start 2024-05-08T22:38:20.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 -- start start 2024-05-08T22:38:20.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35d8158b80 0x7f35d8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c6d00 con 0x7f35d806c230 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c7ff0 con 0x7f35d8158b80 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c6f50 con 0x7f35d80c8d70 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39204/0 (socket says 172.21.15.179:39204) 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 -- 172.21.15.179:0/596775164 learned_addr learned my addr 172.21.15.179:0/596775164 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d7fff700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35d8158b80 0x7f35d8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d6ffd700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 -- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 msgr2=0x7f35d8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 -- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35d8158b80 msgr2=0x7f35d8150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35d8158b80 0x7f35d8150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 -- 172.21.15.179:0/596775164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009570 con 0x7f35d806c230 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35d77fe700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc004510 con 0x7f35d806c230 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc0036c0 con 0x7f35d806c230 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d80c7230 con 0x7f35d806c230 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.144+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc003a70 con 0x7f35d806c230 2024-05-08T22:38:20.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d80c77f0 con 0x7f35d806c230 2024-05-08T22:38:20.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8067770 con 0x7f35d806c230 2024-05-08T22:38:20.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35cc003bd0 con 0x7f35d806c230 2024-05-08T22:38:20.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35bffff700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 0x7f35c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f35cc021030 con 0x7f35d806c230 2024-05-08T22:38:20.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.148+0000 7f35d6ffd700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 0x7f35c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.152+0000 7f35d6ffd700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 0x7f35c4068bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f35d815ded0 tx=0x7f35c8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.156+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f35cc052ac0 con 0x7f35d806c230 2024-05-08T22:38:20.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.356+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35d806f000 con 0x7f35d806c230 2024-05-08T22:38:20.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35bffff700 1 -- 172.21.15.179:0/596775164 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4749 (secure 0 0 0) 0x7f35cc015090 con 0x7f35d806c230 2024-05-08T22:38:20.362 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:20.362 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:00.821878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56162":{"gid":56162,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/123759130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":123759130},{"type":"v1","addr":"172.21.15.195:6837","nonce":123759130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 msgr2=0x7f35c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 0x7f35c4068bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f35d815ded0 tx=0x7f35c8009240 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 msgr2=0x7f35d80be850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc004770 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 shutdown_connections 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f35c4066730 0x7f35c4068bf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35d8158b80 0x7f35d8150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35d80c8d70 0x7f35d8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 --2- 172.21.15.179:0/596775164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35d806c230 0x7f35d80be850 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 >> 172.21.15.179:0/596775164 conn(0x7f35d80cb660 msgr2=0x7f35d815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 shutdown_connections 2024-05-08T22:38:20.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.360+0000 7f35de22b700 1 -- 172.21.15.179:0/596775164 wait complete. 2024-05-08T22:38:20.366 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 39 2024-05-08T22:38:20.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T22:38:20.520 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:38:20.520 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:38:20.521 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:38:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.640+0000 7fadaffb9700 1 Processor -- start 2024-05-08T22:38:20.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.640+0000 7fadaffb9700 1 -- start start 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fada815d3b0 0x7fada80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada815d790 con 0x7fada80af300 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada80cef20 con 0x7fada815d3b0 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada80cf200 con 0x7fada80b3200 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fada815d3b0 0x7fada80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fada815d3b0 0x7fada80ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39210/0 (socket says 172.21.15.179:39210) 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 -- 172.21.15.179:0/2946044012 learned_addr learned my addr 172.21.15.179:0/2946044012 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadae556700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 -- 172.21.15.179:0/2946044012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 msgr2=0x7fada80cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:20.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 -- 172.21.15.179:0/2946044012 >> v1:172.21.15.195:6789/0 conn(0x7fada80af300 legacy=0x7fada80b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadacd53700 1 -- 172.21.15.179:0/2946044012 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016679572 0 0) 0x7fada4001490 con 0x7fada80af300 2024-05-08T22:38:20.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 -- 172.21.15.179:0/2946044012 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80cf4e0 con 0x7fada815d3b0 2024-05-08T22:38:20.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadadd55700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada80ce580 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fad98009940 tx=0x7fad98009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94745bde74a7cda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadacd53700 1 -- 172.21.15.179:0/2946044012 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98011830 con 0x7fada815d3b0 2024-05-08T22:38:20.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadacd53700 1 -- 172.21.15.179:0/2946044012 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad98011990 con 0x7fada815d3b0 2024-05-08T22:38:20.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadacd53700 1 -- 172.21.15.179:0/2946044012 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98011d20 con 0x7fada815d3b0 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- 172.21.15.179:0/2946044012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 msgr2=0x7fada80ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada80ce580 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fad98009940 tx=0x7fad98009c50 comp rx=0 tx=0).stop 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- 172.21.15.179:0/2946044012 shutdown_connections 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 --2- 172.21.15.179:0/2946044012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada80ce580 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- 172.21.15.179:0/2946044012 >> 172.21.15.179:0/2946044012 conn(0x7fada80cc720 msgr2=0x7fada80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- 172.21.15.179:0/2946044012 shutdown_connections 2024-05-08T22:38:20.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.644+0000 7fadaffb9700 1 -- 172.21.15.179:0/2946044012 wait complete. 2024-05-08T22:38:20.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.648+0000 7fadaffb9700 1 Processor -- start 2024-05-08T22:38:20.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.648+0000 7fadaffb9700 1 -- start start 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada80af300 0x7fada8151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815a950 con 0x7fada815d3b0 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada8151ac0 con 0x7fada80af300 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada806c500 con 0x7fada80b3200 2024-05-08T22:38:20.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39212/0 (socket says 172.21.15.179:39212) 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 -- 172.21.15.179:0/1762535332 learned_addr learned my addr 172.21.15.179:0/1762535332 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadad554700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadadd55700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada80af300 0x7fada8151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 -- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 msgr2=0x7fada80ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 -- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada80af300 msgr2=0x7fada8151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada80af300 0x7fada8151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 -- 172.21.15.179:0/1762535332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98009570 con 0x7fada815d3b0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadad554700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadae556700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fada0009fd0 tx=0x7fada000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00096e0 con 0x7fada815d3b0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000eed0 con 0x7fada815d3b0 2024-05-08T22:38:20.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada80aecc0 con 0x7fada815d3b0 2024-05-08T22:38:20.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000ca70 con 0x7fada815d3b0 2024-05-08T22:38:20.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada806c7a0 con 0x7fada815d3b0 2024-05-08T22:38:20.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.652+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada8073290 con 0x7fada815d3b0 2024-05-08T22:38:20.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.656+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fada000cc90 con 0x7fada815d3b0 2024-05-08T22:38:20.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.656+0000 7fad9e7fc700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 0x7fad8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.656+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fada0084340 con 0x7fada815d3b0 2024-05-08T22:38:20.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.656+0000 7fadadd55700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 0x7fad8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.656+0000 7fadadd55700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 0x7fad8c068bf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad98009fd0 tx=0x7fad98008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.660+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fada0051570 con 0x7fada815d3b0 2024-05-08T22:38:20.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.872+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fada815dcb0 con 0x7fada815d3b0 2024-05-08T22:38:20.876 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:38:20.872+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:38:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.888+0000 7fad9e7fc700 1 -- 172.21.15.179:0/1762535332 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7fada0054b90 con 0x7fada815d3b0 2024-05-08T22:38:20.894 INFO:tasks.ceph.mds.d.smithi195.stderr: -3612> 2024-05-08T22:37:58.732+0000 7fb366198700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 msgr2=0x7fad8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 0x7fad8c068bf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad98009fd0 tx=0x7fad98008030 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 msgr2=0x7fada815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fada0009fd0 tx=0x7fada000ec90 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 shutdown_connections 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fad8c066730 0x7fad8c068bf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada815d3b0 0x7fada815b600 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fada80b3200 0x7fada80ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 --2- 172.21.15.179:0/1762535332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fada80af300 0x7fada8151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 >> 172.21.15.179:0/1762535332 conn(0x7fada80cc720 msgr2=0x7fada80b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 shutdown_connections 2024-05-08T22:38:20.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.892+0000 7fadaffb9700 1 -- 172.21.15.179:0/1762535332 wait complete. 2024-05-08T22:38:20.898 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:38:20.912 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-08T22:38:20.927 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:38:20.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 Processor -- start 2024-05-08T22:38:20.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- start start 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 0x563c272936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2715d8f0 0x563c27293c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 0x563c271fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c270c1a00 con 0x563c2712a220 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c270bd2f0 con 0x563c27201740 2024-05-08T22:38:20.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c270798e0 con 0x563c2715d8f0 2024-05-08T22:38:20.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403329700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 0x563c272936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 0x563c271fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0402b28700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2715d8f0 0x563c27293c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403329700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 0x563c272936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39222/0 (socket says 172.21.15.179:39222) 2024-05-08T22:38:20.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403329700 1 -- 172.21.15.179:0/3702260266 learned_addr learned my addr 172.21.15.179:0/3702260266 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 -- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2715d8f0 msgr2=0x563c27293c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:20.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2715d8f0 0x563c27293c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 -- 172.21.15.179:0/3702260266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 msgr2=0x563c272936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 0x563c272936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 -- 172.21.15.179:0/3702260266 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec005d20 con 0x563c27201740 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0403b2a700 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 0x563c271fc720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006fa0 tx=0x7f03f4001910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f76d1acc5ccf90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0401b26700 1 -- 172.21.15.179:0/3702260266 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4010560 con 0x563c27201740 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0401b26700 1 -- 172.21.15.179:0/3702260266 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f40106e0 con 0x563c27201740 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f0401b26700 1 -- 172.21.15.179:0/3702260266 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4010a90 con 0x563c27201740 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 msgr2=0x563c271fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 0x563c271fc720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006fa0 tx=0x7f03f4001910 comp rx=0 tx=0).stop 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- 172.21.15.179:0/3702260266 shutdown_connections 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c27201740 0x563c271fc720 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2715d8f0 0x563c27293c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 --2- 172.21.15.179:0/3702260266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c2712a220 0x563c272936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- 172.21.15.179:0/3702260266 >> 172.21.15.179:0/3702260266 conn(0x563c27130780 msgr2=0x563c271fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- 172.21.15.179:0/3702260266 shutdown_connections 2024-05-08T22:38:20.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- 172.21.15.179:0/3702260266 wait complete. 2024-05-08T22:38:20.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 Processor -- start 2024-05-08T22:38:20.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.952+0000 7f04057f4140 1 -- start start 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2712a220 0x563c2716d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c2715d8f0 0x563c271f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 0x563c271f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c27079580 con 0x563c27201740 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2704f8d0 con 0x563c2715d8f0 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c271faa60 con 0x563c2712a220 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 0x563c271f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 0x563c271f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39234/0 (socket says 172.21.15.179:39234) 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 -- 172.21.15.179:0/206065759 learned_addr learned my addr 172.21.15.179:0/206065759 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403329700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2712a220 0x563c2716d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0402b28700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c2715d8f0 0x563c271f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 -- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2712a220 msgr2=0x563c2716d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563c2712a220 0x563c2716d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 -- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c2715d8f0 msgr2=0x563c271f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c2715d8f0 0x563c271f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 -- 172.21.15.179:0/206065759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4006ba0 con 0x563c27201740 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0403b2a700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 0x563c271f8430 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f03f40105c0 tx=0x7f03f4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0400b24700 1 -- 172.21.15.179:0/206065759 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4011810 con 0x563c27201740 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0400b24700 1 -- 172.21.15.179:0/206065759 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4011990 con 0x563c27201740 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f0400b24700 1 -- 172.21.15.179:0/206065759 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4011d40 con 0x563c27201740 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563c2705af40 con 0x563c27201740 2024-05-08T22:38:20.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.956+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c2705a9a0 con 0x563c27201740 2024-05-08T22:38:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0400b24700 1 -- 172.21.15.179:0/206065759 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03f4019070 con 0x563c27201740 2024-05-08T22:38:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0400b24700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03e8065060 0x7f03e8067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0400b24700 1 -- 172.21.15.179:0/206065759 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f03f4084960 con 0x563c27201740 2024-05-08T22:38:20.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f04057f4140 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563c27294190 0x563c272945a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x563c271fdfa0 con 0x563c27294190 2024-05-08T22:38:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0403329700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03e8065060 0x7f03e8067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0402b28700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563c27294190 0x563c272945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0402b28700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563c27294190 0x563c272945a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.960+0000 7f0403329700 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03e8065060 0x7f03e8067520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0011e0 tx=0x7f03fc001730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f0402b28700 1 -- 172.21.15.179:0/206065759 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v84'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f03ec008a70 con 0x563c27294190 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563c27294190 msgr2=0x563c272945a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563c27294190 0x563c272945a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03e8065060 msgr2=0x7f03e8067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03e8065060 0x7f03e8067520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0011e0 tx=0x7f03fc001730 comp rx=0 tx=0).stop 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 msgr2=0x563c271f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:20.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f04057f4140 1 --2- 172.21.15.179:0/206065759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c27201740 0x563c271f8430 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f03f40105c0 tx=0x7f03f4001250 comp rx=0 tx=0).stop 2024-05-08T22:38:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.964+0000 7f0403b2a700 1 -- 172.21.15.179:0/206065759 reap_dead start 2024-05-08T22:38:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.968+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 shutdown_connections 2024-05-08T22:38:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.968+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 >> 172.21.15.179:0/206065759 conn(0x563c27130780 msgr2=0x563c2715c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.968+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 shutdown_connections 2024-05-08T22:38:20.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:20.968+0000 7f04057f4140 1 -- 172.21.15.179:0/206065759 wait complete. 2024-05-08T22:38:20.973 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:38:21.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.088+0000 7f22088b7700 1 Processor -- start 2024-05-08T22:38:21.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.088+0000 7f22088b7700 1 -- start start 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2204142890 0x7f22040b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 0x7f220406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040b22d0 con 0x7f2204142fd0 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040b19e0 con 0x7f2204142890 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220415e640 con 0x7f22040b2ca0 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 0x7f220406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2203fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2204142fd0 0x7f220415d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46394/0 (socket says 172.21.15.179:46394) 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2203fff700 1 -- 172.21.15.179:0/3607660822 learned_addr learned my addr 172.21.15.179:0/3607660822 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22037fe700 1 --2- 172.21.15.179:0/3607660822 >> v2:172.21.15.195:3300/0 conn(0x7f2204142890 0x7f22040b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22027fc700 1 -- 172.21.15.179:0/3607660822 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079908981 0 0) 0x7f22040b22d0 con 0x7f2204142fd0 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 -- 172.21.15.179:0/3607660822 >> v2:172.21.15.195:3300/0 conn(0x7f2204142890 msgr2=0x7f22040b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 --2- 172.21.15.179:0/3607660822 >> v2:172.21.15.195:3300/0 conn(0x7f2204142890 0x7f22040b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 -- 172.21.15.179:0/3607660822 >> v1:172.21.15.195:6789/0 conn(0x7f2204142fd0 legacy=0x7f220415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 -- 172.21.15.179:0/3607660822 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040b1c80 con 0x7f22040b2ca0 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f2202ffd700 1 --2- 172.21.15.179:0/3607660822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 0x7f220406cda0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009850 tx=0x7f21f4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29878621dad0d9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22027fc700 1 -- 172.21.15.179:0/3607660822 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011830 con 0x7f22040b2ca0 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22027fc700 1 -- 172.21.15.179:0/3607660822 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4011990 con 0x7f22040b2ca0 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22027fc700 1 -- 172.21.15.179:0/3607660822 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011d40 con 0x7f22040b2ca0 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- 172.21.15.179:0/3607660822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 msgr2=0x7f220406cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 --2- 172.21.15.179:0/3607660822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 0x7f220406cda0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009850 tx=0x7f21f4009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- 172.21.15.179:0/3607660822 shutdown_connections 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 --2- 172.21.15.179:0/3607660822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22040b2ca0 0x7f220406cda0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 --2- 172.21.15.179:0/3607660822 >> v2:172.21.15.195:3300/0 conn(0x7f2204142890 0x7f22040b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- 172.21.15.179:0/3607660822 >> 172.21.15.179:0/3607660822 conn(0x7f22040cea30 msgr2=0x7f22040cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:21.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- 172.21.15.179:0/3607660822 shutdown_connections 2024-05-08T22:38:21.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- 172.21.15.179:0/3607660822 wait complete. 2024-05-08T22:38:21.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 Processor -- start 2024-05-08T22:38:21.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.092+0000 7f22088b7700 1 -- start start 2024-05-08T22:38:21.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2204142890 0x7f22041ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2204142fd0 0x7f2204136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204140f40 con 0x7f22040b2ca0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204141220 con 0x7f2204142890 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2204141500 con 0x7f2204142fd0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39252/0 (socket says 172.21.15.179:39252) 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 -- 172.21.15.179:0/3769450414 learned_addr learned my addr 172.21.15.179:0/3769450414 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f2203fff700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2204142fd0 0x7f2204136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f2202ffd700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2204142890 0x7f22041ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 -- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2204142fd0 msgr2=0x7f2204136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2204142fd0 0x7f2204136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 -- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2204142890 msgr2=0x7f22041ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2204142890 0x7f22041ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 -- 172.21.15.179:0/3769450414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40094b0 con 0x7f22040b2ca0 2024-05-08T22:38:21.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22037fe700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9d0 tx=0x7f21ec00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec009720 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22041417e0 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec00c850 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cbc0 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.096+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22041ee720 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204141970 con 0x7f22040b2ca0 2024-05-08T22:38:21.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21ec00cd20 con 0x7f22040b2ca0 2024-05-08T22:38:21.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f21fbfff700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 0x7f21e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:21.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f21ec084430 con 0x7f22040b2ca0 2024-05-08T22:38:21.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f2202ffd700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 0x7f21e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:21.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.100+0000 7f2202ffd700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 0x7f21e4068c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f220406b410 tx=0x7f21f4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:21.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.104+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f21ec051580 con 0x7f22040b2ca0 2024-05-08T22:38:21.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.308+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f220404e700 con 0x7f22040b2ca0 2024-05-08T22:38:21.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.892+0000 7f21fbfff700 1 -- 172.21.15.179:0/3769450414 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v41) v1 ==== 162+0+0 (secure 0 0 0) 0x7f21ec054ba0 con 0x7f22040b2ca0 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 msgr2=0x7f21e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 0x7f21e4068c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f220406b410 tx=0x7f21f4008030 comp rx=0 tx=0).stop 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 msgr2=0x7f220413e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9d0 tx=0x7f21ec00ece0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 shutdown_connections 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f21e4066760 0x7f21e4068c20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2204142890 0x7f22041ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2204142fd0 0x7f2204136c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 --2- 172.21.15.179:0/3769450414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22040b2ca0 0x7f220413e3e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:21.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 >> 172.21.15.179:0/3769450414 conn(0x7f22040cea30 msgr2=0x7f220406d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:21.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 shutdown_connections 2024-05-08T22:38:21.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:21.896+0000 7f22088b7700 1 -- 172.21.15.179:0/3769450414 wait complete. 2024-05-08T22:38:21.902 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:38:21.918 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:22.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 Processor -- start 2024-05-08T22:38:22.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 -- start start 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f86fc0ce2d0 0x7f86fc06bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc06afc0 con 0x7f86fc15d5b0 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc15d990 con 0x7f86fc0cde70 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f87009f3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86fc0d26f0 con 0x7f86fc0ce2d0 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fb7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f86fc15d5b0 0x7f86fc0b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39220/0 (socket says 172.21.15.179:39220) 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fb7fe700 1 -- 172.21.15.179:0/1132862478 learned_addr learned my addr 172.21.15.179:0/1132862478 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 --2- 172.21.15.179:0/1132862478 >> v2:172.21.15.195:3300/0 conn(0x7f86fc0ce2d0 0x7f86fc06bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306935130 0 0) 0x7f86fc06afc0 con 0x7f86fc15d5b0 2024-05-08T22:38:22.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e80035d0 con 0x7f86fc15d5b0 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202207975 0 0) 0x7f86fc15d990 con 0x7f86fc0cde70 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc06afc0 con 0x7f86fc0cde70 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673636351 0 0) 0x7f86e80035d0 con 0x7f86fc15d5b0 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 >> v2:172.21.15.195:3300/0 conn(0x7f86fc0ce2d0 msgr2=0x7f86fc06bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 --2- 172.21.15.179:0/1132862478 >> v2:172.21.15.195:3300/0 conn(0x7f86fc0ce2d0 0x7f86fc06bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 >> v1:172.21.15.195:6789/0 conn(0x7f86fc0cde70 legacy=0x7f86fc06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0d2130 con 0x7f86fc15d5b0 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f86ec001670 con 0x7f86fc15d5b0 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 >> v1:172.21.15.179:6789/0 conn(0x7f86fc15d5b0 legacy=0x7f86fc0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e8003bb0 0x7f86e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8004870 0x7f86e8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 0x7f86e800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0d26f0 con 0x7f86e8005230 2024-05-08T22:38:22.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e800c0b0 con 0x7f86e8003bb0 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e800c270 con 0x7f86e8004870 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 0x7f86e800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fb7fe700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e8003bb0 0x7f86e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86f2ffd700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8004870 0x7f86e8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 -- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8004870 msgr2=0x7f86e8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8004870 0x7f86e8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 -- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e8003bb0 msgr2=0x7f86e8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e8003bb0 0x7f86e8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 -- 172.21.15.179:0/1132862478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80035d0 con 0x7f86e8005230 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.028+0000 7f86fbfff700 1 -- 172.21.15.179:0/1132862478 reap_dead start 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 0x7f86e800ba30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fc0 tx=0x7f86e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18403854df642219 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003a40 con 0x7f86e8005230 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f400f040 con 0x7f86e8005230 2024-05-08T22:38:22.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86faffd700 1 -- 172.21.15.179:0/1132862478 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003e00 con 0x7f86e8005230 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 msgr2=0x7f86e800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 0x7f86e800ba30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fc0 tx=0x7f86e80058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- 172.21.15.179:0/1132862478 shutdown_connections 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 --2- 172.21.15.179:0/1132862478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8005230 0x7f86e800ba30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- 172.21.15.179:0/1132862478 >> 172.21.15.179:0/1132862478 conn(0x7f86fc0b4940 msgr2=0x7f86fc0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- 172.21.15.179:0/1132862478 shutdown_connections 2024-05-08T22:38:22.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- 172.21.15.179:0/1132862478 wait complete. 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 Processor -- start 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- start start 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8005230 0x7f86fc0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 0x7f86fc156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0c6240 con 0x7f86fc154cd0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc156f40 con 0x7f86fc156580 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f87009f3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc155690 con 0x7f86e8005230 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39264/0 (socket says 172.21.15.179:39264) 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 -- 172.21.15.179:0/2010578172 learned_addr learned my addr 172.21.15.179:0/2010578172 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fb7fe700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8005230 0x7f86fc0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86f2ffd700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 0x7f86fc156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 -- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8005230 msgr2=0x7f86fc0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8005230 0x7f86fc0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 -- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 msgr2=0x7f86fc156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 0x7f86fc156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86fbfff700 1 -- 172.21.15.179:0/2010578172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0cc2a0 con 0x7f86fc154cd0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.032+0000 7f86f2ffd700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 0x7f86fc156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86fbfff700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f86fc08a5e0 tx=0x7f86f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:22.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003e30 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4004230 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc0c9f20 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40043c0 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc155b90 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc0cf0e0 con 0x7f86fc154cd0 2024-05-08T22:38:22.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86f4004520 con 0x7f86fc154cd0 2024-05-08T22:38:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 0x7f86dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f86f4085870 con 0x7f86fc154cd0 2024-05-08T22:38:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.036+0000 7f86fb7fe700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 0x7f86dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:22.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.040+0000 7f86fb7fe700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 0x7f86dc068bf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f86ec001050 tx=0x7f86ec001e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:22.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.044+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f86f4052ad0 con 0x7f86fc154cd0 2024-05-08T22:38:22.146 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:38:22.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:38:22.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.252+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86fc04e700 con 0x7f86fc154cd0 2024-05-08T22:38:22.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f86f8ff9700 1 -- 172.21.15.179:0/2010578172 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f86f40560f0 con 0x7f86fc154cd0 2024-05-08T22:38:22.259 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:22.259 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:21.896811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:22.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 msgr2=0x7f86dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 0x7f86dc068bf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f86ec001050 tx=0x7f86ec001e20 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 msgr2=0x7f86fc155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f86fc08a5e0 tx=0x7f86f4009750 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 shutdown_connections 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86dc066730 0x7f86dc068bf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86fc156580 0x7f86fc156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86fc154cd0 0x7f86fc155150 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 --2- 172.21.15.179:0/2010578172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8005230 0x7f86fc0c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 >> 172.21.15.179:0/2010578172 conn(0x7f86fc0b4940 msgr2=0x7f86fc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 shutdown_connections 2024-05-08T22:38:22.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:22.256+0000 7f87009f3700 1 -- 172.21.15.179:0/2010578172 wait complete. 2024-05-08T22:38:22.263 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 42 2024-05-08T22:38:22.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:37:57.812267+0000', 'modified': '2024-05-08T22:38:21.896811+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/1691042534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 1691042534}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1691042534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:38:22.677 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_56354:up:replay 2024-05-08T22:38:23.677 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:23.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 Processor -- start 2024-05-08T22:38:23.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 -- start start 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 0x7f40ac0e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f40ac0e0250 0x7f40ac0e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f40ac006150 0x7f40ac038ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0e49a0 con 0x7f40ac006150 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0540b0 con 0x7f40ac0e0250 2024-05-08T22:38:23.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.796+0000 7f40b359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0532f0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f40ac006150 0x7f40ac038ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f40ac006150 0x7f40ac038ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39272/0 (socket says 172.21.15.179:39272) 2024-05-08T22:38:23.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 -- 172.21.15.179:0/1641054234 learned_addr learned my addr 172.21.15.179:0/1641054234 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:23.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b1d9b700 1 --2- 172.21.15.179:0/1641054234 >> v2:172.21.15.195:3300/0 conn(0x7f40ac0e0250 0x7f40ac0e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b259c700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 0x7f40ac0e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 -- 172.21.15.179:0/1641054234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 msgr2=0x7f40ac0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 0x7f40ac0e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 -- 172.21.15.179:0/1641054234 >> v2:172.21.15.195:3300/0 conn(0x7f40ac0e0250 msgr2=0x7f40ac0e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 --2- 172.21.15.179:0/1641054234 >> v2:172.21.15.195:3300/0 conn(0x7f40ac0e0250 0x7f40ac0e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 -- 172.21.15.179:0/1641054234 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0535d0 con 0x7f40ac006150 2024-05-08T22:38:23.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b259c700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 0x7f40ac0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b2d9d700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac006150 0x7f40ac038ec0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f40b4068290 tx=0x7f40b404f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517324d67723c3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b159a700 1 -- 172.21.15.179:0/1641054234 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4067250 con 0x7f40ac006150 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b159a700 1 -- 172.21.15.179:0/1641054234 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40673b0 con 0x7f40ac006150 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b159a700 1 -- 172.21.15.179:0/1641054234 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4067960 con 0x7f40ac006150 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 -- 172.21.15.179:0/1641054234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac006150 msgr2=0x7f40ac038ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac006150 0x7f40ac038ec0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f40b4068290 tx=0x7f40b404f740 comp rx=0 tx=0).stop 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 -- 172.21.15.179:0/1641054234 shutdown_connections 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac006150 0x7f40ac038ec0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 --2- 172.21.15.179:0/1641054234 >> v2:172.21.15.195:3300/0 conn(0x7f40ac0e0250 0x7f40ac0e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 --2- 172.21.15.179:0/1641054234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac0e3b40 0x7f40ac0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 -- 172.21.15.179:0/1641054234 >> 172.21.15.179:0/1641054234 conn(0x7f40ac054b70 msgr2=0x7f40ac054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 -- 172.21.15.179:0/1641054234 shutdown_connections 2024-05-08T22:38:23.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.800+0000 7f40b359e700 1 -- 172.21.15.179:0/1641054234 wait complete. 2024-05-08T22:38:23.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 Processor -- start 2024-05-08T22:38:23.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- start start 2024-05-08T22:38:23.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac006150 0x7f40ac04dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 0x7f40ac0dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac04d840 con 0x7f40ac0e3b40 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0dcce0 con 0x7f40ac0dc2f0 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac04ad90 con 0x7f40ac006150 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39284/0 (socket says 172.21.15.179:39284) 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 -- 172.21.15.179:0/1524461722 learned_addr learned my addr 172.21.15.179:0/1524461722 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b259c700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac006150 0x7f40ac04dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b2d9d700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 0x7f40ac0dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 -- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac006150 msgr2=0x7f40ac04dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac006150 0x7f40ac04dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 -- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 msgr2=0x7f40ac0dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 0x7f40ac0dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 -- 172.21.15.179:0/1524461722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac04cbc0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b2d9d700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 0x7f40ac0dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:23.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b1d9b700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e7f0 tx=0x7f409c00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c013680 con 0x7f40ac0e3b40 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0137e0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac04cea0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c013b90 con 0x7f40ac0e3b40 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.804+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac04d460 con 0x7f40ac0e3b40 2024-05-08T22:38:23.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac005440 con 0x7f40ac0e3b40 2024-05-08T22:38:23.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f409c013cf0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40aaffd700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 0x7f4094068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:23.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f409c0842d0 con 0x7f40ac0e3b40 2024-05-08T22:38:23.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40b259c700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 0x7f4094068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:23.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.808+0000 7f40b259c700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 0x7f4094068bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c920 tx=0x7f40a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:23.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:23.816+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f409c051410 con 0x7f40ac0e3b40 2024-05-08T22:38:24.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.024+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40ac054280 con 0x7f40ac0e3b40 2024-05-08T22:38:24.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.024+0000 7f40aaffd700 1 -- 172.21.15.179:0/1524461722 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f409c054a30 con 0x7f40ac0e3b40 2024-05-08T22:38:24.029 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:24.029 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:23.901756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 msgr2=0x7f4094068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 0x7f4094068bf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c920 tx=0x7f40a4008040 comp rx=0 tx=0).stop 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 msgr2=0x7f40ac04a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e7f0 tx=0x7f409c00ebb0 comp rx=0 tx=0).stop 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 shutdown_connections 2024-05-08T22:38:24.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4094066730 0x7f4094068bf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40ac0dc2f0 0x7f40ac0dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40ac0e3b40 0x7f40ac04a850 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 --2- 172.21.15.179:0/1524461722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40ac006150 0x7f40ac04dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 >> 172.21.15.179:0/1524461722 conn(0x7f40ac054b70 msgr2=0x7f40ac04ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 shutdown_connections 2024-05-08T22:38:24.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:24.028+0000 7f40b359e700 1 -- 172.21.15.179:0/1524461722 wait complete. 2024-05-08T22:38:24.033 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 44 2024-05-08T22:38:24.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:37:57.812267+0000', 'modified': '2024-05-08T22:38:23.901756+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.195:6837/1691042534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 1691042534}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1691042534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:38:24.446 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_56354:up:rejoin 2024-05-08T22:38:25.447 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:25.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 Processor -- start 2024-05-08T22:38:25.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- start start 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efe580ce380 0x7efe580cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 0x7efe5806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5806abb0 con 0x7efe580cc700 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe580cb9a0 con 0x7efe580ce380 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5806ae90 con 0x7efe580cb510 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efe580ce380 0x7efe580cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efe580ce380 0x7efe580cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53920/0 (socket says 172.21.15.179:53920) 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 -- 172.21.15.179:0/1399632525 learned_addr learned my addr 172.21.15.179:0/1399632525 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5ddfa700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 0x7efe5806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 -- 172.21.15.179:0/1399632525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 msgr2=0x7efe5806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 0x7efe5806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 -- 172.21.15.179:0/1399632525 >> v1:172.21.15.179:6789/0 conn(0x7efe580cc700 legacy=0x7efe5815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 -- 172.21.15.179:0/1399632525 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580ce800 con 0x7efe580ce380 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5ddfa700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 0x7efe5806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5cdf8700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe580cafd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efe54009910 tx=0x7efe54009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea40a21cc9cc219 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe4ffff700 1 -- 172.21.15.179:0/1399632525 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5400ba60 con 0x7efe580ce380 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe4ffff700 1 -- 172.21.15.179:0/1399632525 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5400bbc0 con 0x7efe580ce380 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe4ffff700 1 -- 172.21.15.179:0/1399632525 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54009e60 con 0x7efe580ce380 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- 172.21.15.179:0/1399632525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 msgr2=0x7efe580cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe580cafd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efe54009910 tx=0x7efe54009c20 comp rx=0 tx=0).stop 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- 172.21.15.179:0/1399632525 shutdown_connections 2024-05-08T22:38:25.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cb510 0x7efe5806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/1399632525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe580cafd0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- 172.21.15.179:0/1399632525 >> 172.21.15.179:0/1399632525 conn(0x7efe580ce9d0 msgr2=0x7efe580cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:25.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- 172.21.15.179:0/1399632525 shutdown_connections 2024-05-08T22:38:25.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.568+0000 7efe5e5fb700 1 -- 172.21.15.179:0/1399632525 wait complete. 2024-05-08T22:38:25.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 Processor -- start 2024-05-08T22:38:25.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 -- start start 2024-05-08T22:38:25.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe580cb510 0x7efe580b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cc700 0x7efe5814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe580b3b80 con 0x7efe580ce380 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe581f5d80 con 0x7efe580cb510 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5815d590 con 0x7efe580cc700 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53934/0 (socket says 172.21.15.179:53934) 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 -- 172.21.15.179:0/2591636959 learned_addr learned my addr 172.21.15.179:0/2591636959 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5cdf8700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cc700 0x7efe5814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5d5f9700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe580cb510 0x7efe580b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 -- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cc700 msgr2=0x7efe5814f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cc700 0x7efe5814f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 -- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe580cb510 msgr2=0x7efe580b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe580cb510 0x7efe580b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 -- 172.21.15.179:0/2591636959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54009570 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5ddfa700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efe5000cef0 tx=0x7efe5000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5000ad00 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5000ae60 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.572+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5815d870 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe50015680 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5815de30 con 0x7efe580ce380 2024-05-08T22:38:25.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe58067740 con 0x7efe580ce380 2024-05-08T22:38:25.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe5000d430 con 0x7efe580ce380 2024-05-08T22:38:25.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe4dffb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 0x7efe3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:25.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.576+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7efe50084720 con 0x7efe580ce380 2024-05-08T22:38:25.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.580+0000 7efe5d5f9700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 0x7efe3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:25.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.580+0000 7efe5d5f9700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 0x7efe3c068bf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efe48000f60 tx=0x7efe48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:25.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.584+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe50051980 con 0x7efe580ce380 2024-05-08T22:38:25.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.792+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe581f5ef0 con 0x7efe580ce380 2024-05-08T22:38:25.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.792+0000 7efe4dffb700 1 -- 172.21.15.179:0/2591636959 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4753 (secure 0 0 0) 0x7efe50054fa0 con 0x7efe580ce380 2024-05-08T22:38:25.795 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:25.796 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:24.904013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 msgr2=0x7efe3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 0x7efe3c068bf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efe48000f60 tx=0x7efe48009040 comp rx=0 tx=0).stop 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 msgr2=0x7efe581f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efe5000cef0 tx=0x7efe5000cf20 comp rx=0 tx=0).stop 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 shutdown_connections 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efe3c066730 0x7efe3c068bf0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe580ce380 0x7efe581f27b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe580cc700 0x7efe5814f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 --2- 172.21.15.179:0/2591636959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe580cb510 0x7efe580b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 >> 172.21.15.179:0/2591636959 conn(0x7efe580ce9d0 msgr2=0x7efe5815a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 shutdown_connections 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:25.796+0000 7efe5e5fb700 1 -- 172.21.15.179:0/2591636959 wait complete. 2024-05-08T22:38:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 45 2024-05-08T22:38:26.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:37:57.812267+0000', 'modified': '2024-05-08T22:38:24.904013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6837/1691042534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 1691042534}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 1691042534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:38:26.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:38:26.216 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:38:26.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.340+0000 7fce59727700 1 Processor -- start 2024-05-08T22:38:26.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.340+0000 7fce59727700 1 -- start start 2024-05-08T22:38:26.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fce4c0e0250 0x7fce4c04f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0540b0 con 0x7fce4c006150 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0532f0 con 0x7fce4c0e3b40 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0535d0 con 0x7fce4c0e0250 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce537fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fce4c0e0250 0x7fce4c04f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce58f26700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fce4c006150 0x7fce4c04f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56478/0 (socket says 172.21.15.179:56478) 2024-05-08T22:38:26.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce58f26700 1 -- 172.21.15.179:0/1536375296 learned_addr learned my addr 172.21.15.179:0/1536375296 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce537fe700 1 -- 172.21.15.179:0/1536375296 >> v1:172.21.15.195:6789/0 conn(0x7fce4c0e3b40 legacy=0x7fce4c0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce52ffd700 1 -- 172.21.15.179:0/1536375296 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021761143 0 0) 0x7fce4c0532f0 con 0x7fce4c0e3b40 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce537fe700 1 -- 172.21.15.179:0/1536375296 >> v1:172.21.15.179:6789/0 conn(0x7fce4c006150 legacy=0x7fce4c04f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce537fe700 1 -- 172.21.15.179:0/1536375296 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0538b0 con 0x7fce4c0e0250 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce537fe700 1 --2- 172.21.15.179:0/1536375296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c0e0250 0x7fce4c04f1c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fce40009850 tx=0x7fce40009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25474b34c2184436 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce52ffd700 1 -- 172.21.15.179:0/1536375296 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40004230 con 0x7fce4c0e0250 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce52ffd700 1 -- 172.21.15.179:0/1536375296 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40004390 con 0x7fce4c0e0250 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce52ffd700 1 -- 172.21.15.179:0/1536375296 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400036c0 con 0x7fce4c0e0250 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- 172.21.15.179:0/1536375296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c0e0250 msgr2=0x7fce4c04f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 --2- 172.21.15.179:0/1536375296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c0e0250 0x7fce4c04f1c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fce40009850 tx=0x7fce40009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- 172.21.15.179:0/1536375296 shutdown_connections 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 --2- 172.21.15.179:0/1536375296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c0e0250 0x7fce4c04f1c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- 172.21.15.179:0/1536375296 >> 172.21.15.179:0/1536375296 conn(0x7fce4c054b70 msgr2=0x7fce4c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- 172.21.15.179:0/1536375296 shutdown_connections 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.344+0000 7fce59727700 1 -- 172.21.15.179:0/1536375296 wait complete. 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 Processor -- start 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 -- start start 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce4c0e3b40 0x7fce4c04c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4c0d3520 0x7fce4c037310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c04dc40 con 0x7fce4c006150 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c04b370 con 0x7fce4c0d3520 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c04cfc0 con 0x7fce4c0e3b40 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53952/0 (socket says 172.21.15.179:53952) 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 -- 172.21.15.179:0/2223416308 learned_addr learned my addr 172.21.15.179:0/2223416308 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce537fe700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce4c0e3b40 0x7fce4c04c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce58f26700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4c0d3520 0x7fce4c037310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 -- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce4c0e3b40 msgr2=0x7fce4c04c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce4c0e3b40 0x7fce4c04c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 -- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4c0d3520 msgr2=0x7fce4c037310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4c0d3520 0x7fce4c037310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 -- 172.21.15.179:0/2223416308 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce400094b0 con 0x7fce4c006150 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce53fff700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c450 tx=0x7fce4800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce50ff9700 1 -- 172.21.15.179:0/2223416308 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480054e0 con 0x7fce4c006150 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce50ff9700 1 -- 172.21.15.179:0/2223416308 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800ce60 con 0x7fce4c006150 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c04d2a0 con 0x7fce4c006150 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.348+0000 7fce50ff9700 1 -- 172.21.15.179:0/2223416308 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480038e0 con 0x7fce4c006150 2024-05-08T22:38:26.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.352+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c04d860 con 0x7fce4c006150 2024-05-08T22:38:26.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.352+0000 7fce50ff9700 1 -- 172.21.15.179:0/2223416308 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce48003a40 con 0x7fce4c006150 2024-05-08T22:38:26.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.352+0000 7fce50ff9700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.352+0000 7fce50ff9700 1 -- 172.21.15.179:0/2223416308 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fce48083650 con 0x7fce4c006150 2024-05-08T22:38:26.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.352+0000 7fce537fe700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 Processor -- start 2024-05-08T22:38:26.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- start start 2024-05-08T22:38:26.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce537fe700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 0x7fce38068bf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce40009850 tx=0x7fce40008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea8be0 0x1e79470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 0x1e799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd8d80 con 0x1eb1b00 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd6510 con 0x1eb2420 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd1e00 con 0x1ea8be0 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53958/0 (socket says 172.21.15.179:53958) 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 -- 172.21.15.179:0/553443774 learned_addr learned my addr 172.21.15.179:0/553443774 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce58f26700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 0x1e799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce537fe700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea8be0 0x1e79470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 -- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea8be0 msgr2=0x1e79470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea8be0 0x1e79470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 -- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 msgr2=0x1e799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 0x1e799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 -- 172.21.15.179:0/553443774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c44960 con 0x1eb1b00 2024-05-08T22:38:26.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce58f26700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 0x1e799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:26.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce53fff700 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce480050a0 tx=0x7fce48001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a016bef0ea4af46b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce3dffb700 1 -- 172.21.15.179:0/553443774 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4805c8d0 con 0x1eb1b00 2024-05-08T22:38:26.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce3dffb700 1 -- 172.21.15.179:0/553443774 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48052620 con 0x1eb1b00 2024-05-08T22:38:26.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce3dffb700 1 -- 172.21.15.179:0/553443774 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48054740 con 0x1eb1b00 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- 172.21.15.179:0/553443774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 msgr2=0x1eb1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce480050a0 tx=0x7fce48001260 comp rx=0 tx=0).stop 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- 172.21.15.179:0/553443774 shutdown_connections 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb2420 0x1e799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea8be0 0x1e79470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 --2- 172.21.15.179:0/553443774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1eb1ee0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- 172.21.15.179:0/553443774 >> 172.21.15.179:0/553443774 conn(0x1bf94d0 msgr2=0x1e93250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- 172.21.15.179:0/553443774 shutdown_connections 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.356+0000 7fce5dc38740 1 -- 172.21.15.179:0/553443774 wait complete. 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 -- start start 2024-05-08T22:38:26.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 Processor -- start 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1eb7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d8f730 con 0x1eb1b00 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d955b0 con 0x1ea8be0 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c7daa0 con 0x1eb2420 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce5dc38740 10 client.0 _ll_register_callbacks cb 0x1dcb610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53960/0 (socket says 172.21.15.179:53960) 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 -- 172.21.15.179:0/2893167206 learned_addr learned my addr 172.21.15.179:0/2893167206 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce53fff700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1eb7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce58f26700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 -- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 msgr2=0x1e7c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e7c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 -- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 msgr2=0x1eb7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1eb7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 -- 172.21.15.179:0/2893167206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c04d860 con 0x1eb1b00 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce53fff700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1eb7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce537fe700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007930 tx=0x7fce400047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb089b87584099f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.360+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40006ea0 con 0x1eb1b00 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fce40003e30 con 0x1eb1b00 2024-05-08T22:38:26.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c061e0 con 0x1eb1b00 2024-05-08T22:38:26.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce40006c80 con 0x1eb1b00 2024-05-08T22:38:26.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 10 client.7404 fetch_fsmap learned FSMap version 45 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x1d98220 con 0x1eb1b00 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c061e0 con 0x1eb1b00 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fce4001c450 con 0x1eb1b00 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 10 client.7404 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 10 client.7404 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 20 client.7404 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1e6bc30 0x1eaa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 4 client.7404 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c50460 con 0x1e6bc30 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce53fff700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1e6bc30 0x1eaa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 10 client.7404 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce53fff700 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1e6bc30 0x1eaa2e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5526b2676aa6366 server_cookie=812decf35789274a in_seq=0 out_seq=0 2024-05-08T22:38:26.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 1 -- 172.21.15.179:0/2893167206 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1c50460 con 0x1e6bc30 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce32ffd700 10 client.7404 handle_command_reply: tid=0 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 client.7404 shutdown 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 msgr2=0x1ebaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007930 tx=0x7fce400047e0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 shutdown_connections 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb1b00 0x1ebaae0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.364+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1e6bc30 0x1eaa2e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2893167206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1eb7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 >> 172.21.15.179:0/2893167206 conn(0x1bf94d0 msgr2=0x1e928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce32ffd700 0 client.7404 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 shutdown_connections 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce5dc38740 1 -- 172.21.15.179:0/2893167206 wait complete. 2024-05-08T22:38:26.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.368+0000 7fce5dc38740 20 client.7404 trim_cache size 0 max 16384 2024-05-08T22:38:26.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 Processor -- start 2024-05-08T22:38:26.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- start start 2024-05-08T22:38:26.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1e8f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb1b00 0x1e8f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eab670 con 0x1eb2420 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd0940 con 0x1ea8be0 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d70470 con 0x1eb1b00 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53964/0 (socket says 172.21.15.179:53964) 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 -- 172.21.15.179:0/1260397033 learned_addr learned my addr 172.21.15.179:0/1260397033 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce53fff700 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1e8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce537fe700 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb1b00 0x1e8f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 -- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb1b00 msgr2=0x1e8f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb1b00 0x1e8f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 -- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 msgr2=0x1e8f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1e8f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 -- 172.21.15.179:0/1260397033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d7e450 con 0x1eb2420 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce58f26700 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fce54063cf0 tx=0x7fce5408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e6184600f22c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce3e7fc700 1 -- 172.21.15.179:0/1260397033 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5408e040 con 0x1eb2420 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce3e7fc700 1 -- 172.21.15.179:0/1260397033 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5408f040 con 0x1eb2420 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce3e7fc700 1 -- 172.21.15.179:0/1260397033 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54067da0 con 0x1eb2420 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- 172.21.15.179:0/1260397033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 msgr2=0x1e69730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fce54063cf0 tx=0x7fce5408b040 comp rx=0 tx=0).stop 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- 172.21.15.179:0/1260397033 shutdown_connections 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb2420 0x1e69730 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb1b00 0x1e8f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 --2- 172.21.15.179:0/1260397033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea8be0 0x1e8f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.388+0000 7fce5dc38740 1 -- 172.21.15.179:0/1260397033 >> 172.21.15.179:0/1260397033 conn(0x1bf94d0 msgr2=0x1eb07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- 172.21.15.179:0/1260397033 shutdown_connections 2024-05-08T22:38:26.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- 172.21.15.179:0/1260397033 wait complete. 2024-05-08T22:38:26.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- start start 2024-05-08T22:38:26.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 Processor -- start 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb1b00 0x1e69bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e68bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ed0bc0 con 0x1ea8be0 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ec1ec0 con 0x1eb1b00 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e6b420 con 0x1eb2420 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce53fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53970/0 (socket says 172.21.15.179:53970) 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce53fff700 1 -- 172.21.15.179:0/2399386194 learned_addr learned my addr 172.21.15.179:0/2399386194 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce5dc38740 10 client.0 _ll_register_callbacks cb 0x7fce480508a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce537fe700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb1b00 0x1e69bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.392+0000 7fce58f26700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e68bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 -- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 msgr2=0x1e68bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e68bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 -- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb1b00 msgr2=0x1e69bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb1b00 0x1e69bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 -- 172.21.15.179:0/2399386194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d72720 con 0x1ea8be0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce53fff700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce48001eb0 tx=0x7fce4801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e3d12231df749b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4805c8d0 con 0x1ea8be0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fce48080070 con 0x1ea8be0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce48052620 con 0x1ea8be0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce480539d0 con 0x1ea8be0 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 10 client.7410 fetch_fsmap learned FSMap version 45 2024-05-08T22:38:26.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x1d09c70 con 0x1ea8be0 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48052620 con 0x1ea8be0 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fce4807b460 con 0x1ea8be0 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 10 client.7410 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 10 client.7410 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 20 client.7410 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f13670 0x1f13a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 4 client.7410 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1c751e0 con 0x1f13670 2024-05-08T22:38:26.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce537fe700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f13670 0x1f13a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 10 client.7410 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce537fe700 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f13670 0x1f13a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f40de847ad7905 server_cookie=50f4cac32a85bf01 in_seq=0 out_seq=0 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 1 -- 172.21.15.179:0/2399386194 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x1c751e0 con 0x1f13670 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 10 client.7410 handle_command_reply: tid=0 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 client.7410 shutdown 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 msgr2=0x1f126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce48001eb0 tx=0x7fce4801e070 comp rx=0 tx=0).stop 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 shutdown_connections 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f13670 0x1f13a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eb1b00 0x1e69bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb2420 0x1e68bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce32ffd700 0 client.7410 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 --2- 172.21.15.179:0/2399386194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea8be0 0x1f126e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 >> 172.21.15.179:0/2399386194 conn(0x1bf94d0 msgr2=0x1e8f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 shutdown_connections 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.396+0000 7fce5dc38740 1 -- 172.21.15.179:0/2399386194 wait complete. 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce5dc38740 20 client.7410 trim_cache size 0 max 16384 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 56354, 2024-05-08T22:38:26.405 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.4969163300000004, 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 45, 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 84, 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 5.4711967660000003 2024-05-08T22:38:26.406 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:26.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 msgr2=0x7fce38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 0x7fce38068bf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce40009850 tx=0x7fce40008030 comp rx=0 tx=0).stop 2024-05-08T22:38:26.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 msgr2=0x7fce4c04ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c450 tx=0x7fce4800c760 comp rx=0 tx=0).stop 2024-05-08T22:38:26.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 shutdown_connections 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce38066730 0x7fce38068bf0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce4c0d3520 0x7fce4c037310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce4c0e3b40 0x7fce4c04c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 --2- 172.21.15.179:0/2223416308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce4c006150 0x7fce4c04ae30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 >> 172.21.15.179:0/2223416308 conn(0x7fce4c054b70 msgr2=0x7fce4c04f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 shutdown_connections 2024-05-08T22:38:26.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.400+0000 7fce59727700 1 -- 172.21.15.179:0/2223416308 wait complete. 2024-05-08T22:38:26.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 Processor -- start 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 -- start start 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57100cb6e0 0x7f5710165b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571006abb0 con 0x7f57100cc700 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571006ae90 con 0x7f57100ce380 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716f02700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100ce800 con 0x7f57100cb6e0 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716701700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57100cb6e0 0x7f5710165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5715f00700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f57100cc700 0x7f571015b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56488/0 (socket says 172.21.15.179:56488) 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5715f00700 1 -- 172.21.15.179:0/2169663102 learned_addr learned my addr 172.21.15.179:0/2169663102 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716701700 1 -- 172.21.15.179:0/2169663102 >> v1:172.21.15.195:6789/0 conn(0x7f57100ce380 legacy=0x7f57100cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716701700 1 -- 172.21.15.179:0/2169663102 >> v1:172.21.15.179:6789/0 conn(0x7f57100cc700 legacy=0x7f571015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5714efe700 1 -- 172.21.15.179:0/2169663102 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428846115 0 0) 0x7f571006abb0 con 0x7f57100cc700 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.520+0000 7f5716701700 1 -- 172.21.15.179:0/2169663102 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571015b9c0 con 0x7f57100cb6e0 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716701700 1 --2- 172.21.15.179:0/2169663102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f5710165b60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5708009850 tx=0x7f5708009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2531a3dd27341e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5714efe700 1 -- 172.21.15.179:0/2169663102 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708011830 con 0x7f57100cb6e0 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5714efe700 1 -- 172.21.15.179:0/2169663102 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5708011990 con 0x7f57100cb6e0 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5714efe700 1 -- 172.21.15.179:0/2169663102 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708011d20 con 0x7f57100cb6e0 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- 172.21.15.179:0/2169663102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 msgr2=0x7f5710165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 --2- 172.21.15.179:0/2169663102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f5710165b60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5708009850 tx=0x7f5708009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- 172.21.15.179:0/2169663102 shutdown_connections 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 --2- 172.21.15.179:0/2169663102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f5710165b60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- 172.21.15.179:0/2169663102 >> 172.21.15.179:0/2169663102 conn(0x7f57100ce9d0 msgr2=0x7f57100cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- 172.21.15.179:0/2169663102 shutdown_connections 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- 172.21.15.179:0/2169663102 wait complete. 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 Processor -- start 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.524+0000 7f5716f02700 1 -- start start 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 0x7f571013d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57100ce380 0x7f571013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571013b370 con 0x7f57100cb6e0 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571013b650 con 0x7f57100cc700 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571013b930 con 0x7f57100ce380 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53984/0 (socket says 172.21.15.179:53984) 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 -- 172.21.15.179:0/925201869 learned_addr learned my addr 172.21.15.179:0/925201869 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f57156ff700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 0x7f571013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716701700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57100ce380 0x7f571013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 -- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57100ce380 msgr2=0x7f571013a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57100ce380 0x7f571013a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 -- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 msgr2=0x7f571013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 0x7f571013d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 -- 172.21.15.179:0/925201869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080094b0 con 0x7f57100cb6e0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f57156ff700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 0x7f571013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5715f00700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f570000c4a0 tx=0x7f570000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700004ca0 con 0x7f57100cb6e0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700004e00 con 0x7f57100cb6e0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571013bc10 con 0x7f57100cb6e0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007310 con 0x7f57100cb6e0 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571014d680 con 0x7f57100cb6e0 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.528+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571013c130 con 0x7f57100cb6e0 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.532+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5700007470 con 0x7f57100cb6e0 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.532+0000 7f57067fc700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.532+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5700084130 con 0x7f57100cb6e0 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.532+0000 7f57156ff700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.532+0000 7f57156ff700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 0x7f56f4068bf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5710072e50 tx=0x7f570c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.540+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f571013c130 con 0x7f57100cb6e0 2024-05-08T22:38:26.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.748+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f571013c130 con 0x7f57100cb6e0 2024-05-08T22:38:26.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.748+0000 7f57067fc700 1 -- 172.21.15.179:0/925201869 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f571013c130 con 0x7f57100cb6e0 2024-05-08T22:38:26.752 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.748+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 msgr2=0x7f56f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.748+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 0x7f56f4068bf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5710072e50 tx=0x7f570c009040 comp rx=0 tx=0).stop 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.748+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 msgr2=0x7f57100ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f570000c4a0 tx=0x7f570000c860 comp rx=0 tx=0).stop 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 shutdown_connections 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f56f4066730 0x7f56f4068bf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57100cc700 0x7f571013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57100ce380 0x7f571013a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 --2- 172.21.15.179:0/925201869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57100cb6e0 0x7f57100ad010 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:26.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 >> 172.21.15.179:0/925201869 conn(0x7f57100ce9d0 msgr2=0x7f571015b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:26.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 shutdown_connections 2024-05-08T22:38:26.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:26.752+0000 7f5716f02700 1 -- 172.21.15.179:0/925201869 wait complete. 2024-05-08T22:38:26.767 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:38:26.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:38:26.769 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:26.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.770 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:38:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42264sec preferred_lft 42264sec 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.777 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:38:26.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.779 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:38:26.791 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:38:26.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.792 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:38:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:38:26.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:38:26.843 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:38:26.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.883 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:38:26.905 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:38:26.903+0000 7fb216e91700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:38:26.933 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:38:26.933 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:38:26.933 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:26.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:26.933 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:38:26.947 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:38:26.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:38:26.948 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:26.952 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:38:26.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:38:26.953 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:26.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:27.019 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:27.019 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:27.033 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:27.034 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:27.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:27.035 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:27.040 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:38:27.041 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:38:27.043 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:27.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:27.059 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:27.114 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:27.115 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:27.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:27.115 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:27.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:38:27.124+0000 7fd4dbb7f640 -1 init, newargv = 0x559591a0b860 newargc=15 2024-05-08T22:38:27.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[43578]: starting ceph client 2024-05-08T22:38:27.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[43578]: starting fuse 2024-05-08T22:38:28.225 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:38:28.243 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:38:28.243 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:38:28.301 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:38:28.302 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:38:28.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:28.303 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:38:28.307 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:38:28.308 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:38:28.308 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.309 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:38:28.310 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:38:28.393 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.43578.asok 2024-05-08T22:38:28.578 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43578.asok 2024-05-08T22:38:28.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:28.579 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43578.asok status 2024-05-08T22:38:28.727 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:28.727 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "43578", 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 7428, 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:38:28.728 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 7428 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 528078214 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 528078214 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.7428 192.168.0.1:0/528078214", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/528078214", 2024-05-08T22:38:28.729 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 45, 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 84, 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:38:28.730 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:28.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:28.737 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:38:28.737 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:28.744 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:38:28.745 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:28.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:28.745 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:28.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:28.757 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-08T22:38:28.816 INFO:teuthology.orchestra.run.smithi179.stdout:file_undamaged 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> import json 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> import sys 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> try: 2024-05-08T22:38:28.818 DEBUG:teuthology.orchestra.run.smithi179:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> except OSError as e: 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> sys.exit(e.errno) 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> print(json.dumps( 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> indent=2)) 2024-05-08T22:38:28.819 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:38:29.065 INFO:teuthology.orchestra.run.smithi179.stderr:daemon-helper: command failed with exit status 2 2024-05-08T22:38:29.077 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T22:38:29.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:29.078 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-08T22:38:29.091 INFO:teuthology.orchestra.run.smithi179.stdout:2 2024-05-08T22:38:29.092 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:38:29.092 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:29.141 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:29.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 Processor -- start 2024-05-08T22:38:29.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 -- start start 2024-05-08T22:38:29.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f830006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000cd200 con 0x7f830006dc20 2024-05-08T22:38:29.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000cd4e0 con 0x7f830006ed50 2024-05-08T22:38:29.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f8304b22700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000cd7c0 con 0x7f83001434d0 2024-05-08T22:38:29.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82feffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f830006dc20 0x7f830006e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56496/0 (socket says 172.21.15.179:56496) 2024-05-08T22:38:29.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82feffd700 1 -- 172.21.15.179:0/2831454021 learned_addr learned my addr 172.21.15.179:0/2831454021 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:29.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82ff7fe700 1 --2- 172.21.15.179:0/2831454021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f830006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625467201 0 0) 0x7f83000cd200 con 0x7f830006dc20 2024-05-08T22:38:29.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e80035d0 con 0x7f830006dc20 2024-05-08T22:38:29.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732659858 0 0) 0x7f83000cd4e0 con 0x7f830006ed50 2024-05-08T22:38:29.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83000cd200 con 0x7f830006ed50 2024-05-08T22:38:29.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82ff7fe700 1 -- 172.21.15.179:0/2831454021 >> v1:172.21.15.195:6789/0 conn(0x7f830006ed50 legacy=0x7f83000b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1838108342 0 0) 0x7f82e80035d0 con 0x7f830006dc20 2024-05-08T22:38:29.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82ff7fe700 1 -- 172.21.15.179:0/2831454021 >> v1:172.21.15.179:6789/0 conn(0x7f830006dc20 legacy=0x7f830006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.256+0000 7f82ff7fe700 1 -- 172.21.15.179:0/2831454021 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000cdaa0 con 0x7f83001434d0 2024-05-08T22:38:29.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f82ff7fe700 1 --2- 172.21.15.179:0/2831454021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f830006d6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009880 tx=0x7f82f4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a395aa20e196ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00019f0 con 0x7f83001434d0 2024-05-08T22:38:29.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4004030 con 0x7f83001434d0 2024-05-08T22:38:29.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f82fe7fc700 1 -- 172.21.15.179:0/2831454021 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40043c0 con 0x7f83001434d0 2024-05-08T22:38:29.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- 172.21.15.179:0/2831454021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 msgr2=0x7f830006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 --2- 172.21.15.179:0/2831454021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f830006d6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009880 tx=0x7f82f4009b90 comp rx=0 tx=0).stop 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- 172.21.15.179:0/2831454021 shutdown_connections 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 --2- 172.21.15.179:0/2831454021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f830006d6e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- 172.21.15.179:0/2831454021 >> 172.21.15.179:0/2831454021 conn(0x7f83000ce9d0 msgr2=0x7f83000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- 172.21.15.179:0/2831454021 shutdown_connections 2024-05-08T22:38:29.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- 172.21.15.179:0/2831454021 wait complete. 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 Processor -- start 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.260+0000 7f8304b22700 1 -- start start 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f8300159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 0x7f8300159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001f2640 con 0x7f830006ed50 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001392a0 con 0x7f83001ef270 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8300139580 con 0x7f83001434d0 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53988/0 (socket says 172.21.15.179:53988) 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 -- 172.21.15.179:0/514650621 learned_addr learned my addr 172.21.15.179:0/514650621 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:29.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82feffd700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f8300159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82fffff700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 0x7f8300159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 -- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 msgr2=0x7f8300159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f8300159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 -- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 msgr2=0x7f8300159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 0x7f8300159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 -- 172.21.15.179:0/514650621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094b0 con 0x7f830006ed50 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82fffff700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 0x7f8300159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82ff7fe700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097d0 tx=0x7f82f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003c90 con 0x7f830006ed50 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4003df0 con 0x7f830006ed50 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8300139860 con 0x7f830006ed50 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011870 con 0x7f830006ed50 2024-05-08T22:38:29.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.264+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000b1f80 con 0x7f830006ed50 2024-05-08T22:38:29.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8300067740 con 0x7f830006ed50 2024-05-08T22:38:29.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82f4018050 con 0x7f830006ed50 2024-05-08T22:38:29.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f82e7fff700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 0x7f82ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f82f4084c70 con 0x7f830006ed50 2024-05-08T22:38:29.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f82feffd700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 0x7f82ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.268+0000 7f82feffd700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 0x7f82ec068bf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8300141450 tx=0x7f82f0000f40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.276+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f82f4051ed0 con 0x7f830006ed50 2024-05-08T22:38:29.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.448+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83000b2110 con 0x7f830006ed50 2024-05-08T22:38:29.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.448+0000 7f82e7fff700 1 -- 172.21.15.179:0/514650621 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f82f402b090 con 0x7f830006ed50 2024-05-08T22:38:29.453 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:29.453 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 msgr2=0x7f82ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 0x7f82ec068bf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8300141450 tx=0x7f82f0000f40 comp rx=0 tx=0).stop 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 msgr2=0x7f83000b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097d0 tx=0x7f82f4004770 comp rx=0 tx=0).stop 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 shutdown_connections 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82ec066730 0x7f82ec068bf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83001ef270 0x7f8300159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83001434d0 0x7f8300159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 --2- 172.21.15.179:0/514650621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f830006ed50 0x7f83000b1a40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 >> 172.21.15.179:0/514650621 conn(0x7f83000ce9d0 msgr2=0x7f83001416c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 shutdown_connections 2024-05-08T22:38:29.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.452+0000 7f8304b22700 1 -- 172.21.15.179:0/514650621 wait complete. 2024-05-08T22:38:29.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:29.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:29.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 Processor -- start 2024-05-08T22:38:29.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 -- start start 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f46e015d510 0x7f46e00cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f46e00b51b0 0x7f46e00b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e00b4ac0 con 0x7f46e00b46b0 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00cde60 con 0x7f46e015d510 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46e4a2c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00cd1e0 con 0x7f46e00b51b0 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46ded9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f46e00b46b0 0x7f46e00c9ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56510/0 (socket says 172.21.15.179:56510) 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46de59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f46e015d510 0x7f46e00cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46ded9d700 1 -- 172.21.15.179:0/2956757948 learned_addr learned my addr 172.21.15.179:0/2956757948 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.588+0000 7f46d7fff700 1 --2- 172.21.15.179:0/2956757948 >> v2:172.21.15.195:3300/0 conn(0x7f46e00b51b0 0x7f46e00b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208988572 0 0) 0x7f46e00b4ac0 con 0x7f46e00b46b0 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc0035d0 con 0x7f46e00b46b0 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3616301175 0 0) 0x7f46cc0035d0 con 0x7f46e00b46b0 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 >> v2:172.21.15.195:3300/0 conn(0x7f46e00b51b0 msgr2=0x7f46e00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 --2- 172.21.15.179:0/2956757948 >> v2:172.21.15.195:3300/0 conn(0x7f46e00b51b0 0x7f46e00b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 >> v2:172.21.15.179:3300/0 conn(0x7f46e015d510 msgr2=0x7f46e00cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 --2- 172.21.15.179:0/2956757948 >> v2:172.21.15.179:3300/0 conn(0x7f46e015d510 0x7f46e00cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00cd4c0 con 0x7f46e00b46b0 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f46d80019f0 con 0x7f46e00b46b0 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 >> v1:172.21.15.179:6789/0 conn(0x7f46e00b46b0 legacy=0x7f46e00c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46cc003a20 0x7f46cc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc004870 0x7f46cc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 0x7f46cc00dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00cde60 con 0x7f46cc005230 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00cd1e0 con 0x7f46cc004870 2024-05-08T22:38:29.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc00e1d0 con 0x7f46cc003a20 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46de59c700 1 --2- 172.21.15.179:0/2956757948 >> v2:172.21.15.179:3300/0 conn(0x7f46e015d510 0x7f46e00cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 0x7f46cc00dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46de59c700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46cc003a20 0x7f46cc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46d7fff700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc004870 0x7f46cc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 -- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46cc003a20 msgr2=0x7f46cc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46cc003a20 0x7f46cc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 -- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc004870 msgr2=0x7f46cc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc004870 0x7f46cc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 -- 172.21.15.179:0/2956757948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0035d0 con 0x7f46cc005230 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46d7fff700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc004870 0x7f46cc004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 -- 172.21.15.179:0/2956757948 reap_dead start 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ded9d700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 0x7f46cc00dc40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f46d800a860 tx=0x7f46cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43058b72b5f731f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80072d0 con 0x7f46cc005230 2024-05-08T22:38:29.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46ddd9b700 1 -- 172.21.15.179:0/2956757948 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8007450 con 0x7f46cc005230 2024-05-08T22:38:29.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 -- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 msgr2=0x7f46cc00dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 0x7f46cc00dc40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f46d800a860 tx=0x7f46cc0058a0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 -- 172.21.15.179:0/2956757948 shutdown_connections 2024-05-08T22:38:29.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/2956757948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46cc005230 0x7f46cc00dc40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 -- 172.21.15.179:0/2956757948 >> 172.21.15.179:0/2956757948 conn(0x7f46e0098a70 msgr2=0x7f46e00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:29.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 -- 172.21.15.179:0/2956757948 shutdown_connections 2024-05-08T22:38:29.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.592+0000 7f46e4a2c700 1 -- 172.21.15.179:0/2956757948 wait complete. 2024-05-08T22:38:29.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 Processor -- start 2024-05-08T22:38:29.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 -- start start 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc005230 0x7f46e00ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 0x7f46e00cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00aee50 con 0x7f46e01ee090 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00af900 con 0x7f46cc005230 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46e4a2c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e014b560 con 0x7f46e00cac80 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54012/0 (socket says 172.21.15.179:54012) 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46d7fff700 1 -- 172.21.15.179:0/1419327211 learned_addr learned my addr 172.21.15.179:0/1419327211 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46de59c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc005230 0x7f46e00ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.596+0000 7f46ded9d700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 0x7f46e00cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 -- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 msgr2=0x7f46e00cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 0x7f46e00cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 -- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc005230 msgr2=0x7f46e00ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc005230 0x7f46e00ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 -- 172.21.15.179:0/1419327211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00af060 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46ded9d700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 0x7f46e00cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d7fff700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f46c80108c0 tx=0x7f46c8010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004020 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8004180 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e00af340 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004530 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e01ee510 con 0x7f46e01ee090 2024-05-08T22:38:29.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e00990b0 con 0x7f46e01ee090 2024-05-08T22:38:29.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46c80036c0 con 0x7f46e01ee090 2024-05-08T22:38:29.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.600+0000 7f46d6ffd700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 0x7f46c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.604+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f46c8084160 con 0x7f46e01ee090 2024-05-08T22:38:29.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.604+0000 7f46de59c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 0x7f46c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.604+0000 7f46de59c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 0x7f46c0068bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009480 tx=0x7f46d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.608+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f46c80513c0 con 0x7f46e01ee090 2024-05-08T22:38:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.776+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46e0073290 con 0x7f46e01ee090 2024-05-08T22:38:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.776+0000 7f46d6ffd700 1 -- 172.21.15.179:0/1419327211 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46c80549e0 con 0x7f46e01ee090 2024-05-08T22:38:29.781 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:38:29.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 msgr2=0x7f46c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 0x7f46c0068bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009480 tx=0x7f46d0008040 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 msgr2=0x7f46e00ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f46c80108c0 tx=0x7f46c8010bd0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 shutdown_connections 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46c0066730 0x7f46c0068bf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46e00cac80 0x7f46e00cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ee090 0x7f46e00ca740 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 --2- 172.21.15.179:0/1419327211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46cc005230 0x7f46e00ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 >> 172.21.15.179:0/1419327211 conn(0x7f46e0098a70 msgr2=0x7f46e006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 shutdown_connections 2024-05-08T22:38:29.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.780+0000 7f46e4a2c700 1 -- 172.21.15.179:0/1419327211 wait complete. 2024-05-08T22:38:29.796 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:29.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 Processor -- start 2024-05-08T22:38:29.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 -- start start 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f542c0cbce0 0x7f542c0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0b4820 con 0x7f542c15b190 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0cc6b0 con 0x7f542c0cbce0 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.912+0000 7f5433668700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0b4b00 con 0x7f542c0b69e0 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f542c0cbce0 0x7f542c0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5430c03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431404700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f542c15b190 0x7f542c0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:51938/0 (socket says 172.21.15.179:51938) 2024-05-08T22:38:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431404700 1 -- 172.21.15.179:0/518907950 learned_addr learned my addr 172.21.15.179:0/518907950 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 -- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 msgr2=0x7f542c0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 --2- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 -- 172.21.15.179:0/518907950 >> v1:172.21.15.195:6789/0 conn(0x7f542c15b190 legacy=0x7f542c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 -- 172.21.15.179:0/518907950 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0b4de0 con 0x7f542c0cbce0 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5431c05700 1 --2- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c0cc170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5424009dc0 tx=0x7f542400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f247f174dabbfd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5423fff700 1 -- 172.21.15.179:0/518907950 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e070 con 0x7f542c0cbce0 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5423fff700 1 -- 172.21.15.179:0/518907950 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400bae0 con 0x7f542c0cbce0 2024-05-08T22:38:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5423fff700 1 -- 172.21.15.179:0/518907950 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400be90 con 0x7f542c0cbce0 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 -- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 msgr2=0x7f542c0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 --2- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c0cc170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5424009dc0 tx=0x7f542400b330 comp rx=0 tx=0).stop 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 -- 172.21.15.179:0/518907950 shutdown_connections 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 --2- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c0cc170 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 --2- 172.21.15.179:0/518907950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 -- 172.21.15.179:0/518907950 >> 172.21.15.179:0/518907950 conn(0x7f542c0d0f10 msgr2=0x7f542c0d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 -- 172.21.15.179:0/518907950 shutdown_connections 2024-05-08T22:38:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.916+0000 7f5433668700 1 -- 172.21.15.179:0/518907950 wait complete. 2024-05-08T22:38:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 Processor -- start 2024-05-08T22:38:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- start start 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0c9170 con 0x7f542c15b190 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0ca9d0 con 0x7f542c0cbce0 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0ca6f0 con 0x7f542c0b69e0 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54016/0 (socket says 172.21.15.179:54016) 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 -- 172.21.15.179:0/4137009145 learned_addr learned my addr 172.21.15.179:0/4137009145 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5430c03700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431404700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 -- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 msgr2=0x7f542c0c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 -- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 msgr2=0x7f542c153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 -- 172.21.15.179:0/4137009145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009a70 con 0x7f542c15b190 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5431c05700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f542400b370 tx=0x7f5424004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54240045d0 con 0x7f542c15b190 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424020070 con 0x7f542c15b190 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c0c8b20 con 0x7f542c15b190 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424004990 con 0x7f542c15b190 2024-05-08T22:38:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.920+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c150c10 con 0x7f542c15b190 2024-05-08T22:38:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c0c92e0 con 0x7f542c15b190 2024-05-08T22:38:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5424012430 con 0x7f542c15b190 2024-05-08T22:38:29.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5421ffb700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:29.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5424022070 con 0x7f542c15b190 2024-05-08T22:38:29.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5431404700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:29.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.924+0000 7f5431404700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f541c005930 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:29.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:29.932+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5424056b80 con 0x7f542c15b190 2024-05-08T22:38:30.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f542c073290 con 0x7f542c15b190 2024-05-08T22:38:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5421ffb700 1 -- 172.21.15.179:0/4137009145 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f542402c0a0 con 0x7f542c15b190 2024-05-08T22:38:30.103 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:38:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 msgr2=0x7f5410068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f541c005930 tx=0x7f541c009040 comp rx=0 tx=0).stop 2024-05-08T22:38:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 msgr2=0x7f542c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f542400b370 tx=0x7f5424004af0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 shutdown_connections 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f542c0cbce0 0x7f542c153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c15b190 0x7f542c154170 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 --2- 172.21.15.179:0/4137009145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f542c0b69e0 0x7f542c0c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 >> 172.21.15.179:0/4137009145 conn(0x7f542c0d0f10 msgr2=0x7f542c1580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 shutdown_connections 2024-05-08T22:38:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.100+0000 7f5433668700 1 -- 172.21.15.179:0/4137009145 wait complete. 2024-05-08T22:38:30.117 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:30.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 Processor -- start 2024-05-08T22:38:30.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 -- start start 2024-05-08T22:38:30.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 0x7f68700b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 0x7f68700cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f68700ce0e0 0x7f6870160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68700ce570 con 0x7f68700ce0e0 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687015d4a0 con 0x7f687015d0c0 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6875244700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687006abb0 con 0x7f68700d1d20 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f68700ce0e0 0x7f6870160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f68700ce0e0 0x7f6870160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54030/0 (socket says 172.21.15.179:54030) 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 -- 172.21.15.179:0/2508470809 learned_addr learned my addr 172.21.15.179:0/2508470809 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f686ffff700 1 --2- 172.21.15.179:0/2508470809 >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 0x7f68700b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f68677fe700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 0x7f68700cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 -- 172.21.15.179:0/2508470809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 msgr2=0x7f68700cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 0x7f68700cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 -- 172.21.15.179:0/2508470809 >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 msgr2=0x7f68700b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 --2- 172.21.15.179:0/2508470809 >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 0x7f68700b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 -- 172.21.15.179:0/2508470809 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700d2500 con 0x7f68700ce0e0 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f686ffff700 1 --2- 172.21.15.179:0/2508470809 >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 0x7f68700b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f68677fe700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 0x7f68700cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f6874a43700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700ce0e0 0x7f6870160ba0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f687008b1c0 tx=0x7f686800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4148d1d3229fb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f686f7fe700 1 -- 172.21.15.179:0/2508470809 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800e070 con 0x7f68700ce0e0 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f686f7fe700 1 -- 172.21.15.179:0/2508470809 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800b9c0 con 0x7f68700ce0e0 2024-05-08T22:38:30.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.240+0000 7f686f7fe700 1 -- 172.21.15.179:0/2508470809 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800bd70 con 0x7f68700ce0e0 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/2508470809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700ce0e0 msgr2=0x7f6870160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700ce0e0 0x7f6870160ba0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f687008b1c0 tx=0x7f686800b330 comp rx=0 tx=0).stop 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/2508470809 shutdown_connections 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700ce0e0 0x7f6870160ba0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- 172.21.15.179:0/2508470809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68700d1d20 0x7f68700cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- 172.21.15.179:0/2508470809 >> v2:172.21.15.195:3300/0 conn(0x7f687015d0c0 0x7f68700b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/2508470809 >> 172.21.15.179:0/2508470809 conn(0x7f68700d2ed0 msgr2=0x7f68700d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/2508470809 shutdown_connections 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/2508470809 wait complete. 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 Processor -- start 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- start start 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68700ce0e0 0x7f687015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f687015d0c0 0x7f6870158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687015ae60 con 0x7f68700d1d20 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687015b140 con 0x7f68700ce0e0 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687015b420 con 0x7f687015d0c0 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54038/0 (socket says 172.21.15.179:54038) 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 -- 172.21.15.179:0/585846748 learned_addr learned my addr 172.21.15.179:0/585846748 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:30.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f686ffff700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68700ce0e0 0x7f687015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6874a43700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f687015d0c0 0x7f6870158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 -- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f687015d0c0 msgr2=0x7f6870158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f687015d0c0 0x7f6870158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 -- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68700ce0e0 msgr2=0x7f687015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68700ce0e0 0x7f687015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 -- 172.21.15.179:0/585846748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868009b30 con 0x7f68700d1d20 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f68677fe700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7c0 tx=0x7f685800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858011800 con 0x7f68700d1d20 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858011960 con 0x7f68700d1d20 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858011cd0 con 0x7f68700d1d20 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.244+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687015b700 con 0x7f68700d1d20 2024-05-08T22:38:30.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687015c440 con 0x7f68700d1d20 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687015bc20 con 0x7f68700d1d20 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6858021350 con 0x7f68700d1d20 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f686d7fa700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6858084090 con 0x7f68700d1d20 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f686ffff700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.248+0000 7f686ffff700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 0x7f6850068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6860006fd0 tx=0x7f6860008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.256+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f687015bc20 con 0x7f68700d1d20 2024-05-08T22:38:30.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.420+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f687015bc20 con 0x7f68700d1d20 2024-05-08T22:38:30.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.420+0000 7f686d7fa700 1 -- 172.21.15.179:0/585846748 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f687015bc20 con 0x7f68700d1d20 2024-05-08T22:38:30.424 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:38:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 msgr2=0x7f6850068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 0x7f6850068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6860006fd0 tx=0x7f6860008040 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 msgr2=0x7f68700ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7c0 tx=0x7f685800cad0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 shutdown_connections 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f687015d0c0 0x7f6870158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68700d1d20 0x7f68700ccbc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 --2- 172.21.15.179:0/585846748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68700ce0e0 0x7f687015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 >> 172.21.15.179:0/585846748 conn(0x7f68700d2ed0 msgr2=0x7f68700ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 shutdown_connections 2024-05-08T22:38:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.424+0000 7f6875244700 1 -- 172.21.15.179:0/585846748 wait complete. 2024-05-08T22:38:30.439 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:38:30.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 Processor -- start 2024-05-08T22:38:30.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 -- start start 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 0x7f67000cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670015f1a0 con 0x7f67000cf1b0 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6700070100 con 0x7f67001439d0 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f6705fe2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000703e0 con 0x7f670015f630 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f67057e1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f67000cf1b0 0x7f670006b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56520/0 (socket says 172.21.15.179:56520) 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f67057e1700 1 -- 172.21.15.179:0/197705301 learned_addr learned my addr 172.21.15.179:0/197705301 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66fffff700 1 --2- 172.21.15.179:0/197705301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 0x7f67000cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514642791 0 0) 0x7f670015f1a0 con 0x7f67000cf1b0 2024-05-08T22:38:30.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f00035d0 con 0x7f67000cf1b0 2024-05-08T22:38:30.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283717838 0 0) 0x7f6700070100 con 0x7f67001439d0 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66fffff700 1 -- 172.21.15.179:0/197705301 >> v1:172.21.15.195:6789/0 conn(0x7f67001439d0 legacy=0x7f670006f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66fffff700 1 -- 172.21.15.179:0/197705301 >> v1:172.21.15.179:6789/0 conn(0x7f67000cf1b0 legacy=0x7f670006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66fffff700 1 -- 172.21.15.179:0/197705301 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670015eb50 con 0x7f670015f630 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66fffff700 1 --2- 172.21.15.179:0/197705301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 0x7f67000cec70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009880 tx=0x7f66ec009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625b1a910cf1dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011830 con 0x7f670015f630 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec011990 con 0x7f670015f630 2024-05-08T22:38:30.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.552+0000 7f66ff7fe700 1 -- 172.21.15.179:0/197705301 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec011d20 con 0x7f670015f630 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- 172.21.15.179:0/197705301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 msgr2=0x7f67000cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 --2- 172.21.15.179:0/197705301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 0x7f67000cec70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009880 tx=0x7f66ec009b90 comp rx=0 tx=0).stop 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- 172.21.15.179:0/197705301 shutdown_connections 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 --2- 172.21.15.179:0/197705301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f670015f630 0x7f67000cec70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- 172.21.15.179:0/197705301 >> 172.21.15.179:0/197705301 conn(0x7f67000cfb50 msgr2=0x7f67000cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- 172.21.15.179:0/197705301 shutdown_connections 2024-05-08T22:38:30.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- 172.21.15.179:0/197705301 wait complete. 2024-05-08T22:38:30.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 Processor -- start 2024-05-08T22:38:30.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.556+0000 7f6705fe2700 1 -- start start 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67001439d0 0x7f67001384f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f670015f630 0x7f670015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700159ce0 con 0x7f67000cf1b0 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700134f00 con 0x7f670015f630 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6700138a30 con 0x7f67001439d0 2024-05-08T22:38:30.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54046/0 (socket says 172.21.15.179:54046) 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 -- 172.21.15.179:0/1801541119 learned_addr learned my addr 172.21.15.179:0/1801541119 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f66fffff700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67001439d0 0x7f67001384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f67057e1700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f670015f630 0x7f670015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 -- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67001439d0 msgr2=0x7f67001384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67001439d0 0x7f67001384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 -- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f670015f630 msgr2=0x7f670015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f670015f630 0x7f670015bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 -- 172.21.15.179:0/1801541119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec0094b0 con 0x7f67000cf1b0 2024-05-08T22:38:30.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6704fe0700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f66f400c750 tx=0x7f66f400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40054e0 con 0x7f67000cf1b0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f40036a0 con 0x7f67000cf1b0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4003a50 con 0x7f67000cf1b0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670015a140 con 0x7f67000cf1b0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6700135430 con 0x7f67000cf1b0 2024-05-08T22:38:30.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.560+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670006e9d0 con 0x7f67000cf1b0 2024-05-08T22:38:30.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.564+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66f4003bb0 con 0x7f67000cf1b0 2024-05-08T22:38:30.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.564+0000 7f66fd7fa700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 0x7f66e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.564+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f66f4083890 con 0x7f67000cf1b0 2024-05-08T22:38:30.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.564+0000 7f66fffff700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 0x7f66e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.564+0000 7f66fffff700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 0x7f66e4068bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0090b0 tx=0x7f66ec01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.568+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f66f4050930 con 0x7f67000cf1b0 2024-05-08T22:38:30.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.752+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f670004e700 con 0x7f67000cf1b0 2024-05-08T22:38:30.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.752+0000 7f66fd7fa700 1 -- 172.21.15.179:0/1801541119 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) v1 ==== 80+0+122 (secure 0 0 0) 0x7f66f4003e60 con 0x7f67000cf1b0 2024-05-08T22:38:30.757 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6837/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:38:30.757 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:38:30.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 msgr2=0x7f66e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 0x7f66e4068bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0090b0 tx=0x7f66ec01c040 comp rx=0 tx=0).stop 2024-05-08T22:38:30.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 msgr2=0x7f67001597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f66f400c750 tx=0x7f66f400ca60 comp rx=0 tx=0).stop 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 shutdown_connections 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f66e4066730 0x7f66e4068bf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f670015f630 0x7f670015bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67001439d0 0x7f67001384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 --2- 172.21.15.179:0/1801541119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67000cf1b0 0x7f67001597a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 >> 172.21.15.179:0/1801541119 conn(0x7f67000cfb50 msgr2=0x7f670013de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 shutdown_connections 2024-05-08T22:38:30.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.756+0000 7f6705fe2700 1 -- 172.21.15.179:0/1801541119 wait complete. 2024-05-08T22:38:30.761 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:38:30.771 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:38:30.771 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:30.779 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:30.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 Processor -- start 2024-05-08T22:38:30.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 -- start start 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fec440b19e0 0x7fec440b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fec440b3550 0x7fec4415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440b3980 con 0x7fec441539c0 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec44159b60 con 0x7fec440b19e0 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec4359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec44159e40 con 0x7fec440b3550 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fec440b19e0 0x7fec440b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fec440b19e0 0x7fec440b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54052/0 (socket says 172.21.15.179:54052) 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 -- 172.21.15.179:0/3708686764 learned_addr learned my addr 172.21.15.179:0/3708686764 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 -- 172.21.15.179:0/3708686764 >> v2:172.21.15.195:3300/0 conn(0x7fec440b3550 msgr2=0x7fec4415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec42d9d700 1 --2- 172.21.15.179:0/3708686764 >> v2:172.21.15.195:3300/0 conn(0x7fec440b3550 0x7fec4415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 --2- 172.21.15.179:0/3708686764 >> v2:172.21.15.195:3300/0 conn(0x7fec440b3550 0x7fec4415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 -- 172.21.15.179:0/3708686764 >> v1:172.21.15.195:6789/0 conn(0x7fec441539c0 legacy=0x7fec4406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.884+0000 7fec41d9b700 1 -- 172.21.15.179:0/3708686764 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4415a120 con 0x7fec440b19e0 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec41d9b700 1 --2- 172.21.15.179:0/3708686764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec440b3010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec440708c0 tx=0x7fec2c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a23d0d72254d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4159a700 1 -- 172.21.15.179:0/3708686764 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c010070 con 0x7fec440b19e0 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4159a700 1 -- 172.21.15.179:0/3708686764 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00dd20 con 0x7fec440b19e0 2024-05-08T22:38:30.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4159a700 1 -- 172.21.15.179:0/3708686764 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00b310 con 0x7fec440b19e0 2024-05-08T22:38:30.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 -- 172.21.15.179:0/3708686764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 msgr2=0x7fec440b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 --2- 172.21.15.179:0/3708686764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec440b3010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec440708c0 tx=0x7fec2c00d430 comp rx=0 tx=0).stop 2024-05-08T22:38:30.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 -- 172.21.15.179:0/3708686764 shutdown_connections 2024-05-08T22:38:30.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 --2- 172.21.15.179:0/3708686764 >> v2:172.21.15.195:3300/0 conn(0x7fec440b3550 0x7fec4415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 --2- 172.21.15.179:0/3708686764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec440b3010 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 -- 172.21.15.179:0/3708686764 >> 172.21.15.179:0/3708686764 conn(0x7fec440ce9d0 msgr2=0x7fec440cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:30.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 -- 172.21.15.179:0/3708686764 shutdown_connections 2024-05-08T22:38:30.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.888+0000 7fec4359e700 1 -- 172.21.15.179:0/3708686764 wait complete. 2024-05-08T22:38:30.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 Processor -- start 2024-05-08T22:38:30.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- start start 2024-05-08T22:38:30.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec440b3550 0x7fec44151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec441539c0 0x7fec44158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4406cea0 con 0x7fec440b19e0 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44152a00 con 0x7fec441539c0 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec44152ce0 con 0x7fec440b3550 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54062/0 (socket says 172.21.15.179:54062) 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 -- 172.21.15.179:0/623094004 learned_addr learned my addr 172.21.15.179:0/623094004 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec41d9b700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec440b3550 0x7fec44151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec42d9d700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec441539c0 0x7fec44158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 -- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec440b3550 msgr2=0x7fec44151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec440b3550 0x7fec44151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 -- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec441539c0 msgr2=0x7fec44158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec441539c0 0x7fec44158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 -- 172.21.15.179:0/623094004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c00bd40 con 0x7fec440b19e0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4259c700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c4e0 tx=0x7fec3c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011790 con 0x7fec440b19e0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c011910 con 0x7fec440b19e0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011cc0 con 0x7fec440b19e0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4406d0f0 con 0x7fec440b19e0 2024-05-08T22:38:30.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.892+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec44148df0 con 0x7fec440b19e0 2024-05-08T22:38:30.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44067740 con 0x7fec440b19e0 2024-05-08T22:38:30.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec3c01c460 con 0x7fec440b19e0 2024-05-08T22:38:30.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec3affd700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 0x7fec28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:30.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fec3c0841b0 con 0x7fec440b19e0 2024-05-08T22:38:30.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec41d9b700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 0x7fec28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:30.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.896+0000 7fec41d9b700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 0x7fec28068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec440708c0 tx=0x7fec2c00b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:30.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:30.900+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec3c051370 con 0x7fec440b19e0 2024-05-08T22:38:31.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.072+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec4406d5b0 con 0x7fec440b19e0 2024-05-08T22:38:31.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec3affd700 1 -- 172.21.15.179:0/623094004 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7fec3c054990 con 0x7fec440b19e0 2024-05-08T22:38:31.079 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:38:31.079 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:31.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 msgr2=0x7fec28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 0x7fec28068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec440708c0 tx=0x7fec2c00b000 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 msgr2=0x7fec441488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c4e0 tx=0x7fec3c00c8a0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 shutdown_connections 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec28066730 0x7fec28068bf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec441539c0 0x7fec44158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec440b3550 0x7fec44151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 --2- 172.21.15.179:0/623094004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec440b19e0 0x7fec441488b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 >> 172.21.15.179:0/623094004 conn(0x7fec440ce9d0 msgr2=0x7fec440cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 shutdown_connections 2024-05-08T22:38:31.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.076+0000 7fec4359e700 1 -- 172.21.15.179:0/623094004 wait complete. 2024-05-08T22:38:31.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:31.095 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:31.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 Processor -- start 2024-05-08T22:38:31.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- start start 2024-05-08T22:38:31.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feca0161ed0 0x7feca006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feca006d2a0 0x7feca01622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca006b9a0 con 0x7feca01627f0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca006def0 con 0x7feca006d2a0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca006bc80 con 0x7feca0161ed0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca494f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7feca01627f0 0x7feca00abfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56526/0 (socket says 172.21.15.179:56526) 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feca006d2a0 0x7feca01622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca494f700 1 -- 172.21.15.179:0/4158339955 learned_addr learned my addr 172.21.15.179:0/4158339955 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9ffff700 1 --2- 172.21.15.179:0/4158339955 >> v2:172.21.15.195:3300/0 conn(0x7feca0161ed0 0x7feca006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9effd700 1 -- 172.21.15.179:0/4158339955 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927488064 0 0) 0x7feca006b9a0 con 0x7feca01627f0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9effd700 1 -- 172.21.15.179:0/4158339955 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec880035d0 con 0x7feca01627f0 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 -- 172.21.15.179:0/4158339955 >> v2:172.21.15.195:3300/0 conn(0x7feca0161ed0 msgr2=0x7feca006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 --2- 172.21.15.179:0/4158339955 >> v2:172.21.15.195:3300/0 conn(0x7feca0161ed0 0x7feca006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 -- 172.21.15.179:0/4158339955 >> v1:172.21.15.179:6789/0 conn(0x7feca01627f0 legacy=0x7feca00abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 -- 172.21.15.179:0/4158339955 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca006bf60 con 0x7feca006d2a0 2024-05-08T22:38:31.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9f7fe700 1 --2- 172.21.15.179:0/4158339955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca006d2a0 0x7feca01622b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feca00729b0 tx=0x7fec9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1501bcddf626895f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9effd700 1 -- 172.21.15.179:0/4158339955 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000e070 con 0x7feca006d2a0 2024-05-08T22:38:31.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9effd700 1 -- 172.21.15.179:0/4158339955 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9000bb20 con 0x7feca006d2a0 2024-05-08T22:38:31.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7fec9effd700 1 -- 172.21.15.179:0/4158339955 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90012430 con 0x7feca006d2a0 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- 172.21.15.179:0/4158339955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca006d2a0 msgr2=0x7feca01622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 --2- 172.21.15.179:0/4158339955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca006d2a0 0x7feca01622b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feca00729b0 tx=0x7fec9000b330 comp rx=0 tx=0).stop 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- 172.21.15.179:0/4158339955 shutdown_connections 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 --2- 172.21.15.179:0/4158339955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca006d2a0 0x7feca01622b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 --2- 172.21.15.179:0/4158339955 >> v2:172.21.15.195:3300/0 conn(0x7feca0161ed0 0x7feca006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- 172.21.15.179:0/4158339955 >> 172.21.15.179:0/4158339955 conn(0x7feca00ceca0 msgr2=0x7feca00cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.208+0000 7feca63b2700 1 -- 172.21.15.179:0/4158339955 shutdown_connections 2024-05-08T22:38:31.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- 172.21.15.179:0/4158339955 wait complete. 2024-05-08T22:38:31.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 Processor -- start 2024-05-08T22:38:31.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- start start 2024-05-08T22:38:31.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 0x7feca0164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feca0161ed0 0x7feca01fa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca0164e80 con 0x7feca01627f0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca015df80 con 0x7feca006d2a0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca015e220 con 0x7feca0161ed0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54088/0 (socket says 172.21.15.179:54088) 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 -- 172.21.15.179:0/2539606492 learned_addr learned my addr 172.21.15.179:0/2539606492 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7fec9f7fe700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feca0161ed0 0x7feca01fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7fec9ffff700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 0x7feca0164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 -- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feca0161ed0 msgr2=0x7feca01fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feca0161ed0 0x7feca01fa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 -- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 msgr2=0x7feca0164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 0x7feca0164940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 -- 172.21.15.179:0/2539606492 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009a70 con 0x7feca01627f0 2024-05-08T22:38:31.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7fec9ffff700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 0x7feca0164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca494f700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fec9800c4f0 tx=0x7fec9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98005d90 con 0x7feca01627f0 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.212+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca015e500 con 0x7feca01627f0 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec980070d0 con 0x7feca01627f0 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98007460 con 0x7feca01627f0 2024-05-08T22:38:31.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca015eac0 con 0x7feca01627f0 2024-05-08T22:38:31.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca00afe30 con 0x7feca01627f0 2024-05-08T22:38:31.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec980075c0 con 0x7feca01627f0 2024-05-08T22:38:31.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9cff9700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 0x7fec8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fec98083910 con 0x7feca01627f0 2024-05-08T22:38:31.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.216+0000 7fec9ffff700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 0x7fec8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.220+0000 7fec9ffff700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 0x7fec8c068bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fec94009c40 tx=0x7fec94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.224+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec98050a50 con 0x7feca01627f0 2024-05-08T22:38:31.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.392+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feca0073290 con 0x7feca01627f0 2024-05-08T22:38:31.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.392+0000 7fec9cff9700 1 -- 172.21.15.179:0/2539606492 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec98054070 con 0x7feca01627f0 2024-05-08T22:38:31.397 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:38:31.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 msgr2=0x7fec8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 0x7fec8c068bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fec94009c40 tx=0x7fec94008040 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 msgr2=0x7feca00b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fec9800c4f0 tx=0x7fec9800c8b0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 shutdown_connections 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fec8c066730 0x7fec8c068bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feca01627f0 0x7feca00b1aa0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feca0161ed0 0x7feca01fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 --2- 172.21.15.179:0/2539606492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feca006d2a0 0x7feca0164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 >> 172.21.15.179:0/2539606492 conn(0x7feca00ceca0 msgr2=0x7feca00af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 shutdown_connections 2024-05-08T22:38:31.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.396+0000 7feca63b2700 1 -- 172.21.15.179:0/2539606492 wait complete. 2024-05-08T22:38:31.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:31.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.520+0000 7f1166ee0700 1 Processor -- start 2024-05-08T22:38:31.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.520+0000 7f1166ee0700 1 -- start start 2024-05-08T22:38:31.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1160142fd0 0x7f116006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1160143520 0x7f116006d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11600b22d0 con 0x7f1160142890 2024-05-08T22:38:31.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600b19e0 con 0x7f1160142fd0 2024-05-08T22:38:31.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116015e640 con 0x7f1160143520 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1160142fd0 0x7f116006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1160142fd0 0x7f116006cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54100/0 (socket says 172.21.15.179:54100) 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 -- 172.21.15.179:0/578742081 learned_addr learned my addr 172.21.15.179:0/578742081 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11666df700 1 --2- 172.21.15.179:0/578742081 >> v2:172.21.15.195:3300/0 conn(0x7f1160143520 0x7f116006d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 -- 172.21.15.179:0/578742081 >> v2:172.21.15.195:3300/0 conn(0x7f1160143520 msgr2=0x7f116006d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 --2- 172.21.15.179:0/578742081 >> v2:172.21.15.195:3300/0 conn(0x7f1160143520 0x7f116006d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 -- 172.21.15.179:0/578742081 >> v1:172.21.15.195:6789/0 conn(0x7f1160142890 legacy=0x7f11600b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 -- 172.21.15.179:0/578742081 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600b1c80 con 0x7f1160142fd0 2024-05-08T22:38:31.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f11656dd700 1 --2- 172.21.15.179:0/578742081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160142fd0 0x7f116006cda0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f116015a260 tx=0x7f115c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95ccc8b71824455 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1164edc700 1 -- 172.21.15.179:0/578742081 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00e070 con 0x7f1160142fd0 2024-05-08T22:38:31.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1164edc700 1 -- 172.21.15.179:0/578742081 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c00b910 con 0x7f1160142fd0 2024-05-08T22:38:31.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1164edc700 1 -- 172.21.15.179:0/578742081 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00bcc0 con 0x7f1160142fd0 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- 172.21.15.179:0/578742081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160142fd0 msgr2=0x7f116006cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 --2- 172.21.15.179:0/578742081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160142fd0 0x7f116006cda0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f116015a260 tx=0x7f115c00b330 comp rx=0 tx=0).stop 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- 172.21.15.179:0/578742081 shutdown_connections 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 --2- 172.21.15.179:0/578742081 >> v2:172.21.15.195:3300/0 conn(0x7f1160143520 0x7f116006d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 --2- 172.21.15.179:0/578742081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160142fd0 0x7f116006cda0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- 172.21.15.179:0/578742081 >> 172.21.15.179:0/578742081 conn(0x7f11600cea30 msgr2=0x7f11600cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- 172.21.15.179:0/578742081 shutdown_connections 2024-05-08T22:38:31.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.524+0000 7f1166ee0700 1 -- 172.21.15.179:0/578742081 wait complete. 2024-05-08T22:38:31.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 Processor -- start 2024-05-08T22:38:31.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 -- start start 2024-05-08T22:38:31.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1160142fd0 0x7f11601574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160143520 0x7f1160134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160142030 con 0x7f1160143520 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600ace80 con 0x7f1160142890 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1166ee0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600ad590 con 0x7f1160142fd0 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f11666df700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160143520 0x7f1160134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f11656dd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1160142fd0 0x7f11601574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:47458/0 (socket says 172.21.15.179:47458) 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 -- 172.21.15.179:0/2202836761 learned_addr learned my addr 172.21.15.179:0/2202836761 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 -- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1160142fd0 msgr2=0x7f11601574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1160142fd0 0x7f11601574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 -- 172.21.15.179:0/2202836761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160143520 msgr2=0x7f1160134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160143520 0x7f1160134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 -- 172.21.15.179:0/2202836761 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c009b30 con 0x7f1160142890 2024-05-08T22:38:31.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.528+0000 7f1165ede700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f115000c500 tx=0x7f115000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500052d0 con 0x7f1160142890 2024-05-08T22:38:31.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150005430 con 0x7f1160142890 2024-05-08T22:38:31.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500055a0 con 0x7f1160142890 2024-05-08T22:38:31.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1160141a70 con 0x7f1160142890 2024-05-08T22:38:31.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11600ac420 con 0x7f1160142890 2024-05-08T22:38:31.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11600b2df0 con 0x7f1160142890 2024-05-08T22:38:31.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11500036a0 con 0x7f1160142890 2024-05-08T22:38:31.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.532+0000 7f11567fc700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 0x7f114c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.536+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1150083960 con 0x7f1160142890 2024-05-08T22:38:31.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.536+0000 7f11656dd700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 0x7f114c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.536+0000 7f11656dd700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 0x7f114c068c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f115c000f80 tx=0x7f115c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.540+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1150050bc0 con 0x7f1160142890 2024-05-08T22:38:31.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.712+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f116004e700 con 0x7f1160142890 2024-05-08T22:38:31.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.712+0000 7f11567fc700 1 -- 172.21.15.179:0/2202836761 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11500541e0 con 0x7f1160142890 2024-05-08T22:38:31.717 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:38:31.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 msgr2=0x7f114c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 0x7f114c068c20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f115c000f80 tx=0x7f115c009000 comp rx=0 tx=0).stop 2024-05-08T22:38:31.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 msgr2=0x7f11600ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f115000c500 tx=0x7f115000c810 comp rx=0 tx=0).stop 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 shutdown_connections 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f114c066760 0x7f114c068c20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1160143520 0x7f1160134d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1160142fd0 0x7f11601574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 --2- 172.21.15.179:0/2202836761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1160142890 0x7f11600ac940 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 >> 172.21.15.179:0/2202836761 conn(0x7f11600cea30 msgr2=0x7f116006ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 shutdown_connections 2024-05-08T22:38:31.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.716+0000 7f1166ee0700 1 -- 172.21.15.179:0/2202836761 wait complete. 2024-05-08T22:38:31.737 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:31.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 Processor -- start 2024-05-08T22:38:31.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 -- start start 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540cd070 con 0x7f20540cffc0 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540b3d10 con 0x7f205415d000 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.856+0000 7f205971a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b3b00 con 0x7f20540b47f0 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f20537fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f20540cffc0 0x7f20540cb320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56530/0 (socket says 172.21.15.179:56530) 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f20537fe700 1 -- 172.21.15.179:0/397972576 learned_addr learned my addr 172.21.15.179:0/397972576 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:31.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2058f19700 1 --2- 172.21.15.179:0/397972576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493273018 0 0) 0x7f20540cd070 con 0x7f20540cffc0 2024-05-08T22:38:31.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c0035d0 con 0x7f20540cffc0 2024-05-08T22:38:31.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596583030 0 0) 0x7f20540b3d10 con 0x7f205415d000 2024-05-08T22:38:31.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20540cd070 con 0x7f205415d000 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2058f19700 1 -- 172.21.15.179:0/397972576 >> v1:172.21.15.195:6789/0 conn(0x7f205415d000 legacy=0x7f20540cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2058f19700 1 -- 172.21.15.179:0/397972576 >> v1:172.21.15.179:6789/0 conn(0x7f20540cffc0 legacy=0x7f20540cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2058f19700 1 -- 172.21.15.179:0/397972576 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540b3ff0 con 0x7f20540b47f0 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2058f19700 1 --2- 172.21.15.179:0/397972576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540cba30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f204c009dc0 tx=0x7f204c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d816834bfd91bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c00e070 con 0x7f20540b47f0 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00bb20 con 0x7f20540b47f0 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f2052ffd700 1 -- 172.21.15.179:0/397972576 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c00be90 con 0x7f20540b47f0 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 -- 172.21.15.179:0/397972576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 msgr2=0x7f20540cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 --2- 172.21.15.179:0/397972576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540cba30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f204c009dc0 tx=0x7f204c00b330 comp rx=0 tx=0).stop 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 -- 172.21.15.179:0/397972576 shutdown_connections 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 --2- 172.21.15.179:0/397972576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540cba30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 -- 172.21.15.179:0/397972576 >> 172.21.15.179:0/397972576 conn(0x7f20540b27c0 msgr2=0x7f20540b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 -- 172.21.15.179:0/397972576 shutdown_connections 2024-05-08T22:38:31.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.860+0000 7f205971a700 1 -- 172.21.15.179:0/397972576 wait complete. 2024-05-08T22:38:31.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 Processor -- start 2024-05-08T22:38:31.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- start start 2024-05-08T22:38:31.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 0x7f20540c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054158d50 con 0x7f20540cffc0 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205415fbe0 con 0x7f205415d000 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20540afac0 con 0x7f20540b47f0 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54126/0 (socket says 172.21.15.179:54126) 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 -- 172.21.15.179:0/2063037211 learned_addr learned my addr 172.21.15.179:0/2063037211 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2053fff700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2058f19700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 0x7f20540c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 -- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 msgr2=0x7f20540ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 -- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 msgr2=0x7f20540c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 0x7f20540c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 -- 172.21.15.179:0/2063037211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c009a70 con 0x7f20540cffc0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2058f19700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 0x7f20540c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f20537fe700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f204800c700 tx=0x7f204800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048005d90 con 0x7f20540cffc0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480070d0 con 0x7f20540cffc0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205415f2b0 con 0x7f20540cffc0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048007440 con 0x7f20540cffc0 2024-05-08T22:38:31.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.864+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205415f870 con 0x7f20540cffc0 2024-05-08T22:38:31.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.868+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2054067740 con 0x7f20540cffc0 2024-05-08T22:38:31.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.872+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20480075a0 con 0x7f20540cffc0 2024-05-08T22:38:31.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.872+0000 7f2050ff9700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.872+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f20480839b0 con 0x7f20540cffc0 2024-05-08T22:38:31.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.872+0000 7f2053fff700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.872+0000 7f2053fff700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 0x7f2040068bf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f20440011e0 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:31.876+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2048050d10 con 0x7f20540cffc0 2024-05-08T22:38:31.945 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:38:31.940+0000 7fce8f998700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T22:38:32.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.048+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f205415fd50 con 0x7f20540cffc0 2024-05-08T22:38:32.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.048+0000 7f2050ff9700 1 -- 172.21.15.179:0/2063037211 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20480210a0 con 0x7f20540cffc0 2024-05-08T22:38:32.053 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:38:32.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 msgr2=0x7f2040068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 0x7f2040068bf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f20440011e0 tx=0x7f2044009040 comp rx=0 tx=0).stop 2024-05-08T22:38:32.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 msgr2=0x7f20540af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f204800c700 tx=0x7f204800ca10 comp rx=0 tx=0).stop 2024-05-08T22:38:32.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 shutdown_connections 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f205415d000 0x7f20540c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20540cffc0 0x7f20540af0c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 --2- 172.21.15.179:0/2063037211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20540b47f0 0x7f20540ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 >> 172.21.15.179:0/2063037211 conn(0x7f20540b27c0 msgr2=0x7f20540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 shutdown_connections 2024-05-08T22:38:32.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.052+0000 7f205971a700 1 -- 172.21.15.179:0/2063037211 wait complete. 2024-05-08T22:38:32.069 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:38:32.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 Processor -- start 2024-05-08T22:38:32.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 -- start start 2024-05-08T22:38:32.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17d406dda0 0x7f17d40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17d40b25d0 0x7f17d40b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d406b540 con 0x7f17d406e180 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d406ac50 con 0x7f17d406dda0 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d88c1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d406aef0 con 0x7f17d40b25d0 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d2ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f17d406e180 0x7f17d406e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:56538/0 (socket says 172.21.15.179:56538) 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d2ffd700 1 -- 172.21.15.179:0/3410427142 learned_addr learned my addr 172.21.15.179:0/3410427142 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17d406dda0 0x7f17d40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d3fff700 1 --2- 172.21.15.179:0/3410427142 >> v2:172.21.15.195:3300/0 conn(0x7f17d40b25d0 0x7f17d40b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d27fc700 1 -- 172.21.15.179:0/3410427142 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139853899 0 0) 0x7f17d406b540 con 0x7f17d406e180 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d27fc700 1 -- 172.21.15.179:0/3410427142 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c00035d0 con 0x7f17d406e180 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 -- 172.21.15.179:0/3410427142 >> v2:172.21.15.195:3300/0 conn(0x7f17d40b25d0 msgr2=0x7f17d40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 --2- 172.21.15.179:0/3410427142 >> v2:172.21.15.195:3300/0 conn(0x7f17d40b25d0 0x7f17d40b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 -- 172.21.15.179:0/3410427142 >> v1:172.21.15.179:6789/0 conn(0x7f17d406e180 legacy=0x7f17d406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 -- 172.21.15.179:0/3410427142 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d406b1d0 con 0x7f17d406dda0 2024-05-08T22:38:32.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d37fe700 1 --2- 172.21.15.179:0/3410427142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d40cbc30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17d4070af0 tx=0x7f17bc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209c4c35e82e0cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d27fc700 1 -- 172.21.15.179:0/3410427142 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc011830 con 0x7f17d406dda0 2024-05-08T22:38:32.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d27fc700 1 -- 172.21.15.179:0/3410427142 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc011990 con 0x7f17d406dda0 2024-05-08T22:38:32.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.176+0000 7f17d27fc700 1 -- 172.21.15.179:0/3410427142 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc011d20 con 0x7f17d406dda0 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- 172.21.15.179:0/3410427142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 msgr2=0x7f17d40cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- 172.21.15.179:0/3410427142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d40cbc30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17d4070af0 tx=0x7f17bc009b30 comp rx=0 tx=0).stop 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- 172.21.15.179:0/3410427142 shutdown_connections 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- 172.21.15.179:0/3410427142 >> v2:172.21.15.195:3300/0 conn(0x7f17d40b25d0 0x7f17d40b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- 172.21.15.179:0/3410427142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d40cbc30 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- 172.21.15.179:0/3410427142 >> 172.21.15.179:0/3410427142 conn(0x7f17d40ce960 msgr2=0x7f17d40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- 172.21.15.179:0/3410427142 shutdown_connections 2024-05-08T22:38:32.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- 172.21.15.179:0/3410427142 wait complete. 2024-05-08T22:38:32.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 Processor -- start 2024-05-08T22:38:32.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- start start 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 0x7f17d413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 0x7f17d4163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d413b230 con 0x7f17d406dda0 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d413d230 con 0x7f17d406e180 2024-05-08T22:38:32.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d88c1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d414a680 con 0x7f17d40b25d0 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54136/0 (socket says 172.21.15.179:54136) 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 -- 172.21.15.179:0/2500442098 learned_addr learned my addr 172.21.15.179:0/2500442098 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d2ffd700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 0x7f17d413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d3fff700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 0x7f17d4163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 -- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 msgr2=0x7f17d4163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 0x7f17d4163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 -- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 msgr2=0x7f17d413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 0x7f17d413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d37fe700 1 -- 172.21.15.179:0/2500442098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0094b0 con 0x7f17d406dda0 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d3fff700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 0x7f17d4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.180+0000 7f17d2ffd700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 0x7f17d413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:32.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17d37fe700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f17bc014040 tx=0x7f17bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004610 con 0x7f17d406dda0 2024-05-08T22:38:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc01e070 con 0x7f17d406dda0 2024-05-08T22:38:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d414a960 con 0x7f17d406dda0 2024-05-08T22:38:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0038e0 con 0x7f17d406dda0 2024-05-08T22:38:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d414af20 con 0x7f17d406dda0 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17bc003a40 con 0x7f17d406dda0 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d413b370 con 0x7f17d406dda0 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 0x7f17b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f17bc085320 con 0x7f17d406dda0 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.184+0000 7f17d2ffd700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 0x7f17b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.188+0000 7f17d2ffd700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 0x7f17b4068bf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f17d414bb20 tx=0x7f17c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.192+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17d413b370 con 0x7f17d406dda0 2024-05-08T22:38:32.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17d413b370 con 0x7f17d406dda0 2024-05-08T22:38:32.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17cbfff700 1 -- 172.21.15.179:0/2500442098 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) v1 ==== 80+0+122 (secure 0 0 0) 0x7f17d413b370 con 0x7f17d406dda0 2024-05-08T22:38:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6837/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:38:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 msgr2=0x7f17b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 0x7f17b4068bf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f17d414bb20 tx=0x7f17c4009040 comp rx=0 tx=0).stop 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 msgr2=0x7f17d414a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f17bc014040 tx=0x7f17bc004770 comp rx=0 tx=0).stop 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 shutdown_connections 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f17b4066730 0x7f17b4068bf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17d406e180 0x7f17d413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17d40b25d0 0x7f17d4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 --2- 172.21.15.179:0/2500442098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d406dda0 0x7f17d414a0a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 >> 172.21.15.179:0/2500442098 conn(0x7f17d40ce960 msgr2=0x7f17d40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 shutdown_connections 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:32.376+0000 7f17d88c1700 1 -- 172.21.15.179:0/2500442098 wait complete. 2024-05-08T22:38:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:38:32.394 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:38:32.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:32.394 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:32.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[43578]: fuse finished with error 0 and tester_r 0 2024-05-08T22:38:32.404 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:38:38.506 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:38:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:38.506 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:38:38.514 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-08T22:38:38.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.683+0000 7fd0b3605700 1 Processor -- start 2024-05-08T22:38:38.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.683+0000 7fd0b3605700 1 -- start start 2024-05-08T22:38:38.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 0x7fd0ac0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0ac0ae140 0x7fd0ac0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac0cad20 con 0x7fd0ac15e650 2024-05-08T22:38:38.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0cd5f0 con 0x7fd0ac0ae140 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0cd030 con 0x7fd0ac0ca390 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0ac0ae140 0x7fd0ac0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b13a1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd0ac15e650 0x7fd0ac0cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60972/0 (socket says 172.21.15.179:60972) 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b13a1700 1 -- 172.21.15.179:0/3759985792 learned_addr learned my addr 172.21.15.179:0/3759985792 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/3759985792 >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 0x7fd0ac0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3759985792 >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 msgr2=0x7fd0ac0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/3759985792 >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 0x7fd0ac0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3759985792 >> v1:172.21.15.179:6789/0 conn(0x7fd0ac15e650 legacy=0x7fd0ac0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3759985792 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0cd310 con 0x7fd0ac0ae140 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/3759985792 >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 0x7fd0ac0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0a3fff700 1 -- 172.21.15.179:0/3759985792 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764798368 0 0) 0x7fd0a8001110 con 0x7fd0ac15e650 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/3759985792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0ae140 0x7fd0ac0ae540 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0cd810 tx=0x7fd0a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5dfbcd2b081260 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0a3fff700 1 -- 172.21.15.179:0/3759985792 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400e070 con 0x7fd0ac0ae140 2024-05-08T22:38:38.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0a3fff700 1 -- 172.21.15.179:0/3759985792 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400bbd0 con 0x7fd0ac0ae140 2024-05-08T22:38:38.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0a3fff700 1 -- 172.21.15.179:0/3759985792 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4012430 con 0x7fd0ac0ae140 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- 172.21.15.179:0/3759985792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0ae140 msgr2=0x7fd0ac0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3759985792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0ae140 0x7fd0ac0ae540 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0cd810 tx=0x7fd0a400b330 comp rx=0 tx=0).stop 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- 172.21.15.179:0/3759985792 shutdown_connections 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3759985792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0ae140 0x7fd0ac0ae540 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3759985792 >> v2:172.21.15.195:3300/0 conn(0x7fd0ac0ca390 0x7fd0ac0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- 172.21.15.179:0/3759985792 >> 172.21.15.179:0/3759985792 conn(0x7fd0ac0990e0 msgr2=0x7fd0ac0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- 172.21.15.179:0/3759985792 shutdown_connections 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- 172.21.15.179:0/3759985792 wait complete. 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 Processor -- start 2024-05-08T22:38:38.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.687+0000 7fd0b3605700 1 -- start start 2024-05-08T22:38:38.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 0x7fd0ac1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac0ca390 0x7fd0ac1481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac148730 con 0x7fd0ac15e650 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac152530 con 0x7fd0ac0ca390 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac152810 con 0x7fd0ac0ae140 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44578/0 (socket says 172.21.15.179:44578) 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3974367796 learned_addr learned my addr 172.21.15.179:0/3974367796 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 0x7fd0ac1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac0ca390 0x7fd0ac1481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 msgr2=0x7fd0ac1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 0x7fd0ac1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac0ca390 msgr2=0x7fd0ac1481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac0ca390 0x7fd0ac1481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/3974367796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4009a70 con 0x7fd0ac15e650 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 0x7fd0ac1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0cd8c0 tx=0x7fd0a4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0a1ffb700 1 -- 172.21.15.179:0/3974367796 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4012430 con 0x7fd0ac15e650 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0a1ffb700 1 -- 172.21.15.179:0/3974367796 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4020070 con 0x7fd0ac15e650 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac152af0 con 0x7fd0ac15e650 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0a1ffb700 1 -- 172.21.15.179:0/3974367796 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4004280 con 0x7fd0ac15e650 2024-05-08T22:38:38.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.691+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac1530b0 con 0x7fd0ac15e650 2024-05-08T22:38:38.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0a1ffb700 1 -- 172.21.15.179:0/3974367796 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0a40043e0 con 0x7fd0ac15e650 2024-05-08T22:38:38.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0a1ffb700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0a1ffb700 1 -- 172.21.15.179:0/3974367796 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd0a4022070 con 0x7fd0ac15e650 2024-05-08T22:38:38.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0b68b4740 1 Processor -- start 2024-05-08T22:38:38.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0b68b4740 1 -- start start 2024-05-08T22:38:38.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.695+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000f60 tx=0x7fd0a8007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d0a10 0x19d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1940da0 con 0x19ced10 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x193e530 con 0x19d0a10 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1939e20 con 0x19d1330 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44590/0 (socket says 172.21.15.179:44590) 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 -- 172.21.15.179:0/2090475978 learned_addr learned my addr 172.21.15.179:0/2090475978 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d0a10 0x19d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 -- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 msgr2=0x1a189f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a189f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 -- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d0a10 msgr2=0x19d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d0a10 0x19d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 -- 172.21.15.179:0/2090475978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17acbd0 con 0x19ced10 2024-05-08T22:38:38.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8001b80 tx=0x7fd0a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4825ffdbccea988d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd096ffd700 1 -- 172.21.15.179:0/2090475978 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8013780 con 0x19ced10 2024-05-08T22:38:38.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd096ffd700 1 -- 172.21.15.179:0/2090475978 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a80138e0 con 0x19ced10 2024-05-08T22:38:38.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd096ffd700 1 -- 172.21.15.179:0/2090475978 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8013c50 con 0x19ced10 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- 172.21.15.179:0/2090475978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 msgr2=0x19cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8001b80 tx=0x7fd0a8013000 comp rx=0 tx=0).stop 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- 172.21.15.179:0/2090475978 shutdown_connections 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d0a10 0x19d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a189f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/2090475978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19ced10 0x19cf0f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- 172.21.15.179:0/2090475978 >> 172.21.15.179:0/2090475978 conn(0x1761740 msgr2=0x19fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- 172.21.15.179:0/2090475978 shutdown_connections 2024-05-08T22:38:38.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.699+0000 7fd0b68b4740 1 -- 172.21.15.179:0/2090475978 wait complete. 2024-05-08T22:38:38.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- start start 2024-05-08T22:38:38.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 Processor -- start 2024-05-08T22:38:38.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x19dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d0a10 0x19dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18d8260 con 0x19d1330 2024-05-08T22:38:38.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18fd5f0 con 0x19ced10 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x17e5d10 con 0x19d0a10 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 10 client.0 _ll_register_callbacks cb 0x1942640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x19dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44606/0 (socket says 172.21.15.179:44606) 2024-05-08T22:38:38.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/1210294645 learned_addr learned my addr 172.21.15.179:0/1210294645 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d0a10 0x19dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d0a10 msgr2=0x19dbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d0a10 0x19dbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 msgr2=0x19dad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x19dad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 -- 172.21.15.179:0/1210294645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a8001800 con 0x19d1330 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400b730 tx=0x7fd0a40573c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a18a5f1c80b3ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a405bdd0 con 0x19d1330 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd0a4057580 con 0x19d1330 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x176e450 con 0x19d1330 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a4084080 con 0x19d1330 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 10 client.7509 fetch_fsmap learned FSMap version 46 2024-05-08T22:38:38.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x1900330 con 0x19d1330 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x176e450 con 0x19d1330 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd0a405ede0 con 0x19d1330 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 10 client.7509 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 10 client.7509 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 20 client.7509 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a22ba0 0x1a22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 4 client.7509 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.703+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x17b86d0 con 0x1a22ba0 2024-05-08T22:38:38.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a22ba0 0x1a22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a22ba0 0x1a22fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fb11dc52b8f4e server_cookie=d2420e1594507ae0 in_seq=0 out_seq=0 2024-05-08T22:38:38.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd08bfff700 10 client.7509 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd08bfff700 1 -- 172.21.15.179:0/1210294645 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x17b86d0 con 0x1a22ba0 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd08bfff700 10 client.7509 handle_command_reply: tid=0 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 client.7509 shutdown 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 msgr2=0x19dc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400b730 tx=0x7fd0a40573c0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 shutdown_connections 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d1330 0x19dc450 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d0a10 0x19dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a22ba0 0x1a22fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1210294645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x19dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd08bfff700 0 client.7509 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 >> 172.21.15.179:0/1210294645 conn(0x1761740 msgr2=0x19d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 shutdown_connections 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1210294645 wait complete. 2024-05-08T22:38:38.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.707+0000 7fd0b68b4740 20 client.7509 trim_cache size 0 max 16384 2024-05-08T22:38:38.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 Processor -- start 2024-05-08T22:38:38.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 -- start start 2024-05-08T22:38:38.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x19cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d1330 0x19d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1969b90 con 0x19d0a10 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x193b3b0 con 0x19d1330 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18e8420 con 0x19ced10 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b0ba0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x19cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:36938/0 (socket says 172.21.15.179:36938) 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 -- 172.21.15.179:0/1593478840 learned_addr learned my addr 172.21.15.179:0/1593478840 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d1330 0x19d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 -- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d1330 msgr2=0x19d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d1330 0x19d3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 -- 172.21.15.179:0/1593478840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 msgr2=0x19cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x19cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 -- 172.21.15.179:0/1593478840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18e6560 con 0x19ced10 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x19cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8023040 tx=0x7fd0a8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c597d46dcfbe708 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.731+0000 7fd0977fe700 1 -- 172.21.15.179:0/1593478840 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8007b50 con 0x19ced10 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0977fe700 1 -- 172.21.15.179:0/1593478840 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800b070 con 0x19ced10 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0977fe700 1 -- 172.21.15.179:0/1593478840 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8034de0 con 0x19ced10 2024-05-08T22:38:38.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 msgr2=0x19cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8023040 tx=0x7fd0a8014000 comp rx=0 tx=0).stop 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1593478840 shutdown_connections 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19d1330 0x19d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x19cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1593478840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19ced10 0x19cd280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1593478840 >> 172.21.15.179:0/1593478840 conn(0x1761740 msgr2=0x1a1e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1593478840 shutdown_connections 2024-05-08T22:38:38.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1593478840 wait complete. 2024-05-08T22:38:38.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- start start 2024-05-08T22:38:38.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 Processor -- start 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x1a287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a7ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x19090f0 con 0x19d0a10 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18c5140 con 0x19ced10 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x190a5f0 con 0x19d1330 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b68b4740 10 client.0 _ll_register_callbacks cb 0x19389f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b13a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x1a287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b1ba2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a7ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44622/0 (socket says 172.21.15.179:44622) 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 -- 172.21.15.179:0/1617301257 learned_addr learned my addr 172.21.15.179:0/1617301257 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 -- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 msgr2=0x1a7ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a7ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 -- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 msgr2=0x1a287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x1a287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 -- 172.21.15.179:0/1617301257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c0044d0 con 0x19d0a10 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b1ba2700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a7ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd0b0ba0700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd09c003cd0 tx=0x7fd09c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514b0b8ee6c8769 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.735+0000 7fd08bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c01c070 con 0x19d0a10 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd09c009d20 con 0x19d0a10 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0a80138e0 con 0x19d0a10 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd09c017440 con 0x19d0a10 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 10 client.7515 fetch_fsmap learned FSMap version 46 2024-05-08T22:38:38.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x17acbd0 con 0x19d0a10 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a80138e0 con 0x19d0a10 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd09c0175a0 con 0x19d0a10 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 10 client.7515 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 10 client.7515 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 20 client.7515 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a7bf60 0x1a7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 4 client.7515 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x17dd450 con 0x1a7bf60 2024-05-08T22:38:38.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a7bf60 0x1a7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 10 client.7515 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b13a1700 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a7bf60 0x1a7c370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f156aaf51c9fcf3 server_cookie=eab75d262daf21e1 in_seq=0 out_seq=0 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 1 -- 172.21.15.179:0/1617301257 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x17dd450 con 0x1a7bf60 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd08bfff700 10 client.7515 handle_command_reply: tid=0 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.739+0000 7fd0b68b4740 1 client.7515 shutdown 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 msgr2=0x1a7a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd09c003cd0 tx=0x7fd09c009650 comp rx=0 tx=0).stop 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 shutdown_connections 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19d1330 0x1a7ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19d0a10 0x1a7a890 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1a7bf60 0x1a7c370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 --2- 172.21.15.179:0/1617301257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19ced10 0x1a287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd08bfff700 0 client.7515 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 >> 172.21.15.179:0/1617301257 conn(0x1761740 msgr2=0x19d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 shutdown_connections 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 1 -- 172.21.15.179:0/1617301257 wait complete. 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b68b4740 20 client.7515 trim_cache size 0 max 16384 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "ec8103ae-80f2-42ce-9fac-cea05f68428e", 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:38:38.749 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 msgr2=0x7fd090068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000f60 tx=0x7fd0a8007e90 comp rx=0 tx=0).stop 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 msgr2=0x7fd0ac1576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0cd8c0 tx=0x7fd0a4004af0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 shutdown_connections 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac0ca390 0x7fd0ac1481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15e650 0x7fd0ac1576b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 --2- 172.21.15.179:0/3974367796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac0ae140 0x7fd0ac1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 >> 172.21.15.179:0/3974367796 conn(0x7fd0ac0990e0 msgr2=0x7fd0ac0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 shutdown_connections 2024-05-08T22:38:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.743+0000 7fd0b3605700 1 -- 172.21.15.179:0/3974367796 wait complete. 2024-05-08T22:38:38.766 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T22:38:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 Processor -- start 2024-05-08T22:38:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- start start 2024-05-08T22:38:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb41c06c2d0 0x7fb41c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c15e050 con 0x7fb41c143580 2024-05-08T22:38:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c15b2a0 con 0x7fb41c15ae10 2024-05-08T22:38:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c06f0a0 con 0x7fb41c06c2d0 2024-05-08T22:38:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb41c143580 0x7fb41c06bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60978/0 (socket says 172.21.15.179:60978) 2024-05-08T22:38:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41b7fe700 1 -- 172.21.15.179:0/3492214779 learned_addr learned my addr 172.21.15.179:0/3492214779 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:38:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb413fff700 1 --2- 172.21.15.179:0/3492214779 >> v2:172.21.15.195:3300/0 conn(0x7fb41c06c2d0 0x7fb41c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41affd700 1 -- 172.21.15.179:0/3492214779 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562457428 0 0) 0x7fb41c15e050 con 0x7fb41c143580 2024-05-08T22:38:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41affd700 1 -- 172.21.15.179:0/3492214779 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4080035d0 con 0x7fb41c143580 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb413fff700 1 -- 172.21.15.179:0/3492214779 >> v1:172.21.15.195:6789/0 conn(0x7fb41c15ae10 legacy=0x7fb41c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb413fff700 1 -- 172.21.15.179:0/3492214779 >> v1:172.21.15.179:6789/0 conn(0x7fb41c143580 legacy=0x7fb41c06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb413fff700 1 -- 172.21.15.179:0/3492214779 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c06edc0 con 0x7fb41c06c2d0 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb413fff700 1 --2- 172.21.15.179:0/3492214779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c06c2d0 0x7fb41c15e4e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb404009850 tx=0x7fb404009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5edc7afa3ed3e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41affd700 1 -- 172.21.15.179:0/3492214779 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011830 con 0x7fb41c06c2d0 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41affd700 1 -- 172.21.15.179:0/3492214779 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404011990 con 0x7fb41c06c2d0 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb41affd700 1 -- 172.21.15.179:0/3492214779 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011d00 con 0x7fb41c06c2d0 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- 172.21.15.179:0/3492214779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c06c2d0 msgr2=0x7fb41c15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 --2- 172.21.15.179:0/3492214779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c06c2d0 0x7fb41c15e4e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb404009850 tx=0x7fb404009b60 comp rx=0 tx=0).stop 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- 172.21.15.179:0/3492214779 shutdown_connections 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 --2- 172.21.15.179:0/3492214779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c06c2d0 0x7fb41c15e4e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- 172.21.15.179:0/3492214779 >> 172.21.15.179:0/3492214779 conn(0x7fb41c070d80 msgr2=0x7fb41c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- 172.21.15.179:0/3492214779 shutdown_connections 2024-05-08T22:38:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.883+0000 7fb420f03700 1 -- 172.21.15.179:0/3492214779 wait complete. 2024-05-08T22:38:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 Processor -- start 2024-05-08T22:38:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 -- start start 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb41c06c2d0 0x7fb41c140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c143580 0x7fb41c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c140eb0 con 0x7fb41c15ae10 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0c8450 con 0x7fb41c143580 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb420f03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0a7570 con 0x7fb41c06c2d0 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44634/0 (socket says 172.21.15.179:44634) 2024-05-08T22:38:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 -- 172.21.15.179:0/2134031368 learned_addr learned my addr 172.21.15.179:0/2134031368 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb41c06c2d0 0x7fb41c140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb413fff700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c143580 0x7fb41c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 -- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb41c06c2d0 msgr2=0x7fb41c140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb41c06c2d0 0x7fb41c140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 -- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c143580 msgr2=0x7fb41c0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c143580 0x7fb41c0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 -- 172.21.15.179:0/2134031368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040094b0 con 0x7fb41c15ae10 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.887+0000 7fb41bfff700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cb50 tx=0x7fb41400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb418ff9700 1 -- 172.21.15.179:0/2134031368 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414003c20 con 0x7fb41c15ae10 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb418ff9700 1 -- 172.21.15.179:0/2134031368 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414003d80 con 0x7fb41c15ae10 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0a7850 con 0x7fb41c15ae10 2024-05-08T22:38:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb418ff9700 1 -- 172.21.15.179:0/2134031368 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400aa20 con 0x7fb41c15ae10 2024-05-08T22:38:38.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c0a7e10 con 0x7fb41c15ae10 2024-05-08T22:38:38.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.891+0000 7fb418ff9700 1 -- 172.21.15.179:0/2134031368 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb41400ab80 con 0x7fb41c15ae10 2024-05-08T22:38:38.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb418ff9700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 0x7fb3fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb418ff9700 1 -- 172.21.15.179:0/2134031368 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb4140842d0 con 0x7fb41c15ae10 2024-05-08T22:38:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 0x7fb3fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 0x7fb3fc068bf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007fd0 tx=0x7fb40c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 Processor -- start 2024-05-08T22:38:38.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 -- start start 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 0x15f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16166f0 0x15ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x157f010 con 0x16195d0 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x150cf40 con 0x16166f0 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x155fe30 con 0x15ec840 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 0x15f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44646/0 (socket says 172.21.15.179:44646) 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb41bfff700 1 -- 172.21.15.179:0/3759471132 learned_addr learned my addr 172.21.15.179:0/3759471132 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.895+0000 7fb413fff700 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16166f0 0x15ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 -- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 msgr2=0x15f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 0x15f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 -- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16166f0 msgr2=0x15ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16166f0 0x15ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 -- 172.21.15.179:0/3759471132 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13cba40 con 0x16195d0 2024-05-08T22:38:38.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 0x15f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb414051660 tx=0x7fb41400af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fbe1a43d4921a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb4117fa700 1 -- 172.21.15.179:0/3759471132 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140840f0 con 0x16195d0 2024-05-08T22:38:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb4117fa700 1 -- 172.21.15.179:0/3759471132 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414058f90 con 0x16195d0 2024-05-08T22:38:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb4117fa700 1 -- 172.21.15.179:0/3759471132 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41405d550 con 0x16195d0 2024-05-08T22:38:38.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- 172.21.15.179:0/3759471132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 msgr2=0x15ed810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb414051660 tx=0x7fb41400af00 comp rx=0 tx=0).stop 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- 172.21.15.179:0/3759471132 shutdown_connections 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16166f0 0x15ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16195d0 0x15ed810 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- 172.21.15.179:0/3759471132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15ec840 0x15f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- 172.21.15.179:0/3759471132 >> 172.21.15.179:0/3759471132 conn(0x13805b0 msgr2=0x1618ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- 172.21.15.179:0/3759471132 shutdown_connections 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- 172.21.15.179:0/3759471132 wait complete. 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- start start 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 Processor -- start 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x161d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16195d0 0x1636a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x155b150 con 0x15ec840 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1558eb0 con 0x16195d0 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x15168e0 con 0x16166f0 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb425414740 10 client.0 _ll_register_callbacks cb 0x14f5670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44662/0 (socket says 172.21.15.179:44662) 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 -- 172.21.15.179:0/905664879 learned_addr learned my addr 172.21.15.179:0/905664879 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41bfff700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16195d0 0x1636a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb413fff700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x161d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 -- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 msgr2=0x161d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x161d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 -- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16195d0 msgr2=0x1636a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16195d0 0x1636a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.899+0000 7fb41b7fe700 1 -- 172.21.15.179:0/905664879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4080035d0 con 0x15ec840 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb40c014450 tx=0x7fb40c001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca46ba3429bdcf01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c026030 con 0x15ec840 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb40c027070 con 0x15ec840 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x138d2c0 con 0x15ec840 2024-05-08T22:38:38.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40c014b60 con 0x15ec840 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 10 client.7527 fetch_fsmap learned FSMap version 46 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x7fb41c0a7e10 con 0x15ec840 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c031020 con 0x15ec840 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb40c014cc0 con 0x15ec840 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 10 client.7527 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 10 client.7527 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 20 client.7527 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15efe70 0x15f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 4 client.7527 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x13d7540 con 0x15efe70 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb413fff700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15efe70 0x15f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb413fff700 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15efe70 0x15f0280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93766d963a7d28c server_cookie=25177cc83da51328 in_seq=0 out_seq=0 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 10 client.7527 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/905664879 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x13d7540 con 0x15efe70 2024-05-08T22:38:38.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb3fa7fc700 10 client.7527 handle_command_reply: tid=0 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 client.7527 shutdown 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.903+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 msgr2=0x161b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb40c014450 tx=0x7fb40c001150 comp rx=0 tx=0).stop 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 shutdown_connections 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15efe70 0x15f0280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x16195d0 0x1636a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb3fa7fc700 0 client.7527 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x161d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 --2- 172.21.15.179:0/905664879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x161b010 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 >> 172.21.15.179:0/905664879 conn(0x13805b0 msgr2=0x1618cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 shutdown_connections 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 1 -- 172.21.15.179:0/905664879 wait complete. 2024-05-08T22:38:38.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.907+0000 7fb425414740 20 client.7527 trim_cache size 0 max 16384 2024-05-08T22:38:38.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 Processor -- start 2024-05-08T22:38:38.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 -- start start 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 0x1647620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15efe70 0x1647b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1404b80 con 0x16166f0 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x15566c0 con 0x15ec840 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1529c00 con 0x15efe70 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44670/0 (socket says 172.21.15.179:44670) 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 -- 172.21.15.179:0/1926449347 learned_addr learned my addr 172.21.15.179:0/1926449347 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 0x1647620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb413fff700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15efe70 0x1647b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 -- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15efe70 msgr2=0x1647b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15efe70 0x1647b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 -- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 msgr2=0x1647620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 0x1647620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 -- 172.21.15.179:0/1926449347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15e6a10 con 0x16166f0 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 0x1647620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.927+0000 7fb41bfff700 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb414082630 tx=0x7fb414084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c953491bd085796 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb411ffb700 1 -- 172.21.15.179:0/1926449347 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41405d550 con 0x16166f0 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb411ffb700 1 -- 172.21.15.179:0/1926449347 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414084070 con 0x16166f0 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb411ffb700 1 -- 172.21.15.179:0/1926449347 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140822a0 con 0x16166f0 2024-05-08T22:38:38.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- 172.21.15.179:0/1926449347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 msgr2=0x15eadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb414082630 tx=0x7fb414084000 comp rx=0 tx=0).stop 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- 172.21.15.179:0/1926449347 shutdown_connections 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x16166f0 0x15eadb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15efe70 0x1647b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- 172.21.15.179:0/1926449347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15ec840 0x1647620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- 172.21.15.179:0/1926449347 >> 172.21.15.179:0/1926449347 conn(0x13805b0 msgr2=0x1614710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- 172.21.15.179:0/1926449347 shutdown_connections 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- 172.21.15.179:0/1926449347 wait complete. 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- start start 2024-05-08T22:38:38.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 Processor -- start 2024-05-08T22:38:38.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15efe70 0x164db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x164e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1580610 con 0x15ec840 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e2a50 con 0x15efe70 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x16572e0 con 0x16166f0 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb425414740 10 client.0 _ll_register_callbacks cb 0x1512d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb413fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15efe70 0x164db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44678/0 (socket says 172.21.15.179:44678) 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 -- 172.21.15.179:0/45388624 learned_addr learned my addr 172.21.15.179:0/45388624 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41bfff700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x164e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 -- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 msgr2=0x164e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x164e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 -- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15efe70 msgr2=0x164db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15efe70 0x164db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 -- 172.21.15.179:0/45388624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x13cba40 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb41b7fe700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb40c014810 tx=0x7fb40c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3073cf9744f180b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.931+0000 7fb3fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c026040 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb414058f90 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb40c032080 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40c007760 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 10 client.7533 fetch_fsmap learned FSMap version 46 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x151f290 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414058f90 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb40c014b60 con 0x15ec840 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 10 client.7533 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:38:38.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 10 client.7533 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 20 client.7533 populate_metadata read hostname 'smithi179' 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x16195d0 0x164bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 4 client.7533 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x13fc2c0 con 0x16195d0 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb413fff700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x16195d0 0x164bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 10 client.7533 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb413fff700 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x16195d0 0x164bba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b074332177c3ae5f server_cookie=a875a99d67d15a04 in_seq=0 out_seq=0 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 1 -- 172.21.15.179:0/45388624 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x13fc2c0 con 0x16195d0 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 10 client.7533 handle_command_reply: tid=0 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 client.7533 shutdown 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 msgr2=0x1698bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb40c014810 tx=0x7fb40c015000 comp rx=0 tx=0).stop 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 shutdown_connections 2024-05-08T22:38:38.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x16195d0 0x164bba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15efe70 0x164db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb3fa7fc700 0 client.7533 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x16166f0 0x164e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 --2- 172.21.15.179:0/45388624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15ec840 0x1698bd0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 >> 172.21.15.179:0/45388624 conn(0x13805b0 msgr2=0x1614fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 shutdown_connections 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 1 -- 172.21.15.179:0/45388624 wait complete. 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.935+0000 7fb425414740 20 client.7533 trim_cache size 0 max 16384 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:38:38.945 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: "ec8103ae-80f2-42ce-9fac-cea05f68428e": { 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/subdir", 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "ec8103ae-80f2-42ce-9fac-cea05f68428e", 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive" 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:38:38.946 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 msgr2=0x7fb3fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 0x7fb3fc068bf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007fd0 tx=0x7fb40c007de0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 msgr2=0x7fb41c0c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cb50 tx=0x7fb41400ce60 comp rx=0 tx=0).stop 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 shutdown_connections 2024-05-08T22:38:38.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb3fc066730 0x7fb3fc068bf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb41c143580 0x7fb41c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41c15ae10 0x7fb41c0c4440 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 --2- 172.21.15.179:0/2134031368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb41c06c2d0 0x7fb41c140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 >> 172.21.15.179:0/2134031368 conn(0x7fb41c070d80 msgr2=0x7fb41c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 shutdown_connections 2024-05-08T22:38:38.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:38:38.939+0000 7fb420f03700 1 -- 172.21.15.179:0/2134031368 wait complete. 2024-05-08T22:38:38.960 INFO:tasks.cephfs.filesystem:scrub status for tag:ec8103ae-80f2-42ce-9fac-cea05f68428e - {'path': '/subdir', 'tag': 'ec8103ae-80f2-42ce-9fac-cea05f68428e', 'options': 'recursive'} 2024-05-08T22:38:52.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:38:52.221 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:39:08.961 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T22:39:09.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 Processor -- start 2024-05-08T22:39:09.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 -- start start 2024-05-08T22:39:09.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ce40cb940 0x7f2ce40ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ce40cd490 0x7f2ce4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce406e520 con 0x7f2ce415d2b0 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce40ccb30 con 0x7f2ce40cb940 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2cea75f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce40cbeb0 con 0x7f2ce40cd490 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ce40cb940 0x7f2ce40ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce3fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2ce415d2b0 0x7f2ce406de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52466/0 (socket says 172.21.15.179:52466) 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1023374718 learned_addr learned my addr 172.21.15.179:0/1023374718 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/1023374718 >> v2:172.21.15.195:3300/0 conn(0x7f2ce40cd490 0x7f2ce4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 -- 172.21.15.179:0/1023374718 >> v2:172.21.15.195:3300/0 conn(0x7f2ce40cd490 msgr2=0x7f2ce4160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/1023374718 >> v2:172.21.15.195:3300/0 conn(0x7f2ce40cd490 0x7f2ce4160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 -- 172.21.15.179:0/1023374718 >> v1:172.21.15.179:6789/0 conn(0x7f2ce415d2b0 legacy=0x7f2ce406de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 -- 172.21.15.179:0/1023374718 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40cc190 con 0x7f2ce40cb940 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce2ffd700 1 -- 172.21.15.179:0/1023374718 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151437775 0 0) 0x7f2ce406e520 con 0x7f2ce415d2b0 2024-05-08T22:39:09.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/1023374718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce40cb940 0x7f2ce40ccf50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40cb710 tx=0x7f2ccc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52094ab877ed5d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce2ffd700 1 -- 172.21.15.179:0/1023374718 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc011830 con 0x7f2ce40cb940 2024-05-08T22:39:09.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce2ffd700 1 -- 172.21.15.179:0/1023374718 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc011990 con 0x7f2ce40cb940 2024-05-08T22:39:09.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.083+0000 7f2ce2ffd700 1 -- 172.21.15.179:0/1023374718 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc011d40 con 0x7f2ce40cb940 2024-05-08T22:39:09.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- 172.21.15.179:0/1023374718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce40cb940 msgr2=0x7f2ce40ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 --2- 172.21.15.179:0/1023374718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce40cb940 0x7f2ce40ccf50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40cb710 tx=0x7f2ccc009b30 comp rx=0 tx=0).stop 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- 172.21.15.179:0/1023374718 shutdown_connections 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 --2- 172.21.15.179:0/1023374718 >> v2:172.21.15.195:3300/0 conn(0x7f2ce40cd490 0x7f2ce4160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 --2- 172.21.15.179:0/1023374718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce40cb940 0x7f2ce40ccf50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- 172.21.15.179:0/1023374718 >> 172.21.15.179:0/1023374718 conn(0x7f2ce40d0f10 msgr2=0x7f2ce40d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- 172.21.15.179:0/1023374718 shutdown_connections 2024-05-08T22:39:09.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- 172.21.15.179:0/1023374718 wait complete. 2024-05-08T22:39:09.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 Processor -- start 2024-05-08T22:39:09.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.087+0000 7f2cea75f700 1 -- start start 2024-05-08T22:39:09.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ce40cb940 0x7f2ce40abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce40cd490 0x7f2ce40b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40ac560 con 0x7f2ce415d2b0 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40ac840 con 0x7f2ce40cd490 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40acb20 con 0x7f2ce40cb940 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ce40cb940 0x7f2ce40abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44638/0 (socket says 172.21.15.179:44638) 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 -- 172.21.15.179:0/2119586412 learned_addr learned my addr 172.21.15.179:0/2119586412 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce40cd490 0x7f2ce40b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 -- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ce40cb940 msgr2=0x7f2ce40abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ce40cb940 0x7f2ce40abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 -- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce40cd490 msgr2=0x7f2ce40b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce40cd490 0x7f2ce40b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 -- 172.21.15.179:0/2119586412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0094b0 con 0x7f2ce415d2b0 2024-05-08T22:39:09.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc007e60 tx=0x7f2cdc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce0ff9700 1 -- 172.21.15.179:0/2119586412 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00ed30 con 0x7f2ce415d2b0 2024-05-08T22:39:09.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce0ff9700 1 -- 172.21.15.179:0/2119586412 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc014420 con 0x7f2ce415d2b0 2024-05-08T22:39:09.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce40ace00 con 0x7f2ce415d2b0 2024-05-08T22:39:09.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2ce0ff9700 1 -- 172.21.15.179:0/2119586412 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0147d0 con 0x7f2ce415d2b0 2024-05-08T22:39:09.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.091+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce40ac250 con 0x7f2ce415d2b0 2024-05-08T22:39:09.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.095+0000 7f2ce0ff9700 1 -- 172.21.15.179:0/2119586412 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2cdc014930 con 0x7f2ce415d2b0 2024-05-08T22:39:09.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.095+0000 7f2ce0ff9700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 0x7f2cc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.095+0000 7f2ce0ff9700 1 -- 172.21.15.179:0/2119586412 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2cdc084910 con 0x7f2ce415d2b0 2024-05-08T22:39:09.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.095+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 0x7f2cc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.095+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 0x7f2cc4068bf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000f60 tx=0x7f2cd4007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 Processor -- start 2024-05-08T22:39:09.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- start start 2024-05-08T22:39:09.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f45d0 0x1913240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1913780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x185a010 con 0x18c7840 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x17e7f40 con 0x18f49b0 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x183ae30 con 0x18f45d0 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44642/0 (socket says 172.21.15.179:44642) 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1177804485 learned_addr learned my addr 172.21.15.179:0/1177804485 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f45d0 0x1913240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1913780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f45d0 msgr2=0x1913240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f45d0 0x1913240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 msgr2=0x1913780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1913780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1177804485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16a6a40 con 0x18c7840 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40016e0 tx=0x7f2cd4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0400edaba20dca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2cd9ffb700 1 -- 172.21.15.179:0/1177804485 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4021450 con 0x18c7840 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2cd9ffb700 1 -- 172.21.15.179:0/1177804485 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd40215b0 con 0x18c7840 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2cd9ffb700 1 -- 172.21.15.179:0/1177804485 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4021920 con 0x18c7840 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1177804485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 msgr2=0x18c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40016e0 tx=0x7f2cd4004f90 comp rx=0 tx=0).stop 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1177804485 shutdown_connections 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1913780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f45d0 0x1913240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1177804485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18c7c20 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1177804485 >> 172.21.15.179:0/1177804485 conn(0x165b5b0 msgr2=0x18f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1177804485 shutdown_connections 2024-05-08T22:39:09.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.099+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1177804485 wait complete. 2024-05-08T22:39:09.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 -- start start 2024-05-08T22:39:09.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 Processor -- start 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x18f9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x18f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1836150 con 0x18f45d0 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1833eb0 con 0x18f49b0 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x17f18e0 con 0x18c7840 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ceda0e740 10 client.0 _ll_register_callbacks cb 0x17d0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x18f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:56746/0 (socket says 172.21.15.179:56746) 2024-05-08T22:39:09.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1798621174 learned_addr learned my addr 172.21.15.179:0/1798621174 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x18f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 msgr2=0x18f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x18f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1798621174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 msgr2=0x18f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x18f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 -- 172.21.15.179:0/1798621174 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd4001450 con 0x18c7840 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x17d9380 tx=0x7f2cd4000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94838a513dd013f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2cc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3301/0 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.103+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4027070 con 0x18c7840 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mon.2 v2:172.21.15.195:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2cd4022e30 con 0x18c7840 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16682c0 con 0x18c7840 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd4021450 con 0x18c7840 2024-05-08T22:39:09.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 10 client.56480 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x17fa290 con 0x18c7840 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mon.2 v2:172.21.15.195:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16682c0 con 0x18c7840 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mon.2 v2:172.21.15.195:3301/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2cd4013600 con 0x18c7840 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 10 client.56480 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 10 client.56480 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 20 client.56480 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1911860 0x1911c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 4 client.56480 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16b2540 con 0x1911860 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1911860 0x1911c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 10 client.56480 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1911860 0x1911c70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a593381e359ebf server_cookie=487309cccf45cb86 in_seq=0 out_seq=0 2024-05-08T22:39:09.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/1798621174 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x16b2540 con 0x1911860 2024-05-08T22:39:09.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 10 client.56480 handle_command_reply: tid=0 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 client.56480 shutdown 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 msgr2=0x18c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x17d9380 tx=0x7f2cd4000ed0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 shutdown_connections 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1911860 0x1911c70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x18f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x18f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2cc2ffd700 0 client.56480 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/1798621174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x18c61b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.107+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 >> 172.21.15.179:0/1798621174 conn(0x165b5b0 msgr2=0x18c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.111+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 shutdown_connections 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.111+0000 7f2ceda0e740 1 -- 172.21.15.179:0/1798621174 wait complete. 2024-05-08T22:39:09.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.111+0000 7f2ceda0e740 20 client.56480 trim_cache size 0 max 16384 2024-05-08T22:39:09.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.127+0000 7f2ceda0e740 1 Processor -- start 2024-05-08T22:39:09.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.127+0000 7f2ceda0e740 1 -- start start 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x191b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1918260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x16dfb80 con 0x18f45d0 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18316c0 con 0x18f49b0 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x17d2620 con 0x18c7840 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44672/0 (socket says 172.21.15.179:44672) 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 -- 172.21.15.179:0/806209120 learned_addr learned my addr 172.21.15.179:0/806209120 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x191b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1918260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 -- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 msgr2=0x191b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x191b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 -- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 msgr2=0x1918260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1918260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 -- 172.21.15.179:0/806209120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17e04c0 con 0x18f45d0 2024-05-08T22:39:09.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc013000 tx=0x7f2ccc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a79a4ddca0d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2cda7fc700 1 -- 172.21.15.179:0/806209120 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc031480 con 0x18f45d0 2024-05-08T22:39:09.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2cda7fc700 1 -- 172.21.15.179:0/806209120 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02a070 con 0x18f45d0 2024-05-08T22:39:09.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2cda7fc700 1 -- 172.21.15.179:0/806209120 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc031840 con 0x18f45d0 2024-05-08T22:39:09.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- 172.21.15.179:0/806209120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 msgr2=0x191b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc013000 tx=0x7f2ccc008030 comp rx=0 tx=0).stop 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- 172.21.15.179:0/806209120 shutdown_connections 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f49b0 0x1918260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18f45d0 0x191b660 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/806209120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18c7840 0x191b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- 172.21.15.179:0/806209120 >> 172.21.15.179:0/806209120 conn(0x165b5b0 msgr2=0x18df690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- 172.21.15.179:0/806209120 shutdown_connections 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- 172.21.15.179:0/806209120 wait complete. 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 -- start start 2024-05-08T22:39:09.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.131+0000 7f2ceda0e740 1 Processor -- start 2024-05-08T22:39:09.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 0x1974560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f49b0 0x1931530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x17bda50 con 0x18c7840 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1932b20 con 0x18f45d0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x191c1e0 con 0x18f49b0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 10 client.0 _ll_register_callbacks cb 0x18cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 0x1974560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44674/0 (socket says 172.21.15.179:44674) 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 -- 172.21.15.179:0/779458320 learned_addr learned my addr 172.21.15.179:0/779458320 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce8cfc700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f49b0 0x1931530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 -- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f49b0 msgr2=0x1931530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f49b0 0x1931530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 -- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 msgr2=0x1974560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 0x1974560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 -- 172.21.15.179:0/779458320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce40cc190 con 0x18c7840 2024-05-08T22:39:09.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 0x1974560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:09.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:09.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce3fff700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4001e60 tx=0x7f2cd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66458b9326bac77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4021450 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2cd4004b30 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cd40215b0 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd402b760 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 10 client.7554 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x17d4790 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd40215b0 con 0x18c7840 2024-05-08T22:39:09.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2cd402bb10 con 0x18c7840 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 10 client.7554 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 10 client.7554 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 20 client.7554 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1931a70 0x18f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 4 client.7554 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x16d72c0 con 0x1931a70 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.135+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1931a70 0x18f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ce37fe700 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1931a70 0x18f7430 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387202dba86c24cb server_cookie=dd75cbac4b499904 in_seq=0 out_seq=0 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cc2ffd700 10 client.7554 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cc2ffd700 1 -- 172.21.15.179:0/779458320 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x16d72c0 con 0x1931a70 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cc2ffd700 10 client.7554 handle_command_reply: tid=0 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 client.7554 shutdown 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 msgr2=0x18e16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4001e60 tx=0x7f2cd4013000 comp rx=0 tx=0).stop 2024-05-08T22:39:09.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 shutdown_connections 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1931a70 0x18f7430 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18f45d0 0x1974560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18f49b0 0x1931530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 --2- 172.21.15.179:0/779458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18c7840 0x18e16d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 >> 172.21.15.179:0/779458320 conn(0x165b5b0 msgr2=0x18dffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cc2ffd700 0 client.7554 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 shutdown_connections 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 1 -- 172.21.15.179:0/779458320 wait complete. 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2ceda0e740 20 client.7554 trim_cache size 0 max 16384 2024-05-08T22:39:09.141 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:09.142 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:39:09.142 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:39:09.142 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:09.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 msgr2=0x7f2cc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 0x7f2cc4068bf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000f60 tx=0x7f2cd4007e00 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 msgr2=0x7f2ce40ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc007e60 tx=0x7f2cdc00e3c0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 shutdown_connections 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2cc4066730 0x7f2cc4068bf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce40cd490 0x7f2ce40b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce415d2b0 0x7f2ce40ae100 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 --2- 172.21.15.179:0/2119586412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ce40cb940 0x7f2ce40abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 >> 172.21.15.179:0/2119586412 conn(0x7f2ce40d0f10 msgr2=0x7f2ce406ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 shutdown_connections 2024-05-08T22:39:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.139+0000 7f2cea75f700 1 -- 172.21.15.179:0/2119586412 wait complete. 2024-05-08T22:39:09.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:39:09.156 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:09.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 Processor -- start 2024-05-08T22:39:09.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 -- start start 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fefd0161640 0x7fefd00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd00d2640 con 0x7fefd00d18a0 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd006b3a0 con 0x7fefd015d580 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6994700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd00d2360 con 0x7fefd0161640 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd5191700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fefd00d18a0 0x7fefd0160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52472/0 (socket says 172.21.15.179:52472) 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd5191700 1 -- 172.21.15.179:0/2529857724 learned_addr learned my addr 172.21.15.179:0/2529857724 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:09.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6193700 1 --2- 172.21.15.179:0/2529857724 >> v2:172.21.15.195:3300/0 conn(0x7fefd0161640 0x7fefd00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6193700 1 -- 172.21.15.179:0/2529857724 >> v1:172.21.15.195:6789/0 conn(0x7fefd015d580 legacy=0x7fefd00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6193700 1 -- 172.21.15.179:0/2529857724 >> v1:172.21.15.179:6789/0 conn(0x7fefd00d18a0 legacy=0x7fefd0160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6193700 1 -- 172.21.15.179:0/2529857724 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd006ad50 con 0x7fefd0161640 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd4990700 1 -- 172.21.15.179:0/2529857724 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676433496 0 0) 0x7fefd00d2640 con 0x7fefd00d18a0 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd6193700 1 --2- 172.21.15.179:0/2529857724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd0161640 0x7fefd00d1d30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009850 tx=0x7fefc8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c256acd288a07d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd4990700 1 -- 172.21.15.179:0/2529857724 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011830 con 0x7fefd0161640 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd4990700 1 -- 172.21.15.179:0/2529857724 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8011990 con 0x7fefd0161640 2024-05-08T22:39:09.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.275+0000 7fefd4990700 1 -- 172.21.15.179:0/2529857724 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011d20 con 0x7fefd0161640 2024-05-08T22:39:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2529857724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd0161640 msgr2=0x7fefd00d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 --2- 172.21.15.179:0/2529857724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd0161640 0x7fefd00d1d30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009850 tx=0x7fefc8009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2529857724 shutdown_connections 2024-05-08T22:39:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 --2- 172.21.15.179:0/2529857724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd0161640 0x7fefd00d1d30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2529857724 >> 172.21.15.179:0/2529857724 conn(0x7fefd00d2d90 msgr2=0x7fefd00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2529857724 shutdown_connections 2024-05-08T22:39:09.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2529857724 wait complete. 2024-05-08T22:39:09.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 Processor -- start 2024-05-08T22:39:09.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- start start 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd015d580 0x7fefd014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd0161640 0x7fefd014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd0158aa0 con 0x7fefd00d18a0 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd015de00 con 0x7fefd015d580 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd00b4b80 con 0x7fefd0161640 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44676/0 (socket says 172.21.15.179:44676) 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 -- 172.21.15.179:0/2446848890 learned_addr learned my addr 172.21.15.179:0/2446848890 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6193700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd0161640 0x7fefd014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5191700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd015d580 0x7fefd014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 -- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd0161640 msgr2=0x7fefd014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd0161640 0x7fefd014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 -- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd015d580 msgr2=0x7fefd014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd015d580 0x7fefd014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 -- 172.21.15.179:0/2446848890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80094b0 con 0x7fefd00d18a0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd5992700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fefb8001ed0 tx=0x7fefb80021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8003c20 con 0x7fefd00d18a0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb8003d80 con 0x7fefd00d18a0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb800ab40 con 0x7fefd00d18a0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd00b4e60 con 0x7fefd00d18a0 2024-05-08T22:39:09.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.279+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd01586c0 con 0x7fefd00d18a0 2024-05-08T22:39:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd0158520 con 0x7fefd00d18a0 2024-05-08T22:39:09.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefb800aca0 con 0x7fefd00d18a0 2024-05-08T22:39:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefce7fc700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 0x7fefb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fefb80842e0 con 0x7fefd00d18a0 2024-05-08T22:39:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefd5191700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 0x7fefb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.283+0000 7fefd5191700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 0x7fefb0068bf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fefc0007b30 tx=0x7fefc0007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.291+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fefd0158520 con 0x7fefd00d18a0 2024-05-08T22:39:09.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.499+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefd0158be0 con 0x7fefd00d18a0 2024-05-08T22:39:09.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.499+0000 7fefce7fc700 1 -- 172.21.15.179:0/2446848890 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fefd0158be0 con 0x7fefd00d18a0 2024-05-08T22:39:09.501 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:09.501 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:38:31.946718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":6,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 msgr2=0x7fefb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 0x7fefb0068bf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fefc0007b30 tx=0x7fefc0007a80 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 msgr2=0x7fefd00b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fefb8001ed0 tx=0x7fefb80021e0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 shutdown_connections 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fefb0066730 0x7fefb0068bf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd015d580 0x7fefd014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd0161640 0x7fefd014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 --2- 172.21.15.179:0/2446848890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd00d18a0 0x7fefd00b41d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 >> 172.21.15.179:0/2446848890 conn(0x7fefd00d2d90 msgr2=0x7fefd00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 shutdown_connections 2024-05-08T22:39:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.503+0000 7fefd6994700 1 -- 172.21.15.179:0/2446848890 wait complete. 2024-05-08T22:39:09.505 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 46 2024-05-08T22:39:09.516 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-05-08T22:39:09.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.631+0000 7f1c1b35f700 1 Processor -- start 2024-05-08T22:39:09.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.631+0000 7f1c1b35f700 1 -- start start 2024-05-08T22:39:09.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 0x7f1c1415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 0x7f1c140cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1415e050 con 0x7f1c1415ae10 2024-05-08T22:39:09.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1415b2c0 con 0x7f1c14143580 2024-05-08T22:39:09.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406f0a0 con 0x7f1c1406c2d0 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1a35d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 0x7f1c1415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 0x7f1c140cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c19b5c700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1c1415ae10 0x7f1c1406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:35684/0 (socket says 172.21.15.179:35684) 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c19b5c700 1 -- 172.21.15.179:0/4173830850 learned_addr learned my addr 172.21.15.179:0/4173830850 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1935b700 1 -- 172.21.15.179:0/4173830850 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514325784 0 0) 0x7f1c1415e050 con 0x7f1c1415ae10 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1935b700 1 -- 172.21.15.179:0/4173830850 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c000035d0 con 0x7f1c1415ae10 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/4173830850 >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 msgr2=0x7f1c1415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/4173830850 >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 0x7f1c1415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/4173830850 >> v1:172.21.15.195:6789/0 conn(0x7f1c1415ae10 legacy=0x7f1c1406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/4173830850 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1406edc0 con 0x7f1c1406c2d0 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/4173830850 >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 0x7f1c1415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/4173830850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 0x7f1c140cd010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009850 tx=0x7f1c0c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85c8223215999a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1935b700 1 -- 172.21.15.179:0/4173830850 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c011830 con 0x7f1c1406c2d0 2024-05-08T22:39:09.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1935b700 1 -- 172.21.15.179:0/4173830850 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c011990 con 0x7f1c1406c2d0 2024-05-08T22:39:09.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1935b700 1 -- 172.21.15.179:0/4173830850 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c011d40 con 0x7f1c1406c2d0 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- 172.21.15.179:0/4173830850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 msgr2=0x7f1c140cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/4173830850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 0x7f1c140cd010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009850 tx=0x7f1c0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- 172.21.15.179:0/4173830850 shutdown_connections 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/4173830850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1406c2d0 0x7f1c140cd010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/4173830850 >> v2:172.21.15.195:3300/0 conn(0x7f1c14143580 0x7f1c1415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- 172.21.15.179:0/4173830850 >> 172.21.15.179:0/4173830850 conn(0x7f1c14070d80 msgr2=0x7f1c14071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- 172.21.15.179:0/4173830850 shutdown_connections 2024-05-08T22:39:09.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.635+0000 7f1c1b35f700 1 -- 172.21.15.179:0/4173830850 wait complete. 2024-05-08T22:39:09.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 Processor -- start 2024-05-08T22:39:09.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 -- start start 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c1406c2d0 0x7f1c141596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 0x7f1c140cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140a7600 con 0x7f1c14143580 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140a78e0 con 0x7f1c1406c2d0 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1b35f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1413cab0 con 0x7f1c1415ae10 2024-05-08T22:39:09.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c19b5c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c19b5c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44682/0 (socket says 172.21.15.179:44682) 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c19b5c700 1 -- 172.21.15.179:0/3985536940 learned_addr learned my addr 172.21.15.179:0/3985536940 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 0x7f1c140cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.639+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c1406c2d0 0x7f1c141596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 -- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 msgr2=0x7f1c140cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 0x7f1c140cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 -- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c1406c2d0 msgr2=0x7f1c141596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c1406c2d0 0x7f1c141596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 -- 172.21.15.179:0/3985536940 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1000d950 con 0x7f1c14143580 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 0x7f1c140cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800cbc0 tx=0x7f1c0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c06ffd700 1 -- 172.21.15.179:0/3985536940 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08007480 con 0x7f1c14143580 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c0094b0 con 0x7f1c14143580 2024-05-08T22:39:09.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c06ffd700 1 -- 172.21.15.179:0/3985536940 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c080075e0 con 0x7f1c14143580 2024-05-08T22:39:09.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1415ce20 con 0x7f1c14143580 2024-05-08T22:39:09.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c06ffd700 1 -- 172.21.15.179:0/3985536940 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08007950 con 0x7f1c14143580 2024-05-08T22:39:09.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.643+0000 7f1c06ffd700 1 -- 172.21.15.179:0/3985536940 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c0801c440 con 0x7f1c14143580 2024-05-08T22:39:09.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c06ffd700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 0x7f1bf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c06ffd700 1 -- 172.21.15.179:0/3985536940 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1c08083cb0 con 0x7f1c14143580 2024-05-08T22:39:09.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 0x7f1bf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 0x7f1bf8068bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1c10015460 tx=0x7f1c1000f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c1f870740 1 Processor -- start 2024-05-08T22:39:09.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.647+0000 7f1c1f870740 1 -- start start 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12932a0 0x125bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1293680 0x125c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ba810 con 0x11db520 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b7fa0 con 0x1293680 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x11b3890 con 0x12932a0 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44686/0 (socket says 172.21.15.179:44686) 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 -- 172.21.15.179:0/51435376 learned_addr learned my addr 172.21.15.179:0/51435376 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1293680 0x125c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12932a0 0x125bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 -- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12932a0 msgr2=0x125bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12932a0 0x125bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 -- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1293680 msgr2=0x125c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1293680 0x125c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 -- 172.21.15.179:0/51435376 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1025ba0 con 0x11db520 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001b7f0 tx=0x7f1c10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7d0836eb5c4baa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1bf7fff700 1 -- 172.21.15.179:0/51435376 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10012040 con 0x11db520 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1bf7fff700 1 -- 172.21.15.179:0/51435376 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10013040 con 0x11db520 2024-05-08T22:39:09.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1bf7fff700 1 -- 172.21.15.179:0/51435376 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10019a20 con 0x11db520 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- 172.21.15.179:0/51435376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 msgr2=0x1292d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001b7f0 tx=0x7f1c10018000 comp rx=0 tx=0).stop 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- 172.21.15.179:0/51435376 shutdown_connections 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1293680 0x125c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12932a0 0x125bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 --2- 172.21.15.179:0/51435376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x1292d60 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- 172.21.15.179:0/51435376 >> 172.21.15.179:0/51435376 conn(0xfda710 msgr2=0x1274d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- 172.21.15.179:0/51435376 shutdown_connections 2024-05-08T22:39:09.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.651+0000 7f1c1f870740 1 -- 172.21.15.179:0/51435376 wait complete. 2024-05-08T22:39:09.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- start start 2024-05-08T22:39:09.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 Processor -- start 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1293d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1293680 0x12942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1176d80 con 0x11db520 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1176a20 con 0x12932a0 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x10a6770 con 0x1293680 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44702/0 (socket says 172.21.15.179:44702) 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 -- 172.21.15.179:0/3970920224 learned_addr learned my addr 172.21.15.179:0/3970920224 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 10 client.0 _ll_register_callbacks cb 0x11acf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1293680 0x12942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1293d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 -- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1293680 msgr2=0x12942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1293680 0x12942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 -- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 msgr2=0x1293d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1293d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 -- 172.21.15.179:0/3970920224 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1001b440 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c10015bb0 tx=0x7f1c100047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e55138b2d9a24b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10012070 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xfe7420 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1c10020040 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c10003c90 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 10 client.7569 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x1179910 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xfe7420 con 0x11db520 2024-05-08T22:39:09.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1c10019620 con 0x11db520 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 10 client.7569 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 10 client.7569 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 20 client.7569 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x124ea40 0x124ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 4 client.7569 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.655+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10316a0 con 0x124ea40 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x124ea40 0x124ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x124ea40 0x124ce70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239bbbb3fc1205db server_cookie=f0ed8dabb42d7f57 in_seq=0 out_seq=0 2024-05-08T22:39:09.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1bf57fa700 10 client.7569 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3970920224 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x10316a0 con 0x124ea40 2024-05-08T22:39:09.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1bf57fa700 10 client.7569 handle_command_reply: tid=0 2024-05-08T22:39:09.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 client.7569 shutdown 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 msgr2=0x12923f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c10015bb0 tx=0x7f1c100047e0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 shutdown_connections 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x124ea40 0x124ce70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1293d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1293680 0x12942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1bf57fa700 0 client.7569 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3970920224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x12923f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 >> 172.21.15.179:0/3970920224 conn(0xfda710 msgr2=0x12721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 shutdown_connections 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 1 -- 172.21.15.179:0/3970920224 wait complete. 2024-05-08T22:39:09.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.659+0000 7f1c1f870740 20 client.7569 trim_cache size 0 max 16384 2024-05-08T22:39:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.675+0000 7f1c1f870740 1 Processor -- start 2024-05-08T22:39:09.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.675+0000 7f1c1f870740 1 -- start start 2024-05-08T22:39:09.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x124d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1290000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x11da9a0 con 0x11db520 2024-05-08T22:39:09.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b0c90 con 0x12932a0 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x11af080 con 0x124ea40 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44712/0 (socket says 172.21.15.179:44712) 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 -- 172.21.15.179:0/437458366 learned_addr learned my addr 172.21.15.179:0/437458366 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x124d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1290000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 -- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 msgr2=0x124d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x124d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 -- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 msgr2=0x1290000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1290000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 -- 172.21.15.179:0/437458366 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x115fb40 con 0x11db520 2024-05-08T22:39:09.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004820 tx=0x7f1c10024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3431c2578cd5aca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c04ff9700 1 -- 172.21.15.179:0/437458366 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1001be90 con 0x11db520 2024-05-08T22:39:09.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c04ff9700 1 -- 172.21.15.179:0/437458366 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1001b660 con 0x11db520 2024-05-08T22:39:09.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c04ff9700 1 -- 172.21.15.179:0/437458366 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10019b30 con 0x11db520 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- 172.21.15.179:0/437458366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 msgr2=0x124ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004820 tx=0x7f1c10024000 comp rx=0 tx=0).stop 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- 172.21.15.179:0/437458366 shutdown_connections 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12932a0 0x1290000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x124d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 --2- 172.21.15.179:0/437458366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x11db520 0x124ce70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- 172.21.15.179:0/437458366 >> 172.21.15.179:0/437458366 conn(0xfda710 msgr2=0x1275380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- 172.21.15.179:0/437458366 shutdown_connections 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- 172.21.15.179:0/437458366 wait complete. 2024-05-08T22:39:09.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 -- start start 2024-05-08T22:39:09.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.679+0000 7f1c1f870740 1 Processor -- start 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 0x12b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x12f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b10a0 con 0x12932a0 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12b12f0 con 0x11db520 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12509f0 con 0x124ea40 2024-05-08T22:39:09.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 10 client.0 _ll_register_callbacks cb 0x11b2460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44720/0 (socket says 172.21.15.179:44720) 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/3791325646 learned_addr learned my addr 172.21.15.179:0/3791325646 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c19b5c700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x12f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 0x12b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 msgr2=0x12f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x12f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 msgr2=0x12b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 0x12b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 -- 172.21.15.179:0/3791325646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1179910 con 0x12932a0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 0x12b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1ab5e700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013000 tx=0x7f1c0c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415956939b23131 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:09.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c020040 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1c0c004450 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c10013040 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c0c004610 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 10 client.7575 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x1153c20 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10013040 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1c0c030430 con 0x12932a0 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 10 client.7575 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 10 client.7575 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 20 client.7575 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1293680 0x12f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 4 client.7575 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x1056420 con 0x1293680 2024-05-08T22:39:09.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.683+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1293680 0x12f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1a35d700 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1293680 0x12f3cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f174ea5f1fbe2645 server_cookie=3422ea45effa4303 in_seq=0 out_seq=0 2024-05-08T22:39:09.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1bf57fa700 10 client.7575 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1bf57fa700 1 -- 172.21.15.179:0/3791325646 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x1056420 con 0x1293680 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1bf57fa700 10 client.7575 handle_command_reply: tid=0 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 client.7575 shutdown 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 msgr2=0x12912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013000 tx=0x7f1c0c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 shutdown_connections 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12932a0 0x12912f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x124ea40 0x12f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1293680 0x12f3cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 --2- 172.21.15.179:0/3791325646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x11db520 0x12b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1bf57fa700 0 client.7575 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 >> 172.21.15.179:0/3791325646 conn(0xfda710 msgr2=0x124ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 shutdown_connections 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 1 -- 172.21.15.179:0/3791325646 wait complete. 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1f870740 20 client.7575 trim_cache size 0 max 16384 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:09.691 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "dentry", 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 2328003840, 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627776, 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "frag": "*", 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "dname": "file_to_be_damaged", 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "snap_id": "head", 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:09.692 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 msgr2=0x7f1bf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 0x7f1bf8068bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1c10015460 tx=0x7f1c1000f040 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 msgr2=0x7f1c141eeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800cbc0 tx=0x7f1c0800cbf0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 shutdown_connections 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1bf8066730 0x7f1bf8068bf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c1415ae10 0x7f1c140cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c14143580 0x7f1c141eeba0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 --2- 172.21.15.179:0/3985536940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c1406c2d0 0x7f1c141596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 >> 172.21.15.179:0/3985536940 conn(0x7f1c14070d80 msgr2=0x7f1c1413fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 shutdown_connections 2024-05-08T22:39:09.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.687+0000 7f1c1b35f700 1 -- 172.21.15.179:0/3985536940 wait complete. 2024-05-08T22:39:09.703 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:39:09.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.815+0000 7f095359e700 1 Processor -- start 2024-05-08T22:39:09.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.815+0000 7f095359e700 1 -- start start 2024-05-08T22:39:09.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f0954161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0954165d50 con 0x7f095406de70 2024-05-08T22:39:09.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095406ad70 con 0x7f0954164db0 2024-05-08T22:39:09.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0954165230 con 0x7f095415ef80 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0952d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f095406de70 0x7f095406b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52482/0 (socket says 172.21.15.179:52482) 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0952d9d700 1 -- 172.21.15.179:0/4175989821 learned_addr learned my addr 172.21.15.179:0/4175989821 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095259c700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f0954161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618450841 0 0) 0x7f0954165d50 con 0x7f095406de70 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c0035d0 con 0x7f095406de70 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006925816 0 0) 0x7f095406ad70 con 0x7f0954164db0 2024-05-08T22:39:09.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0954165d50 con 0x7f0954164db0 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3359969520 0 0) 0x7f093c0035d0 con 0x7f095406de70 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 msgr2=0x7f0954161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f0954161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 >> v1:172.21.15.195:6789/0 conn(0x7f0954164db0 legacy=0x7f095406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095406e4b0 con 0x7f095406de70 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f094c0019f0 con 0x7f095406de70 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 >> v1:172.21.15.179:6789/0 conn(0x7f095406de70 legacy=0x7f095406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f093c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f093c004140 0x7f093c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004db0 0x7f093c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954165230 con 0x7f093c004140 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c00bbc0 con 0x7f093c004db0 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c00bd50 con 0x7f095415ef80 2024-05-08T22:39:09.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f093c004140 0x7f093c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095259c700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f093c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0952d9d700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004db0 0x7f093c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 -- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 msgr2=0x7f093c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095415ef80 0x7f093c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 -- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004db0 msgr2=0x7f093c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004db0 0x7f093c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 -- 172.21.15.179:0/4175989821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0035d0 con 0x7f093c004140 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0952d9d700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004db0 0x7f093c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0951d9b700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f093c004140 0x7f093c004870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0944001960 tx=0x7f093c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6025ffef3d2984f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944001450 con 0x7f093c004140 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09440052f0 con 0x7f093c004140 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095159a700 1 -- 172.21.15.179:0/4175989821 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09440036a0 con 0x7f093c004140 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- 172.21.15.179:0/4175989821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f093c004140 msgr2=0x7f093c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 --2- 172.21.15.179:0/4175989821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f093c004140 0x7f093c004870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0944001960 tx=0x7f093c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f0952d9d700 1 -- 172.21.15.179:0/4175989821 reap_dead start 2024-05-08T22:39:09.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- 172.21.15.179:0/4175989821 shutdown_connections 2024-05-08T22:39:09.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- 172.21.15.179:0/4175989821 >> 172.21.15.179:0/4175989821 conn(0x7f09540ce9d0 msgr2=0x7f09540cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:09.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- 172.21.15.179:0/4175989821 shutdown_connections 2024-05-08T22:39:09.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.819+0000 7f095359e700 1 -- 172.21.15.179:0/4175989821 wait complete. 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 Processor -- start 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- start start 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095414cb10 0x7f09541eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954163d20 con 0x7f095414cf90 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540ce0d0 con 0x7f093c004140 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0954163380 con 0x7f095414cb10 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:60510/0 (socket says 172.21.15.179:60510) 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095259c700 1 -- 172.21.15.179:0/3578004945 learned_addr learned my addr 172.21.15.179:0/3578004945 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0951d9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095414cb10 0x7f09541eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44728/0 (socket says 172.21.15.179:44728) 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 -- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095414cb10 msgr2=0x7f09541eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095414cb10 0x7f09541eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 -- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 msgr2=0x7f09540cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 -- 172.21.15.179:0/3578004945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954163660 con 0x7f095414cf90 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095259c700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0952d9d700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f093c003bb0 tx=0x7f094c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0074e0 con 0x7f095414cf90 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c007660 con 0x7f095414cf90 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c007a10 con 0x7f095414cf90 2024-05-08T22:39:09.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0954163940 con 0x7f095414cf90 2024-05-08T22:39:09.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.823+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09540cd330 con 0x7f095414cf90 2024-05-08T22:39:09.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.827+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0954163e60 con 0x7f095414cf90 2024-05-08T22:39:09.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.827+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f094c0123d0 con 0x7f095414cf90 2024-05-08T22:39:09.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.827+0000 7f0942ffd700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 0x7f0934068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:09.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.827+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f094c083b70 con 0x7f095414cf90 2024-05-08T22:39:09.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.827+0000 7f095259c700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 0x7f0934068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:09.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.831+0000 7f095259c700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 0x7f0934068c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0948009f80 tx=0x7f0948008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:09.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:09.839+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0954163e60 con 0x7f095414cf90 2024-05-08T22:39:10.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.043+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0954163e60 con 0x7f095414cf90 2024-05-08T22:39:10.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f0942ffd700 1 -- 172.21.15.179:0/3578004945 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f0954163e60 con 0x7f095414cf90 2024-05-08T22:39:10.047 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:10.047 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T22:39:10.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 msgr2=0x7f0934068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 0x7f0934068c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0948009f80 tx=0x7f0948008040 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 msgr2=0x7f09541eb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f093c003bb0 tx=0x7f094c00dbe0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 shutdown_connections 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0934066760 0x7f0934068c20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f095414cf90 0x7f09541eb150 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095414cb10 0x7f09541eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 --2- 172.21.15.179:0/3578004945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093c004140 0x7f09540cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 >> 172.21.15.179:0/3578004945 conn(0x7f09540ce9d0 msgr2=0x7f0954161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 shutdown_connections 2024-05-08T22:39:10.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.047+0000 7f095359e700 1 -- 172.21.15.179:0/3578004945 wait complete. 2024-05-08T22:39:10.061 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:39:10.062 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-08T22:39:10.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 Processor -- start 2024-05-08T22:39:10.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 -- start start 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb04c159770 0x7fb04c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c0ce7c0 con 0x7fb04c06b980 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c06be10 con 0x7fb04c159770 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051a54700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c06b2c0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051253700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb04c06b980 0x7fb04c06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52490/0 (socket says 172.21.15.179:52490) 2024-05-08T22:39:10.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb051253700 1 -- 172.21.15.179:0/3815456490 learned_addr learned my addr 172.21.15.179:0/3815456490 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:10.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb050a52700 1 --2- 172.21.15.179:0/3815456490 >> v2:172.21.15.195:3300/0 conn(0x7fb04c159770 0x7fb04c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb04bfff700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb04b7fe700 1 -- 172.21.15.179:0/3815456490 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297206638 0 0) 0x7fb04c0ce7c0 con 0x7fb04c06b980 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.183+0000 7fb04b7fe700 1 -- 172.21.15.179:0/3815456490 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0340035d0 con 0x7fb04c06b980 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb050a52700 1 -- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 msgr2=0x7fb04c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb050a52700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb050a52700 1 -- 172.21.15.179:0/3815456490 >> v1:172.21.15.179:6789/0 conn(0x7fb04c06b980 legacy=0x7fb04c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb050a52700 1 -- 172.21.15.179:0/3815456490 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0ccd60 con 0x7fb04c159770 2024-05-08T22:39:10.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb04bfff700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:10.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb050a52700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c159770 0x7fb04c0cc760 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb040009880 tx=0x7fb040009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c989e86553cc3b69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb04b7fe700 1 -- 172.21.15.179:0/3815456490 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011830 con 0x7fb04c159770 2024-05-08T22:39:10.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb04b7fe700 1 -- 172.21.15.179:0/3815456490 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040011990 con 0x7fb04c159770 2024-05-08T22:39:10.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb04b7fe700 1 -- 172.21.15.179:0/3815456490 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011d20 con 0x7fb04c159770 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 -- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c159770 msgr2=0x7fb04c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c159770 0x7fb04c0cc760 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb040009880 tx=0x7fb040009b90 comp rx=0 tx=0).stop 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 -- 172.21.15.179:0/3815456490 shutdown_connections 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 --2- 172.21.15.179:0/3815456490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c159770 0x7fb04c0cc760 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 -- 172.21.15.179:0/3815456490 >> 172.21.15.179:0/3815456490 conn(0x7fb04c0ce9a0 msgr2=0x7fb04c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 -- 172.21.15.179:0/3815456490 shutdown_connections 2024-05-08T22:39:10.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.187+0000 7fb051a54700 1 -- 172.21.15.179:0/3815456490 wait complete. 2024-05-08T22:39:10.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 Processor -- start 2024-05-08T22:39:10.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- start start 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c06b980 0x7fb04c159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c1484b0 con 0x7fb04c143250 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c15a130 con 0x7fb04c06b980 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c15a410 con 0x7fb04c15dbb0 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44730/0 (socket says 172.21.15.179:44730) 2024-05-08T22:39:10.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051253700 1 -- 172.21.15.179:0/1280964756 learned_addr learned my addr 172.21.15.179:0/1280964756 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb050a52700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c06b980 0x7fb04c159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 -- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c06b980 msgr2=0x7fb04c159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c06b980 0x7fb04c159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 -- 172.21.15.179:0/1280964756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 msgr2=0x7fb04c147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 -- 172.21.15.179:0/1280964756 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0094b0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051253700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb04bfff700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c15b1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9b0 tx=0x7fb03c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb0497fa700 1 -- 172.21.15.179:0/1280964756 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00c640 con 0x7fb04c15dbb0 2024-05-08T22:39:10.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb0497fa700 1 -- 172.21.15.179:0/1280964756 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00c7a0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0400094b0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb0497fa700 1 -- 172.21.15.179:0/1280964756 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00cb50 con 0x7fb04c15dbb0 2024-05-08T22:39:10.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.191+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c15a9d0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.195+0000 7fb0497fa700 1 -- 172.21.15.179:0/1280964756 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb03c00ccb0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.195+0000 7fb0497fa700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 0x7fb038068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.195+0000 7fb0497fa700 1 -- 172.21.15.179:0/1280964756 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb03c0842b0 con 0x7fb04c15dbb0 2024-05-08T22:39:10.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.195+0000 7fb050a52700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 0x7fb038068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb050a52700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 0x7fb038068bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0b4060 tx=0x7fb040008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 Processor -- start 2024-05-08T22:39:10.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 -- start start 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x2116fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x20cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2039530 con 0x21174e0 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2036cc0 con 0x2116bc0 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20325b0 con 0x20cd590 2024-05-08T22:39:10.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x20cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb050a52700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x2116fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44736/0 (socket says 172.21.15.179:44736) 2024-05-08T22:39:10.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x20cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:56876/0 (socket says 172.21.15.179:56876) 2024-05-08T22:39:10.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb051253700 1 -- 172.21.15.179:0/3717012339 learned_addr learned my addr 172.21.15.179:0/3717012339 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.199+0000 7fb051253700 1 -- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 msgr2=0x20cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb051253700 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x20cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb051253700 1 -- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 msgr2=0x2116fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb051253700 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x2116fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb051253700 1 -- 172.21.15.179:0/3717012339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f24f10 con 0x21174e0 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb050a52700 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x2116fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb051253700 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb04400d500 tx=0x7fb044001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99f504a8e2716cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb0327fc700 1 -- 172.21.15.179:0/3717012339 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400dc90 con 0x21174e0 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb0327fc700 1 -- 172.21.15.179:0/3717012339 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400ddf0 con 0x21174e0 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb0327fc700 1 -- 172.21.15.179:0/3717012339 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044007310 con 0x21174e0 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- 172.21.15.179:0/3717012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 msgr2=0x20cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb04400d500 tx=0x7fb044001ad0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- 172.21.15.179:0/3717012339 shutdown_connections 2024-05-08T22:39:10.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x20cb750 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x20cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 --2- 172.21.15.179:0/3717012339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x2116fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- 172.21.15.179:0/3717012339 >> 172.21.15.179:0/3717012339 conn(0x1e597b0 msgr2=0x20f40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- 172.21.15.179:0/3717012339 shutdown_connections 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- 172.21.15.179:0/3717012339 wait complete. 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 -- start start 2024-05-08T22:39:10.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 Processor -- start 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.203+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20cd590 0x20f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x2112780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ff5950 con 0x21174e0 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ff55f0 con 0x20cd590 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fc7e40 con 0x2116bc0 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 10 client.0 _ll_register_callbacks cb 0x202bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44744/0 (socket says 172.21.15.179:44744) 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 -- 172.21.15.179:0/2079121720 learned_addr learned my addr 172.21.15.179:0/2079121720 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb050a52700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20cd590 0x20f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb04bfff700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x2112780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 -- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 msgr2=0x2112780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x2112780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 -- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20cd590 msgr2=0x20f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20cd590 0x20f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 -- 172.21.15.179:0/2079121720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0ccd60 con 0x21174e0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb051253700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb044001580 tx=0x7fb044001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e23ad5e3829509 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400dc90 con 0x21174e0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb044021430 con 0x21174e0 2024-05-08T22:39:10.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e664c0 con 0x21174e0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb044001810 con 0x21174e0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 10 client.7593 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x7fb0340035d0 con 0x21174e0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e664c0 con 0x21174e0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb044007d20 con 0x21174e0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 10 client.7593 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 10 client.7593 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 20 client.7593 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x2115fe0 0x2110d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 4 client.7593 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1eb0740 con 0x2115fe0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb050a52700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x2115fe0 0x2110d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb02b7fe700 10 client.7593 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.207+0000 7fb050a52700 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x2115fe0 0x2110d60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd188f323a7439c server_cookie=27264a35690b7f6c in_seq=0 out_seq=0 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb02b7fe700 1 -- 172.21.15.179:0/2079121720 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1eb0740 con 0x2115fe0 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb02b7fe700 10 client.7593 handle_command_reply: tid=0 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 client.7593 shutdown 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 msgr2=0x2112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb044001580 tx=0x7fb044001250 comp rx=0 tx=0).stop 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 shutdown_connections 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2112cc0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x2112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x2115fe0 0x2110d60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 --2- 172.21.15.179:0/2079121720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20cd590 0x20f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 >> 172.21.15.179:0/2079121720 conn(0x1e597b0 msgr2=0x20cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb02b7fe700 0 client.7593 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 shutdown_connections 2024-05-08T22:39:10.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 1 -- 172.21.15.179:0/2079121720 wait complete. 2024-05-08T22:39:10.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.211+0000 7fb055f65740 20 client.7593 trim_cache size 0 max 16384 2024-05-08T22:39:10.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 Processor -- start 2024-05-08T22:39:10.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 -- start start 2024-05-08T22:39:10.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x210f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x210f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x202dda0 con 0x21174e0 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd05d0 con 0x2116bc0 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fec040 con 0x20cd590 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb051253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44756/0 (socket says 172.21.15.179:44756) 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.231+0000 7fb051253700 1 -- 172.21.15.179:0/1218135281 learned_addr learned my addr 172.21.15.179:0/1218135281 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb050a52700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x210f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb04bfff700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x210f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 -- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 msgr2=0x210f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x210f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 -- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 msgr2=0x210f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x210f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 -- 172.21.15.179:0/1218135281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ff8630 con 0x21174e0 2024-05-08T22:39:10.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb04bfff700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x210f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:10.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb051253700 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb04402a4d0 tx=0x7fb044021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c140683f51f9b5ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb032ffd700 1 -- 172.21.15.179:0/1218135281 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400dc90 con 0x21174e0 2024-05-08T22:39:10.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb032ffd700 1 -- 172.21.15.179:0/1218135281 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04402a560 con 0x21174e0 2024-05-08T22:39:10.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb032ffd700 1 -- 172.21.15.179:0/1218135281 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04402a8f0 con 0x21174e0 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- 172.21.15.179:0/1218135281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 msgr2=0x2125930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb04402a4d0 tx=0x7fb044021000 comp rx=0 tx=0).stop 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- 172.21.15.179:0/1218135281 shutdown_connections 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2116bc0 0x210f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21174e0 0x2125930 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 --2- 172.21.15.179:0/1218135281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20cd590 0x210f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- 172.21.15.179:0/1218135281 >> 172.21.15.179:0/1218135281 conn(0x1e597b0 msgr2=0x210ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- 172.21.15.179:0/1218135281 shutdown_connections 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- 172.21.15.179:0/1218135281 wait complete. 2024-05-08T22:39:10.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 -- start start 2024-05-08T22:39:10.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.235+0000 7fb055f65740 1 Processor -- start 2024-05-08T22:39:10.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x212e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21174e0 0x217bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2035510 con 0x20cd590 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2114530 con 0x21174e0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x21361b0 con 0x2116bc0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 10 client.0 _ll_register_callbacks cb 0x2031180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44758/0 (socket says 172.21.15.179:44758) 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 -- 172.21.15.179:0/4077358120 learned_addr learned my addr 172.21.15.179:0/4077358120 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb04bfff700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x212e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb051253700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21174e0 0x217bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 -- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 msgr2=0x212e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x212e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 -- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21174e0 msgr2=0x217bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21174e0 0x217bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 -- 172.21.15.179:0/4077358120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0340035d0 con 0x20cd590 2024-05-08T22:39:10.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb050a52700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb040000c00 tx=0x7fb0400047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c57f2e032b7824f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04002d040 con 0x20cd590 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb040003740 con 0x20cd590 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb04400ddf0 con 0x20cd590 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb040020070 con 0x20cd590 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 10 client.7599 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x1fde860 con 0x20cd590 2024-05-08T22:39:10.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400ddf0 con 0x20cd590 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb040003900 con 0x20cd590 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 10 client.7599 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.239+0000 7fb055f65740 10 client.7599 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 20 client.7599 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x217c2e0 0x2178be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 4 client.7599 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x1ed7620 con 0x217c2e0 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb04bfff700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x217c2e0 0x2178be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb04bfff700 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x217c2e0 0x2178be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e010eaa3ecf935bb server_cookie=83c8b15ecac8f389 in_seq=0 out_seq=0 2024-05-08T22:39:10.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb02b7fe700 10 client.7599 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb02b7fe700 1 -- 172.21.15.179:0/4077358120 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1ed7620 con 0x217c2e0 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb02b7fe700 10 client.7599 handle_command_reply: tid=0 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 client.7599 shutdown 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 msgr2=0x212d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb040000c00 tx=0x7fb0400047e0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 shutdown_connections 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x217c2e0 0x2178be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb02b7fe700 0 client.7599 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21174e0 0x217bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2116bc0 0x212e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 --2- 172.21.15.179:0/4077358120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20cd590 0x212d370 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 >> 172.21.15.179:0/4077358120 conn(0x1e597b0 msgr2=0x210ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 shutdown_connections 2024-05-08T22:39:10.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 1 -- 172.21.15.179:0/4077358120 wait complete. 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.243+0000 7fb055f65740 20 client.7599 trim_cache size 0 max 16384 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "72d56be7-c27e-48ab-8d7d-b3b4329a7ef9", 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:39:10.247 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 msgr2=0x7fb038068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 0x7fb038068bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0b4060 tx=0x7fb040008030 comp rx=0 tx=0).stop 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 msgr2=0x7fb04c15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c15b1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9b0 tx=0x7fb03c00ecc0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 shutdown_connections 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb038066730 0x7fb038068bf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb04c143250 0x7fb04c147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb04c15dbb0 0x7fb04c15b1d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 --2- 172.21.15.179:0/1280964756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb04c06b980 0x7fb04c159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 >> 172.21.15.179:0/1280964756 conn(0x7fb04c0ce9a0 msgr2=0x7fb04c0aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 shutdown_connections 2024-05-08T22:39:10.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.247+0000 7fb051a54700 1 -- 172.21.15.179:0/1280964756 wait complete. 2024-05-08T22:39:10.262 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T22:39:10.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 Processor -- start 2024-05-08T22:39:10.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 -- start start 2024-05-08T22:39:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4a415c0b0 0x7fa4a40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40cbfd0 con 0x7fa4a40cc9a0 2024-05-08T22:39:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40cb3c0 con 0x7fa4a41448b0 2024-05-08T22:39:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a9177700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a40cd970 con 0x7fa4a415c0b0 2024-05-08T22:39:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4a415c0b0 0x7fa4a40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a8976700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa4a40cc9a0 0x7fa4a406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52506/0 (socket says 172.21.15.179:52506) 2024-05-08T22:39:10.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4a415c0b0 0x7fa4a40cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44774/0 (socket says 172.21.15.179:44774) 2024-05-08T22:39:10.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a8976700 1 -- 172.21.15.179:0/2812430 learned_addr learned my addr 172.21.15.179:0/2812430 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:10.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a37fe700 1 -- 172.21.15.179:0/2812430 >> v1:172.21.15.195:6789/0 conn(0x7fa4a41448b0 legacy=0x7fa4a40ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a37fe700 1 -- 172.21.15.179:0/2812430 >> v1:172.21.15.179:6789/0 conn(0x7fa4a40cc9a0 legacy=0x7fa4a406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a37fe700 1 -- 172.21.15.179:0/2812430 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a40cb6a0 con 0x7fa4a415c0b0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.387+0000 7fa4a2ffd700 1 -- 172.21.15.179:0/2812430 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153539716 0 0) 0x7fa4a40cbfd0 con 0x7fa4a40cc9a0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/2812430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a40cc460 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009850 tx=0x7fa48c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789efbd6b8d5951f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a2ffd700 1 -- 172.21.15.179:0/2812430 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c011830 con 0x7fa4a415c0b0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a2ffd700 1 -- 172.21.15.179:0/2812430 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c011990 con 0x7fa4a415c0b0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a2ffd700 1 -- 172.21.15.179:0/2812430 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c011d20 con 0x7fa4a415c0b0 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- 172.21.15.179:0/2812430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 msgr2=0x7fa4a40cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 --2- 172.21.15.179:0/2812430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a40cc460 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009850 tx=0x7fa48c009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- 172.21.15.179:0/2812430 shutdown_connections 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 --2- 172.21.15.179:0/2812430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a40cc460 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- 172.21.15.179:0/2812430 >> 172.21.15.179:0/2812430 conn(0x7fa4a4070d80 msgr2=0x7fa4a4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- 172.21.15.179:0/2812430 shutdown_connections 2024-05-08T22:39:10.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- 172.21.15.179:0/2812430 wait complete. 2024-05-08T22:39:10.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 Processor -- start 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- start start 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a40cc9a0 0x7fa4a40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a41448b0 0x7fa4a40acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a415ad10 con 0x7fa4a415c0b0 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ac880 con 0x7fa4a41448b0 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a9177700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40abfe0 con 0x7fa4a40cc9a0 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a8976700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a8976700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44782/0 (socket says 172.21.15.179:44782) 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a8976700 1 -- 172.21.15.179:0/640539338 learned_addr learned my addr 172.21.15.179:0/640539338 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a41448b0 0x7fa4a40acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.391+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a40cc9a0 0x7fa4a40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 -- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a40cc9a0 msgr2=0x7fa4a40ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a40cc9a0 0x7fa4a40ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 -- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a41448b0 msgr2=0x7fa4a40acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a41448b0 0x7fa4a40acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 -- 172.21.15.179:0/640539338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0094b0 con 0x7fa4a415c0b0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a8976700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00cca0 tx=0x7fa49c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 -- 172.21.15.179:0/640539338 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00ab70 con 0x7fa4a415c0b0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 -- 172.21.15.179:0/640539338 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c00acd0 con 0x7fa4a415c0b0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40ac2c0 con 0x7fa4a415c0b0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 -- 172.21.15.179:0/640539338 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c007310 con 0x7fa4a415c0b0 2024-05-08T22:39:10.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a415af40 con 0x7fa4a415c0b0 2024-05-08T22:39:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 -- 172.21.15.179:0/640539338 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa49c007470 con 0x7fa4a415c0b0 2024-05-08T22:39:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 0x7fa488068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a0ff9700 1 -- 172.21.15.179:0/640539338 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa49c0840a0 con 0x7fa4a415c0b0 2024-05-08T22:39:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.395+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 0x7fa488068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 0x7fa488068bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa494000f60 tx=0x7fa494009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 Processor -- start 2024-05-08T22:39:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 -- start start 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1588840 0x158dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x1589810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x151b010 con 0x15b26f0 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x14a8f40 con 0x1588840 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x14fbe30 con 0x15b55d0 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44794/0 (socket says 172.21.15.179:44794) 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a37fe700 1 -- 172.21.15.179:0/933568819 learned_addr learned my addr 172.21.15.179:0/933568819 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1588840 0x158dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.399+0000 7fa4a8976700 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x1589810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 -- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 msgr2=0x1589810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x1589810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 -- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1588840 msgr2=0x158dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1588840 0x158dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 -- 172.21.15.179:0/933568819 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1367a40 con 0x15b26f0 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa48c014040 tx=0x7fa48c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43d53f77bb5b8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa499ffb700 1 -- 172.21.15.179:0/933568819 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c028070 con 0x15b26f0 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa499ffb700 1 -- 172.21.15.179:0/933568819 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c011b00 con 0x15b26f0 2024-05-08T22:39:10.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa499ffb700 1 -- 172.21.15.179:0/933568819 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c01a420 con 0x15b26f0 2024-05-08T22:39:10.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- 172.21.15.179:0/933568819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 msgr2=0x1588c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa48c014040 tx=0x7fa48c008030 comp rx=0 tx=0).stop 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- 172.21.15.179:0/933568819 shutdown_connections 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x1589810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1588c20 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 --2- 172.21.15.179:0/933568819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1588840 0x158dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- 172.21.15.179:0/933568819 >> 172.21.15.179:0/933568819 conn(0x131c5b0 msgr2=0x15b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- 172.21.15.179:0/933568819 shutdown_connections 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- 172.21.15.179:0/933568819 wait complete. 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 -- start start 2024-05-08T22:39:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.403+0000 7fa4ad688740 1 Processor -- start 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 0x158bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x158c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f7150 con 0x1588840 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f4eb0 con 0x15b26f0 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x14b28e0 con 0x15b55d0 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 10 client.0 _ll_register_callbacks cb 0x1491670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44804/0 (socket says 172.21.15.179:44804) 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a3fff700 1 -- 172.21.15.179:0/1585057094 learned_addr learned my addr 172.21.15.179:0/1585057094 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 0x158bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a8976700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x158c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 -- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 msgr2=0x158c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x158c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 -- 172.21.15.179:0/1585057094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 msgr2=0x15d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 -- 172.21.15.179:0/1585057094 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40cb6a0 con 0x15b26f0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 0x158bf20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009fd0 tx=0x7fa48c0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392031f6fce0944b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c028070 con 0x15b26f0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa48c01f070 con 0x15b26f0 2024-05-08T22:39:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13292c0 con 0x15b26f0 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c0045d0 con 0x15b26f0 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 10 client.7447 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x14bb290 con 0x15b26f0 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13292c0 con 0x15b26f0 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa48c0036c0 con 0x15b26f0 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 10 client.7447 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 10 client.7447 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 20 client.7447 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b7390 0x15dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 4 client.7447 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1373540 con 0x15b7390 2024-05-08T22:39:10.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b7390 0x15dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa492ffd700 10 client.7447 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.407+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b7390 0x15dad60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12c635cadb07549 server_cookie=3cedd3fd503ac193 in_seq=0 out_seq=0 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa492ffd700 1 -- 172.21.15.179:0/1585057094 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1373540 con 0x15b7390 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa492ffd700 10 client.7447 handle_command_reply: tid=0 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 client.7447 shutdown 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 msgr2=0x158bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 0x158bf20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009fd0 tx=0x7fa48c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 shutdown_connections 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b7390 0x15dad60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b26f0 0x158bf20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b55d0 0x158c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1585057094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa492ffd700 0 client.7447 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 >> 172.21.15.179:0/1585057094 conn(0x131c5b0 msgr2=0x1586db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 shutdown_connections 2024-05-08T22:39:10.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 1 -- 172.21.15.179:0/1585057094 wait complete. 2024-05-08T22:39:10.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.411+0000 7fa4ad688740 20 client.7447 trim_cache size 0 max 16384 2024-05-08T22:39:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 Processor -- start 2024-05-08T22:39:10.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 -- start start 2024-05-08T22:39:10.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b26f0 0x15b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x15d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x13a0b80 con 0x1588840 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f5760 con 0x15b55d0 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x14f3530 con 0x15b26f0 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4a3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44808/0 (socket says 172.21.15.179:44808) 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4a8976700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x15d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4a3fff700 1 -- 172.21.15.179:0/1451185997 learned_addr learned my addr 172.21.15.179:0/1451185997 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.431+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b26f0 0x15b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b26f0 msgr2=0x15b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b26f0 0x15b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/1451185997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 msgr2=0x15b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/1451185997 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14a14c0 con 0x15b55d0 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:10.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x15d1f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa49c082480 tx=0x7fa49c05b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dcaa601ef87b4d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa49a7fc700 1 -- 172.21.15.179:0/1451185997 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c055630 con 0x15b55d0 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa49a7fc700 1 -- 172.21.15.179:0/1451185997 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c054f30 con 0x15b55d0 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa49a7fc700 1 -- 172.21.15.179:0/1451185997 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c058d60 con 0x15b55d0 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 msgr2=0x15d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x15d1f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa49c082480 tx=0x7fa49c05b040 comp rx=0 tx=0).stop 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- 172.21.15.179:0/1451185997 shutdown_connections 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x15d1f70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x15b26f0 0x15b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- 172.21.15.179:0/1451185997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1588840 0x15b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- 172.21.15.179:0/1451185997 >> 172.21.15.179:0/1451185997 conn(0x131c5b0 msgr2=0x15d9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- 172.21.15.179:0/1451185997 shutdown_connections 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- 172.21.15.179:0/1451185997 wait complete. 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- start start 2024-05-08T22:39:10.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 Processor -- start 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1588840 0x15880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b8690 con 0x15b26f0 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1588600 con 0x15b55d0 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x15d22d0 con 0x1588840 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4ad688740 10 client.0 _ll_register_callbacks cb 0x14aed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1588840 0x15880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44812/0 (socket says 172.21.15.179:44812) 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:60636/0 (socket says 172.21.15.179:60636) 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/584227869 learned_addr learned my addr 172.21.15.179:0/584227869 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1588840 msgr2=0x15880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1588840 0x15880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.435+0000 7fa4a8976700 1 -- 172.21.15.179:0/584227869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 msgr2=0x1634e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a8976700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a8976700 1 -- 172.21.15.179:0/584227869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40cb3c0 con 0x15b55d0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a37fe700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a8976700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa49c054c30 tx=0x7fa49c009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c9086c536c50f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c082070 con 0x15b55d0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa48c011b00 con 0x15b55d0 2024-05-08T22:39:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa49c055630 con 0x15b55d0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49c054a70 con 0x15b55d0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 10 client.7453 fetch_fsmap learned FSMap version 46 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x7fa4a415af40 con 0x15b55d0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c011b00 con 0x15b55d0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa49c06cb40 con 0x15b55d0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 10 client.7453 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 10 client.7453 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 20 client.7453 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b13c0 0x15eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 4 client.7453 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x13982c0 con 0x15b13c0 2024-05-08T22:39:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b13c0 0x15eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4a3fff700 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b13c0 0x15eab40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4980a34912bee018 server_cookie=98fefe00b5b257c5 in_seq=0 out_seq=0 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 10 client.7453 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 1 -- 172.21.15.179:0/584227869 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x13982c0 con 0x15b13c0 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 10 client.7453 handle_command_reply: tid=0 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 client.7453 shutdown 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 msgr2=0x16353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa49c054c30 tx=0x7fa49c009040 comp rx=0 tx=0).stop 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 shutdown_connections 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x15b13c0 0x15eab40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15b55d0 0x16353a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x15b26f0 0x1634e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa492ffd700 0 client.7453 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 --2- 172.21.15.179:0/584227869 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1588840 0x15880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 >> 172.21.15.179:0/584227869 conn(0x131c5b0 msgr2=0x15b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 shutdown_connections 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.439+0000 7fa4ad688740 1 -- 172.21.15.179:0/584227869 wait complete. 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4ad688740 20 client.7453 trim_cache size 0 max 16384 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "72d56be7-c27e-48ab-8d7d-b3b4329a7ef9": { 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/subdir", 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "72d56be7-c27e-48ab-8d7d-b3b4329a7ef9", 2024-05-08T22:39:10.445 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:39:10.446 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:10.446 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:10.446 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 msgr2=0x7fa488068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 0x7fa488068bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa494000f60 tx=0x7fa494009040 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 msgr2=0x7fa4a406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00cca0 tx=0x7fa49c00a060 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 shutdown_connections 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa488066730 0x7fa488068bf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4a41448b0 0x7fa4a40acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a415c0b0 0x7fa4a406cde0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 --2- 172.21.15.179:0/640539338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4a40cc9a0 0x7fa4a40ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 >> 172.21.15.179:0/640539338 conn(0x7fa4a4070d80 msgr2=0x7fa4a415d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 shutdown_connections 2024-05-08T22:39:10.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:10.443+0000 7fa4a9177700 1 -- 172.21.15.179:0/640539338 wait complete. 2024-05-08T22:39:10.459 INFO:tasks.cephfs.filesystem:scrub status for tag:72d56be7-c27e-48ab-8d7d-b3b4329a7ef9 - {'path': '/subdir', 'tag': '72d56be7-c27e-48ab-8d7d-b3b4329a7ef9', 'options': 'recursive,repair'} 2024-05-08T22:39:22.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:39:22.320 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:39:40.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T22:39:40.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 Processor -- start 2024-05-08T22:39:40.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 -- start start 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780cbfd0 con 0x7fe9780ce650 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780cb3c0 con 0x7fe97815c0b0 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97e67e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780cd970 con 0x7fe9781448b0 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97ce7b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe9780ce650 0x7fe9780cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35560/0 (socket says 172.21.15.179:35560) 2024-05-08T22:39:40.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97ce7b700 1 -- 172.21.15.179:0/3553264519 learned_addr learned my addr 172.21.15.179:0/3553264519 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:40.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648938636 0 0) 0x7fe9780cbfd0 con 0x7fe9780ce650 2024-05-08T22:39:40.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9600035d0 con 0x7fe9780ce650 2024-05-08T22:39:40.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936248727 0 0) 0x7fe9780cb3c0 con 0x7fe97815c0b0 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9780cbfd0 con 0x7fe97815c0b0 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636686529 0 0) 0x7fe9600035d0 con 0x7fe9780ce650 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 msgr2=0x7fe9780ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.586+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 >> v1:172.21.15.195:6789/0 conn(0x7fe97815c0b0 legacy=0x7fe97806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780cb6a0 con 0x7fe9780ce650 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fe968001670 con 0x7fe9780ce650 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 >> v1:172.21.15.179:6789/0 conn(0x7fe9780ce650 legacy=0x7fe9780cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe960003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe960004120 0x7fe960004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe960004db0 0x7fe960005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96000bae0 con 0x7fe960004120 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96000bc20 con 0x7fe960004db0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96000bde0 con 0x7fe9781448b0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe960004120 0x7fe960004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe960003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97de7d700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe960004db0 0x7fe960005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 -- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe960004db0 msgr2=0x7fe960005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe960004db0 0x7fe960005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 -- 172.21.15.179:0/3553264519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe960004120 msgr2=0x7fe960004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe960004120 0x7fe960004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 -- 172.21.15.179:0/3553264519 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0094b0 con 0x7fe9781448b0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe960004120 0x7fe960004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97d67c700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe960003be0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009d50 tx=0x7fe9600058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127045e44838761e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c009ea0 con 0x7fe9781448b0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c00e9a0 con 0x7fe9781448b0 2024-05-08T22:39:40.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe977fff700 1 -- 172.21.15.179:0/3553264519 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00ed50 con 0x7fe9781448b0 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 -- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 msgr2=0x7fe960003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 --2- 172.21.15.179:0/3553264519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe960003be0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009d50 tx=0x7fe9600058a0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97de7d700 1 -- 172.21.15.179:0/3553264519 reap_dead start 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 -- 172.21.15.179:0/3553264519 shutdown_connections 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 -- 172.21.15.179:0/3553264519 >> 172.21.15.179:0/3553264519 conn(0x7fe978070d80 msgr2=0x7fe978071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 -- 172.21.15.179:0/3553264519 shutdown_connections 2024-05-08T22:39:40.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.590+0000 7fe97e67e700 1 -- 172.21.15.179:0/3553264519 wait complete. 2024-05-08T22:39:40.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 Processor -- start 2024-05-08T22:39:40.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 -- start start 2024-05-08T22:39:40.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 0x7fe9780b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe978159220 0x7fe9781596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806cf80 con 0x7fe97815b100 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780ad420 con 0x7fe978159220 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97e67e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815be80 con 0x7fe9781448b0 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 0x7fe9780b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:47978/0 (socket says 172.21.15.179:47978) 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97d67c700 1 -- 172.21.15.179:0/407040143 learned_addr learned my addr 172.21.15.179:0/407040143 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97de7d700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe978159220 0x7fe9781596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 -- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 msgr2=0x7fe9780acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 -- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe978159220 msgr2=0x7fe9781596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe978159220 0x7fe9781596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 -- 172.21.15.179:0/407040143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97815b5e0 con 0x7fe97815b100 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 0x7fe9780b2280 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe968007fd0 tx=0x7fe9680075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe975ffb700 1 -- 172.21.15.179:0/407040143 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968012560 con 0x7fe97815b100 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.594+0000 7fe975ffb700 1 -- 172.21.15.179:0/407040143 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9680126c0 con 0x7fe97815b100 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97815b8c0 con 0x7fe97815b100 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe975ffb700 1 -- 172.21.15.179:0/407040143 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968012a50 con 0x7fe97815b100 2024-05-08T22:39:40.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978159c10 con 0x7fe97815b100 2024-05-08T22:39:40.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe975ffb700 1 -- 172.21.15.179:0/407040143 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe968012bb0 con 0x7fe97815b100 2024-05-08T22:39:40.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe975ffb700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 0x7fe964068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.598+0000 7fe975ffb700 1 -- 172.21.15.179:0/407040143 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe9680842c0 con 0x7fe97815b100 2024-05-08T22:39:40.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 0x7fe964068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 0x7fe964068c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009cf0 tx=0x7fe96c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 Processor -- start 2024-05-08T22:39:40.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 -- start start 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 0x1f08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f355d0 0x1f09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e9b010 con 0x1f326f0 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e28f40 con 0x1f355d0 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e7be30 con 0x1f08840 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97ce7b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 0x1f08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:47984/0 (socket says 172.21.15.179:47984) 2024-05-08T22:39:40.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 -- 172.21.15.179:0/178499616 learned_addr learned my addr 172.21.15.179:0/178499616 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97ce7b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 0x1f08c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39050/0 (socket says 172.21.15.179:39050) 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97de7d700 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f355d0 0x1f09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 -- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f355d0 msgr2=0x1f09810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f355d0 0x1f09810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 -- 172.21.15.179:0/178499616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 msgr2=0x1f08c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 0x1f08c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 -- 172.21.15.179:0/178499616 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ce7a40 con 0x1f08840 2024-05-08T22:39:40.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.602+0000 7fe97d67c700 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00ee80 tx=0x7fe96c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e9b3fd54aa8513 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe95effd700 1 -- 172.21.15.179:0/178499616 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02e070 con 0x1f08840 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe95effd700 1 -- 172.21.15.179:0/178499616 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c021040 con 0x1f08840 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe95effd700 1 -- 172.21.15.179:0/178499616 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004230 con 0x1f08840 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 msgr2=0x1f0dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00ee80 tx=0x7fe96c013000 comp rx=0 tx=0).stop 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- 172.21.15.179:0/178499616 shutdown_connections 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f355d0 0x1f09810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f326f0 0x1f08c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 --2- 172.21.15.179:0/178499616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f0dd90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- 172.21.15.179:0/178499616 >> 172.21.15.179:0/178499616 conn(0x1c9c5b0 msgr2=0x1f34ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- 172.21.15.179:0/178499616 shutdown_connections 2024-05-08T22:39:40.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- 172.21.15.179:0/178499616 wait complete. 2024-05-08T22:39:40.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 -- start start 2024-05-08T22:39:40.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.606+0000 7fe982b8f740 1 Processor -- start 2024-05-08T22:39:40.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f0b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f0bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e77150 con 0x1f08840 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e74eb0 con 0x1f326f0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e328e0 con 0x1f355d0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 10 client.0 _ll_register_callbacks cb 0x1e11670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39060/0 (socket says 172.21.15.179:39060) 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 -- 172.21.15.179:0/126533629 learned_addr learned my addr 172.21.15.179:0/126533629 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97de7d700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f0bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f0b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 -- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 msgr2=0x1f0bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f0bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 -- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 msgr2=0x1f0b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f0b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 -- 172.21.15.179:0/126533629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c0094b0 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97d67c700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe96c034040 tx=0x7fe96c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7c290ee2e74fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02e040 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe96c003ad0 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ca92c0 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c009ab0 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 10 client.7632 fetch_fsmap learned FSMap version 47 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x1e3b290 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ca92c0 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe96c003c90 con 0x1f08840 2024-05-08T22:39:40.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 10 client.7632 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:39:40.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 10 client.7632 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:40.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 20 client.7632 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:40.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f360f0 0x1f5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 4 client.7632 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:40.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1cf3540 con 0x1f360f0 2024-05-08T22:39:40.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f360f0 0x1f5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe957fff700 10 client.7632 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:40.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.610+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f360f0 0x1f5da10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96053a040a5c72d server_cookie=3ba2a17c75a623f5 in_seq=0 out_seq=0 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe957fff700 1 -- 172.21.15.179:0/126533629 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1cf3540 con 0x1f360f0 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe957fff700 10 client.7632 handle_command_reply: tid=0 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 client.7632 shutdown 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 msgr2=0x1f38eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe96c034040 tx=0x7fe96c004770 comp rx=0 tx=0).stop 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 shutdown_connections 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f360f0 0x1f5da10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f0b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f0bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 --2- 172.21.15.179:0/126533629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f38eb0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe957fff700 0 client.7632 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 >> 172.21.15.179:0/126533629 conn(0x1c9c5b0 msgr2=0x1f53a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 shutdown_connections 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 1 -- 172.21.15.179:0/126533629 wait complete. 2024-05-08T22:39:40.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.614+0000 7fe982b8f740 20 client.7632 trim_cache size 0 max 16384 2024-05-08T22:39:40.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.630+0000 7fe982b8f740 1 Processor -- start 2024-05-08T22:39:40.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.630+0000 7fe982b8f740 1 -- start start 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.630+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f6bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.630+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.630+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d20b80 con 0x1f355d0 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e13620 con 0x1f326f0 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e726c0 con 0x1f08840 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39068/0 (socket says 172.21.15.179:39068) 2024-05-08T22:39:40.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 -- 172.21.15.179:0/1521715293 learned_addr learned my addr 172.21.15.179:0/1521715293 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97d67c700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f6bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 -- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 msgr2=0x1f6bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f6bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 -- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 msgr2=0x1f638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 -- 172.21.15.179:0/1521715293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e214c0 con 0x1f355d0 2024-05-08T22:39:40.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97d67c700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f6bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:40.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe97de7d700 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9780b0750 tx=0x7fe970015820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91947a260f6f21db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe95f7fe700 1 -- 172.21.15.179:0/1521715293 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97001b070 con 0x1f355d0 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe95f7fe700 1 -- 172.21.15.179:0/1521715293 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970015e80 con 0x1f355d0 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe95f7fe700 1 -- 172.21.15.179:0/1521715293 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970003c30 con 0x1f355d0 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- 172.21.15.179:0/1521715293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 msgr2=0x1f63e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9780b0750 tx=0x7fe970015820 comp rx=0 tx=0).stop 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- 172.21.15.179:0/1521715293 shutdown_connections 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f355d0 0x1f63e30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 --2- 172.21.15.179:0/1521715293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f08840 0x1f6bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- 172.21.15.179:0/1521715293 >> 172.21.15.179:0/1521715293 conn(0x1c9c5b0 msgr2=0x1f30710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- 172.21.15.179:0/1521715293 shutdown_connections 2024-05-08T22:39:40.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.634+0000 7fe982b8f740 1 -- 172.21.15.179:0/1521715293 wait complete. 2024-05-08T22:39:40.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 -- start start 2024-05-08T22:39:40.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 Processor -- start 2024-05-08T22:39:40.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f68eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f0dbe0 con 0x1f08840 2024-05-08T22:39:40.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f5a100 con 0x1f326f0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f6c580 con 0x1f355d0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe982b8f740 10 client.0 _ll_register_callbacks cb 0x1e2ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39084/0 (socket says 172.21.15.179:39084) 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 -- 172.21.15.179:0/248299504 learned_addr learned my addr 172.21.15.179:0/248299504 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97ce7b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f68eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97de7d700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 -- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 msgr2=0x1f693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 -- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 msgr2=0x1f68eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f68eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 -- 172.21.15.179:0/248299504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9780cbfd0 con 0x1f08840 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe97d67c700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006ac0 tx=0x7fe96c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8183e438edeb3239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe957fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02e040 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.638+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe96c0038d0 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe96c021040 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c00ed30 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 10 client.7638 fetch_fsmap learned FSMap version 47 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x1e15790 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c021040 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe96c003a90 con 0x1f08840 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 10 client.7638 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 10 client.7638 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 20 client.7638 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f6bda0 0x1f69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 4 client.7638 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:39:40.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1d182c0 con 0x1f6bda0 2024-05-08T22:39:40.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f6bda0 0x1f69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97ce7b700 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f6bda0 0x1f69930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2569246f9e0a4067 server_cookie=bf214184c426d0b0 in_seq=0 out_seq=0 2024-05-08T22:39:40.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 10 client.7638 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:40.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 1 -- 172.21.15.179:0/248299504 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1d182c0 con 0x1f6bda0 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 10 client.7638 handle_command_reply: tid=0 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 client.7638 shutdown 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 msgr2=0x1f07b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006ac0 tx=0x7fe96c014000 comp rx=0 tx=0).stop 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 shutdown_connections 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0x1f6bda0 0x1f69930 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1f326f0 0x1f68eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe957fff700 0 client.7638 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1f355d0 0x1f693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 --2- 172.21.15.179:0/248299504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1f08840 0x1f07b20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 >> 172.21.15.179:0/248299504 conn(0x1c9c5b0 msgr2=0x1f0c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 shutdown_connections 2024-05-08T22:39:40.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 1 -- 172.21.15.179:0/248299504 wait complete. 2024-05-08T22:39:40.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe982b8f740 20 client.7638 trim_cache size 0 max 16384 2024-05-08T22:39:40.649 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:40.649 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:39:40.649 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:39:40.649 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 msgr2=0x7fe964068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 0x7fe964068c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009cf0 tx=0x7fe96c008030 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 msgr2=0x7fe9780b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 0x7fe9780b2280 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe968007fd0 tx=0x7fe9680075f0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 shutdown_connections 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe964066760 0x7fe964068c20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe978159220 0x7fe9781596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97815b100 0x7fe9780b2280 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 --2- 172.21.15.179:0/407040143 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9781448b0 0x7fe9780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 >> 172.21.15.179:0/407040143 conn(0x7fe978070d80 msgr2=0x7fe9780718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.642+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 shutdown_connections 2024-05-08T22:39:40.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.646+0000 7fe97e67e700 1 -- 172.21.15.179:0/407040143 wait complete. 2024-05-08T22:39:40.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:39:40.664 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:40.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.774+0000 7f2bed036700 1 Processor -- start 2024-05-08T22:39:40.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.774+0000 7f2bed036700 1 -- start start 2024-05-08T22:39:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2be815d1d0 0x7f2be806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be815d5b0 con 0x7f2be80ce180 2024-05-08T22:39:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be806c5d0 con 0x7f2be815d1d0 2024-05-08T22:39:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80ce630 con 0x7f2be806bb00 2024-05-08T22:39:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2be815d1d0 0x7f2be806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bec835700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2be80ce180 0x7f2be8160c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35576/0 (socket says 172.21.15.179:35576) 2024-05-08T22:39:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bec835700 1 -- 172.21.15.179:0/3524465836 learned_addr learned my addr 172.21.15.179:0/3524465836 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be77fe700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 -- 172.21.15.179:0/3524465836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 msgr2=0x7f2be80cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be80cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 -- 172.21.15.179:0/3524465836 >> v1:172.21.15.179:6789/0 conn(0x7f2be80ce180 legacy=0x7f2be8160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 -- 172.21.15.179:0/3524465836 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be806c010 con 0x7f2be815d1d0 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be6ffd700 1 -- 172.21.15.179:0/3524465836 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859413664 0 0) 0x7f2be0001110 con 0x7f2be80ce180 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be7fff700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be815d1d0 0x7f2be806b5c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009850 tx=0x7f2bdc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c17f5ab07b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be6ffd700 1 -- 172.21.15.179:0/3524465836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011830 con 0x7f2be815d1d0 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be6ffd700 1 -- 172.21.15.179:0/3524465836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011990 con 0x7f2be815d1d0 2024-05-08T22:39:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2be6ffd700 1 -- 172.21.15.179:0/3524465836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011d40 con 0x7f2be815d1d0 2024-05-08T22:39:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- 172.21.15.179:0/3524465836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be815d1d0 msgr2=0x7f2be806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be815d1d0 0x7f2be806b5c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009850 tx=0x7f2bdc009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- 172.21.15.179:0/3524465836 shutdown_connections 2024-05-08T22:39:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be80cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 --2- 172.21.15.179:0/3524465836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be815d1d0 0x7f2be806b5c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- 172.21.15.179:0/3524465836 >> 172.21.15.179:0/3524465836 conn(0x7f2be80d2e70 msgr2=0x7f2be80d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- 172.21.15.179:0/3524465836 shutdown_connections 2024-05-08T22:39:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.778+0000 7f2bed036700 1 -- 172.21.15.179:0/3524465836 wait complete. 2024-05-08T22:39:40.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 Processor -- start 2024-05-08T22:39:40.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 -- start start 2024-05-08T22:39:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be81f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815d1d0 0x7f2be815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8144cb0 con 0x7f2be80ce180 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be815af70 con 0x7f2be815d1d0 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.782+0000 7f2bed036700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80cca90 con 0x7f2be806bb00 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39100/0 (socket says 172.21.15.179:39100) 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 -- 172.21.15.179:0/664326593 learned_addr learned my addr 172.21.15.179:0/664326593 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be7fff700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be81f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2bec835700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815d1d0 0x7f2be815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 -- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 msgr2=0x7f2be81f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be81f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 -- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815d1d0 msgr2=0x7f2be815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815d1d0 0x7f2be815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 -- 172.21.15.179:0/664326593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0094b0 con 0x7f2be80ce180 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be77fe700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800efc0 tx=0x7f2bd8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800cae0 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd800f040 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be8159ed0 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800ce60 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be815a490 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be804e700 con 0x7f2be80ce180 2024-05-08T22:39:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.786+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bd8014480 con 0x7f2be80ce180 2024-05-08T22:39:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.790+0000 7f2be4ff9700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.790+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2bd8084dd0 con 0x7f2be80ce180 2024-05-08T22:39:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.790+0000 7f2be7fff700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:40.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.790+0000 7f2be7fff700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 0x7f2bd4068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:40.798+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2bd8051f00 con 0x7f2be80ce180 2024-05-08T22:39:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.006+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2be8153980 con 0x7f2be80ce180 2024-05-08T22:39:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2be4ff9700 1 -- 172.21.15.179:0/664326593 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f2bd8055520 con 0x7f2be80ce180 2024-05-08T22:39:41.010 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:41.011 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:39:12.053827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 msgr2=0x7f2bd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 0x7f2bd4068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc008030 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 msgr2=0x7f2be815b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800efc0 tx=0x7f2bd8009f80 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 shutdown_connections 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815d1d0 0x7f2be815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2be80ce180 0x7f2be815b410 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 --2- 172.21.15.179:0/664326593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be806bb00 0x7f2be81f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 >> 172.21.15.179:0/664326593 conn(0x7f2be80d2e70 msgr2=0x7f2be80c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 shutdown_connections 2024-05-08T22:39:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.010+0000 7f2bed036700 1 -- 172.21.15.179:0/664326593 wait complete. 2024-05-08T22:39:41.014 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 47 2024-05-08T22:39:41.025 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-05-08T22:39:41.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 Processor -- start 2024-05-08T22:39:41.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 -- start start 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 0x7f07280dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07280e0160 0x7f07280067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728006d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280d6b70 con 0x7f07280e0160 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280e0bc0 con 0x7f07280507d0 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f073495c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07280e0600 con 0x7f072804ff30 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f072e59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07280e0160 0x7f07280067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f072e59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07280e0160 0x7f07280067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39116/0 (socket says 172.21.15.179:39116) 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.138+0000 7f072e59c700 1 -- 172.21.15.179:0/3013617632 learned_addr learned my addr 172.21.15.179:0/3013617632 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072f59e700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728006d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072ed9d700 1 --2- 172.21.15.179:0/3013617632 >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 0x7f07280dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 -- 172.21.15.179:0/3013617632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 msgr2=0x7f0728006d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728006d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 -- 172.21.15.179:0/3013617632 >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 msgr2=0x7f07280dfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 --2- 172.21.15.179:0/3013617632 >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 0x7f07280dfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 -- 172.21.15.179:0/3013617632 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280e08e0 con 0x7f07280e0160 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072f59e700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728006d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072ed9d700 1 --2- 172.21.15.179:0/3013617632 >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 0x7f07280dfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072e59c700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07280067d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f072800da10 tx=0x7f071800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828264e1f352df18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072dd9b700 1 -- 172.21.15.179:0/3013617632 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800dd90 con 0x7f07280e0160 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072dd9b700 1 -- 172.21.15.179:0/3013617632 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718010470 con 0x7f07280e0160 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f072dd9b700 1 -- 172.21.15.179:0/3013617632 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718010800 con 0x7f07280e0160 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- 172.21.15.179:0/3013617632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 msgr2=0x7f07280067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07280067d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f072800da10 tx=0x7f071800be90 comp rx=0 tx=0).stop 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- 172.21.15.179:0/3013617632 shutdown_connections 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728006d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 --2- 172.21.15.179:0/3013617632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07280067d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 --2- 172.21.15.179:0/3013617632 >> v2:172.21.15.195:3300/0 conn(0x7f07280507d0 0x7f07280dfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- 172.21.15.179:0/3013617632 >> 172.21.15.179:0/3013617632 conn(0x7f0728054b70 msgr2=0x7f0728054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- 172.21.15.179:0/3013617632 shutdown_connections 2024-05-08T22:39:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- 172.21.15.179:0/3013617632 wait complete. 2024-05-08T22:39:41.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 Processor -- start 2024-05-08T22:39:41.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.142+0000 7f073495c700 1 -- start start 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728039380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07280507d0 0x7f07280398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728176080 con 0x7f07280e0160 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728175da0 con 0x7f07280507d0 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0728175460 con 0x7f072804ff30 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072ed9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728039380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39122/0 (socket says 172.21.15.179:39122) 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 -- 172.21.15.179:0/3148024732 learned_addr learned my addr 172.21.15.179:0/3148024732 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072e59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07280507d0 0x7f07280398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 -- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 msgr2=0x7f0728039380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728039380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 -- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07280507d0 msgr2=0x7f07280398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07280507d0 0x7f07280398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 -- 172.21.15.179:0/3148024732 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071800b810 con 0x7f07280e0160 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f072f59e700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f073002ce60 tx=0x7f073004f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f07277fe700 1 -- 172.21.15.179:0/3148024732 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300669e0 con 0x7f07280e0160 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f07277fe700 1 -- 172.21.15.179:0/3148024732 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730066b60 con 0x7f07280e0160 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f07277fe700 1 -- 172.21.15.179:0/3148024732 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730066f50 con 0x7f07280e0160 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0728175740 con 0x7f07280e0160 2024-05-08T22:39:41.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.146+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0728037330 con 0x7f07280e0160 2024-05-08T22:39:41.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f07277fe700 1 -- 172.21.15.179:0/3148024732 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0730066150 con 0x7f07280e0160 2024-05-08T22:39:41.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f07277fe700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 0x7f0710068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f07277fe700 1 -- 172.21.15.179:0/3148024732 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f073008f030 con 0x7f07280e0160 2024-05-08T22:39:41.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f072ed9d700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 0x7f0710068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f072ed9d700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 0x7f0710068bf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0720006fd0 tx=0x7f0720008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f073840c740 1 Processor -- start 2024-05-08T22:39:41.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.150+0000 7f073840c740 1 -- start start 2024-05-08T22:39:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xebc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xe71180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xddf810 con 0xebc660 2024-05-08T22:39:41.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xddcfa0 con 0xe00520 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xdd8890 con 0xebca40 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39138/0 (socket says 172.21.15.179:39138) 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 -- 172.21.15.179:0/1838456079 learned_addr learned my addr 172.21.15.179:0/1838456079 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072ed9d700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xebc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072f59e700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xe71180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 -- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 msgr2=0xe71180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xe71180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 -- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 msgr2=0xebc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xebc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 -- 172.21.15.179:0/1838456079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc4aba0 con 0xebc660 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072f59e700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xe71180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072ed9d700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xebc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:41.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f072e59c700 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f071800bfd0 tx=0x7f0718013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad02702244ba7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f0724ff9700 1 -- 172.21.15.179:0/1838456079 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071801f040 con 0xebc660 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f0724ff9700 1 -- 172.21.15.179:0/1838456079 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718004260 con 0xebc660 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f0724ff9700 1 -- 172.21.15.179:0/1838456079 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718004610 con 0xebc660 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- 172.21.15.179:0/1838456079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 msgr2=0xe70c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f071800bfd0 tx=0x7f0718013000 comp rx=0 tx=0).stop 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- 172.21.15.179:0/1838456079 shutdown_connections 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xe71180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe70c40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 --2- 172.21.15.179:0/1838456079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xebc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- 172.21.15.179:0/1838456079 >> 172.21.15.179:0/1838456079 conn(0xbff710 msgr2=0xe99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- 172.21.15.179:0/1838456079 shutdown_connections 2024-05-08T22:39:41.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.154+0000 7f073840c740 1 -- 172.21.15.179:0/1838456079 wait complete. 2024-05-08T22:39:41.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- start start 2024-05-08T22:39:41.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 Processor -- start 2024-05-08T22:39:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xebb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xec2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd9bd80 con 0xebc660 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xd9ba20 con 0xebca40 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xccb770 con 0xe00520 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 10 client.0 _ll_register_callbacks cb 0xdd1f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39152/0 (socket says 172.21.15.179:39152) 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 -- 172.21.15.179:0/769141116 learned_addr learned my addr 172.21.15.179:0/769141116 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072ed9d700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xebb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072f59e700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xec2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 -- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 msgr2=0xebb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xebb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 -- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 msgr2=0xec2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xec2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 -- 172.21.15.179:0/769141116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07280e08e0 con 0xebc660 2024-05-08T22:39:41.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f072e59c700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0718004600 tx=0x7f07180047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de075cb6f295e622 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071801f050 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f07180109f0 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc0c420 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0718010bb0 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 10 client.7659 fetch_fsmap learned FSMap version 47 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0xd9e910 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc0c420 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f071800bbb0 con 0xebc660 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 10 client.7659 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 10 client.7659 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 20 client.7659 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xe90190 0xe95290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 4 client.7659 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:39:41.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.158+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc566a0 con 0xe90190 2024-05-08T22:39:41.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f072ed9d700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xe90190 0xe95290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f072ed9d700 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xe90190 0xe95290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59baf3fc9ae660c6 server_cookie=e492cde631cff314 in_seq=0 out_seq=0 2024-05-08T22:39:41.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f070dffb700 10 client.7659 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:41.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f070dffb700 1 -- 172.21.15.179:0/769141116 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xc566a0 con 0xe90190 2024-05-08T22:39:41.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f070dffb700 10 client.7659 handle_command_reply: tid=0 2024-05-08T22:39:41.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 client.7659 shutdown 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 msgr2=0xe93110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0718004600 tx=0x7f07180047e0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 shutdown_connections 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xe90190 0xe95290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xec2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xe93110 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f070dffb700 0 client.7659 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 --2- 172.21.15.179:0/769141116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xebb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 >> 172.21.15.179:0/769141116 conn(0xbff710 msgr2=0xe99af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 shutdown_connections 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 1 -- 172.21.15.179:0/769141116 wait complete. 2024-05-08T22:39:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.162+0000 7f073840c740 20 client.7659 trim_cache size 0 max 16384 2024-05-08T22:39:41.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 Processor -- start 2024-05-08T22:39:41.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 -- start start 2024-05-08T22:39:41.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xf17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xecae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xddc890 con 0xebc660 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xdff9a0 con 0xebca40 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xdd60a0 con 0xe00520 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39160/0 (socket says 172.21.15.179:39160) 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 -- 172.21.15.179:0/2079149711 learned_addr learned my addr 172.21.15.179:0/2079149711 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072f59e700 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xecae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072ed9d700 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xf17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 -- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 msgr2=0xf17f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xf17f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 -- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 msgr2=0xecae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xecae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 -- 172.21.15.179:0/2079149711 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd84b40 con 0xebc660 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f072e59c700 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f07180046e0 tx=0x7f071802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7be0afa3c90da1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f07257fa700 1 -- 172.21.15.179:0/2079149711 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071801f070 con 0xebc660 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f07257fa700 1 -- 172.21.15.179:0/2079149711 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071800bbb0 con 0xebc660 2024-05-08T22:39:41.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.178+0000 7f07257fa700 1 -- 172.21.15.179:0/2079149711 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180109f0 con 0xebc660 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- 172.21.15.179:0/2079149711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 msgr2=0xf18480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f07180046e0 tx=0x7f071802b000 comp rx=0 tx=0).stop 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- 172.21.15.179:0/2079149711 shutdown_connections 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xebca40 0xecae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xf18480 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- 172.21.15.179:0/2079149711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xe00520 0xf17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- 172.21.15.179:0/2079149711 >> 172.21.15.179:0/2079149711 conn(0xbff710 msgr2=0xec0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- 172.21.15.179:0/2079149711 shutdown_connections 2024-05-08T22:39:41.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- 172.21.15.179:0/2079149711 wait complete. 2024-05-08T22:39:41.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- start start 2024-05-08T22:39:41.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 Processor -- start 2024-05-08T22:39:41.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xeb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xeb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xda76b0 con 0xebc660 2024-05-08T22:39:41.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xdd7720 con 0xe00520 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xec2800 con 0xebca40 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f073840c740 10 client.0 _ll_register_callbacks cb 0xdd7460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f072e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39176/0 (socket says 172.21.15.179:39176) 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f072e59c700 1 -- 172.21.15.179:0/2062028145 learned_addr learned my addr 172.21.15.179:0/2062028145 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f072ed9d700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xeb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.182+0000 7f072f59e700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xeb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 -- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 msgr2=0xeb82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xeb82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 -- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 msgr2=0xeb9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xeb9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 -- 172.21.15.179:0/2062028145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd9e910 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072e59c700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0718011bd0 tx=0x7f0718008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d1c075c4218056 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071801f040 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0718000e40 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0718004260 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f071800bd20 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 10 client.7665 fetch_fsmap learned FSMap version 47 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0xd78c20 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718004260 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0718017020 con 0xebc660 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 10 client.7665 fetch_fsmap finished waiting for FSMap version 47 2024-05-08T22:39:41.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 10 client.7665 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:39:41.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 20 client.7665 populate_metadata read hostname 'smithi179' 2024-05-08T22:39:41.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xeb89e0 0xeda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 4 client.7665 mds_command: new command op to 56354 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:39:41.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 --> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0xc7b420 con 0xeb89e0 2024-05-08T22:39:41.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072ed9d700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xeb89e0 0xeda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 10 client.7665 ms_handle_connect on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f072ed9d700 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xeb89e0 0xeda550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10bac2b8e7a1474a server_cookie=d2d95f962ec2cc20 in_seq=0 out_seq=0 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 1 -- 172.21.15.179:0/2062028145 <== mds.0 v2:172.21.15.195:6835/1691042534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0xc7b420 con 0xeb89e0 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f070dffb700 10 client.7665 handle_command_reply: tid=0 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 client.7665 shutdown 2024-05-08T22:39:41.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 msgr2=0xebacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.186+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0718011bd0 tx=0x7f0718008040 comp rx=0 tx=0).stop 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 shutdown_connections 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xebca40 0xeb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xebc660 0xebacd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:6835/1691042534,v1:172.21.15.195:6837/1691042534] conn(0xeb89e0 0xeda550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 --2- 172.21.15.179:0/2062028145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xe00520 0xeb9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f070dffb700 0 client.7665 ms_handle_reset on v2:172.21.15.195:6835/1691042534 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 >> 172.21.15.179:0/2062028145 conn(0xbff710 msgr2=0xecb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 shutdown_connections 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 1 -- 172.21.15.179:0/2062028145 wait complete. 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073840c740 20 client.7665 trim_cache size 0 max 16384 2024-05-08T22:39:41.193 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:41.194 INFO:teuthology.orchestra.run.smithi179.stdout:[] 2024-05-08T22:39:41.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 msgr2=0x7f0710068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 0x7f0710068bf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0720006fd0 tx=0x7f0720008040 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 msgr2=0x7f07281745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f073002ce60 tx=0x7f073004f700 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 shutdown_connections 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0710066730 0x7f0710068bf0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07280507d0 0x7f07280398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07280e0160 0x7f07281745b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 --2- 172.21.15.179:0/3148024732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f072804ff30 0x7f0728039380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 >> 172.21.15.179:0/3148024732 conn(0x7f0728054b70 msgr2=0x7f07280d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 shutdown_connections 2024-05-08T22:39:41.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.190+0000 7f073495c700 1 -- 172.21.15.179:0/3148024732 wait complete. 2024-05-08T22:39:41.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:39:41.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 Processor -- start 2024-05-08T22:39:41.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 -- start start 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07c40cab60 0x7f07c4164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c406eb20 con 0x7f07c406ac90 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c406df10 con 0x7f07c415df60 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07ca5aa700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c406e1f0 con 0x7f07c40cab60 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f07c406ac90 0x7f07c40ca450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35590/0 (socket says 172.21.15.179:35590) 2024-05-08T22:39:41.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c8b47700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07c40cab60 0x7f07c4164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c37fe700 1 -- 172.21.15.179:0/4098725191 learned_addr learned my addr 172.21.15.179:0/4098725191 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c8b47700 1 -- 172.21.15.179:0/4098725191 >> v1:172.21.15.195:6789/0 conn(0x7f07c415df60 legacy=0x7f07c415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c2ffd700 1 -- 172.21.15.179:0/4098725191 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785781390 0 0) 0x7f07c406eb20 con 0x7f07c406ac90 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c8b47700 1 -- 172.21.15.179:0/4098725191 >> v1:172.21.15.179:6789/0 conn(0x7f07c406ac90 legacy=0x7f07c40ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c8b47700 1 -- 172.21.15.179:0/4098725191 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c406e4d0 con 0x7f07c40cab60 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c8b47700 1 --2- 172.21.15.179:0/4098725191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c40cab60 0x7f07c4164cc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009850 tx=0x7f07bc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624796050a779980 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c2ffd700 1 -- 172.21.15.179:0/4098725191 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc004230 con 0x7f07c40cab60 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c2ffd700 1 -- 172.21.15.179:0/4098725191 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc004390 con 0x7f07c40cab60 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.322+0000 7f07c2ffd700 1 -- 172.21.15.179:0/4098725191 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0036c0 con 0x7f07c40cab60 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- 172.21.15.179:0/4098725191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c40cab60 msgr2=0x7f07c4164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/4098725191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c40cab60 0x7f07c4164cc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009850 tx=0x7f07bc009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- 172.21.15.179:0/4098725191 shutdown_connections 2024-05-08T22:39:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/4098725191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c40cab60 0x7f07c4164cc0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- 172.21.15.179:0/4098725191 >> 172.21.15.179:0/4098725191 conn(0x7f07c40cb660 msgr2=0x7f07c40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- 172.21.15.179:0/4098725191 shutdown_connections 2024-05-08T22:39:41.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- 172.21.15.179:0/4098725191 wait complete. 2024-05-08T22:39:41.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 Processor -- start 2024-05-08T22:39:41.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.326+0000 7f07ca5aa700 1 -- start start 2024-05-08T22:39:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07c40cab60 0x7f07c41f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07c415df60 0x7f07c40c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c40c45f0 con 0x7f07c406ac90 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41f8450 con 0x7f07c40cab60 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c40c2ee0 con 0x7f07c415df60 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39194/0 (socket says 172.21.15.179:39194) 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 -- 172.21.15.179:0/49914950 learned_addr learned my addr 172.21.15.179:0/49914950 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c8b47700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07c415df60 0x7f07c40c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c37fe700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07c40cab60 0x7f07c41f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 -- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07c415df60 msgr2=0x7f07c40c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07c415df60 0x7f07c40c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 -- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07c40cab60 msgr2=0x7f07c41f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07c40cab60 0x7f07c41f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 -- 172.21.15.179:0/49914950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0094b0 con 0x7f07c406ac90 2024-05-08T22:39:41.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c3fff700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00c4e0 tx=0x7f07ac00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011790 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac0118f0 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011ca0 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c41f7e90 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c40c2b70 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.330+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c4067770 con 0x7f07c406ac90 2024-05-08T22:39:41.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.334+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f07ac005080 con 0x7f07c406ac90 2024-05-08T22:39:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.334+0000 7f07c0ff9700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 0x7f07a4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.334+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f07ac083b60 con 0x7f07c406ac90 2024-05-08T22:39:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.334+0000 7f07c37fe700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 0x7f07a4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.334+0000 7f07c37fe700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 0x7f07a4068c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f07c4158690 tx=0x7f07b4001aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.338+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f07ac050d20 con 0x7f07c406ac90 2024-05-08T22:39:41.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.546+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f07c415db10 con 0x7f07c406ac90 2024-05-08T22:39:41.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.546+0000 7f07c0ff9700 1 -- 172.21.15.179:0/49914950 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f07ac054340 con 0x7f07c406ac90 2024-05-08T22:39:41.550 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:41.550 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:39:41.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 msgr2=0x7f07a4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 0x7f07a4068c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f07c4158690 tx=0x7f07b4001aa0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 msgr2=0x7f07c4155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00c4e0 tx=0x7f07ac00c7f0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 shutdown_connections 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f07a4066760 0x7f07a4068c20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07c40cab60 0x7f07c41f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07c415df60 0x7f07c40c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 --2- 172.21.15.179:0/49914950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07c406ac90 0x7f07c4155a70 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 >> 172.21.15.179:0/49914950 conn(0x7f07c40cb660 msgr2=0x7f07c415c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 shutdown_connections 2024-05-08T22:39:41.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.550+0000 7f07ca5aa700 1 -- 172.21.15.179:0/49914950 wait complete. 2024-05-08T22:39:41.565 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:39:41.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:39:41.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 Processor -- start 2024-05-08T22:39:41.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 -- start start 2024-05-08T22:39:41.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40cc420 0x7f88a40cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88a40ce500 0x7f88a40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40cb6f0 0x7f88a415e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a40cbba0 con 0x7f88a40cb6f0 2024-05-08T22:39:41.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a415c2f0 con 0x7f88a40ce500 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88aa31b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a406abb0 con 0x7f88a40cc420 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40cb6f0 0x7f88a415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40cb6f0 0x7f88a415e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39208/0 (socket says 172.21.15.179:39208) 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 -- 172.21.15.179:0/3982846198 learned_addr learned my addr 172.21.15.179:0/3982846198 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a8b18700 1 --2- 172.21.15.179:0/3982846198 >> v2:172.21.15.195:3300/0 conn(0x7f88a40ce500 0x7f88a40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9319700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40cc420 0x7f88a40cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 -- 172.21.15.179:0/3982846198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40cc420 msgr2=0x7f88a40cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40cc420 0x7f88a40cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 -- 172.21.15.179:0/3982846198 >> v2:172.21.15.195:3300/0 conn(0x7f88a40ce500 msgr2=0x7f88a40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 --2- 172.21.15.179:0/3982846198 >> v2:172.21.15.195:3300/0 conn(0x7f88a40ce500 0x7f88a40cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 -- 172.21.15.179:0/3982846198 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a415b770 con 0x7f88a40cb6f0 2024-05-08T22:39:41.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a9b1a700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a415e930 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c770 tx=0x7f889c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d5994ba734034c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.674+0000 7f88a3fff700 1 -- 172.21.15.179:0/3982846198 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c011830 con 0x7f88a40cb6f0 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88a3fff700 1 -- 172.21.15.179:0/3982846198 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c011990 con 0x7f88a40cb6f0 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88a3fff700 1 -- 172.21.15.179:0/3982846198 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c011d40 con 0x7f88a40cb6f0 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 -- 172.21.15.179:0/3982846198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 msgr2=0x7f88a415e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a415e930 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c770 tx=0x7f889c009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 -- 172.21.15.179:0/3982846198 shutdown_connections 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a415e930 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 --2- 172.21.15.179:0/3982846198 >> v2:172.21.15.195:3300/0 conn(0x7f88a40ce500 0x7f88a40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 --2- 172.21.15.179:0/3982846198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40cc420 0x7f88a40cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 -- 172.21.15.179:0/3982846198 >> 172.21.15.179:0/3982846198 conn(0x7f88a40ce9d0 msgr2=0x7f88a40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 -- 172.21.15.179:0/3982846198 shutdown_connections 2024-05-08T22:39:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 -- 172.21.15.179:0/3982846198 wait complete. 2024-05-08T22:39:41.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.678+0000 7f88aa31b700 1 Processor -- start 2024-05-08T22:39:41.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- start start 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a40cc420 0x7f88a40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40ce500 0x7f88a41472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4147820 con 0x7f88a40cb6f0 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40b39e0 con 0x7f88a40cc420 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40b1ae0 con 0x7f88a40ce500 2024-05-08T22:39:41.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39212/0 (socket says 172.21.15.179:39212) 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 -- 172.21.15.179:0/770889243 learned_addr learned my addr 172.21.15.179:0/770889243 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a8b18700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a40cc420 0x7f88a40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9b1a700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40ce500 0x7f88a41472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 -- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40ce500 msgr2=0x7f88a41472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40ce500 0x7f88a41472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 -- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a40cc420 msgr2=0x7f88a40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a40cc420 0x7f88a40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 -- 172.21.15.179:0/770889243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c009570 con 0x7f88a40cb6f0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a9319700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e960 tx=0x7f888c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0096c0 con 0x7f88a40cb6f0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c013680 con 0x7f88a40cb6f0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a40b1dc0 con 0x7f88a40cb6f0 2024-05-08T22:39:41.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c013a10 con 0x7f88a40cb6f0 2024-05-08T22:39:41.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.682+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a40b2380 con 0x7f88a40cb6f0 2024-05-08T22:39:41.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a4067740 con 0x7f88a40cb6f0 2024-05-08T22:39:41.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f888c013b90 con 0x7f88a40cb6f0 2024-05-08T22:39:41.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88a1ffb700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 0x7f8884068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:41.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f888c084330 con 0x7f88a40cb6f0 2024-05-08T22:39:41.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88a8b18700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 0x7f8884068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:41.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.686+0000 7f88a8b18700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 0x7f8884068bf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f88a41f6160 tx=0x7f889400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:41.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.694+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f888c051460 con 0x7f88a40cb6f0 2024-05-08T22:39:41.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.906+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f88a40b18b0 con 0x7f88a40cb6f0 2024-05-08T22:39:41.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.906+0000 7f88a1ffb700 1 -- 172.21.15.179:0/770889243 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f888c054a80 con 0x7f88a40cb6f0 2024-05-08T22:39:41.910 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 msgr2=0x7f8884068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 0x7f8884068bf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f88a41f6160 tx=0x7f889400b040 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 msgr2=0x7f88a40b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e960 tx=0x7f888c00ec70 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 shutdown_connections 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8884066730 0x7f8884068bf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a40cc420 0x7f88a40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a40ce500 0x7f88a41472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 --2- 172.21.15.179:0/770889243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40cb6f0 0x7f88a40b3400 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 >> 172.21.15.179:0/770889243 conn(0x7f88a40ce9d0 msgr2=0x7f88a415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 shutdown_connections 2024-05-08T22:39:41.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:41.910+0000 7f88aa31b700 1 -- 172.21.15.179:0/770889243 wait complete. 2024-05-08T22:39:41.924 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:39:41.924 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:39:41.925 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:39:41.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:41.925 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.931 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42189sec preferred_lft 42189sec 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.932 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:39:41.933 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:39:41.934 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:39:41.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:41.934 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:39:41.948 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:39:41.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:41.949 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:39:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:39:41.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:39:41.959 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:39:42.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:42.040 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:39:42.046 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:39:42.046 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:39:42.047 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:39:42.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:42.047 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:39:42.061 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:39:42.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:39:42.062 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:42.067 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:39:42.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:39:42.068 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:42.073 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:39:42.129 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:39:42.129 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:39:42.145 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:39:42.145 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:39:42.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:42.146 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:39:42.151 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:39:42.151 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:39:42.195 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:39:42.212 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:39:42.212 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:39:42.222 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:39:42.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:39:42.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:42.224 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:39:42.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:39:42.282+0000 7f25b600c640 -1 init, newargv = 0x560c94f59ec0 newargc=15 2024-05-08T22:39:42.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[44496]: starting ceph client 2024-05-08T22:39:42.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[44496]: starting fuse 2024-05-08T22:39:43.330 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:39:43.348 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:39:43.348 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:39:43.405 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:39:43.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:39:43.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:43.406 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:39:43.412 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:39:43.412 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:39:43.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:39:43.412 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:39:43.412 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:39:43.412 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:39:43.413 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:39:43.414 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:39:43.497 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.44496.asok 2024-05-08T22:39:43.681 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44496.asok 2024-05-08T22:39:43.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:43.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44496.asok status 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "44496", 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:39:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 7689, 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 7689 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:39:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1171868436 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1171868436 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.7689 192.168.0.1:0/1171868436", 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/1171868436", 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:39:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 47, 2024-05-08T22:39:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 84, 2024-05-08T22:39:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:39:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:39:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:39:43.831 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:43.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:43.838 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:39:43.838 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:43.845 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:39:43.846 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:43.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:43.846 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:43.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:43.862 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-08T22:39:43.919 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:39:43.920 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-08T22:39:43.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.958+0000 7fdcf8ae6140 1 Processor -- start 2024-05-08T22:39:43.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.958+0000 7fdcf8ae6140 1 -- start start 2024-05-08T22:39:43.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685c1cac0 0x555685d499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555685b7c2f0 con 0x555685cc0740 2024-05-08T22:39:43.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555685b388e0 con 0x555685be9220 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555685b38580 con 0x555685c1cac0 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf661b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39234/0 (socket says 172.21.15.179:39234) 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/1210226926 learned_addr learned my addr 172.21.15.179:0/1210226926 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf661b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:36056/0 (socket says 172.21.15.179:36056) 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf5e1a700 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685c1cac0 0x555685d499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685c1cac0 msgr2=0x555685d499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685c1cac0 0x555685d499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 msgr2=0x555685cbdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/1210226926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0005de0 con 0x555685cc0740 2024-05-08T22:39:43.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf661b700 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:43.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdce800a7d0 tx=0x7fdce800aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e104ef9c921853a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:43.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf4e18700 1 -- 172.21.15.179:0/1210226926 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800f070 con 0x555685cc0740 2024-05-08T22:39:43.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf4e18700 1 -- 172.21.15.179:0/1210226926 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce8007600 con 0x555685cc0740 2024-05-08T22:39:43.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf4e18700 1 -- 172.21.15.179:0/1210226926 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8007970 con 0x555685cc0740 2024-05-08T22:39:43.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/1210226926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 msgr2=0x555685d53400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdce800a7d0 tx=0x7fdce800aae0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/1210226926 shutdown_connections 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685cc0740 0x555685d53400 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685c1cac0 0x555685d499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/1210226926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685be9220 0x555685cbdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/1210226926 >> 172.21.15.179:0/1210226926 conn(0x555685bef780 msgr2=0x555685cbe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/1210226926 shutdown_connections 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.962+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/1210226926 wait complete. 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 Processor -- start 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- start start 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685be9220 0x555685d48500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 0x555685d4b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685cc0740 0x555685c32610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555685b0e8d0 con 0x555685c1cac0 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555685d4bdd0 con 0x555685cc0740 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555685d47e30 con 0x555685be9220 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 0x555685d4b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 0x555685d4b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39244/0 (socket says 172.21.15.179:39244) 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf6e1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685cc0740 0x555685c32610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 -- 172.21.15.179:0/2276541801 learned_addr learned my addr 172.21.15.179:0/2276541801 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf661b700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685be9220 0x555685d48500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 -- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685be9220 msgr2=0x555685d48500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555685be9220 0x555685d48500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 -- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685cc0740 msgr2=0x555685c32610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685cc0740 0x555685c32610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 -- 172.21.15.179:0/2276541801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555685b40ca0 con 0x555685c1cac0 2024-05-08T22:39:43.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555685cc0740 0x555685c32610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:43.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf5e1a700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 0x555685d4b890 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdce000d420 tx=0x7fdce000d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcef7fe700 1 -- 172.21.15.179:0/2276541801 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0013480 con 0x555685c1cac0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcef7fe700 1 -- 172.21.15.179:0/2276541801 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce00135e0 con 0x555685c1cac0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce800a420 con 0x555685c1cac0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcef7fe700 1 -- 172.21.15.179:0/2276541801 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0013950 con 0x555685c1cac0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.966+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555685b26b00 con 0x555685c1cac0 2024-05-08T22:39:43.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcef7fe700 1 -- 172.21.15.179:0/2276541801 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdce0013ab0 con 0x555685c1cac0 2024-05-08T22:39:43.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcef7fe700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdcd8065060 0x7fdcd8067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcef7fe700 1 -- 172.21.15.179:0/2276541801 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fdce00824f0 con 0x555685c1cac0 2024-05-08T22:39:43.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x555685cbbaf0 0x555685bec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:43.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) v8 -- 0x555685cbbf00 con 0x555685cbbaf0 2024-05-08T22:39:43.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcf661b700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdcd8065060 0x7fdcd8067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x555685cbbaf0 0x555685bec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:43.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.970+0000 7fdcf6e1c700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x555685cbbaf0 0x555685bec3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:43.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf661b700 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdcd8065060 0x7fdcd8067520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000a4f0 tx=0x7fdcf000f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:43.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/2276541801 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v84'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdce8007a10 con 0x555685cbbaf0 2024-05-08T22:39:43.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x555685cbbaf0 msgr2=0x555685bec3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x555685cbbaf0 0x555685bec3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:43.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdcd8065060 msgr2=0x7fdcd8067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdcd8065060 0x7fdcd8067520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000a4f0 tx=0x7fdcf000f000 comp rx=0 tx=0).stop 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 msgr2=0x555685d4b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 --2- 172.21.15.179:0/2276541801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555685c1cac0 0x555685d4b890 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdce000d420 tx=0x7fdce000d730 comp rx=0 tx=0).stop 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf6e1c700 1 -- 172.21.15.179:0/2276541801 reap_dead start 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 shutdown_connections 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 >> 172.21.15.179:0/2276541801 conn(0x555685bef780 msgr2=0x555685d499e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 shutdown_connections 2024-05-08T22:39:43.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:43.974+0000 7fdcf8ae6140 1 -- 172.21.15.179:0/2276541801 wait complete. 2024-05-08T22:39:43.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:39:43.982 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:39:43.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:39:44.020 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:39:44.021 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:39:44.021 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:44.027 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:44.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 Processor -- start 2024-05-08T22:39:44.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 -- start start 2024-05-08T22:39:44.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f973006ac90 0x7f97300ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973006eb20 con 0x7f973015df60 2024-05-08T22:39:44.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973006df10 con 0x7f97300ca990 2024-05-08T22:39:44.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9737a19700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300cae40 con 0x7f973006ac90 2024-05-08T22:39:44.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f97357b5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f973015df60 0x7f973015e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35594/0 (socket says 172.21.15.179:35594) 2024-05-08T22:39:44.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f97357b5700 1 -- 172.21.15.179:0/514156528 learned_addr learned my addr 172.21.15.179:0/514156528 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:44.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 --2- 172.21.15.179:0/514156528 >> v2:172.21.15.195:3300/0 conn(0x7f973006ac90 0x7f97300ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014490570 0 0) 0x7f973006eb20 con 0x7f973015df60 2024-05-08T22:39:44.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97180035d0 con 0x7f973015df60 2024-05-08T22:39:44.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457104260 0 0) 0x7f97180035d0 con 0x7f973015df60 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 >> v2:172.21.15.195:3300/0 conn(0x7f973006ac90 msgr2=0x7f97300ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 --2- 172.21.15.179:0/514156528 >> v2:172.21.15.195:3300/0 conn(0x7f973006ac90 0x7f97300ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 >> v1:172.21.15.195:6789/0 conn(0x7f97300ca990 legacy=0x7f97300cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973006e1f0 con 0x7f973015df60 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f97200019f0 con 0x7f973015df60 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 >> v1:172.21.15.179:6789/0 conn(0x7f973015df60 legacy=0x7f973015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9718003a20 0x7f9718003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f9718004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9718005230 0x7f971800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97300cae40 con 0x7f9718005230 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971800c0d0 con 0x7f9718004870 2024-05-08T22:39:44.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971800c290 con 0x7f9718003a20 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f9718004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f97357b5700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9718003a20 0x7f9718003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9735fb6700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9718005230 0x7f971800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 -- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9718003a20 msgr2=0x7f9718003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9718003a20 0x7f9718003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 -- 172.21.15.179:0/514156528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9718005230 msgr2=0x7f971800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9718005230 0x7f971800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 -- 172.21.15.179:0/514156528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180035d0 con 0x7f9718004870 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9735fb6700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9718005230 0x7f971800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9735fb6700 1 -- 172.21.15.179:0/514156528 reap_dead start 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f9734fb4700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f9718004cf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9730067cf0 tx=0x7f97180058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6c6fc1f28af2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972400e6c0 con 0x7f9718004870 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972400e820 con 0x7f9718004870 2024-05-08T22:39:44.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.190+0000 7f972ffff700 1 -- 172.21.15.179:0/514156528 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972400eb90 con 0x7f9718004870 2024-05-08T22:39:44.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 msgr2=0x7f9718004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f9718004cf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9730067cf0 tx=0x7f97180058a0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- 172.21.15.179:0/514156528 shutdown_connections 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 --2- 172.21.15.179:0/514156528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f9718004cf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- 172.21.15.179:0/514156528 >> 172.21.15.179:0/514156528 conn(0x7f97300cb660 msgr2=0x7f97300cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- 172.21.15.179:0/514156528 shutdown_connections 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- 172.21.15.179:0/514156528 wait complete. 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 Processor -- start 2024-05-08T22:39:44.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- start start 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f97300b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300af4f0 0x7f97300c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730158830 con 0x7f973015c050 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97300afd50 con 0x7f9718004870 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9737a19700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9730158550 con 0x7f97300af4f0 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9735fb6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.194+0000 7f9735fb6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39264/0 (socket says 172.21.15.179:39264) 2024-05-08T22:39:44.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 -- 172.21.15.179:0/3915698508 learned_addr learned my addr 172.21.15.179:0/3915698508 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:44.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9734fb4700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300af4f0 0x7f97300c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f97357b5700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f97300b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 -- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300af4f0 msgr2=0x7f97300c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300af4f0 0x7f97300c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 -- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 msgr2=0x7f97300b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f97300b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 -- 172.21.15.179:0/3915698508 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300c3850 con 0x7f973015c050 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9735fb6700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f973006d3b0 tx=0x7f972800f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800fe50 con 0x7f973015c050 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728005320 con 0x7f973015c050 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97300c3b30 con 0x7f973015c050 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728006880 con 0x7f973015c050 2024-05-08T22:39:44.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97300c7a20 con 0x7f973015c050 2024-05-08T22:39:44.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9730067770 con 0x7f973015c050 2024-05-08T22:39:44.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.198+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9728006a00 con 0x7f973015c050 2024-05-08T22:39:44.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.202+0000 7f972dffb700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 0x7f971c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.202+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f97280833c0 con 0x7f973015c050 2024-05-08T22:39:44.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.202+0000 7f97357b5700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 0x7f971c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.202+0000 7f97357b5700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 0x7f971c068bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9720002010 tx=0x7f9720006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.206+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9728050620 con 0x7f973015c050 2024-05-08T22:39:44.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.378+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973006ef90 con 0x7f973015c050 2024-05-08T22:39:44.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f972dffb700 1 -- 172.21.15.179:0/3915698508 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f973006ef90 con 0x7f973015c050 2024-05-08T22:39:44.382 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:44.382 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:44.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 msgr2=0x7f971c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 0x7f971c068bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9720002010 tx=0x7f9720006040 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 msgr2=0x7f97300ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f973006d3b0 tx=0x7f972800f9a0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 shutdown_connections 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f971c066730 0x7f971c068bf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973015c050 0x7f97300ad5a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97300af4f0 0x7f97300c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 --2- 172.21.15.179:0/3915698508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9718004870 0x7f97300b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 >> 172.21.15.179:0/3915698508 conn(0x7f97300cb660 msgr2=0x7f97300ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 shutdown_connections 2024-05-08T22:39:44.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.382+0000 7f9737a19700 1 -- 172.21.15.179:0/3915698508 wait complete. 2024-05-08T22:39:44.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:39:44.398 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:39:44.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.522+0000 7f4f0b59e700 1 Processor -- start 2024-05-08T22:39:44.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.522+0000 7f4f0b59e700 1 -- start start 2024-05-08T22:39:44.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.522+0000 7f4f0b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4f0c15d0c0 0x7f4f0c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.522+0000 7f4f0b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f0c0ce2b0 0x7f4f0c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0c15d4a0 con 0x7f4f0c0d1d20 2024-05-08T22:39:44.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0c06abb0 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0c0d2500 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f09d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4f0c0d1d20 0x7f4f0c0cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35602/0 (socket says 172.21.15.179:35602) 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f09d9b700 1 -- 172.21.15.179:0/1726248379 learned_addr learned my addr 172.21.15.179:0/1726248379 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1726248379 >> v2:172.21.15.179:3300/0 conn(0x7f4f0c0ce2b0 0x7f4f0c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0a59c700 1 --2- 172.21.15.179:0/1726248379 >> v2:172.21.15.195:3300/0 conn(0x7f4f0c15d0c0 0x7f4f0c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1726248379 >> v2:172.21.15.195:3300/0 conn(0x7f4f0c15d0c0 msgr2=0x7f4f0c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1726248379 >> v2:172.21.15.195:3300/0 conn(0x7f4f0c15d0c0 0x7f4f0c0b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1726248379 >> v1:172.21.15.179:6789/0 conn(0x7f4f0c0d1d20 legacy=0x7f4f0c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1726248379 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0d27e0 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0959a700 1 -- 172.21.15.179:0/1726248379 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330845508 0 0) 0x7f4f0c15d4a0 con 0x7f4f0c0d1d20 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1726248379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c0ce2b0 0x7f4f0c06b370 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009d90 tx=0x7f4f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab8626933a9e310 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0959a700 1 -- 172.21.15.179:0/1726248379 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400e070 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0959a700 1 -- 172.21.15.179:0/1726248379 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0400bae0 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0959a700 1 -- 172.21.15.179:0/1726248379 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400be90 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1726248379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c0ce2b0 msgr2=0x7f4f0c06b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1726248379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c0ce2b0 0x7f4f0c06b370 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009d90 tx=0x7f4f0400b330 comp rx=0 tx=0).stop 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1726248379 shutdown_connections 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1726248379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c0ce2b0 0x7f4f0c06b370 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1726248379 >> v2:172.21.15.195:3300/0 conn(0x7f4f0c15d0c0 0x7f4f0c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1726248379 >> 172.21.15.179:0/1726248379 conn(0x7f4f0c0d2ed0 msgr2=0x7f4f0c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:44.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1726248379 shutdown_connections 2024-05-08T22:39:44.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.526+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1726248379 wait complete. 2024-05-08T22:39:44.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 Processor -- start 2024-05-08T22:39:44.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- start start 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f0c0ce2b0 0x7f4f0c0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f0c0d1d20 0x7f4f0c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0c6860 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0c7700 con 0x7f4f0c0d1d20 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0cb940 con 0x7f4f0c0ce2b0 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39276/0 (socket says 172.21.15.179:39276) 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0a59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f0c0ce2b0 0x7f4f0c0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1936022646 learned_addr learned my addr 172.21.15.179:0/1936022646 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f09d9b700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f0c0d1d20 0x7f4f0c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f0c0ce2b0 msgr2=0x7f4f0c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f0c0ce2b0 0x7f4f0c0cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f0c0d1d20 msgr2=0x7f4f0c0c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f0c0d1d20 0x7f4f0c0c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 -- 172.21.15.179:0/1936022646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009a70 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0ad9d700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4f04014040 tx=0x7f4f04004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040045d0 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f04020070 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0c0cbc00 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04004970 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.530+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c0cc1c0 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0c04e700 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f04012430 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f02ffd700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 0x7f4eec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4f04022070 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f0a59c700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 0x7f4eec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.534+0000 7f4f0a59c700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 0x7f4eec068bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c161570 tx=0x7f4efc009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.542+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4f04056be0 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.710+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f0c0676e0 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.710+0000 7f4f02ffd700 1 -- 172.21.15.179:0/1936022646 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f04056770 con 0x7f4f0c15d0c0 2024-05-08T22:39:44.712 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:39:44.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 msgr2=0x7f4eec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 0x7f4eec068bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c161570 tx=0x7f4efc009240 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 msgr2=0x7f4f0c148370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4f04014040 tx=0x7f4f04004af0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 shutdown_connections 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4eec066730 0x7f4eec068bf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f0c0d1d20 0x7f4f0c0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c15d0c0 0x7f4f0c148370 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 --2- 172.21.15.179:0/1936022646 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f0c0ce2b0 0x7f4f0c0cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 >> 172.21.15.179:0/1936022646 conn(0x7f4f0c0d2ed0 msgr2=0x7f4f0c0d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 shutdown_connections 2024-05-08T22:39:44.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.714+0000 7f4f0b59e700 1 -- 172.21.15.179:0/1936022646 wait complete. 2024-05-08T22:39:44.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:39:44.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.846+0000 7fe998a3b700 1 Processor -- start 2024-05-08T22:39:44.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.846+0000 7fe998a3b700 1 -- start start 2024-05-08T22:39:44.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe99415cfb0 0x7fe9940b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 0x7fe9940b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9940d0270 con 0x7fe9940ca970 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9940b4a20 con 0x7fe99415cfb0 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940b6c30 con 0x7fe99415d390 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe98a59c700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fe9940ca970 0x7fe994160cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:53676/0 (socket says 172.21.15.179:53676) 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe98a59c700 1 -- 172.21.15.179:0/2700989769 learned_addr learned my addr 172.21.15.179:0/2700989769 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 --2- 172.21.15.179:0/2700989769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 0x7fe9940b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe992d9d700 1 --2- 172.21.15.179:0/2700989769 >> v2:172.21.15.195:3300/0 conn(0x7fe99415cfb0 0x7fe9940b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99259c700 1 -- 172.21.15.179:0/2700989769 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996873632 0 0) 0x7fe9940d0270 con 0x7fe9940ca970 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99259c700 1 -- 172.21.15.179:0/2700989769 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9800035d0 con 0x7fe9940ca970 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 -- 172.21.15.179:0/2700989769 >> v2:172.21.15.195:3300/0 conn(0x7fe99415cfb0 msgr2=0x7fe9940b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 --2- 172.21.15.179:0/2700989769 >> v2:172.21.15.195:3300/0 conn(0x7fe99415cfb0 0x7fe9940b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 -- 172.21.15.179:0/2700989769 >> v1:172.21.15.195:6789/0 conn(0x7fe9940ca970 legacy=0x7fe994160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 -- 172.21.15.179:0/2700989769 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9940d07a0 con 0x7fe99415d390 2024-05-08T22:39:44.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99359e700 1 --2- 172.21.15.179:0/2700989769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 0x7fe9940b66f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009d90 tx=0x7fe98c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d944623e4c0dd0eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99259c700 1 -- 172.21.15.179:0/2700989769 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00e070 con 0x7fe99415d390 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99259c700 1 -- 172.21.15.179:0/2700989769 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c00bb20 con 0x7fe99415d390 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe99259c700 1 -- 172.21.15.179:0/2700989769 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c012430 con 0x7fe99415d390 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- 172.21.15.179:0/2700989769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 msgr2=0x7fe9940b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 --2- 172.21.15.179:0/2700989769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 0x7fe9940b66f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009d90 tx=0x7fe98c00b330 comp rx=0 tx=0).stop 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- 172.21.15.179:0/2700989769 shutdown_connections 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 --2- 172.21.15.179:0/2700989769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99415d390 0x7fe9940b66f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 --2- 172.21.15.179:0/2700989769 >> v2:172.21.15.195:3300/0 conn(0x7fe99415cfb0 0x7fe9940b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- 172.21.15.179:0/2700989769 >> 172.21.15.179:0/2700989769 conn(0x7fe9940b2770 msgr2=0x7fe9940b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- 172.21.15.179:0/2700989769 shutdown_connections 2024-05-08T22:39:44.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 -- 172.21.15.179:0/2700989769 wait complete. 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.850+0000 7fe998a3b700 1 Processor -- start 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- start start 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9940ca970 0x7fe99414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 0x7fe99406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99415dce0 con 0x7fe99415cfb0 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99414c8f0 con 0x7fe99415d390 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99406d960 con 0x7fe9940ca970 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39286/0 (socket says 172.21.15.179:39286) 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98a59c700 1 -- 172.21.15.179:0/728157644 learned_addr learned my addr 172.21.15.179:0/728157644 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 0x7fe99406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe992d9d700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9940ca970 0x7fe99414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 -- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9940ca970 msgr2=0x7fe99414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9940ca970 0x7fe99414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 -- 172.21.15.179:0/728157644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 msgr2=0x7fe99406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:44.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 -- 172.21.15.179:0/728157644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009a70 con 0x7fe99415d390 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98a59c700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe99359e700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 0x7fe99406e5d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe994074520 tx=0x7fe98c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c012430 con 0x7fe99415d390 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c020070 con 0x7fe99415d390 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004280 con 0x7fe99415d390 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99406d3a0 con 0x7fe99415d390 2024-05-08T22:39:44.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.854+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99415d900 con 0x7fe99415d390 2024-05-08T22:39:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99415de20 con 0x7fe99415d390 2024-05-08T22:39:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe98c0043e0 con 0x7fe99415d390 2024-05-08T22:39:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe98bfff700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 0x7fe974068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe98c022070 con 0x7fe99415d390 2024-05-08T22:39:44.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe992d9d700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 0x7fe974068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:44.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.858+0000 7fe992d9d700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 0x7fe974068bf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe984009b50 tx=0x7fe984009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:44.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:44.870+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe98c0534d0 con 0x7fe99415d390 2024-05-08T22:39:45.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.034+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe99415de20 con 0x7fe99415d390 2024-05-08T22:39:45.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe98bfff700 1 -- 172.21.15.179:0/728157644 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe98c02c020 con 0x7fe99415d390 2024-05-08T22:39:45.039 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:39:45.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 msgr2=0x7fe974068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 0x7fe974068bf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe984009b50 tx=0x7fe984009240 comp rx=0 tx=0).stop 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 msgr2=0x7fe99406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 0x7fe99406e5d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe994074520 tx=0x7fe98c004af0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 shutdown_connections 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe974066730 0x7fe974068bf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99415d390 0x7fe99406e5d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99415cfb0 0x7fe99406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 --2- 172.21.15.179:0/728157644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9940ca970 0x7fe99414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 >> 172.21.15.179:0/728157644 conn(0x7fe9940b2770 msgr2=0x7fe9940c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:45.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 shutdown_connections 2024-05-08T22:39:45.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.038+0000 7fe998a3b700 1 -- 172.21.15.179:0/728157644 wait complete. 2024-05-08T22:39:45.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:39:45.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.170+0000 7fa8b20fb700 1 Processor -- start 2024-05-08T22:39:45.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.170+0000 7fa8b20fb700 1 -- start start 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa8a40e33a0 0x7fa8a4058950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a40578b0 con 0x7fa8a4007280 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a4057b90 con 0x7fa8a403ebd0 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a40576c0 con 0x7fa8a40e33a0 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b18fa700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa8a4007280 0x7fa8a403c050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35608/0 (socket says 172.21.15.179:35608) 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b18fa700 1 -- 172.21.15.179:0/2120799952 learned_addr learned my addr 172.21.15.179:0/2120799952 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b10f9700 1 --2- 172.21.15.179:0/2120799952 >> v2:172.21.15.195:3300/0 conn(0x7fa8a40e33a0 0x7fa8a4058950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366866553 0 0) 0x7fa8a40578b0 con 0x7fa8a4007280 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8980035d0 con 0x7fa8a4007280 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395447514 0 0) 0x7fa8a4057b90 con 0x7fa8a403ebd0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a40578b0 con 0x7fa8a403ebd0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b10f9700 1 -- 172.21.15.179:0/2120799952 >> v1:172.21.15.195:6789/0 conn(0x7fa8a403ebd0 legacy=0x7fa8a4056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b10f9700 1 -- 172.21.15.179:0/2120799952 >> v1:172.21.15.179:6789/0 conn(0x7fa8a4007280 legacy=0x7fa8a403c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b10f9700 1 -- 172.21.15.179:0/2120799952 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4057e70 con 0x7fa8a40e33a0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b10f9700 1 --2- 172.21.15.179:0/2120799952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a40e33a0 0x7fa8a4058950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009850 tx=0x7fa89c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d325dd896af617 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011830 con 0x7fa8a40e33a0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c011990 con 0x7fa8a40e33a0 2024-05-08T22:39:45.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8a3fff700 1 -- 172.21.15.179:0/2120799952 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011d40 con 0x7fa8a40e33a0 2024-05-08T22:39:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- 172.21.15.179:0/2120799952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a40e33a0 msgr2=0x7fa8a4058950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/2120799952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a40e33a0 0x7fa8a4058950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009850 tx=0x7fa89c009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- 172.21.15.179:0/2120799952 shutdown_connections 2024-05-08T22:39:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/2120799952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a40e33a0 0x7fa8a4058950 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- 172.21.15.179:0/2120799952 >> 172.21.15.179:0/2120799952 conn(0x7fa8a4055a10 msgr2=0x7fa8a4055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- 172.21.15.179:0/2120799952 shutdown_connections 2024-05-08T22:39:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.174+0000 7fa8b20fb700 1 -- 172.21.15.179:0/2120799952 wait complete. 2024-05-08T22:39:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 Processor -- start 2024-05-08T22:39:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 -- start start 2024-05-08T22:39:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a4007280 0x7fa8a4050060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8a40e33a0 0x7fa8a40e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40e63f0 con 0x7fa8a403ebd0 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40d8c40 con 0x7fa8a4007280 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b20fb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40505a0 con 0x7fa8a40e33a0 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.178+0000 7fa8b08f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39288/0 (socket says 172.21.15.179:39288) 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 -- 172.21.15.179:0/1616912921 learned_addr learned my addr 172.21.15.179:0/1616912921 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b10f9700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a4007280 0x7fa8a4050060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b18fa700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8a40e33a0 0x7fa8a40e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 -- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8a40e33a0 msgr2=0x7fa8a40e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8a40e33a0 0x7fa8a40e4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 -- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a4007280 msgr2=0x7fa8a4050060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a4007280 0x7fa8a4050060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 -- 172.21.15.179:0/1616912921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0094b0 con 0x7fa8a403ebd0 2024-05-08T22:39:45.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b08f8700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800c520 tx=0x7fa8a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80052d0 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a8005450 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a4052d80 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80038e0 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a40e6010 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a40e6530 con 0x7fa8a403ebd0 2024-05-08T22:39:45.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.182+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8a8003a40 con 0x7fa8a403ebd0 2024-05-08T22:39:45.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.186+0000 7fa8a1ffb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 0x7fa890068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.186+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa8a8083660 con 0x7fa8a403ebd0 2024-05-08T22:39:45.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.186+0000 7fa8b10f9700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 0x7fa890068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.186+0000 7fa8b10f9700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 0x7fa890068bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0090b0 tx=0x7fa89c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.190+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8a40e6530 con 0x7fa8a403ebd0 2024-05-08T22:39:45.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8a40e6530 con 0x7fa8a403ebd0 2024-05-08T22:39:45.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8a1ffb700 1 -- 172.21.15.179:0/1616912921 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8a40e6530 con 0x7fa8a403ebd0 2024-05-08T22:39:45.364 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 msgr2=0x7fa890068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 0x7fa890068bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0090b0 tx=0x7fa89c01c040 comp rx=0 tx=0).stop 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 msgr2=0x7fa8a4178940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.362+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800c520 tx=0x7fa8a800c830 comp rx=0 tx=0).stop 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 shutdown_connections 2024-05-08T22:39:45.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa890066730 0x7fa890068bf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8a40e33a0 0x7fa8a40e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8a403ebd0 0x7fa8a4178940 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 --2- 172.21.15.179:0/1616912921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8a4007280 0x7fa8a4050060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 >> 172.21.15.179:0/1616912921 conn(0x7fa8a4055a10 msgr2=0x7fa8a404cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 shutdown_connections 2024-05-08T22:39:45.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.366+0000 7fa8b20fb700 1 -- 172.21.15.179:0/1616912921 wait complete. 2024-05-08T22:39:45.379 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:39:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 Processor -- start 2024-05-08T22:39:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 -- start start 2024-05-08T22:39:45.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc800ad740 0x7fcc800adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800ae0d0 con 0x7fcc800ccdd0 2024-05-08T22:39:45.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800b1cb0 con 0x7fcc80150bc0 2024-05-08T22:39:45.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc863c2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc800b2930 con 0x7fcc800ad740 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc84bbf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc800ad740 0x7fcc800adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc85bc1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcc800ccdd0 0x7fcc800cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46544/0 (socket says 172.21.15.179:46544) 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc85bc1700 1 -- 172.21.15.179:0/1578439557 learned_addr learned my addr 172.21.15.179:0/1578439557 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc84bbf700 1 -- 172.21.15.179:0/1578439557 >> v1:172.21.15.195:6789/0 conn(0x7fcc80150bc0 legacy=0x7fcc8015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc84bbf700 1 -- 172.21.15.179:0/1578439557 >> v1:172.21.15.179:6789/0 conn(0x7fcc800ccdd0 legacy=0x7fcc800cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc73fff700 1 -- 172.21.15.179:0/1578439557 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582948269 0 0) 0x7fcc800b1cb0 con 0x7fcc80150bc0 2024-05-08T22:39:45.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.494+0000 7fcc84bbf700 1 -- 172.21.15.179:0/1578439557 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc800b1f90 con 0x7fcc800ad740 2024-05-08T22:39:45.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc84bbf700 1 --2- 172.21.15.179:0/1578439557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc800adb90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009880 tx=0x7fcc74009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3468e197d1c2b6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc73fff700 1 -- 172.21.15.179:0/1578439557 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011830 con 0x7fcc800ad740 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc73fff700 1 -- 172.21.15.179:0/1578439557 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc74011990 con 0x7fcc800ad740 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc73fff700 1 -- 172.21.15.179:0/1578439557 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011d20 con 0x7fcc800ad740 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- 172.21.15.179:0/1578439557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 msgr2=0x7fcc800adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 --2- 172.21.15.179:0/1578439557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc800adb90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009880 tx=0x7fcc74009b90 comp rx=0 tx=0).stop 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- 172.21.15.179:0/1578439557 shutdown_connections 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 --2- 172.21.15.179:0/1578439557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc800adb90 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- 172.21.15.179:0/1578439557 >> 172.21.15.179:0/1578439557 conn(0x7fcc800ce960 msgr2=0x7fcc800ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- 172.21.15.179:0/1578439557 shutdown_connections 2024-05-08T22:39:45.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- 172.21.15.179:0/1578439557 wait complete. 2024-05-08T22:39:45.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 Processor -- start 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- start start 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc800ccdd0 0x7fcc80158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc80150bc0 0x7fcc8014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801593a0 con 0x7fcc800ad740 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80158b30 con 0x7fcc800ccdd0 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc863c2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801f6c90 con 0x7fcc80150bc0 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52524/0 (socket says 172.21.15.179:52524) 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 -- 172.21.15.179:0/3170367256 learned_addr learned my addr 172.21.15.179:0/3170367256 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc84bbf700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc800ccdd0 0x7fcc80158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc85bc1700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc80150bc0 0x7fcc8014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 -- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc80150bc0 msgr2=0x7fcc8014c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc80150bc0 0x7fcc8014c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 -- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc800ccdd0 msgr2=0x7fcc80158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc800ccdd0 0x7fcc80158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.498+0000 7fcc853c0700 1 -- 172.21.15.179:0/3170367256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740094b0 con 0x7fcc800ad740 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc853c0700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c750 tx=0x7fcc7c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0054e0 con 0x7fcc800ad740 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c0036a0 con 0x7fcc800ad740 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc801f63f0 con 0x7fcc800ad740 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003a50 con 0x7fcc800ad740 2024-05-08T22:39:45.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc801f69b0 con 0x7fcc800ad740 2024-05-08T22:39:45.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8004e700 con 0x7fcc800ad740 2024-05-08T22:39:45.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc7c003bb0 con 0x7fcc800ad740 2024-05-08T22:39:45.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 0x7fcc64068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:45.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.502+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fcc7c083910 con 0x7fcc800ad740 2024-05-08T22:39:45.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.506+0000 7fcc84bbf700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 0x7fcc64068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:45.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.506+0000 7fcc84bbf700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 0x7fcc64068c20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcc740090b0 tx=0x7fcc74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:45.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.510+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcc7c050a50 con 0x7fcc800ad740 2024-05-08T22:39:45.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.694+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcc8006e980 con 0x7fcc800ad740 2024-05-08T22:39:45.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc71ffb700 1 -- 172.21.15.179:0/3170367256 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) v1 ==== 80+0+122 (secure 0 0 0) 0x7fcc7c017080 con 0x7fcc800ad740 2024-05-08T22:39:45.698 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6837/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:39:45.698 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/123759130 2024-05-09T22:38:20.877311+0000 2024-05-08T22:39:45.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 msgr2=0x7fcc64068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 0x7fcc64068c20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcc740090b0 tx=0x7fcc74008030 comp rx=0 tx=0).stop 2024-05-08T22:39:45.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 msgr2=0x7fcc801f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:45.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c750 tx=0x7fcc7c00cb10 comp rx=0 tx=0).stop 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 shutdown_connections 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcc64066760 0x7fcc64068c20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc800ccdd0 0x7fcc80158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc80150bc0 0x7fcc8014c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 --2- 172.21.15.179:0/3170367256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc800ad740 0x7fcc801f5e10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 >> 172.21.15.179:0/3170367256 conn(0x7fcc800ce960 msgr2=0x7fcc80159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 shutdown_connections 2024-05-08T22:39:45.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:45.698+0000 7fcc863c2700 1 -- 172.21.15.179:0/3170367256 wait complete. 2024-05-08T22:39:45.703 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:39:45.713 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:39:45.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:45.714 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:39:45.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[44496]: fuse finished with error 0 and tester_r 0 2024-05-08T22:39:45.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:39:45.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:45.728 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:39:45.851 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:39:45.851 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:45.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:39:45.860 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:39:45.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:45.860 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:39:45.867 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:39:45.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:39:45.868 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:39:45.868 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:45.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:39:45.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:39:45.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:45.876 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:39:45.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:39:45.926 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:39:45.927 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:39:45.927 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:45.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:39:45.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:39:45.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:39:45.935 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:39:45.941 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:39:45.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:39:45.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:46.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.062+0000 7f14567bc700 1 Processor -- start 2024-05-08T22:39:46.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.062+0000 7f14567bc700 1 -- start start 2024-05-08T22:39:46.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14500b50b0 0x7f14500cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f14500b1620 0x7f145015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14500ca310 con 0x7f14500b1620 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145015a200 con 0x7f14500b50b0 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14567bc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b4e20 con 0x7f1450159e20 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f14500b1620 0x7f145015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f14500b1620 0x7f145015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52534/0 (socket says 172.21.15.179:52534) 2024-05-08T22:39:46.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 -- 172.21.15.179:0/3112476187 learned_addr learned my addr 172.21.15.179:0/3112476187 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f14557ba700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1454fb9700 1 --2- 172.21.15.179:0/3112476187 >> v2:172.21.15.195:3300/0 conn(0x7f14500b50b0 0x7f14500cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 -- 172.21.15.179:0/3112476187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 msgr2=0x7f14500cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 -- 172.21.15.179:0/3112476187 >> v2:172.21.15.195:3300/0 conn(0x7f14500b50b0 msgr2=0x7f14500cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 --2- 172.21.15.179:0/3112476187 >> v2:172.21.15.195:3300/0 conn(0x7f14500b50b0 0x7f14500cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 -- 172.21.15.179:0/3112476187 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500c99e0 con 0x7f14500b1620 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1455fbb700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f145015d950 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f1450070d90 tx=0x7f144800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcd18cc64fda923 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1447fff700 1 -- 172.21.15.179:0/3112476187 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800e070 con 0x7f14500b1620 2024-05-08T22:39:46.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1447fff700 1 -- 172.21.15.179:0/3112476187 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144800b910 con 0x7f14500b1620 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.066+0000 7f1447fff700 1 -- 172.21.15.179:0/3112476187 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800bcc0 con 0x7f14500b1620 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- 172.21.15.179:0/3112476187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 msgr2=0x7f145015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f145015d950 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f1450070d90 tx=0x7f144800b330 comp rx=0 tx=0).stop 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- 172.21.15.179:0/3112476187 shutdown_connections 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f145015d950 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- 172.21.15.179:0/3112476187 >> v2:172.21.15.195:3300/0 conn(0x7f14500b50b0 0x7f14500cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- 172.21.15.179:0/3112476187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- 172.21.15.179:0/3112476187 >> 172.21.15.179:0/3112476187 conn(0x7f1450098a70 msgr2=0x7f14500b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- 172.21.15.179:0/3112476187 shutdown_connections 2024-05-08T22:39:46.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- 172.21.15.179:0/3112476187 wait complete. 2024-05-08T22:39:46.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 Processor -- start 2024-05-08T22:39:46.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- start start 2024-05-08T22:39:46.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14500b50b0 0x7f14500afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450153bb0 con 0x7f14500b1620 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b06c0 con 0x7f14500b50b0 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14567bc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b09a0 con 0x7f1450159e20 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52548/0 (socket says 172.21.15.179:52548) 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 -- 172.21.15.179:0/214072219 learned_addr learned my addr 172.21.15.179:0/214072219 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f1454fb9700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14500b50b0 0x7f14500afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f1455fbb700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 -- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 msgr2=0x7f14500ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 -- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14500b50b0 msgr2=0x7f14500afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14500b50b0 0x7f14500afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 -- 172.21.15.179:0/214072219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1448009b30 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f1455fbb700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.070+0000 7f14557ba700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f144c00ebe0 tx=0x7f144c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c009740 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00c850 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14500b0c80 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145015a3d0 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00cbe0 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1430005310 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.074+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f144c00cd40 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.078+0000 7f1445ffb700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 0x7f1434068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.078+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f144c084370 con 0x7f14500b1620 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.078+0000 7f1454fb9700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 0x7f1434068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.078+0000 7f1454fb9700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 0x7f1434068bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f14500afd70 tx=0x7f1440009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.082+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f144c0515d0 con 0x7f14500b1620 2024-05-08T22:39:46.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.294+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14300055c0 con 0x7f14500b1620 2024-05-08T22:39:46.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f1445ffb700 1 -- 172.21.15.179:0/214072219 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f144c054bf0 con 0x7f14500b1620 2024-05-08T22:39:46.298 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:46.298 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:39:12.053827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 msgr2=0x7f1434068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 0x7f1434068bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f14500afd70 tx=0x7f1440009240 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 msgr2=0x7f1450153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f144c00ebe0 tx=0x7f144c00eef0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 shutdown_connections 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1434066730 0x7f1434068bf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14500b50b0 0x7f14500afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1450159e20 0x7f14500ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 --2- 172.21.15.179:0/214072219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14500b1620 0x7f1450153670 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 >> 172.21.15.179:0/214072219 conn(0x7f1450098a70 msgr2=0x7f14500ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 shutdown_connections 2024-05-08T22:39:46.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.298+0000 7f14567bc700 1 -- 172.21.15.179:0/214072219 wait complete. 2024-05-08T22:39:46.302 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 47 2024-05-08T22:39:46.313 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:46.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 Processor -- start 2024-05-08T22:39:46.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 -- start start 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93600b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93600ce580 0x7f936006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936006afc0 con 0x7f93600b2120 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936015d990 con 0x7f93600ce580 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365845700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600d26f0 con 0x7f936015d5b0 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93600ce580 0x7f936006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93600ce580 0x7f936006b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52550/0 (socket says 172.21.15.179:52550) 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 -- 172.21.15.179:0/960196728 learned_addr learned my addr 172.21.15.179:0/960196728 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9364843700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93600b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 -- 172.21.15.179:0/960196728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 msgr2=0x7f93600b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93600b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 -- 172.21.15.179:0/960196728 >> v1:172.21.15.195:6789/0 conn(0x7f93600b2120 legacy=0x7f93600cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 -- 172.21.15.179:0/960196728 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600d2130 con 0x7f93600ce580 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f9365044700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600ce580 0x7f936006b450 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9358009910 tx=0x7f9358009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bd527401eb8561 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f935f7fe700 1 -- 172.21.15.179:0/960196728 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358011830 con 0x7f93600ce580 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f935f7fe700 1 -- 172.21.15.179:0/960196728 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9358011990 con 0x7f93600ce580 2024-05-08T22:39:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.418+0000 7f935f7fe700 1 -- 172.21.15.179:0/960196728 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358011d40 con 0x7f93600ce580 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- 172.21.15.179:0/960196728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600ce580 msgr2=0x7f936006b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600ce580 0x7f936006b450 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9358009910 tx=0x7f9358009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- 172.21.15.179:0/960196728 shutdown_connections 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600ce580 0x7f936006b450 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- 172.21.15.179:0/960196728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93600b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- 172.21.15.179:0/960196728 >> 172.21.15.179:0/960196728 conn(0x7f93600b4940 msgr2=0x7f93600b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- 172.21.15.179:0/960196728 shutdown_connections 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- 172.21.15.179:0/960196728 wait complete. 2024-05-08T22:39:46.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 Processor -- start 2024-05-08T22:39:46.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- start start 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 0x7f936015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93601f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601f4580 con 0x7f93600b2120 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360160500 con 0x7f93600ce580 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365845700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936015ff40 con 0x7f936015d5b0 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9364843700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 0x7f936015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f935ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f935ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:57106/0 (socket says 172.21.15.179:57106) 2024-05-08T22:39:46.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9364843700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 0x7f936015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52566/0 (socket says 172.21.15.179:52566) 2024-05-08T22:39:46.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f935ffff700 1 -- 172.21.15.179:0/1020062166 learned_addr learned my addr 172.21.15.179:0/1020062166 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.422+0000 7f9365044700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93601f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 -- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 msgr2=0x7f93601f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93601f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 -- 172.21.15.179:0/1020062166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 msgr2=0x7f936015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 0x7f936015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 -- 172.21.15.179:0/1020062166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009570 con 0x7f93600ce580 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935ffff700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f934800c500 tx=0x7f934800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480052d0 con 0x7f93600ce580 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348005430 con 0x7f93600ce580 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480055a0 con 0x7f93600ce580 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9360160220 con 0x7f93600ce580 2024-05-08T22:39:46.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600cd380 con 0x7f93600ce580 2024-05-08T22:39:46.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93601f46c0 con 0x7f93600ce580 2024-05-08T22:39:46.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.426+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f93480038e0 con 0x7f93600ce580 2024-05-08T22:39:46.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.430+0000 7f935d7fa700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 0x7f9340068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.430+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f93480837b0 con 0x7f93600ce580 2024-05-08T22:39:46.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.430+0000 7f9364843700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 0x7f9340068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.430+0000 7f9364843700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 0x7f9340068bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9360072d00 tx=0x7f935000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.434+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9348050af0 con 0x7f93600ce580 2024-05-08T22:39:46.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.618+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93601f46c0 con 0x7f93600ce580 2024-05-08T22:39:46.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.622+0000 7f935d7fa700 1 -- 172.21.15.179:0/1020062166 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f93601f46c0 con 0x7f93600ce580 2024-05-08T22:39:46.624 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:46.624 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.622+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 msgr2=0x7f9340068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.622+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 0x7f9340068bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9360072d00 tx=0x7f935000b040 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.622+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 msgr2=0x7f93601f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.622+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f934800c500 tx=0x7f934800c810 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 shutdown_connections 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9340066730 0x7f9340068bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93600ce580 0x7f93601f64a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f936015d5b0 0x7f93601f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 --2- 172.21.15.179:0/1020062166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93600b2120 0x7f936015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 >> 172.21.15.179:0/1020062166 conn(0x7f93600b4940 msgr2=0x7f936015c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 shutdown_connections 2024-05-08T22:39:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.626+0000 7f9365845700 1 -- 172.21.15.179:0/1020062166 wait complete. 2024-05-08T22:39:46.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:37:56.159297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:37:56.796186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:39:46.640 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:39:46.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.762+0000 7f1f7a7fd700 1 Processor -- start 2024-05-08T22:39:46.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.762+0000 7f1f7a7fd700 1 -- start start 2024-05-08T22:39:46.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f7a7fd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1f74159550 0x7f1f7406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f7a7fd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f7a7fd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740c5a70 con 0x7f1f740b18b0 2024-05-08T22:39:46.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f7a7fd700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740b1e00 con 0x7f1f74159550 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f7a7fd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7406c710 con 0x7f1f740c4180 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f797fb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1f74159550 0x7f1f7406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:40192/0 (socket says 172.21.15.179:40192) 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 -- 172.21.15.179:0/1062136941 learned_addr learned my addr 172.21.15.179:0/1062136941 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 -- 172.21.15.179:0/1062136941 >> v2:172.21.15.195:3300/0 conn(0x7f1f74159550 msgr2=0x7f1f7406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/1062136941 >> v2:172.21.15.195:3300/0 conn(0x7f1f74159550 0x7f1f7406b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 -- 172.21.15.179:0/1062136941 >> v1:172.21.15.195:6789/0 conn(0x7f1f740b18b0 legacy=0x7f1f7415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 -- 172.21.15.179:0/1062136941 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7406ce80 con 0x7f1f740c4180 2024-05-08T22:39:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/1062136941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009e50 tx=0x7f1f6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f68ac5404943fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f6bfff700 1 -- 172.21.15.179:0/1062136941 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00e070 con 0x7f1f740c4180 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f6bfff700 1 -- 172.21.15.179:0/1062136941 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00b910 con 0x7f1f740c4180 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.766+0000 7f1f6bfff700 1 -- 172.21.15.179:0/1062136941 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00bcc0 con 0x7f1f740c4180 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/1062136941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 msgr2=0x7f1f7406b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/1062136941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009e50 tx=0x7f1f6c00b330 comp rx=0 tx=0).stop 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/1062136941 shutdown_connections 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/1062136941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c4180 0x7f1f7406b980 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/1062136941 >> v2:172.21.15.195:3300/0 conn(0x7f1f74159550 0x7f1f7406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/1062136941 >> 172.21.15.179:0/1062136941 conn(0x7f1f74098a70 msgr2=0x7f1f740cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/1062136941 shutdown_connections 2024-05-08T22:39:46.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/1062136941 wait complete. 2024-05-08T22:39:46.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 Processor -- start 2024-05-08T22:39:46.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.770+0000 7f1f7a7fd700 1 -- start start 2024-05-08T22:39:46.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740b18b0 0x7f1f74156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 0x7f1f740aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 0x7f1f74152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740afd90 con 0x7f1f740c4180 2024-05-08T22:39:46.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740af270 con 0x7f1f74159550 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74152c90 con 0x7f1f740b18b0 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f78ffa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 0x7f1f740aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f78ffa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 0x7f1f740aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52568/0 (socket says 172.21.15.179:52568) 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f78ffa700 1 -- 172.21.15.179:0/494597156 learned_addr learned my addr 172.21.15.179:0/494597156 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 0x7f1f74152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f797fb700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740b18b0 0x7f1f74156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 -- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740b18b0 msgr2=0x7f1f74156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740b18b0 0x7f1f74156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 -- 172.21.15.179:0/494597156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 msgr2=0x7f1f740aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 0x7f1f740aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 -- 172.21.15.179:0/494597156 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c009b30 con 0x7f1f74159550 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f79ffc700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 0x7f1f74152750 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c015000 tx=0x7f1f6c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004880 con 0x7f1f74159550 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0092e0 con 0x7f1f74159550 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740af4f0 con 0x7f1f74159550 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c01c680 con 0x7f1f74159550 2024-05-08T22:39:46.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740afab0 con 0x7f1f74159550 2024-05-08T22:39:46.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.774+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7404e700 con 0x7f1f74159550 2024-05-08T22:39:46.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.778+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f6c01c7e0 con 0x7f1f74159550 2024-05-08T22:39:46.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.778+0000 7f1f69ffb700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 0x7f1f58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:46.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.778+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1f6c085bd0 con 0x7f1f74159550 2024-05-08T22:39:46.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.778+0000 7f1f797fb700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 0x7f1f58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:46.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.778+0000 7f1f797fb700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 0x7f1f58068bf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000ac30 tx=0x7f1f7000a350 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:46.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.790+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1f6c052fd0 con 0x7f1f74159550 2024-05-08T22:39:46.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.958+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f74158b60 con 0x7f1f74159550 2024-05-08T22:39:46.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.958+0000 7f1f69ffb700 1 -- 172.21.15.179:0/494597156 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f6c0565f0 con 0x7f1f74159550 2024-05-08T22:39:46.962 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:39:46.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 msgr2=0x7f1f58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 0x7f1f58068bf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000ac30 tx=0x7f1f7000a350 comp rx=0 tx=0).stop 2024-05-08T22:39:46.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 msgr2=0x7f1f74152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 0x7f1f74152750 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c015000 tx=0x7f1f6c004af0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 shutdown_connections 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1f58066730 0x7f1f58068bf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f74159550 0x7f1f74152750 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f740c4180 0x7f1f740aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 --2- 172.21.15.179:0/494597156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740b18b0 0x7f1f74156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 >> 172.21.15.179:0/494597156 conn(0x7f1f74098a70 msgr2=0x7f1f7406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 shutdown_connections 2024-05-08T22:39:46.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:46.962+0000 7f1f7a7fd700 1 -- 172.21.15.179:0/494597156 wait complete. 2024-05-08T22:39:46.977 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:39:47.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.094+0000 7f736b59e700 1 Processor -- start 2024-05-08T22:39:47.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.094+0000 7f736b59e700 1 -- start start 2024-05-08T22:39:47.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f736c14ae30 0x7f736c165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c0b2de0 0x7f736c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c166090 con 0x7f736c0b1a50 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c06e9d0 con 0x7f736c14ae30 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c06dd50 con 0x7f736c0b2de0 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f736c14ae30 0x7f736c165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736ad9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c0b2de0 0x7f736c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f7369d9b700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f736c0b1a50 0x7f736c0b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43132/0 (socket says 172.21.15.179:43132) 2024-05-08T22:39:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f7369d9b700 1 -- 172.21.15.179:0/2794069253 learned_addr learned my addr 172.21.15.179:0/2794069253 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 -- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c0b2de0 msgr2=0x7f736c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 --2- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c0b2de0 0x7f736c06abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 -- 172.21.15.179:0/2794069253 >> v1:172.21.15.195:6789/0 conn(0x7f736c0b1a50 legacy=0x7f736c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 -- 172.21.15.179:0/2794069253 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c06e030 con 0x7f736c14ae30 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736a59c700 1 --2- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c14ae30 0x7f736c165b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7364009850 tx=0x7f7364009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68882b31e5a292a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736959a700 1 -- 172.21.15.179:0/2794069253 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364011830 con 0x7f736c14ae30 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736959a700 1 -- 172.21.15.179:0/2794069253 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7364011990 con 0x7f736c14ae30 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736959a700 1 -- 172.21.15.179:0/2794069253 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364011d40 con 0x7f736c14ae30 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c14ae30 msgr2=0x7f736c165b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 --2- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c14ae30 0x7f736c165b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7364009850 tx=0x7f7364009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- 172.21.15.179:0/2794069253 shutdown_connections 2024-05-08T22:39:47.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 --2- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c0b2de0 0x7f736c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 --2- 172.21.15.179:0/2794069253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c14ae30 0x7f736c165b50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- 172.21.15.179:0/2794069253 >> 172.21.15.179:0/2794069253 conn(0x7f736c0cea30 msgr2=0x7f736c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- 172.21.15.179:0/2794069253 shutdown_connections 2024-05-08T22:39:47.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.098+0000 7f736b59e700 1 -- 172.21.15.179:0/2794069253 wait complete. 2024-05-08T22:39:47.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 Processor -- start 2024-05-08T22:39:47.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 -- start start 2024-05-08T22:39:47.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 0x7f736c1f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c0b2de0 0x7f736c147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 0x7f736c0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c147350 con 0x7f736c0b1a50 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c1460e0 con 0x7f736c0b2de0 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c1463c0 con 0x7f736c14ae30 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 0x7f736c1f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 0x7f736c1f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52574/0 (socket says 172.21.15.179:52574) 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736a59c700 1 -- 172.21.15.179:0/1892974299 learned_addr learned my addr 172.21.15.179:0/1892974299 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:47.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 0x7f736c0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f7369d9b700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c0b2de0 0x7f736c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 -- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c0b2de0 msgr2=0x7f736c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c0b2de0 0x7f736c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 -- 172.21.15.179:0/1892974299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 msgr2=0x7f736c1f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 0x7f736c1f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.102+0000 7f736ad9d700 1 -- 172.21.15.179:0/1892974299 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640094b0 con 0x7f736c14ae30 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f736ad9d700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 0x7f736c0cc720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f736c162cc0 tx=0x7f735c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00cc20 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00f040 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c009e60 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c1466a0 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c1f7650 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c04e700 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.106+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f735c014480 con 0x7f736c14ae30 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.110+0000 7f7362ffd700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.110+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f735c084db0 con 0x7f736c14ae30 2024-05-08T22:39:47.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.110+0000 7f736a59c700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.110+0000 7f736a59c700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 0x7f734c068c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7364000c00 tx=0x7f736401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.122+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f735c089070 con 0x7f736c14ae30 2024-05-08T22:39:47.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.290+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f736c06d460 con 0x7f736c14ae30 2024-05-08T22:39:47.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.290+0000 7f7362ffd700 1 -- 172.21.15.179:0/1892974299 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f735c018020 con 0x7f736c14ae30 2024-05-08T22:39:47.293 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 msgr2=0x7f734c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 0x7f734c068c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7364000c00 tx=0x7f736401c040 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 msgr2=0x7f736c0cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 0x7f736c0cc720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f736c162cc0 tx=0x7f735c00ef90 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 shutdown_connections 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f736c0b2de0 0x7f736c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f736c14ae30 0x7f736c0cc720 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 --2- 172.21.15.179:0/1892974299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c0b1a50 0x7f736c1f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 >> 172.21.15.179:0/1892974299 conn(0x7f736c0cea30 msgr2=0x7f736c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 shutdown_connections 2024-05-08T22:39:47.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.294+0000 7f736b59e700 1 -- 172.21.15.179:0/1892974299 wait complete. 2024-05-08T22:39:47.309 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:39:47.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 Processor -- start 2024-05-08T22:39:47.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 -- start start 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f96d41544e0 0x7f96d406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415d8f0 0x7f96d415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f96d415e2b0 0x7f96d415a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d40cd6e0 con 0x7f96d415e2b0 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d406db50 con 0x7f96d41544e0 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b4650 con 0x7f96d415d8f0 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f96d415e2b0 0x7f96d415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f96d415e2b0 0x7f96d415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52590/0 (socket says 172.21.15.179:52590) 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d37fe700 1 -- 172.21.15.179:0/3358789179 learned_addr learned my addr 172.21.15.179:0/3358789179 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:47.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d27fc700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415d8f0 0x7f96d415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.430+0000 7f96d2ffd700 1 --2- 172.21.15.179:0/3358789179 >> v2:172.21.15.195:3300/0 conn(0x7f96d41544e0 0x7f96d406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 -- 172.21.15.179:0/3358789179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415d8f0 msgr2=0x7f96d415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415d8f0 0x7f96d415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 -- 172.21.15.179:0/3358789179 >> v2:172.21.15.195:3300/0 conn(0x7f96d41544e0 msgr2=0x7f96d406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 --2- 172.21.15.179:0/3358789179 >> v2:172.21.15.195:3300/0 conn(0x7f96d41544e0 0x7f96d406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 -- 172.21.15.179:0/3358789179 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d415e740 con 0x7f96d415e2b0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d37fe700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415e2b0 0x7f96d415a300 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f96d4070850 tx=0x7f96cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be30f3cb06ea29f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d1ffb700 1 -- 172.21.15.179:0/3358789179 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00bb10 con 0x7f96d415e2b0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d1ffb700 1 -- 172.21.15.179:0/3358789179 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00bc70 con 0x7f96d415e2b0 2024-05-08T22:39:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d1ffb700 1 -- 172.21.15.179:0/3358789179 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc011460 con 0x7f96d415e2b0 2024-05-08T22:39:47.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 -- 172.21.15.179:0/3358789179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415e2b0 msgr2=0x7f96d415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415e2b0 0x7f96d415a300 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f96d4070850 tx=0x7f96cc00b330 comp rx=0 tx=0).stop 2024-05-08T22:39:47.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 -- 172.21.15.179:0/3358789179 shutdown_connections 2024-05-08T22:39:47.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415e2b0 0x7f96d415a300 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3358789179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415d8f0 0x7f96d415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3358789179 >> v2:172.21.15.195:3300/0 conn(0x7f96d41544e0 0x7f96d406ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 -- 172.21.15.179:0/3358789179 >> 172.21.15.179:0/3358789179 conn(0x7f96d40b18d0 msgr2=0x7f96d40b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 -- 172.21.15.179:0/3358789179 shutdown_connections 2024-05-08T22:39:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.434+0000 7f96d3fff700 1 -- 172.21.15.179:0/3358789179 wait complete. 2024-05-08T22:39:47.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 Processor -- start 2024-05-08T22:39:47.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 -- start start 2024-05-08T22:39:47.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96d41544e0 0x7f96d4158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415e2b0 0x7f96d4147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d414ec20 con 0x7f96d415d8f0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b0ba0 con 0x7f96d41544e0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d414eec0 con 0x7f96d415e2b0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d27fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52594/0 (socket says 172.21.15.179:52594) 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96d41544e0 0x7f96d4158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d27fc700 1 -- 172.21.15.179:0/3257024386 learned_addr learned my addr 172.21.15.179:0/3257024386 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.438+0000 7f96d37fe700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415e2b0 0x7f96d4147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 -- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415e2b0 msgr2=0x7f96d4147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415e2b0 0x7f96d4147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 -- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96d41544e0 msgr2=0x7f96d4158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96d41544e0 0x7f96d4158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 -- 172.21.15.179:0/3257024386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009b30 con 0x7f96d415d8f0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d27fc700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f96c800ebe0 tx=0x7f96c800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8009740 con 0x7f96d415d8f0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c800c850 con 0x7f96d415d8f0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d414f1a0 con 0x7f96d415d8f0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800cbe0 con 0x7f96d415d8f0 2024-05-08T22:39:47.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4158c80 con 0x7f96d415d8f0 2024-05-08T22:39:47.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d414f3e0 con 0x7f96d415d8f0 2024-05-08T22:39:47.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96c800cd40 con 0x7f96d415d8f0 2024-05-08T22:39:47.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 0x7f96c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f96c8084360 con 0x7f96d415d8f0 2024-05-08T22:39:47.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.442+0000 7f96d2ffd700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 0x7f96c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.446+0000 7f96d2ffd700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 0x7f96c0068bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009970 tx=0x7f96c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.450+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f96d414f3e0 con 0x7f96d415d8f0 2024-05-08T22:39:47.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.618+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96d414f3e0 con 0x7f96d415d8f0 2024-05-08T22:39:47.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.618+0000 7f96bbfff700 1 -- 172.21.15.179:0/3257024386 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96d414f3e0 con 0x7f96d415d8f0 2024-05-08T22:39:47.622 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 msgr2=0x7f96c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 0x7f96c0068bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009970 tx=0x7f96c4008040 comp rx=0 tx=0).stop 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 msgr2=0x7f96d40b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f96c800ebe0 tx=0x7f96c800eef0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 shutdown_connections 2024-05-08T22:39:47.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f96c0066730 0x7f96c0068bf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f96d415e2b0 0x7f96d4147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96d415d8f0 0x7f96d40b0100 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 --2- 172.21.15.179:0/3257024386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96d41544e0 0x7f96d4158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 >> 172.21.15.179:0/3257024386 conn(0x7f96d40b18d0 msgr2=0x7f96d40cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 shutdown_connections 2024-05-08T22:39:47.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.622+0000 7f96d3fff700 1 -- 172.21.15.179:0/3257024386 wait complete. 2024-05-08T22:39:47.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:47.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 Processor -- start 2024-05-08T22:39:47.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 -- start start 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 0x7f8c840ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8c8415b1b0 0x7f8c840cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8406abb0 con 0x7f8c840cb510 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c840cb9a0 con 0x7f8c8415b1b0 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.746+0000 7f8c8884c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8406ae90 con 0x7f8c840cc700 2024-05-08T22:39:47.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8c8415b1b0 0x7f8c840cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c83fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8c840cb510 0x7f8c8415e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46546/0 (socket says 172.21.15.179:46546) 2024-05-08T22:39:47.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c83fff700 1 -- 172.21.15.179:0/19031998 learned_addr learned my addr 172.21.15.179:0/19031998 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:47.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c837fe700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 0x7f8c840ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 -- 172.21.15.179:0/19031998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 msgr2=0x7f8c840ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 0x7f8c840ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 -- 172.21.15.179:0/19031998 >> v1:172.21.15.179:6789/0 conn(0x7f8c840cb510 legacy=0x7f8c8415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c827fc700 1 -- 172.21.15.179:0/19031998 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666209484 0 0) 0x7f8c8406abb0 con 0x7f8c840cb510 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 -- 172.21.15.179:0/19031998 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8415b9c0 con 0x7f8c8415b1b0 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c837fe700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 0x7f8c840ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c82ffd700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8415b1b0 0x7f8c840cafd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009850 tx=0x7f8c6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64a1590da76db36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c827fc700 1 -- 172.21.15.179:0/19031998 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c011830 con 0x7f8c8415b1b0 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c827fc700 1 -- 172.21.15.179:0/19031998 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c011990 con 0x7f8c8415b1b0 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c827fc700 1 -- 172.21.15.179:0/19031998 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c011d20 con 0x7f8c8415b1b0 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- 172.21.15.179:0/19031998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8415b1b0 msgr2=0x7f8c840cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8415b1b0 0x7f8c840cafd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009850 tx=0x7f8c6c009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- 172.21.15.179:0/19031998 shutdown_connections 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8415b1b0 0x7f8c840cafd0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 --2- 172.21.15.179:0/19031998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cc700 0x7f8c840ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- 172.21.15.179:0/19031998 >> 172.21.15.179:0/19031998 conn(0x7f8c840ce9d0 msgr2=0x7f8c840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- 172.21.15.179:0/19031998 shutdown_connections 2024-05-08T22:39:47.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- 172.21.15.179:0/19031998 wait complete. 2024-05-08T22:39:47.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 Processor -- start 2024-05-08T22:39:47.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.750+0000 7f8c8884c700 1 -- start start 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cb510 0x7f8c8414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c8415b1b0 0x7f8c8415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8414f080 con 0x7f8c840cc700 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b2700 con 0x7f8c8415b1b0 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b1af0 con 0x7f8c840cb510 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52606/0 (socket says 172.21.15.179:52606) 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 -- 172.21.15.179:0/483613125 learned_addr learned my addr 172.21.15.179:0/483613125 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c837fe700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cb510 0x7f8c8414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c83fff700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c8415b1b0 0x7f8c8415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 -- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cb510 msgr2=0x7f8c8414eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cb510 0x7f8c8414eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 -- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c8415b1b0 msgr2=0x7f8c8415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c8415b1b0 0x7f8c8415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 -- 172.21.15.179:0/483613125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0094b0 con 0x7f8c840cc700 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c82ffd700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009fd0 tx=0x7f8c6c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0045d0 con 0x7f8c840cc700 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01f070 con 0x7f8c840cc700 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c840b1dd0 con 0x7f8c840cc700 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c003900 con 0x7f8c840cc700 2024-05-08T22:39:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.754+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c840b2390 con 0x7f8c840cc700 2024-05-08T22:39:47.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c84067740 con 0x7f8c840cc700 2024-05-08T22:39:47.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c6c003a60 con 0x7f8c840cc700 2024-05-08T22:39:47.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c7bfff700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 0x7f8c68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:47.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8c6c021070 con 0x7f8c840cc700 2024-05-08T22:39:47.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c837fe700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 0x7f8c68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:47.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.758+0000 7f8c837fe700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 0x7f8c68068bf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c84151be0 tx=0x7f8c74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:47.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.766+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c6c052d80 con 0x7f8c840cc700 2024-05-08T22:39:47.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.974+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c8404e700 con 0x7f8c840cc700 2024-05-08T22:39:47.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.974+0000 7f8c7bfff700 1 -- 172.21.15.179:0/483613125 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f8c6c02b090 con 0x7f8c840cc700 2024-05-08T22:39:47.978 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:47.978 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:39:12.053827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:47.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 msgr2=0x7f8c68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 0x7f8c68068bf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c84151be0 tx=0x7f8c74008040 comp rx=0 tx=0).stop 2024-05-08T22:39:47.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 msgr2=0x7f8c8415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:47.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009fd0 tx=0x7f8c6c004770 comp rx=0 tx=0).stop 2024-05-08T22:39:47.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 shutdown_connections 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8c68066730 0x7f8c68068bf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c8415b1b0 0x7f8c8415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c840cc700 0x7f8c8415c2f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 --2- 172.21.15.179:0/483613125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c840cb510 0x7f8c8414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 >> 172.21.15.179:0/483613125 conn(0x7f8c840ce9d0 msgr2=0x7f8c840aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 shutdown_connections 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:47.978+0000 7f8c8884c700 1 -- 172.21.15.179:0/483613125 wait complete. 2024-05-08T22:39:47.983 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 47 2024-05-08T22:39:47.992 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:48.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 Processor -- start 2024-05-08T22:39:48.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 -- start start 2024-05-08T22:39:48.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5f600cde70 0x7f5f600ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600ce810 con 0x7f5f6015d5b0 2024-05-08T22:39:48.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6006afc0 con 0x7f5f6006b450 2024-05-08T22:39:48.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f6485b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6015d990 con 0x7f5f600cde70 2024-05-08T22:39:48.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f5ed9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5f6015d5b0 0x7f5f600b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46554/0 (socket says 172.21.15.179:46554) 2024-05-08T22:39:48.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f5ed9d700 1 -- 172.21.15.179:0/2941150778 learned_addr learned my addr 172.21.15.179:0/2941150778 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:48.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f5e59c700 1 --2- 172.21.15.179:0/2941150778 >> v2:172.21.15.195:3300/0 conn(0x7f5f600cde70 0x7f5f600ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542592422 0 0) 0x7f5f600ce810 con 0x7f5f6015d5b0 2024-05-08T22:39:48.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.102+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f4c0035d0 con 0x7f5f6015d5b0 2024-05-08T22:39:48.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671814259 0 0) 0x7f5f6006afc0 con 0x7f5f6006b450 2024-05-08T22:39:48.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f600ce810 con 0x7f5f6006b450 2024-05-08T22:39:48.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5e59c700 1 -- 172.21.15.179:0/2941150778 >> v1:172.21.15.195:6789/0 conn(0x7f5f6006b450 legacy=0x7f5f6006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5e59c700 1 -- 172.21.15.179:0/2941150778 >> v1:172.21.15.179:6789/0 conn(0x7f5f6015d5b0 legacy=0x7f5f600b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443383487 0 0) 0x7f5f4c0035d0 con 0x7f5f6015d5b0 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5e59c700 1 -- 172.21.15.179:0/2941150778 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600d26f0 con 0x7f5f600cde70 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5e59c700 1 --2- 172.21.15.179:0/2941150778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600ce2d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009850 tx=0x7f5f50009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d64fc08b42987725 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48001670 con 0x7f5f600cde70 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50004030 con 0x7f5f600cde70 2024-05-08T22:39:48.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f5dd9b700 1 -- 172.21.15.179:0/2941150778 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500043a0 con 0x7f5f600cde70 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 -- 172.21.15.179:0/2941150778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 msgr2=0x7f5f600ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 --2- 172.21.15.179:0/2941150778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600ce2d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009850 tx=0x7f5f50009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 -- 172.21.15.179:0/2941150778 shutdown_connections 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 --2- 172.21.15.179:0/2941150778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600ce2d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 -- 172.21.15.179:0/2941150778 >> 172.21.15.179:0/2941150778 conn(0x7f5f600b4940 msgr2=0x7f5f600b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 -- 172.21.15.179:0/2941150778 shutdown_connections 2024-05-08T22:39:48.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.106+0000 7f5f6485b700 1 -- 172.21.15.179:0/2941150778 wait complete. 2024-05-08T22:39:48.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 Processor -- start 2024-05-08T22:39:48.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 -- start start 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f6015d5b0 0x7f5f60155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6015fb10 con 0x7f5f6006b450 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6015ebb0 con 0x7f5f600cde70 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f6485b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6015ee90 con 0x7f5f6015d5b0 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f5ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f5ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52612/0 (socket says 172.21.15.179:52612) 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f5ed9d700 1 -- 172.21.15.179:0/4178198720 learned_addr learned my addr 172.21.15.179:0/4178198720 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f5e59c700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.110+0000 7f5f5f59e700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f6015d5b0 0x7f5f60155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 -- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f6015d5b0 msgr2=0x7f5f60155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f6015d5b0 0x7f5f60155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 -- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 msgr2=0x7f5f600cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 -- 172.21.15.179:0/4178198720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f500094b0 con 0x7f5f6006b450 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5ed9d700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c750 tx=0x7f5f4800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48004ca0 con 0x7f5f6006b450 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f48004e00 con 0x7f5f6006b450 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6015f170 con 0x7f5f6006b450 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480072d0 con 0x7f5f6006b450 2024-05-08T22:39:48.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6015f730 con 0x7f5f6006b450 2024-05-08T22:39:48.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6015fd80 con 0x7f5f6006b450 2024-05-08T22:39:48.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f48007430 con 0x7f5f6006b450 2024-05-08T22:39:48.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 0x7f5f40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5f48084030 con 0x7f5f6006b450 2024-05-08T22:39:48.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.114+0000 7f5f5e59c700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 0x7f5f40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.118+0000 7f5f5e59c700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 0x7f5f40068bf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f500090b0 tx=0x7f5f50008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.122+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5f48051200 con 0x7f5f6006b450 2024-05-08T22:39:48.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.298+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f6006e4a0 con 0x7f5f6006b450 2024-05-08T22:39:48.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f577fe700 1 -- 172.21.15.179:0/4178198720 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f5f48054820 con 0x7f5f6006b450 2024-05-08T22:39:48.303 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:48.303 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:48.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 msgr2=0x7f5f40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 0x7f5f40068bf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f500090b0 tx=0x7f5f50008030 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 msgr2=0x7f5f6015ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c750 tx=0x7f5f4800ca60 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 shutdown_connections 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f40066730 0x7f5f40068bf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f600cde70 0x7f5f600cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f6015d5b0 0x7f5f60155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 --2- 172.21.15.179:0/4178198720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f6006b450 0x7f5f6015ff30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 >> 172.21.15.179:0/4178198720 conn(0x7f5f600b4940 msgr2=0x7f5f6006b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 shutdown_connections 2024-05-08T22:39:48.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.302+0000 7f5f6485b700 1 -- 172.21.15.179:0/4178198720 wait complete. 2024-05-08T22:39:48.318 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:39:48.319 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:48.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 Processor -- start 2024-05-08T22:39:48.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 -- start start 2024-05-08T22:39:48.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cb510 0x7f0c9c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c06abb0 con 0x7f0c9c0cc700 2024-05-08T22:39:48.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9c0cb9a0 con 0x7f0c9c0ce380 2024-05-08T22:39:48.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c06ae90 con 0x7f0c9c0cb510 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52622/0 (socket says 172.21.15.179:52622) 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 -- 172.21.15.179:0/3520520276 learned_addr learned my addr 172.21.15.179:0/3520520276 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca1bae700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cb510 0x7f0c9c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 -- 172.21.15.179:0/3520520276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cb510 msgr2=0x7f0c9c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cb510 0x7f0c9c06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 -- 172.21.15.179:0/3520520276 >> v1:172.21.15.179:6789/0 conn(0x7f0c9c0cc700 legacy=0x7f0c9c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 -- 172.21.15.179:0/3520520276 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c0ce800 con 0x7f0c9c0ce380 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca0bac700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1c4384f4d630f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0c9bfff700 1 -- 172.21.15.179:0/3520520276 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00e070 con 0x7f0c9c0ce380 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0c9bfff700 1 -- 172.21.15.179:0/3520520276 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c00b9c0 con 0x7f0c9c0ce380 2024-05-08T22:39:48.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0c9bfff700 1 -- 172.21.15.179:0/3520520276 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00bd70 con 0x7f0c9c0ce380 2024-05-08T22:39:48.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 -- 172.21.15.179:0/3520520276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0ce380 msgr2=0x7f0c9c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.430+0000 7f0ca23af700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009e50 tx=0x7f0c8c00b330 comp rx=0 tx=0).stop 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/3520520276 shutdown_connections 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cb510 0x7f0c9c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 --2- 172.21.15.179:0/3520520276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c0cafd0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/3520520276 >> 172.21.15.179:0/3520520276 conn(0x7f0c9c0ce9d0 msgr2=0x7f0c9c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/3520520276 shutdown_connections 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/3520520276 wait complete. 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 Processor -- start 2024-05-08T22:39:48.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- start start 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cc700 0x7f0c9c1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c06d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c13f490 con 0x7f0c9c0cb510 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1f16b0 con 0x7f0c9c0ce380 2024-05-08T22:39:48.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c13ece0 con 0x7f0c9c0cc700 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52634/0 (socket says 172.21.15.179:52634) 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 -- 172.21.15.179:0/4000016422 learned_addr learned my addr 172.21.15.179:0/4000016422 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca0bac700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cc700 0x7f0c9c1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca1bae700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 -- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cc700 msgr2=0x7f0c9c1f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cc700 0x7f0c9c1f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 -- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 msgr2=0x7f0c9c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c06d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 -- 172.21.15.179:0/4000016422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009b30 con 0x7f0c9c0cb510 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca1bae700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c06d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca13ad700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c500 tx=0x7f0c9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900052d0 con 0x7f0c9c0cb510 2024-05-08T22:39:48.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90005430 con 0x7f0c9c0cb510 2024-05-08T22:39:48.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1f10f0 con 0x7f0c9c0cb510 2024-05-08T22:39:48.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900055a0 con 0x7f0c9c0cb510 2024-05-08T22:39:48.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.434+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c13f120 con 0x7f0c9c0cb510 2024-05-08T22:39:48.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c900038e0 con 0x7f0c9c0cb510 2024-05-08T22:39:48.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c067740 con 0x7f0c9c0cb510 2024-05-08T22:39:48.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0c99ffb700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0c90083850 con 0x7f0c9c0cb510 2024-05-08T22:39:48.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0ca0bac700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.438+0000 7f0ca0bac700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 0x7f0c88068bf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b370 tx=0x7f0c8c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.446+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0c90050a00 con 0x7f0c9c0cb510 2024-05-08T22:39:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.654+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c9c04e700 con 0x7f0c9c0cb510 2024-05-08T22:39:48.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0c99ffb700 1 -- 172.21.15.179:0/4000016422 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f0c90054020 con 0x7f0c9c0cb510 2024-05-08T22:39:48.659 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:48.659 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:39:12.053827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 msgr2=0x7f0c88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 0x7f0c88068bf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b370 tx=0x7f0c8c009000 comp rx=0 tx=0).stop 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 msgr2=0x7f0c9c13e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c500 tx=0x7f0c9000c8c0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 shutdown_connections 2024-05-08T22:39:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0c88066730 0x7f0c88068bf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c9c0ce380 0x7f0c9c06d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c9c0cc700 0x7f0c9c1f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 --2- 172.21.15.179:0/4000016422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9c0cb510 0x7f0c9c13e7a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 >> 172.21.15.179:0/4000016422 conn(0x7f0c9c0ce9d0 msgr2=0x7f0c9c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 shutdown_connections 2024-05-08T22:39:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.658+0000 7f0ca23af700 1 -- 172.21.15.179:0/4000016422 wait complete. 2024-05-08T22:39:48.664 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 47 2024-05-08T22:39:48.674 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:48.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 Processor -- start 2024-05-08T22:39:48.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 -- start start 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 0x7f63280cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632806abb0 con 0x7f63280cb510 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632806ae90 con 0x7f63280cc700 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e8ef700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280cb9a0 con 0x7f63280ce380 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e0ee700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f63280cb510 0x7f632806b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46564/0 (socket says 172.21.15.179:46564) 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.786+0000 7f632e0ee700 1 -- 172.21.15.179:0/2277621760 learned_addr learned my addr 172.21.15.179:0/2277621760 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:48.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632d0ec700 1 --2- 172.21.15.179:0/2277621760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 0x7f63280cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399817924 0 0) 0x7f632806abb0 con 0x7f63280cb510 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63140035d0 con 0x7f63280cb510 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739464701 0 0) 0x7f632806ae90 con 0x7f63280cc700 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632806abb0 con 0x7f63280cc700 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632d0ec700 1 -- 172.21.15.179:0/2277621760 >> v1:172.21.15.195:6789/0 conn(0x7f63280cc700 legacy=0x7f632815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632d0ec700 1 -- 172.21.15.179:0/2277621760 >> v1:172.21.15.179:6789/0 conn(0x7f63280cb510 legacy=0x7f632806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632d0ec700 1 -- 172.21.15.179:0/2277621760 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280ce800 con 0x7f63280ce380 2024-05-08T22:39:48.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632d0ec700 1 --2- 172.21.15.179:0/2277621760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 0x7f63280cafd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6318009880 tx=0x7f6318009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb733c94c78ea1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318011830 con 0x7f63280ce380 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318011990 con 0x7f63280ce380 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632c8eb700 1 -- 172.21.15.179:0/2277621760 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318011d00 con 0x7f63280ce380 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 -- 172.21.15.179:0/2277621760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 msgr2=0x7f63280cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 --2- 172.21.15.179:0/2277621760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 0x7f63280cafd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6318009880 tx=0x7f6318009b90 comp rx=0 tx=0).stop 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 -- 172.21.15.179:0/2277621760 shutdown_connections 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 --2- 172.21.15.179:0/2277621760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280ce380 0x7f63280cafd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 -- 172.21.15.179:0/2277621760 >> 172.21.15.179:0/2277621760 conn(0x7f63280ce9d0 msgr2=0x7f63280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 -- 172.21.15.179:0/2277621760 shutdown_connections 2024-05-08T22:39:48.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.790+0000 7f632e8ef700 1 -- 172.21.15.179:0/2277621760 wait complete. 2024-05-08T22:39:48.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 Processor -- start 2024-05-08T22:39:48.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- start start 2024-05-08T22:39:48.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 0x7f6328157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63280cc700 0x7f63281f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281577c0 con 0x7f63280ce380 2024-05-08T22:39:48.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281f35b0 con 0x7f63280cc700 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6328156bb0 con 0x7f63280cb510 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52636/0 (socket says 172.21.15.179:52636) 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 -- 172.21.15.179:0/3154951571 learned_addr learned my addr 172.21.15.179:0/3154951571 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632d0ec700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63280cc700 0x7f63281f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632d8ed700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 0x7f6328157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 -- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 msgr2=0x7f6328157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 0x7f6328157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 -- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63280cc700 msgr2=0x7f63281f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63280cc700 0x7f63281f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 -- 172.21.15.179:0/3154951571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180094b0 con 0x7f63280ce380 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632d8ed700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 0x7f6328157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e0ee700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4f0 tx=0x7f632000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320005d90 con 0x7f63280ce380 2024-05-08T22:39:48.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6328156e90 con 0x7f63280ce380 2024-05-08T22:39:48.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63200070d0 con 0x7f63280ce380 2024-05-08T22:39:48.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320007440 con 0x7f63280ce380 2024-05-08T22:39:48.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.794+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6328157450 con 0x7f63280ce380 2024-05-08T22:39:48.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6328157970 con 0x7f63280ce380 2024-05-08T22:39:48.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63200075a0 con 0x7f63280ce380 2024-05-08T22:39:48.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f631e7fc700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 0x7f630c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:48.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6320083630 con 0x7f63280ce380 2024-05-08T22:39:48.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f632d8ed700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 0x7f630c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:48.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.798+0000 7f632d8ed700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 0x7f630c068bf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6324002230 tx=0x7f6324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:48.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:48.806+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6328157970 con 0x7f63280ce380 2024-05-08T22:39:49.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.014+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6328157970 con 0x7f63280ce380 2024-05-08T22:39:49.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.014+0000 7f631e7fc700 1 -- 172.21.15.179:0/3154951571 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f6328157970 con 0x7f63280ce380 2024-05-08T22:39:49.016 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:49.016 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:37:57.812267+0000","modified":"2024-05-08T22:39:12.053827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.195:6837/1691042534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":1691042534},{"type":"v1","addr":"172.21.15.195:6837","nonce":1691042534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T22:39:49.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 msgr2=0x7f630c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 0x7f630c068bf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6324002230 tx=0x7f6324009040 comp rx=0 tx=0).stop 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 msgr2=0x7f63280b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4f0 tx=0x7f632000c800 comp rx=0 tx=0).stop 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 shutdown_connections 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f630c066730 0x7f630c068bf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63280cc700 0x7f63281f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63280ce380 0x7f63280b3570 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 --2- 172.21.15.179:0/3154951571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63280cb510 0x7f6328157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 >> 172.21.15.179:0/3154951571 conn(0x7f63280ce9d0 msgr2=0x7f63280ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 shutdown_connections 2024-05-08T22:39:49.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.018+0000 7f632e8ef700 1 -- 172.21.15.179:0/3154951571 wait complete. 2024-05-08T22:39:49.022 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 47 2024-05-08T22:39:49.031 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:49.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 Processor -- start 2024-05-08T22:39:49.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 -- start start 2024-05-08T22:39:49.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5a80b6b90 0x7ff5a80c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80b46f0 con 0x7ff5a815b0c0 2024-05-08T22:39:49.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80b49c0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5aebf5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80b4ca0 con 0x7ff5a80b6b90 2024-05-08T22:39:49.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5ac991700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5a815b0c0 0x7ff5a80b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46580/0 (socket says 172.21.15.179:46580) 2024-05-08T22:39:49.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5ac991700 1 -- 172.21.15.179:0/994501824 learned_addr learned my addr 172.21.15.179:0/994501824 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:49.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a7fff700 1 --2- 172.21.15.179:0/994501824 >> v2:172.21.15.179:3300/0 conn(0x7ff5a80b6b90 0x7ff5a80c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856998674 0 0) 0x7ff5a80b46f0 con 0x7ff5a815b0c0 2024-05-08T22:39:49.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5880035d0 con 0x7ff5a815b0c0 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a7fff700 1 -- 172.21.15.179:0/994501824 >> v1:172.21.15.195:6789/0 conn(0x7ff5a80c4f40 legacy=0x7ff5a80d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187492045 0 0) 0x7ff5a80b49c0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a7fff700 1 -- 172.21.15.179:0/994501824 >> v1:172.21.15.179:6789/0 conn(0x7ff5a815b0c0 legacy=0x7ff5a80b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a7fff700 1 -- 172.21.15.179:0/994501824 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80b4f80 con 0x7ff5a80b6b90 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a7fff700 1 --2- 172.21.15.179:0/994501824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80b6b90 0x7ff5a80c4a00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff598009850 tx=0x7ff598009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eee77c0383b3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590001970 con 0x7ff5a80b6b90 2024-05-08T22:39:49.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598010590 con 0x7ff5a80b6b90 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.146+0000 7ff5a77fe700 1 -- 172.21.15.179:0/994501824 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598010940 con 0x7ff5a80b6b90 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- 172.21.15.179:0/994501824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80b6b90 msgr2=0x7ff5a80c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/994501824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80b6b90 0x7ff5a80c4a00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff598009850 tx=0x7ff598009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- 172.21.15.179:0/994501824 shutdown_connections 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/994501824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80b6b90 0x7ff5a80c4a00 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- 172.21.15.179:0/994501824 >> 172.21.15.179:0/994501824 conn(0x7ff5a80d0eb0 msgr2=0x7ff5a80d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- 172.21.15.179:0/994501824 shutdown_connections 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- 172.21.15.179:0/994501824 wait complete. 2024-05-08T22:39:49.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 Processor -- start 2024-05-08T22:39:49.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- start start 2024-05-08T22:39:49.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5a80b6b90 0x7ff5a8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 0x7ff5a80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c8de0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c90c0 con 0x7ff5a815b0c0 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5aebf5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c93a0 con 0x7ff5a80b6b90 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52652/0 (socket says 172.21.15.179:52652) 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5a7fff700 1 -- 172.21.15.179:0/1794028648 learned_addr learned my addr 172.21.15.179:0/1794028648 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5ac991700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5a80b6b90 0x7ff5a8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.150+0000 7ff5ad192700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 0x7ff5a80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 -- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5a80b6b90 msgr2=0x7ff5a8155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5a80b6b90 0x7ff5a8155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 -- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 msgr2=0x7ff5a80c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 0x7ff5a80c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 -- 172.21.15.179:0/1794028648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980094b0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5ad192700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 0x7ff5a80c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:49.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a7fff700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff598013040 tx=0x7ff598004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980036c0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598003820 con 0x7ff5a80c4f40 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598003bd0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a80c9680 con 0x7ff5a80c4f40 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8155960 con 0x7ff5a80c4f40 2024-05-08T22:39:49.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.154+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a8067770 con 0x7ff5a80c4f40 2024-05-08T22:39:49.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.158+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff598003d30 con 0x7ff5a80c4f40 2024-05-08T22:39:49.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.158+0000 7ff5a57fa700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 0x7ff58c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.158+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff598085820 con 0x7ff5a80c4f40 2024-05-08T22:39:49.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.158+0000 7ff5ac991700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 0x7ff58c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.158+0000 7ff5ac991700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 0x7ff58c068bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff5a815e820 tx=0x7ff590001bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.162+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff598052a80 con 0x7ff5a80c4f40 2024-05-08T22:39:49.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.338+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5a80c9870 con 0x7ff5a80c4f40 2024-05-08T22:39:49.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.338+0000 7ff5a57fa700 1 -- 172.21.15.179:0/1794028648 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16486 (secure 0 0 0) 0x7ff5980560a0 con 0x7ff5a80c4f40 2024-05-08T22:39:49.342 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:49.342 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":84,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:38:20.877365+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:37:56.159297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T22:37:56.796186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 msgr2=0x7ff58c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 0x7ff58c068bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff5a815e820 tx=0x7ff590001bc0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 msgr2=0x7ff5a80c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff598013040 tx=0x7ff598004770 comp rx=0 tx=0).stop 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 shutdown_connections 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff58c066730 0x7ff58c068bf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5a815b0c0 0x7ff5a80c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5a80c4f40 0x7ff5a80c8340 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 --2- 172.21.15.179:0/1794028648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5a80b6b90 0x7ff5a8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 >> 172.21.15.179:0/1794028648 conn(0x7ff5a80d0eb0 msgr2=0x7ff5a80cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 shutdown_connections 2024-05-08T22:39:49.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.342+0000 7ff5aebf5700 1 -- 172.21.15.179:0/1794028648 wait complete. 2024-05-08T22:39:49.358 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:39:49.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.470+0000 7f4930e03700 1 Processor -- start 2024-05-08T22:39:49.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.470+0000 7f4930e03700 1 -- start start 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f492c15ae40 0x7f492c06d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c06e480 0x7f492c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c06db90 con 0x7f492c143880 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c06ee50 con 0x7f492c15ae40 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0ce2f0 con 0x7f492c06e480 2024-05-08T22:39:49.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f492c15ae40 0x7f492c06d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f492c15ae40 0x7f492c06d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52668/0 (socket says 172.21.15.179:52668) 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 -- 172.21.15.179:0/1026932661 learned_addr learned my addr 172.21.15.179:0/1026932661 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f492bfff700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c06e480 0x7f492c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 -- 172.21.15.179:0/1026932661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c06e480 msgr2=0x7f492c06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c06e480 0x7f492c06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 -- 172.21.15.179:0/1026932661 >> v1:172.21.15.195:6789/0 conn(0x7f492c143880 legacy=0x7f492c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 -- 172.21.15.179:0/1026932661 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0ce5d0 con 0x7f492c15ae40 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4923fff700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c15ae40 0x7f492c06d340 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f4914009910 tx=0x7f4914009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd82de8edfbae5a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f492affd700 1 -- 172.21.15.179:0/1026932661 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914011830 con 0x7f492c15ae40 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f492affd700 1 -- 172.21.15.179:0/1026932661 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4914011990 con 0x7f492c15ae40 2024-05-08T22:39:49.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f492affd700 1 -- 172.21.15.179:0/1026932661 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914011d40 con 0x7f492c15ae40 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- 172.21.15.179:0/1026932661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c15ae40 msgr2=0x7f492c06d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c15ae40 0x7f492c06d340 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f4914009910 tx=0x7f4914009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- 172.21.15.179:0/1026932661 shutdown_connections 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c06e480 0x7f492c06e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 --2- 172.21.15.179:0/1026932661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c15ae40 0x7f492c06d340 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- 172.21.15.179:0/1026932661 >> 172.21.15.179:0/1026932661 conn(0x7f492c0b1ed0 msgr2=0x7f492c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- 172.21.15.179:0/1026932661 shutdown_connections 2024-05-08T22:39:49.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.474+0000 7f4930e03700 1 -- 172.21.15.179:0/1026932661 wait complete. 2024-05-08T22:39:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 Processor -- start 2024-05-08T22:39:49.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 -- start start 2024-05-08T22:39:49.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c143880 0x7f492c0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c15ae40 0x7f492c1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c1f30c0 con 0x7f492c06e480 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0af4f0 con 0x7f492c15ae40 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4930e03700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c15ddb0 con 0x7f492c143880 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52684/0 (socket says 172.21.15.179:52684) 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 -- 172.21.15.179:0/312376215 learned_addr learned my addr 172.21.15.179:0/312376215 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:49.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492bfff700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c15ae40 0x7f492c1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f4923fff700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c143880 0x7f492c0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 -- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c143880 msgr2=0x7f492c0aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c143880 0x7f492c0aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 -- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c15ae40 msgr2=0x7f492c1f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.478+0000 7f492b7fe700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c15ae40 0x7f492c1f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f492b7fe700 1 -- 172.21.15.179:0/312376215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4914009570 con 0x7f492c06e480 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f492b7fe700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c530 tx=0x7f491c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0052d0 con 0x7f492c06e480 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c15e090 con 0x7f492c06e480 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c005430 con 0x7f492c06e480 2024-05-08T22:39:49.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0055a0 con 0x7f492c06e480 2024-05-08T22:39:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c1f2ce0 con 0x7f492c06e480 2024-05-08T22:39:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f491c0038e0 con 0x7f492c06e480 2024-05-08T22:39:49.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c04e700 con 0x7f492c06e480 2024-05-08T22:39:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 0x7f490c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f491c0836e0 con 0x7f492c06e480 2024-05-08T22:39:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.482+0000 7f4923fff700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 0x7f490c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.486+0000 7f4923fff700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 0x7f490c068bf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f4914008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.490+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f491c0508b0 con 0x7f492c06e480 2024-05-08T22:39:49.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.690+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f492c0b4da0 con 0x7f492c06e480 2024-05-08T22:39:49.694 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:39:49.694+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:39:49.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4928ff9700 1 -- 172.21.15.179:0/312376215 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f491c053ed0 con 0x7f492c06e480 2024-05-08T22:39:49.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 msgr2=0x7f490c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 0x7f490c068bf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f4914008030 comp rx=0 tx=0).stop 2024-05-08T22:39:49.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 msgr2=0x7f492c15d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c530 tx=0x7f491c00c840 comp rx=0 tx=0).stop 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 shutdown_connections 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f490c066730 0x7f490c068bf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c15ae40 0x7f492c1f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c143880 0x7f492c0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 --2- 172.21.15.179:0/312376215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f492c06e480 0x7f492c15d760 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 >> 172.21.15.179:0/312376215 conn(0x7f492c0b1ed0 msgr2=0x7f492c15a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 shutdown_connections 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.706+0000 7f4930e03700 1 -- 172.21.15.179:0/312376215 wait complete. 2024-05-08T22:39:49.711 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:39:49.714 INFO:tasks.ceph.mds.d.smithi195.stderr: -4081> 2024-05-08T22:38:26.903+0000 7fb216e91700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:39:49.725 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:39:49.747 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:39:49.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.846+0000 7f4713fff700 1 Processor -- start 2024-05-08T22:39:49.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.846+0000 7f4713fff700 1 -- start start 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4713fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4714143580 0x7f471415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4713fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471406bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4713fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471415e050 con 0x7f471406c100 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4713fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471406c570 con 0x7f4714143580 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4713fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471406f0a0 con 0x7f471415ae10 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4714143580 0x7f471415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f47127fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471406bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f47137fe700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f471406c100 0x7f47140cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43178/0 (socket says 172.21.15.179:43178) 2024-05-08T22:39:49.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4714143580 0x7f471415e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:57240/0 (socket says 172.21.15.179:57240) 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f47137fe700 1 -- 172.21.15.179:0/939823260 learned_addr learned my addr 172.21.15.179:0/939823260 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 -- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 msgr2=0x7f471406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 --2- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471406bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 -- 172.21.15.179:0/939823260 >> v1:172.21.15.195:6789/0 conn(0x7f471406c100 legacy=0x7f47140cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 -- 172.21.15.179:0/939823260 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471415b290 con 0x7f4714143580 2024-05-08T22:39:49.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.850+0000 7f4712ffd700 1 --2- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415e4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f471415b480 tx=0x7f4704009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3400f84188e9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4711ffb700 1 -- 172.21.15.179:0/939823260 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004020 con 0x7f4714143580 2024-05-08T22:39:49.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4711ffb700 1 -- 172.21.15.179:0/939823260 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704004180 con 0x7f4714143580 2024-05-08T22:39:49.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4711ffb700 1 -- 172.21.15.179:0/939823260 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004530 con 0x7f4714143580 2024-05-08T22:39:49.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 msgr2=0x7f471415e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415e4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f471415b480 tx=0x7f4704009b30 comp rx=0 tx=0).stop 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- 172.21.15.179:0/939823260 shutdown_connections 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- 172.21.15.179:0/939823260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415e4e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- 172.21.15.179:0/939823260 >> 172.21.15.179:0/939823260 conn(0x7f4714070d80 msgr2=0x7f4714071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- 172.21.15.179:0/939823260 shutdown_connections 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- 172.21.15.179:0/939823260 wait complete. 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 Processor -- start 2024-05-08T22:39:49.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- start start 2024-05-08T22:39:49.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47141f1a30 con 0x7f471406c100 2024-05-08T22:39:49.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47141f15a0 con 0x7f4714143580 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4713fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140cab10 con 0x7f471415ae10 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4712ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4712ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52688/0 (socket says 172.21.15.179:52688) 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f4712ffd700 1 -- 172.21.15.179:0/1385244104 learned_addr learned my addr 172.21.15.179:0/1385244104 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f47127fc700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.854+0000 7f47137fe700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 -- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 msgr2=0x7f471415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 -- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 msgr2=0x7f471415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 -- 172.21.15.179:0/1385244104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040094b0 con 0x7f471406c100 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4712ffd700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f4714072220 tx=0x7f4704004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704011cb0 con 0x7f471406c100 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704011e10 con 0x7f471406c100 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47141f09a0 con 0x7f471406c100 2024-05-08T22:39:49.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704003900 con 0x7f471406c100 2024-05-08T22:39:49.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47141f0f50 con 0x7f471406c100 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47140ccbf0 con 0x7f471406c100 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4704003a60 con 0x7f471406c100 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 0x7f4700068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f4704085390 con 0x7f471406c100 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.858+0000 7f47127fc700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 0x7f4700068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:49.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.862+0000 7f47127fc700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 0x7f4700068bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f47140acc20 tx=0x7f4708009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:49.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:49.866+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47040526f0 con 0x7f471406c100 2024-05-08T22:39:50.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.078+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f471404e700 con 0x7f471406c100 2024-05-08T22:39:50.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.710+0000 7f46fb7fe700 1 -- 172.21.15.179:0/1385244104 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4704055d10 con 0x7f471406c100 2024-05-08T22:39:50.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 msgr2=0x7f4700068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 0x7f4700068bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f47140acc20 tx=0x7f4708009240 comp rx=0 tx=0).stop 2024-05-08T22:39:50.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 msgr2=0x7f47140ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f4714072220 tx=0x7f4704004770 comp rx=0 tx=0).stop 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 shutdown_connections 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4700066730 0x7f4700068bf0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714143580 0x7f471415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f471415ae10 0x7f471415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 --2- 172.21.15.179:0/1385244104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f471406c100 0x7f47140ca5d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 >> 172.21.15.179:0/1385244104 conn(0x7f4714070d80 msgr2=0x7f47140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:50.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 shutdown_connections 2024-05-08T22:39:50.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.714+0000 7f4713fff700 1 -- 172.21.15.179:0/1385244104 wait complete. 2024-05-08T22:39:50.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:39:50.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 Processor -- start 2024-05-08T22:39:50.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 -- start start 2024-05-08T22:39:50.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b18146760 0x7f7b180cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7b1806dc90 0x7f7b1806af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1806e2b0 con 0x7f7b180cd640 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b180cdaf0 con 0x7f7b18146760 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.854+0000 7f7b1df85700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1806e0b0 con 0x7f7b1806dc90 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b18146760 0x7f7b180cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b18146760 0x7f7b180cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52690/0 (socket says 172.21.15.179:52690) 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 -- 172.21.15.179:0/3394350592 learned_addr learned my addr 172.21.15.179:0/3394350592 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b17fff700 1 --2- 172.21.15.179:0/3394350592 >> v2:172.21.15.195:3300/0 conn(0x7f7b1806dc90 0x7f7b1806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 -- 172.21.15.179:0/3394350592 >> v2:172.21.15.195:3300/0 conn(0x7f7b1806dc90 msgr2=0x7f7b1806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 --2- 172.21.15.179:0/3394350592 >> v2:172.21.15.195:3300/0 conn(0x7f7b1806dc90 0x7f7b1806af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 -- 172.21.15.179:0/3394350592 >> v1:172.21.15.195:6789/0 conn(0x7f7b180cd640 legacy=0x7f7b1806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 -- 172.21.15.179:0/3394350592 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1806e590 con 0x7f7b18146760 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b177fe700 1 --2- 172.21.15.179:0/3394350592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b18146760 0x7f7b180cd100 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806d040 tx=0x7f7b08009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e1f15411cf6f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b16ffd700 1 -- 172.21.15.179:0/3394350592 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08011830 con 0x7f7b18146760 2024-05-08T22:39:50.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b16ffd700 1 -- 172.21.15.179:0/3394350592 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08011990 con 0x7f7b18146760 2024-05-08T22:39:50.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b16ffd700 1 -- 172.21.15.179:0/3394350592 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08011d20 con 0x7f7b18146760 2024-05-08T22:39:50.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 -- 172.21.15.179:0/3394350592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b18146760 msgr2=0x7f7b180cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3394350592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b18146760 0x7f7b180cd100 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806d040 tx=0x7f7b08009bf0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 -- 172.21.15.179:0/3394350592 shutdown_connections 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3394350592 >> v2:172.21.15.195:3300/0 conn(0x7f7b1806dc90 0x7f7b1806af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3394350592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b18146760 0x7f7b180cd100 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 -- 172.21.15.179:0/3394350592 >> 172.21.15.179:0/3394350592 conn(0x7f7b180cb610 msgr2=0x7f7b180cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 -- 172.21.15.179:0/3394350592 shutdown_connections 2024-05-08T22:39:50.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.858+0000 7f7b1df85700 1 -- 172.21.15.179:0/3394350592 wait complete. 2024-05-08T22:39:50.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 Processor -- start 2024-05-08T22:39:50.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- start start 2024-05-08T22:39:50.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1806dc90 0x7f7b180ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b1813f360 0x7f7b180b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180b3780 con 0x7f7b180cd640 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180b2b00 con 0x7f7b1806dc90 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180b2de0 con 0x7f7b1813f360 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52692/0 (socket says 172.21.15.179:52692) 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b177fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1806dc90 0x7f7b180ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 -- 172.21.15.179:0/3578178489 learned_addr learned my addr 172.21.15.179:0/3578178489 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b17fff700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b1813f360 0x7f7b180b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 -- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b1813f360 msgr2=0x7f7b180b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b1813f360 0x7f7b180b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 -- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1806dc90 msgr2=0x7f7b180ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1806dc90 0x7f7b180ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 -- 172.21.15.179:0/3578178489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009570 con 0x7f7b180cd640 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b0effd700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c500 tx=0x7f7b0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000052d0 con 0x7f7b180cd640 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00005430 con 0x7f7b180cd640 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b180b30c0 con 0x7f7b180cd640 2024-05-08T22:39:50.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000055a0 con 0x7f7b180cd640 2024-05-08T22:39:50.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.862+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b180ac6c0 con 0x7f7b180cd640 2024-05-08T22:39:50.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.866+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b180b38c0 con 0x7f7b180cd640 2024-05-08T22:39:50.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.866+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b000038e0 con 0x7f7b180cd640 2024-05-08T22:39:50.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.866+0000 7f7b14ff9700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 0x7f7af8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:50.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.866+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f7b000835c0 con 0x7f7b180cd640 2024-05-08T22:39:50.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.866+0000 7f7b177fe700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 0x7f7af8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:50.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.870+0000 7f7b177fe700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 0x7f7af8068bf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7b18070630 tx=0x7f7b08008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:50.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:50.874+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7b180b38c0 con 0x7f7b180cd640 2024-05-08T22:39:51.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.046+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b180b38c0 con 0x7f7b180cd640 2024-05-08T22:39:51.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.714+0000 7f7b14ff9700 1 -- 172.21.15.179:0/3578178489 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7b180b38c0 con 0x7f7b180cd640 2024-05-08T22:39:51.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.718+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 msgr2=0x7f7af8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 0x7f7af8068bf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7b18070630 tx=0x7f7b08008030 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 msgr2=0x7f7b1813ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c500 tx=0x7f7b0000c810 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 shutdown_connections 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7af8066730 0x7f7af8068bf0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b1813f360 0x7f7b180b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b180cd640 0x7f7b1813ee20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 --2- 172.21.15.179:0/3578178489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b1806dc90 0x7f7b180ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 >> 172.21.15.179:0/3578178489 conn(0x7f7b180cb610 msgr2=0x7f7b1815ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 shutdown_connections 2024-05-08T22:39:51.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.722+0000 7f7b1df85700 1 -- 172.21.15.179:0/3578178489 wait complete. 2024-05-08T22:39:51.724 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:39:51.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:39:51.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 Processor -- start 2024-05-08T22:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 -- start start 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7b54157be0 0x7f7b5406e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b5406cd50 0x7f7b5415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5406ddf0 con 0x7f7b54157800 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5406e7c0 con 0x7f7b5406cd50 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b5a625700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5406d1e0 con 0x7f7b54157be0 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59623700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7b54157800 0x7f7b540cdd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46588/0 (socket says 172.21.15.179:46588) 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59e24700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b5406cd50 0x7f7b5415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59623700 1 -- 172.21.15.179:0/1913840232 learned_addr learned my addr 172.21.15.179:0/1913840232 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b58e22700 1 --2- 172.21.15.179:0/1913840232 >> v2:172.21.15.195:3300/0 conn(0x7f7b54157be0 0x7f7b5406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59e24700 1 -- 172.21.15.179:0/1913840232 >> v2:172.21.15.195:3300/0 conn(0x7f7b54157be0 msgr2=0x7f7b5406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59e24700 1 --2- 172.21.15.179:0/1913840232 >> v2:172.21.15.195:3300/0 conn(0x7f7b54157be0 0x7f7b5406e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59e24700 1 -- 172.21.15.179:0/1913840232 >> v1:172.21.15.179:6789/0 conn(0x7f7b54157800 legacy=0x7f7b540cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b59e24700 1 -- 172.21.15.179:0/1913840232 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5406d4c0 con 0x7f7b5406cd50 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.854+0000 7f7b4bfff700 1 -- 172.21.15.179:0/1913840232 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213931149 0 0) 0x7f7b5406ddf0 con 0x7f7b54157800 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b59e24700 1 --2- 172.21.15.179:0/1913840232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b5406cd50 0x7f7b5415da30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7b540ae5b0 tx=0x7f7b4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce070b8928f4db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:51.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b4bfff700 1 -- 172.21.15.179:0/1913840232 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00e070 con 0x7f7b5406cd50 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b4bfff700 1 -- 172.21.15.179:0/1913840232 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00bb20 con 0x7f7b5406cd50 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b4bfff700 1 -- 172.21.15.179:0/1913840232 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c012430 con 0x7f7b5406cd50 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- 172.21.15.179:0/1913840232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b5406cd50 msgr2=0x7f7b5415da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1913840232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b5406cd50 0x7f7b5415da30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7b540ae5b0 tx=0x7f7b4c00b330 comp rx=0 tx=0).stop 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- 172.21.15.179:0/1913840232 shutdown_connections 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1913840232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b5406cd50 0x7f7b5415da30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1913840232 >> v2:172.21.15.195:3300/0 conn(0x7f7b54157be0 0x7f7b5406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- 172.21.15.179:0/1913840232 >> 172.21.15.179:0/1913840232 conn(0x7f7b540ce9a0 msgr2=0x7f7b540cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- 172.21.15.179:0/1913840232 shutdown_connections 2024-05-08T22:39:51.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- 172.21.15.179:0/1913840232 wait complete. 2024-05-08T22:39:51.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 Processor -- start 2024-05-08T22:39:51.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.858+0000 7f7b5a625700 1 -- start start 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 0x7f7b540c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b540abea0 0x7f7b540ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540aba10 con 0x7f7b540acb70 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c56d0 con 0x7f7b540abea0 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c3fe0 con 0x7f7b54157800 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52708/0 (socket says 172.21.15.179:52708) 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 -- 172.21.15.179:0/1891870349 learned_addr learned my addr 172.21.15.179:0/1891870349 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:51.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b59e24700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b540abea0 0x7f7b540ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b59623700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 0x7f7b540c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 -- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 msgr2=0x7f7b540c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 0x7f7b540c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 -- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b540abea0 msgr2=0x7f7b540ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b540abea0 0x7f7b540ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 -- 172.21.15.179:0/1891870349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c009a70 con 0x7f7b540acb70 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b59623700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 0x7f7b540c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b58e22700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e840 tx=0x7f7b5000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500095a0 con 0x7f7b540acb70 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50013680 con 0x7f7b540acb70 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540c42c0 con 0x7f7b540acb70 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500139f0 con 0x7f7b540acb70 2024-05-08T22:39:51.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540c4880 con 0x7f7b540acb70 2024-05-08T22:39:51.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b50013b50 con 0x7f7b540acb70 2024-05-08T22:39:51.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.862+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b540abbc0 con 0x7f7b540acb70 2024-05-08T22:39:51.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.866+0000 7f7b49ffb700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 0x7f7b38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:51.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.866+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f7b50084160 con 0x7f7b540acb70 2024-05-08T22:39:51.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.866+0000 7f7b59623700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 0x7f7b38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:51.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.866+0000 7f7b59623700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 0x7f7b38068bf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b44000f60 tx=0x7f7b44007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:51.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:51.870+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7b540abbc0 con 0x7f7b540acb70 2024-05-08T22:39:52.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.042+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b540cbd80 con 0x7f7b540acb70 2024-05-08T22:39:52.387 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:39:52.390 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:39:52.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.722+0000 7f7b49ffb700 1 -- 172.21.15.179:0/1891870349 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7b50051410 con 0x7f7b540acb70 2024-05-08T22:39:52.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 msgr2=0x7f7b38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 0x7f7b38068bf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b44000f60 tx=0x7f7b44007cc0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 msgr2=0x7f7b540afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e840 tx=0x7f7b5000eb50 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 shutdown_connections 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7b38066730 0x7f7b38068bf0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b540abea0 0x7f7b540ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b540acb70 0x7f7b540afb90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 --2- 172.21.15.179:0/1891870349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b54157800 0x7f7b540c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 >> 172.21.15.179:0/1891870349 conn(0x7f7b540ce9a0 msgr2=0x7f7b540ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 shutdown_connections 2024-05-08T22:39:52.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.726+0000 7f7b5a625700 1 -- 172.21.15.179:0/1891870349 wait complete. 2024-05-08T22:39:52.729 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:39:52.745 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:39:52.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 Processor -- start 2024-05-08T22:39:52.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 -- start start 2024-05-08T22:39:52.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5f641605e0 0x7f5f640d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f640b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f640d16c0 con 0x7f5f640cdb20 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f640d2ca0 con 0x7f5f641605e0 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f699e5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640d19a0 con 0x7f5f640d1190 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f691e4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5f640cdb20 0x7f5f640b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46598/0 (socket says 172.21.15.179:46598) 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f691e4700 1 -- 172.21.15.179:0/1313255032 learned_addr learned my addr 172.21.15.179:0/1313255032 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 --2- 172.21.15.179:0/1313255032 >> v2:172.21.15.195:3300/0 conn(0x7f5f641605e0 0x7f5f640d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580493140 0 0) 0x7f5f640d16c0 con 0x7f5f640cdb20 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f4c0035d0 con 0x7f5f640cdb20 2024-05-08T22:39:52.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f63fff700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f640b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1457315575 0 0) 0x7f5f4c0035d0 con 0x7f5f640cdb20 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 msgr2=0x7f5f640b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f640b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 >> v2:172.21.15.195:3300/0 conn(0x7f5f641605e0 msgr2=0x7f5f640d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 --2- 172.21.15.179:0/1313255032 >> v2:172.21.15.195:3300/0 conn(0x7f5f641605e0 0x7f5f640d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640d1c80 con 0x7f5f640cdb20 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5f5c0019f0 con 0x7f5f640cdb20 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 >> v1:172.21.15.179:6789/0 conn(0x7f5f640cdb20 legacy=0x7f5f640b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f4c003a20 0x7f5f4c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f4c004870 0x7f5f4c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f4c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640d2ca0 con 0x7f5f4c003a20 2024-05-08T22:39:52.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640d19a0 con 0x7f5f4c004870 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c00ba30 con 0x7f5f640d1190 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f4c003a20 0x7f5f4c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f63fff700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f4c004870 0x7f5f4c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f691e4700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f4c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 -- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 msgr2=0x7f5f4c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f640d1190 0x7f5f4c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 -- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f4c004870 msgr2=0x7f5f4c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f4c004870 0x7f5f4c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.870+0000 7f5f689e3700 1 -- 172.21.15.179:0/1313255032 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0035d0 con 0x7f5f4c003a20 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f689e3700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f4c003a20 0x7f5f4c003e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009b70 tx=0x7f5f4c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ae310fef76b0cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800e6c0 con 0x7f5f4c003a20 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800e820 con 0x7f5f4c003a20 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f637fe700 1 -- 172.21.15.179:0/1313255032 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800eb90 con 0x7f5f4c003a20 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 -- 172.21.15.179:0/1313255032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f4c003a20 msgr2=0x7f5f4c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 --2- 172.21.15.179:0/1313255032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f4c003a20 0x7f5f4c003e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009b70 tx=0x7f5f4c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f691e4700 1 -- 172.21.15.179:0/1313255032 reap_dead start 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 -- 172.21.15.179:0/1313255032 shutdown_connections 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 -- 172.21.15.179:0/1313255032 >> 172.21.15.179:0/1313255032 conn(0x7f5f640b4940 msgr2=0x7f5f640b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 -- 172.21.15.179:0/1313255032 shutdown_connections 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.874+0000 7f5f699e5700 1 -- 172.21.15.179:0/1313255032 wait complete. 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 Processor -- start 2024-05-08T22:39:52.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- start start 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f4c003a20 0x7f5f64155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f640c7550 0x7f5f640c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6415ba40 con 0x7f5f64159720 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6415a0e0 con 0x7f5f640c7550 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640c7f40 con 0x7f5f4c003a20 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52718/0 (socket says 172.21.15.179:52718) 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 -- 172.21.15.179:0/2672605338 learned_addr learned my addr 172.21.15.179:0/2672605338 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f689e3700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f4c003a20 0x7f5f64155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f691e4700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f640c7550 0x7f5f640c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 -- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f4c003a20 msgr2=0x7f5f64155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f4c003a20 0x7f5f64155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 -- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f640c7550 msgr2=0x7f5f640c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f640c7550 0x7f5f640c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 -- 172.21.15.179:0/2672605338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640c8160 con 0x7f5f64159720 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f63fff700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5f540119e0 tx=0x7f5f54011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400f940 con 0x7f5f64159720 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5400faa0 con 0x7f5f64159720 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f640c8440 con 0x7f5f64159720 2024-05-08T22:39:52.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400fe30 con 0x7f5f64159720 2024-05-08T22:39:52.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.878+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f640c8a00 con 0x7f5f64159720 2024-05-08T22:39:52.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f640d1000 con 0x7f5f64159720 2024-05-08T22:39:52.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f54014440 con 0x7f5f64159720 2024-05-08T22:39:52.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f617fa700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 0x7f5f50068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:52.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f5f54084710 con 0x7f5f64159720 2024-05-08T22:39:52.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f689e3700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 0x7f5f50068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:52.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.882+0000 7f5f689e3700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 0x7f5f50068c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5f580090b0 tx=0x7f5f58008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:52.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:52.886+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5f54051c90 con 0x7f5f64159720 2024-05-08T22:39:53.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.054+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5f6404e700 con 0x7f5f64159720 2024-05-08T22:39:53.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.058+0000 7f5f617fa700 1 -- 172.21.15.179:0/2672605338 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f540552b0 con 0x7f5f64159720 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 msgr2=0x7f5f50068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 0x7f5f50068c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5f580090b0 tx=0x7f5f58008030 comp rx=0 tx=0).stop 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 msgr2=0x7f5f64159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5f540119e0 tx=0x7f5f54011da0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 shutdown_connections 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5f50066760 0x7f5f50068c20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f640c7550 0x7f5f640c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f64159720 0x7f5f64159ba0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 --2- 172.21.15.179:0/2672605338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f4c003a20 0x7f5f64155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 >> 172.21.15.179:0/2672605338 conn(0x7f5f640b4940 msgr2=0x7f5f6406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:53.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 shutdown_connections 2024-05-08T22:39:53.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.062+0000 7f5f699e5700 1 -- 172.21.15.179:0/2672605338 wait complete. 2024-05-08T22:39:53.077 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-08T22:39:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.190+0000 7fb042199700 1 Processor -- start 2024-05-08T22:39:53.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.190+0000 7fb042199700 1 -- start start 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.190+0000 7fb042199700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 0x7fb03c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.190+0000 7fb042199700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0b2a50 con 0x7fb03c145ab0 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0b32f0 con 0x7fb03c0cd2a0 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0ca350 con 0x7fb03c0ceff0 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb041197700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb03c145ab0 0x7fb03c0ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46612/0 (socket says 172.21.15.179:46612) 2024-05-08T22:39:53.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb040996700 1 --2- 172.21.15.179:0/3897971485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 0x7fb03c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb041197700 1 -- 172.21.15.179:0/3897971485 learned_addr learned my addr 172.21.15.179:0/3897971485 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835294717 0 0) 0x7fb03c0b2a50 con 0x7fb03c145ab0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0280035d0 con 0x7fb03c145ab0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753136172 0 0) 0x7fb03c0b32f0 con 0x7fb03c0cd2a0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb040996700 1 -- 172.21.15.179:0/3897971485 >> v1:172.21.15.195:6789/0 conn(0x7fb03c0cd2a0 legacy=0x7fb03c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb040996700 1 -- 172.21.15.179:0/3897971485 >> v1:172.21.15.179:6789/0 conn(0x7fb03c145ab0 legacy=0x7fb03c0ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb040996700 1 -- 172.21.15.179:0/3897971485 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0cdfb0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb040996700 1 --2- 172.21.15.179:0/3897971485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 0x7fb03c0c9e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb038009880 tx=0x7fb038009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec068d6d14476cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038011830 con 0x7fb03c0ceff0 2024-05-08T22:39:53.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038011990 con 0x7fb03c0ceff0 2024-05-08T22:39:53.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb033fff700 1 -- 172.21.15.179:0/3897971485 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038011d20 con 0x7fb03c0ceff0 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- 172.21.15.179:0/3897971485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 msgr2=0x7fb03c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 --2- 172.21.15.179:0/3897971485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 0x7fb03c0c9e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb038009880 tx=0x7fb038009b90 comp rx=0 tx=0).stop 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- 172.21.15.179:0/3897971485 shutdown_connections 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 --2- 172.21.15.179:0/3897971485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0ceff0 0x7fb03c0c9e10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- 172.21.15.179:0/3897971485 >> 172.21.15.179:0/3897971485 conn(0x7fb03c0cfae0 msgr2=0x7fb03c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- 172.21.15.179:0/3897971485 shutdown_connections 2024-05-08T22:39:53.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.194+0000 7fb042199700 1 -- 172.21.15.179:0/3897971485 wait complete. 2024-05-08T22:39:53.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.198+0000 7fb042199700 1 Processor -- start 2024-05-08T22:39:53.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.198+0000 7fb042199700 1 -- start start 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cd2a0 0x7fb03c143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c145ab0 0x7fb03c13ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c06c5c0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c06c8a0 con 0x7fb03c145ab0 2024-05-08T22:39:53.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c141d60 con 0x7fb03c0cd2a0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52724/0 (socket says 172.21.15.179:52724) 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 -- 172.21.15.179:0/1687137360 learned_addr learned my addr 172.21.15.179:0/1687137360 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb041197700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cd2a0 0x7fb03c143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb041998700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c145ab0 0x7fb03c13ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 -- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cd2a0 msgr2=0x7fb03c143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cd2a0 0x7fb03c143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 -- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c145ab0 msgr2=0x7fb03c13ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c145ab0 0x7fb03c13ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 -- 172.21.15.179:0/1687137360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380094b0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb040996700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb0380090b0 tx=0x7fb038004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03801a8c0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03801e070 con 0x7fb03c0ceff0 2024-05-08T22:39:53.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c142040 con 0x7fb03c0ceff0 2024-05-08T22:39:53.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03801ac40 con 0x7fb03c0ceff0 2024-05-08T22:39:53.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c1419e0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.202+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c04e700 con 0x7fb03c0ceff0 2024-05-08T22:39:53.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.206+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb03802d450 con 0x7fb03c0ceff0 2024-05-08T22:39:53.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.206+0000 7fb031ffb700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 0x7fb020068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.206+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fb038084e20 con 0x7fb03c0ceff0 2024-05-08T22:39:53.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.206+0000 7fb041197700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 0x7fb020068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.206+0000 7fb041197700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 0x7fb020068bf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb02c007fd0 tx=0x7fb02c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.210+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb038052380 con 0x7fb03c0ceff0 2024-05-08T22:39:53.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.442+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fb03c06c9e0 con 0x7fb03c0ceff0 2024-05-08T22:39:53.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.726+0000 7fb031ffb700 1 -- 172.21.15.179:0/1687137360 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v315) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb038020020 con 0x7fb03c0ceff0 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 msgr2=0x7fb020068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 0x7fb020068bf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb02c007fd0 tx=0x7fb02c009040 comp rx=0 tx=0).stop 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 msgr2=0x7fb03c1414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb0380090b0 tx=0x7fb038004770 comp rx=0 tx=0).stop 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 shutdown_connections 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb020066730 0x7fb020068bf0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb03c145ab0 0x7fb03c13ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03c0ceff0 0x7fb03c1414a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 --2- 172.21.15.179:0/1687137360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb03c0cd2a0 0x7fb03c143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 >> 172.21.15.179:0/1687137360 conn(0x7fb03c0cfae0 msgr2=0x7fb03c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:53.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 shutdown_connections 2024-05-08T22:39:53.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.730+0000 7fb042199700 1 -- 172.21.15.179:0/1687137360 wait complete. 2024-05-08T22:39:53.746 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T22:39:53.747 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T22:39:53.747 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:39:53.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:53.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 Processor -- start 2024-05-08T22:39:53.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- start start 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0cc310 0x7f38fc06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38fc0add40 0x7f38fc06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc161ac0 con 0x7f38fc0ce060 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0ae550 con 0x7f38fc0add40 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc06abb0 con 0x7f38fc0cc310 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38fc0add40 0x7f38fc06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38fc0add40 0x7f38fc06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52734/0 (socket says 172.21.15.179:52734) 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 -- 172.21.15.179:0/1629663376 learned_addr learned my addr 172.21.15.179:0/1629663376 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38faffd700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0cc310 0x7f38fc06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 -- 172.21.15.179:0/1629663376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0cc310 msgr2=0x7f38fc06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0cc310 0x7f38fc06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 -- 172.21.15.179:0/1629663376 >> v1:172.21.15.195:6789/0 conn(0x7f38fc0ce060 legacy=0x7f38fc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 -- 172.21.15.179:0/1629663376 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0cdaa0 con 0x7f38fc0add40 2024-05-08T22:39:53.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fa7fc700 1 -- 172.21.15.179:0/1629663376 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692306800 0 0) 0x7f38fc161ac0 con 0x7f38fc0ce060 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fbfff700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc06b8a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009910 tx=0x7f38ec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf86c0f471b44989 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fa7fc700 1 -- 172.21.15.179:0/1629663376 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec011830 con 0x7f38fc0add40 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fa7fc700 1 -- 172.21.15.179:0/1629663376 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec011990 con 0x7f38fc0add40 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f38fa7fc700 1 -- 172.21.15.179:0/1629663376 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec011d40 con 0x7f38fc0add40 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- 172.21.15.179:0/1629663376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 msgr2=0x7f38fc06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc06b8a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009910 tx=0x7f38ec009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- 172.21.15.179:0/1629663376 shutdown_connections 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0cc310 0x7f38fc06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 --2- 172.21.15.179:0/1629663376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc06b8a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- 172.21.15.179:0/1629663376 >> 172.21.15.179:0/1629663376 conn(0x7f38fc0cb660 msgr2=0x7f38fc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:53.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- 172.21.15.179:0/1629663376 shutdown_connections 2024-05-08T22:39:53.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.866+0000 7f3901e61700 1 -- 172.21.15.179:0/1629663376 wait complete. 2024-05-08T22:39:53.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 Processor -- start 2024-05-08T22:39:53.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 -- start start 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 0x7f38fc0c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0ce060 0x7f38fc155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b2810 con 0x7f38fc0add40 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b2af0 con 0x7f38fc0cc310 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f3901e61700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b2dd0 con 0x7f38fc0ce060 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52746/0 (socket says 172.21.15.179:52746) 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 0x7f38fc0c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38fb7fe700 1 -- 172.21.15.179:0/2056958091 learned_addr learned my addr 172.21.15.179:0/2056958091 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:53.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38fbfff700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0ce060 0x7f38fc155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 -- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0ce060 msgr2=0x7f38fc155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0ce060 0x7f38fc155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 -- 172.21.15.179:0/2056958091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 msgr2=0x7f38fc0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 -- 172.21.15.179:0/2056958091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec009570 con 0x7f38fc0cc310 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38fb7fe700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.870+0000 7f38faffd700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 0x7f38fc0c4b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f38f000e870 tx=0x7f38f000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0013680 con 0x7f38fc0cc310 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f00137e0 con 0x7f38fc0cc310 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0b30b0 con 0x7f38fc0cc310 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0013b70 con 0x7f38fc0cc310 2024-05-08T22:39:53.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0c50b0 con 0x7f38fc0cc310 2024-05-08T22:39:53.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc067770 con 0x7f38fc0cc310 2024-05-08T22:39:53.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.874+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38f0013cd0 con 0x7f38fc0cc310 2024-05-08T22:39:53.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.878+0000 7f38e3fff700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 0x7f38e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:53.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.878+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f38f00840e0 con 0x7f38fc0cc310 2024-05-08T22:39:53.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.878+0000 7f38fb7fe700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 0x7f38e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:53.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.878+0000 7f38fb7fe700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 0x7f38e8068bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f38f4002230 tx=0x7f38f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:53.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:53.886+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f38f0051720 con 0x7f38fc0cc310 2024-05-08T22:39:54.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.054+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38fc0b35d0 con 0x7f38fc0cc310 2024-05-08T22:39:54.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.058+0000 7f38e3fff700 1 -- 172.21.15.179:0/2056958091 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+12904 (secure 0 0 0) 0x7f38fc0b35d0 con 0x7f38fc0cc310 2024-05-08T22:39:54.060 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:54.060 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":87,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:39:52.711828+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/1691042534":"2024-05-09T22:39:49.694983+0000","172.21.15.195:6835/1691042534":"2024-05-09T22:39:49.694983+0000","172.21.15.195:6837/123759130":"2024-05-09T22:38:20.877311+0000","172.21.15.195:6835/123759130":"2024-05-09T22:38:20.877311+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:54.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 msgr2=0x7f38e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 0x7f38e8068bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f38f4002230 tx=0x7f38f4009040 comp rx=0 tx=0).stop 2024-05-08T22:39:54.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 msgr2=0x7f38fc0c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 0x7f38fc0c4b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f38f000e870 tx=0x7f38f000eb80 comp rx=0 tx=0).stop 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 shutdown_connections 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f38e8066730 0x7f38e8068bf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38fc0cc310 0x7f38fc0c4b70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38fc0ce060 0x7f38fc155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 --2- 172.21.15.179:0/2056958091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38fc0add40 0x7f38fc0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 >> 172.21.15.179:0/2056958091 conn(0x7f38fc0cb660 msgr2=0x7f38fc0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 shutdown_connections 2024-05-08T22:39:54.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.062+0000 7f3901e61700 1 -- 172.21.15.179:0/2056958091 wait complete. 2024-05-08T22:39:54.079 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:39:54.079 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:39:54.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 Processor -- start 2024-05-08T22:39:54.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 -- start start 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b3406e090 0x7f8b3415e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 0x7f8b340cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340cb370 con 0x7f8b34158850 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3415f370 con 0x7f8b3406e090 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.194+0000 7f8b3a750700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406e5c0 con 0x7f8b3415eec0 2024-05-08T22:39:54.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3974e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8b34158850 0x7f8b340ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46626/0 (socket says 172.21.15.179:46626) 2024-05-08T22:39:54.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3974e700 1 -- 172.21.15.179:0/1858881840 learned_addr learned my addr 172.21.15.179:0/1858881840 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:54.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 --2- 172.21.15.179:0/1858881840 >> v2:172.21.15.195:3300/0 conn(0x7f8b3406e090 0x7f8b3415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b39f4f700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 0x7f8b340cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b2bfff700 1 -- 172.21.15.179:0/1858881840 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754240297 0 0) 0x7f8b340cb370 con 0x7f8b34158850 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b2bfff700 1 -- 172.21.15.179:0/1858881840 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c0035d0 con 0x7f8b34158850 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 -- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 msgr2=0x7f8b340cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 0x7f8b340cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 -- 172.21.15.179:0/1858881840 >> v1:172.21.15.179:6789/0 conn(0x7f8b34158850 legacy=0x7f8b340ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 -- 172.21.15.179:0/1858881840 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3406e8a0 con 0x7f8b3406e090 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b39f4f700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 0x7f8b340cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b38f4d700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b3415e980 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009dc0 tx=0x7f8b3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fdb672f7b3a8b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b2bfff700 1 -- 172.21.15.179:0/1858881840 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000e070 con 0x7f8b3406e090 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b2bfff700 1 -- 172.21.15.179:0/1858881840 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3000bae0 con 0x7f8b3406e090 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b2bfff700 1 -- 172.21.15.179:0/1858881840 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000be50 con 0x7f8b3406e090 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 -- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 msgr2=0x7f8b3415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b3415e980 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009dc0 tx=0x7f8b3000b330 comp rx=0 tx=0).stop 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 -- 172.21.15.179:0/1858881840 shutdown_connections 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b3415eec0 0x7f8b340cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 --2- 172.21.15.179:0/1858881840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b3415e980 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 -- 172.21.15.179:0/1858881840 >> 172.21.15.179:0/1858881840 conn(0x7f8b340cea30 msgr2=0x7f8b340cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 -- 172.21.15.179:0/1858881840 shutdown_connections 2024-05-08T22:39:54.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.198+0000 7f8b3a750700 1 -- 172.21.15.179:0/1858881840 wait complete. 2024-05-08T22:39:54.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 Processor -- start 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- start start 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b34156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b34158850 0x7f8b341f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341571d0 con 0x7f8b3415eec0 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3413a780 con 0x7f8b3406e090 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3413aa60 con 0x7f8b34158850 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52754/0 (socket says 172.21.15.179:52754) 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 -- 172.21.15.179:0/122882664 learned_addr learned my addr 172.21.15.179:0/122882664 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3974e700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b34156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b38f4d700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b34158850 0x7f8b341f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 -- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b34158850 msgr2=0x7f8b341f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b34158850 0x7f8b341f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 -- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 msgr2=0x7f8b34156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b34156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 -- 172.21.15.179:0/122882664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240094b0 con 0x7f8b3415eec0 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b39f4f700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400eb60 tx=0x7f8b2400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240097b0 con 0x7f8b3415eec0 2024-05-08T22:39:54.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24009e70 con 0x7f8b3415eec0 2024-05-08T22:39:54.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b30009a70 con 0x7f8b3415eec0 2024-05-08T22:39:54.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24004220 con 0x7f8b3415eec0 2024-05-08T22:39:54.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.202+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b341f07f0 con 0x7f8b3415eec0 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b240043c0 con 0x7f8b3415eec0 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3404e700 con 0x7f8b3415eec0 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b29ffb700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 0x7f8b20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8b24083ce0 con 0x7f8b3415eec0 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b3974e700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 0x7f8b20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.206+0000 7f8b3974e700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 0x7f8b20068bf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007fd0 tx=0x7f8b2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.214+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b240511b0 con 0x7f8b3415eec0 2024-05-08T22:39:54.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.382+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b34143870 con 0x7f8b3415eec0 2024-05-08T22:39:54.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.382+0000 7f8b29ffb700 1 -- 172.21.15.179:0/122882664 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b240547d0 con 0x7f8b3415eec0 2024-05-08T22:39:54.385 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:39:54.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 msgr2=0x7f8b20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 0x7f8b20068bf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007fd0 tx=0x7f8b2c009040 comp rx=0 tx=0).stop 2024-05-08T22:39:54.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 msgr2=0x7f8b34139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400eb60 tx=0x7f8b2400ee70 comp rx=0 tx=0).stop 2024-05-08T22:39:54.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 shutdown_connections 2024-05-08T22:39:54.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b20066730 0x7f8b20068bf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b3415eec0 0x7f8b34139c40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b34158850 0x7f8b341f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 --2- 172.21.15.179:0/122882664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b3406e090 0x7f8b34156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 >> 172.21.15.179:0/122882664 conn(0x7f8b340cea30 msgr2=0x7f8b3406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:54.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 shutdown_connections 2024-05-08T22:39:54.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.386+0000 7f8b3a750700 1 -- 172.21.15.179:0/122882664 wait complete. 2024-05-08T22:39:54.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-08T22:39:54.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.510+0000 7f02abfff700 1 Processor -- start 2024-05-08T22:39:54.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.510+0000 7f02abfff700 1 -- start start 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f02ac15b150 0x7f02ac0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0af490 0x7f02ac0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0acbc0 con 0x7f02ac0ae740 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac0ac290 con 0x7f02ac15b150 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0ac570 con 0x7f02ac0af490 2024-05-08T22:39:54.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f02ac15b150 0x7f02ac0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f02ac15b150 0x7f02ac0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52756/0 (socket says 172.21.15.179:52756) 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 -- 172.21.15.179:0/1633574493 learned_addr learned my addr 172.21.15.179:0/1633574493 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aa7fc700 1 --1- 172.21.15.179:0/1633574493 >> v1:172.21.15.179:6789/0 conn(0x7f02ac0ae740 0x7f02ac0aed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:46634/0 (socket says 172.21.15.179:46634) 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02ab7fe700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0af490 0x7f02ac0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 -- 172.21.15.179:0/1633574493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0af490 msgr2=0x7f02ac0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0af490 0x7f02ac0ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 -- 172.21.15.179:0/1633574493 >> v1:172.21.15.179:6789/0 conn(0x7f02ac0ae740 legacy=0x7f02ac0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 -- 172.21.15.179:0/1633574493 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0ac850 con 0x7f02ac15b150 2024-05-08T22:39:54.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02aaffd700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac0b4a30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f029c009910 tx=0x7f029c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94676302cd8f8ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02a9ffb700 1 -- 172.21.15.179:0/1633574493 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c011830 con 0x7f02ac15b150 2024-05-08T22:39:54.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02a9ffb700 1 -- 172.21.15.179:0/1633574493 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c011990 con 0x7f02ac15b150 2024-05-08T22:39:54.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02a9ffb700 1 -- 172.21.15.179:0/1633574493 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c011d40 con 0x7f02ac15b150 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- 172.21.15.179:0/1633574493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 msgr2=0x7f02ac0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac0b4a30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f029c009910 tx=0x7f029c009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- 172.21.15.179:0/1633574493 shutdown_connections 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0af490 0x7f02ac0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 --2- 172.21.15.179:0/1633574493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac0b4a30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- 172.21.15.179:0/1633574493 >> 172.21.15.179:0/1633574493 conn(0x7f02ac098a70 msgr2=0x7f02ac0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- 172.21.15.179:0/1633574493 shutdown_connections 2024-05-08T22:39:54.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.514+0000 7f02abfff700 1 -- 172.21.15.179:0/1633574493 wait complete. 2024-05-08T22:39:54.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 Processor -- start 2024-05-08T22:39:54.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 -- start start 2024-05-08T22:39:54.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0ae740 0x7f02ac09cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02ac0af490 0x7f02ac147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac159980 con 0x7f02ac15b150 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac145580 con 0x7f02ac0af490 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02abfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac158e00 con 0x7f02ac0ae740 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52770/0 (socket says 172.21.15.179:52770) 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02ab7fe700 1 -- 172.21.15.179:0/3868035760 learned_addr learned my addr 172.21.15.179:0/3868035760 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02aa7fc700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02ac0af490 0x7f02ac147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.518+0000 7f02aaffd700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0ae740 0x7f02ac09cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 -- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0ae740 msgr2=0x7f02ac09cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0ae740 0x7f02ac09cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 -- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02ac0af490 msgr2=0x7f02ac147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02ac0af490 0x7f02ac147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 -- 172.21.15.179:0/3868035760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009570 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02ab7fe700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f02a400efc0 tx=0x7f02a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a400cc90 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a400f040 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40146c0 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac06ad30 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1593c0 con 0x7f02ac15b150 2024-05-08T22:39:54.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac1458d0 con 0x7f02ac15b150 2024-05-08T22:39:54.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02a4014820 con 0x7f02ac15b150 2024-05-08T22:39:54.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.522+0000 7f02937fe700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 0x7f0298068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:54.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.526+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f02a4084b90 con 0x7f02ac15b150 2024-05-08T22:39:54.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.526+0000 7f02aaffd700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 0x7f0298068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:54.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.526+0000 7f02aaffd700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 0x7f0298068bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f029c000c00 tx=0x7f029c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:54.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.530+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f02a4052050 con 0x7f02ac15b150 2024-05-08T22:39:54.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:54.758+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f02ac04e700 con 0x7f02ac15b150 2024-05-08T22:39:55.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.734+0000 7f02937fe700 1 -- 172.21.15.179:0/3868035760 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v317) v1 ==== 170+0+0 (secure 0 0 0) 0x7f02a4018020 con 0x7f02ac15b150 2024-05-08T22:39:55.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 msgr2=0x7f0298068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 0x7f0298068bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f029c000c00 tx=0x7f029c008030 comp rx=0 tx=0).stop 2024-05-08T22:39:55.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 msgr2=0x7f02ac147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f02a400efc0 tx=0x7f02a400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 shutdown_connections 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0298066730 0x7f0298068bf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02ac0af490 0x7f02ac147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02ac15b150 0x7f02ac147db0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 --2- 172.21.15.179:0/3868035760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02ac0ae740 0x7f02ac09cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 >> 172.21.15.179:0/3868035760 conn(0x7f02ac098a70 msgr2=0x7f02ac0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 shutdown_connections 2024-05-08T22:39:55.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.738+0000 7f02abfff700 1 -- 172.21.15.179:0/3868035760 wait complete. 2024-05-08T22:39:55.759 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:39:55.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.882+0000 7f892dbee700 1 Processor -- start 2024-05-08T22:39:55.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.882+0000 7f892dbee700 1 -- start start 2024-05-08T22:39:55.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f892815d5d0 0x7f89280ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89280ac6c0 0x7f89280b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89280ad590 con 0x7f892815d1f0 2024-05-08T22:39:55.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892815da60 con 0x7f892815d5d0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892815dcf0 con 0x7f89280ac6c0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f892815d5d0 0x7f89280ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892cbec700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f892815d1f0 0x7f89280b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47852/0 (socket says 172.21.15.179:47852) 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892cbec700 1 -- 172.21.15.179:0/1584939298 learned_addr learned my addr 172.21.15.179:0/1584939298 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892d3ed700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89280ac6c0 0x7f89280b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 -- 172.21.15.179:0/1584939298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89280ac6c0 msgr2=0x7f89280b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89280ac6c0 0x7f89280b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 -- 172.21.15.179:0/1584939298 >> v1:172.21.15.179:6789/0 conn(0x7f892815d1f0 legacy=0x7f89280b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 -- 172.21.15.179:0/1584939298 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892815dfd0 con 0x7f892815d5d0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f8927fff700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89280ac180 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8918009d90 tx=0x7f891800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d21a00b436155 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f89277fe700 1 -- 172.21.15.179:0/1584939298 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800e070 con 0x7f892815d5d0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f89277fe700 1 -- 172.21.15.179:0/1584939298 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891800bbd0 con 0x7f892815d5d0 2024-05-08T22:39:55.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f89277fe700 1 -- 172.21.15.179:0/1584939298 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918012430 con 0x7f892815d5d0 2024-05-08T22:39:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- 172.21.15.179:0/1584939298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 msgr2=0x7f89280ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89280ac180 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8918009d90 tx=0x7f891800b330 comp rx=0 tx=0).stop 2024-05-08T22:39:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- 172.21.15.179:0/1584939298 shutdown_connections 2024-05-08T22:39:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89280ac6c0 0x7f89280b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 --2- 172.21.15.179:0/1584939298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89280ac180 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- 172.21.15.179:0/1584939298 >> 172.21.15.179:0/1584939298 conn(0x7f89280cea30 msgr2=0x7f89280cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:55.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- 172.21.15.179:0/1584939298 shutdown_connections 2024-05-08T22:39:55.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.886+0000 7f892dbee700 1 -- 172.21.15.179:0/1584939298 wait complete. 2024-05-08T22:39:55.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 Processor -- start 2024-05-08T22:39:55.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 -- start start 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89280ac6c0 0x7f892814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 0x7f8928155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928155090 con 0x7f892815d5d0 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928155650 con 0x7f89280ac6c0 2024-05-08T22:39:55.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892814ba40 con 0x7f892815d1f0 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892cbec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89280ac6c0 0x7f892814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58854/0 (socket says 172.21.15.179:58854) 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 -- 172.21.15.179:0/743856712 learned_addr learned my addr 172.21.15.179:0/743856712 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f8927fff700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 0x7f8928155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 -- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 msgr2=0x7f8928155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 0x7f8928155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 -- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89280ac6c0 msgr2=0x7f892814b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89280ac6c0 0x7f892814b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 -- 172.21.15.179:0/743856712 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009a70 con 0x7f892815d5d0 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f8927fff700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 0x7f8928155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:39:55.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892d3ed700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f892000e840 tx=0x7f892000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:55.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892000c070 con 0x7f892815d5d0 2024-05-08T22:39:55.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920013630 con 0x7f892815d5d0 2024-05-08T22:39:55.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89281f34a0 con 0x7f892815d5d0 2024-05-08T22:39:55.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.890+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89200139e0 con 0x7f892815d5d0 2024-05-08T22:39:55.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89281552e0 con 0x7f892815d5d0 2024-05-08T22:39:55.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89281f39c0 con 0x7f892815d5d0 2024-05-08T22:39:55.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8920013b40 con 0x7f892815d5d0 2024-05-08T22:39:55.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f89257fa700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 0x7f8914068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:55.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8920083d50 con 0x7f892815d5d0 2024-05-08T22:39:55.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.894+0000 7f892cbec700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 0x7f8914068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:55.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.898+0000 7f892cbec700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 0x7f8914068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f891c005930 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:55.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:55.902+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8920087030 con 0x7f892815d5d0 2024-05-08T22:39:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.074+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f892804e700 con 0x7f892815d5d0 2024-05-08T22:39:56.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.078+0000 7f89257fa700 1 -- 172.21.15.179:0/743856712 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f89200512b0 con 0x7f892815d5d0 2024-05-08T22:39:56.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 msgr2=0x7f8914068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 0x7f8914068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f891c005930 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-05-08T22:39:56.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 msgr2=0x7f89281f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f892000e840 tx=0x7f892000eb50 comp rx=0 tx=0).stop 2024-05-08T22:39:56.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 shutdown_connections 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8914066730 0x7f8914068bf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f892815d5d0 0x7f89281f29b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f892815d1f0 0x7f8928155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 --2- 172.21.15.179:0/743856712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89280ac6c0 0x7f892814b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 >> 172.21.15.179:0/743856712 conn(0x7f89280cea30 msgr2=0x7f89280b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 shutdown_connections 2024-05-08T22:39:56.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.082+0000 7f892dbee700 1 -- 172.21.15.179:0/743856712 wait complete. 2024-05-08T22:39:56.095 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:39:56.095 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:39:56.113 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:39:56.133 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:39:56.133 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:39:56.133 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:56.140 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:39:56.140 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:39:56.140 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:56.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:39:56.188 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:39:56.188 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:56.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:39:56.195 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:39:56.195 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:39:56.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:39:56.203 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:39:56.203 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:39:56.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.318+0000 7ff72dc87700 1 Processor -- start 2024-05-08T22:39:56.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.318+0000 7ff72dc87700 1 -- start start 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff72815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff7280c9230 0x7ff7280c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff728158f60 con 0x7ff7280c8d70 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72806b860 con 0x7ff7280c9230 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72806ac50 con 0x7ff728158b80 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff7280c9230 0x7ff7280c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7277fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff72815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff726ffd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ff7280c8d70 0x7ff72806bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46752/0 (socket says 172.21.15.179:46752) 2024-05-08T22:39:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff726ffd700 1 -- 172.21.15.179:0/1167868828 learned_addr learned my addr 172.21.15.179:0/1167868828 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 -- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 msgr2=0x7ff72815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff72815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 -- 172.21.15.179:0/1167868828 >> v1:172.21.15.195:6789/0 conn(0x7ff7280c8d70 legacy=0x7ff72806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 -- 172.21.15.179:0/1167868828 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72806af30 con 0x7ff7280c9230 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7267fc700 1 -- 172.21.15.179:0/1167868828 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459061985 0 0) 0x7ff728158f60 con 0x7ff7280c8d70 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7277fe700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff72815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff727fff700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c9230 0x7ff7280c96c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff720009850 tx=0x7ff720009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7df21de35016d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7267fc700 1 -- 172.21.15.179:0/1167868828 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720011830 con 0x7ff7280c9230 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7267fc700 1 -- 172.21.15.179:0/1167868828 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720011990 con 0x7ff7280c9230 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff7267fc700 1 -- 172.21.15.179:0/1167868828 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720011d40 con 0x7ff7280c9230 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c9230 msgr2=0x7ff7280c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c9230 0x7ff7280c96c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff720009850 tx=0x7ff720009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- 172.21.15.179:0/1167868828 shutdown_connections 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c9230 0x7ff7280c96c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 --2- 172.21.15.179:0/1167868828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff72815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- 172.21.15.179:0/1167868828 >> 172.21.15.179:0/1167868828 conn(0x7ff7280cb660 msgr2=0x7ff7280cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- 172.21.15.179:0/1167868828 shutdown_connections 2024-05-08T22:39:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.322+0000 7ff72dc87700 1 -- 172.21.15.179:0/1167868828 wait complete. 2024-05-08T22:39:56.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 Processor -- start 2024-05-08T22:39:56.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 -- start start 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c8d70 0x7ff7280af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff728151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280af710 con 0x7ff7280c9230 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72814fad0 con 0x7ff7280c8d70 2024-05-08T22:39:56.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff72dc87700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72814f1e0 con 0x7ff728158b80 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff726ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff726ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58860/0 (socket says 172.21.15.179:58860) 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff726ffd700 1 -- 172.21.15.179:0/3283167296 learned_addr learned my addr 172.21.15.179:0/3283167296 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff728151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff7277fe700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c8d70 0x7ff7280af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 -- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c8d70 msgr2=0x7ff7280af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c8d70 0x7ff7280af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 -- 172.21.15.179:0/3283167296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 msgr2=0x7ff7280c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 -- 172.21.15.179:0/3283167296 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0094b0 con 0x7ff728158b80 2024-05-08T22:39:56.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff726ffd700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:56.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.326+0000 7ff727fff700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff728151fe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff720016000 tx=0x7ff720004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72001a990 con 0x7ff728158b80 2024-05-08T22:39:56.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72001f070 con 0x7ff728158b80 2024-05-08T22:39:56.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7200094b0 con 0x7ff728158b80 2024-05-08T22:39:56.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72001ad50 con 0x7ff728158b80 2024-05-08T22:39:56.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72814f480 con 0x7ff728158b80 2024-05-08T22:39:56.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff728067770 con 0x7ff728158b80 2024-05-08T22:39:56.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.330+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff72002e450 con 0x7ff728158b80 2024-05-08T22:39:56.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.334+0000 7ff70ffff700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 0x7ff714068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.334+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff720085340 con 0x7ff728158b80 2024-05-08T22:39:56.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.334+0000 7ff7277fe700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 0x7ff714068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.334+0000 7ff7277fe700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 0x7ff714068bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009ce0 tx=0x7ff71c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.342+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff720052880 con 0x7ff728158b80 2024-05-08T22:39:56.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.554+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff72806f010 con 0x7ff728158b80 2024-05-08T22:39:56.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff70ffff700 1 -- 172.21.15.179:0/3283167296 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3542 (secure 0 0 0) 0x7ff720015df0 con 0x7ff728158b80 2024-05-08T22:39:56.558 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:56.558 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6982,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3798867821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3798867821},{"type":"v1","addr":"172.21.15.179:6835","nonce":3798867821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/2254587083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2254587083},{"type":"v1","addr":"172.21.15.195:6836","nonce":2254587083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":7627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4279567157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4279567157},{"type":"v1","addr":"172.21.15.195:6837","nonce":4279567157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":56147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1449078518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1449078518},{"type":"v1","addr":"172.21.15.179:6834","nonce":1449078518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 msgr2=0x7ff714068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 0x7ff714068bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009ce0 tx=0x7ff71c009240 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 msgr2=0x7ff728151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff728151fe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff720016000 tx=0x7ff720004770 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 shutdown_connections 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff714066730 0x7ff714068bf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff728158b80 0x7ff728151fe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7280c9230 0x7ff7280c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 --2- 172.21.15.179:0/3283167296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7280c8d70 0x7ff7280af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 >> 172.21.15.179:0/3283167296 conn(0x7ff7280cb660 msgr2=0x7ff7280c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 shutdown_connections 2024-05-08T22:39:56.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.558+0000 7ff72dc87700 1 -- 172.21.15.179:0/3283167296 wait complete. 2024-05-08T22:39:56.562 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 49 2024-05-08T22:39:56.573 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:39:56.573 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:39:56.573 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:39:56.573 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:39:56.573 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:39:56.573 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:39:56.573 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:39:56.574 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:39:56.574 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:39:56.574 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:39:56.574 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:39:56.574 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:39:56.574 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:39:56.574 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:39:56.575 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:39:56.575 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:39:56.584 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:39:56.584 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:39:56.588 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:39:56.588 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:39:56.590 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:39:56.590 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:39:56.593 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:39:56.594 DEBUG:teuthology.parallel:result is None 2024-05-08T22:39:56.594 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:39:56.594 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:39:56.596 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:39:56.596 DEBUG:teuthology.parallel:result is None 2024-05-08T22:39:56.635 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:39:56.635 DEBUG:teuthology.parallel:result is None 2024-05-08T22:39:56.636 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:39:56.637 DEBUG:teuthology.parallel:result is None 2024-05-08T22:39:56.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:39:56.681 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:39:56.684 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:39:56.708 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:39:56.714 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:39:56.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.742+0000 7f8d26cd9700 1 Processor -- start 2024-05-08T22:39:56.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.742+0000 7f8d26cd9700 1 -- start start 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 0x7f8d2006adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 0x7f8d200cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d200ce2f0 con 0x7f8d200ccd90 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2006e8d0 con 0x7f8d2015d400 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200ceb00 con 0x7f8d200c92d0 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d25276700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8d200ccd90 0x7f8d2015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47878/0 (socket says 172.21.15.179:47878) 2024-05-08T22:39:56.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d25276700 1 -- 172.21.15.179:0/21096235 learned_addr learned my addr 172.21.15.179:0/21096235 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 --2- 172.21.15.179:0/21096235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 0x7f8d200cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d24a75700 1 --2- 172.21.15.179:0/21096235 >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 0x7f8d2006adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d177fe700 1 -- 172.21.15.179:0/21096235 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257828547 0 0) 0x7f8d200ce2f0 con 0x7f8d200ccd90 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d177fe700 1 -- 172.21.15.179:0/21096235 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d080035d0 con 0x7f8d200ccd90 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 -- 172.21.15.179:0/21096235 >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 msgr2=0x7f8d2006adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 --2- 172.21.15.179:0/21096235 >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 0x7f8d2006adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 -- 172.21.15.179:0/21096235 >> v1:172.21.15.179:6789/0 conn(0x7f8d200ccd90 legacy=0x7f8d2015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 -- 172.21.15.179:0/21096235 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200cac10 con 0x7f8d200c92d0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d24a75700 1 --2- 172.21.15.179:0/21096235 >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 0x7f8d2006adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d17fff700 1 --2- 172.21.15.179:0/21096235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 0x7f8d200cc850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009850 tx=0x7f8d10009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29c70caeef3e434 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d177fe700 1 -- 172.21.15.179:0/21096235 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000f8a0 con 0x7f8d200c92d0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d177fe700 1 -- 172.21.15.179:0/21096235 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000fa00 con 0x7f8d200c92d0 2024-05-08T22:39:56.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d177fe700 1 -- 172.21.15.179:0/21096235 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000fdb0 con 0x7f8d200c92d0 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- 172.21.15.179:0/21096235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 msgr2=0x7f8d200cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/21096235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 0x7f8d200cc850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009850 tx=0x7f8d10009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- 172.21.15.179:0/21096235 shutdown_connections 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/21096235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200c92d0 0x7f8d200cc850 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/21096235 >> v2:172.21.15.195:3300/0 conn(0x7f8d2015d400 0x7f8d2006adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- 172.21.15.179:0/21096235 >> 172.21.15.179:0/21096235 conn(0x7f8d200cb700 msgr2=0x7f8d200cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- 172.21.15.179:0/21096235 shutdown_connections 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.746+0000 7f8d26cd9700 1 -- 172.21.15.179:0/21096235 wait complete. 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 Processor -- start 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- start start 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 0x7f8d200b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 0x7f8d200b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200b2300 con 0x7f8d2015d400 2024-05-08T22:39:56.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201539d0 con 0x7f8d200c92d0 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200b2820 con 0x7f8d200ccd90 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d17fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 0x7f8d200b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d17fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 0x7f8d200b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45608/0 (socket says 172.21.15.179:45608) 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d17fff700 1 -- 172.21.15.179:0/1907446335 learned_addr learned my addr 172.21.15.179:0/1907446335 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58902/0 (socket says 172.21.15.179:58902) 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d24a75700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 0x7f8d200b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 -- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 msgr2=0x7f8d200b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 0x7f8d200b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 -- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 msgr2=0x7f8d200b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 0x7f8d200b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 -- 172.21.15.179:0/1907446335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0094b0 con 0x7f8d2015d400 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d24a75700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 0x7f8d200b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d25276700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800cab0 tx=0x7f8d1800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18007480 con 0x7f8d2015d400 2024-05-08T22:39:56.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d180075e0 con 0x7f8d2015d400 2024-05-08T22:39:56.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d100094b0 con 0x7f8d2015d400 2024-05-08T22:39:56.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18007950 con 0x7f8d2015d400 2024-05-08T22:39:56.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.750+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200b24f0 con 0x7f8d2015d400 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20067770 con 0x7f8d2015d400 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d1801b3b0 con 0x7f8d2015d400 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d157fa700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 0x7f8d0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8d18083810 con 0x7f8d2015d400 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d24a75700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 0x7f8d0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:56.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.754+0000 7f8d24a75700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 0x7f8d0c068c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0113d0 tx=0x7f8d1c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:56.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.762+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8d18050d10 con 0x7f8d2015d400 2024-05-08T22:39:56.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:56.938+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8d200b2990 con 0x7f8d2015d400 2024-05-08T22:39:57.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.745+0000 7f8d157fa700 1 -- 172.21.15.179:0/1907446335 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d18017080 con 0x7f8d2015d400 2024-05-08T22:39:57.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 msgr2=0x7f8d0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 0x7f8d0c068c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0113d0 tx=0x7f8d1c009240 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 msgr2=0x7f8d201525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800cab0 tx=0x7f8d1800cae0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 shutdown_connections 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066760 0x7f8d0c068c20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2015d400 0x7f8d201525d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d200ccd90 0x7f8d200b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 --2- 172.21.15.179:0/1907446335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c92d0 0x7f8d200b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.749+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 >> 172.21.15.179:0/1907446335 conn(0x7f8d200cb700 msgr2=0x7f8d2015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.753+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 shutdown_connections 2024-05-08T22:39:57.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.753+0000 7f8d26cd9700 1 -- 172.21.15.179:0/1907446335 wait complete. 2024-05-08T22:39:57.756 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:39:57.771 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:39:57.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.885+0000 7fbdb1a54700 1 Processor -- start 2024-05-08T22:39:57.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.885+0000 7fbdb1a54700 1 -- start start 2024-05-08T22:39:57.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac0cb5d0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdac0cf8e0 con 0x7fbdac0cd2a0 2024-05-08T22:39:57.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0cdc70 con 0x7fbdac0cef40 2024-05-08T22:39:57.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1253700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb0a52700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fbdac15d0e0 0x7fbdac0cc8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:46784/0 (socket says 172.21.15.179:46784) 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45616/0 (socket says 172.21.15.179:45616) 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 -- 172.21.15.179:0/83426342 learned_addr learned my addr 172.21.15.179:0/83426342 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1253700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:39794/0 (socket says 172.21.15.179:39794) 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 -- 172.21.15.179:0/83426342 >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 msgr2=0x7fbdac0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 --2- 172.21.15.179:0/83426342 >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 -- 172.21.15.179:0/83426342 >> v1:172.21.15.195:6789/0 conn(0x7fbdac15d0e0 legacy=0x7fbdac0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 -- 172.21.15.179:0/83426342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac0cc410 con 0x7fbdac0cef40 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda3fff700 1 --2- 172.21.15.179:0/83426342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009910 tx=0x7fbd9c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8eeb09f79636714 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda37fe700 1 -- 172.21.15.179:0/83426342 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c011830 con 0x7fbdac0cef40 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda37fe700 1 -- 172.21.15.179:0/83426342 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c011990 con 0x7fbdac0cef40 2024-05-08T22:39:57.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbda37fe700 1 -- 172.21.15.179:0/83426342 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c011d40 con 0x7fbdac0cef40 2024-05-08T22:39:57.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- 172.21.15.179:0/83426342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 msgr2=0x7fbdac0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/83426342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009910 tx=0x7fbd9c009c20 comp rx=0 tx=0).stop 2024-05-08T22:39:57.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- 172.21.15.179:0/83426342 shutdown_connections 2024-05-08T22:39:57.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/83426342 >> v2:172.21.15.195:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/83426342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac0cf3a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- 172.21.15.179:0/83426342 >> 172.21.15.179:0/83426342 conn(0x7fbdac0b2780 msgr2=0x7fbdac0b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:57.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- 172.21.15.179:0/83426342 shutdown_connections 2024-05-08T22:39:57.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.889+0000 7fbdb1a54700 1 -- 172.21.15.179:0/83426342 wait complete. 2024-05-08T22:39:57.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 Processor -- start 2024-05-08T22:39:57.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 -- start start 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0b59f0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0b4de0 con 0x7fbdac0cd2a0 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1a54700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0b50c0 con 0x7fbdac0cef40 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1253700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58904/0 (socket says 172.21.15.179:58904) 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb1253700 1 -- 172.21.15.179:0/962048030 learned_addr learned my addr 172.21.15.179:0/962048030 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbda3fff700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.893+0000 7fbdb0a52700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 -- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 msgr2=0x7fbdac158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 -- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdac0cd2a0 msgr2=0x7fbdac0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac0b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 -- 172.21.15.179:0/962048030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009570 con 0x7fbdac15d0e0 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbda3fff700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:39:57.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1253700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbda400efc0 tx=0x7fbda400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4009ea0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda400f040 con 0x7fbdac15d0e0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdac0b53a0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda400ced0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac0b4a40 con 0x7fbdac15d0e0 2024-05-08T22:39:57.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac0b2b60 con 0x7fbdac15d0e0 2024-05-08T22:39:57.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.897+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbda4014480 con 0x7fbdac15d0e0 2024-05-08T22:39:57.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.901+0000 7fbda17fa700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 0x7fbd98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:57.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.901+0000 7fbdb0a52700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 0x7fbd98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:57.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.901+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbda40847a0 con 0x7fbdac15d0e0 2024-05-08T22:39:57.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.901+0000 7fbdb0a52700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 0x7fbd98068bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbda8000f60 tx=0x7fbda8007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:57.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:57.905+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbda4051d30 con 0x7fbdac15d0e0 2024-05-08T22:39:58.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.129+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbdac04e700 con 0x7fbdac15d0e0 2024-05-08T22:39:58.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbda17fa700 1 -- 172.21.15.179:0/962048030 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbda4018020 con 0x7fbdac15d0e0 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.135 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:39:58.136 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.137 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.138 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.139 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.140 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.141 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.142 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.143 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.144 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:39:58.145 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.146 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.147 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.148 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:39:58.149 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:39:58.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 msgr2=0x7fbd98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 0x7fbd98068bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbda8000f60 tx=0x7fbda8007ef0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 msgr2=0x7fbdac0ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbda400efc0 tx=0x7fbda400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 shutdown_connections 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066730 0x7fbd98068bf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac15d0e0 0x7fbdac0ca950 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdac0cef40 0x7fbdac158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 --2- 172.21.15.179:0/962048030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 >> 172.21.15.179:0/962048030 conn(0x7fbdac0b2780 msgr2=0x7fbdac0cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 shutdown_connections 2024-05-08T22:39:58.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.133+0000 7fbdb1a54700 1 -- 172.21.15.179:0/962048030 wait complete. 2024-05-08T22:39:58.153 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:39:58.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.261+0000 7fbcd5887700 1 Processor -- start 2024-05-08T22:39:58.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.261+0000 7fbcd5887700 1 -- start start 2024-05-08T22:39:58.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 0x7fbcd006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbcd00cc310 0x7fbcd00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd0161ac0 con 0x7fbcd00add40 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd00ae550 con 0x7fbcd00cc310 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd006abb0 con 0x7fbcd00ce060 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbcd00cc310 0x7fbcd00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbcd00cc310 0x7fbcd00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58912/0 (socket says 172.21.15.179:58912) 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 -- 172.21.15.179:0/3698577862 learned_addr learned my addr 172.21.15.179:0/3698577862 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 -- 172.21.15.179:0/3698577862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 msgr2=0x7fbcd006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcceffd700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 0x7fbcd006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 0x7fbcd006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 -- 172.21.15.179:0/3698577862 >> v1:172.21.15.195:6789/0 conn(0x7fbcd00add40 legacy=0x7fbcd006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 -- 172.21.15.179:0/3698577862 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00ce980 con 0x7fbcd00cc310 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcceffd700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 0x7fbcd006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:58.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcce7fc700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00cc310 0x7fbcd00ce440 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d68ed3d91cd4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbccdffb700 1 -- 172.21.15.179:0/3698577862 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000e070 con 0x7fbcd00cc310 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbccdffb700 1 -- 172.21.15.179:0/3698577862 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc000b9c0 con 0x7fbcd00cc310 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbccdffb700 1 -- 172.21.15.179:0/3698577862 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000bd70 con 0x7fbcd00cc310 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- 172.21.15.179:0/3698577862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00cc310 msgr2=0x7fbcd00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00cc310 0x7fbcd00ce440 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc000b330 comp rx=0 tx=0).stop 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- 172.21.15.179:0/3698577862 shutdown_connections 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00cc310 0x7fbcd00ce440 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3698577862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00ce060 0x7fbcd006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- 172.21.15.179:0/3698577862 >> 172.21.15.179:0/3698577862 conn(0x7fbcd00cb660 msgr2=0x7fbcd00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- 172.21.15.179:0/3698577862 shutdown_connections 2024-05-08T22:39:58.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.265+0000 7fbcd5887700 1 -- 172.21.15.179:0/3698577862 wait complete. 2024-05-08T22:39:58.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 Processor -- start 2024-05-08T22:39:58.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- start start 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcd00add40 0x7fbcd00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00cc310 0x7fbcd00c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00c2d70 con 0x7fbcd00ce060 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00b4780 con 0x7fbcd00add40 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00c27b0 con 0x7fbcd00cc310 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcd00add40 0x7fbcd00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58928/0 (socket says 172.21.15.179:58928) 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 -- 172.21.15.179:0/3845477261 learned_addr learned my addr 172.21.15.179:0/3845477261 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcce7fc700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00cc310 0x7fbcd00c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 -- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00cc310 msgr2=0x7fbcd00c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00cc310 0x7fbcd00c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 -- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcd00add40 msgr2=0x7fbcd00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcd00add40 0x7fbcd00b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 -- 172.21.15.179:0/3845477261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0009b30 con 0x7fbcd00ce060 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbccf7fe700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c3c0 tx=0x7fbcc800c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80052d0 con 0x7fbcd00ce060 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8005450 con 0x7fbcd00ce060 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd00c2a90 con 0x7fbcd00ce060 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8006ac0 con 0x7fbcd00ce060 2024-05-08T22:39:58.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.269+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd00c6f40 con 0x7fbcd00ce060 2024-05-08T22:39:58.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0067770 con 0x7fbcd00ce060 2024-05-08T22:39:58.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcc80036a0 con 0x7fbcd00ce060 2024-05-08T22:39:58.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcb7fff700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 0x7fbcbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbcc8083080 con 0x7fbcd00ce060 2024-05-08T22:39:58.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcceffd700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 0x7fbcbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.273+0000 7fbcceffd700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 0x7fbcbc068bf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4006fd0 tx=0x7fbcc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.277+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbcc80b4cf0 con 0x7fbcd00ce060 2024-05-08T22:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.449+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd0073290 con 0x7fbcd00ce060 2024-05-08T22:39:58.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.449+0000 7fbcb7fff700 1 -- 172.21.15.179:0/3845477261 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12643 (secure 0 0 0) 0x7fbcc8050560 con 0x7fbcd00ce060 2024-05-08T22:39:58.452 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:58.453 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":92,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:39:57.743110+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 msgr2=0x7fbcbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 0x7fbcbc068bf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4006fd0 tx=0x7fbcc4008040 comp rx=0 tx=0).stop 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 msgr2=0x7fbcd00c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c3c0 tx=0x7fbcc800c6d0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 shutdown_connections 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbcbc066730 0x7fbcbc068bf0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd00ce060 0x7fbcd00c61d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbcd00cc310 0x7fbcd00c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 --2- 172.21.15.179:0/3845477261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcd00add40 0x7fbcd00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 >> 172.21.15.179:0/3845477261 conn(0x7fbcd00cb660 msgr2=0x7fbcd00aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 shutdown_connections 2024-05-08T22:39:58.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.453+0000 7fbcd5887700 1 -- 172.21.15.179:0/3845477261 wait complete. 2024-05-08T22:39:58.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:39:58.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:39:58.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.585+0000 7f914647f700 1 Processor -- start 2024-05-08T22:39:58.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.585+0000 7f914647f700 1 -- start start 2024-05-08T22:39:58.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9140142890 0x7f91400b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f914006d4b0 0x7f914015d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400b22d0 con 0x7f91400b2ca0 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400b19e0 con 0x7f9140142890 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914015e640 con 0x7f914006d4b0 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9140142890 0x7f91400b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9140142890 0x7f91400b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58944/0 (socket says 172.21.15.179:58944) 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 -- 172.21.15.179:0/1875484644 learned_addr learned my addr 172.21.15.179:0/1875484644 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f9145c7e700 1 --2- 172.21.15.179:0/1875484644 >> v2:172.21.15.195:3300/0 conn(0x7f914006d4b0 0x7f914015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 -- 172.21.15.179:0/1875484644 >> v2:172.21.15.195:3300/0 conn(0x7f914006d4b0 msgr2=0x7f914015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 --2- 172.21.15.179:0/1875484644 >> v2:172.21.15.195:3300/0 conn(0x7f914006d4b0 0x7f914015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 -- 172.21.15.179:0/1875484644 >> v1:172.21.15.195:6789/0 conn(0x7f91400b2ca0 legacy=0x7f914006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 -- 172.21.15.179:0/1875484644 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400b1c80 con 0x7f9140142890 2024-05-08T22:39:58.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914547d700 1 --2- 172.21.15.179:0/1875484644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9140142890 0x7f91400b2760 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f91400cc9d0 tx=0x7f913c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b297639f17c27de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f9137fff700 1 -- 172.21.15.179:0/1875484644 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c011830 con 0x7f9140142890 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f9137fff700 1 -- 172.21.15.179:0/1875484644 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c011990 con 0x7f9140142890 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f9137fff700 1 -- 172.21.15.179:0/1875484644 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c011d20 con 0x7f9140142890 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- 172.21.15.179:0/1875484644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9140142890 msgr2=0x7f91400b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 --2- 172.21.15.179:0/1875484644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9140142890 0x7f91400b2760 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f91400cc9d0 tx=0x7f913c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- 172.21.15.179:0/1875484644 shutdown_connections 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 --2- 172.21.15.179:0/1875484644 >> v2:172.21.15.195:3300/0 conn(0x7f914006d4b0 0x7f914015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 --2- 172.21.15.179:0/1875484644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9140142890 0x7f91400b2760 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- 172.21.15.179:0/1875484644 >> 172.21.15.179:0/1875484644 conn(0x7f91400cea30 msgr2=0x7f91400cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- 172.21.15.179:0/1875484644 shutdown_connections 2024-05-08T22:39:58.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.589+0000 7f914647f700 1 -- 172.21.15.179:0/1875484644 wait complete. 2024-05-08T22:39:58.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 Processor -- start 2024-05-08T22:39:58.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- start start 2024-05-08T22:39:58.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f914006d4b0 0x7f91401f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140142890 0x7f9140070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401f3c00 con 0x7f91400b2ca0 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914013cb50 con 0x7f914006d4b0 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91401f4d30 con 0x7f9140142890 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58958/0 (socket says 172.21.15.179:58958) 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 -- 172.21.15.179:0/23352689 learned_addr learned my addr 172.21.15.179:0/23352689 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914547d700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f914006d4b0 0x7f91401f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9145c7e700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140142890 0x7f9140070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 -- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140142890 msgr2=0x7f9140070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140142890 0x7f9140070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 -- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f914006d4b0 msgr2=0x7f91401f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f914006d4b0 0x7f91401f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 -- 172.21.15.179:0/23352689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009570 con 0x7f91400b2ca0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9144c7c700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f913000c500 tx=0x7f913000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300052d0 con 0x7f91400b2ca0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400aa060 con 0x7f91400b2ca0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130005430 con 0x7f91400b2ca0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300055a0 con 0x7f91400b2ca0 2024-05-08T22:39:58.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.593+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91401f0f70 con 0x7f91400b2ca0 2024-05-08T22:39:58.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.597+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914004e700 con 0x7f91400b2ca0 2024-05-08T22:39:58.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.597+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91300038e0 con 0x7f91400b2ca0 2024-05-08T22:39:58.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.597+0000 7f9135ffb700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 0x7f912c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.597+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9130083450 con 0x7f91400b2ca0 2024-05-08T22:39:58.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.597+0000 7f914547d700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 0x7f912c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.601+0000 7f914547d700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 0x7f912c068bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f913c000c00 tx=0x7f913c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.601+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9130050950 con 0x7f91400b2ca0 2024-05-08T22:39:58.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.777+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91400af300 con 0x7f91400b2ca0 2024-05-08T22:39:58.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.777+0000 7f9135ffb700 1 -- 172.21.15.179:0/23352689 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9130053f70 con 0x7f91400b2ca0 2024-05-08T22:39:58.780 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 msgr2=0x7f912c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 0x7f912c068bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f913c000c00 tx=0x7f913c008030 comp rx=0 tx=0).stop 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 msgr2=0x7f91401f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f913000c500 tx=0x7f913000c810 comp rx=0 tx=0).stop 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 shutdown_connections 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f912c066730 0x7f912c068bf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9140142890 0x7f9140070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91400b2ca0 0x7f91401f47f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 --2- 172.21.15.179:0/23352689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f914006d4b0 0x7f91401f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 >> 172.21.15.179:0/23352689 conn(0x7f91400cea30 msgr2=0x7f91400afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 shutdown_connections 2024-05-08T22:39:58.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.781+0000 7f914647f700 1 -- 172.21.15.179:0/23352689 wait complete. 2024-05-08T22:39:58.795 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:39:58.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 Processor -- start 2024-05-08T22:39:58.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- start start 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffae8161640 0x7ffae80d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80d2640 con 0x7ffae815d580 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae806b3a0 con 0x7ffae80d18a0 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae80d2360 con 0x7ffae8161640 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffae815d580 0x7ffae80b6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47894/0 (socket says 172.21.15.179:47894) 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae77fe700 1 -- 172.21.15.179:0/3531018510 learned_addr learned my addr 172.21.15.179:0/3531018510 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:58.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae7fff700 1 --2- 172.21.15.179:0/3531018510 >> v2:172.21.15.179:3300/0 conn(0x7ffae8161640 0x7ffae80d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae67fc700 1 -- 172.21.15.179:0/3531018510 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762182784 0 0) 0x7ffae80d2640 con 0x7ffae815d580 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae67fc700 1 -- 172.21.15.179:0/3531018510 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad00035d0 con 0x7ffae815d580 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae7fff700 1 -- 172.21.15.179:0/3531018510 >> v1:172.21.15.195:6789/0 conn(0x7ffae80d18a0 legacy=0x7ffae8160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae7fff700 1 -- 172.21.15.179:0/3531018510 >> v1:172.21.15.179:6789/0 conn(0x7ffae815d580 legacy=0x7ffae80b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae7fff700 1 -- 172.21.15.179:0/3531018510 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae806ad50 con 0x7ffae8161640 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae7fff700 1 --2- 172.21.15.179:0/3531018510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae80d1d30 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009d90 tx=0x7ffae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb60c271edadbc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae67fc700 1 -- 172.21.15.179:0/3531018510 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000e070 con 0x7ffae8161640 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae67fc700 1 -- 172.21.15.179:0/3531018510 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000bb20 con 0x7ffae8161640 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffae67fc700 1 -- 172.21.15.179:0/3531018510 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0012430 con 0x7ffae8161640 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- 172.21.15.179:0/3531018510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 msgr2=0x7ffae80d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/3531018510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae80d1d30 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009d90 tx=0x7ffae000b330 comp rx=0 tx=0).stop 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- 172.21.15.179:0/3531018510 shutdown_connections 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/3531018510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae80d1d30 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- 172.21.15.179:0/3531018510 >> 172.21.15.179:0/3531018510 conn(0x7ffae80d2d90 msgr2=0x7ffae80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- 172.21.15.179:0/3531018510 shutdown_connections 2024-05-08T22:39:58.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.901+0000 7ffaeccb4700 1 -- 172.21.15.179:0/3531018510 wait complete. 2024-05-08T22:39:58.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 Processor -- start 2024-05-08T22:39:58.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- start start 2024-05-08T22:39:58.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffae80d18a0 0x7ffae80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 0x7ffae8159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80b5730 con 0x7ffae8161640 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8148f10 con 0x7ffae815d580 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80b5450 con 0x7ffae80d18a0 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58968/0 (socket says 172.21.15.179:58968) 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 -- 172.21.15.179:0/4086134968 learned_addr learned my addr 172.21.15.179:0/4086134968 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae77fe700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffae80d18a0 0x7ffae80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae6ffd700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 0x7ffae8159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 -- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffae80d18a0 msgr2=0x7ffae80b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffae80d18a0 0x7ffae80b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 -- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 msgr2=0x7ffae8159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 0x7ffae8159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 -- 172.21.15.179:0/4086134968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0009a70 con 0x7ffae8161640 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae6ffd700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 0x7ffae8159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffae7fff700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ffae0013000 tx=0x7ffae0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0012430 con 0x7ffae8161640 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae0020070 con 0x7ffae8161640 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae815e4f0 con 0x7ffae8161640 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0004280 con 0x7ffae8161640 2024-05-08T22:39:58.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.905+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae815eab0 con 0x7ffae8161640 2024-05-08T22:39:58.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae815e320 con 0x7ffae8161640 2024-05-08T22:39:58.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffae00043e0 con 0x7ffae8161640 2024-05-08T22:39:58.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffacffff700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 0x7ffad4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:58.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ffae0022070 con 0x7ffae8161640 2024-05-08T22:39:58.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffae77fe700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 0x7ffad4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:58.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.909+0000 7ffae77fe700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 0x7ffad4068bf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0011e0 tx=0x7ffadc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:58.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:58.917+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffae0053660 con 0x7ffae8161640 2024-05-08T22:39:59.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.089+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffae804e700 con 0x7ffae8161640 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffacffff700 1 -- 172.21.15.179:0/4086134968 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffae0056c80 con 0x7ffae8161640 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:39:59.094 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:39:59.095 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:39:59.095 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 msgr2=0x7ffad4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 0x7ffad4068bf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0011e0 tx=0x7ffadc009040 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 msgr2=0x7ffae81f1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ffae0013000 tx=0x7ffae0004af0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 shutdown_connections 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffad4066730 0x7ffad4068bf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffae815d580 0x7ffae8159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae8161640 0x7ffae81f1dc0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 --2- 172.21.15.179:0/4086134968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffae80d18a0 0x7ffae80b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 >> 172.21.15.179:0/4086134968 conn(0x7ffae80d2d90 msgr2=0x7ffae80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 shutdown_connections 2024-05-08T22:39:59.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.093+0000 7ffaeccb4700 1 -- 172.21.15.179:0/4086134968 wait complete. 2024-05-08T22:39:59.109 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:39:59.109 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:39:59.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 Processor -- start 2024-05-08T22:39:59.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 -- start start 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03c003a570 0x7f03c0006920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c0053380 con 0x7f03c0053810 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c0007250 con 0x7f03c004f840 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ceb66700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c0007530 con 0x7f03c003a570 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.225+0000 7f03ce365700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f03c0053810 0x7f03c0053ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47896/0 (socket says 172.21.15.179:47896) 2024-05-08T22:39:59.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ce365700 1 -- 172.21.15.179:0/2084540780 learned_addr learned my addr 172.21.15.179:0/2084540780 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:59.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03cd363700 1 --2- 172.21.15.179:0/2084540780 >> v2:172.21.15.195:3300/0 conn(0x7f03c003a570 0x7f03c0006920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ccb62700 1 -- 172.21.15.179:0/2084540780 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957495143 0 0) 0x7f03c0053380 con 0x7f03c0053810 2024-05-08T22:39:59.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ccb62700 1 -- 172.21.15.179:0/2084540780 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b40035d0 con 0x7f03c0053810 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03cd363700 1 -- 172.21.15.179:0/2084540780 >> v1:172.21.15.195:6789/0 conn(0x7f03c004f840 legacy=0x7f03c004fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03cd363700 1 -- 172.21.15.179:0/2084540780 >> v1:172.21.15.179:6789/0 conn(0x7f03c0053810 legacy=0x7f03c0053ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03cd363700 1 -- 172.21.15.179:0/2084540780 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0007810 con 0x7f03c003a570 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03cd363700 1 --2- 172.21.15.179:0/2084540780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c003a570 0x7f03c0006920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009850 tx=0x7f03c4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8592b11aad4179d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ccb62700 1 -- 172.21.15.179:0/2084540780 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011830 con 0x7f03c003a570 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ccb62700 1 -- 172.21.15.179:0/2084540780 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4011990 con 0x7f03c003a570 2024-05-08T22:39:59.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ccb62700 1 -- 172.21.15.179:0/2084540780 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011d40 con 0x7f03c003a570 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 -- 172.21.15.179:0/2084540780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c003a570 msgr2=0x7f03c0006920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2084540780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c003a570 0x7f03c0006920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009850 tx=0x7f03c4009b60 comp rx=0 tx=0).stop 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 -- 172.21.15.179:0/2084540780 shutdown_connections 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2084540780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c003a570 0x7f03c0006920 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 -- 172.21.15.179:0/2084540780 >> 172.21.15.179:0/2084540780 conn(0x7f03c0054b70 msgr2=0x7f03c0054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 -- 172.21.15.179:0/2084540780 shutdown_connections 2024-05-08T22:39:59.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.229+0000 7f03ceb66700 1 -- 172.21.15.179:0/2084540780 wait complete. 2024-05-08T22:39:59.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 Processor -- start 2024-05-08T22:39:59.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- start start 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c004f840 0x7f03c00e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03c0053810 0x7f03c00c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00d7d20 con 0x7f03c003a570 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00d70a0 con 0x7f03c004f840 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00d7380 con 0x7f03c0053810 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58970/0 (socket says 172.21.15.179:58970) 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 -- 172.21.15.179:0/2506014515 learned_addr learned my addr 172.21.15.179:0/2506014515 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cd363700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c004f840 0x7f03c00e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ce365700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03c0053810 0x7f03c00c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 -- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03c0053810 msgr2=0x7f03c00c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03c0053810 0x7f03c00c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 -- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c004f840 msgr2=0x7f03c00e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c004f840 0x7f03c00e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 -- 172.21.15.179:0/2506014515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40094b0 con 0x7f03c003a570 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03cdb64700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c500 tx=0x7f03b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b80052d0 con 0x7f03c003a570 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b8005430 con 0x7f03c003a570 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b80055a0 con 0x7f03c003a570 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c00e3600 con 0x7f03c003a570 2024-05-08T22:39:59.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.233+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00d7940 con 0x7f03c003a570 2024-05-08T22:39:59.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c00d7e60 con 0x7f03c003a570 2024-05-08T22:39:59.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03b80038e0 con 0x7f03c003a570 2024-05-08T22:39:59.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03be7fc700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 0x7f03ac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f03b8083450 con 0x7f03c003a570 2024-05-08T22:39:59.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03cd363700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 0x7f03ac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.237+0000 7f03cd363700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 0x7f03ac068c20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f03c003a9b0 tx=0x7f03c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.245+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03b80509d0 con 0x7f03c003a570 2024-05-08T22:39:59.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.413+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f03c00d80b0 con 0x7f03c003a570 2024-05-08T22:39:59.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.757+0000 7f03be7fc700 1 -- 172.21.15.179:0/2506014515 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7f03b8053ff0 con 0x7f03c003a570 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 msgr2=0x7f03ac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 0x7f03ac068c20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f03c003a9b0 tx=0x7f03c4008030 comp rx=0 tx=0).stop 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 msgr2=0x7f03c00d44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c500 tx=0x7f03b800c8c0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 shutdown_connections 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f03ac066760 0x7f03ac068c20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03c004f840 0x7f03c00e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03c0053810 0x7f03c00c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 --2- 172.21.15.179:0/2506014515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c003a570 0x7f03c00d44f0 secure :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c500 tx=0x7f03b800c8c0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.761+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 >> 172.21.15.179:0/2506014515 conn(0x7f03c0054b70 msgr2=0x7f03c00de4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:59.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.765+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 shutdown_connections 2024-05-08T22:39:59.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.765+0000 7f03ceb66700 1 -- 172.21.15.179:0/2506014515 wait complete. 2024-05-08T22:39:59.768 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:39:59.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:39:59.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 Processor -- start 2024-05-08T22:39:59.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 -- start start 2024-05-08T22:39:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc1b40ae760 0x7fc1b40cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc1b40cb560 0x7fc1b40ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40cd860 con 0x7fc1b40cd450 2024-05-08T22:39:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40ca3a0 con 0x7fc1b40cb560 2024-05-08T22:39:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40ca680 con 0x7fc1b40ae760 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b27fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc1b40cb560 0x7fc1b40ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc1b40cd450 0x7fc1b40b4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47906/0 (socket says 172.21.15.179:47906) 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.901+0000 7fc1b37fe700 1 -- 172.21.15.179:0/568235681 learned_addr learned my addr 172.21.15.179:0/568235681 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b2ffd700 1 --2- 172.21.15.179:0/568235681 >> v2:172.21.15.195:3300/0 conn(0x7fc1b40ae760 0x7fc1b40cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b27fc700 1 -- 172.21.15.179:0/568235681 >> v2:172.21.15.195:3300/0 conn(0x7fc1b40ae760 msgr2=0x7fc1b40cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b27fc700 1 --2- 172.21.15.179:0/568235681 >> v2:172.21.15.195:3300/0 conn(0x7fc1b40ae760 0x7fc1b40cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b27fc700 1 -- 172.21.15.179:0/568235681 >> v1:172.21.15.179:6789/0 conn(0x7fc1b40cd450 legacy=0x7fc1b40b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b27fc700 1 -- 172.21.15.179:0/568235681 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40ca960 con 0x7fc1b40cb560 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b1ffb700 1 -- 172.21.15.179:0/568235681 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250944170 0 0) 0x7fc1b40cd860 con 0x7fc1b40cd450 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b27fc700 1 --2- 172.21.15.179:0/568235681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40cb560 0x7fc1b40ccf10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc1b406fea0 tx=0x7fc1a8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e78dd01083ccb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b1ffb700 1 -- 172.21.15.179:0/568235681 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8011830 con 0x7fc1b40cb560 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b1ffb700 1 -- 172.21.15.179:0/568235681 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8011990 con 0x7fc1b40cb560 2024-05-08T22:39:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b1ffb700 1 -- 172.21.15.179:0/568235681 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8011d20 con 0x7fc1b40cb560 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 -- 172.21.15.179:0/568235681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40cb560 msgr2=0x7fc1b40ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/568235681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40cb560 0x7fc1b40ccf10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc1b406fea0 tx=0x7fc1a8009b30 comp rx=0 tx=0).stop 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 -- 172.21.15.179:0/568235681 shutdown_connections 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/568235681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40cb560 0x7fc1b40ccf10 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/568235681 >> v2:172.21.15.195:3300/0 conn(0x7fc1b40ae760 0x7fc1b40cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 -- 172.21.15.179:0/568235681 >> 172.21.15.179:0/568235681 conn(0x7fc1b4098a70 msgr2=0x7fc1b40b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 -- 172.21.15.179:0/568235681 shutdown_connections 2024-05-08T22:39:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.905+0000 7fc1b3fff700 1 -- 172.21.15.179:0/568235681 wait complete. 2024-05-08T22:39:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 Processor -- start 2024-05-08T22:39:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- start start 2024-05-08T22:39:59.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b40cb560 0x7fc1b40c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1b40cd450 0x7fc1b4150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4150c90 con 0x7fc1b40ae760 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40af8d0 con 0x7fc1b40cb560 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40afbb0 con 0x7fc1b40cd450 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58986/0 (socket says 172.21.15.179:58986) 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 -- 172.21.15.179:0/2820737989 learned_addr learned my addr 172.21.15.179:0/2820737989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b27fc700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b40cb560 0x7fc1b40c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b37fe700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1b40cd450 0x7fc1b4150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 -- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1b40cd450 msgr2=0x7fc1b4150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1b40cd450 0x7fc1b4150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 -- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b40cb560 msgr2=0x7fc1b40c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b40cb560 0x7fc1b40c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 -- 172.21.15.179:0/2820737989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80094b0 con 0x7fc1b40ae760 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b2ffd700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400e840 tx=0x7fc1a400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40095a0 con 0x7fc1b40ae760 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4013680 con 0x7fc1b40ae760 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b40afe90 con 0x7fc1b40ae760 2024-05-08T22:39:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40139f0 con 0x7fc1b40ae760 2024-05-08T22:39:59.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.909+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b40b0450 con 0x7fc1b40ae760 2024-05-08T22:39:59.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.913+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b404e700 con 0x7fc1b40ae760 2024-05-08T22:39:59.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.913+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc1a4013b50 con 0x7fc1b40ae760 2024-05-08T22:39:59.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.913+0000 7fc19b7fe700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 0x7fc1a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:39:59.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.913+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fc1a4083e30 con 0x7fc1b40ae760 2024-05-08T22:39:59.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.913+0000 7fc1b27fc700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 0x7fc1a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:39:59.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.917+0000 7fc1b27fc700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 0x7fc1a0068c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40cb9f0 tx=0x7fc1a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:39:59.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:39:59.921+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc1a40512d0 con 0x7fc1b40ae760 2024-05-08T22:40:00.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.089+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc1b406dda0 con 0x7fc1b40ae760 2024-05-08T22:40:00.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.757+0000 7fc19b7fe700 1 -- 172.21.15.179:0/2820737989 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc1a40548f0 con 0x7fc1b40ae760 2024-05-08T22:40:00.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 msgr2=0x7fc1a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 0x7fc1a0068c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40cb9f0 tx=0x7fc1a8008030 comp rx=0 tx=0).stop 2024-05-08T22:40:00.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 msgr2=0x7fc1b40c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400e840 tx=0x7fc1a400eb50 comp rx=0 tx=0).stop 2024-05-08T22:40:00.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 shutdown_connections 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc1a0066760 0x7fc1a0068c20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b40cb560 0x7fc1b40c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1b40cd450 0x7fc1b4150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 --2- 172.21.15.179:0/2820737989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1b40ae760 0x7fc1b40c54b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 >> 172.21.15.179:0/2820737989 conn(0x7fc1b4098a70 msgr2=0x7fc1b406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 shutdown_connections 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.781+0000 7fc1b3fff700 1 -- 172.21.15.179:0/2820737989 wait complete. 2024-05-08T22:40:00.785 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:40:00.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:40:00.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 Processor -- start 2024-05-08T22:40:00.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 -- start start 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3980158850 0x7f39800ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f398006e090 0x7f398015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39800ccb40 con 0x7f398015eec0 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398015f330 con 0x7f3980158850 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3987161700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39800cb370 con 0x7f398006e090 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f398615f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3980158850 0x7f39800ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3986960700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f398015eec0 0x7f398015b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47916/0 (socket says 172.21.15.179:47916) 2024-05-08T22:40:00.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f3986960700 1 -- 172.21.15.179:0/4064979583 learned_addr learned my addr 172.21.15.179:0/4064979583 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f398595e700 1 --2- 172.21.15.179:0/4064979583 >> v2:172.21.15.195:3300/0 conn(0x7f398006e090 0x7f398015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f398615f700 1 -- 172.21.15.179:0/4064979583 >> v2:172.21.15.195:3300/0 conn(0x7f398006e090 msgr2=0x7f398015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f398615f700 1 --2- 172.21.15.179:0/4064979583 >> v2:172.21.15.195:3300/0 conn(0x7f398006e090 0x7f398015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.909+0000 7f398515d700 1 -- 172.21.15.179:0/4064979583 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468182763 0 0) 0x7f39800ccb40 con 0x7f398015eec0 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398615f700 1 -- 172.21.15.179:0/4064979583 >> v1:172.21.15.179:6789/0 conn(0x7f398015eec0 legacy=0x7f398015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398615f700 1 -- 172.21.15.179:0/4064979583 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398006e5c0 con 0x7f3980158850 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398615f700 1 --2- 172.21.15.179:0/4064979583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3980158850 0x7f39800ce230 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3970009850 tx=0x7f3970009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39acf0df2b40bd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398515d700 1 -- 172.21.15.179:0/4064979583 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011830 con 0x7f3980158850 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398515d700 1 -- 172.21.15.179:0/4064979583 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970011990 con 0x7f3980158850 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f398515d700 1 -- 172.21.15.179:0/4064979583 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970011d20 con 0x7f3980158850 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 -- 172.21.15.179:0/4064979583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3980158850 msgr2=0x7f39800ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 --2- 172.21.15.179:0/4064979583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3980158850 0x7f39800ce230 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3970009850 tx=0x7f3970009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:00.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 -- 172.21.15.179:0/4064979583 shutdown_connections 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 --2- 172.21.15.179:0/4064979583 >> v2:172.21.15.195:3300/0 conn(0x7f398006e090 0x7f398015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 --2- 172.21.15.179:0/4064979583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3980158850 0x7f39800ce230 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 -- 172.21.15.179:0/4064979583 >> 172.21.15.179:0/4064979583 conn(0x7f39800cea30 msgr2=0x7f39800cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 -- 172.21.15.179:0/4064979583 shutdown_connections 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 -- 172.21.15.179:0/4064979583 wait complete. 2024-05-08T22:40:00.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.913+0000 7f3987161700 1 Processor -- start 2024-05-08T22:40:00.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- start start 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3980158850 0x7f398006b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398015eec0 0x7f398006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801f3770 con 0x7f398006e090 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801f05f0 con 0x7f398015eec0 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39800b1ae0 con 0x7f3980158850 2024-05-08T22:40:00.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58994/0 (socket says 172.21.15.179:58994) 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 -- 172.21.15.179:0/418473653 learned_addr learned my addr 172.21.15.179:0/418473653 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398595e700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3980158850 0x7f398006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3986960700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398015eec0 0x7f398006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 -- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3980158850 msgr2=0x7f398006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3980158850 0x7f398006b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 -- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398015eec0 msgr2=0x7f398006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398015eec0 0x7f398006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 -- 172.21.15.179:0/418473653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700094b0 con 0x7f398006e090 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f398615f700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f39700090b0 tx=0x7f3970004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970019a00 con 0x7f398006e090 2024-05-08T22:40:00.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397001e070 con 0x7f398006e090 2024-05-08T22:40:00.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39801f2e40 con 0x7f398006e090 2024-05-08T22:40:00.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970019dc0 con 0x7f398006e090 2024-05-08T22:40:00.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.917+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39801f3400 con 0x7f398006e090 2024-05-08T22:40:00.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398004e700 con 0x7f398006e090 2024-05-08T22:40:00.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f397002d450 con 0x7f398006e090 2024-05-08T22:40:00.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f397effd700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 0x7f3968068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:00.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f3970085130 con 0x7f398006e090 2024-05-08T22:40:00.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f398595e700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 0x7f3968068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:00.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.921+0000 7f398595e700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 0x7f3968068bf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f39800ad570 tx=0x7f396c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:00.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:00.925+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3970052450 con 0x7f398006e090 2024-05-08T22:40:01.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.133+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f398015aab0 con 0x7f398006e090 2024-05-08T22:40:01.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.149+0000 7f397effd700 1 -- 172.21.15.179:0/418473653 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f397005ae60 con 0x7f398006e090 2024-05-08T22:40:01.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 msgr2=0x7f3968068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 0x7f3968068bf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f39800ad570 tx=0x7f396c00b040 comp rx=0 tx=0).stop 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 msgr2=0x7f39801f00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f39700090b0 tx=0x7f3970004770 comp rx=0 tx=0).stop 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 shutdown_connections 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3968066730 0x7f3968068bf0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398015eec0 0x7f398006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3980158850 0x7f398006b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 --2- 172.21.15.179:0/418473653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398006e090 0x7f39801f00b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.157+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 >> 172.21.15.179:0/418473653 conn(0x7f39800cea30 msgr2=0x7f39800cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.165+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 shutdown_connections 2024-05-08T22:40:01.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.165+0000 7f3987161700 1 -- 172.21.15.179:0/418473653 wait complete. 2024-05-08T22:40:01.170 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 13 and data pool 14 2024-05-08T22:40:01.187 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:40:01.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 Processor -- start 2024-05-08T22:40:01.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- start start 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f75e80ce700 0x7f75e80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e80b5500 con 0x7f75e81447c0 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e80b48f0 con 0x7f75e80d0650 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e80ceb70 con 0x7f75e80ce700 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ecf68700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f75e81447c0 0x7f75e80cff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47920/0 (socket says 172.21.15.179:47920) 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ecf68700 1 -- 172.21.15.179:0/1859105080 learned_addr learned my addr 172.21.15.179:0/1859105080 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ed769700 1 --2- 172.21.15.179:0/1859105080 >> v2:172.21.15.179:3300/0 conn(0x7f75e80ce700 0x7f75e80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258800066 0 0) 0x7f75e80b5500 con 0x7f75e81447c0 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d40035d0 con 0x7f75e81447c0 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580859046 0 0) 0x7f75d40035d0 con 0x7f75e81447c0 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ed769700 1 -- 172.21.15.179:0/1859105080 >> v1:172.21.15.195:6789/0 conn(0x7f75e80d0650 legacy=0x7f75e80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ed769700 1 -- 172.21.15.179:0/1859105080 >> v1:172.21.15.179:6789/0 conn(0x7f75e81447c0 legacy=0x7f75e80cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ed769700 1 -- 172.21.15.179:0/1859105080 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80b4bd0 con 0x7f75e80ce700 2024-05-08T22:40:01.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75ed769700 1 --2- 172.21.15.179:0/1859105080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80ce700 0x7f75e80b41c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009850 tx=0x7f75d8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecff5bbcadc5bf39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0001670 con 0x7f75e80ce700 2024-05-08T22:40:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8004030 con 0x7f75e80ce700 2024-05-08T22:40:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75e77fe700 1 -- 172.21.15.179:0/1859105080 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80043c0 con 0x7f75e80ce700 2024-05-08T22:40:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- 172.21.15.179:0/1859105080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80ce700 msgr2=0x7f75e80b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 --2- 172.21.15.179:0/1859105080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80ce700 0x7f75e80b41c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009850 tx=0x7f75d8009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- 172.21.15.179:0/1859105080 shutdown_connections 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 --2- 172.21.15.179:0/1859105080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80ce700 0x7f75e80b41c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- 172.21.15.179:0/1859105080 >> 172.21.15.179:0/1859105080 conn(0x7f75e80b2840 msgr2=0x7f75e80b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.285+0000 7f75edf6a700 1 -- 172.21.15.179:0/1859105080 shutdown_connections 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- 172.21.15.179:0/1859105080 wait complete. 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 Processor -- start 2024-05-08T22:40:01.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- start start 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 0x7f75e81f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 0x7f75e81f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81f5cc0 con 0x7f75e80d0650 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81f3850 con 0x7f75e81447c0 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80a7b80 con 0x7f75e80ce700 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59022/0 (socket says 172.21.15.179:59022) 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 -- 172.21.15.179:0/435633126 learned_addr learned my addr 172.21.15.179:0/435633126 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75ed769700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 0x7f75e81f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75ecf68700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 0x7f75e81f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 -- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 msgr2=0x7f75e81f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 0x7f75e81f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 -- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 msgr2=0x7f75e81f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 0x7f75e81f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 -- 172.21.15.179:0/435633126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80094b0 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75ed769700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 0x7f75e81f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75ecf68700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 0x7f75e81f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e7fff700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f75e0001ed0 tx=0x7f75e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e000ade0 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e00020d0 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0003c40 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e80a7e60 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.289+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e80a8420 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e804e700 con 0x7f75e80d0650 2024-05-08T22:40:01.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75e00050d0 con 0x7f75e80d0650 2024-05-08T22:40:01.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75e57fa700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 0x7f75c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f75e0084400 con 0x7f75e80d0650 2024-05-08T22:40:01.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75ecf68700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 0x7f75c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.293+0000 7f75ecf68700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 0x7f75c8068c20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f75d0007910 tx=0x7f75d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.301+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75e0051630 con 0x7f75e80d0650 2024-05-08T22:40:01.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.473+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75e80676e0 con 0x7f75e80d0650 2024-05-08T22:40:01.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.477+0000 7f75e57fa700 1 -- 172.21.15.179:0/435633126 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16310 (secure 0 0 0) 0x7f75e0054c50 con 0x7f75e80d0650 2024-05-08T22:40:01.480 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:01.480 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":95,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:01.140164+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:40:01.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 msgr2=0x7f75c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 0x7f75c8068c20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f75d0007910 tx=0x7f75d0009040 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 msgr2=0x7f75e81f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f75e0001ed0 tx=0x7f75e000a340 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 shutdown_connections 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75c8066760 0x7f75c8068c20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e81447c0 0x7f75e81f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e80d0650 0x7f75e81f5780 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 --2- 172.21.15.179:0/435633126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75e80ce700 0x7f75e81f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 >> 172.21.15.179:0/435633126 conn(0x7f75e80b2840 msgr2=0x7f75e80b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 shutdown_connections 2024-05-08T22:40:01.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.481+0000 7f75edf6a700 1 -- 172.21.15.179:0/435633126 wait complete. 2024-05-08T22:40:01.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:40:01.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.609+0000 7f3e4ad43700 1 Processor -- start 2024-05-08T22:40:01.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.609+0000 7f3e4ad43700 1 -- start start 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3e4406de70 0x7f3e44161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e44164db0 0x7f3e4406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e44165d50 con 0x7f3e4415ef80 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4406ad70 con 0x7f3e4406de70 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e44165260 con 0x7f3e44164db0 2024-05-08T22:40:01.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49d41700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3e4415ef80 0x7f3e4406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47934/0 (socket says 172.21.15.179:47934) 2024-05-08T22:40:01.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49d41700 1 -- 172.21.15.179:0/1485010642 learned_addr learned my addr 172.21.15.179:0/1485010642 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:01.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 --2- 172.21.15.179:0/1485010642 >> v2:172.21.15.195:3300/0 conn(0x7f3e4406de70 0x7f3e44161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4a542700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e44164db0 0x7f3e4406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e48d3f700 1 -- 172.21.15.179:0/1485010642 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290631357 0 0) 0x7f3e44165d50 con 0x7f3e4415ef80 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e48d3f700 1 -- 172.21.15.179:0/1485010642 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e300035d0 con 0x7f3e4415ef80 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 -- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e44164db0 msgr2=0x7f3e4406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e44164db0 0x7f3e4406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 -- 172.21.15.179:0/1485010642 >> v1:172.21.15.179:6789/0 conn(0x7f3e4415ef80 legacy=0x7f3e4406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 -- 172.21.15.179:0/1485010642 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4406e4b0 con 0x7f3e4406de70 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e49540700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4406de70 0x7f3e44161790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009850 tx=0x7f3e38009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115e4517429b52c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e48d3f700 1 -- 172.21.15.179:0/1485010642 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38011830 con 0x7f3e4406de70 2024-05-08T22:40:01.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e48d3f700 1 -- 172.21.15.179:0/1485010642 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38011990 con 0x7f3e4406de70 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e48d3f700 1 -- 172.21.15.179:0/1485010642 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38011d40 con 0x7f3e4406de70 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4406de70 msgr2=0x7f3e44161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4406de70 0x7f3e44161790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009850 tx=0x7f3e38009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- 172.21.15.179:0/1485010642 shutdown_connections 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e44164db0 0x7f3e4406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/1485010642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4406de70 0x7f3e44161790 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- 172.21.15.179:0/1485010642 >> 172.21.15.179:0/1485010642 conn(0x7f3e440ce9d0 msgr2=0x7f3e440cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- 172.21.15.179:0/1485010642 shutdown_connections 2024-05-08T22:40:01.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.613+0000 7f3e4ad43700 1 -- 172.21.15.179:0/1485010642 wait complete. 2024-05-08T22:40:01.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 Processor -- start 2024-05-08T22:40:01.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- start start 2024-05-08T22:40:01.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e4415ef80 0x7f3e44150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e44164db0 0x7f3e4406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440c75e0 con 0x7f3e4406de70 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440c39f0 con 0x7f3e44164db0 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440c3cd0 con 0x7f3e4415ef80 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59036/0 (socket says 172.21.15.179:59036) 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 -- 172.21.15.179:0/2218326200 learned_addr learned my addr 172.21.15.179:0/2218326200 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49540700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e4415ef80 0x7f3e44150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4a542700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e44164db0 0x7f3e4406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 -- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e4415ef80 msgr2=0x7f3e44150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e4415ef80 0x7f3e44150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 -- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e44164db0 msgr2=0x7f3e4406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e44164db0 0x7f3e4406bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 -- 172.21.15.179:0/2218326200 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380094b0 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e49d41700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c700 tx=0x7f3e4000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40005d90 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e400070d0 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e440c3fb0 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40007480 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.617+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e44151170 con 0x7f3e4406de70 2024-05-08T22:40:01.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e44067740 con 0x7f3e4406de70 2024-05-08T22:40:01.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e400075e0 con 0x7f3e4406de70 2024-05-08T22:40:01.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e367fc700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 0x7f3e28068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3e40083760 con 0x7f3e4406de70 2024-05-08T22:40:01.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e49540700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 0x7f3e28068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.621+0000 7f3e49540700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 0x7f3e28068c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3e380090b0 tx=0x7f3e38008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.629+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e40050900 con 0x7f3e4406de70 2024-05-08T22:40:01.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.805+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e440c41f0 con 0x7f3e4406de70 2024-05-08T22:40:01.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e367fc700 1 -- 172.21.15.179:0/2218326200 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16310 (secure 0 0 0) 0x7f3e40053f20 con 0x7f3e4406de70 2024-05-08T22:40:01.810 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:01.811 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":95,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:01.140164+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 msgr2=0x7f3e28068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 0x7f3e28068c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3e380090b0 tx=0x7f3e38008030 comp rx=0 tx=0).stop 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 msgr2=0x7f3e440c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c700 tx=0x7f3e4000ca10 comp rx=0 tx=0).stop 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 shutdown_connections 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e28066760 0x7f3e28068c20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e44164db0 0x7f3e4406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e4415ef80 0x7f3e44150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 --2- 172.21.15.179:0/2218326200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e4406de70 0x7f3e440c6d90 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 >> 172.21.15.179:0/2218326200 conn(0x7f3e440ce9d0 msgr2=0x7f3e44160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 shutdown_connections 2024-05-08T22:40:01.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.809+0000 7f3e4ad43700 1 -- 172.21.15.179:0/2218326200 wait complete. 2024-05-08T22:40:01.826 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:40:01.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 Processor -- start 2024-05-08T22:40:01.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 -- start start 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 0x7fb5080cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5080cf590 0x7fb5080b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5080b11f0 con 0x7fb50815d3b0 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080cf9e0 con 0x7fb5080cf590 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb50c997700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080cd9b0 con 0x7fb5080cd0f0 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb506d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5080cf590 0x7fb5080b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb506d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5080cf590 0x7fb5080b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59038/0 (socket says 172.21.15.179:59038) 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb506d9d700 1 -- 172.21.15.179:0/4099558115 learned_addr learned my addr 172.21.15.179:0/4099558115 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.941+0000 7fb505d9b700 1 --2- 172.21.15.179:0/4099558115 >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 0x7fb5080cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb506d9d700 1 -- 172.21.15.179:0/4099558115 >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 msgr2=0x7fb5080cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb506d9d700 1 --2- 172.21.15.179:0/4099558115 >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 0x7fb5080cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb506d9d700 1 -- 172.21.15.179:0/4099558115 >> v1:172.21.15.195:6789/0 conn(0x7fb50815d3b0 legacy=0x7fb5080cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb506d9d700 1 -- 172.21.15.179:0/4099558115 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080cd6d0 con 0x7fb5080cf590 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb505d9b700 1 --2- 172.21.15.179:0/4099558115 >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 0x7fb5080cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb506d9d700 1 --2- 172.21.15.179:0/4099558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5080cf590 0x7fb5080b1680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb5080cceb0 tx=0x7fb50000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7a5a4d75979b18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50559a700 1 -- 172.21.15.179:0/4099558115 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000e070 con 0x7fb5080cf590 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50559a700 1 -- 172.21.15.179:0/4099558115 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50000b9c0 con 0x7fb5080cf590 2024-05-08T22:40:01.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50559a700 1 -- 172.21.15.179:0/4099558115 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000bd70 con 0x7fb5080cf590 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 -- 172.21.15.179:0/4099558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5080cf590 msgr2=0x7fb5080b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 --2- 172.21.15.179:0/4099558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5080cf590 0x7fb5080b1680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb5080cceb0 tx=0x7fb50000b330 comp rx=0 tx=0).stop 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 -- 172.21.15.179:0/4099558115 shutdown_connections 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 --2- 172.21.15.179:0/4099558115 >> v2:172.21.15.195:3300/0 conn(0x7fb5080cd0f0 0x7fb5080cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 --2- 172.21.15.179:0/4099558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5080cf590 0x7fb5080b1680 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 -- 172.21.15.179:0/4099558115 >> 172.21.15.179:0/4099558115 conn(0x7fb5080d1180 msgr2=0x7fb5080d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 -- 172.21.15.179:0/4099558115 shutdown_connections 2024-05-08T22:40:01.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.945+0000 7fb50c997700 1 -- 172.21.15.179:0/4099558115 wait complete. 2024-05-08T22:40:01.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 Processor -- start 2024-05-08T22:40:01.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 -- start start 2024-05-08T22:40:01.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 0x7fb508148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5080cf590 0x7fb508157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50815a6a0 con 0x7fb50815d3b0 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50815a980 con 0x7fb5080cd0f0 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50c997700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50815ac60 con 0x7fb5080cf590 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb506d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5080cf590 0x7fb508157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb506d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59054/0 (socket says 172.21.15.179:59054) 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb50659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 0x7fb508148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.949+0000 7fb506d9d700 1 -- 172.21.15.179:0/1457605219 learned_addr learned my addr 172.21.15.179:0/1457605219 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 -- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5080cf590 msgr2=0x7fb508157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5080cf590 0x7fb508157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 -- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 msgr2=0x7fb508148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 0x7fb508148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 -- 172.21.15.179:0/1457605219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500009b30 con 0x7fb50815d3b0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb50659c700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 0x7fb508148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb506d9d700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb50808a490 tx=0x7fb500004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500004880 con 0x7fb50815d3b0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5000099b0 con 0x7fb50815d3b0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50815af40 con 0x7fb50815d3b0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50001c680 con 0x7fb50815d3b0 2024-05-08T22:40:01.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081f6cc0 con 0x7fb50815d3b0 2024-05-08T22:40:01.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080ce980 con 0x7fb50815d3b0 2024-05-08T22:40:01.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb50001c7e0 con 0x7fb50815d3b0 2024-05-08T22:40:01.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 0x7fb4e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:01.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb500085a40 con 0x7fb50815d3b0 2024-05-08T22:40:01.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.953+0000 7fb50659c700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 0x7fb4e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:01.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.957+0000 7fb50659c700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 0x7fb4e8068c20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc000f60 tx=0x7fb4fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:01.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:01.965+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb500087e60 con 0x7fb50815d3b0 2024-05-08T22:40:02.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:02.173+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb50815a7e0 con 0x7fb50815d3b0 2024-05-08T22:40:03.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.157+0000 7fb4f6ffd700 1 -- 172.21.15.179:0/1457605219 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb50815a7e0 con 0x7fb50815d3b0 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 msgr2=0x7fb4e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 0x7fb4e8068c20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc000f60 tx=0x7fb4fc009040 comp rx=0 tx=0).stop 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 msgr2=0x7fb5081f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb50808a490 tx=0x7fb500004af0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 shutdown_connections 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb4e8066760 0x7fb4e8068c20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50815d3b0 0x7fb5081f6780 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5080cf590 0x7fb508157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 --2- 172.21.15.179:0/1457605219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5080cd0f0 0x7fb508148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 >> 172.21.15.179:0/1457605219 conn(0x7fb5080d1180 msgr2=0x7fb50806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:03.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 shutdown_connections 2024-05-08T22:40:03.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.161+0000 7fb50c997700 1 -- 172.21.15.179:0/1457605219 wait complete. 2024-05-08T22:40:03.181 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:40:03.181 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:40:03.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 Processor -- start 2024-05-08T22:40:03.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 -- start start 2024-05-08T22:40:03.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd4d40b0210 0x7fd4d40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:03.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40b5340 con 0x7fd4d40b4170 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40b4730 con 0x7fd4d40ae720 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4db927700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40b4a10 con 0x7fd4d40b0210 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d9ec4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd4d40b4170 0x7fd4d406ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47944/0 (socket says 172.21.15.179:47944) 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d9ec4700 1 -- 172.21.15.179:0/1692560610 learned_addr learned my addr 172.21.15.179:0/1692560610 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d8ec2700 1 --2- 172.21.15.179:0/1692560610 >> v2:172.21.15.179:3300/0 conn(0x7fd4d40b0210 0x7fd4d40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229724742 0 0) 0x7fd4d40b5340 con 0x7fd4d40b4170 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c00035d0 con 0x7fd4d40b4170 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/1692560610 >> v1:172.21.15.195:6789/0 conn(0x7fd4d40ae720 legacy=0x7fd4d40afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179089255 0 0) 0x7fd4d40b4730 con 0x7fd4d40ae720 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/1692560610 >> v1:172.21.15.179:6789/0 conn(0x7fd4d40b4170 legacy=0x7fd4d406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/1692560610 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40b4cf0 con 0x7fd4d40b0210 2024-05-08T22:40:03.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4d8ec2700 1 --2- 172.21.15.179:0/1692560610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d40aeb00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009850 tx=0x7fd4c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9de29d049a0418e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4011830 con 0x7fd4d40b0210 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4011990 con 0x7fd4d40b0210 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.301+0000 7fd4cbfff700 1 -- 172.21.15.179:0/1692560610 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4011d40 con 0x7fd4d40b0210 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- 172.21.15.179:0/1692560610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 msgr2=0x7fd4d40aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 --2- 172.21.15.179:0/1692560610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d40aeb00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009850 tx=0x7fd4c4009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- 172.21.15.179:0/1692560610 shutdown_connections 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 --2- 172.21.15.179:0/1692560610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d40aeb00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- 172.21.15.179:0/1692560610 >> 172.21.15.179:0/1692560610 conn(0x7fd4d4098a70 msgr2=0x7fd4d40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- 172.21.15.179:0/1692560610 shutdown_connections 2024-05-08T22:40:03.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- 172.21.15.179:0/1692560610 wait complete. 2024-05-08T22:40:03.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 Processor -- start 2024-05-08T22:40:03.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- start start 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d40ae720 0x7fd4d4159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415d2f0 con 0x7fd4d40b0210 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415bbc0 con 0x7fd4d40b4170 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4db927700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415d570 con 0x7fd4d40ae720 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d8ec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d9ec4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d96c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d40ae720 0x7fd4d4159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d9ec4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:39936/0 (socket says 172.21.15.179:39936) 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d9ec4700 1 -- 172.21.15.179:0/3740945373 learned_addr learned my addr 172.21.15.179:0/3740945373 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:03.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.305+0000 7fd4d8ec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59072/0 (socket says 172.21.15.179:59072) 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d40ae720 msgr2=0x7fd4d4159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d40ae720 0x7fd4d4159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 msgr2=0x7fd4d40c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 -- 172.21.15.179:0/3740945373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40094b0 con 0x7fd4d40b0210 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d9ec4700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4d8ec2700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4016000 tx=0x7fd4c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004550 con 0x7fd4d40b0210 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c401f070 con 0x7fd4d40b0210 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d415d850 con 0x7fd4d40b0210 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4003900 con 0x7fd4d40b0210 2024-05-08T22:40:03.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d415de10 con 0x7fd4d40b0210 2024-05-08T22:40:03.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.309+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4067770 con 0x7fd4d40b0210 2024-05-08T22:40:03.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.313+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4c4003a60 con 0x7fd4d40b0210 2024-05-08T22:40:03.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.313+0000 7fd4c9ffb700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 0x7fd4b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:03.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.313+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd4c4021070 con 0x7fd4d40b0210 2024-05-08T22:40:03.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.313+0000 7fd4d96c3700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 0x7fd4b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:03.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.313+0000 7fd4d96c3700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 0x7fd4b8068bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0000f60 tx=0x7fd4d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:03.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.317+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd4c4052c60 con 0x7fd4d40b0210 2024-05-08T22:40:03.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:03.529+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd4d4073290 con 0x7fd4d40b0210 2024-05-08T22:40:04.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.165+0000 7fd4c9ffb700 1 -- 172.21.15.179:0/3740945373 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4c40150c0 con 0x7fd4d40b0210 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 msgr2=0x7fd4b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 0x7fd4b8068bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0000f60 tx=0x7fd4d0009040 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 msgr2=0x7fd4d415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4016000 tx=0x7fd4c4004770 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 shutdown_connections 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd4b8066730 0x7fd4b8068bf0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4d40b4170 0x7fd4d40c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4d40b0210 0x7fd4d415b680 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 --2- 172.21.15.179:0/3740945373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4d40ae720 0x7fd4d4159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 >> 172.21.15.179:0/3740945373 conn(0x7fd4d4098a70 msgr2=0x7fd4d406ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 shutdown_connections 2024-05-08T22:40:04.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.169+0000 7fd4db927700 1 -- 172.21.15.179:0/3740945373 wait complete. 2024-05-08T22:40:04.191 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:40:04.191 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:04.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 Processor -- start 2024-05-08T22:40:04.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 -- start start 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f532806de70 0x7f5328161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5328164db0 0x7f532806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5328165d50 con 0x7f532815ef80 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532806ad70 con 0x7f532806de70 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5327fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5328165220 con 0x7f5328164db0 2024-05-08T22:40:04.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5326ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f532815ef80 0x7f532806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47954/0 (socket says 172.21.15.179:47954) 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5326ffd700 1 -- 172.21.15.179:0/2017429964 learned_addr learned my addr 172.21.15.179:0/2017429964 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53267fc700 1 --2- 172.21.15.179:0/2017429964 >> v2:172.21.15.179:3300/0 conn(0x7f532806de70 0x7f5328161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 --2- 172.21.15.179:0/2017429964 >> v2:172.21.15.195:3300/0 conn(0x7f5328164db0 0x7f532806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643028003 0 0) 0x7f5328165d50 con 0x7f532815ef80 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53140035d0 con 0x7f532815ef80 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265396661 0 0) 0x7f53140035d0 con 0x7f532815ef80 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 >> v2:172.21.15.195:3300/0 conn(0x7f5328164db0 msgr2=0x7f532806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 --2- 172.21.15.179:0/2017429964 >> v2:172.21.15.195:3300/0 conn(0x7f5328164db0 0x7f532806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 >> v2:172.21.15.179:3300/0 conn(0x7f532806de70 msgr2=0x7f5328161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 --2- 172.21.15.179:0/2017429964 >> v2:172.21.15.179:3300/0 conn(0x7f532806de70 0x7f5328161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532806e4b0 con 0x7f532815ef80 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5310001670 con 0x7f532815ef80 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53267fc700 1 --2- 172.21.15.179:0/2017429964 >> v2:172.21.15.179:3300/0 conn(0x7f532806de70 0x7f5328161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:04.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 >> v1:172.21.15.179:6789/0 conn(0x7f532815ef80 legacy=0x7f532806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314003a20 0x7f5314003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5314004870 0x7f5314004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 0x7f531400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532806ad70 con 0x7f5314005230 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328165220 con 0x7f5314003a20 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531400c070 con 0x7f5314004870 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 0x7f531400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53267fc700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5314004870 0x7f5314004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5326ffd700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314003a20 0x7f5314003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 -- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5314004870 msgr2=0x7f5314004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5314004870 0x7f5314004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 -- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314003a20 msgr2=0x7f5314003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314003a20 0x7f5314003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 -- 172.21.15.179:0/2017429964 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140035d0 con 0x7f5314005230 2024-05-08T22:40:04.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 -- 172.21.15.179:0/2017429964 reap_dead start 2024-05-08T22:40:04.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f53277fe700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 0x7f531400ba30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5318009b70 tx=0x7f53140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f360d9eae99b539 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.305+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318004020 con 0x7f5314005230 2024-05-08T22:40:04.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318004180 con 0x7f5314005230 2024-05-08T22:40:04.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5325ffb700 1 -- 172.21.15.179:0/2017429964 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180044f0 con 0x7f5314005230 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 -- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 msgr2=0x7f531400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 0x7f531400ba30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5318009b70 tx=0x7f53140058a0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 -- 172.21.15.179:0/2017429964 shutdown_connections 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 --2- 172.21.15.179:0/2017429964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5314005230 0x7f531400ba30 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 -- 172.21.15.179:0/2017429964 >> 172.21.15.179:0/2017429964 conn(0x7f53280ce9d0 msgr2=0x7f53280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 -- 172.21.15.179:0/2017429964 shutdown_connections 2024-05-08T22:40:04.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.309+0000 7f5327fff700 1 -- 172.21.15.179:0/2017429964 wait complete. 2024-05-08T22:40:04.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 Processor -- start 2024-05-08T22:40:04.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 -- start start 2024-05-08T22:40:04.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314005230 0x7f5328159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 0x7f53280cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280c6f30 con 0x7f53280cd900 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280ca850 con 0x7f5314005230 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5327fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53280c46e0 con 0x7f53280c4b70 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59098/0 (socket says 172.21.15.179:59098) 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53267fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 0x7f53280cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 -- 172.21.15.179:0/3575339147 learned_addr learned my addr 172.21.15.179:0/3575339147 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f5326ffd700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314005230 0x7f5328159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 -- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 msgr2=0x7f53280cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 0x7f53280cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 -- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314005230 msgr2=0x7f5328159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314005230 0x7f5328159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 -- 172.21.15.179:0/3575339147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280c3ad0 con 0x7f53280cd900 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53267fc700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 0x7f53280cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:04.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f53277fe700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f53280b3eb0 tx=0x7f53180047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180038c0 con 0x7f53280cd900 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.313+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318003a40 con 0x7f53280cd900 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.317+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53280c3db0 con 0x7f53280cd900 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.317+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318003df0 con 0x7f53280cd900 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.317+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53280c4370 con 0x7f53280cd900 2024-05-08T22:40:04.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.317+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532804e700 con 0x7f53280cd900 2024-05-08T22:40:04.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.317+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5318013470 con 0x7f53280cd900 2024-05-08T22:40:04.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.321+0000 7f531ffff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 0x7f5308068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.321+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5318085730 con 0x7f53280cd900 2024-05-08T22:40:04.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.321+0000 7f5326ffd700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 0x7f5308068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.321+0000 7f5326ffd700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 0x7f5308068c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f53100079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.325+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5318052b10 con 0x7f53280cd900 2024-05-08T22:40:04.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.529+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53280c4820 con 0x7f53280cd900 2024-05-08T22:40:04.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f531ffff700 1 -- 172.21.15.179:0/3575339147 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f5318056130 con 0x7f53280cd900 2024-05-08T22:40:04.535 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:04.535 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:04.156894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/1693330449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1693330449},{"type":"v1","addr":"172.21.15.179:6835","nonce":1693330449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 msgr2=0x7f5308068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 0x7f5308068c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f53100079a0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 msgr2=0x7f53281604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f53280b3eb0 tx=0x7f53180047e0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 shutdown_connections 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5308066760 0x7f5308068c20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53280cd900 0x7f53281604f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53280c4b70 0x7f53280cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 --2- 172.21.15.179:0/3575339147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314005230 0x7f5328159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 >> 172.21.15.179:0/3575339147 conn(0x7f53280ce9d0 msgr2=0x7f532815e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 shutdown_connections 2024-05-08T22:40:04.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.533+0000 7f5327fff700 1 -- 172.21.15.179:0/3575339147 wait complete. 2024-05-08T22:40:04.539 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 55 2024-05-08T22:40:04.550 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:40:04.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 Processor -- start 2024-05-08T22:40:04.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- start start 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e74158b80 0x7f7e7415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c9230 0x7f7e740c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e74158f60 con 0x7f7e740c8d70 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7406b860 con 0x7f7e74158b80 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7406ac50 con 0x7f7e740c9230 2024-05-08T22:40:04.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e74158b80 0x7f7e7415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e74158b80 0x7f7e7415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59100/0 (socket says 172.21.15.179:59100) 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 -- 172.21.15.179:0/4265221234 learned_addr learned my addr 172.21.15.179:0/4265221234 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7ab53700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c9230 0x7f7e740c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 -- 172.21.15.179:0/4265221234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c9230 msgr2=0x7f7e740c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c9230 0x7f7e740c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 -- 172.21.15.179:0/4265221234 >> v1:172.21.15.195:6789/0 conn(0x7f7e740c8d70 legacy=0x7f7e7406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 -- 172.21.15.179:0/4265221234 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7406af30 con 0x7f7e74158b80 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7a352700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e7415b4d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009e50 tx=0x7f7e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a52b569cebfc66c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e79350700 1 -- 172.21.15.179:0/4265221234 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400e070 con 0x7f7e74158b80 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e79350700 1 -- 172.21.15.179:0/4265221234 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400b910 con 0x7f7e74158b80 2024-05-08T22:40:04.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e79350700 1 -- 172.21.15.179:0/4265221234 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400bcc0 con 0x7f7e74158b80 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/4265221234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 msgr2=0x7f7e7415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e7415b4d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009e50 tx=0x7f7e6400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/4265221234 shutdown_connections 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c9230 0x7f7e740c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/4265221234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e7415b4d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/4265221234 >> 172.21.15.179:0/4265221234 conn(0x7f7e740cb660 msgr2=0x7f7e740cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/4265221234 shutdown_connections 2024-05-08T22:40:04.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.665+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/4265221234 wait complete. 2024-05-08T22:40:04.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 Processor -- start 2024-05-08T22:40:04.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 -- start start 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c8d70 0x7f7e741f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e740c9230 0x7f7e740cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cd000 con 0x7f7e74158b80 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cd2e0 con 0x7f7e740c9230 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7c5b6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cd5c0 con 0x7f7e740c8d70 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59108/0 (socket says 172.21.15.179:59108) 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 -- 172.21.15.179:0/3088100105 learned_addr learned my addr 172.21.15.179:0/3088100105 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7a352700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c8d70 0x7f7e741f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e79b51700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e740c9230 0x7f7e740cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 -- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c8d70 msgr2=0x7f7e741f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c8d70 0x7f7e741f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 -- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e740c9230 msgr2=0x7f7e740cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e740c9230 0x7f7e740cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 -- 172.21.15.179:0/3088100105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64009b30 con 0x7f7e74158b80 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e7ab53700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00eb60 tx=0x7f7e6c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c009740 con 0x7f7e74158b80 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.669+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c009e00 con 0x7f7e74158b80 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e740cd8a0 con 0x7f7e74158b80 2024-05-08T22:40:04.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00ca70 con 0x7f7e74158b80 2024-05-08T22:40:04.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741f2ac0 con 0x7f7e74158b80 2024-05-08T22:40:04.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e74067770 con 0x7f7e74158b80 2024-05-08T22:40:04.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e6c00cbd0 con 0x7f7e74158b80 2024-05-08T22:40:04.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.673+0000 7f7e72ffd700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 0x7f7e60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.677+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7e6c084210 con 0x7f7e74158b80 2024-05-08T22:40:04.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.677+0000 7f7e7a352700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 0x7f7e60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.677+0000 7f7e7a352700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 0x7f7e60068bf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b370 tx=0x7f7e64009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.681+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e6c051440 con 0x7f7e74158b80 2024-05-08T22:40:04.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.857+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e740cddc0 con 0x7f7e74158b80 2024-05-08T22:40:04.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.857+0000 7f7e72ffd700 1 -- 172.21.15.179:0/3088100105 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16297 (secure 0 0 0) 0x7f7e6c054a60 con 0x7f7e74158b80 2024-05-08T22:40:04.862 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:04.862 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":96,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:02.144206+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 msgr2=0x7f7e60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 0x7f7e60068bf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b370 tx=0x7f7e64009000 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 msgr2=0x7f7e740ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00eb60 tx=0x7f7e6c00ee70 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 shutdown_connections 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e60066730 0x7f7e60068bf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e740c9230 0x7f7e740cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74158b80 0x7f7e740ce0a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 --2- 172.21.15.179:0/3088100105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e740c8d70 0x7f7e741f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 >> 172.21.15.179:0/3088100105 conn(0x7f7e740cb660 msgr2=0x7f7e74071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 shutdown_connections 2024-05-08T22:40:04.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.861+0000 7f7e7c5b6700 1 -- 172.21.15.179:0/3088100105 wait complete. 2024-05-08T22:40:04.877 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:40:04.878 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:40:04.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 Processor -- start 2024-05-08T22:40:04.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- start start 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f50e806de70 0x7f50e8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e8165d50 con 0x7f50e815ef80 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e806ad70 con 0x7f50e8164db0 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e8165220 con 0x7f50e806de70 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f50e806de70 0x7f50e8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:04.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50edec7700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f50e815ef80 0x7f50e806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:47970/0 (socket says 172.21.15.179:47970) 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f50e806de70 0x7f50e8161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59124/0 (socket says 172.21.15.179:59124) 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50edec7700 1 -- 172.21.15.179:0/2349057121 learned_addr learned my addr 172.21.15.179:0/2349057121 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 -- 172.21.15.179:0/2349057121 >> v1:172.21.15.195:6789/0 conn(0x7f50e8164db0 legacy=0x7f50e806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 -- 172.21.15.179:0/2349057121 >> v1:172.21.15.179:6789/0 conn(0x7f50e815ef80 legacy=0x7f50e806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ecec5700 1 -- 172.21.15.179:0/2349057121 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857524588 0 0) 0x7f50e8165d50 con 0x7f50e815ef80 2024-05-08T22:40:04.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 -- 172.21.15.179:0/2349057121 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e806e4b0 con 0x7f50e806de70 2024-05-08T22:40:04.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ed6c6700 1 --2- 172.21.15.179:0/2349057121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e806de70 0x7f50e8161790 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009850 tx=0x7f50e4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905a65f47c4ff5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:04.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ecec5700 1 -- 172.21.15.179:0/2349057121 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011830 con 0x7f50e806de70 2024-05-08T22:40:04.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ecec5700 1 -- 172.21.15.179:0/2349057121 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4011990 con 0x7f50e806de70 2024-05-08T22:40:04.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50ecec5700 1 -- 172.21.15.179:0/2349057121 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4011d20 con 0x7f50e806de70 2024-05-08T22:40:04.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- 172.21.15.179:0/2349057121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e806de70 msgr2=0x7f50e8161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 --2- 172.21.15.179:0/2349057121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e806de70 0x7f50e8161790 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009850 tx=0x7f50e4009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- 172.21.15.179:0/2349057121 shutdown_connections 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 --2- 172.21.15.179:0/2349057121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e806de70 0x7f50e8161790 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- 172.21.15.179:0/2349057121 >> 172.21.15.179:0/2349057121 conn(0x7f50e80ce9d0 msgr2=0x7f50e80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- 172.21.15.179:0/2349057121 shutdown_connections 2024-05-08T22:40:04.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.989+0000 7f50eeec9700 1 -- 172.21.15.179:0/2349057121 wait complete. 2024-05-08T22:40:05.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 Processor -- start 2024-05-08T22:40:05.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 -- start start 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 0x7f50e8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50e815ef80 0x7f50e8146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81fa000 con 0x7f50e8164db0 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81f6890 con 0x7f50e806de70 2024-05-08T22:40:05.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50eeec9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81fa250 con 0x7f50e815ef80 2024-05-08T22:40:05.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59130/0 (socket says 172.21.15.179:59130) 2024-05-08T22:40:05.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 -- 172.21.15.179:0/454598528 learned_addr learned my addr 172.21.15.179:0/454598528 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:05.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ed6c6700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50e815ef80 0x7f50e8146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50edec7700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 0x7f50e8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 -- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50e815ef80 msgr2=0x7f50e8146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50e815ef80 0x7f50e8146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 -- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 msgr2=0x7f50e8154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 0x7f50e8154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 -- 172.21.15.179:0/454598528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40094b0 con 0x7f50e8164db0 2024-05-08T22:40:05.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50edec7700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 0x7f50e8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.993+0000 7f50ee6c8700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f50e000cb50 tx=0x7f50e000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003c20 con 0x7f50e8164db0 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0003d80 con 0x7f50e8164db0 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e81fa530 con 0x7f50e8164db0 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000aac0 con 0x7f50e8164db0 2024-05-08T22:40:05.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e81faaf0 con 0x7f50e8164db0 2024-05-08T22:40:05.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e8067740 con 0x7f50e8164db0 2024-05-08T22:40:05.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50e000ac20 con 0x7f50e8164db0 2024-05-08T22:40:05.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 0x7f50cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f50e0084250 con 0x7f50e8164db0 2024-05-08T22:40:05.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:04.997+0000 7f50edec7700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 0x7f50cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.001+0000 7f50edec7700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 0x7f50cc068bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000f60 tx=0x7f50d8007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.009+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f50e0051480 con 0x7f50e8164db0 2024-05-08T22:40:05.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.177+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50e8165a70 con 0x7f50cc066730 2024-05-08T22:40:05.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.189+0000 7f50de7fc700 1 -- 172.21.15.179:0/454598528 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284709 (secure 0 0 0) 0x7f50d8005bb0 con 0x7f50cc066730 2024-05-08T22:40:05.199 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 msgr2=0x7f50cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 0x7f50cc068bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000f60 tx=0x7f50d8007d00 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 msgr2=0x7f50e8143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f50e000cb50 tx=0x7f50e000ce60 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 shutdown_connections 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f50cc066730 0x7f50cc068bf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8164db0 0x7f50e8143550 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50e815ef80 0x7f50e8146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 --2- 172.21.15.179:0/454598528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50e806de70 0x7f50e8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 >> 172.21.15.179:0/454598528 conn(0x7f50e80ce9d0 msgr2=0x7f50e80cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:05.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 shutdown_connections 2024-05-08T22:40:05.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.197+0000 7f50eeec9700 1 -- 172.21.15.179:0/454598528 wait complete. 2024-05-08T22:40:05.203 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:40:05.215 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-05-08T22:40:03.523173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":191,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":263344,"kb_used_data":10308,"kb_used_omap":12,"kb_used_meta":252915,"kb_avail":754711376,"statfs":{"total":773094113280,"available":772824449024,"internally_reserved":0,"allocated":10555392,"data_stored":5424872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":258985528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001140"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776788+0000","last_change":"2024-05-08T22:40:01.165659+0000","last_active":"2024-05-08T22:40:02.776788+0000","last_peered":"2024-05-08T22:40:02.776788+0000","last_clean":"2024-05-08T22:40:02.776788+0000","last_became_active":"2024-05-08T22:40:01.165368+0000","last_became_peered":"2024-05-08T22:40:01.165368+0000","last_unstale":"2024-05-08T22:40:02.776788+0000","last_undegraded":"2024-05-08T22:40:02.776788+0000","last_fullsized":"2024-05-08T22:40:02.776788+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:39.746907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158010+0000","last_change":"2024-05-08T22:40:01.157440+0000","last_active":"2024-05-08T22:40:02.158010+0000","last_peered":"2024-05-08T22:40:02.158010+0000","last_clean":"2024-05-08T22:40:02.158010+0000","last_became_active":"2024-05-08T22:40:01.156861+0000","last_became_peered":"2024-05-08T22:40:01.156861+0000","last_unstale":"2024-05-08T22:40:02.158010+0000","last_undegraded":"2024-05-08T22:40:02.158010+0000","last_fullsized":"2024-05-08T22:40:02.158010+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:01.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777657+0000","last_change":"2024-05-08T22:40:01.176314+0000","last_active":"2024-05-08T22:40:02.777657+0000","last_peered":"2024-05-08T22:40:02.777657+0000","last_clean":"2024-05-08T22:40:02.777657+0000","last_became_active":"2024-05-08T22:40:01.170247+0000","last_became_peered":"2024-05-08T22:40:01.170247+0000","last_unstale":"2024-05-08T22:40:02.777657+0000","last_undegraded":"2024-05-08T22:40:02.777657+0000","last_fullsized":"2024-05-08T22:40:02.777657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:54.552531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778046+0000","last_change":"2024-05-08T22:40:01.160763+0000","last_active":"2024-05-08T22:40:02.778046+0000","last_peered":"2024-05-08T22:40:02.778046+0000","last_clean":"2024-05-08T22:40:02.778046+0000","last_became_active":"2024-05-08T22:40:01.160411+0000","last_became_peered":"2024-05-08T22:40:01.160411+0000","last_unstale":"2024-05-08T22:40:02.778046+0000","last_undegraded":"2024-05-08T22:40:02.778046+0000","last_fullsized":"2024-05-08T22:40:02.778046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:50.548005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159500+0000","last_change":"2024-05-08T22:40:01.170682+0000","last_active":"2024-05-08T22:40:02.159500+0000","last_peered":"2024-05-08T22:40:02.159500+0000","last_clean":"2024-05-08T22:40:02.159500+0000","last_became_active":"2024-05-08T22:40:01.170408+0000","last_became_peered":"2024-05-08T22:40:01.170408+0000","last_unstale":"2024-05-08T22:40:02.159500+0000","last_undegraded":"2024-05-08T22:40:02.159500+0000","last_fullsized":"2024-05-08T22:40:02.159500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:15.318176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158711+0000","last_change":"2024-05-08T22:40:01.170718+0000","last_active":"2024-05-08T22:40:02.158711+0000","last_peered":"2024-05-08T22:40:02.158711+0000","last_clean":"2024-05-08T22:40:02.158711+0000","last_became_active":"2024-05-08T22:40:01.169054+0000","last_became_peered":"2024-05-08T22:40:01.169054+0000","last_unstale":"2024-05-08T22:40:02.158711+0000","last_undegraded":"2024-05-08T22:40:02.158711+0000","last_fullsized":"2024-05-08T22:40:02.158711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:14.931477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158629+0000","last_change":"2024-05-08T22:40:01.179236+0000","last_active":"2024-05-08T22:40:02.158629+0000","last_peered":"2024-05-08T22:40:02.158629+0000","last_clean":"2024-05-08T22:40:02.158629+0000","last_became_active":"2024-05-08T22:40:01.179069+0000","last_became_peered":"2024-05-08T22:40:01.179069+0000","last_unstale":"2024-05-08T22:40:02.158629+0000","last_undegraded":"2024-05-08T22:40:02.158629+0000","last_fullsized":"2024-05-08T22:40:02.158629+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:22.920302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158389+0000","last_change":"2024-05-08T22:40:01.168157+0000","last_active":"2024-05-08T22:40:02.158389+0000","last_peered":"2024-05-08T22:40:02.158389+0000","last_clean":"2024-05-08T22:40:02.158389+0000","last_became_active":"2024-05-08T22:40:01.167883+0000","last_became_peered":"2024-05-08T22:40:01.167883+0000","last_unstale":"2024-05-08T22:40:02.158389+0000","last_undegraded":"2024-05-08T22:40:02.158389+0000","last_fullsized":"2024-05-08T22:40:02.158389+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:02.269420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157596+0000","last_change":"2024-05-08T22:40:01.166264+0000","last_active":"2024-05-08T22:40:02.157596+0000","last_peered":"2024-05-08T22:40:02.157596+0000","last_clean":"2024-05-08T22:40:02.157596+0000","last_became_active":"2024-05-08T22:40:01.165967+0000","last_became_peered":"2024-05-08T22:40:01.165967+0000","last_unstale":"2024-05-08T22:40:02.157596+0000","last_undegraded":"2024-05-08T22:40:02.157596+0000","last_fullsized":"2024-05-08T22:40:02.157596+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:45.929756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777378+0000","last_change":"2024-05-08T22:40:01.176520+0000","last_active":"2024-05-08T22:40:02.777378+0000","last_peered":"2024-05-08T22:40:02.777378+0000","last_clean":"2024-05-08T22:40:02.777378+0000","last_became_active":"2024-05-08T22:40:01.174004+0000","last_became_peered":"2024-05-08T22:40:01.174004+0000","last_unstale":"2024-05-08T22:40:02.777378+0000","last_undegraded":"2024-05-08T22:40:02.777378+0000","last_fullsized":"2024-05-08T22:40:02.777378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:30.934567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778358+0000","last_change":"2024-05-08T22:40:01.174406+0000","last_active":"2024-05-08T22:40:02.778358+0000","last_peered":"2024-05-08T22:40:02.778358+0000","last_clean":"2024-05-08T22:40:02.778358+0000","last_became_active":"2024-05-08T22:40:01.174147+0000","last_became_peered":"2024-05-08T22:40:01.174147+0000","last_unstale":"2024-05-08T22:40:02.778358+0000","last_undegraded":"2024-05-08T22:40:02.778358+0000","last_fullsized":"2024-05-08T22:40:02.778358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:39.165962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158768+0000","last_change":"2024-05-08T22:40:01.165738+0000","last_active":"2024-05-08T22:40:02.158768+0000","last_peered":"2024-05-08T22:40:02.158768+0000","last_clean":"2024-05-08T22:40:02.158768+0000","last_became_active":"2024-05-08T22:40:01.165475+0000","last_became_peered":"2024-05-08T22:40:01.165475+0000","last_unstale":"2024-05-08T22:40:02.158768+0000","last_undegraded":"2024-05-08T22:40:02.158768+0000","last_fullsized":"2024-05-08T22:40:02.158768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:33.091502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157869+0000","last_change":"2024-05-08T22:40:01.158150+0000","last_active":"2024-05-08T22:40:02.157869+0000","last_peered":"2024-05-08T22:40:02.157869+0000","last_clean":"2024-05-08T22:40:02.157869+0000","last_became_active":"2024-05-08T22:40:01.157169+0000","last_became_peered":"2024-05-08T22:40:01.157169+0000","last_unstale":"2024-05-08T22:40:02.157869+0000","last_undegraded":"2024-05-08T22:40:02.157869+0000","last_fullsized":"2024-05-08T22:40:02.157869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:54.095539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157585+0000","last_change":"2024-05-08T22:40:01.159819+0000","last_active":"2024-05-08T22:40:02.157585+0000","last_peered":"2024-05-08T22:40:02.157585+0000","last_clean":"2024-05-08T22:40:02.157585+0000","last_became_active":"2024-05-08T22:40:01.159558+0000","last_became_peered":"2024-05-08T22:40:01.159558+0000","last_unstale":"2024-05-08T22:40:02.157585+0000","last_undegraded":"2024-05-08T22:40:02.157585+0000","last_fullsized":"2024-05-08T22:40:02.157585+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:39.038788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157876+0000","last_change":"2024-05-08T22:40:01.161934+0000","last_active":"2024-05-08T22:40:02.157876+0000","last_peered":"2024-05-08T22:40:02.157876+0000","last_clean":"2024-05-08T22:40:02.157876+0000","last_became_active":"2024-05-08T22:40:01.159856+0000","last_became_peered":"2024-05-08T22:40:01.159856+0000","last_unstale":"2024-05-08T22:40:02.157876+0000","last_undegraded":"2024-05-08T22:40:02.157876+0000","last_fullsized":"2024-05-08T22:40:02.157876+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.627596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777322+0000","last_change":"2024-05-08T22:40:01.169075+0000","last_active":"2024-05-08T22:40:02.777322+0000","last_peered":"2024-05-08T22:40:02.777322+0000","last_clean":"2024-05-08T22:40:02.777322+0000","last_became_active":"2024-05-08T22:40:01.168810+0000","last_became_peered":"2024-05-08T22:40:01.168810+0000","last_unstale":"2024-05-08T22:40:02.777322+0000","last_undegraded":"2024-05-08T22:40:02.777322+0000","last_fullsized":"2024-05-08T22:40:02.777322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:49.903039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159206+0000","last_change":"2024-05-08T22:40:01.165423+0000","last_active":"2024-05-08T22:40:02.159206+0000","last_peered":"2024-05-08T22:40:02.159206+0000","last_clean":"2024-05-08T22:40:02.159206+0000","last_became_active":"2024-05-08T22:40:01.165160+0000","last_became_peered":"2024-05-08T22:40:01.165160+0000","last_unstale":"2024-05-08T22:40:02.159206+0000","last_undegraded":"2024-05-08T22:40:02.159206+0000","last_fullsized":"2024-05-08T22:40:02.159206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:43.911356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158540+0000","last_change":"2024-05-08T22:40:01.162310+0000","last_active":"2024-05-08T22:40:02.158540+0000","last_peered":"2024-05-08T22:40:02.158540+0000","last_clean":"2024-05-08T22:40:02.158540+0000","last_became_active":"2024-05-08T22:40:01.162077+0000","last_became_peered":"2024-05-08T22:40:01.162077+0000","last_unstale":"2024-05-08T22:40:02.158540+0000","last_undegraded":"2024-05-08T22:40:02.158540+0000","last_fullsized":"2024-05-08T22:40:02.158540+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:19.151595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778350+0000","last_change":"2024-05-08T22:40:01.162156+0000","last_active":"2024-05-08T22:40:02.778350+0000","last_peered":"2024-05-08T22:40:02.778350+0000","last_clean":"2024-05-08T22:40:02.778350+0000","last_became_active":"2024-05-08T22:40:01.161909+0000","last_became_peered":"2024-05-08T22:40:01.161909+0000","last_unstale":"2024-05-08T22:40:02.778350+0000","last_undegraded":"2024-05-08T22:40:02.778350+0000","last_fullsized":"2024-05-08T22:40:02.778350+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:51.515199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157676+0000","last_change":"2024-05-08T22:40:01.170095+0000","last_active":"2024-05-08T22:40:02.157676+0000","last_peered":"2024-05-08T22:40:02.157676+0000","last_clean":"2024-05-08T22:40:02.157676+0000","last_became_active":"2024-05-08T22:40:01.169821+0000","last_became_peered":"2024-05-08T22:40:01.169821+0000","last_unstale":"2024-05-08T22:40:02.157676+0000","last_undegraded":"2024-05-08T22:40:02.157676+0000","last_fullsized":"2024-05-08T22:40:02.157676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:07.808574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.775945+0000","last_change":"2024-05-08T22:40:01.167498+0000","last_active":"2024-05-08T22:40:02.775945+0000","last_peered":"2024-05-08T22:40:02.775945+0000","last_clean":"2024-05-08T22:40:02.775945+0000","last_became_active":"2024-05-08T22:40:01.167206+0000","last_became_peered":"2024-05-08T22:40:01.167206+0000","last_unstale":"2024-05-08T22:40:02.775945+0000","last_undegraded":"2024-05-08T22:40:02.775945+0000","last_fullsized":"2024-05-08T22:40:02.775945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:16.869390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777430+0000","last_change":"2024-05-08T22:40:01.171406+0000","last_active":"2024-05-08T22:40:02.777430+0000","last_peered":"2024-05-08T22:40:02.777430+0000","last_clean":"2024-05-08T22:40:02.777430+0000","last_became_active":"2024-05-08T22:40:01.170943+0000","last_became_peered":"2024-05-08T22:40:01.170943+0000","last_unstale":"2024-05-08T22:40:02.777430+0000","last_undegraded":"2024-05-08T22:40:02.777430+0000","last_fullsized":"2024-05-08T22:40:02.777430+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:15.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777843+0000","last_change":"2024-05-08T22:40:01.176643+0000","last_active":"2024-05-08T22:40:02.777843+0000","last_peered":"2024-05-08T22:40:02.777843+0000","last_clean":"2024-05-08T22:40:02.777843+0000","last_became_active":"2024-05-08T22:40:01.176371+0000","last_became_peered":"2024-05-08T22:40:01.176371+0000","last_unstale":"2024-05-08T22:40:02.777843+0000","last_undegraded":"2024-05-08T22:40:02.777843+0000","last_fullsized":"2024-05-08T22:40:02.777843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:25.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"95'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158637+0000","last_change":"2024-05-08T22:40:00.774315+0000","last_active":"2024-05-08T22:40:02.158637+0000","last_peered":"2024-05-08T22:40:02.158637+0000","last_clean":"2024-05-08T22:40:02.158637+0000","last_became_active":"2024-05-08T22:40:00.774083+0000","last_became_peered":"2024-05-08T22:40:00.774083+0000","last_unstale":"2024-05-08T22:40:02.158637+0000","last_undegraded":"2024-05-08T22:40:02.158637+0000","last_fullsized":"2024-05-08T22:40:02.158637+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:27.360721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777922+0000","last_change":"2024-05-08T22:40:01.178935+0000","last_active":"2024-05-08T22:40:02.777922+0000","last_peered":"2024-05-08T22:40:02.777922+0000","last_clean":"2024-05-08T22:40:02.777922+0000","last_became_active":"2024-05-08T22:40:01.177949+0000","last_became_peered":"2024-05-08T22:40:01.177949+0000","last_unstale":"2024-05-08T22:40:02.777922+0000","last_undegraded":"2024-05-08T22:40:02.777922+0000","last_fullsized":"2024-05-08T22:40:02.777922+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:07.587283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159395+0000","last_change":"2024-05-08T22:40:00.771757+0000","last_active":"2024-05-08T22:40:02.159395+0000","last_peered":"2024-05-08T22:40:02.159395+0000","last_clean":"2024-05-08T22:40:02.159395+0000","last_became_active":"2024-05-08T22:40:00.771526+0000","last_became_peered":"2024-05-08T22:40:00.771526+0000","last_unstale":"2024-05-08T22:40:02.159395+0000","last_undegraded":"2024-05-08T22:40:02.159395+0000","last_fullsized":"2024-05-08T22:40:02.159395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:15.689463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778570+0000","last_change":"2024-05-08T22:40:01.160428+0000","last_active":"2024-05-08T22:40:02.778570+0000","last_peered":"2024-05-08T22:40:02.778570+0000","last_clean":"2024-05-08T22:40:02.778570+0000","last_became_active":"2024-05-08T22:40:01.160002+0000","last_became_peered":"2024-05-08T22:40:01.160002+0000","last_unstale":"2024-05-08T22:40:02.778570+0000","last_undegraded":"2024-05-08T22:40:02.778570+0000","last_fullsized":"2024-05-08T22:40:02.778570+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:01.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778433+0000","last_change":"2024-05-08T22:40:00.776580+0000","last_active":"2024-05-08T22:40:02.778433+0000","last_peered":"2024-05-08T22:40:02.778433+0000","last_clean":"2024-05-08T22:40:02.778433+0000","last_became_active":"2024-05-08T22:40:00.776269+0000","last_became_peered":"2024-05-08T22:40:00.776269+0000","last_unstale":"2024-05-08T22:40:02.778433+0000","last_undegraded":"2024-05-08T22:40:02.778433+0000","last_fullsized":"2024-05-08T22:40:02.778433+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:54.249895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778439+0000","last_change":"2024-05-08T22:40:01.174367+0000","last_active":"2024-05-08T22:40:02.778439+0000","last_peered":"2024-05-08T22:40:02.778439+0000","last_clean":"2024-05-08T22:40:02.778439+0000","last_became_active":"2024-05-08T22:40:01.173707+0000","last_became_peered":"2024-05-08T22:40:01.173707+0000","last_unstale":"2024-05-08T22:40:02.778439+0000","last_undegraded":"2024-05-08T22:40:02.778439+0000","last_fullsized":"2024-05-08T22:40:02.778439+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:44.349613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158782+0000","last_change":"2024-05-08T22:40:00.778514+0000","last_active":"2024-05-08T22:40:02.158782+0000","last_peered":"2024-05-08T22:40:02.158782+0000","last_clean":"2024-05-08T22:40:02.158782+0000","last_became_active":"2024-05-08T22:40:00.778232+0000","last_became_peered":"2024-05-08T22:40:00.778232+0000","last_unstale":"2024-05-08T22:40:02.158782+0000","last_undegraded":"2024-05-08T22:40:02.158782+0000","last_fullsized":"2024-05-08T22:40:02.158782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:33.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158023+0000","last_change":"2024-05-08T22:40:00.773224+0000","last_active":"2024-05-08T22:40:02.158023+0000","last_peered":"2024-05-08T22:40:02.158023+0000","last_clean":"2024-05-08T22:40:02.158023+0000","last_became_active":"2024-05-08T22:40:00.772790+0000","last_became_peered":"2024-05-08T22:40:00.772790+0000","last_unstale":"2024-05-08T22:40:02.158023+0000","last_undegraded":"2024-05-08T22:40:02.158023+0000","last_fullsized":"2024-05-08T22:40:02.158023+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:45.105491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158103+0000","last_change":"2024-05-08T22:40:01.180089+0000","last_active":"2024-05-08T22:40:02.158103+0000","last_peered":"2024-05-08T22:40:02.158103+0000","last_clean":"2024-05-08T22:40:02.158103+0000","last_became_active":"2024-05-08T22:40:01.179809+0000","last_became_peered":"2024-05-08T22:40:01.179809+0000","last_unstale":"2024-05-08T22:40:02.158103+0000","last_undegraded":"2024-05-08T22:40:02.158103+0000","last_fullsized":"2024-05-08T22:40:02.158103+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:18.761217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777655+0000","last_change":"2024-05-08T22:40:01.174364+0000","last_active":"2024-05-08T22:40:02.777655+0000","last_peered":"2024-05-08T22:40:02.777655+0000","last_clean":"2024-05-08T22:40:02.777655+0000","last_became_active":"2024-05-08T22:40:01.166300+0000","last_became_peered":"2024-05-08T22:40:01.166300+0000","last_unstale":"2024-05-08T22:40:02.777655+0000","last_undegraded":"2024-05-08T22:40:02.777655+0000","last_fullsized":"2024-05-08T22:40:02.777655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.396021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777033+0000","last_change":"2024-05-08T22:40:00.787251+0000","last_active":"2024-05-08T22:40:02.777033+0000","last_peered":"2024-05-08T22:40:02.777033+0000","last_clean":"2024-05-08T22:40:02.777033+0000","last_became_active":"2024-05-08T22:40:00.787037+0000","last_became_peered":"2024-05-08T22:40:00.787037+0000","last_unstale":"2024-05-08T22:40:02.777033+0000","last_undegraded":"2024-05-08T22:40:02.777033+0000","last_fullsized":"2024-05-08T22:40:02.777033+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:48.270689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158147+0000","last_change":"2024-05-08T22:40:01.170124+0000","last_active":"2024-05-08T22:40:02.158147+0000","last_peered":"2024-05-08T22:40:02.158147+0000","last_clean":"2024-05-08T22:40:02.158147+0000","last_became_active":"2024-05-08T22:40:01.169895+0000","last_became_peered":"2024-05-08T22:40:01.169895+0000","last_unstale":"2024-05-08T22:40:02.158147+0000","last_undegraded":"2024-05-08T22:40:02.158147+0000","last_fullsized":"2024-05-08T22:40:02.158147+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:23.768473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777448+0000","last_change":"2024-05-08T22:40:00.771376+0000","last_active":"2024-05-08T22:40:02.777448+0000","last_peered":"2024-05-08T22:40:02.777448+0000","last_clean":"2024-05-08T22:40:02.777448+0000","last_became_active":"2024-05-08T22:40:00.771107+0000","last_became_peered":"2024-05-08T22:40:00.771107+0000","last_unstale":"2024-05-08T22:40:02.777448+0000","last_undegraded":"2024-05-08T22:40:02.777448+0000","last_fullsized":"2024-05-08T22:40:02.777448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:58.552604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778309+0000","last_change":"2024-05-08T22:40:01.160124+0000","last_active":"2024-05-08T22:40:02.778309+0000","last_peered":"2024-05-08T22:40:02.778309+0000","last_clean":"2024-05-08T22:40:02.778309+0000","last_became_active":"2024-05-08T22:40:01.159805+0000","last_became_peered":"2024-05-08T22:40:01.159805+0000","last_unstale":"2024-05-08T22:40:02.778309+0000","last_undegraded":"2024-05-08T22:40:02.778309+0000","last_fullsized":"2024-05-08T22:40:02.778309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:44.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"95'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159275+0000","last_change":"2024-05-08T22:40:00.775095+0000","last_active":"2024-05-08T22:40:02.159275+0000","last_peered":"2024-05-08T22:40:02.159275+0000","last_clean":"2024-05-08T22:40:02.159275+0000","last_became_active":"2024-05-08T22:40:00.774654+0000","last_became_peered":"2024-05-08T22:40:00.774654+0000","last_unstale":"2024-05-08T22:40:02.159275+0000","last_undegraded":"2024-05-08T22:40:02.159275+0000","last_fullsized":"2024-05-08T22:40:02.159275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:46.623988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777997+0000","last_change":"2024-05-08T22:40:00.781468+0000","last_active":"2024-05-08T22:40:02.777997+0000","last_peered":"2024-05-08T22:40:02.777997+0000","last_clean":"2024-05-08T22:40:02.777997+0000","last_became_active":"2024-05-08T22:40:00.781113+0000","last_became_peered":"2024-05-08T22:40:00.781113+0000","last_unstale":"2024-05-08T22:40:02.777997+0000","last_undegraded":"2024-05-08T22:40:02.777997+0000","last_fullsized":"2024-05-08T22:40:02.777997+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:06.805376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778410+0000","last_change":"2024-05-08T22:40:00.787240+0000","last_active":"2024-05-08T22:40:02.778410+0000","last_peered":"2024-05-08T22:40:02.778410+0000","last_clean":"2024-05-08T22:40:02.778410+0000","last_became_active":"2024-05-08T22:40:00.786988+0000","last_became_peered":"2024-05-08T22:40:00.786988+0000","last_unstale":"2024-05-08T22:40:02.778410+0000","last_undegraded":"2024-05-08T22:40:02.778410+0000","last_fullsized":"2024-05-08T22:40:02.778410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:01.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157449+0000","last_change":"2024-05-08T22:40:01.157968+0000","last_active":"2024-05-08T22:40:02.157449+0000","last_peered":"2024-05-08T22:40:02.157449+0000","last_clean":"2024-05-08T22:40:02.157449+0000","last_became_active":"2024-05-08T22:40:01.157643+0000","last_became_peered":"2024-05-08T22:40:01.157643+0000","last_unstale":"2024-05-08T22:40:02.157449+0000","last_undegraded":"2024-05-08T22:40:02.157449+0000","last_fullsized":"2024-05-08T22:40:02.157449+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:44.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777890+0000","last_change":"2024-05-08T22:40:00.769475+0000","last_active":"2024-05-08T22:40:02.777890+0000","last_peered":"2024-05-08T22:40:02.777890+0000","last_clean":"2024-05-08T22:40:02.777890+0000","last_became_active":"2024-05-08T22:40:00.769190+0000","last_became_peered":"2024-05-08T22:40:00.769190+0000","last_unstale":"2024-05-08T22:40:02.777890+0000","last_undegraded":"2024-05-08T22:40:02.777890+0000","last_fullsized":"2024-05-08T22:40:02.777890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:15.028535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158273+0000","last_change":"2024-05-08T22:40:01.167324+0000","last_active":"2024-05-08T22:40:02.158273+0000","last_peered":"2024-05-08T22:40:02.158273+0000","last_clean":"2024-05-08T22:40:02.158273+0000","last_became_active":"2024-05-08T22:40:01.167051+0000","last_became_peered":"2024-05-08T22:40:01.167051+0000","last_unstale":"2024-05-08T22:40:02.158273+0000","last_undegraded":"2024-05-08T22:40:02.158273+0000","last_fullsized":"2024-05-08T22:40:02.158273+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:08.376851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158174+0000","last_change":"2024-05-08T22:40:01.173627+0000","last_active":"2024-05-08T22:40:02.158174+0000","last_peered":"2024-05-08T22:40:02.158174+0000","last_clean":"2024-05-08T22:40:02.158174+0000","last_became_active":"2024-05-08T22:40:01.173347+0000","last_became_peered":"2024-05-08T22:40:01.173347+0000","last_unstale":"2024-05-08T22:40:02.158174+0000","last_undegraded":"2024-05-08T22:40:02.158174+0000","last_fullsized":"2024-05-08T22:40:02.158174+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:12.709608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777880+0000","last_change":"2024-05-08T22:40:01.181440+0000","last_active":"2024-05-08T22:40:02.777880+0000","last_peered":"2024-05-08T22:40:02.777880+0000","last_clean":"2024-05-08T22:40:02.777880+0000","last_became_active":"2024-05-08T22:40:01.181181+0000","last_became_peered":"2024-05-08T22:40:01.181181+0000","last_unstale":"2024-05-08T22:40:02.777880+0000","last_undegraded":"2024-05-08T22:40:02.777880+0000","last_fullsized":"2024-05-08T22:40:02.777880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:23.375670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158700+0000","last_change":"2024-05-08T22:40:00.769915+0000","last_active":"2024-05-08T22:40:02.158700+0000","last_peered":"2024-05-08T22:40:02.158700+0000","last_clean":"2024-05-08T22:40:02.158700+0000","last_became_active":"2024-05-08T22:40:00.769657+0000","last_became_peered":"2024-05-08T22:40:00.769657+0000","last_unstale":"2024-05-08T22:40:02.158700+0000","last_undegraded":"2024-05-08T22:40:02.158700+0000","last_fullsized":"2024-05-08T22:40:02.158700+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:49:10.561141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158920+0000","last_change":"2024-05-08T22:40:01.158895+0000","last_active":"2024-05-08T22:40:02.158920+0000","last_peered":"2024-05-08T22:40:02.158920+0000","last_clean":"2024-05-08T22:40:02.158920+0000","last_became_active":"2024-05-08T22:40:01.158575+0000","last_became_peered":"2024-05-08T22:40:01.158575+0000","last_unstale":"2024-05-08T22:40:02.158920+0000","last_undegraded":"2024-05-08T22:40:02.158920+0000","last_fullsized":"2024-05-08T22:40:02.158920+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:10.201272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778766+0000","last_change":"2024-05-08T22:40:00.772430+0000","last_active":"2024-05-08T22:40:02.778766+0000","last_peered":"2024-05-08T22:40:02.778766+0000","last_clean":"2024-05-08T22:40:02.778766+0000","last_became_active":"2024-05-08T22:40:00.772183+0000","last_became_peered":"2024-05-08T22:40:00.772183+0000","last_unstale":"2024-05-08T22:40:02.778766+0000","last_undegraded":"2024-05-08T22:40:02.778766+0000","last_fullsized":"2024-05-08T22:40:02.778766+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159328+0000","last_change":"2024-05-08T22:40:01.160519+0000","last_active":"2024-05-08T22:40:02.159328+0000","last_peered":"2024-05-08T22:40:02.159328+0000","last_clean":"2024-05-08T22:40:02.159328+0000","last_became_active":"2024-05-08T22:40:01.160271+0000","last_became_peered":"2024-05-08T22:40:01.160271+0000","last_unstale":"2024-05-08T22:40:02.159328+0000","last_undegraded":"2024-05-08T22:40:02.159328+0000","last_fullsized":"2024-05-08T22:40:02.159328+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:04.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159206+0000","last_change":"2024-05-08T22:40:00.782143+0000","last_active":"2024-05-08T22:40:02.159206+0000","last_peered":"2024-05-08T22:40:02.159206+0000","last_clean":"2024-05-08T22:40:02.159206+0000","last_became_active":"2024-05-08T22:40:00.781867+0000","last_became_peered":"2024-05-08T22:40:00.781867+0000","last_unstale":"2024-05-08T22:40:02.159206+0000","last_undegraded":"2024-05-08T22:40:02.159206+0000","last_fullsized":"2024-05-08T22:40:02.159206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777842+0000","last_change":"2024-05-08T22:40:01.171579+0000","last_active":"2024-05-08T22:40:02.777842+0000","last_peered":"2024-05-08T22:40:02.777842+0000","last_clean":"2024-05-08T22:40:02.777842+0000","last_became_active":"2024-05-08T22:40:01.170571+0000","last_became_peered":"2024-05-08T22:40:01.170571+0000","last_unstale":"2024-05-08T22:40:02.777842+0000","last_undegraded":"2024-05-08T22:40:02.777842+0000","last_fullsized":"2024-05-08T22:40:02.777842+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:24.889483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776286+0000","last_change":"2024-05-08T22:40:00.775780+0000","last_active":"2024-05-08T22:40:02.776286+0000","last_peered":"2024-05-08T22:40:02.776286+0000","last_clean":"2024-05-08T22:40:02.776286+0000","last_became_active":"2024-05-08T22:40:00.775350+0000","last_became_peered":"2024-05-08T22:40:00.775350+0000","last_unstale":"2024-05-08T22:40:02.776286+0000","last_undegraded":"2024-05-08T22:40:02.776286+0000","last_fullsized":"2024-05-08T22:40:02.776286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:32.117465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776225+0000","last_change":"2024-05-08T22:40:01.169014+0000","last_active":"2024-05-08T22:40:02.776225+0000","last_peered":"2024-05-08T22:40:02.776225+0000","last_clean":"2024-05-08T22:40:02.776225+0000","last_became_active":"2024-05-08T22:40:01.168647+0000","last_became_peered":"2024-05-08T22:40:01.168647+0000","last_unstale":"2024-05-08T22:40:02.776225+0000","last_undegraded":"2024-05-08T22:40:02.776225+0000","last_fullsized":"2024-05-08T22:40:02.776225+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:29.344124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158374+0000","last_change":"2024-05-08T22:40:00.784050+0000","last_active":"2024-05-08T22:40:02.158374+0000","last_peered":"2024-05-08T22:40:02.158374+0000","last_clean":"2024-05-08T22:40:02.158374+0000","last_became_active":"2024-05-08T22:40:00.783815+0000","last_became_peered":"2024-05-08T22:40:00.783815+0000","last_unstale":"2024-05-08T22:40:02.158374+0000","last_undegraded":"2024-05-08T22:40:02.158374+0000","last_fullsized":"2024-05-08T22:40:02.158374+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:59.988151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776025+0000","last_change":"2024-05-08T22:40:01.165354+0000","last_active":"2024-05-08T22:40:02.776025+0000","last_peered":"2024-05-08T22:40:02.776025+0000","last_clean":"2024-05-08T22:40:02.776025+0000","last_became_active":"2024-05-08T22:40:01.165075+0000","last_became_peered":"2024-05-08T22:40:01.165075+0000","last_unstale":"2024-05-08T22:40:02.776025+0000","last_undegraded":"2024-05-08T22:40:02.776025+0000","last_fullsized":"2024-05-08T22:40:02.776025+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:46:02.769323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776860+0000","last_change":"2024-05-08T22:40:00.774648+0000","last_active":"2024-05-08T22:40:02.776860+0000","last_peered":"2024-05-08T22:40:02.776860+0000","last_clean":"2024-05-08T22:40:02.776860+0000","last_became_active":"2024-05-08T22:40:00.774301+0000","last_became_peered":"2024-05-08T22:40:00.774301+0000","last_unstale":"2024-05-08T22:40:02.776860+0000","last_undegraded":"2024-05-08T22:40:02.776860+0000","last_fullsized":"2024-05-08T22:40:02.776860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:11.152944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777758+0000","last_change":"2024-05-08T22:40:01.173266+0000","last_active":"2024-05-08T22:40:02.777758+0000","last_peered":"2024-05-08T22:40:02.777758+0000","last_clean":"2024-05-08T22:40:02.777758+0000","last_became_active":"2024-05-08T22:40:01.173029+0000","last_became_peered":"2024-05-08T22:40:01.173029+0000","last_unstale":"2024-05-08T22:40:02.777758+0000","last_undegraded":"2024-05-08T22:40:02.777758+0000","last_fullsized":"2024-05-08T22:40:02.777758+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:05.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157944+0000","last_change":"2024-05-08T22:40:00.772913+0000","last_active":"2024-05-08T22:40:02.157944+0000","last_peered":"2024-05-08T22:40:02.157944+0000","last_clean":"2024-05-08T22:40:02.157944+0000","last_became_active":"2024-05-08T22:40:00.772489+0000","last_became_peered":"2024-05-08T22:40:00.772489+0000","last_unstale":"2024-05-08T22:40:02.157944+0000","last_undegraded":"2024-05-08T22:40:02.157944+0000","last_fullsized":"2024-05-08T22:40:02.157944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:43.832037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158647+0000","last_change":"2024-05-08T22:40:01.154094+0000","last_active":"2024-05-08T22:40:02.158647+0000","last_peered":"2024-05-08T22:40:02.158647+0000","last_clean":"2024-05-08T22:40:02.158647+0000","last_became_active":"2024-05-08T22:40:01.153749+0000","last_became_peered":"2024-05-08T22:40:01.153749+0000","last_unstale":"2024-05-08T22:40:02.158647+0000","last_undegraded":"2024-05-08T22:40:02.158647+0000","last_fullsized":"2024-05-08T22:40:02.158647+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:20.847730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779036+0000","last_change":"2024-05-08T22:40:00.771029+0000","last_active":"2024-05-08T22:40:02.779036+0000","last_peered":"2024-05-08T22:40:02.779036+0000","last_clean":"2024-05-08T22:40:02.779036+0000","last_became_active":"2024-05-08T22:40:00.770723+0000","last_became_peered":"2024-05-08T22:40:00.770723+0000","last_unstale":"2024-05-08T22:40:02.779036+0000","last_undegraded":"2024-05-08T22:40:02.779036+0000","last_fullsized":"2024-05-08T22:40:02.779036+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:46.623988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778178+0000","last_change":"2024-05-08T22:40:01.178824+0000","last_active":"2024-05-08T22:40:02.778178+0000","last_peered":"2024-05-08T22:40:02.778178+0000","last_clean":"2024-05-08T22:40:02.778178+0000","last_became_active":"2024-05-08T22:40:01.176113+0000","last_became_peered":"2024-05-08T22:40:01.176113+0000","last_unstale":"2024-05-08T22:40:02.778178+0000","last_undegraded":"2024-05-08T22:40:02.778178+0000","last_fullsized":"2024-05-08T22:40:02.778178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:14.063300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159423+0000","last_change":"2024-05-08T22:40:00.772227+0000","last_active":"2024-05-08T22:40:02.159423+0000","last_peered":"2024-05-08T22:40:02.159423+0000","last_clean":"2024-05-08T22:40:02.159423+0000","last_became_active":"2024-05-08T22:40:00.771988+0000","last_became_peered":"2024-05-08T22:40:00.771988+0000","last_unstale":"2024-05-08T22:40:02.159423+0000","last_undegraded":"2024-05-08T22:40:02.159423+0000","last_fullsized":"2024-05-08T22:40:02.159423+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.970913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778850+0000","last_change":"2024-05-08T22:40:01.172783+0000","last_active":"2024-05-08T22:40:02.778850+0000","last_peered":"2024-05-08T22:40:02.778850+0000","last_clean":"2024-05-08T22:40:02.778850+0000","last_became_active":"2024-05-08T22:40:01.172550+0000","last_became_peered":"2024-05-08T22:40:01.172550+0000","last_unstale":"2024-05-08T22:40:02.778850+0000","last_undegraded":"2024-05-08T22:40:02.778850+0000","last_fullsized":"2024-05-08T22:40:02.778850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:11.272797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159320+0000","last_change":"2024-05-08T22:40:00.774935+0000","last_active":"2024-05-08T22:40:02.159320+0000","last_peered":"2024-05-08T22:40:02.159320+0000","last_clean":"2024-05-08T22:40:02.159320+0000","last_became_active":"2024-05-08T22:40:00.774334+0000","last_became_peered":"2024-05-08T22:40:00.774334+0000","last_unstale":"2024-05-08T22:40:02.159320+0000","last_undegraded":"2024-05-08T22:40:02.159320+0000","last_fullsized":"2024-05-08T22:40:02.159320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:34.090155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777396+0000","last_change":"2024-05-08T22:40:01.168627+0000","last_active":"2024-05-08T22:40:02.777396+0000","last_peered":"2024-05-08T22:40:02.777396+0000","last_clean":"2024-05-08T22:40:02.777396+0000","last_became_active":"2024-05-08T22:40:01.167471+0000","last_became_peered":"2024-05-08T22:40:01.167471+0000","last_unstale":"2024-05-08T22:40:02.777396+0000","last_undegraded":"2024-05-08T22:40:02.777396+0000","last_fullsized":"2024-05-08T22:40:02.777396+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.902752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158850+0000","last_change":"2024-05-08T22:40:00.772765+0000","last_active":"2024-05-08T22:40:02.158850+0000","last_peered":"2024-05-08T22:40:02.158850+0000","last_clean":"2024-05-08T22:40:02.158850+0000","last_became_active":"2024-05-08T22:40:00.772512+0000","last_became_peered":"2024-05-08T22:40:00.772512+0000","last_unstale":"2024-05-08T22:40:02.158850+0000","last_undegraded":"2024-05-08T22:40:02.158850+0000","last_fullsized":"2024-05-08T22:40:02.158850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:24.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157892+0000","last_change":"2024-05-08T22:40:01.179700+0000","last_active":"2024-05-08T22:40:02.157892+0000","last_peered":"2024-05-08T22:40:02.157892+0000","last_clean":"2024-05-08T22:40:02.157892+0000","last_became_active":"2024-05-08T22:40:01.179428+0000","last_became_peered":"2024-05-08T22:40:01.179428+0000","last_unstale":"2024-05-08T22:40:02.157892+0000","last_undegraded":"2024-05-08T22:40:02.157892+0000","last_fullsized":"2024-05-08T22:40:02.157892+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:28.955691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777914+0000","last_change":"2024-05-08T22:40:00.778567+0000","last_active":"2024-05-08T22:40:02.777914+0000","last_peered":"2024-05-08T22:40:02.777914+0000","last_clean":"2024-05-08T22:40:02.777914+0000","last_became_active":"2024-05-08T22:40:00.778286+0000","last_became_peered":"2024-05-08T22:40:00.778286+0000","last_unstale":"2024-05-08T22:40:02.777914+0000","last_undegraded":"2024-05-08T22:40:02.777914+0000","last_fullsized":"2024-05-08T22:40:02.777914+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:00.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:40:00.750923+0000","last_change":"2024-05-08T22:40:00.750923+0000","last_active":"2024-05-08T22:40:00.750923+0000","last_peered":"2024-05-08T22:40:00.750923+0000","last_clean":"2024-05-08T22:40:00.750923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:40:00.750923+0000","last_undegraded":"2024-05-08T22:40:00.750923+0000","last_fullsized":"2024-05-08T22:40:00.750923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776268+0000","last_change":"2024-05-08T22:40:00.775741+0000","last_active":"2024-05-08T22:40:02.776268+0000","last_peered":"2024-05-08T22:40:02.776268+0000","last_clean":"2024-05-08T22:40:02.776268+0000","last_became_active":"2024-05-08T22:40:00.775321+0000","last_became_peered":"2024-05-08T22:40:00.775321+0000","last_unstale":"2024-05-08T22:40:02.776268+0000","last_undegraded":"2024-05-08T22:40:02.776268+0000","last_fullsized":"2024-05-08T22:40:02.776268+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:20.359593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158379+0000","last_change":"2024-05-08T22:40:01.160276+0000","last_active":"2024-05-08T22:40:02.158379+0000","last_peered":"2024-05-08T22:40:02.158379+0000","last_clean":"2024-05-08T22:40:02.158379+0000","last_became_active":"2024-05-08T22:40:01.159983+0000","last_became_peered":"2024-05-08T22:40:01.159983+0000","last_unstale":"2024-05-08T22:40:02.158379+0000","last_undegraded":"2024-05-08T22:40:02.158379+0000","last_fullsized":"2024-05-08T22:40:02.158379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:07.802805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158311+0000","last_change":"2024-05-08T22:40:00.772313+0000","last_active":"2024-05-08T22:40:02.158311+0000","last_peered":"2024-05-08T22:40:02.158311+0000","last_clean":"2024-05-08T22:40:02.158311+0000","last_became_active":"2024-05-08T22:40:00.772060+0000","last_became_peered":"2024-05-08T22:40:00.772060+0000","last_unstale":"2024-05-08T22:40:02.158311+0000","last_undegraded":"2024-05-08T22:40:02.158311+0000","last_fullsized":"2024-05-08T22:40:02.158311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:11.332444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777812+0000","last_change":"2024-05-08T22:40:01.171539+0000","last_active":"2024-05-08T22:40:02.777812+0000","last_peered":"2024-05-08T22:40:02.777812+0000","last_clean":"2024-05-08T22:40:02.777812+0000","last_became_active":"2024-05-08T22:40:01.170634+0000","last_became_peered":"2024-05-08T22:40:01.170634+0000","last_unstale":"2024-05-08T22:40:02.777812+0000","last_undegraded":"2024-05-08T22:40:02.777812+0000","last_fullsized":"2024-05-08T22:40:02.777812+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:55.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777731+0000","last_change":"2024-05-08T22:40:00.777821+0000","last_active":"2024-05-08T22:40:02.777731+0000","last_peered":"2024-05-08T22:40:02.777731+0000","last_clean":"2024-05-08T22:40:02.777731+0000","last_became_active":"2024-05-08T22:40:00.777599+0000","last_became_peered":"2024-05-08T22:40:00.777599+0000","last_unstale":"2024-05-08T22:40:02.777731+0000","last_undegraded":"2024-05-08T22:40:02.777731+0000","last_fullsized":"2024-05-08T22:40:02.777731+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:09.000232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779054+0000","last_change":"2024-05-08T22:40:01.162842+0000","last_active":"2024-05-08T22:40:02.779054+0000","last_peered":"2024-05-08T22:40:02.779054+0000","last_clean":"2024-05-08T22:40:02.779054+0000","last_became_active":"2024-05-08T22:40:01.162571+0000","last_became_peered":"2024-05-08T22:40:01.162571+0000","last_unstale":"2024-05-08T22:40:02.779054+0000","last_undegraded":"2024-05-08T22:40:02.779054+0000","last_fullsized":"2024-05-08T22:40:02.779054+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:16.031733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T22:39:59.747725+0000","last_change":"2024-05-08T22:39:59.747725+0000","last_active":"2024-05-08T22:39:59.747725+0000","last_peered":"2024-05-08T22:39:59.747725+0000","last_clean":"2024-05-08T22:39:59.747725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T22:39:59.747725+0000","last_undegraded":"2024-05-08T22:39:59.747725+0000","last_fullsized":"2024-05-08T22:39:59.747725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-08T22:40:01.147393+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:40:01.147393+0000","last_peered":"2024-05-08T22:40:01.147393+0000","last_clean":"2024-05-08T22:40:01.147393+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:40:01.147393+0000","last_undegraded":"2024-05-08T22:40:01.147393+0000","last_fullsized":"2024-05-08T22:40:01.147393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":52,"acting":52,"num_store_stats":6},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32220,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1126400,"data_stored":510100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.5620000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.50700000000000001,"5min":0.309,"15min":0.307},"max":{"1min":1.214,"5min":1.214,"15min":1.6899999999999999},"last":0.502}]},{"osd":1,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.35799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97699999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.75800000000000001,"15min":0.748},"min":{"1min":0.60699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1459999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0149999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.91500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2170000000000001,"5min":1.306,"15min":1.306},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95999999999999996,"15min":0.94599999999999995},"min":{"1min":0.77200000000000002,"5min":0.64600000000000002,"15min":0.64100000000000001},"max":{"1min":1.2809999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.98399999999999999,"15min":0.97199999999999998},"min":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.68300000000000005},"max":{"1min":1.3100000000000001,"5min":1.339,"15min":1.339},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.006,"15min":1.026},"min":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.0149999999999999}]},{"osd":6,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.86299999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.163,"5min":1.446,"15min":1.446},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.0600000000000001},"min":{"1min":0.83699999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3300000000000001,"5min":1.3480000000000001,"15min":1.3580000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.081,"15min":1.1020000000000001},"min":{"1min":0.755,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.552,"5min":1.6779999999999999,"15min":1.7070000000000001},"last":1.087},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0489999999999999,"15min":1.038},"min":{"1min":0.84699999999999998,"5min":0.622,"15min":0.61699999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.8360000000000001},"last":0.81100000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32224,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1126400,"data_stored":510100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.42299999999999999,"15min":0.38500000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":1.23,"15min":1.23},"last":0.63}]},{"osd":2,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65100000000000002,"15min":0.69199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85799999999999998,"5min":1.0029999999999999,"15min":1.04},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76200000000000001,"15min":0.73999999999999999},"min":{"1min":0.54800000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.113,"5min":1.1499999999999999,"15min":1.1779999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.751},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":0.88600000000000001,"5min":1.583,"15min":1.583},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.92700000000000005,"15min":0.96999999999999997},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2949999999999999,"5min":1.333,"15min":1.431},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97399999999999998,"15min":0.97199999999999998},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4730000000000001,"5min":1.55,"15min":1.55},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92700000000000005,"15min":0.94999999999999996},"min":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.153,"5min":1.5,"15min":1.5},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":1.075},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3240000000000001,"5min":1.605,"15min":1.726},"last":0.91100000000000003}]},{"osd":6,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96499999999999997,"15min":0.98899999999999999},"min":{"1min":0.74199999999999999,"5min":0.72399999999999998,"15min":0.60899999999999999},"max":{"1min":1.274,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.0549999999999999,"15min":1.101},"min":{"1min":0.89900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.24,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.401,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.028},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0409999999999999,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.65800000000000003},"max":{"1min":1.3120000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.879}]}]},{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1777664,"data_stored":1126971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85799999999999998,"15min":0.82999999999999996},"min":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0920000000000001,"5min":1.282,"15min":1.282},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.68200000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.78400000000000003}]},{"osd":1,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91100000000000003,"15min":0.90800000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89200000000000002,"15min":0.86699999999999999},"min":{"1min":0.63600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1850000000000001,"5min":1.488,"15min":1.488},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86799999999999999,"15min":0.90800000000000003},"min":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.04},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95299999999999996,"15min":0.96299999999999997},"min":{"1min":0.66300000000000003,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.337},"last":0.91300000000000003}]},{"osd":3,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92800000000000005,"15min":0.91200000000000003},"min":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3919999999999999,"5min":1.5269999999999999,"15min":1.5589999999999999},"last":1.081},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.86799999999999999,"15min":0.84699999999999998},"min":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.2290000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.94399999999999995,"15min":0.96699999999999997},"min":{"1min":0.64100000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.92000000000000004,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"max":{"1min":1.103,"5min":1.482,"15min":1.482},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0329999999999999,"15min":1.048},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.70599999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1040000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.044,"15min":1.0800000000000001},"min":{"1min":0.80000000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":1.599,"15min":1.599},"last":1.111}]}]},{"osd":6,"up_from":14,"seq":60129542229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1196032,"data_stored":538715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.38200000000000001},"max":{"1min":1.0940000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80300000000000005,"15min":0.84899999999999998},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.038,"5min":1.0669999999999999,"15min":1.141},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80600000000000005,"15min":0.86499999999999999},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95299999999999996,"5min":1.3300000000000001,"15min":1.758},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90800000000000003,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1499999999999999,"5min":1.4790000000000001,"15min":1.8089999999999999},"last":1.2070000000000001}]},{"osd":2,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.97199999999999998},"min":{"1min":0.64100000000000001,"5min":0.55700000000000005,"15min":0.49099999999999999},"max":{"1min":1.167,"5min":1.375,"15min":1.4319999999999999},"last":1.165},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.92300000000000004},"min":{"1min":0.66900000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.21,"5min":1.397,"15min":1.419},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.97399999999999998,"15min":1.0760000000000001},"min":{"1min":0.72699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1910000000000001,"5min":1.3580000000000001,"15min":3.2040000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90700000000000003,"15min":0.97699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.1120000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93300000000000005,"15min":0.98199999999999998},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1970000000000001,"5min":1.538,"15min":1.538},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.92700000000000005,"15min":1.026},"min":{"1min":0.629,"5min":0.53200000000000003,"15min":0.44400000000000001},"max":{"1min":1.2050000000000001,"5min":1.4430000000000001,"15min":1.458},"last":0.73299999999999998}]},{"osd":5,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.038,"15min":1.123},"min":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2549999999999999,"5min":1.528,"15min":1.681},"last":1.117},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.1259999999999999},"min":{"1min":0.93899999999999995,"5min":0.68000000000000005,"15min":0.66000000000000003},"max":{"1min":1.177,"5min":1.466,"15min":1.6000000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94699999999999995,"15min":1.0629999999999999},"min":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.091,"5min":1.4330000000000001,"15min":1.835},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1200000000000001,"15min":1.2150000000000001},"min":{"1min":0.748,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5600000000000001,"5min":1.6970000000000001,"15min":2.2999999999999998},"last":0.98299999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":536547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83499999999999996,"15min":0.81299999999999994},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.33000000000000002},"max":{"1min":1.226,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84199999999999997,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.53400000000000003,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.52,"15min":1.52},"last":1.119}]},{"osd":1,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.56100000000000005,"15min":0.35199999999999998},"max":{"1min":1.1519999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.85599999999999998,"15min":0.81299999999999994},"min":{"1min":0.70799999999999996,"5min":0.57299999999999995,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.96799999999999997}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90900000000000003,"15min":0.88100000000000001},"min":{"1min":0.62,"5min":0.51500000000000001,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.90600000000000003,"15min":0.85599999999999998},"min":{"1min":0.67000000000000004,"5min":0.57499999999999996,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.89800000000000002,"15min":0.874},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.435},"max":{"1min":1.1379999999999999,"5min":1.298,"15min":1.298},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.97299999999999998,"15min":0.91400000000000003},"min":{"1min":0.67500000000000004,"5min":0.63900000000000001,"15min":0.46300000000000002},"max":{"1min":1.1779999999999999,"5min":1.349,"15min":1.349},"last":0.875},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98899999999999999,"15min":0.92300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.1719999999999999,"5min":1.605,"15min":1.605},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.0740000000000001,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.69899999999999995,"15min":0.54900000000000004},"max":{"1min":1.2330000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.997}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.81399999999999995,"5min":0.71399999999999997,"15min":0.57299999999999995},"max":{"1min":1.282,"5min":1.538,"15min":1.538},"last":1.024},{"interface":"front","average":{"1min":1.012,"5min":1.095,"15min":1.0309999999999999},"min":{"1min":0.83699999999999997,"5min":0.81000000000000005,"15min":0.53400000000000003},"max":{"1min":1.2569999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.84199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542229,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1183744,"data_stored":536707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81299999999999994,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.083,"5min":1.242,"15min":1.28},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1519999999999999,"5min":1.294,"15min":1.294},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.85899999999999999,"15min":0.86699999999999999},"min":{"1min":0.623,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0920000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.91400000000000003,"15min":0.88800000000000001},"min":{"1min":0.63200000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.80000000000000004}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84499999999999997,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.224,"15min":1.29},"last":1.093},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.91100000000000003,"15min":0.93100000000000005},"min":{"1min":0.71299999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1779999999999999,"5min":1.351,"15min":1.637},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93500000000000005,"15min":1.022},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60599999999999998},"max":{"1min":1.159,"5min":1.337,"15min":2.601},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95199999999999996,"15min":1.0780000000000001},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.4259999999999999,"15min":2.8769999999999998},"last":0.876}]},{"osd":5,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.792},"last":0.622},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96399999999999997,"15min":1.0149999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.177,"5min":1.3839999999999999,"15min":1.575},"last":1.1439999999999999}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.2390000000000001,"5min":1.377,"15min":1.831},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.0600000000000001,"15min":1.091},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3129999999999999,"5min":1.347,"15min":1.8899999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0189999999999999,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2290000000000001,"5min":1.5169999999999999,"15min":1.8500000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.135,"15min":1.169},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.76000000000000001},"max":{"1min":1.2749999999999999,"5min":1.5169999999999999,"15min":1.944},"last":0.91600000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":536547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":1.0369999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.39700000000000002,"15min":0.39300000000000002},"max":{"1min":1.022,"5min":1.133,"15min":1.2190000000000001},"last":0.45300000000000001}]},{"osd":1,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.66100000000000003},"min":{"1min":0.41799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.53400000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93999999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.56200000000000006,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.071,"15min":1.071},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.59799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96499999999999997,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94699999999999995,"15min":0.91900000000000004},"min":{"1min":0.78800000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2310000000000001,"5min":1.504,"15min":1.504},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.99299999999999999,"15min":1.0229999999999999},"min":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.157,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.91500000000000004}]},{"osd":5,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.01,"15min":1.0329999999999999},"min":{"1min":0.76700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.3580000000000001,"5min":1.407,"15min":1.484},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.208,"5min":1.456,"15min":1.456},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.76400000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2010000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.075,"15min":1.0860000000000001},"min":{"1min":0.879,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.008}]},{"osd":7,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.073,"15min":1.081},"min":{"1min":0.73199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.345,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.89000000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.288,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.034}]}]},{"osd":0,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1794048,"data_stored":1129185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.62,"15min":0.67100000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":1.012,"15min":1.617},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.62,"15min":0.69999999999999996},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.33600000000000002},"max":{"1min":0.91100000000000003,"5min":1.113,"15min":1.5129999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.45700000000000002,"15min":0.44500000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.446},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.97799999999999998,"5min":1.4410000000000001,"15min":1.6910000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed May 8 22:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.80400000000000005},"min":{"1min":0.49299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89900000000000002,"5min":1.609,"15min":1.714},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.621}]},{"osd":4,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.93500000000000005,"15min":0.90600000000000003},"min":{"1min":0.71999999999999997,"5min":0.66000000000000003,"15min":0.61699999999999999},"max":{"1min":1.1930000000000001,"5min":4.391,"15min":4.391},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":0.995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3169999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.97399999999999998}]},{"osd":5,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.89600000000000002,"15min":0.91000000000000003},"min":{"1min":0.71599999999999997,"5min":0.65500000000000003,"15min":0.64600000000000002},"max":{"1min":1.052,"5min":1.419,"15min":1.419},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.97099999999999997,"15min":0.997},"min":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.423,"5min":1.423,"15min":1.474},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.68600000000000005},"max":{"1min":1.1619999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.036,"5min":1.048,"15min":1.0569999999999999},"min":{"1min":0.85799999999999998,"5min":0.72699999999999998,"15min":0.66300000000000003},"max":{"1min":1.474,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":0.95099999999999996}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.054,"15min":1.0860000000000001},"min":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.72199999999999998},"max":{"1min":1.2509999999999999,"5min":1.532,"15min":1.6459999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.024},"min":{"1min":0.88,"5min":0.74299999999999999,"15min":0.60399999999999998},"max":{"1min":1.2150000000000001,"5min":1.365,"15min":1.4470000000000001},"last":0.995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:40:05.220 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:40:05.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.333+0000 7fc7c7fff700 1 Processor -- start 2024-05-08T22:40:05.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.333+0000 7fc7c7fff700 1 -- start start 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7c8164db0 0x7fc7c806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c8165d50 con 0x7fc7c815ef80 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c806ad70 con 0x7fc7c8164db0 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8165260 con 0x7fc7c806de70 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7c8164db0 0x7fc7c806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7c8164db0 0x7fc7c806b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39972/0 (socket says 172.21.15.179:39972) 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2450535080 learned_addr learned my addr 172.21.15.179:0/2450535080 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:05.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c67fc700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2450535080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 msgr2=0x7fc7c8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:05.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2450535080 >> v1:172.21.15.195:6789/0 conn(0x7fc7c815ef80 legacy=0x7fc7c806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2450535080 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c806e4b0 con 0x7fc7c8164db0 2024-05-08T22:40:05.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c77fe700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c806b200 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009910 tx=0x7fc7c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178ce2c266c36d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c5ffb700 1 -- 172.21.15.179:0/2450535080 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0011830 con 0x7fc7c8164db0 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c5ffb700 1 -- 172.21.15.179:0/2450535080 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0011990 con 0x7fc7c8164db0 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c5ffb700 1 -- 172.21.15.179:0/2450535080 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0011d40 con 0x7fc7c8164db0 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2450535080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 msgr2=0x7fc7c806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c806b200 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009910 tx=0x7fc7c0009c20 comp rx=0 tx=0).stop 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2450535080 shutdown_connections 2024-05-08T22:40:05.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2450535080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c806b200 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2450535080 >> 172.21.15.179:0/2450535080 conn(0x7fc7c80ce9d0 msgr2=0x7fc7c80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2450535080 shutdown_connections 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2450535080 wait complete. 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 Processor -- start 2024-05-08T22:40:05.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.337+0000 7fc7c7fff700 1 -- start start 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ef80 0x7fc7c8159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815ab70 con 0x7fc7c8164db0 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815b000 con 0x7fc7c815ef80 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8159f60 con 0x7fc7c806de70 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39988/0 (socket says 172.21.15.179:39988) 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2448130491 learned_addr learned my addr 172.21.15.179:0/2448130491 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c67fc700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ef80 0x7fc7c8159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c6ffd700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 msgr2=0x7fc7c8149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ef80 msgr2=0x7fc7c8159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ef80 0x7fc7c8159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 -- 172.21.15.179:0/2448130491 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009570 con 0x7fc7c8164db0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c6ffd700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c77fe700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8072020 tx=0x7fc7c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004590 con 0x7fc7c8164db0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c001e070 con 0x7fc7c8164db0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0003900 con 0x7fc7c8164db0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c815a240 con 0x7fc7c8164db0 2024-05-08T22:40:05.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c815a800 con 0x7fc7c8164db0 2024-05-08T22:40:05.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.341+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c815ad20 con 0x7fc7c8164db0 2024-05-08T22:40:05.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.345+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7c0003a60 con 0x7fc7c8164db0 2024-05-08T22:40:05.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.345+0000 7fc7bffff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 0x7fc7a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.345+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc7c0085190 con 0x7fc7c8164db0 2024-05-08T22:40:05.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.345+0000 7fc7c6ffd700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 0x7fc7a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.345+0000 7fc7c6ffd700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 0x7fc7a8068bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0000f60 tx=0x7fc7b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.353+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7c815ad20 con 0x7fc7c8164db0 2024-05-08T22:40:05.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.525+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7c8165a70 con 0x7fc7a8066730 2024-05-08T22:40:05.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.537+0000 7fc7bffff700 1 -- 172.21.15.179:0/2448130491 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288140 (secure 0 0 0) 0x7fc7b0001d70 con 0x7fc7a8066730 2024-05-08T22:40:05.547 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:05.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 msgr2=0x7fc7a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 0x7fc7a8068bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0000f60 tx=0x7fc7b0009040 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 msgr2=0x7fc7c8161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8072020 tx=0x7fc7c0004770 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 shutdown_connections 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7a8066730 0x7fc7a8068bf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ef80 0x7fc7c8159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7c8164db0 0x7fc7c8161d10 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 --2- 172.21.15.179:0/2448130491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c806de70 0x7fc7c8149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 >> 172.21.15.179:0/2448130491 conn(0x7fc7c80ce9d0 msgr2=0x7fc7c80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 shutdown_connections 2024-05-08T22:40:05.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.545+0000 7fc7c7fff700 1 -- 172.21.15.179:0/2448130491 wait complete. 2024-05-08T22:40:05.551 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:40:05.563 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-05-08T22:40:05.523911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263440,"kb_used_data":10408,"kb_used_omap":12,"kb_used_meta":252915,"kb_avail":754711280,"statfs":{"total":773094113280,"available":772824350720,"internally_reserved":0,"allocated":10657792,"data_stored":5477836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":258985528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.772799"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770737+0000","last_change":"2024-05-08T22:40:01.175542+0000","last_active":"2024-05-08T22:40:02.770737+0000","last_peered":"2024-05-08T22:40:02.770737+0000","last_clean":"2024-05-08T22:40:02.770737+0000","last_became_active":"2024-05-08T22:40:01.175286+0000","last_became_peered":"2024-05-08T22:40:01.175286+0000","last_unstale":"2024-05-08T22:40:02.770737+0000","last_undegraded":"2024-05-08T22:40:02.770737+0000","last_fullsized":"2024-05-08T22:40:02.770737+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:08:00.085505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776788+0000","last_change":"2024-05-08T22:40:01.165659+0000","last_active":"2024-05-08T22:40:02.776788+0000","last_peered":"2024-05-08T22:40:02.776788+0000","last_clean":"2024-05-08T22:40:02.776788+0000","last_became_active":"2024-05-08T22:40:01.165368+0000","last_became_peered":"2024-05-08T22:40:01.165368+0000","last_unstale":"2024-05-08T22:40:02.776788+0000","last_undegraded":"2024-05-08T22:40:02.776788+0000","last_fullsized":"2024-05-08T22:40:02.776788+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:39.746907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770366+0000","last_change":"2024-05-08T22:40:01.175416+0000","last_active":"2024-05-08T22:40:02.770366+0000","last_peered":"2024-05-08T22:40:02.770366+0000","last_clean":"2024-05-08T22:40:02.770366+0000","last_became_active":"2024-05-08T22:40:01.175137+0000","last_became_peered":"2024-05-08T22:40:01.175137+0000","last_unstale":"2024-05-08T22:40:02.770366+0000","last_undegraded":"2024-05-08T22:40:02.770366+0000","last_fullsized":"2024-05-08T22:40:02.770366+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:50:44.672867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779338+0000","last_change":"2024-05-08T22:40:01.177087+0000","last_active":"2024-05-08T22:40:02.779338+0000","last_peered":"2024-05-08T22:40:02.779338+0000","last_clean":"2024-05-08T22:40:02.779338+0000","last_became_active":"2024-05-08T22:40:01.176814+0000","last_became_peered":"2024-05-08T22:40:01.176814+0000","last_unstale":"2024-05-08T22:40:02.779338+0000","last_undegraded":"2024-05-08T22:40:02.779338+0000","last_fullsized":"2024-05-08T22:40:02.779338+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:42:24.158606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158010+0000","last_change":"2024-05-08T22:40:01.157440+0000","last_active":"2024-05-08T22:40:02.158010+0000","last_peered":"2024-05-08T22:40:02.158010+0000","last_clean":"2024-05-08T22:40:02.158010+0000","last_became_active":"2024-05-08T22:40:01.156861+0000","last_became_peered":"2024-05-08T22:40:01.156861+0000","last_unstale":"2024-05-08T22:40:02.158010+0000","last_undegraded":"2024-05-08T22:40:02.158010+0000","last_fullsized":"2024-05-08T22:40:02.158010+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:01.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777657+0000","last_change":"2024-05-08T22:40:01.176314+0000","last_active":"2024-05-08T22:40:02.777657+0000","last_peered":"2024-05-08T22:40:02.777657+0000","last_clean":"2024-05-08T22:40:02.777657+0000","last_became_active":"2024-05-08T22:40:01.170247+0000","last_became_peered":"2024-05-08T22:40:01.170247+0000","last_unstale":"2024-05-08T22:40:02.777657+0000","last_undegraded":"2024-05-08T22:40:02.777657+0000","last_fullsized":"2024-05-08T22:40:02.777657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:54.552531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778046+0000","last_change":"2024-05-08T22:40:01.160763+0000","last_active":"2024-05-08T22:40:02.778046+0000","last_peered":"2024-05-08T22:40:02.778046+0000","last_clean":"2024-05-08T22:40:02.778046+0000","last_became_active":"2024-05-08T22:40:01.160411+0000","last_became_peered":"2024-05-08T22:40:01.160411+0000","last_unstale":"2024-05-08T22:40:02.778046+0000","last_undegraded":"2024-05-08T22:40:02.778046+0000","last_fullsized":"2024-05-08T22:40:02.778046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:50.548005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779010+0000","last_change":"2024-05-08T22:40:01.164376+0000","last_active":"2024-05-08T22:40:02.779010+0000","last_peered":"2024-05-08T22:40:02.779010+0000","last_clean":"2024-05-08T22:40:02.779010+0000","last_became_active":"2024-05-08T22:40:01.164167+0000","last_became_peered":"2024-05-08T22:40:01.164167+0000","last_unstale":"2024-05-08T22:40:02.779010+0000","last_undegraded":"2024-05-08T22:40:02.779010+0000","last_fullsized":"2024-05-08T22:40:02.779010+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:10.201272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159500+0000","last_change":"2024-05-08T22:40:01.170682+0000","last_active":"2024-05-08T22:40:02.159500+0000","last_peered":"2024-05-08T22:40:02.159500+0000","last_clean":"2024-05-08T22:40:02.159500+0000","last_became_active":"2024-05-08T22:40:01.170408+0000","last_became_peered":"2024-05-08T22:40:01.170408+0000","last_unstale":"2024-05-08T22:40:02.159500+0000","last_undegraded":"2024-05-08T22:40:02.159500+0000","last_fullsized":"2024-05-08T22:40:02.159500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:15.318176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158711+0000","last_change":"2024-05-08T22:40:01.170718+0000","last_active":"2024-05-08T22:40:02.158711+0000","last_peered":"2024-05-08T22:40:02.158711+0000","last_clean":"2024-05-08T22:40:02.158711+0000","last_became_active":"2024-05-08T22:40:01.169054+0000","last_became_peered":"2024-05-08T22:40:01.169054+0000","last_unstale":"2024-05-08T22:40:02.158711+0000","last_undegraded":"2024-05-08T22:40:02.158711+0000","last_fullsized":"2024-05-08T22:40:02.158711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:14.931477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158629+0000","last_change":"2024-05-08T22:40:01.179236+0000","last_active":"2024-05-08T22:40:02.158629+0000","last_peered":"2024-05-08T22:40:02.158629+0000","last_clean":"2024-05-08T22:40:02.158629+0000","last_became_active":"2024-05-08T22:40:01.179069+0000","last_became_peered":"2024-05-08T22:40:01.179069+0000","last_unstale":"2024-05-08T22:40:02.158629+0000","last_undegraded":"2024-05-08T22:40:02.158629+0000","last_fullsized":"2024-05-08T22:40:02.158629+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:22.920302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158389+0000","last_change":"2024-05-08T22:40:01.168157+0000","last_active":"2024-05-08T22:40:02.158389+0000","last_peered":"2024-05-08T22:40:02.158389+0000","last_clean":"2024-05-08T22:40:02.158389+0000","last_became_active":"2024-05-08T22:40:01.167883+0000","last_became_peered":"2024-05-08T22:40:01.167883+0000","last_unstale":"2024-05-08T22:40:02.158389+0000","last_undegraded":"2024-05-08T22:40:02.158389+0000","last_fullsized":"2024-05-08T22:40:02.158389+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:02.269420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779131+0000","last_change":"2024-05-08T22:40:01.166959+0000","last_active":"2024-05-08T22:40:02.779131+0000","last_peered":"2024-05-08T22:40:02.779131+0000","last_clean":"2024-05-08T22:40:02.779131+0000","last_became_active":"2024-05-08T22:40:01.166677+0000","last_became_peered":"2024-05-08T22:40:01.166677+0000","last_unstale":"2024-05-08T22:40:02.779131+0000","last_undegraded":"2024-05-08T22:40:02.779131+0000","last_fullsized":"2024-05-08T22:40:02.779131+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:41:26.673169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157596+0000","last_change":"2024-05-08T22:40:01.166264+0000","last_active":"2024-05-08T22:40:02.157596+0000","last_peered":"2024-05-08T22:40:02.157596+0000","last_clean":"2024-05-08T22:40:02.157596+0000","last_became_active":"2024-05-08T22:40:01.165967+0000","last_became_peered":"2024-05-08T22:40:01.165967+0000","last_unstale":"2024-05-08T22:40:02.157596+0000","last_undegraded":"2024-05-08T22:40:02.157596+0000","last_fullsized":"2024-05-08T22:40:02.157596+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:45.929756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778483+0000","last_change":"2024-05-08T22:40:01.177949+0000","last_active":"2024-05-08T22:40:02.778483+0000","last_peered":"2024-05-08T22:40:02.778483+0000","last_clean":"2024-05-08T22:40:02.778483+0000","last_became_active":"2024-05-08T22:40:01.177677+0000","last_became_peered":"2024-05-08T22:40:01.177677+0000","last_unstale":"2024-05-08T22:40:02.778483+0000","last_undegraded":"2024-05-08T22:40:02.778483+0000","last_fullsized":"2024-05-08T22:40:02.778483+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:07.802805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777378+0000","last_change":"2024-05-08T22:40:01.176520+0000","last_active":"2024-05-08T22:40:02.777378+0000","last_peered":"2024-05-08T22:40:02.777378+0000","last_clean":"2024-05-08T22:40:02.777378+0000","last_became_active":"2024-05-08T22:40:01.174004+0000","last_became_peered":"2024-05-08T22:40:01.174004+0000","last_unstale":"2024-05-08T22:40:02.777378+0000","last_undegraded":"2024-05-08T22:40:02.777378+0000","last_fullsized":"2024-05-08T22:40:02.777378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:30.934567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778358+0000","last_change":"2024-05-08T22:40:01.174406+0000","last_active":"2024-05-08T22:40:02.778358+0000","last_peered":"2024-05-08T22:40:02.778358+0000","last_clean":"2024-05-08T22:40:02.778358+0000","last_became_active":"2024-05-08T22:40:01.174147+0000","last_became_peered":"2024-05-08T22:40:01.174147+0000","last_unstale":"2024-05-08T22:40:02.778358+0000","last_undegraded":"2024-05-08T22:40:02.778358+0000","last_fullsized":"2024-05-08T22:40:02.778358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:39.165962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770452+0000","last_change":"2024-05-08T22:40:01.169643+0000","last_active":"2024-05-08T22:40:02.770452+0000","last_peered":"2024-05-08T22:40:02.770452+0000","last_clean":"2024-05-08T22:40:02.770452+0000","last_became_active":"2024-05-08T22:40:01.169401+0000","last_became_peered":"2024-05-08T22:40:01.169401+0000","last_unstale":"2024-05-08T22:40:02.770452+0000","last_undegraded":"2024-05-08T22:40:02.770452+0000","last_fullsized":"2024-05-08T22:40:02.770452+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:13.752047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158768+0000","last_change":"2024-05-08T22:40:01.165738+0000","last_active":"2024-05-08T22:40:02.158768+0000","last_peered":"2024-05-08T22:40:02.158768+0000","last_clean":"2024-05-08T22:40:02.158768+0000","last_became_active":"2024-05-08T22:40:01.165475+0000","last_became_peered":"2024-05-08T22:40:01.165475+0000","last_unstale":"2024-05-08T22:40:02.158768+0000","last_undegraded":"2024-05-08T22:40:02.158768+0000","last_fullsized":"2024-05-08T22:40:02.158768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:33.091502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779200+0000","last_change":"2024-05-08T22:40:01.163266+0000","last_active":"2024-05-08T22:40:02.779200+0000","last_peered":"2024-05-08T22:40:02.779200+0000","last_clean":"2024-05-08T22:40:02.779200+0000","last_became_active":"2024-05-08T22:40:01.162491+0000","last_became_peered":"2024-05-08T22:40:01.162491+0000","last_unstale":"2024-05-08T22:40:02.779200+0000","last_undegraded":"2024-05-08T22:40:02.779200+0000","last_fullsized":"2024-05-08T22:40:02.779200+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:20.847730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157869+0000","last_change":"2024-05-08T22:40:01.158150+0000","last_active":"2024-05-08T22:40:02.157869+0000","last_peered":"2024-05-08T22:40:02.157869+0000","last_clean":"2024-05-08T22:40:02.157869+0000","last_became_active":"2024-05-08T22:40:01.157169+0000","last_became_peered":"2024-05-08T22:40:01.157169+0000","last_unstale":"2024-05-08T22:40:02.157869+0000","last_undegraded":"2024-05-08T22:40:02.157869+0000","last_fullsized":"2024-05-08T22:40:02.157869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:54.095539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157585+0000","last_change":"2024-05-08T22:40:01.159819+0000","last_active":"2024-05-08T22:40:02.157585+0000","last_peered":"2024-05-08T22:40:02.157585+0000","last_clean":"2024-05-08T22:40:02.157585+0000","last_became_active":"2024-05-08T22:40:01.159558+0000","last_became_peered":"2024-05-08T22:40:01.159558+0000","last_unstale":"2024-05-08T22:40:02.157585+0000","last_undegraded":"2024-05-08T22:40:02.157585+0000","last_fullsized":"2024-05-08T22:40:02.157585+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:39.038788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157876+0000","last_change":"2024-05-08T22:40:01.161934+0000","last_active":"2024-05-08T22:40:02.157876+0000","last_peered":"2024-05-08T22:40:02.157876+0000","last_clean":"2024-05-08T22:40:02.157876+0000","last_became_active":"2024-05-08T22:40:01.159856+0000","last_became_peered":"2024-05-08T22:40:01.159856+0000","last_unstale":"2024-05-08T22:40:02.157876+0000","last_undegraded":"2024-05-08T22:40:02.157876+0000","last_fullsized":"2024-05-08T22:40:02.157876+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.627596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770043+0000","last_change":"2024-05-08T22:40:01.174485+0000","last_active":"2024-05-08T22:40:02.770043+0000","last_peered":"2024-05-08T22:40:02.770043+0000","last_clean":"2024-05-08T22:40:02.770043+0000","last_became_active":"2024-05-08T22:40:01.174248+0000","last_became_peered":"2024-05-08T22:40:01.174248+0000","last_unstale":"2024-05-08T22:40:02.770043+0000","last_undegraded":"2024-05-08T22:40:02.770043+0000","last_fullsized":"2024-05-08T22:40:02.770043+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:16:20.333349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777322+0000","last_change":"2024-05-08T22:40:01.169075+0000","last_active":"2024-05-08T22:40:02.777322+0000","last_peered":"2024-05-08T22:40:02.777322+0000","last_clean":"2024-05-08T22:40:02.777322+0000","last_became_active":"2024-05-08T22:40:01.168810+0000","last_became_peered":"2024-05-08T22:40:01.168810+0000","last_unstale":"2024-05-08T22:40:02.777322+0000","last_undegraded":"2024-05-08T22:40:02.777322+0000","last_fullsized":"2024-05-08T22:40:02.777322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:49.903039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159206+0000","last_change":"2024-05-08T22:40:01.165423+0000","last_active":"2024-05-08T22:40:02.159206+0000","last_peered":"2024-05-08T22:40:02.159206+0000","last_clean":"2024-05-08T22:40:02.159206+0000","last_became_active":"2024-05-08T22:40:01.165160+0000","last_became_peered":"2024-05-08T22:40:01.165160+0000","last_unstale":"2024-05-08T22:40:02.159206+0000","last_undegraded":"2024-05-08T22:40:02.159206+0000","last_fullsized":"2024-05-08T22:40:02.159206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:43.911356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158540+0000","last_change":"2024-05-08T22:40:01.162310+0000","last_active":"2024-05-08T22:40:02.158540+0000","last_peered":"2024-05-08T22:40:02.158540+0000","last_clean":"2024-05-08T22:40:02.158540+0000","last_became_active":"2024-05-08T22:40:01.162077+0000","last_became_peered":"2024-05-08T22:40:01.162077+0000","last_unstale":"2024-05-08T22:40:02.158540+0000","last_undegraded":"2024-05-08T22:40:02.158540+0000","last_fullsized":"2024-05-08T22:40:02.158540+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:19.151595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779369+0000","last_change":"2024-05-08T22:40:01.164015+0000","last_active":"2024-05-08T22:40:02.779369+0000","last_peered":"2024-05-08T22:40:02.779369+0000","last_clean":"2024-05-08T22:40:02.779369+0000","last_became_active":"2024-05-08T22:40:01.163540+0000","last_became_peered":"2024-05-08T22:40:01.163540+0000","last_unstale":"2024-05-08T22:40:02.779369+0000","last_undegraded":"2024-05-08T22:40:02.779369+0000","last_fullsized":"2024-05-08T22:40:02.779369+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.627596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778350+0000","last_change":"2024-05-08T22:40:01.162156+0000","last_active":"2024-05-08T22:40:02.778350+0000","last_peered":"2024-05-08T22:40:02.778350+0000","last_clean":"2024-05-08T22:40:02.778350+0000","last_became_active":"2024-05-08T22:40:01.161909+0000","last_became_peered":"2024-05-08T22:40:01.161909+0000","last_unstale":"2024-05-08T22:40:02.778350+0000","last_undegraded":"2024-05-08T22:40:02.778350+0000","last_fullsized":"2024-05-08T22:40:02.778350+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:58:51.515199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157676+0000","last_change":"2024-05-08T22:40:01.170095+0000","last_active":"2024-05-08T22:40:02.157676+0000","last_peered":"2024-05-08T22:40:02.157676+0000","last_clean":"2024-05-08T22:40:02.157676+0000","last_became_active":"2024-05-08T22:40:01.169821+0000","last_became_peered":"2024-05-08T22:40:01.169821+0000","last_unstale":"2024-05-08T22:40:02.157676+0000","last_undegraded":"2024-05-08T22:40:02.157676+0000","last_fullsized":"2024-05-08T22:40:02.157676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:07.808574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.775945+0000","last_change":"2024-05-08T22:40:01.167498+0000","last_active":"2024-05-08T22:40:02.775945+0000","last_peered":"2024-05-08T22:40:02.775945+0000","last_clean":"2024-05-08T22:40:02.775945+0000","last_became_active":"2024-05-08T22:40:01.167206+0000","last_became_peered":"2024-05-08T22:40:01.167206+0000","last_unstale":"2024-05-08T22:40:02.775945+0000","last_undegraded":"2024-05-08T22:40:02.775945+0000","last_fullsized":"2024-05-08T22:40:02.775945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:16.869390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777430+0000","last_change":"2024-05-08T22:40:01.171406+0000","last_active":"2024-05-08T22:40:02.777430+0000","last_peered":"2024-05-08T22:40:02.777430+0000","last_clean":"2024-05-08T22:40:02.777430+0000","last_became_active":"2024-05-08T22:40:01.170943+0000","last_became_peered":"2024-05-08T22:40:01.170943+0000","last_unstale":"2024-05-08T22:40:02.777430+0000","last_undegraded":"2024-05-08T22:40:02.777430+0000","last_fullsized":"2024-05-08T22:40:02.777430+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:15.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777843+0000","last_change":"2024-05-08T22:40:01.176643+0000","last_active":"2024-05-08T22:40:02.777843+0000","last_peered":"2024-05-08T22:40:02.777843+0000","last_clean":"2024-05-08T22:40:02.777843+0000","last_became_active":"2024-05-08T22:40:01.176371+0000","last_became_peered":"2024-05-08T22:40:01.176371+0000","last_unstale":"2024-05-08T22:40:02.777843+0000","last_undegraded":"2024-05-08T22:40:02.777843+0000","last_fullsized":"2024-05-08T22:40:02.777843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:25.367298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"95'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158637+0000","last_change":"2024-05-08T22:40:00.774315+0000","last_active":"2024-05-08T22:40:02.158637+0000","last_peered":"2024-05-08T22:40:02.158637+0000","last_clean":"2024-05-08T22:40:02.158637+0000","last_became_active":"2024-05-08T22:40:00.774083+0000","last_became_peered":"2024-05-08T22:40:00.774083+0000","last_unstale":"2024-05-08T22:40:02.158637+0000","last_undegraded":"2024-05-08T22:40:02.158637+0000","last_fullsized":"2024-05-08T22:40:02.158637+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:27.360721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777922+0000","last_change":"2024-05-08T22:40:01.178935+0000","last_active":"2024-05-08T22:40:02.777922+0000","last_peered":"2024-05-08T22:40:02.777922+0000","last_clean":"2024-05-08T22:40:02.777922+0000","last_became_active":"2024-05-08T22:40:01.177949+0000","last_became_peered":"2024-05-08T22:40:01.177949+0000","last_unstale":"2024-05-08T22:40:02.777922+0000","last_undegraded":"2024-05-08T22:40:02.777922+0000","last_fullsized":"2024-05-08T22:40:02.777922+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:07.587283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159395+0000","last_change":"2024-05-08T22:40:00.771757+0000","last_active":"2024-05-08T22:40:02.159395+0000","last_peered":"2024-05-08T22:40:02.159395+0000","last_clean":"2024-05-08T22:40:02.159395+0000","last_became_active":"2024-05-08T22:40:00.771526+0000","last_became_peered":"2024-05-08T22:40:00.771526+0000","last_unstale":"2024-05-08T22:40:02.159395+0000","last_undegraded":"2024-05-08T22:40:02.159395+0000","last_fullsized":"2024-05-08T22:40:02.159395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:15.689463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778570+0000","last_change":"2024-05-08T22:40:01.160428+0000","last_active":"2024-05-08T22:40:02.778570+0000","last_peered":"2024-05-08T22:40:02.778570+0000","last_clean":"2024-05-08T22:40:02.778570+0000","last_became_active":"2024-05-08T22:40:01.160002+0000","last_became_peered":"2024-05-08T22:40:01.160002+0000","last_unstale":"2024-05-08T22:40:02.778570+0000","last_undegraded":"2024-05-08T22:40:02.778570+0000","last_fullsized":"2024-05-08T22:40:02.778570+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:01.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778433+0000","last_change":"2024-05-08T22:40:00.776580+0000","last_active":"2024-05-08T22:40:02.778433+0000","last_peered":"2024-05-08T22:40:02.778433+0000","last_clean":"2024-05-08T22:40:02.778433+0000","last_became_active":"2024-05-08T22:40:00.776269+0000","last_became_peered":"2024-05-08T22:40:00.776269+0000","last_unstale":"2024-05-08T22:40:02.778433+0000","last_undegraded":"2024-05-08T22:40:02.778433+0000","last_fullsized":"2024-05-08T22:40:02.778433+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:54.249895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778439+0000","last_change":"2024-05-08T22:40:01.174367+0000","last_active":"2024-05-08T22:40:02.778439+0000","last_peered":"2024-05-08T22:40:02.778439+0000","last_clean":"2024-05-08T22:40:02.778439+0000","last_became_active":"2024-05-08T22:40:01.173707+0000","last_became_peered":"2024-05-08T22:40:01.173707+0000","last_unstale":"2024-05-08T22:40:02.778439+0000","last_undegraded":"2024-05-08T22:40:02.778439+0000","last_fullsized":"2024-05-08T22:40:02.778439+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:44.349613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158782+0000","last_change":"2024-05-08T22:40:00.778514+0000","last_active":"2024-05-08T22:40:02.158782+0000","last_peered":"2024-05-08T22:40:02.158782+0000","last_clean":"2024-05-08T22:40:02.158782+0000","last_became_active":"2024-05-08T22:40:00.778232+0000","last_became_peered":"2024-05-08T22:40:00.778232+0000","last_unstale":"2024-05-08T22:40:02.158782+0000","last_undegraded":"2024-05-08T22:40:02.158782+0000","last_fullsized":"2024-05-08T22:40:02.158782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:33.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778291+0000","last_change":"2024-05-08T22:40:01.166910+0000","last_active":"2024-05-08T22:40:02.778291+0000","last_peered":"2024-05-08T22:40:02.778291+0000","last_clean":"2024-05-08T22:40:02.778291+0000","last_became_active":"2024-05-08T22:40:01.166635+0000","last_became_peered":"2024-05-08T22:40:01.166635+0000","last_unstale":"2024-05-08T22:40:02.778291+0000","last_undegraded":"2024-05-08T22:40:02.778291+0000","last_fullsized":"2024-05-08T22:40:02.778291+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:18.761217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158023+0000","last_change":"2024-05-08T22:40:00.773224+0000","last_active":"2024-05-08T22:40:02.158023+0000","last_peered":"2024-05-08T22:40:02.158023+0000","last_clean":"2024-05-08T22:40:02.158023+0000","last_became_active":"2024-05-08T22:40:00.772790+0000","last_became_peered":"2024-05-08T22:40:00.772790+0000","last_unstale":"2024-05-08T22:40:02.158023+0000","last_undegraded":"2024-05-08T22:40:02.158023+0000","last_fullsized":"2024-05-08T22:40:02.158023+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:45.105491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158103+0000","last_change":"2024-05-08T22:40:01.180089+0000","last_active":"2024-05-08T22:40:02.158103+0000","last_peered":"2024-05-08T22:40:02.158103+0000","last_clean":"2024-05-08T22:40:02.158103+0000","last_became_active":"2024-05-08T22:40:01.179809+0000","last_became_peered":"2024-05-08T22:40:01.179809+0000","last_unstale":"2024-05-08T22:40:02.158103+0000","last_undegraded":"2024-05-08T22:40:02.158103+0000","last_fullsized":"2024-05-08T22:40:02.158103+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:18.761217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779534+0000","last_change":"2024-05-08T22:40:00.782692+0000","last_active":"2024-05-08T22:40:02.779534+0000","last_peered":"2024-05-08T22:40:02.779534+0000","last_clean":"2024-05-08T22:40:02.779534+0000","last_became_active":"2024-05-08T22:40:00.782426+0000","last_became_peered":"2024-05-08T22:40:00.782426+0000","last_unstale":"2024-05-08T22:40:02.779534+0000","last_undegraded":"2024-05-08T22:40:02.779534+0000","last_fullsized":"2024-05-08T22:40:02.779534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:19.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777655+0000","last_change":"2024-05-08T22:40:01.174364+0000","last_active":"2024-05-08T22:40:02.777655+0000","last_peered":"2024-05-08T22:40:02.777655+0000","last_clean":"2024-05-08T22:40:02.777655+0000","last_became_active":"2024-05-08T22:40:01.166300+0000","last_became_peered":"2024-05-08T22:40:01.166300+0000","last_unstale":"2024-05-08T22:40:02.777655+0000","last_undegraded":"2024-05-08T22:40:02.777655+0000","last_fullsized":"2024-05-08T22:40:02.777655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:35.396021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770452+0000","last_change":"2024-05-08T22:40:00.777023+0000","last_active":"2024-05-08T22:40:02.770452+0000","last_peered":"2024-05-08T22:40:02.770452+0000","last_clean":"2024-05-08T22:40:02.770452+0000","last_became_active":"2024-05-08T22:40:00.776764+0000","last_became_peered":"2024-05-08T22:40:00.776764+0000","last_unstale":"2024-05-08T22:40:02.770452+0000","last_undegraded":"2024-05-08T22:40:02.770452+0000","last_fullsized":"2024-05-08T22:40:02.770452+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:31.195877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779299+0000","last_change":"2024-05-08T22:40:01.158046+0000","last_active":"2024-05-08T22:40:02.779299+0000","last_peered":"2024-05-08T22:40:02.779299+0000","last_clean":"2024-05-08T22:40:02.779299+0000","last_became_active":"2024-05-08T22:40:01.157792+0000","last_became_peered":"2024-05-08T22:40:01.157792+0000","last_unstale":"2024-05-08T22:40:02.779299+0000","last_undegraded":"2024-05-08T22:40:02.779299+0000","last_fullsized":"2024-05-08T22:40:02.779299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:10.115995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777033+0000","last_change":"2024-05-08T22:40:00.787251+0000","last_active":"2024-05-08T22:40:02.777033+0000","last_peered":"2024-05-08T22:40:02.777033+0000","last_clean":"2024-05-08T22:40:02.777033+0000","last_became_active":"2024-05-08T22:40:00.787037+0000","last_became_peered":"2024-05-08T22:40:00.787037+0000","last_unstale":"2024-05-08T22:40:02.777033+0000","last_undegraded":"2024-05-08T22:40:02.777033+0000","last_fullsized":"2024-05-08T22:40:02.777033+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:48.270689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158147+0000","last_change":"2024-05-08T22:40:01.170124+0000","last_active":"2024-05-08T22:40:02.158147+0000","last_peered":"2024-05-08T22:40:02.158147+0000","last_clean":"2024-05-08T22:40:02.158147+0000","last_became_active":"2024-05-08T22:40:01.169895+0000","last_became_peered":"2024-05-08T22:40:01.169895+0000","last_unstale":"2024-05-08T22:40:02.158147+0000","last_undegraded":"2024-05-08T22:40:02.158147+0000","last_fullsized":"2024-05-08T22:40:02.158147+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:23.768473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777448+0000","last_change":"2024-05-08T22:40:00.771376+0000","last_active":"2024-05-08T22:40:02.777448+0000","last_peered":"2024-05-08T22:40:02.777448+0000","last_clean":"2024-05-08T22:40:02.777448+0000","last_became_active":"2024-05-08T22:40:00.771107+0000","last_became_peered":"2024-05-08T22:40:00.771107+0000","last_unstale":"2024-05-08T22:40:02.777448+0000","last_undegraded":"2024-05-08T22:40:02.777448+0000","last_fullsized":"2024-05-08T22:40:02.777448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:58.552604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778309+0000","last_change":"2024-05-08T22:40:01.160124+0000","last_active":"2024-05-08T22:40:02.778309+0000","last_peered":"2024-05-08T22:40:02.778309+0000","last_clean":"2024-05-08T22:40:02.778309+0000","last_became_active":"2024-05-08T22:40:01.159805+0000","last_became_peered":"2024-05-08T22:40:01.159805+0000","last_unstale":"2024-05-08T22:40:02.778309+0000","last_undegraded":"2024-05-08T22:40:02.778309+0000","last_fullsized":"2024-05-08T22:40:02.778309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:44.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770686+0000","last_change":"2024-05-08T22:40:00.779626+0000","last_active":"2024-05-08T22:40:02.770686+0000","last_peered":"2024-05-08T22:40:02.770686+0000","last_clean":"2024-05-08T22:40:02.770686+0000","last_became_active":"2024-05-08T22:40:00.779344+0000","last_became_peered":"2024-05-08T22:40:00.779344+0000","last_unstale":"2024-05-08T22:40:02.770686+0000","last_undegraded":"2024-05-08T22:40:02.770686+0000","last_fullsized":"2024-05-08T22:40:02.770686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:30.459140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770587+0000","last_change":"2024-05-08T22:40:01.167597+0000","last_active":"2024-05-08T22:40:02.770587+0000","last_peered":"2024-05-08T22:40:02.770587+0000","last_clean":"2024-05-08T22:40:02.770587+0000","last_became_active":"2024-05-08T22:40:01.167353+0000","last_became_peered":"2024-05-08T22:40:01.167353+0000","last_unstale":"2024-05-08T22:40:02.770587+0000","last_undegraded":"2024-05-08T22:40:02.770587+0000","last_fullsized":"2024-05-08T22:40:02.770587+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:23:32.452630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"95'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159275+0000","last_change":"2024-05-08T22:40:00.775095+0000","last_active":"2024-05-08T22:40:02.159275+0000","last_peered":"2024-05-08T22:40:02.159275+0000","last_clean":"2024-05-08T22:40:02.159275+0000","last_became_active":"2024-05-08T22:40:00.774654+0000","last_became_peered":"2024-05-08T22:40:00.774654+0000","last_unstale":"2024-05-08T22:40:02.159275+0000","last_undegraded":"2024-05-08T22:40:02.159275+0000","last_fullsized":"2024-05-08T22:40:02.159275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:46.623988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779302+0000","last_change":"2024-05-08T22:40:01.169734+0000","last_active":"2024-05-08T22:40:02.779302+0000","last_peered":"2024-05-08T22:40:02.779302+0000","last_clean":"2024-05-08T22:40:02.779302+0000","last_became_active":"2024-05-08T22:40:01.169463+0000","last_became_peered":"2024-05-08T22:40:01.169463+0000","last_unstale":"2024-05-08T22:40:02.779302+0000","last_undegraded":"2024-05-08T22:40:02.779302+0000","last_fullsized":"2024-05-08T22:40:02.779302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:31.192463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777997+0000","last_change":"2024-05-08T22:40:00.781468+0000","last_active":"2024-05-08T22:40:02.777997+0000","last_peered":"2024-05-08T22:40:02.777997+0000","last_clean":"2024-05-08T22:40:02.777997+0000","last_became_active":"2024-05-08T22:40:00.781113+0000","last_became_peered":"2024-05-08T22:40:00.781113+0000","last_unstale":"2024-05-08T22:40:02.777997+0000","last_undegraded":"2024-05-08T22:40:02.777997+0000","last_fullsized":"2024-05-08T22:40:02.777997+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:06.805376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778432+0000","last_change":"2024-05-08T22:40:01.166800+0000","last_active":"2024-05-08T22:40:02.778432+0000","last_peered":"2024-05-08T22:40:02.778432+0000","last_clean":"2024-05-08T22:40:02.778432+0000","last_became_active":"2024-05-08T22:40:01.166325+0000","last_became_peered":"2024-05-08T22:40:01.166325+0000","last_unstale":"2024-05-08T22:40:02.778432+0000","last_undegraded":"2024-05-08T22:40:02.778432+0000","last_fullsized":"2024-05-08T22:40:02.778432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:56.095117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778410+0000","last_change":"2024-05-08T22:40:00.787240+0000","last_active":"2024-05-08T22:40:02.778410+0000","last_peered":"2024-05-08T22:40:02.778410+0000","last_clean":"2024-05-08T22:40:02.778410+0000","last_became_active":"2024-05-08T22:40:00.786988+0000","last_became_peered":"2024-05-08T22:40:00.786988+0000","last_unstale":"2024-05-08T22:40:02.778410+0000","last_undegraded":"2024-05-08T22:40:02.778410+0000","last_fullsized":"2024-05-08T22:40:02.778410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:44:01.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157449+0000","last_change":"2024-05-08T22:40:01.157968+0000","last_active":"2024-05-08T22:40:02.157449+0000","last_peered":"2024-05-08T22:40:02.157449+0000","last_clean":"2024-05-08T22:40:02.157449+0000","last_became_active":"2024-05-08T22:40:01.157643+0000","last_became_peered":"2024-05-08T22:40:01.157643+0000","last_unstale":"2024-05-08T22:40:02.157449+0000","last_undegraded":"2024-05-08T22:40:02.157449+0000","last_fullsized":"2024-05-08T22:40:02.157449+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:44.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777890+0000","last_change":"2024-05-08T22:40:00.769475+0000","last_active":"2024-05-08T22:40:02.777890+0000","last_peered":"2024-05-08T22:40:02.777890+0000","last_clean":"2024-05-08T22:40:02.777890+0000","last_became_active":"2024-05-08T22:40:00.769190+0000","last_became_peered":"2024-05-08T22:40:00.769190+0000","last_unstale":"2024-05-08T22:40:02.777890+0000","last_undegraded":"2024-05-08T22:40:02.777890+0000","last_fullsized":"2024-05-08T22:40:02.777890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:15.028535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158273+0000","last_change":"2024-05-08T22:40:01.167324+0000","last_active":"2024-05-08T22:40:02.158273+0000","last_peered":"2024-05-08T22:40:02.158273+0000","last_clean":"2024-05-08T22:40:02.158273+0000","last_became_active":"2024-05-08T22:40:01.167051+0000","last_became_peered":"2024-05-08T22:40:01.167051+0000","last_unstale":"2024-05-08T22:40:02.158273+0000","last_undegraded":"2024-05-08T22:40:02.158273+0000","last_fullsized":"2024-05-08T22:40:02.158273+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:08.376851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778714+0000","last_change":"2024-05-08T22:40:00.781211+0000","last_active":"2024-05-08T22:40:02.778714+0000","last_peered":"2024-05-08T22:40:02.778714+0000","last_clean":"2024-05-08T22:40:02.778714+0000","last_became_active":"2024-05-08T22:40:00.780958+0000","last_became_peered":"2024-05-08T22:40:00.780958+0000","last_unstale":"2024-05-08T22:40:02.778714+0000","last_undegraded":"2024-05-08T22:40:02.778714+0000","last_fullsized":"2024-05-08T22:40:02.778714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:57:46.542422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158174+0000","last_change":"2024-05-08T22:40:01.173627+0000","last_active":"2024-05-08T22:40:02.158174+0000","last_peered":"2024-05-08T22:40:02.158174+0000","last_clean":"2024-05-08T22:40:02.158174+0000","last_became_active":"2024-05-08T22:40:01.173347+0000","last_became_peered":"2024-05-08T22:40:01.173347+0000","last_unstale":"2024-05-08T22:40:02.158174+0000","last_undegraded":"2024-05-08T22:40:02.158174+0000","last_fullsized":"2024-05-08T22:40:02.158174+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:12.709608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"95'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778561+0000","last_change":"2024-05-08T22:40:00.782692+0000","last_active":"2024-05-08T22:40:02.778561+0000","last_peered":"2024-05-08T22:40:02.778561+0000","last_clean":"2024-05-08T22:40:02.778561+0000","last_became_active":"2024-05-08T22:40:00.782400+0000","last_became_peered":"2024-05-08T22:40:00.782400+0000","last_unstale":"2024-05-08T22:40:02.778561+0000","last_undegraded":"2024-05-08T22:40:02.778561+0000","last_fullsized":"2024-05-08T22:40:02.778561+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:14.003045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777880+0000","last_change":"2024-05-08T22:40:01.181440+0000","last_active":"2024-05-08T22:40:02.777880+0000","last_peered":"2024-05-08T22:40:02.777880+0000","last_clean":"2024-05-08T22:40:02.777880+0000","last_became_active":"2024-05-08T22:40:01.181181+0000","last_became_peered":"2024-05-08T22:40:01.181181+0000","last_unstale":"2024-05-08T22:40:02.777880+0000","last_undegraded":"2024-05-08T22:40:02.777880+0000","last_fullsized":"2024-05-08T22:40:02.777880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:23.375670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158700+0000","last_change":"2024-05-08T22:40:00.769915+0000","last_active":"2024-05-08T22:40:02.158700+0000","last_peered":"2024-05-08T22:40:02.158700+0000","last_clean":"2024-05-08T22:40:02.158700+0000","last_became_active":"2024-05-08T22:40:00.769657+0000","last_became_peered":"2024-05-08T22:40:00.769657+0000","last_unstale":"2024-05-08T22:40:02.158700+0000","last_undegraded":"2024-05-08T22:40:02.158700+0000","last_fullsized":"2024-05-08T22:40:02.158700+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:49:10.561141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158920+0000","last_change":"2024-05-08T22:40:01.158895+0000","last_active":"2024-05-08T22:40:02.158920+0000","last_peered":"2024-05-08T22:40:02.158920+0000","last_clean":"2024-05-08T22:40:02.158920+0000","last_became_active":"2024-05-08T22:40:01.158575+0000","last_became_peered":"2024-05-08T22:40:01.158575+0000","last_unstale":"2024-05-08T22:40:02.158920+0000","last_undegraded":"2024-05-08T22:40:02.158920+0000","last_fullsized":"2024-05-08T22:40:02.158920+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:10.201272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778766+0000","last_change":"2024-05-08T22:40:00.772430+0000","last_active":"2024-05-08T22:40:02.778766+0000","last_peered":"2024-05-08T22:40:02.778766+0000","last_clean":"2024-05-08T22:40:02.778766+0000","last_became_active":"2024-05-08T22:40:00.772183+0000","last_became_peered":"2024-05-08T22:40:00.772183+0000","last_unstale":"2024-05-08T22:40:02.778766+0000","last_undegraded":"2024-05-08T22:40:02.778766+0000","last_fullsized":"2024-05-08T22:40:02.778766+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159328+0000","last_change":"2024-05-08T22:40:01.160519+0000","last_active":"2024-05-08T22:40:02.159328+0000","last_peered":"2024-05-08T22:40:02.159328+0000","last_clean":"2024-05-08T22:40:02.159328+0000","last_became_active":"2024-05-08T22:40:01.160271+0000","last_became_peered":"2024-05-08T22:40:01.160271+0000","last_unstale":"2024-05-08T22:40:02.159328+0000","last_undegraded":"2024-05-08T22:40:02.159328+0000","last_fullsized":"2024-05-08T22:40:02.159328+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:36:04.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159206+0000","last_change":"2024-05-08T22:40:00.782143+0000","last_active":"2024-05-08T22:40:02.159206+0000","last_peered":"2024-05-08T22:40:02.159206+0000","last_clean":"2024-05-08T22:40:02.159206+0000","last_became_active":"2024-05-08T22:40:00.781867+0000","last_became_peered":"2024-05-08T22:40:00.781867+0000","last_unstale":"2024-05-08T22:40:02.159206+0000","last_undegraded":"2024-05-08T22:40:02.159206+0000","last_fullsized":"2024-05-08T22:40:02.159206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777842+0000","last_change":"2024-05-08T22:40:01.171579+0000","last_active":"2024-05-08T22:40:02.777842+0000","last_peered":"2024-05-08T22:40:02.777842+0000","last_clean":"2024-05-08T22:40:02.777842+0000","last_became_active":"2024-05-08T22:40:01.170571+0000","last_became_peered":"2024-05-08T22:40:01.170571+0000","last_unstale":"2024-05-08T22:40:02.777842+0000","last_undegraded":"2024-05-08T22:40:02.777842+0000","last_fullsized":"2024-05-08T22:40:02.777842+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:24.889483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776286+0000","last_change":"2024-05-08T22:40:00.775780+0000","last_active":"2024-05-08T22:40:02.776286+0000","last_peered":"2024-05-08T22:40:02.776286+0000","last_clean":"2024-05-08T22:40:02.776286+0000","last_became_active":"2024-05-08T22:40:00.775350+0000","last_became_peered":"2024-05-08T22:40:00.775350+0000","last_unstale":"2024-05-08T22:40:02.776286+0000","last_undegraded":"2024-05-08T22:40:02.776286+0000","last_fullsized":"2024-05-08T22:40:02.776286+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:32.117465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776225+0000","last_change":"2024-05-08T22:40:01.169014+0000","last_active":"2024-05-08T22:40:02.776225+0000","last_peered":"2024-05-08T22:40:02.776225+0000","last_clean":"2024-05-08T22:40:02.776225+0000","last_became_active":"2024-05-08T22:40:01.168647+0000","last_became_peered":"2024-05-08T22:40:01.168647+0000","last_unstale":"2024-05-08T22:40:02.776225+0000","last_undegraded":"2024-05-08T22:40:02.776225+0000","last_fullsized":"2024-05-08T22:40:02.776225+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:29.344124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158374+0000","last_change":"2024-05-08T22:40:00.784050+0000","last_active":"2024-05-08T22:40:02.158374+0000","last_peered":"2024-05-08T22:40:02.158374+0000","last_clean":"2024-05-08T22:40:02.158374+0000","last_became_active":"2024-05-08T22:40:00.783815+0000","last_became_peered":"2024-05-08T22:40:00.783815+0000","last_unstale":"2024-05-08T22:40:02.158374+0000","last_undegraded":"2024-05-08T22:40:02.158374+0000","last_fullsized":"2024-05-08T22:40:02.158374+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:59.988151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776025+0000","last_change":"2024-05-08T22:40:01.165354+0000","last_active":"2024-05-08T22:40:02.776025+0000","last_peered":"2024-05-08T22:40:02.776025+0000","last_clean":"2024-05-08T22:40:02.776025+0000","last_became_active":"2024-05-08T22:40:01.165075+0000","last_became_peered":"2024-05-08T22:40:01.165075+0000","last_unstale":"2024-05-08T22:40:02.776025+0000","last_undegraded":"2024-05-08T22:40:02.776025+0000","last_fullsized":"2024-05-08T22:40:02.776025+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:46:02.769323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776860+0000","last_change":"2024-05-08T22:40:00.774648+0000","last_active":"2024-05-08T22:40:02.776860+0000","last_peered":"2024-05-08T22:40:02.776860+0000","last_clean":"2024-05-08T22:40:02.776860+0000","last_became_active":"2024-05-08T22:40:00.774301+0000","last_became_peered":"2024-05-08T22:40:00.774301+0000","last_unstale":"2024-05-08T22:40:02.776860+0000","last_undegraded":"2024-05-08T22:40:02.776860+0000","last_fullsized":"2024-05-08T22:40:02.776860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:11.152944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777758+0000","last_change":"2024-05-08T22:40:01.173266+0000","last_active":"2024-05-08T22:40:02.777758+0000","last_peered":"2024-05-08T22:40:02.777758+0000","last_clean":"2024-05-08T22:40:02.777758+0000","last_became_active":"2024-05-08T22:40:01.173029+0000","last_became_peered":"2024-05-08T22:40:01.173029+0000","last_unstale":"2024-05-08T22:40:02.777758+0000","last_undegraded":"2024-05-08T22:40:02.777758+0000","last_fullsized":"2024-05-08T22:40:02.777758+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:05.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157944+0000","last_change":"2024-05-08T22:40:00.772913+0000","last_active":"2024-05-08T22:40:02.157944+0000","last_peered":"2024-05-08T22:40:02.157944+0000","last_clean":"2024-05-08T22:40:02.157944+0000","last_became_active":"2024-05-08T22:40:00.772489+0000","last_became_peered":"2024-05-08T22:40:00.772489+0000","last_unstale":"2024-05-08T22:40:02.157944+0000","last_undegraded":"2024-05-08T22:40:02.157944+0000","last_fullsized":"2024-05-08T22:40:02.157944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:43.832037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158647+0000","last_change":"2024-05-08T22:40:01.154094+0000","last_active":"2024-05-08T22:40:02.158647+0000","last_peered":"2024-05-08T22:40:02.158647+0000","last_clean":"2024-05-08T22:40:02.158647+0000","last_became_active":"2024-05-08T22:40:01.153749+0000","last_became_peered":"2024-05-08T22:40:01.153749+0000","last_unstale":"2024-05-08T22:40:02.158647+0000","last_undegraded":"2024-05-08T22:40:02.158647+0000","last_fullsized":"2024-05-08T22:40:02.158647+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:20.847730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779036+0000","last_change":"2024-05-08T22:40:00.771029+0000","last_active":"2024-05-08T22:40:02.779036+0000","last_peered":"2024-05-08T22:40:02.779036+0000","last_clean":"2024-05-08T22:40:02.779036+0000","last_became_active":"2024-05-08T22:40:00.770723+0000","last_became_peered":"2024-05-08T22:40:00.770723+0000","last_unstale":"2024-05-08T22:40:02.779036+0000","last_undegraded":"2024-05-08T22:40:02.779036+0000","last_fullsized":"2024-05-08T22:40:02.779036+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:46.623988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778178+0000","last_change":"2024-05-08T22:40:01.178824+0000","last_active":"2024-05-08T22:40:02.778178+0000","last_peered":"2024-05-08T22:40:02.778178+0000","last_clean":"2024-05-08T22:40:02.778178+0000","last_became_active":"2024-05-08T22:40:01.176113+0000","last_became_peered":"2024-05-08T22:40:01.176113+0000","last_unstale":"2024-05-08T22:40:02.778178+0000","last_undegraded":"2024-05-08T22:40:02.778178+0000","last_fullsized":"2024-05-08T22:40:02.778178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:14.063300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159423+0000","last_change":"2024-05-08T22:40:00.772227+0000","last_active":"2024-05-08T22:40:02.159423+0000","last_peered":"2024-05-08T22:40:02.159423+0000","last_clean":"2024-05-08T22:40:02.159423+0000","last_became_active":"2024-05-08T22:40:00.771988+0000","last_became_peered":"2024-05-08T22:40:00.771988+0000","last_unstale":"2024-05-08T22:40:02.159423+0000","last_undegraded":"2024-05-08T22:40:02.159423+0000","last_fullsized":"2024-05-08T22:40:02.159423+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:29.970913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.778850+0000","last_change":"2024-05-08T22:40:01.172783+0000","last_active":"2024-05-08T22:40:02.778850+0000","last_peered":"2024-05-08T22:40:02.778850+0000","last_clean":"2024-05-08T22:40:02.778850+0000","last_became_active":"2024-05-08T22:40:01.172550+0000","last_became_peered":"2024-05-08T22:40:01.172550+0000","last_unstale":"2024-05-08T22:40:02.778850+0000","last_undegraded":"2024-05-08T22:40:02.778850+0000","last_fullsized":"2024-05-08T22:40:02.778850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:11.272797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.159320+0000","last_change":"2024-05-08T22:40:00.774935+0000","last_active":"2024-05-08T22:40:02.159320+0000","last_peered":"2024-05-08T22:40:02.159320+0000","last_clean":"2024-05-08T22:40:02.159320+0000","last_became_active":"2024-05-08T22:40:00.774334+0000","last_became_peered":"2024-05-08T22:40:00.774334+0000","last_unstale":"2024-05-08T22:40:02.159320+0000","last_undegraded":"2024-05-08T22:40:02.159320+0000","last_fullsized":"2024-05-08T22:40:02.159320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:34.090155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777396+0000","last_change":"2024-05-08T22:40:01.168627+0000","last_active":"2024-05-08T22:40:02.777396+0000","last_peered":"2024-05-08T22:40:02.777396+0000","last_clean":"2024-05-08T22:40:02.777396+0000","last_became_active":"2024-05-08T22:40:01.167471+0000","last_became_peered":"2024-05-08T22:40:01.167471+0000","last_unstale":"2024-05-08T22:40:02.777396+0000","last_undegraded":"2024-05-08T22:40:02.777396+0000","last_fullsized":"2024-05-08T22:40:02.777396+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.902752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158850+0000","last_change":"2024-05-08T22:40:00.772765+0000","last_active":"2024-05-08T22:40:02.158850+0000","last_peered":"2024-05-08T22:40:02.158850+0000","last_clean":"2024-05-08T22:40:02.158850+0000","last_became_active":"2024-05-08T22:40:00.772512+0000","last_became_peered":"2024-05-08T22:40:00.772512+0000","last_unstale":"2024-05-08T22:40:02.158850+0000","last_undegraded":"2024-05-08T22:40:02.158850+0000","last_fullsized":"2024-05-08T22:40:02.158850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:24.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.157892+0000","last_change":"2024-05-08T22:40:01.179700+0000","last_active":"2024-05-08T22:40:02.157892+0000","last_peered":"2024-05-08T22:40:02.157892+0000","last_clean":"2024-05-08T22:40:02.157892+0000","last_became_active":"2024-05-08T22:40:01.179428+0000","last_became_peered":"2024-05-08T22:40:01.179428+0000","last_unstale":"2024-05-08T22:40:02.157892+0000","last_undegraded":"2024-05-08T22:40:02.157892+0000","last_fullsized":"2024-05-08T22:40:02.157892+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:28.955691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777914+0000","last_change":"2024-05-08T22:40:00.778567+0000","last_active":"2024-05-08T22:40:02.777914+0000","last_peered":"2024-05-08T22:40:02.777914+0000","last_clean":"2024-05-08T22:40:02.777914+0000","last_became_active":"2024-05-08T22:40:00.778286+0000","last_became_peered":"2024-05-08T22:40:00.778286+0000","last_unstale":"2024-05-08T22:40:02.777914+0000","last_undegraded":"2024-05-08T22:40:02.777914+0000","last_fullsized":"2024-05-08T22:40:02.777914+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:00.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.770276+0000","last_change":"2024-05-08T22:40:01.168916+0000","last_active":"2024-05-08T22:40:02.770276+0000","last_peered":"2024-05-08T22:40:02.770276+0000","last_clean":"2024-05-08T22:40:02.770276+0000","last_became_active":"2024-05-08T22:40:01.167869+0000","last_became_peered":"2024-05-08T22:40:01.167869+0000","last_unstale":"2024-05-08T22:40:02.770276+0000","last_undegraded":"2024-05-08T22:40:02.770276+0000","last_fullsized":"2024-05-08T22:40:02.770276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:16.336469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.776268+0000","last_change":"2024-05-08T22:40:00.775741+0000","last_active":"2024-05-08T22:40:02.776268+0000","last_peered":"2024-05-08T22:40:02.776268+0000","last_clean":"2024-05-08T22:40:02.776268+0000","last_became_active":"2024-05-08T22:40:00.775321+0000","last_became_peered":"2024-05-08T22:40:00.775321+0000","last_unstale":"2024-05-08T22:40:02.776268+0000","last_undegraded":"2024-05-08T22:40:02.776268+0000","last_fullsized":"2024-05-08T22:40:02.776268+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:20.359593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158379+0000","last_change":"2024-05-08T22:40:01.160276+0000","last_active":"2024-05-08T22:40:02.158379+0000","last_peered":"2024-05-08T22:40:02.158379+0000","last_clean":"2024-05-08T22:40:02.158379+0000","last_became_active":"2024-05-08T22:40:01.159983+0000","last_became_peered":"2024-05-08T22:40:01.159983+0000","last_unstale":"2024-05-08T22:40:02.158379+0000","last_undegraded":"2024-05-08T22:40:02.158379+0000","last_fullsized":"2024-05-08T22:40:02.158379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:07.802805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.158311+0000","last_change":"2024-05-08T22:40:00.772313+0000","last_active":"2024-05-08T22:40:02.158311+0000","last_peered":"2024-05-08T22:40:02.158311+0000","last_clean":"2024-05-08T22:40:02.158311+0000","last_became_active":"2024-05-08T22:40:00.772060+0000","last_became_peered":"2024-05-08T22:40:00.772060+0000","last_unstale":"2024-05-08T22:40:02.158311+0000","last_undegraded":"2024-05-08T22:40:02.158311+0000","last_fullsized":"2024-05-08T22:40:02.158311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:11.332444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777812+0000","last_change":"2024-05-08T22:40:01.171539+0000","last_active":"2024-05-08T22:40:02.777812+0000","last_peered":"2024-05-08T22:40:02.777812+0000","last_clean":"2024-05-08T22:40:02.777812+0000","last_became_active":"2024-05-08T22:40:01.170634+0000","last_became_peered":"2024-05-08T22:40:01.170634+0000","last_unstale":"2024-05-08T22:40:02.777812+0000","last_undegraded":"2024-05-08T22:40:02.777812+0000","last_fullsized":"2024-05-08T22:40:02.777812+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:55.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.777731+0000","last_change":"2024-05-08T22:40:00.777821+0000","last_active":"2024-05-08T22:40:02.777731+0000","last_peered":"2024-05-08T22:40:02.777731+0000","last_clean":"2024-05-08T22:40:02.777731+0000","last_became_active":"2024-05-08T22:40:00.777599+0000","last_became_peered":"2024-05-08T22:40:00.777599+0000","last_unstale":"2024-05-08T22:40:02.777731+0000","last_undegraded":"2024-05-08T22:40:02.777731+0000","last_fullsized":"2024-05-08T22:40:02.777731+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:09.000232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779054+0000","last_change":"2024-05-08T22:40:01.162842+0000","last_active":"2024-05-08T22:40:02.779054+0000","last_peered":"2024-05-08T22:40:02.779054+0000","last_clean":"2024-05-08T22:40:02.779054+0000","last_became_active":"2024-05-08T22:40:01.162571+0000","last_became_peered":"2024-05-08T22:40:01.162571+0000","last_unstale":"2024-05-08T22:40:02.779054+0000","last_undegraded":"2024-05-08T22:40:02.779054+0000","last_fullsized":"2024-05-08T22:40:02.779054+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:40:00.750923+0000","last_clean_scrub_stamp":"2024-05-08T22:40:00.750923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:16.031733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-08T22:40:02.779585+0000","last_change":"2024-05-08T22:40:00.782239+0000","last_active":"2024-05-08T22:40:02.779585+0000","last_peered":"2024-05-08T22:40:02.779585+0000","last_clean":"2024-05-08T22:40:02.779585+0000","last_became_active":"2024-05-08T22:40:00.781994+0000","last_became_peered":"2024-05-08T22:40:00.781994+0000","last_unstale":"2024-05-08T22:40:02.779585+0000","last_undegraded":"2024-05-08T22:40:02.779585+0000","last_fullsized":"2024-05-08T22:40:02.779585+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:39:59.747725+0000","last_clean_scrub_stamp":"2024-05-08T22:39:59.747725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:10.032705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-08T22:40:01.147393+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:40:01.147393+0000","last_peered":"2024-05-08T22:40:01.147393+0000","last_clean":"2024-05-08T22:40:01.147393+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:40:01.147393+0000","last_undegraded":"2024-05-08T22:40:01.147393+0000","last_fullsized":"2024-05-08T22:40:01.147393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":536547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.5620000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.50700000000000001,"5min":0.309,"15min":0.307},"max":{"1min":1.214,"5min":1.214,"15min":1.6899999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.61099999999999999},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.35799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97699999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":2,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.75800000000000001,"15min":0.748},"min":{"1min":0.60699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1459999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0149999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.97999999999999998}]},{"osd":4,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.91500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2170000000000001,"5min":1.306,"15min":1.306},"last":1.01},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.95999999999999996,"15min":0.94599999999999995},"min":{"1min":0.77200000000000002,"5min":0.64600000000000002,"15min":0.64100000000000001},"max":{"1min":1.2809999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.044}]},{"osd":5,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.98399999999999999,"15min":0.97199999999999998},"min":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.68300000000000005},"max":{"1min":1.3100000000000001,"5min":1.339,"15min":1.339},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.006,"15min":1.026},"min":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.179}]},{"osd":6,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.86299999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.163,"5min":1.446,"15min":1.446},"last":1.359},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.0600000000000001},"min":{"1min":0.83699999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3300000000000001,"5min":1.3480000000000001,"15min":1.3580000000000001},"last":1.25}]},{"osd":7,"last update":"Wed May 8 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.081,"15min":1.1020000000000001},"min":{"1min":0.755,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.552,"5min":1.6779999999999999,"15min":1.7070000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0489999999999999,"15min":1.038},"min":{"1min":0.84699999999999998,"5min":0.622,"15min":0.61699999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.8360000000000001},"last":1.161}]}]},{"osd":1,"up_from":15,"seq":64424509527,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1179648,"data_stored":536617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.42299999999999999,"15min":0.38500000000000001},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":1.23,"15min":1.23},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65100000000000002,"15min":0.69199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85799999999999998,"5min":1.0029999999999999,"15min":1.04},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76200000000000001,"15min":0.73999999999999999},"min":{"1min":0.54800000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.113,"5min":1.1499999999999999,"15min":1.1779999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":0.88600000000000001,"5min":1.583,"15min":1.583},"last":0.67700000000000005}]},{"osd":4,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.92700000000000005,"15min":0.96999999999999997},"min":{"1min":0.745,"5min":0.629,"15min":0.629},"max":{"1min":1.2949999999999999,"5min":1.333,"15min":1.431},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.97399999999999998,"15min":0.97199999999999998},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4730000000000001,"5min":1.55,"15min":1.55},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92700000000000005,"15min":0.94999999999999996},"min":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.153,"5min":1.5,"15min":1.5},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":1.075},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3240000000000001,"5min":1.605,"15min":1.726},"last":0.83399999999999996}]},{"osd":6,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96499999999999997,"15min":0.98899999999999999},"min":{"1min":0.74199999999999999,"5min":0.72399999999999998,"15min":0.60899999999999999},"max":{"1min":1.274,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.877},{"interface":"front","average":{"1min":1.048,"5min":1.0549999999999999,"15min":1.101},"min":{"1min":0.89900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.24,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.401,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0409999999999999,"15min":1.0580000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.65800000000000003},"max":{"1min":1.3120000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0389999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1777664,"data_stored":1126971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85799999999999998,"15min":0.82999999999999996},"min":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0920000000000001,"5min":1.282,"15min":1.282},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.68200000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.78400000000000003}]},{"osd":1,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91100000000000003,"15min":0.90800000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89200000000000002,"15min":0.86699999999999999},"min":{"1min":0.63600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1850000000000001,"5min":1.488,"15min":1.488},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86799999999999999,"15min":0.90800000000000003},"min":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.04},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95299999999999996,"15min":0.96299999999999997},"min":{"1min":0.66300000000000003,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.337},"last":0.91300000000000003}]},{"osd":3,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92800000000000005,"15min":0.91200000000000003},"min":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3919999999999999,"5min":1.5269999999999999,"15min":1.5589999999999999},"last":1.081},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.86799999999999999,"15min":0.84699999999999998},"min":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.2290000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.94399999999999995,"15min":0.96699999999999997},"min":{"1min":0.64100000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.92000000000000004,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"max":{"1min":1.103,"5min":1.482,"15min":1.482},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0329999999999999,"15min":1.048},"min":{"1min":0.71299999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed May 8 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.70599999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1040000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.044,"15min":1.0800000000000001},"min":{"1min":0.80000000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":1.599,"15min":1.599},"last":1.111}]}]},{"osd":6,"up_from":14,"seq":60129542229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1196032,"data_stored":538715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.38200000000000001},"max":{"1min":1.0940000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80300000000000005,"15min":0.84899999999999998},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.038,"5min":1.0669999999999999,"15min":1.141},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80600000000000005,"15min":0.86499999999999999},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95299999999999996,"5min":1.3300000000000001,"15min":1.758},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90800000000000003,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1499999999999999,"5min":1.4790000000000001,"15min":1.8089999999999999},"last":1.2070000000000001}]},{"osd":2,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.97199999999999998},"min":{"1min":0.64100000000000001,"5min":0.55700000000000005,"15min":0.49099999999999999},"max":{"1min":1.167,"5min":1.375,"15min":1.4319999999999999},"last":1.165},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.92300000000000004},"min":{"1min":0.66900000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.21,"5min":1.397,"15min":1.419},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.97399999999999998,"15min":1.0760000000000001},"min":{"1min":0.72699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1910000000000001,"5min":1.3580000000000001,"15min":3.2040000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90700000000000003,"15min":0.97699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.1120000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93300000000000005,"15min":0.98199999999999998},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1970000000000001,"5min":1.538,"15min":1.538},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.92700000000000005,"15min":1.026},"min":{"1min":0.629,"5min":0.53200000000000003,"15min":0.44400000000000001},"max":{"1min":1.2050000000000001,"5min":1.4430000000000001,"15min":1.458},"last":0.73299999999999998}]},{"osd":5,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.038,"15min":1.123},"min":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2549999999999999,"5min":1.528,"15min":1.681},"last":1.117},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.1259999999999999},"min":{"1min":0.93899999999999995,"5min":0.68000000000000005,"15min":0.66000000000000003},"max":{"1min":1.177,"5min":1.466,"15min":1.6000000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94699999999999995,"15min":1.0629999999999999},"min":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.091,"5min":1.4330000000000001,"15min":1.835},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1200000000000001,"15min":1.2150000000000001},"min":{"1min":0.748,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5600000000000001,"5min":1.6970000000000001,"15min":2.2999999999999998},"last":0.98299999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":536547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83499999999999996,"15min":0.81299999999999994},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.33000000000000002},"max":{"1min":1.226,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84199999999999997,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.53400000000000003,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.52,"15min":1.52},"last":1.119}]},{"osd":1,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.56100000000000005,"15min":0.35199999999999998},"max":{"1min":1.1519999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.85599999999999998,"15min":0.81299999999999994},"min":{"1min":0.70799999999999996,"5min":0.57299999999999995,"15min":0.44400000000000001},"max":{"1min":1.1519999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.96799999999999997}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90900000000000003,"15min":0.88100000000000001},"min":{"1min":0.62,"5min":0.51500000000000001,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.90600000000000003,"15min":0.85599999999999998},"min":{"1min":0.67000000000000004,"5min":0.57499999999999996,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.89800000000000002,"15min":0.874},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.435},"max":{"1min":1.1379999999999999,"5min":1.298,"15min":1.298},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.97299999999999998,"15min":0.91400000000000003},"min":{"1min":0.67500000000000004,"5min":0.63900000000000001,"15min":0.46300000000000002},"max":{"1min":1.1779999999999999,"5min":1.349,"15min":1.349},"last":0.875},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98899999999999999,"15min":0.92300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.1719999999999999,"5min":1.605,"15min":1.605},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.0740000000000001,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.69899999999999995,"15min":0.54900000000000004},"max":{"1min":1.2330000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.997}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.81399999999999995,"5min":0.71399999999999997,"15min":0.57299999999999995},"max":{"1min":1.282,"5min":1.538,"15min":1.538},"last":1.024},{"interface":"front","average":{"1min":1.012,"5min":1.095,"15min":1.0309999999999999},"min":{"1min":0.83699999999999997,"5min":0.81000000000000005,"15min":0.53400000000000003},"max":{"1min":1.2569999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.84199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542229,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1183744,"data_stored":536707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81299999999999994,"15min":0.83299999999999996},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.083,"5min":1.242,"15min":1.28},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1519999999999999,"5min":1.294,"15min":1.294},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.85899999999999999,"15min":0.86699999999999999},"min":{"1min":0.623,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0920000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.91400000000000003,"15min":0.88800000000000001},"min":{"1min":0.63200000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.80000000000000004}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84499999999999997,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.224,"15min":1.29},"last":1.093},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.91100000000000003,"15min":0.93100000000000005},"min":{"1min":0.71299999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1779999999999999,"5min":1.351,"15min":1.637},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93500000000000005,"15min":1.022},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60599999999999998},"max":{"1min":1.159,"5min":1.337,"15min":2.601},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95199999999999996,"15min":1.0780000000000001},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.4259999999999999,"15min":2.8769999999999998},"last":0.876}]},{"osd":5,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.792},"last":0.622},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96399999999999997,"15min":1.0149999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.177,"5min":1.3839999999999999,"15min":1.575},"last":1.1439999999999999}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.2390000000000001,"5min":1.377,"15min":1.831},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.0600000000000001,"15min":1.091},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3129999999999999,"5min":1.347,"15min":1.8899999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0189999999999999,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.2290000000000001,"5min":1.5169999999999999,"15min":1.8500000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.135,"15min":1.169},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.76000000000000001},"max":{"1min":1.2749999999999999,"5min":1.5169999999999999,"15min":1.944},"last":0.91600000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":536547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":1.0369999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.39700000000000002,"15min":0.39300000000000002},"max":{"1min":1.022,"5min":1.133,"15min":1.2190000000000001},"last":0.45300000000000001}]},{"osd":1,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.66100000000000003},"min":{"1min":0.41799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.53400000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93999999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 22:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.56200000000000006,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.071,"15min":1.071},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.59799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96499999999999997,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94699999999999995,"15min":0.91900000000000004},"min":{"1min":0.78800000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2310000000000001,"5min":1.504,"15min":1.504},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.99299999999999999,"15min":1.0229999999999999},"min":{"1min":0.81899999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.157,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.91500000000000004}]},{"osd":5,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.01,"15min":1.0329999999999999},"min":{"1min":0.76700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.3580000000000001,"5min":1.407,"15min":1.484},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.208,"5min":1.456,"15min":1.456},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.042,"15min":1.0580000000000001},"min":{"1min":0.76400000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2010000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.075,"15min":1.0860000000000001},"min":{"1min":0.879,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.008}]},{"osd":7,"last update":"Wed May 8 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.073,"15min":1.081},"min":{"1min":0.73199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.345,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.89000000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.288,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.034}]}]},{"osd":0,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1794048,"data_stored":1129185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.62,"15min":0.67100000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":1.012,"15min":1.617},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.62,"15min":0.69999999999999996},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.33600000000000002},"max":{"1min":0.91100000000000003,"5min":1.113,"15min":1.5129999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.754,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.45700000000000002,"15min":0.44500000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.446},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.97799999999999998,"5min":1.4410000000000001,"15min":1.6910000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed May 8 22:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.80400000000000005},"min":{"1min":0.49299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89900000000000002,"5min":1.609,"15min":1.714},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.621}]},{"osd":4,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.93500000000000005,"15min":0.90600000000000003},"min":{"1min":0.71999999999999997,"5min":0.66000000000000003,"15min":0.61699999999999999},"max":{"1min":1.1930000000000001,"5min":4.391,"15min":4.391},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":0.995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3169999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.97399999999999998}]},{"osd":5,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.89600000000000002,"15min":0.91000000000000003},"min":{"1min":0.71599999999999997,"5min":0.65500000000000003,"15min":0.64600000000000002},"max":{"1min":1.052,"5min":1.419,"15min":1.419},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.97099999999999997,"15min":0.997},"min":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.423,"5min":1.423,"15min":1.474},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.68600000000000005},"max":{"1min":1.1619999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.036,"5min":1.048,"15min":1.0569999999999999},"min":{"1min":0.85799999999999998,"5min":0.72699999999999998,"15min":0.66300000000000003},"max":{"1min":1.474,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":0.95099999999999996}]},{"osd":7,"last update":"Wed May 8 22:39:18 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.054,"15min":1.0860000000000001},"min":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.72199999999999998},"max":{"1min":1.2509999999999999,"5min":1.532,"15min":1.6459999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.024},"min":{"1min":0.88,"5min":0.74299999999999999,"15min":0.60399999999999998},"max":{"1min":1.2150000000000001,"5min":1.365,"15min":1.4470000000000001},"last":0.995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:40:05.567 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:40:05.568 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 Processor -- start 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 -- start start 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcba415db00 0x7fcba406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcba415dee0 0x7fcba40cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40cd3c0 0x7fcba40b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba40b1d30 con 0x7fcba415dee0 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba406b110 con 0x7fcba415db00 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaaf88700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b19e0 con 0x7fcba40cd3c0 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcba415dee0 0x7fcba40cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcba415dee0 0x7fcba40cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39992/0 (socket says 172.21.15.179:39992) 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 -- 172.21.15.179:0/368266032 learned_addr learned my addr 172.21.15.179:0/368266032 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcbaa787700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40cd3c0 0x7fcba40b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9f86700 1 --2- 172.21.15.179:0/368266032 >> v2:172.21.15.195:3300/0 conn(0x7fcba415db00 0x7fcba406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 -- 172.21.15.179:0/368266032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40cd3c0 msgr2=0x7fcba40b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40cd3c0 0x7fcba40b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 -- 172.21.15.179:0/368266032 >> v2:172.21.15.195:3300/0 conn(0x7fcba415db00 msgr2=0x7fcba406cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 --2- 172.21.15.179:0/368266032 >> v2:172.21.15.195:3300/0 conn(0x7fcba415db00 0x7fcba406cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.697+0000 7fcba9785700 1 -- 172.21.15.179:0/368266032 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4157cf0 con 0x7fcba415dee0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcba9785700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba415dee0 0x7fcba40cce80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009910 tx=0x7fcba0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512a9fcfc61afbfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcba8f84700 1 -- 172.21.15.179:0/368266032 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011830 con 0x7fcba415dee0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcba8f84700 1 -- 172.21.15.179:0/368266032 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0011990 con 0x7fcba415dee0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcba8f84700 1 -- 172.21.15.179:0/368266032 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011d40 con 0x7fcba415dee0 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- 172.21.15.179:0/368266032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba415dee0 msgr2=0x7fcba40cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba415dee0 0x7fcba40cce80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009910 tx=0x7fcba0009c20 comp rx=0 tx=0).stop 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- 172.21.15.179:0/368266032 shutdown_connections 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba40cd3c0 0x7fcba40b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/368266032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba415dee0 0x7fcba40cce80 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/368266032 >> v2:172.21.15.195:3300/0 conn(0x7fcba415db00 0x7fcba406cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- 172.21.15.179:0/368266032 >> 172.21.15.179:0/368266032 conn(0x7fcba40cea30 msgr2=0x7fcba40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- 172.21.15.179:0/368266032 shutdown_connections 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- 172.21.15.179:0/368266032 wait complete. 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 Processor -- start 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 -- start start 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.701+0000 7fcbaaf88700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba415db00 0x7fcba406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 0x7fcba4149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415a400 con 0x7fcba40cd3c0 2024-05-08T22:40:05.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415a6e0 con 0x7fcba415db00 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415a9c0 con 0x7fcba415dee0 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40000/0 (socket says 172.21.15.179:40000) 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 -- 172.21.15.179:0/540383720 learned_addr learned my addr 172.21.15.179:0/540383720 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9785700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba415db00 0x7fcba406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaa787700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 0x7fcba4149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 -- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 msgr2=0x7fcba4149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 0x7fcba4149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 -- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba415db00 msgr2=0x7fcba406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba415db00 0x7fcba406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 -- 172.21.15.179:0/540383720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0009570 con 0x7fcba40cd3c0 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaa787700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 0x7fcba4149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:05.712 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcba9f86700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e9d0 tx=0x7fcb9400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94009720 con 0x7fcba40cd3c0 2024-05-08T22:40:05.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9400ee70 con 0x7fcba40cd3c0 2024-05-08T22:40:05.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ca50 con 0x7fcba40cd3c0 2024-05-08T22:40:05.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba415aca0 con 0x7fcba40cd3c0 2024-05-08T22:40:05.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba413ea20 con 0x7fcba40cd3c0 2024-05-08T22:40:05.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.705+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba404e700 con 0x7fcba40cd3c0 2024-05-08T22:40:05.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.709+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb9400cbb0 con 0x7fcba40cd3c0 2024-05-08T22:40:05.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.709+0000 7fcb9a7fc700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 0x7fcb88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:05.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.709+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb94084950 con 0x7fcba40cd3c0 2024-05-08T22:40:05.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.709+0000 7fcba9785700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 0x7fcb88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:05.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.709+0000 7fcba9785700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 0x7fcb88068bf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009910 tx=0x7fcba0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:05.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.713+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb94051cf0 con 0x7fcba40cd3c0 2024-05-08T22:40:05.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.945+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcba415afa0 con 0x7fcba40cd3c0 2024-05-08T22:40:05.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcb9a7fc700 1 -- 172.21.15.179:0/540383720 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcb9400ce60 con 0x7fcba40cd3c0 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 msgr2=0x7fcb88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 0x7fcb88068bf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009910 tx=0x7fcba0008030 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 msgr2=0x7fcba413e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e9d0 tx=0x7fcb9400ece0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 shutdown_connections 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcb88066730 0x7fcb88068bf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcba415db00 0x7fcba406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcba415dee0 0x7fcba4149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 --2- 172.21.15.179:0/540383720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcba40cd3c0 0x7fcba413e4e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 >> 172.21.15.179:0/540383720 conn(0x7fcba40cea30 msgr2=0x7fcba406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:05.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 shutdown_connections 2024-05-08T22:40:05.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:05.953+0000 7fcbaaf88700 1 -- 172.21.15.179:0/540383720 wait complete. 2024-05-08T22:40:05.959 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:40:05.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:06.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 Processor -- start 2024-05-08T22:40:06.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 -- start start 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f552415ef80 0x7f5524161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5524164db0 0x7f552406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5524165d50 con 0x7f552406dca0 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552406e130 con 0x7f552415ef80 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f552aa83700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552406ad70 con 0x7f5524164db0 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f552415ef80 0x7f5524161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f552415ef80 0x7f5524161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40006/0 (socket says 172.21.15.179:40006) 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 -- 172.21.15.179:0/2553948375 learned_addr learned my addr 172.21.15.179:0/2553948375 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529280700 1 --2- 172.21.15.179:0/2553948375 >> v2:172.21.15.195:3300/0 conn(0x7f5524164db0 0x7f552406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 -- 172.21.15.179:0/2553948375 >> v2:172.21.15.195:3300/0 conn(0x7f5524164db0 msgr2=0x7f552406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 --2- 172.21.15.179:0/2553948375 >> v2:172.21.15.195:3300/0 conn(0x7f5524164db0 0x7f552406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 -- 172.21.15.179:0/2553948375 >> v1:172.21.15.195:6789/0 conn(0x7f552406dca0 legacy=0x7f55240b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 -- 172.21.15.179:0/2553948375 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524165260 con 0x7f552415ef80 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5529a81700 1 --2- 172.21.15.179:0/2553948375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524161790 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f55240b2020 tx=0x7f5520009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79c1fad376ea09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5528a7f700 1 -- 172.21.15.179:0/2553948375 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520004230 con 0x7f552415ef80 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5528a7f700 1 -- 172.21.15.179:0/2553948375 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520004390 con 0x7f552415ef80 2024-05-08T22:40:06.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.085+0000 7f5528a7f700 1 -- 172.21.15.179:0/2553948375 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200036c0 con 0x7f552415ef80 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- 172.21.15.179:0/2553948375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 msgr2=0x7f5524161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- 172.21.15.179:0/2553948375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524161790 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f55240b2020 tx=0x7f5520009bf0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- 172.21.15.179:0/2553948375 shutdown_connections 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- 172.21.15.179:0/2553948375 >> v2:172.21.15.195:3300/0 conn(0x7f5524164db0 0x7f552406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- 172.21.15.179:0/2553948375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524161790 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- 172.21.15.179:0/2553948375 >> 172.21.15.179:0/2553948375 conn(0x7f55240ce9d0 msgr2=0x7f55240cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- 172.21.15.179:0/2553948375 shutdown_connections 2024-05-08T22:40:06.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- 172.21.15.179:0/2553948375 wait complete. 2024-05-08T22:40:06.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 Processor -- start 2024-05-08T22:40:06.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- start start 2024-05-08T22:40:06.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 0x7f55241ed560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5524164db0 0x7f55240b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240cbbb0 con 0x7f552415ef80 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240cd800 con 0x7f5524164db0 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552aa83700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240cbdc0 con 0x7f552406dca0 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40022/0 (socket says 172.21.15.179:40022) 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 -- 172.21.15.179:0/379061606 learned_addr learned my addr 172.21.15.179:0/379061606 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529a81700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 0x7f55241ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f552a282700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5524164db0 0x7f55240b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 -- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 msgr2=0x7f55241ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 0x7f55241ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 -- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5524164db0 msgr2=0x7f55240b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5524164db0 0x7f55240b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529280700 1 -- 172.21.15.179:0/379061606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520009570 con 0x7f552415ef80 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.089+0000 7f5529a81700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 0x7f55241ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f5529280700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f551400d800 tx=0x7f551400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140125f0 con 0x7f552415ef80 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514012750 con 0x7f552415ef80 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240cc0a0 con 0x7f552415ef80 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514012b00 con 0x7f552415ef80 2024-05-08T22:40:06.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524159580 con 0x7f552415ef80 2024-05-08T22:40:06.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5524067740 con 0x7f552415ef80 2024-05-08T22:40:06.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5514012c60 con 0x7f552415ef80 2024-05-08T22:40:06.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 0x7f5508068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5514084680 con 0x7f552415ef80 2024-05-08T22:40:06.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.093+0000 7f5529a81700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 0x7f5508068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.097+0000 7f5529a81700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 0x7f5508068bf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55240b2020 tx=0x7f5520008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.101+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f55140518b0 con 0x7f552415ef80 2024-05-08T22:40:06.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.305+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5524160130 con 0x7f552415ef80 2024-05-08T22:40:06.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.305+0000 7f551a7fc700 1 -- 172.21.15.179:0/379061606 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f5514054ed0 con 0x7f552415ef80 2024-05-08T22:40:06.308 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:06.308 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:04.156894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/1693330449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1693330449},{"type":"v1","addr":"172.21.15.179:6835","nonce":1693330449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:06.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 msgr2=0x7f5508068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 0x7f5508068bf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55240b2020 tx=0x7f5520008030 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 msgr2=0x7f5524159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f551400d800 tx=0x7f551400db10 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 shutdown_connections 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5508066730 0x7f5508068bf0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5524164db0 0x7f55240b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415ef80 0x7f5524159040 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 --2- 172.21.15.179:0/379061606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f552406dca0 0x7f55241ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 >> 172.21.15.179:0/379061606 conn(0x7f55240ce9d0 msgr2=0x7f55241604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 shutdown_connections 2024-05-08T22:40:06.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.309+0000 7f552aa83700 1 -- 172.21.15.179:0/379061606 wait complete. 2024-05-08T22:40:06.314 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 55 2024-05-08T22:40:06.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:40:04.156894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/1693330449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 1693330449}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 1693330449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:40:06.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:40:06.731 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:40:06.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 Processor -- start 2024-05-08T22:40:06.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 -- start start 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f60080cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080cbf20 con 0x7f600815d1d0 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080cc170 con 0x7f60080cf6e0 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600806cb40 con 0x7f600815cdf0 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f6006d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f600815d1d0 0x7f600806d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35172/0 (socket says 172.21.15.179:35172) 2024-05-08T22:40:06.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f6006d9d700 1 -- 172.21.15.179:0/3975841171 learned_addr learned my addr 172.21.15.179:0/3975841171 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600659c700 1 --2- 172.21.15.179:0/3975841171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f60080cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590124833 0 0) 0x7f60080cbf20 con 0x7f600815d1d0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff40035d0 con 0x7f600815d1d0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456608194 0 0) 0x7f60080cc170 con 0x7f60080cf6e0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60080cbf20 con 0x7f60080cf6e0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600659c700 1 -- 172.21.15.179:0/3975841171 >> v1:172.21.15.195:6789/0 conn(0x7f60080cf6e0 legacy=0x7f60080cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600659c700 1 -- 172.21.15.179:0/3975841171 >> v1:172.21.15.179:6789/0 conn(0x7f600815d1d0 legacy=0x7f600806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600659c700 1 -- 172.21.15.179:0/3975841171 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080cc450 con 0x7f600815cdf0 2024-05-08T22:40:06.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600659c700 1 --2- 172.21.15.179:0/3975841171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f60080cf1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009880 tx=0x7f5ff0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789e98a585f6944f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.861+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011830 con 0x7f600815cdf0 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011990 con 0x7f600815cdf0 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600559a700 1 -- 172.21.15.179:0/3975841171 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011d00 con 0x7f600815cdf0 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- 172.21.15.179:0/3975841171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 msgr2=0x7f60080cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 --2- 172.21.15.179:0/3975841171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f60080cf1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009880 tx=0x7f5ff0009b90 comp rx=0 tx=0).stop 2024-05-08T22:40:06.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- 172.21.15.179:0/3975841171 shutdown_connections 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 --2- 172.21.15.179:0/3975841171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f60080cf1a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- 172.21.15.179:0/3975841171 >> 172.21.15.179:0/3975841171 conn(0x7f60080b2940 msgr2=0x7f60080b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- 172.21.15.179:0/3975841171 shutdown_connections 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- 172.21.15.179:0/3975841171 wait complete. 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 Processor -- start 2024-05-08T22:40:06.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- start start 2024-05-08T22:40:06.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f6008152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f600806f550 0x7f60080c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806fa00 con 0x7f600815d1d0 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080ca120 con 0x7f600806f550 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f600759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008159af0 con 0x7f600815cdf0 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.865+0000 7f6005d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40038/0 (socket says 172.21.15.179:40038) 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 -- 172.21.15.179:0/1450954552 learned_addr learned my addr 172.21.15.179:0/1450954552 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6006d9d700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f600806f550 0x7f60080c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f600659c700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f6008152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 -- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 msgr2=0x7f6008152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f6008152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 -- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f600806f550 msgr2=0x7f60080c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f600806f550 0x7f60080c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 -- 172.21.15.179:0/1450954552 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00094b0 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001f10 tx=0x7f5ff800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f5ffeffd700 1 -- 172.21.15.179:0/1450954552 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800ad00 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f5ffeffd700 1 -- 172.21.15.179:0/1450954552 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff800ae80 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008152ca0 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f5ffeffd700 1 -- 172.21.15.179:0/1450954552 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003c60 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60081f36c0 con 0x7f600815d1d0 2024-05-08T22:40:06.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.869+0000 7f5ffeffd700 1 -- 172.21.15.179:0/1450954552 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5ff80050d0 con 0x7f600815d1d0 2024-05-08T22:40:06.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f5ffeffd700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 0x7f5fe8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f5ffeffd700 1 -- 172.21.15.179:0/1450954552 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5ff8084420 con 0x7f600815d1d0 2024-05-08T22:40:06.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f600659c700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 0x7f5fe8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f600659c700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 0x7f5fe8068bf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00090b0 tx=0x7f5ff0010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 Processor -- start 2024-05-08T22:40:06.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 -- start start 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 0x21d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x20fed80 con 0x21d7770 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20fc510 con 0x21d02a0 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.873+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20f7e00 con 0x21ce5c0 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40040/0 (socket says 172.21.15.179:40040) 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 -- 172.21.15.179:0/3950464989 learned_addr learned my addr 172.21.15.179:0/3950464989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6006d9d700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 0x21d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 -- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 msgr2=0x21d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 0x21d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 -- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 msgr2=0x21d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 -- 172.21.15.179:0/3950464989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f6a960 con 0x21d7770 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 0x21d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f600659c700 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009480 tx=0x7f5ff0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e3c2626e51febb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f5fe7fff700 1 -- 172.21.15.179:0/3950464989 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00038c0 con 0x21d7770 2024-05-08T22:40:06.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f5fe7fff700 1 -- 172.21.15.179:0/3950464989 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0003a20 con 0x21d7770 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f5fe7fff700 1 -- 172.21.15.179:0/3950464989 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0003d90 con 0x21d7770 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- 172.21.15.179:0/3950464989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 msgr2=0x21d7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009480 tx=0x7f5ff0009f90 comp rx=0 tx=0).stop 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- 172.21.15.179:0/3950464989 shutdown_connections 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ce5c0 0x21d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 --2- 172.21.15.179:0/3950464989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21d7770 0x21d7b50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- 172.21.15.179:0/3950464989 >> 172.21.15.179:0/3950464989 conn(0x1f1f4d0 msgr2=0x21b9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- 172.21.15.179:0/3950464989 shutdown_connections 2024-05-08T22:40:06.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- 172.21.15.179:0/3950464989 wait complete. 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 -- start start 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.877+0000 7f6010129740 1 Processor -- start 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x218ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x21d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x20b5730 con 0x21ce5c0 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20bb5b0 con 0x21d7770 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fa3aa0 con 0x21d02a0 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 10 client.0 _ll_register_callbacks cb 0x20f1610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40044/0 (socket says 172.21.15.179:40044) 2024-05-08T22:40:06.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 -- 172.21.15.179:0/1168888537 learned_addr learned my addr 172.21.15.179:0/1168888537 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x218ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6006d9d700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x21d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 -- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 msgr2=0x218ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x218ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 -- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 msgr2=0x21d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x21d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 -- 172.21.15.179:0/1168888537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0021070 con 0x21ce5c0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f600659c700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce770 tx=0x7f5ff00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a60618e31d4681 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0003d90 con 0x21ce5c0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5ff003acb0 con 0x21ce5c0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f2c1e0 con 0x21ce5c0 2024-05-08T22:40:06.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0030420 con 0x21ce5c0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 10 client.8037 fetch_fsmap learned FSMap version 55 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x20be220 con 0x21ce5c0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f2c1e0 con 0x21ce5c0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 55) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5ff0030580 con 0x21ce5c0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 10 client.8037 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 10 client.8037 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 20 client.8037 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x218cd70 0x218d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 4 client.8037 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 --> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f76460 con 0x218cd70 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x218cd70 0x218d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f6005d9b700 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x218cd70 0x218d150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a337271c66f756c server_cookie=5028943f6f98b1ad in_seq=0 out_seq=0 2024-05-08T22:40:06.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.881+0000 7f5fe57fa700 10 client.8037 ms_handle_connect on v2:172.21.15.179:6834/1693330449 2024-05-08T22:40:06.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f5fe57fa700 1 -- 172.21.15.179:0/1168888537 <== mds.0 v2:172.21.15.179:6834/1693330449 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1f76460 con 0x218cd70 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f5fe57fa700 10 client.8037 handle_command_reply: tid=0 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 client.8037 shutdown 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 msgr2=0x21cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce770 tx=0x7f5ff00047e0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 shutdown_connections 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x218cd70 0x218d150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x21d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x218ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f5fe57fa700 0 client.8037 ms_handle_reset on v2:172.21.15.179:6834/1693330449 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 --2- 172.21.15.179:0/1168888537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21cdec0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 >> 172.21.15.179:0/1168888537 conn(0x1f1f4d0 msgr2=0x21b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 shutdown_connections 2024-05-08T22:40:06.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 1 -- 172.21.15.179:0/1168888537 wait complete. 2024-05-08T22:40:06.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.885+0000 7f6010129740 20 client.8037 trim_cache size 0 max 16384 2024-05-08T22:40:06.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 Processor -- start 2024-05-08T22:40:06.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- start start 2024-05-08T22:40:06.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x2234290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x22347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x218e3b0 con 0x21ce5c0 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2096470 con 0x21d7770 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20c24f0 con 0x21d02a0 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40058/0 (socket says 172.21.15.179:40058) 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 -- 172.21.15.179:0/2311409852 learned_addr learned my addr 172.21.15.179:0/2311409852 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6005d9b700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x2234290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6006d9d700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x22347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 -- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 msgr2=0x2234290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x2234290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 -- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 msgr2=0x22347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x22347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 -- 172.21.15.179:0/2311409852 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20a4450 con 0x21ce5c0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6005d9b700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x2234290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f600659c700 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00043a0 tx=0x7f5ff0032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e942eeedada05a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f5ffcff9700 1 -- 172.21.15.179:0/2311409852 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011630 con 0x21ce5c0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f5ffcff9700 1 -- 172.21.15.179:0/2311409852 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011790 con 0x21ce5c0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f5ffcff9700 1 -- 172.21.15.179:0/2311409852 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0030640 con 0x21ce5c0 2024-05-08T22:40:06.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- 172.21.15.179:0/2311409852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 msgr2=0x218b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00043a0 tx=0x7f5ff0032040 comp rx=0 tx=0).stop 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- 172.21.15.179:0/2311409852 shutdown_connections 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d7770 0x22347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d02a0 0x2234290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 --2- 172.21.15.179:0/2311409852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x218b8f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- 172.21.15.179:0/2311409852 >> 172.21.15.179:0/2311409852 conn(0x1f1f4d0 msgr2=0x21cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- 172.21.15.179:0/2311409852 shutdown_connections 2024-05-08T22:40:06.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.905+0000 7f6010129740 1 -- 172.21.15.179:0/2311409852 wait complete. 2024-05-08T22:40:06.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 -- start start 2024-05-08T22:40:06.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 Processor -- start 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d7770 0x2237ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b6990 con 0x21ce5c0 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f6a60 con 0x21d02a0 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x21d5860 con 0x21d7770 2024-05-08T22:40:06.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f600659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40064/0 (socket says 172.21.15.179:40064) 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f600659c700 1 -- 172.21.15.179:0/3556226078 learned_addr learned my addr 172.21.15.179:0/3556226078 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6010129740 10 client.0 _ll_register_callbacks cb 0x20f69d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.909+0000 7f6006d9d700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d7770 0x2237ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 -- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d7770 msgr2=0x2237ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d7770 0x2237ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 -- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 msgr2=0x21cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 -- 172.21.15.179:0/3556226078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x20be220 con 0x21ce5c0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f600659c700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011fd0 tx=0x7f5ff0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8abdf03f4e5e8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:06.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011630 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5ff0043680 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ff0003a20 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0043840 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 10 client.8043 fetch_fsmap learned FSMap version 55 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x2098720 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0003a20 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 55) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5ff00043d0 con 0x21ce5c0 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 10 client.8043 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 10 client.8043 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:06.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 20 client.8043 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:06.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x2239c10 0x21d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:06.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 4 client.8043 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:40:06.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 --> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1f9b1e0 con 0x2239c10 2024-05-08T22:40:06.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x2239c10 0x21d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:06.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 10 client.8043 ms_handle_connect on v2:172.21.15.179:6834/1693330449 2024-05-08T22:40:06.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6005d9b700 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x2239c10 0x21d63b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db3d3c8fb5c6967 server_cookie=a7d44cf526c25729 in_seq=0 out_seq=0 2024-05-08T22:40:06.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 1 -- 172.21.15.179:0/3556226078 <== mds.0 v2:172.21.15.179:6834/1693330449 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1f9b1e0 con 0x2239c10 2024-05-08T22:40:06.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 10 client.8043 handle_command_reply: tid=0 2024-05-08T22:40:06.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 client.8043 shutdown 2024-05-08T22:40:06.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 msgr2=0x21d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011fd0 tx=0x7f5ff0008030 comp rx=0 tx=0).stop 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 shutdown_connections 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:6834/1693330449,v1:172.21.15.179:6835/1693330449] conn(0x2239c10 0x21d63b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21d02a0 0x21cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21d7770 0x2237ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 --2- 172.21.15.179:0/3556226078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21ce5c0 0x21d5320 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 >> 172.21.15.179:0/3556226078 conn(0x1f1f4d0 msgr2=0x21b97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f5fe57fa700 0 client.8043 ms_handle_reset on v2:172.21.15.179:6834/1693330449 2024-05-08T22:40:06.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 shutdown_connections 2024-05-08T22:40:06.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 1 -- 172.21.15.179:0/3556226078 wait complete. 2024-05-08T22:40:06.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.913+0000 7f6010129740 20 client.8043 trim_cache size 0 max 16384 2024-05-08T22:40:06.925 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:06.925 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:40:06.925 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:40:06.925 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 7899, 2024-05-08T22:40:06.925 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 5.7740979279999998, 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 55, 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 97, 2024-05-08T22:40:06.926 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 96, 2024-05-08T22:40:06.927 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 10.200943924000001 2024-05-08T22:40:06.927 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:06.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 msgr2=0x7f5fe8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 0x7f5fe8068bf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00090b0 tx=0x7f5ff0010000 comp rx=0 tx=0).stop 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 msgr2=0x7f600806f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001f10 tx=0x7f5ff800a340 comp rx=0 tx=0).stop 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 shutdown_connections 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5fe8066730 0x7f5fe8068bf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f600806f550 0x7f60080c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f600815d1d0 0x7f600806f010 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 --2- 172.21.15.179:0/1450954552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f600815cdf0 0x7f6008152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:06.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 >> 172.21.15.179:0/1450954552 conn(0x7f60080b2940 msgr2=0x7f60080b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:06.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 shutdown_connections 2024-05-08T22:40:06.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:06.917+0000 7f600759e700 1 -- 172.21.15.179:0/1450954552 wait complete. 2024-05-08T22:40:06.935 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:40:07.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 Processor -- start 2024-05-08T22:40:07.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 -- start start 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb66806cd70 0x7fb6680ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 0x7fb66815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66815deb0 con 0x7fb668159710 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66815ab10 con 0x7fb66806cd70 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66c8da700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb668159710 0x7fb66806c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35186/0 (socket says 172.21.15.179:35186) 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d8dc700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66815d2a0 con 0x7fb6680c4070 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66c8da700 1 -- 172.21.15.179:0/3447645036 learned_addr learned my addr 172.21.15.179:0/3447645036 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb667fff700 1 --2- 172.21.15.179:0/3447645036 >> v2:172.21.15.179:3300/0 conn(0x7fb66806cd70 0x7fb6680ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb66d0db700 1 --2- 172.21.15.179:0/3447645036 >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 0x7fb66815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963318727 0 0) 0x7fb66815deb0 con 0x7fb668159710 2024-05-08T22:40:07.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.045+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6540035d0 con 0x7fb668159710 2024-05-08T22:40:07.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb667fff700 1 -- 172.21.15.179:0/3447645036 >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 msgr2=0x7fb66815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:07.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb667fff700 1 --2- 172.21.15.179:0/3447645036 >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 0x7fb66815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb667fff700 1 -- 172.21.15.179:0/3447645036 >> v1:172.21.15.179:6789/0 conn(0x7fb668159710 legacy=0x7fb66806c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4214078874 0 0) 0x7fb6540035d0 con 0x7fb668159710 2024-05-08T22:40:07.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb667fff700 1 -- 172.21.15.179:0/3447645036 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66815d580 con 0x7fb66806cd70 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d0db700 1 --2- 172.21.15.179:0/3447645036 >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 0x7fb66815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb667fff700 1 --2- 172.21.15.179:0/3447645036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb66806cd70 0x7fb6680ce6b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b15a0 tx=0x7fb650009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6bd344c24dea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658001670 con 0x7fb66806cd70 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650004030 con 0x7fb66806cd70 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb6677fe700 1 -- 172.21.15.179:0/3447645036 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500043c0 con 0x7fb66806cd70 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- 172.21.15.179:0/3447645036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb66806cd70 msgr2=0x7fb6680ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/3447645036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb66806cd70 0x7fb6680ce6b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b15a0 tx=0x7fb650009b30 comp rx=0 tx=0).stop 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- 172.21.15.179:0/3447645036 shutdown_connections 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/3447645036 >> v2:172.21.15.195:3300/0 conn(0x7fb6680c4070 0x7fb66815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/3447645036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb66806cd70 0x7fb6680ce6b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- 172.21.15.179:0/3447645036 >> 172.21.15.179:0/3447645036 conn(0x7fb668071a40 msgr2=0x7fb6680b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- 172.21.15.179:0/3447645036 shutdown_connections 2024-05-08T22:40:07.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- 172.21.15.179:0/3447645036 wait complete. 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 Processor -- start 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.049+0000 7fb66d8dc700 1 -- start start 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66806cd70 0x7fb66814c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb668159710 0x7fb6680c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66814cb80 con 0x7fb6680c4070 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66814d0a0 con 0x7fb66806cd70 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680c7210 con 0x7fb668159710 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40086/0 (socket says 172.21.15.179:40086) 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 -- 172.21.15.179:0/324703344 learned_addr learned my addr 172.21.15.179:0/324703344 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:07.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d0db700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb668159710 0x7fb6680c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66c8da700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66806cd70 0x7fb66814c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 -- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb668159710 msgr2=0x7fb6680c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb668159710 0x7fb6680c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 -- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66806cd70 msgr2=0x7fb66814c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66806cd70 0x7fb66814c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 -- 172.21.15.179:0/324703344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009a70 con 0x7fb6680c4070 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb667fff700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb6500090b0 tx=0x7fb650004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650003df0 con 0x7fb6680c4070 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65001f070 con 0x7fb6680c4070 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6500094b0 con 0x7fb6680c4070 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650011870 con 0x7fb6680c4070 2024-05-08T22:40:07.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb668151010 con 0x7fb6680c4070 2024-05-08T22:40:07.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb668067740 con 0x7fb6680c4070 2024-05-08T22:40:07.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6500119d0 con 0x7fb6680c4070 2024-05-08T22:40:07.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 0x7fb648068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:07.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.053+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb650021070 con 0x7fb6680c4070 2024-05-08T22:40:07.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.057+0000 7fb66c8da700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 0x7fb648068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:07.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.057+0000 7fb66c8da700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 0x7fb648068c20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb658001eb0 tx=0x7fb658009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:07.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.065+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb650052ea0 con 0x7fb6680c4070 2024-05-08T22:40:07.090 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:40:07.089+0000 7fe904d82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:40:07.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.269+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb66804e700 con 0x7fb6680c4070 2024-05-08T22:40:07.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.269+0000 7fb6657fa700 1 -- 172.21.15.179:0/324703344 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb65002b070 con 0x7fb6680c4070 2024-05-08T22:40:07.272 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:40:07.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 msgr2=0x7fb648068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 0x7fb648068c20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb658001eb0 tx=0x7fb658009040 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 msgr2=0x7fb66814df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb6500090b0 tx=0x7fb650004770 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 shutdown_connections 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb648066760 0x7fb648068c20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb668159710 0x7fb6680c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6680c4070 0x7fb66814df10 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 --2- 172.21.15.179:0/324703344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66806cd70 0x7fb66814c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 >> 172.21.15.179:0/324703344 conn(0x7fb668071a40 msgr2=0x7fb66814d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 shutdown_connections 2024-05-08T22:40:07.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:07.273+0000 7fb66d8dc700 1 -- 172.21.15.179:0/324703344 wait complete. 2024-05-08T22:40:07.292 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:40:07.293 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:07.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.293 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:40:07.299 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:40:07.299 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:40:07.299 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:40:07.299 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42164sec preferred_lft 42164sec 2024-05-08T22:40:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:40:07.301 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.302 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:40:07.302 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:40:07.302 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:40:07.302 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:07.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.302 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:40:07.313 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:40:07.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.315 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:40:07.324 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:40:07.325 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:40:07.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:40:07.325 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:40:07.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.403 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:40:07.407 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:40:07.407 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:40:07.408 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:07.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.408 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:40:07.417 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:40:07.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:40:07.417 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:07.421 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:07.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:40:07.422 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:07.427 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:07.487 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:07.487 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:07.497 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:07.498 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:07.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.498 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:07.503 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:40:07.504 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:40:07.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:07.524 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:07.524 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:07.580 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:07.581 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:07.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:07.581 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:07.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:40:07.589+0000 7f0b011a2640 -1 init, newargv = 0x562501717eb0 newargc=15 2024-05-08T22:40:07.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[45949]: starting ceph client 2024-05-08T22:40:07.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[45949]: starting fuse 2024-05-08T22:40:08.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:08.705 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:08.705 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:08.763 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:08.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:08.764 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:08.772 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:40:08.772 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:40:08.773 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:40:08.774 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:40:08.775 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:40:08.855 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.45949.asok 2024-05-08T22:40:09.041 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45949.asok 2024-05-08T22:40:09.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.041 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45949.asok status 2024-05-08T22:40:09.188 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:09.188 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:40:09.188 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:40:09.188 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "45949", 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 8061, 2024-05-08T22:40:09.189 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 8061 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2709342313 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:40:09.190 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2709342313 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.8061 192.168.0.1:0/2709342313", 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/2709342313", 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 55, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 98, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:40:09.191 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:09.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.198 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:40:09.198 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:09.203 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:40:09.204 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:09.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.204 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:09.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.218 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-08T22:40:09.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.282 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-08T22:40:09.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.294 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-08T22:40:09.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:09.306 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-08T22:40:09.325 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:40:09.434 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:40:09.625 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:40:09.625 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:40:09.633 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:40:09.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.749+0000 7f2542c1c700 1 Processor -- start 2024-05-08T22:40:09.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.749+0000 7f2542c1c700 1 -- start start 2024-05-08T22:40:09.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.749+0000 7f2542c1c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f253c15cb90 0x7f253c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c06b440 0x7f253c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0cc590 con 0x7f253c0cc140 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0afc90 con 0x7f253c15cb90 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0b5390 con 0x7f253c06b440 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f253c15cb90 0x7f253c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f253c15cb90 0x7f253c0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40096/0 (socket says 172.21.15.179:40096) 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 -- 172.21.15.179:0/2115141831 learned_addr learned my addr 172.21.15.179:0/2115141831 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:09.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541419700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c06b440 0x7f253c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 -- 172.21.15.179:0/2115141831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c06b440 msgr2=0x7f253c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c06b440 0x7f253c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 -- 172.21.15.179:0/2115141831 >> v1:172.21.15.179:6789/0 conn(0x7f253c0cc140 legacy=0x7f253c06bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 -- 172.21.15.179:0/2115141831 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0b4a80 con 0x7f253c15cb90 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2540c18700 1 -- 172.21.15.179:0/2115141831 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743909765 0 0) 0x7f2534001490 con 0x7f253c0cc140 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2541c1a700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c15cb90 0x7f253c0b0460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2538009910 tx=0x7f2538009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3a6211bf130091 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2540c18700 1 -- 172.21.15.179:0/2115141831 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538011830 con 0x7f253c15cb90 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2540c18700 1 -- 172.21.15.179:0/2115141831 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538011990 con 0x7f253c15cb90 2024-05-08T22:40:09.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2540c18700 1 -- 172.21.15.179:0/2115141831 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538011d40 con 0x7f253c15cb90 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- 172.21.15.179:0/2115141831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c15cb90 msgr2=0x7f253c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c15cb90 0x7f253c0b0460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2538009910 tx=0x7f2538009c20 comp rx=0 tx=0).stop 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- 172.21.15.179:0/2115141831 shutdown_connections 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c06b440 0x7f253c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 --2- 172.21.15.179:0/2115141831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c15cb90 0x7f253c0b0460 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- 172.21.15.179:0/2115141831 >> 172.21.15.179:0/2115141831 conn(0x7f253c098a70 msgr2=0x7f253c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:09.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- 172.21.15.179:0/2115141831 shutdown_connections 2024-05-08T22:40:09.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.753+0000 7f2542c1c700 1 -- 172.21.15.179:0/2115141831 wait complete. 2024-05-08T22:40:09.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 Processor -- start 2024-05-08T22:40:09.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- start start 2024-05-08T22:40:09.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 0x7f253c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 0x7f253c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c14e3a0 con 0x7f253c0cc140 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1f6db0 con 0x7f253c06b440 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1e22b0 con 0x7f253c15cb90 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541419700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541c1a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 0x7f253c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541419700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40112/0 (socket says 172.21.15.179:40112) 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541419700 1 -- 172.21.15.179:0/884141070 learned_addr learned my addr 172.21.15.179:0/884141070 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:09.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 0x7f253c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 -- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 msgr2=0x7f253c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 0x7f253c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 -- 172.21.15.179:0/884141070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 msgr2=0x7f253c1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 -- 172.21.15.179:0/884141070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538009570 con 0x7f253c15cb90 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541c1a700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 0x7f253c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2541419700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f254241b700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 0x7f253c0c8590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f253400cbb0 tx=0x7f253400cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534003e50 con 0x7f253c15cb90 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400a690 con 0x7f253c15cb90 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c1e2590 con 0x7f253c15cb90 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400aa40 con 0x7f253c15cb90 2024-05-08T22:40:09.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.757+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c1e2b50 con 0x7f253c15cb90 2024-05-08T22:40:09.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.761+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c04e700 con 0x7f253c15cb90 2024-05-08T22:40:09.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.761+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f253400aba0 con 0x7f253c15cb90 2024-05-08T22:40:09.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.761+0000 7f25327fc700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 0x7f2520068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:09.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.761+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2534083ae0 con 0x7f253c15cb90 2024-05-08T22:40:09.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.761+0000 7f2541c1a700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 0x7f2520068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:09.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.765+0000 7f2541c1a700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 0x7f2520068bf0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f2538008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:09.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.773+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2534050c90 con 0x7f253c15cb90 2024-05-08T22:40:09.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.945+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f253c0cc590 con 0x7f253c15cb90 2024-05-08T22:40:09.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.949+0000 7f25327fc700 1 -- 172.21.15.179:0/884141070 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16491 (secure 0 0 0) 0x7f253c0cc590 con 0x7f253c15cb90 2024-05-08T22:40:09.953 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:09.953 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":98,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:07.175476+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 msgr2=0x7f2520068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 0x7f2520068bf0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f2538008030 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 msgr2=0x7f253c0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 0x7f253c0c8590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f253400cbb0 tx=0x7f253400cec0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 shutdown_connections 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2520066730 0x7f2520068bf0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f253c15cb90 0x7f253c0c8590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f253c0cc140 0x7f253c1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 --2- 172.21.15.179:0/884141070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f253c06b440 0x7f253c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 >> 172.21.15.179:0/884141070 conn(0x7f253c098a70 msgr2=0x7f253c0c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:09.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 shutdown_connections 2024-05-08T22:40:09.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:09.953+0000 7f2542c1c700 1 -- 172.21.15.179:0/884141070 wait complete. 2024-05-08T22:40:09.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:40:09.969 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:40:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 Processor -- start 2024-05-08T22:40:10.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 -- start start 2024-05-08T22:40:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 0x7f47b006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 0x7f47b006d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00b22d0 con 0x7f47b0142890 2024-05-08T22:40:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00b19e0 con 0x7f47b0142fd0 2024-05-08T22:40:10.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b6446700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b015e640 con 0x7f47b0143520 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5444700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f47b0142890 0x7f47b00b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35202/0 (socket says 172.21.15.179:35202) 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5444700 1 -- 172.21.15.179:0/4107793799 learned_addr learned my addr 172.21.15.179:0/4107793799 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 --2- 172.21.15.179:0/4107793799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 0x7f47b006d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b4c43700 1 --2- 172.21.15.179:0/4107793799 >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 0x7f47b006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460360585 0 0) 0x7f47b00b22d0 con 0x7f47b0142890 2024-05-08T22:40:10.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47980035d0 con 0x7f47b0142890 2024-05-08T22:40:10.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 -- 172.21.15.179:0/4107793799 >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 msgr2=0x7f47b006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:10.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300844152 0 0) 0x7f47980035d0 con 0x7f47b0142890 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 --2- 172.21.15.179:0/4107793799 >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 0x7f47b006cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 -- 172.21.15.179:0/4107793799 >> v1:172.21.15.179:6789/0 conn(0x7f47b0142890 legacy=0x7f47b00b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 -- 172.21.15.179:0/4107793799 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00b1c80 con 0x7f47b0143520 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b4c43700 1 --2- 172.21.15.179:0/4107793799 >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 0x7f47b006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47b5c45700 1 --2- 172.21.15.179:0/4107793799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 0x7f47b006d2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009850 tx=0x7f47a0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b40e985385b187 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0011830 con 0x7f47b0143520 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0011990 con 0x7f47b0143520 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.089+0000 7f47affff700 1 -- 172.21.15.179:0/4107793799 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0011d00 con 0x7f47b0143520 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/4107793799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 msgr2=0x7f47b006d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- 172.21.15.179:0/4107793799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 0x7f47b006d2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009850 tx=0x7f47a0009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/4107793799 shutdown_connections 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- 172.21.15.179:0/4107793799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0143520 0x7f47b006d2e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- 172.21.15.179:0/4107793799 >> v2:172.21.15.195:3300/0 conn(0x7f47b0142fd0 0x7f47b006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/4107793799 >> 172.21.15.179:0/4107793799 conn(0x7f47b00cea30 msgr2=0x7f47b00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/4107793799 shutdown_connections 2024-05-08T22:40:10.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/4107793799 wait complete. 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 Processor -- start 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- start start 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0142fd0 0x7f47b01365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b0143520 0x7f47b0136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0157900 con 0x7f47b0142890 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0156870 con 0x7f47b0143520 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0157b50 con 0x7f47b0142fd0 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40122/0 (socket says 172.21.15.179:40122) 2024-05-08T22:40:10.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 -- 172.21.15.179:0/3586458171 learned_addr learned my addr 172.21.15.179:0/3586458171 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b4c43700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0142fd0 0x7f47b01365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5c45700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b0143520 0x7f47b0136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 -- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0142fd0 msgr2=0x7f47b01365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0142fd0 0x7f47b01365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 -- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b0143520 msgr2=0x7f47b0136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b0143520 0x7f47b0136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 -- 172.21.15.179:0/3586458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40094b0 con 0x7f47b0142890 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b5444700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f47a800cb90 tx=0x7f47a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800a820 con 0x7f47b0142890 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a00094b0 con 0x7f47b0142890 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b0158110 con 0x7f47b0142890 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a800a9a0 con 0x7f47b0142890 2024-05-08T22:40:10.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.093+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800ad10 con 0x7f47b0142890 2024-05-08T22:40:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4790005310 con 0x7f47b0142890 2024-05-08T22:40:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47a801b400 con 0x7f47b0142890 2024-05-08T22:40:10.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47adffb700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 0x7f479c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f47a8084350 con 0x7f47b0142890 2024-05-08T22:40:10.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47b4c43700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 0x7f479c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.097+0000 7f47b4c43700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 0x7f479c068bf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d80 tx=0x7f47a4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.105+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47a8051500 con 0x7f47b0142890 2024-05-08T22:40:10.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.269+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4790005b40 con 0x7f47b0142890 2024-05-08T22:40:10.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.269+0000 7f47adffb700 1 -- 172.21.15.179:0/3586458171 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47a8054b20 con 0x7f47b0142890 2024-05-08T22:40:10.273 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 msgr2=0x7f479c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 0x7f479c068bf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d80 tx=0x7f47a4009240 comp rx=0 tx=0).stop 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 msgr2=0x7f47b01514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f47a800cb90 tx=0x7f47a800cbc0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 shutdown_connections 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f479c066730 0x7f479c068bf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b0143520 0x7f47b0136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47b0142fd0 0x7f47b01365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 --2- 172.21.15.179:0/3586458171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47b0142890 0x7f47b01514f0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 >> 172.21.15.179:0/3586458171 conn(0x7f47b00cea30 msgr2=0x7f47b013caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 shutdown_connections 2024-05-08T22:40:10.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.273+0000 7f47b6446700 1 -- 172.21.15.179:0/3586458171 wait complete. 2024-05-08T22:40:10.287 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:40:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.389+0000 7f1eaca04700 1 Processor -- start 2024-05-08T22:40:10.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.389+0000 7f1eaca04700 1 -- start start 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1eaca04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 0x7f1ea806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1eaca04700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 0x7f1ea815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1eaca04700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80b4650 con 0x7f1ea815e2b0 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1eaca04700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea80cd6e0 con 0x7f1ea81544e0 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1eaca04700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea806db50 con 0x7f1ea815d8f0 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea6d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 0x7f1ea806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea759e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1ea815e2b0 0x7f1ea815a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35204/0 (socket says 172.21.15.179:35204) 2024-05-08T22:40:10.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea759e700 1 -- 172.21.15.179:0/1234207117 learned_addr learned my addr 172.21.15.179:0/1234207117 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/1234207117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 0x7f1ea815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 -- 172.21.15.179:0/1234207117 >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 msgr2=0x7f1ea806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/1234207117 >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 0x7f1ea806ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 -- 172.21.15.179:0/1234207117 >> v1:172.21.15.179:6789/0 conn(0x7f1ea815e2b0 legacy=0x7f1ea815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea659c700 1 -- 172.21.15.179:0/1234207117 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32527147 0 0) 0x7f1ea80b4650 con 0x7f1ea815e2b0 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 -- 172.21.15.179:0/1234207117 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea815e740 con 0x7f1ea815d8f0 2024-05-08T22:40:10.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea6d9d700 1 --2- 172.21.15.179:0/1234207117 >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 0x7f1ea806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/1234207117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 0x7f1ea815dd70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009850 tx=0x7f1e90009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f282a72ef567609 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea659c700 1 -- 172.21.15.179:0/1234207117 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90011830 con 0x7f1ea815d8f0 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea659c700 1 -- 172.21.15.179:0/1234207117 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90011990 con 0x7f1ea815d8f0 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.393+0000 7f1ea659c700 1 -- 172.21.15.179:0/1234207117 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90011d40 con 0x7f1ea815d8f0 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 -- 172.21.15.179:0/1234207117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 msgr2=0x7f1ea815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 --2- 172.21.15.179:0/1234207117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 0x7f1ea815dd70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009850 tx=0x7f1e90009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 -- 172.21.15.179:0/1234207117 shutdown_connections 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 --2- 172.21.15.179:0/1234207117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815d8f0 0x7f1ea815dd70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 --2- 172.21.15.179:0/1234207117 >> v2:172.21.15.179:3300/0 conn(0x7f1ea81544e0 0x7f1ea806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 -- 172.21.15.179:0/1234207117 >> 172.21.15.179:0/1234207117 conn(0x7f1ea80b18d0 msgr2=0x7f1ea80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 -- 172.21.15.179:0/1234207117 shutdown_connections 2024-05-08T22:40:10.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.397+0000 7f1eaca04700 1 -- 172.21.15.179:0/1234207117 wait complete. 2024-05-08T22:40:10.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 Processor -- start 2024-05-08T22:40:10.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- start start 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ea81544e0 0x7f1ea8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 0x7f1ea8158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8147b10 con 0x7f1ea815d8f0 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea814a5e0 con 0x7f1ea81544e0 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8147830 con 0x7f1ea815e2b0 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40128/0 (socket says 172.21.15.179:40128) 2024-05-08T22:40:10.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 -- 172.21.15.179:0/113802551 learned_addr learned my addr 172.21.15.179:0/113802551 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1ea6d9d700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ea81544e0 0x7f1ea8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1ea759e700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 0x7f1ea8158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 -- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 msgr2=0x7f1ea8158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 0x7f1ea8158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 -- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ea81544e0 msgr2=0x7f1ea8157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ea81544e0 0x7f1ea8157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 -- 172.21.15.179:0/113802551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980094b0 con 0x7f1ea815d8f0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1ea759e700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 0x7f1ea8158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9e59c700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900036c0 con 0x7f1ea815d8f0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e900094b0 con 0x7f1ea815d8f0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9001f070 con 0x7f1ea815d8f0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90003a80 con 0x7f1ea815d8f0 2024-05-08T22:40:10.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.401+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea814ab60 con 0x7f1ea815d8f0 2024-05-08T22:40:10.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.405+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea804e700 con 0x7f1ea815d8f0 2024-05-08T22:40:10.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.405+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e90003be0 con 0x7f1ea815d8f0 2024-05-08T22:40:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.405+0000 7f1e9ffff700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 0x7f1e88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.405+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1e90021070 con 0x7f1ea815d8f0 2024-05-08T22:40:10.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.405+0000 7f1ea6d9d700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 0x7f1e88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.409+0000 7f1ea6d9d700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 0x7f1e88068bf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009d10 tx=0x7f1e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.409+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e90052e30 con 0x7f1ea815d8f0 2024-05-08T22:40:10.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.577+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ea814d460 con 0x7f1ea815d8f0 2024-05-08T22:40:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.577+0000 7f1e9ffff700 1 -- 172.21.15.179:0/113802551 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e90015020 con 0x7f1ea815d8f0 2024-05-08T22:40:10.581 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:40:10.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 msgr2=0x7f1e88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 0x7f1e88068bf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009d10 tx=0x7f1e98008040 comp rx=0 tx=0).stop 2024-05-08T22:40:10.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 msgr2=0x7f1ea81586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e90004770 comp rx=0 tx=0).stop 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 shutdown_connections 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e88066730 0x7f1e88068bf0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ea815e2b0 0x7f1ea8158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ea815d8f0 0x7f1ea81586f0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 --2- 172.21.15.179:0/113802551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ea81544e0 0x7f1ea8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 >> 172.21.15.179:0/113802551 conn(0x7f1ea80b18d0 msgr2=0x7f1ea80b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 shutdown_connections 2024-05-08T22:40:10.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.581+0000 7f1eaca04700 1 -- 172.21.15.179:0/113802551 wait complete. 2024-05-08T22:40:10.596 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:40:10.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.709+0000 7f7f3f41a700 1 Processor -- start 2024-05-08T22:40:10.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.709+0000 7f7f3f41a700 1 -- start start 2024-05-08T22:40:10.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f3815d000 0x7f7f380cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 0x7f7f380cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380cd070 con 0x7f7f380ca880 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380b3b00 con 0x7f7f3815d000 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380b3d10 con 0x7f7f380b47f0 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f3815d000 0x7f7f380cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3ec19700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7f380ca880 0x7f7f38160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35216/0 (socket says 172.21.15.179:35216) 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f3815d000 0x7f7f380cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40130/0 (socket says 172.21.15.179:40130) 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3ec19700 1 -- 172.21.15.179:0/1132027377 learned_addr learned my addr 172.21.15.179:0/1132027377 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/1132027377 >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 0x7f7f380cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 -- 172.21.15.179:0/1132027377 >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 msgr2=0x7f7f380cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 --2- 172.21.15.179:0/1132027377 >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 0x7f7f380cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 -- 172.21.15.179:0/1132027377 >> v1:172.21.15.179:6789/0 conn(0x7f7f380ca880 legacy=0x7f7f38160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3d416700 1 -- 172.21.15.179:0/1132027377 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631239467 0 0) 0x7f7f380cd070 con 0x7f7f380ca880 2024-05-08T22:40:10.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 -- 172.21.15.179:0/1132027377 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380b3ff0 con 0x7f7f3815d000 2024-05-08T22:40:10.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/1132027377 >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 0x7f7f380cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:10.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3e418700 1 --2- 172.21.15.179:0/1132027377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3815d000 0x7f7f380cf8b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7f380d0be0 tx=0x7f7f28009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5c7509d6a0c358 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3d416700 1 -- 172.21.15.179:0/1132027377 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011830 con 0x7f7f3815d000 2024-05-08T22:40:10.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3d416700 1 -- 172.21.15.179:0/1132027377 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28011990 con 0x7f7f3815d000 2024-05-08T22:40:10.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3d416700 1 -- 172.21.15.179:0/1132027377 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011d40 con 0x7f7f3815d000 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- 172.21.15.179:0/1132027377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3815d000 msgr2=0x7f7f380cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/1132027377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3815d000 0x7f7f380cf8b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7f380d0be0 tx=0x7f7f28009b30 comp rx=0 tx=0).stop 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- 172.21.15.179:0/1132027377 shutdown_connections 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/1132027377 >> v2:172.21.15.195:3300/0 conn(0x7f7f380b47f0 0x7f7f380cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/1132027377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3815d000 0x7f7f380cf8b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- 172.21.15.179:0/1132027377 >> 172.21.15.179:0/1132027377 conn(0x7f7f380b27c0 msgr2=0x7f7f380b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- 172.21.15.179:0/1132027377 shutdown_connections 2024-05-08T22:40:10.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.713+0000 7f7f3f41a700 1 -- 172.21.15.179:0/1132027377 wait complete. 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 Processor -- start 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- start start 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 0x7f7f380c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 0x7f7f380c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38159250 con 0x7f7f380b47f0 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380c7600 con 0x7f7f3815d000 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815e5f0 con 0x7f7f380ca880 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40132/0 (socket says 172.21.15.179:40132) 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 -- 172.21.15.179:0/3430827436 learned_addr learned my addr 172.21.15.179:0/3430827436 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 0x7f7f380c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3ec19700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 0x7f7f380c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 -- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 msgr2=0x7f7f380c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 0x7f7f380c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 -- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 msgr2=0x7f7f380c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 0x7f7f380c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 -- 172.21.15.179:0/3430827436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280094b0 con 0x7f7f380b47f0 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 0x7f7f380c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3ec19700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 0x7f7f380c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:10.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3e418700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f28004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28019ab0 con 0x7f7f380b47f0 2024-05-08T22:40:10.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2801e070 con 0x7f7f380b47f0 2024-05-08T22:40:10.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f381ed0f0 con 0x7f7f380b47f0 2024-05-08T22:40:10.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28019e70 con 0x7f7f380b47f0 2024-05-08T22:40:10.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.717+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3815e870 con 0x7f7f380b47f0 2024-05-08T22:40:10.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f380ca680 con 0x7f7f380b47f0 2024-05-08T22:40:10.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f2801a420 con 0x7f7f380b47f0 2024-05-08T22:40:10.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f2effd700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 0x7f7f1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:10.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7f280854a0 con 0x7f7f380b47f0 2024-05-08T22:40:10.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 0x7f7f1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:10.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.721+0000 7f7f3dc17700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 0x7f7f1c068c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7f38158dd0 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:10.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.729+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7f28052660 con 0x7f7f380b47f0 2024-05-08T22:40:10.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.893+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f38067740 con 0x7f7f380b47f0 2024-05-08T22:40:10.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.893+0000 7f7f2effd700 1 -- 172.21.15.179:0/3430827436 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f28006ce0 con 0x7f7f380b47f0 2024-05-08T22:40:10.898 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 msgr2=0x7f7f1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 0x7f7f1c068c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7f38158dd0 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 msgr2=0x7f7f38156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f28004770 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 shutdown_connections 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7f1c066760 0x7f7f1c068c20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f3815d000 0x7f7f380c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f380ca880 0x7f7f380c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 --2- 172.21.15.179:0/3430827436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f380b47f0 0x7f7f38156f60 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 >> 172.21.15.179:0/3430827436 conn(0x7f7f380b27c0 msgr2=0x7f7f380b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 shutdown_connections 2024-05-08T22:40:10.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:10.897+0000 7f7f3f41a700 1 -- 172.21.15.179:0/3430827436 wait complete. 2024-05-08T22:40:10.913 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:40:11.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.021+0000 7f0e331f8700 1 Processor -- start 2024-05-08T22:40:11.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.021+0000 7f0e331f8700 1 -- start start 2024-05-08T22:40:11.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0ca360 0x7f0e2c0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0caed0 con 0x7f0e2c0b2a50 2024-05-08T22:40:11.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0cbb50 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0cb1b0 con 0x7f0e2c0ca360 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40142/0 (socket says 172.21.15.179:40142) 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 -- 172.21.15.179:0/2916960951 learned_addr learned my addr 172.21.15.179:0/2916960951 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e319f5700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0ca360 0x7f0e2c0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 -- 172.21.15.179:0/2916960951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0ca360 msgr2=0x7f0e2c0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0ca360 0x7f0e2c0ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 -- 172.21.15.179:0/2916960951 >> v1:172.21.15.195:6789/0 conn(0x7f0e2c0b2a50 legacy=0x7f0e2c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 -- 172.21.15.179:0/2916960951 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0cb490 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e329f7700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009e50 tx=0x7f0e2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52041463cb9e6ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e311f4700 1 -- 172.21.15.179:0/2916960951 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400e070 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e311f4700 1 -- 172.21.15.179:0/2916960951 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2400b9c0 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e311f4700 1 -- 172.21.15.179:0/2916960951 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400bd70 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- 172.21.15.179:0/2916960951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 msgr2=0x7f0e2c0c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009e50 tx=0x7f0e2400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:11.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- 172.21.15.179:0/2916960951 shutdown_connections 2024-05-08T22:40:11.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0ca360 0x7f0e2c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 --2- 172.21.15.179:0/2916960951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c0c4fc0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- 172.21.15.179:0/2916960951 >> 172.21.15.179:0/2916960951 conn(0x7f0e2c0cfb00 msgr2=0x7f0e2c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- 172.21.15.179:0/2916960951 shutdown_connections 2024-05-08T22:40:11.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.025+0000 7f0e331f8700 1 -- 172.21.15.179:0/2916960951 wait complete. 2024-05-08T22:40:11.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 Processor -- start 2024-05-08T22:40:11.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 -- start start 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0b2a50 0x7f0e2c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0ca360 0x7f0e2c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c13c6d0 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c13c9b0 con 0x7f0e2c0ca360 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e331f8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c13cc90 con 0x7f0e2c0b2a50 2024-05-08T22:40:11.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e319f5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e319f5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40158/0 (socket says 172.21.15.179:40158) 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e319f5700 1 -- 172.21.15.179:0/1453961533 learned_addr learned my addr 172.21.15.179:0/1453961533 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e329f7700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0ca360 0x7f0e2c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e321f6700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0b2a50 0x7f0e2c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e319f5700 1 -- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0b2a50 msgr2=0x7f0e2c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.029+0000 7f0e319f5700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0b2a50 0x7f0e2c15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e319f5700 1 -- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0ca360 msgr2=0x7f0e2c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e319f5700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0ca360 0x7f0e2c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e319f5700 1 -- 172.21.15.179:0/1453961533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009b30 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e319f5700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000e760 tx=0x7f0e2000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20013610 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e20013770 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c13cf70 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20013ae0 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c06d430 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c0b3e50 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.033+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e20013c40 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.037+0000 7f0e2affd700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 0x7f0e18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.037+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f0e20084110 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.037+0000 7f0e321f6700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 0x7f0e18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.037+0000 7f0e321f6700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 0x7f0e18068bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0c9730 tx=0x7f0e1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.045+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e20051220 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.225+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0e2c04e700 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.225+0000 7f0e2affd700 1 -- 172.21.15.179:0/1453961533 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0e20054840 con 0x7f0e2c0c4b30 2024-05-08T22:40:11.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 msgr2=0x7f0e18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 0x7f0e18068bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0c9730 tx=0x7f0e1c009040 comp rx=0 tx=0).stop 2024-05-08T22:40:11.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 msgr2=0x7f0e2c13c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000e760 tx=0x7f0e2000eb20 comp rx=0 tx=0).stop 2024-05-08T22:40:11.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 shutdown_connections 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e18066730 0x7f0e18068bf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0ca360 0x7f0e2c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0c4b30 0x7f0e2c13c010 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 --2- 172.21.15.179:0/1453961533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0b2a50 0x7f0e2c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 >> 172.21.15.179:0/1453961533 conn(0x7f0e2c0cfb00 msgr2=0x7f0e2c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 shutdown_connections 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.229+0000 7f0e331f8700 1 -- 172.21.15.179:0/1453961533 wait complete. 2024-05-08T22:40:11.233 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:40:11.244 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:40:11.244 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:40:11.252 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:40:11.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.357+0000 7fe20bfff700 1 Processor -- start 2024-05-08T22:40:11.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20bfff700 1 -- start start 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe20c13de70 0x7fe20c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c15c360 con 0x7fe20c15b900 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c15bda0 con 0x7fe20c06c450 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c15c080 con 0x7fe20c13de70 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe2027fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe20c15b900 0x7fe20c13e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35232/0 (socket says 172.21.15.179:35232) 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe2027fc700 1 -- 172.21.15.179:0/1626618335 learned_addr learned my addr 172.21.15.179:0/1626618335 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:11.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20affd700 1 --2- 172.21.15.179:0/1626618335 >> v2:172.21.15.195:3300/0 conn(0x7fe20c13de70 0x7fe20c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651798204 0 0) 0x7fe20c15c360 con 0x7fe20c15b900 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f80035d0 con 0x7fe20c15b900 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324062812 0 0) 0x7fe20c15bda0 con 0x7fe20c06c450 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe20c15c360 con 0x7fe20c06c450 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20affd700 1 -- 172.21.15.179:0/1626618335 >> v1:172.21.15.195:6789/0 conn(0x7fe20c06c450 legacy=0x7fe20c06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20affd700 1 -- 172.21.15.179:0/1626618335 >> v1:172.21.15.179:6789/0 conn(0x7fe20c15b900 legacy=0x7fe20c13e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20affd700 1 -- 172.21.15.179:0/1626618335 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0ce210 con 0x7fe20c13de70 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20affd700 1 --2- 172.21.15.179:0/1626618335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c13de70 0x7fe20c15b3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009850 tx=0x7fe1fc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f55f3912aa0774 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011830 con 0x7fe20c13de70 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc011990 con 0x7fe20c13de70 2024-05-08T22:40:11.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.361+0000 7fe20a7fc700 1 -- 172.21.15.179:0/1626618335 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011d40 con 0x7fe20c13de70 2024-05-08T22:40:11.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- 172.21.15.179:0/1626618335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c13de70 msgr2=0x7fe20c15b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 --2- 172.21.15.179:0/1626618335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c13de70 0x7fe20c15b3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009850 tx=0x7fe1fc009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:11.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- 172.21.15.179:0/1626618335 shutdown_connections 2024-05-08T22:40:11.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 --2- 172.21.15.179:0/1626618335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c13de70 0x7fe20c15b3c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- 172.21.15.179:0/1626618335 >> 172.21.15.179:0/1626618335 conn(0x7fe20c0cfae0 msgr2=0x7fe20c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- 172.21.15.179:0/1626618335 shutdown_connections 2024-05-08T22:40:11.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- 172.21.15.179:0/1626618335 wait complete. 2024-05-08T22:40:11.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 Processor -- start 2024-05-08T22:40:11.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- start start 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c06c450 0x7fe20c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 0x7fe20c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0ccc60 con 0x7fe20c13de70 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0cbd10 con 0x7fe20c15b900 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0cbfe0 con 0x7fe20c06c450 2024-05-08T22:40:11.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe2027fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe2027fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40170/0 (socket says 172.21.15.179:40170) 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe2027fc700 1 -- 172.21.15.179:0/3424834524 learned_addr learned my addr 172.21.15.179:0/3424834524 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20b7fe700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 0x7fe20c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.365+0000 7fe20affd700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c06c450 0x7fe20c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 -- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c06c450 msgr2=0x7fe20c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c06c450 0x7fe20c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 -- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 msgr2=0x7fe20c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 0x7fe20c15d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 -- 172.21.15.179:0/3424834524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0094b0 con 0x7fe20c13de70 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe20b7fe700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 0x7fe20c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe2027fc700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c4d0 tx=0x7fe1f400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4005d90 con 0x7fe20c13de70 2024-05-08T22:40:11.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f40070d0 con 0x7fe20c13de70 2024-05-08T22:40:11.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c0cc2c0 con 0x7fe20c13de70 2024-05-08T22:40:11.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4007480 con 0x7fe20c13de70 2024-05-08T22:40:11.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c0cc880 con 0x7fe20c13de70 2024-05-08T22:40:11.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c06ef10 con 0x7fe20c13de70 2024-05-08T22:40:11.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.369+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1f40075e0 con 0x7fe20c13de70 2024-05-08T22:40:11.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.373+0000 7fe203fff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.373+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe1f4083750 con 0x7fe20c13de70 2024-05-08T22:40:11.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.373+0000 7fe20affd700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.373+0000 7fe20affd700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0090b0 tx=0x7fe1fc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.381+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe1f4050900 con 0x7fe20c13de70 2024-05-08T22:40:11.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.549+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe20c0ccda0 con 0x7fe20c13de70 2024-05-08T22:40:11.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.549+0000 7fe203fff700 1 -- 172.21.15.179:0/3424834524 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16491 (secure 0 0 0) 0x7fe20c0ccda0 con 0x7fe20c13de70 2024-05-08T22:40:11.553 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:11.554 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":98,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:07.175476+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:40:11.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 msgr2=0x7fe1ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0090b0 tx=0x7fe1fc008030 comp rx=0 tx=0).stop 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 msgr2=0x7fe20c0cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c4d0 tx=0x7fe1f400c7e0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 shutdown_connections 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c15b900 0x7fe20c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c13de70 0x7fe20c0cb7d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 --2- 172.21.15.179:0/3424834524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c06c450 0x7fe20c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 >> 172.21.15.179:0/3424834524 conn(0x7fe20c0cfae0 msgr2=0x7fe20c070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 shutdown_connections 2024-05-08T22:40:11.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.553+0000 7fe20bfff700 1 -- 172.21.15.179:0/3424834524 wait complete. 2024-05-08T22:40:11.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:40:11.569 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:40:11.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 Processor -- start 2024-05-08T22:40:11.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 -- start start 2024-05-08T22:40:11.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f876415d000 0x7f87640cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87640b47f0 0x7f87640cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640cd070 con 0x7f87640ca880 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640b3b00 con 0x7f87640b47f0 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640b3d10 con 0x7f876415d000 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f8762d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f87640ca880 0x7f8764160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35240/0 (socket says 172.21.15.179:35240) 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f8761d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87640b47f0 0x7f87640cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f8762d9d700 1 -- 172.21.15.179:0/4046091678 learned_addr learned my addr 172.21.15.179:0/4046091678 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:11.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.685+0000 7f876259c700 1 --2- 172.21.15.179:0/4046091678 >> v2:172.21.15.195:3300/0 conn(0x7f876415d000 0x7f87640cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f8761d9b700 1 -- 172.21.15.179:0/4046091678 >> v2:172.21.15.195:3300/0 conn(0x7f876415d000 msgr2=0x7f87640cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:11.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f8761d9b700 1 --2- 172.21.15.179:0/4046091678 >> v2:172.21.15.195:3300/0 conn(0x7f876415d000 0x7f87640cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f8761d9b700 1 -- 172.21.15.179:0/4046091678 >> v1:172.21.15.179:6789/0 conn(0x7f87640ca880 legacy=0x7f8764160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876159a700 1 -- 172.21.15.179:0/4046091678 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085957689 0 0) 0x7f87640cd070 con 0x7f87640ca880 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f8761d9b700 1 -- 172.21.15.179:0/4046091678 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640b3ff0 con 0x7f87640b47f0 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f8761d9b700 1 --2- 172.21.15.179:0/4046091678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87640b47f0 0x7f87640cfdf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f87640d0c30 tx=0x7f8758009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c57782353cae49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876159a700 1 -- 172.21.15.179:0/4046091678 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011830 con 0x7f87640b47f0 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876159a700 1 -- 172.21.15.179:0/4046091678 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758011990 con 0x7f87640b47f0 2024-05-08T22:40:11.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876159a700 1 -- 172.21.15.179:0/4046091678 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011d20 con 0x7f87640b47f0 2024-05-08T22:40:11.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 -- 172.21.15.179:0/4046091678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87640b47f0 msgr2=0x7f87640cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 --2- 172.21.15.179:0/4046091678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87640b47f0 0x7f87640cfdf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f87640d0c30 tx=0x7f8758009b30 comp rx=0 tx=0).stop 2024-05-08T22:40:11.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 -- 172.21.15.179:0/4046091678 shutdown_connections 2024-05-08T22:40:11.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 --2- 172.21.15.179:0/4046091678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87640b47f0 0x7f87640cfdf0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 --2- 172.21.15.179:0/4046091678 >> v2:172.21.15.195:3300/0 conn(0x7f876415d000 0x7f87640cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 -- 172.21.15.179:0/4046091678 >> 172.21.15.179:0/4046091678 conn(0x7f87640b27c0 msgr2=0x7f87640b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 -- 172.21.15.179:0/4046091678 shutdown_connections 2024-05-08T22:40:11.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.689+0000 7f876359e700 1 -- 172.21.15.179:0/4046091678 wait complete. 2024-05-08T22:40:11.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 Processor -- start 2024-05-08T22:40:11.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 -- start start 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87640b47f0 0x7f8764150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87640ca880 0x7f87640aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641506a0 con 0x7f876415d000 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640af730 con 0x7f87640b47f0 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640afa10 con 0x7f87640ca880 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f8762d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f8762d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40194/0 (socket says 172.21.15.179:40194) 2024-05-08T22:40:11.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f8762d9d700 1 -- 172.21.15.179:0/3595261485 learned_addr learned my addr 172.21.15.179:0/3595261485 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f8761d9b700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87640ca880 0x7f87640aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.693+0000 7f876259c700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87640b47f0 0x7f8764150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 -- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87640ca880 msgr2=0x7f87640aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87640ca880 0x7f87640aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 -- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87640b47f0 msgr2=0x7f8764150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87640b47f0 0x7f8764150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 -- 172.21.15.179:0/3595261485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580094b0 con 0x7f876415d000 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8762d9d700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cca0 tx=0x7f875c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00ab70 con 0x7f876415d000 2024-05-08T22:40:11.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00acd0 con 0x7f876415d000 2024-05-08T22:40:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876415d560 con 0x7f876415d000 2024-05-08T22:40:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c007310 con 0x7f876415d000 2024-05-08T22:40:11.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876415db20 con 0x7f876415d000 2024-05-08T22:40:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764067740 con 0x7f876415d000 2024-05-08T22:40:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f875c007470 con 0x7f876415d000 2024-05-08T22:40:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 0x7f8744068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:11.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f875c084130 con 0x7f876415d000 2024-05-08T22:40:11.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.697+0000 7f876259c700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 0x7f8744068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:11.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.701+0000 7f876259c700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 0x7f8744068c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8754009b50 tx=0x7f8754009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:11.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.705+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f875c051240 con 0x7f876415d000 2024-05-08T22:40:11.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.873+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f876415dd40 con 0x7f876415d000 2024-05-08T22:40:11.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.873+0000 7f8752ffd700 1 -- 172.21.15.179:0/3595261485 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f875c054860 con 0x7f876415d000 2024-05-08T22:40:11.876 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 msgr2=0x7f8744068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 0x7f8744068c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8754009b50 tx=0x7f8754009240 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 msgr2=0x7f87640c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cca0 tx=0x7f875c00a060 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 shutdown_connections 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8744066760 0x7f8744068c20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f876415d000 0x7f87640c8a00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87640ca880 0x7f87640aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 --2- 172.21.15.179:0/3595261485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87640b47f0 0x7f8764150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 >> 172.21.15.179:0/3595261485 conn(0x7f87640b27c0 msgr2=0x7f87640c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 shutdown_connections 2024-05-08T22:40:11.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:11.877+0000 7f876359e700 1 -- 172.21.15.179:0/3595261485 wait complete. 2024-05-08T22:40:11.891 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:40:12.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 Processor -- start 2024-05-08T22:40:12.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 -- start start 2024-05-08T22:40:12.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f16340cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b8ee0 con 0x7f16340ce0e0 2024-05-08T22:40:12.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163415d4a0 con 0x7f163415d0c0 2024-05-08T22:40:12.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340ce570 con 0x7f16340b8a80 2024-05-08T22:40:12.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1632d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f16340ce0e0 0x7f163406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35254/0 (socket says 172.21.15.179:35254) 2024-05-08T22:40:12.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1632d9d700 1 -- 172.21.15.179:0/4220207387 learned_addr learned my addr 172.21.15.179:0/4220207387 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:12.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1631d9b700 1 --2- 172.21.15.179:0/4220207387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f16340cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558230402 0 0) 0x7f16340b8ee0 con 0x7f16340ce0e0 2024-05-08T22:40:12.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161c0035d0 con 0x7f16340ce0e0 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952180905 0 0) 0x7f163415d4a0 con 0x7f163415d0c0 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16340b8ee0 con 0x7f163415d0c0 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480445434 0 0) 0x7f161c0035d0 con 0x7f16340ce0e0 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1631d9b700 1 -- 172.21.15.179:0/4220207387 >> v1:172.21.15.195:6789/0 conn(0x7f163415d0c0 legacy=0x7f16340d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1631d9b700 1 -- 172.21.15.179:0/4220207387 >> v1:172.21.15.179:6789/0 conn(0x7f16340ce0e0 legacy=0x7f163406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.009+0000 7f1631d9b700 1 -- 172.21.15.179:0/4220207387 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163406abb0 con 0x7f16340b8a80 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f1631d9b700 1 --2- 172.21.15.179:0/4220207387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f16340cdba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16240098b0 tx=0x7f1624009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e91e8f7e06596e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c0019f0 con 0x7f16340b8a80 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624004030 con 0x7f16340b8a80 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163159a700 1 -- 172.21.15.179:0/4220207387 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240043a0 con 0x7f16340b8a80 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- 172.21.15.179:0/4220207387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 msgr2=0x7f16340cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 --2- 172.21.15.179:0/4220207387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f16340cdba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16240098b0 tx=0x7f1624009bc0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- 172.21.15.179:0/4220207387 shutdown_connections 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 --2- 172.21.15.179:0/4220207387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f16340cdba0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- 172.21.15.179:0/4220207387 >> 172.21.15.179:0/4220207387 conn(0x7f16340d2ed0 msgr2=0x7f16340d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- 172.21.15.179:0/4220207387 shutdown_connections 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- 172.21.15.179:0/4220207387 wait complete. 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 Processor -- start 2024-05-08T22:40:12.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.013+0000 7f163359e700 1 -- start start 2024-05-08T22:40:12.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f1634157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 0x7f16341f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340caff0 con 0x7f16340ce0e0 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634152e40 con 0x7f163415d0c0 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cb290 con 0x7f16340b8a80 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40198/0 (socket says 172.21.15.179:40198) 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 -- 172.21.15.179:0/2302823473 learned_addr learned my addr 172.21.15.179:0/2302823473 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1632d9d700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 0x7f16341f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163259c700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f1634157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 -- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 msgr2=0x7f1634157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f1634157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 -- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 msgr2=0x7f16341f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 0x7f16341f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 -- 172.21.15.179:0/2302823473 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240094b0 con 0x7f16340ce0e0 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1632d9d700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 0x7f16341f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1631d9b700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1634161c60 tx=0x7f1624004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624006ea0 con 0x7f16340ce0e0 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162401f070 con 0x7f16340ce0e0 2024-05-08T22:40:12.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340cb570 con 0x7f16340ce0e0 2024-05-08T22:40:12.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624011870 con 0x7f16340ce0e0 2024-05-08T22:40:12.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1634152660 con 0x7f16340ce0e0 2024-05-08T22:40:12.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1624021030 con 0x7f16340ce0e0 2024-05-08T22:40:12.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.017+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163404e700 con 0x7f16340ce0e0 2024-05-08T22:40:12.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.021+0000 7f1622ffd700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 0x7f1614068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.021+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f16240856d0 con 0x7f16340ce0e0 2024-05-08T22:40:12.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.021+0000 7f163259c700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 0x7f1614068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.021+0000 7f163259c700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 0x7f1614068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1634072e00 tx=0x7f1628009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.029+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1624052890 con 0x7f16340ce0e0 2024-05-08T22:40:12.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.189+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16340676e0 con 0x7f16340ce0e0 2024-05-08T22:40:12.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.189+0000 7f1622ffd700 1 -- 172.21.15.179:0/2302823473 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1624083e60 con 0x7f16340ce0e0 2024-05-08T22:40:12.192 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:40:12.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 msgr2=0x7f1614068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 0x7f1614068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1634072e00 tx=0x7f1628009040 comp rx=0 tx=0).stop 2024-05-08T22:40:12.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 msgr2=0x7f16340c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1634161c60 tx=0x7f1624004770 comp rx=0 tx=0).stop 2024-05-08T22:40:12.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 shutdown_connections 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1614066730 0x7f1614068bf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f163415d0c0 0x7f16341f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16340ce0e0 0x7f16340c5b50 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 --2- 172.21.15.179:0/2302823473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16340b8a80 0x7f1634157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 >> 172.21.15.179:0/2302823473 conn(0x7f16340d2ed0 msgr2=0x7f163406b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 shutdown_connections 2024-05-08T22:40:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.193+0000 7f163359e700 1 -- 172.21.15.179:0/2302823473 wait complete. 2024-05-08T22:40:12.208 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:40:12.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.321+0000 7f311b59e700 1 Processor -- start 2024-05-08T22:40:12.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.321+0000 7f311b59e700 1 -- start start 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f311c142890 0x7f311c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f311c06d4b0 0x7f311c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0b22d0 con 0x7f311c0b2ca0 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0b19e0 con 0x7f311c06d4b0 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c15e640 con 0x7f311c142890 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f311c06d4b0 0x7f311c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f311c06d4b0 0x7f311c15d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40214/0 (socket says 172.21.15.179:40214) 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 -- 172.21.15.179:0/3127552088 learned_addr learned my addr 172.21.15.179:0/3127552088 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:12.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311a59c700 1 --2- 172.21.15.179:0/3127552088 >> v2:172.21.15.195:3300/0 conn(0x7f311c142890 0x7f311c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 -- 172.21.15.179:0/3127552088 >> v2:172.21.15.195:3300/0 conn(0x7f311c142890 msgr2=0x7f311c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 --2- 172.21.15.179:0/3127552088 >> v2:172.21.15.195:3300/0 conn(0x7f311c142890 0x7f311c0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 -- 172.21.15.179:0/3127552088 >> v1:172.21.15.195:6789/0 conn(0x7f311c0b2ca0 legacy=0x7f311c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 -- 172.21.15.179:0/3127552088 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0b1c80 con 0x7f311c06d4b0 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311ad9d700 1 --2- 172.21.15.179:0/3127552088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c06d4b0 0x7f311c15d830 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f311c0720b0 tx=0x7f311400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0481c15bb6ea3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f3119d9b700 1 -- 172.21.15.179:0/3127552088 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400e070 con 0x7f311c06d4b0 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f3119d9b700 1 -- 172.21.15.179:0/3127552088 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311400b910 con 0x7f311c06d4b0 2024-05-08T22:40:12.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f3119d9b700 1 -- 172.21.15.179:0/3127552088 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400bcc0 con 0x7f311c06d4b0 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- 172.21.15.179:0/3127552088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c06d4b0 msgr2=0x7f311c15d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 --2- 172.21.15.179:0/3127552088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c06d4b0 0x7f311c15d830 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f311c0720b0 tx=0x7f311400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- 172.21.15.179:0/3127552088 shutdown_connections 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 --2- 172.21.15.179:0/3127552088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c06d4b0 0x7f311c15d830 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 --2- 172.21.15.179:0/3127552088 >> v2:172.21.15.195:3300/0 conn(0x7f311c142890 0x7f311c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- 172.21.15.179:0/3127552088 >> 172.21.15.179:0/3127552088 conn(0x7f311c0cea30 msgr2=0x7f311c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- 172.21.15.179:0/3127552088 shutdown_connections 2024-05-08T22:40:12.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 -- 172.21.15.179:0/3127552088 wait complete. 2024-05-08T22:40:12.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.325+0000 7f311b59e700 1 Processor -- start 2024-05-08T22:40:12.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- start start 2024-05-08T22:40:12.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 0x7f311c135fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 0x7f311c136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c142890 0x7f311c136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c15a2d0 con 0x7f311c0b2ca0 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c138bc0 con 0x7f311c06d4b0 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c139180 con 0x7f311c142890 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f3111d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 0x7f311c136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f3111d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 0x7f311c136520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40222/0 (socket says 172.21.15.179:40222) 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f3111d9b700 1 -- 172.21.15.179:0/3915572166 learned_addr learned my addr 172.21.15.179:0/3915572166 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 0x7f311c135fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311ad9d700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c142890 0x7f311c136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 -- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c142890 msgr2=0x7f311c136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c142890 0x7f311c136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 -- 172.21.15.179:0/3915572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 msgr2=0x7f311c136520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 0x7f311c136520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 -- 172.21.15.179:0/3915572166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114009b30 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311a59c700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 0x7f311c135fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e870 tx=0x7f310c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c013680 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c0137e0 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c1f0780 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c013b70 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c15a7c0 con 0x7f311c06d4b0 2024-05-08T22:40:12.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.329+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c15a9b0 con 0x7f311c06d4b0 2024-05-08T22:40:12.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.333+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f310c013cd0 con 0x7f311c06d4b0 2024-05-08T22:40:12.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.333+0000 7f31137fe700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 0x7f3100068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.333+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f310c084190 con 0x7f311c06d4b0 2024-05-08T22:40:12.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.333+0000 7f3111d9b700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 0x7f3100068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.333+0000 7f3111d9b700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 0x7f3100068bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f311c149c20 tx=0x7f3104009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.341+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f310c0512a0 con 0x7f311c06d4b0 2024-05-08T22:40:12.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.513+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f311c04e700 con 0x7f311c06d4b0 2024-05-08T22:40:12.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.513+0000 7f31137fe700 1 -- 172.21.15.179:0/3915572166 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f310c0548c0 con 0x7f311c06d4b0 2024-05-08T22:40:12.516 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 msgr2=0x7f3100068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 0x7f3100068bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f311c149c20 tx=0x7f3104009040 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 msgr2=0x7f311c135fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 0x7f311c135fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e870 tx=0x7f310c00eb80 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 shutdown_connections 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3100066730 0x7f3100068bf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c142890 0x7f311c136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f311c0b2ca0 0x7f311c136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 --2- 172.21.15.179:0/3915572166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f311c06d4b0 0x7f311c135fe0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 >> 172.21.15.179:0/3915572166 conn(0x7f311c0cea30 msgr2=0x7f311c0cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 shutdown_connections 2024-05-08T22:40:12.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.517+0000 7f311b59e700 1 -- 172.21.15.179:0/3915572166 wait complete. 2024-05-08T22:40:12.532 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:40:12.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 Processor -- start 2024-05-08T22:40:12.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 -- start start 2024-05-08T22:40:12.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f342406dda0 0x7f34240cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f342406e180 0x7f342406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342406b540 con 0x7f34240b25d0 2024-05-08T22:40:12.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342406ac50 con 0x7f342406e180 2024-05-08T22:40:12.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342ab67700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342406aef0 con 0x7f342406dda0 2024-05-08T22:40:12.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342a366700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f34240b25d0 0x7f34240b2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35256/0 (socket says 172.21.15.179:35256) 2024-05-08T22:40:12.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f342a366700 1 -- 172.21.15.179:0/3353689148 learned_addr learned my addr 172.21.15.179:0/3353689148 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:12.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f3429364700 1 --2- 172.21.15.179:0/3353689148 >> v2:172.21.15.195:3300/0 conn(0x7f342406e180 0x7f342406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.645+0000 7f3429b65700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f342406dda0 0x7f34240cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3428b63700 1 -- 172.21.15.179:0/3353689148 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888255810 0 0) 0x7f342406b540 con 0x7f34240b25d0 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3428b63700 1 -- 172.21.15.179:0/3353689148 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34100035d0 con 0x7f34240b25d0 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3429364700 1 -- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f342406dda0 msgr2=0x7f34240cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3429364700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f342406dda0 0x7f34240cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3429364700 1 -- 172.21.15.179:0/3353689148 >> v1:172.21.15.179:6789/0 conn(0x7f34240b25d0 legacy=0x7f34240b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3429364700 1 -- 172.21.15.179:0/3353689148 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342406b1d0 con 0x7f342406e180 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3429364700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406e180 0x7f342406e5e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3420009850 tx=0x7f3420009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39dd304169a23616 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3428b63700 1 -- 172.21.15.179:0/3353689148 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420011830 con 0x7f342406e180 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3428b63700 1 -- 172.21.15.179:0/3353689148 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420011990 con 0x7f342406e180 2024-05-08T22:40:12.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f3428b63700 1 -- 172.21.15.179:0/3353689148 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420011d00 con 0x7f342406e180 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 -- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406e180 msgr2=0x7f342406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406e180 0x7f342406e5e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3420009850 tx=0x7f3420009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 -- 172.21.15.179:0/3353689148 shutdown_connections 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406e180 0x7f342406e5e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 --2- 172.21.15.179:0/3353689148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f342406dda0 0x7f34240cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 -- 172.21.15.179:0/3353689148 >> 172.21.15.179:0/3353689148 conn(0x7f34240ce960 msgr2=0x7f34240ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 -- 172.21.15.179:0/3353689148 shutdown_connections 2024-05-08T22:40:12.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.649+0000 7f342ab67700 1 -- 172.21.15.179:0/3353689148 wait complete. 2024-05-08T22:40:12.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 Processor -- start 2024-05-08T22:40:12.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 -- start start 2024-05-08T22:40:12.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406dda0 0x7f342414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34240b25d0 0x7f342414a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342414e120 con 0x7f342406e180 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241f6510 con 0x7f342406dda0 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.653+0000 7f342ab67700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342415ea90 con 0x7f34240b25d0 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40228/0 (socket says 172.21.15.179:40228) 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 -- 172.21.15.179:0/1991338443 learned_addr learned my addr 172.21.15.179:0/1991338443 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429b65700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406dda0 0x7f342414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f342a366700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34240b25d0 0x7f342414a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 -- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34240b25d0 msgr2=0x7f342414a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34240b25d0 0x7f342414a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 -- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406dda0 msgr2=0x7f342414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406dda0 0x7f342414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 -- 172.21.15.179:0/1991338443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200094b0 con 0x7f342406e180 2024-05-08T22:40:12.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429364700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f3420004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420003ad0 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420003c50 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342415ed70 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200045a0 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342415f330 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342415f4c0 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f342001a430 con 0x7f342406e180 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 0x7f3408068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:12.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3420021030 con 0x7f342406e180 2024-05-08T22:40:12.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.657+0000 7f3429b65700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 0x7f3408068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:12.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.661+0000 7f3429b65700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 0x7f3408068c20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3418009bf0 tx=0x7f3418009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:12.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.665+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3420052dc0 con 0x7f342406e180 2024-05-08T22:40:12.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.845+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f342404e700 con 0x7f342406e180 2024-05-08T22:40:12.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.845+0000 7f34167fc700 1 -- 172.21.15.179:0/1991338443 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) v1 ==== 80+0+0 (secure 0 0 0) 0x7f342002b020 con 0x7f342406e180 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 msgr2=0x7f3408068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 0x7f3408068c20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3418009bf0 tx=0x7f3418009240 comp rx=0 tx=0).stop 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 msgr2=0x7f342414a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f3420004770 comp rx=0 tx=0).stop 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 shutdown_connections 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3408066760 0x7f3408068c20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34240b25d0 0x7f342414a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342406e180 0x7f342414a280 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 --2- 172.21.15.179:0/1991338443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342406dda0 0x7f342414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:12.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 >> 172.21.15.179:0/1991338443 conn(0x7f34240ce960 msgr2=0x7f3424159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:12.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 shutdown_connections 2024-05-08T22:40:12.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:12.849+0000 7f342ab67700 1 -- 172.21.15.179:0/1991338443 wait complete. 2024-05-08T22:40:12.852 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:40:12.863 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:40:12.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:12.863 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:12.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[45949]: fuse finished with error 0 and tester_r 0 2024-05-08T22:40:12.874 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:40:18.975 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:40:18.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:18.976 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:40:18.984 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 Processor -- start 2024-05-08T22:40:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 -- start start 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f08741539c0 0x7f087406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0874159b60 con 0x7f08740b3720 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0874159e40 con 0x7f087406bc50 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087b55c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087415a120 con 0x7f08741539c0 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087ad5b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f08740b3720 0x7f087415d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:40874/0 (socket says 172.21.15.179:40874) 2024-05-08T22:40:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087a55a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f08741539c0 0x7f087406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f087ad5b700 1 -- 172.21.15.179:0/3864396878 learned_addr learned my addr 172.21.15.179:0/3864396878 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982806948 0 0) 0x7f0874159b60 con 0x7f08740b3720 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08600035d0 con 0x7f08740b3720 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623061194 0 0) 0x7f0874159e40 con 0x7f087406bc50 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.101+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0874159b60 con 0x7f087406bc50 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087a55a700 1 -- 172.21.15.179:0/3864396878 >> v1:172.21.15.195:6789/0 conn(0x7f087406bc50 legacy=0x7f08740b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087a55a700 1 -- 172.21.15.179:0/3864396878 >> v1:172.21.15.179:6789/0 conn(0x7f08740b3720 legacy=0x7f087415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087a55a700 1 -- 172.21.15.179:0/3864396878 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087415a400 con 0x7f08741539c0 2024-05-08T22:40:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087a55a700 1 --2- 172.21.15.179:0/3864396878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08741539c0 0x7f087406b710 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0864009850 tx=0x7f0864009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe9e2b2a8a0f962 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864011830 con 0x7f08741539c0 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864011990 con 0x7f08741539c0 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f0879558700 1 -- 172.21.15.179:0/3864396878 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864011d00 con 0x7f08741539c0 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 -- 172.21.15.179:0/3864396878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08741539c0 msgr2=0x7f087406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 --2- 172.21.15.179:0/3864396878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08741539c0 0x7f087406b710 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0864009850 tx=0x7f0864009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 -- 172.21.15.179:0/3864396878 shutdown_connections 2024-05-08T22:40:19.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 --2- 172.21.15.179:0/3864396878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08741539c0 0x7f087406b710 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 -- 172.21.15.179:0/3864396878 >> 172.21.15.179:0/3864396878 conn(0x7f08740ce9d0 msgr2=0x7f08740cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 -- 172.21.15.179:0/3864396878 shutdown_connections 2024-05-08T22:40:19.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.105+0000 7f087b55c700 1 -- 172.21.15.179:0/3864396878 wait complete. 2024-05-08T22:40:19.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 Processor -- start 2024-05-08T22:40:19.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- start start 2024-05-08T22:40:19.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 0x7f0874148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08741539c0 0x7f087414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087414d890 con 0x7f08740b3720 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087414d5b0 con 0x7f087406bc50 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087414fdc0 con 0x7f08741539c0 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35190/0 (socket says 172.21.15.179:35190) 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 -- 172.21.15.179:0/50265394 learned_addr learned my addr 172.21.15.179:0/50265394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087ad5b700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08741539c0 0x7f087414ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087a55a700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 0x7f0874148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 -- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08741539c0 msgr2=0x7f087414ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08741539c0 0x7f087414ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 -- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 msgr2=0x7f0874148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 0x7f0874148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 -- 172.21.15.179:0/50265394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640094b0 con 0x7f08740b3720 2024-05-08T22:40:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087a55a700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 0x7f0874148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f0879d59700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0870001ee0 tx=0x7f08700021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870003c20 con 0x7f08740b3720 2024-05-08T22:40:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870003da0 con 0x7f08740b3720 2024-05-08T22:40:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08741500a0 con 0x7f08740b3720 2024-05-08T22:40:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000abb0 con 0x7f08740b3720 2024-05-08T22:40:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.109+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0874148a40 con 0x7f08740b3720 2024-05-08T22:40:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874067740 con 0x7f08740b3720 2024-05-08T22:40:19.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f087000ad10 con 0x7f08740b3720 2024-05-08T22:40:19.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f086affd700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 0x7f0858068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f08700843f0 con 0x7f08740b3720 2024-05-08T22:40:19.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f087a55a700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 0x7f0858068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.113+0000 7f087a55a700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 0x7f0858068bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f08640090b0 tx=0x7f0864008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.121+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0870051590 con 0x7f08740b3720 2024-05-08T22:40:19.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.329+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087404e700 con 0x7f08740b3720 2024-05-08T22:40:19.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.329+0000 7f086affd700 1 -- 172.21.15.179:0/50265394 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f0870054bb0 con 0x7f08740b3720 2024-05-08T22:40:19.333 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:19.333 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:04.156894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/1693330449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1693330449},{"type":"v1","addr":"172.21.15.179:6835","nonce":1693330449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:19.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 msgr2=0x7f0858068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 0x7f0858068bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f08640090b0 tx=0x7f0864008030 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 msgr2=0x7f087414c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0870001ee0 tx=0x7f08700021f0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 shutdown_connections 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0858066730 0x7f0858068bf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08741539c0 0x7f087414ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08740b3720 0x7f087414c710 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 --2- 172.21.15.179:0/50265394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087406bc50 0x7f0874148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 >> 172.21.15.179:0/50265394 conn(0x7f08740ce9d0 msgr2=0x7f08740afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 shutdown_connections 2024-05-08T22:40:19.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.333+0000 7f087b55c700 1 -- 172.21.15.179:0/50265394 wait complete. 2024-05-08T22:40:19.337 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 55 2024-05-08T22:40:19.348 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T22:40:19.481 INFO:teuthology.orchestra.run.smithi179.stdout:{"message":"","return_code":0} 2024-05-08T22:40:19.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:40:19.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:40:19.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 Processor -- start 2024-05-08T22:40:19.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 -- start start 2024-05-08T22:40:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03a806c2d0 0x7f03a80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03a80b4610 0x7f03a80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a80ca620 con 0x7f03a8159930 2024-05-08T22:40:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80ca900 con 0x7f03a806c2d0 2024-05-08T22:40:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03af96e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80cabe0 con 0x7f03a80b4610 2024-05-08T22:40:19.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.589+0000 7f03acf09700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03a806c2d0 0x7f03a80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03a806c2d0 0x7f03a80b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35192/0 (socket says 172.21.15.179:35192) 2024-05-08T22:40:19.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 -- 172.21.15.179:0/1037856399 learned_addr learned my addr 172.21.15.179:0/1037856399 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03adf0b700 1 --2- 172.21.15.179:0/1037856399 >> v2:172.21.15.195:3300/0 conn(0x7f03a80b4610 0x7f03a80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 -- 172.21.15.179:0/1037856399 >> v2:172.21.15.195:3300/0 conn(0x7f03a80b4610 msgr2=0x7f03a80cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 --2- 172.21.15.179:0/1037856399 >> v2:172.21.15.195:3300/0 conn(0x7f03a80b4610 0x7f03a80cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 -- 172.21.15.179:0/1037856399 >> v1:172.21.15.195:6789/0 conn(0x7f03a8159930 legacy=0x7f03a806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 -- 172.21.15.179:0/1037856399 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80caec0 con 0x7f03a806c2d0 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03acf09700 1 --2- 172.21.15.179:0/1037856399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a806c2d0 0x7f03a80b40d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f03a80ceba0 tx=0x7f03a400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a565e2d4f2aeb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f039ffff700 1 -- 172.21.15.179:0/1037856399 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4010070 con 0x7f03a806c2d0 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f039ffff700 1 -- 172.21.15.179:0/1037856399 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a400dce0 con 0x7f03a806c2d0 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f039ffff700 1 -- 172.21.15.179:0/1037856399 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400b310 con 0x7f03a806c2d0 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 -- 172.21.15.179:0/1037856399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a806c2d0 msgr2=0x7f03a80b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 --2- 172.21.15.179:0/1037856399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a806c2d0 0x7f03a80b40d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f03a80ceba0 tx=0x7f03a400d430 comp rx=0 tx=0).stop 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 -- 172.21.15.179:0/1037856399 shutdown_connections 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 --2- 172.21.15.179:0/1037856399 >> v2:172.21.15.195:3300/0 conn(0x7f03a80b4610 0x7f03a80cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 --2- 172.21.15.179:0/1037856399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a806c2d0 0x7f03a80b40d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 -- 172.21.15.179:0/1037856399 >> 172.21.15.179:0/1037856399 conn(0x7f03a8070e50 msgr2=0x7f03a8071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 -- 172.21.15.179:0/1037856399 shutdown_connections 2024-05-08T22:40:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.593+0000 7f03af96e700 1 -- 172.21.15.179:0/1037856399 wait complete. 2024-05-08T22:40:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 Processor -- start 2024-05-08T22:40:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- start start 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a806c2d0 0x7f03a81ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a8159930 0x7f03a806a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8155a50 con 0x7f03a80b4610 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81f3080 con 0x7f03a806c2d0 2024-05-08T22:40:19.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81f22c0 con 0x7f03a8159930 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35200/0 (socket says 172.21.15.179:35200) 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 -- 172.21.15.179:0/2678688035 learned_addr learned my addr 172.21.15.179:0/2678688035 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03ad70a700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a806c2d0 0x7f03a81ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03adf0b700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a8159930 0x7f03a806a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 -- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a8159930 msgr2=0x7f03a806a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a8159930 0x7f03a806a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 -- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a806c2d0 msgr2=0x7f03a81ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a806c2d0 0x7f03a81ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 -- 172.21.15.179:0/2678688035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a400bd40 con 0x7f03a80b4610 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03acf09700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f03a400d470 tx=0x7f03a4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400bbc0 con 0x7f03a80b4610 2024-05-08T22:40:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a400b4c0 con 0x7f03a80b4610 2024-05-08T22:40:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a81f25a0 con 0x7f03a80b4610 2024-05-08T22:40:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4004280 con 0x7f03a80b4610 2024-05-08T22:40:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.597+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a81f2b60 con 0x7f03a80b4610 2024-05-08T22:40:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.601+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a8067770 con 0x7f03a80b4610 2024-05-08T22:40:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.601+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03a40043e0 con 0x7f03a80b4610 2024-05-08T22:40:19.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.601+0000 7f039dffb700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 0x7f038c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.601+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f03a4088100 con 0x7f03a80b4610 2024-05-08T22:40:19.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.601+0000 7f03ad70a700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 0x7f038c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.605+0000 7f03ad70a700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 0x7f038c068bf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0398000f60 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.609+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03a40553c0 con 0x7f03a80b4610 2024-05-08T22:40:19.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.817+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f03a806f820 con 0x7f03a80b4610 2024-05-08T22:40:19.823 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:40:19.821+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:40:19.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.833+0000 7f039dffb700 1 -- 172.21.15.179:0/2678688035 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f03a40589e0 con 0x7f03a80b4610 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 msgr2=0x7f038c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 0x7f038c068bf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0398000f60 tx=0x7f0398009040 comp rx=0 tx=0).stop 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 msgr2=0x7f03a806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f03a400d470 tx=0x7f03a4004af0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 shutdown_connections 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f038c066730 0x7f038c068bf0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a8159930 0x7f03a806a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a80b4610 0x7f03a806a320 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 --2- 172.21.15.179:0/2678688035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a806c2d0 0x7f03a81ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 >> 172.21.15.179:0/2678688035 conn(0x7f03a8070e50 msgr2=0x7f03a80b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 shutdown_connections 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.837+0000 7f03af96e700 1 -- 172.21.15.179:0/2678688035 wait complete. 2024-05-08T22:40:19.842 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:40:19.845 INFO:tasks.ceph.mds.a.smithi179.stderr: -3762> 2024-05-08T22:40:07.089+0000 7fe904d82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:40:19.855 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-08T22:40:19.877 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:40:19.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 Processor -- start 2024-05-08T22:40:19.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- start start 2024-05-08T22:40:19.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f91174ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117f6700 0x55f9117f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9116b22f0 con 0x55f911752a80 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91166e8e0 con 0x55f9117f6700 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91166e580 con 0x55f91171f1e0 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35246/0 (socket says 172.21.15.179:35246) 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 -- 172.21.15.179:0/3395609087 learned_addr learned my addr 172.21.15.179:0/3395609087 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531de40700 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f91174ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531e641700 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117f6700 0x55f9117f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 -- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 msgr2=0x55f91174ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f91174ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 -- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117f6700 msgr2=0x55f9117f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117f6700 0x55f9117f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 -- 172.21.15.179:0/3395609087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531000a100 con 0x55f911752a80 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f531d63f700 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f531400b370 tx=0x7f531400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80226e15a84b21b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f530ffff700 1 -- 172.21.15.179:0/3395609087 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314014070 con 0x55f911752a80 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f530ffff700 1 -- 172.21.15.179:0/3395609087 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531400f3f0 con 0x55f911752a80 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f530ffff700 1 -- 172.21.15.179:0/3395609087 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400f780 con 0x55f911752a80 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- 172.21.15.179:0/3395609087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 msgr2=0x55f9117f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f531400b370 tx=0x7f531400b680 comp rx=0 tx=0).stop 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- 172.21.15.179:0/3395609087 shutdown_connections 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117f6700 0x55f9117f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f911752a80 0x55f9117f3390 secure :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f531400b370 tx=0x7f531400b680 comp rx=0 tx=0).stop 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- 172.21.15.179:0/3395609087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f91174ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- 172.21.15.179:0/3395609087 >> 172.21.15.179:0/3395609087 conn(0x55f911725740 msgr2=0x55f911768a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- 172.21.15.179:0/3395609087 shutdown_connections 2024-05-08T22:40:19.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- 172.21.15.179:0/3395609087 wait complete. 2024-05-08T22:40:19.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 Processor -- start 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- start start 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f9117e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 0x55f91174c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117e54c0 0x55f9117e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9116448d0 con 0x55f9117f6700 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9117e7310 con 0x55f9117e54c0 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.889+0000 7f532030b140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9117e5eb0 con 0x55f91171f1e0 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 0x55f91174c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 0x55f91174c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35250/0 (socket says 172.21.15.179:35250) 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 -- 172.21.15.179:0/1201780763 learned_addr learned my addr 172.21.15.179:0/1201780763 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531de40700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f9117e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531e641700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117e54c0 0x55f9117e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 -- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 msgr2=0x55f9117e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f91171f1e0 0x55f9117e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 -- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117e54c0 msgr2=0x55f9117e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9117e54c0 0x55f9117e58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 -- 172.21.15.179:0/1201780763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140015b0 con 0x55f9117f6700 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f531d63f700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 0x55f91174c2e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5314000c00 tx=0x7f5314000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f530effd700 1 -- 172.21.15.179:0/1201780763 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314014030 con 0x55f9117f6700 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f530effd700 1 -- 172.21.15.179:0/1201780763 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531401b840 con 0x55f9117f6700 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f530effd700 1 -- 172.21.15.179:0/1201780763 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531401bbd0 con 0x55f9117f6700 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f91165cb00 con 0x55f9117f6700 2024-05-08T22:40:19.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91164ff40 con 0x55f9117f6700 2024-05-08T22:40:19.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.893+0000 7f530effd700 1 -- 172.21.15.179:0/1201780763 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f531400d040 con 0x55f9117f6700 2024-05-08T22:40:19.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f530effd700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5300065090 0x7f5300067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f530effd700 1 -- 172.21.15.179:0/1201780763 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f5314015030 con 0x55f9117f6700 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f532030b140 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55f911766700 0x55f911766b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x55f911767050 con 0x55f911766700 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f531de40700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5300065090 0x7f5300067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f531e641700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55f911766700 0x55f911766b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f531de40700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5300065090 0x7f5300067550 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5310013040 tx=0x7f5310007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.897+0000 7f531e641700 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55f911766700 0x55f911766b10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f531e641700 1 -- 172.21.15.179:0/1201780763 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v99'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5318009860 con 0x55f911766700 2024-05-08T22:40:19.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55f911766700 msgr2=0x55f911766b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55f911766700 0x55f911766b10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5300065090 msgr2=0x7f5300067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5300065090 0x7f5300067550 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5310013040 tx=0x7f5310007030 comp rx=0 tx=0).stop 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 msgr2=0x55f91174c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 --2- 172.21.15.179:0/1201780763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9117f6700 0x55f91174c2e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5314000c00 tx=0x7f5314000ed0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f531e641700 1 -- 172.21.15.179:0/1201780763 reap_dead start 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 shutdown_connections 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 >> 172.21.15.179:0/1201780763 conn(0x55f911725740 msgr2=0x55f9117f3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 shutdown_connections 2024-05-08T22:40:19.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.901+0000 7f532030b140 1 -- 172.21.15.179:0/1201780763 wait complete. 2024-05-08T22:40:19.909 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.941+0000 7f19f844d500 1 -- start start 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.941+0000 7f19f844d500 1 Processor -- start 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a529800 0x55a38a5ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a529000 0x55a38a5cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a55fe00 con 0x55a38a529400 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aaa00 con 0x55a38a529000 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa640 con 0x55a38a529800 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35256/0 (socket says 172.21.15.179:35256) 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 -- 172.21.15.179:0/1730242785 learned_addr learned my addr 172.21.15.179:0/1730242785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6c07700 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a529800 0x55a38a5ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f7408700 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a529000 0x55a38a5cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 -- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a529800 msgr2=0x55a38a5ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a529800 0x55a38a5ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 -- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a529000 msgr2=0x55a38a5cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a529000 0x55a38a5cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 -- 172.21.15.179:0/1730242785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a38a54a9c0 con 0x55a38a529400 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f6406700 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x55a38a6372c0 tx=0x55a38b3a6240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbb1f77195d16a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f4402700 1 -- 172.21.15.179:0/1730242785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a559600 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f4402700 1 -- 172.21.15.179:0/1730242785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a38b312a80 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f4402700 1 -- 172.21.15.179:0/1730242785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a5591e0 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a38a54ab60 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a38a54a340 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 4 waiting for MDS map... 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f4402700 1 -- 172.21.15.179:0/1730242785 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x55a38a562c40 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f4402700 1 -- 172.21.15.179:0/1730242785 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 56) v1 ==== 2573+0+0 (secure 0 0 0) 0x55a38b3b0900 con 0x55a38a529400 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 4 Got MDS map 56 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 10 main: JournalTool::main 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.945+0000 7f19f844d500 4 main: JournalTool: connecting to RADOS... 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 Processor -- start 2024-05-08T22:40:19.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- start start 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa500 con 0x55a38a824c00 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa3c0 con 0x55a38a824400 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aaa00 con 0x55a38a824800 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35266/0 (socket says 172.21.15.179:35266) 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 -- 172.21.15.179:0/3750192259 learned_addr learned my addr 172.21.15.179:0/3750192259 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f7408700 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6c07700 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 -- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 msgr2=0x55a38a5cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 -- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 msgr2=0x55a38a5cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 -- 172.21.15.179:0/3750192259 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a38a54a000 con 0x55a38a824c00 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f6406700 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x55a38a87a690 tx=0x55a38b3a60c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9439f9a687869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f23fe700 1 -- 172.21.15.179:0/3750192259 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a558f20 con 0x55a38a824c00 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f23fe700 1 -- 172.21.15.179:0/3750192259 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a38b313080 con 0x55a38a824c00 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f23fe700 1 -- 172.21.15.179:0/3750192259 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a559600 con 0x55a38a824c00 2024-05-08T22:40:19.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- 172.21.15.179:0/3750192259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 msgr2=0x55a38a5d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x55a38a87a690 tx=0x55a38b3a60c0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- 172.21.15.179:0/3750192259 shutdown_connections 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0100 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 --2- 172.21.15.179:0/3750192259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- 172.21.15.179:0/3750192259 >> 172.21.15.179:0/3750192259 conn(0x55a38a528c00 msgr2=0x55a38a5e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- 172.21.15.179:0/3750192259 shutdown_connections 2024-05-08T22:40:19.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.949+0000 7f19f844d500 1 -- 172.21.15.179:0/3750192259 wait complete. 2024-05-08T22:40:19.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 Processor -- start 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- start start 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa280 con 0x55a38a824c00 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa140 con 0x55a38a824400 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa000 con 0x55a38a824800 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35278/0 (socket says 172.21.15.179:35278) 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6c07700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f7408700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 -- 172.21.15.179:0/1289399956 learned_addr learned my addr 172.21.15.179:0/1289399956 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 -- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 msgr2=0x55a38a5d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 -- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 msgr2=0x55a38a5d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 -- 172.21.15.179:0/1289399956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a38a54a9c0 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f6406700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55a38a87a900 tx=0x55a38a836c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f0bfb700 1 -- 172.21.15.179:0/1289399956 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a5591e0 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f0bfb700 1 -- 172.21.15.179:0/1289399956 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a38b312d80 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f0bfb700 1 -- 172.21.15.179:0/1289399956 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a558dc0 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a38a54ab60 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a38a54a340 con 0x55a38a824c00 2024-05-08T22:40:19.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.953+0000 7f19f844d500 4 main: JournalTool: resolving pool 13 2024-05-08T22:40:19.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f0bfb700 1 -- 172.21.15.179:0/1289399956 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55a38a588840 con 0x55a38a824c00 2024-05-08T22:40:19.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f0bfb700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 0x55a38a5d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f0bfb700 1 -- 172.21.15.179:0/1289399956 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x55a38a562a80 con 0x55a38a824c00 2024-05-08T22:40:19.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f844d500 4 main: JournalTool: creating IoCtx.. 2024-05-08T22:40:19.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f844d500 4 main: Executing for rank 0 2024-05-08T22:40:19.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f7408700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 0x55a38a5d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.957+0000 7f19f7408700 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 0x55a38a5d1700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55a38a57eea0 tx=0x55a38a5ff860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- start start 2024-05-08T22:40:19.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 Processor -- start 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a825400 0x55a38a5d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a97e400 0x55a38a5cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5ab040 con 0x55a38a825800 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa500 con 0x55a38a97e400 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a38a5aa000 con 0x55a38a825400 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35290/0 (socket says 172.21.15.179:35290) 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 learned_addr learned my addr 172.21.15.179:0/4133531721 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:19.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a825400 0x55a38a5d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6406700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a97e400 0x55a38a5cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a825400 msgr2=0x55a38a5d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a825400 0x55a38a5d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a97e400 msgr2=0x55a38a5cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a97e400 0x55a38a5cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a38a8cb860 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f6c07700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x55a38a94f350 tx=0x55a38a989800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4027400aa3d453 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19edbf5700 1 -- 172.21.15.179:0/4133531721 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a559600 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19edbf5700 1 -- 172.21.15.179:0/4133531721 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a38a574480 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19edbf5700 1 -- 172.21.15.179:0/4133531721 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a38a558420 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a38a8c9ba0 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a38a8c9a00 con 0x55a38a825800 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 4 waiting for MDS map... 2024-05-08T22:40:19.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19edbf5700 1 -- 172.21.15.179:0/4133531721 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x55a38a562700 con 0x55a38a825800 2024-05-08T22:40:19.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19edbf5700 1 -- 172.21.15.179:0/4133531721 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 56) v1 ==== 2573+0+0 (secure 0 0 0) 0x55a38a985800 con 0x55a38a825800 2024-05-08T22:40:19.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 4 Got MDS map 56 2024-05-08T22:40:19.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 0x55a38a9f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97ec00 con 0x55a38a97e800 2024-05-08T22:40:19.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 0x55a38a9f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.961+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 0x55a38a9f8000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=1b02ea66cccad422 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.965+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a97e800 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.965+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55a38a97f000 0x55a38a9f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.965+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97f400 con 0x55a38a97f000 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.965+0000 7f19f6406700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55a38a97f000 0x55a38a9f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.965+0000 7f19f6406700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55a38a97f000 0x55a38a9f8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=1dab3e39b4580ada server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f6406700 1 -- 172.21.15.179:0/4133531721 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55a38b306d80 con 0x55a38a97f000 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19ec3f2700 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a9da800 con 0x55a38a97f000 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19ec3f2700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55a38a9dac00 0x55a38a9fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19ec3f2700 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a9db000 con 0x55a38a9dac00 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55a38a9dac00 0x55a38a9fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f6406700 1 -- 172.21.15.179:0/4133531721 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x55a38b306d80 con 0x55a38a97f000 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55a38a9dac00 0x55a38a9fa680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=f5bae2d5aa1f30f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a9dac00 2024-05-08T22:40:19.976 INFO:teuthology.orchestra.run.smithi179.stdout:old journal was 4209788~892 2024-05-08T22:40:19.976 INFO:teuthology.orchestra.run.smithi179.stdout:new journal start will be 8388608 (4177928 bytes past old end) 2024-05-08T22:40:19.976 INFO:teuthology.orchestra.run.smithi179.stdout:writing journal head 2024-05-08T22:40:19.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.969+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97f800 con 0x55a38a97f000 2024-05-08T22:40:19.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f6406700 1 -- 172.21.15.179:0/4133531721 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v99'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306d80 con 0x55a38a97f000 2024-05-08T22:40:19.977 INFO:teuthology.orchestra.run.smithi179.stdout:writing EResetJournal entry 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97fc00 con 0x55a38a9dac00 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55a38a9db000 0x55a38a9f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a9db400 con 0x55a38a9db000 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a9db800 con 0x55a38a97e800 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55a38a9dbc00 0x55a38a9f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97ec00 con 0x55a38a9dbc00 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38aa10000 con 0x55a38a9dac00 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 0x55a38a9f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38aa10800 con 0x55a38aa10400 2024-05-08T22:40:19.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f6406700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55a38a9dbc00 0x55a38a9f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f6c07700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55a38a9db000 0x55a38a9f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 0x55a38a9f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.973+0000 7f19f6406700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55a38a9dbc00 0x55a38a9f9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=f6338f1e8d27c8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f7408700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 0x55a38a9f9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=ed0e00fab0903d34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f6c07700 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55a38a9db000 0x55a38a9f8b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=ff5dc200cb646f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f6406700 1 -- 172.21.15.179:0/4133531721 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(9 200.00000005 [delete] v99'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306d80 con 0x55a38a9dbc00 2024-05-08T22:40:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(8 200.00000004 [delete] v99'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a97e800 2024-05-08T22:40:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(6 200.00000002 [delete] v99'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a9dac00 2024-05-08T22:40:19.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(10 200.00000006 [delete] v99'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a9dac00 2024-05-08T22:40:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(11 200.00000007 [delete] v99'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38aa10400 2024-05-08T22:40:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(7 200.00000003 [delete] v99'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b3078c0 con 0x55a38a9db000 2024-05-08T22:40:19.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.977+0000 7f19ec3f2700 1 -- 172.21.15.179:0/4133531721 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x55a38a97f400 con 0x55a38a9dac00 2024-05-08T22:40:19.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f7408700 1 -- 172.21.15.179:0/4133531721 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v99'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a38b306b40 con 0x55a38a9dac00 2024-05-08T22:40:19.983 INFO:teuthology.orchestra.run.smithi179.stdout:done 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55a38a97f000 msgr2=0x55a38a9f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55a38a97f000 0x55a38a9f8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55a38a9dbc00 msgr2=0x55a38a9f9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55a38a9dbc00 0x55a38a9f9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55a38a9dac00 msgr2=0x55a38a9fa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55a38a9dac00 0x55a38a9fa680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55a38a9db000 msgr2=0x55a38a9f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55a38a9db000 0x55a38a9f8b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 reap_dead start 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 msgr2=0x55a38a9f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f6c07700 1 -- 172.21.15.179:0/4133531721 reap_dead start 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 0x55a38a9f8000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 msgr2=0x55a38a9f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 0x55a38a9f9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 msgr2=0x55a38a5cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x55a38a94f350 tx=0x55a38a989800 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 shutdown_connections 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55a38aa10400 0x55a38a9f9600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55a38a97e800 0x55a38a9f8000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/4133531721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a825800 0x55a38a5cfb80 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 >> 172.21.15.179:0/4133531721 conn(0x55a38a825000 msgr2=0x55a38a5e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 shutdown_connections 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/4133531721 wait complete. 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 msgr2=0x55a38a5d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 0x55a38a5d1700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55a38a57eea0 tx=0x55a38a5ff860 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 msgr2=0x55a38a5d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55a38a87a900 tx=0x55a38a836c00 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 shutdown_connections 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55a38a97e000 0x55a38a5d1700 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a824c00 0x55a38a5d0680 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a824800 0x55a38a5d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1289399956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a824400 0x55a38a5d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 >> 172.21.15.179:0/1289399956 conn(0x55a38a528c00 msgr2=0x55a38a5e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 shutdown_connections 2024-05-08T22:40:19.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1289399956 wait complete. 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 msgr2=0x55a38a5ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x55a38a6372c0 tx=0x55a38b3a6240 comp rx=0 tx=0).stop 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 shutdown_connections 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a38a529000 0x55a38a5cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a38a529400 0x55a38a5ceb00 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 --2- 172.21.15.179:0/1730242785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a38a529800 0x55a38a5ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 >> 172.21.15.179:0/1730242785 conn(0x55a38a529c00 msgr2=0x55a38a5e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 shutdown_connections 2024-05-08T22:40:19.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:19.981+0000 7f19f844d500 1 -- 172.21.15.179:0/1730242785 wait complete. 2024-05-08T22:40:19.990 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.080836, result: old journal was 4209788~892 new journal start will be 8388608 (4177928 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-08T22:40:19.990 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:40:20.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 Processor -- start 2024-05-08T22:40:20.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 -- start start 2024-05-08T22:40:20.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8540159320 0x7f85400cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f85400b19e0 0x7f85400cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854006cda0 con 0x7f8540159700 2024-05-08T22:40:20.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400b1f10 con 0x7f8540159320 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400b21f0 con 0x7f85400b19e0 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8540159320 0x7f85400cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8540159320 0x7f85400cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35294/0 (socket says 172.21.15.179:35294) 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f853effd700 1 -- 172.21.15.179:0/1916704730 learned_addr learned my addr 172.21.15.179:0/1916704730 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.153+0000 7f85367fc700 1 --2- 172.21.15.179:0/1916704730 >> v2:172.21.15.195:3300/0 conn(0x7f85400b19e0 0x7f85400cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853effd700 1 -- 172.21.15.179:0/1916704730 >> v2:172.21.15.195:3300/0 conn(0x7f85400b19e0 msgr2=0x7f85400cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853effd700 1 --2- 172.21.15.179:0/1916704730 >> v2:172.21.15.195:3300/0 conn(0x7f85400b19e0 0x7f85400cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853effd700 1 -- 172.21.15.179:0/1916704730 >> v1:172.21.15.195:6789/0 conn(0x7f8540159700 legacy=0x7f854006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853effd700 1 -- 172.21.15.179:0/1916704730 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400b24d0 con 0x7f8540159320 2024-05-08T22:40:20.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853effd700 1 --2- 172.21.15.179:0/1916704730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f85400cadb0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f854006ef00 tx=0x7f8530009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a65af5cea38983 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853e7fc700 1 -- 172.21.15.179:0/1916704730 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000f8a0 con 0x7f8540159320 2024-05-08T22:40:20.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853e7fc700 1 -- 172.21.15.179:0/1916704730 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000fa00 con 0x7f8540159320 2024-05-08T22:40:20.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853e7fc700 1 -- 172.21.15.179:0/1916704730 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000fd90 con 0x7f8540159320 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 -- 172.21.15.179:0/1916704730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 msgr2=0x7f85400cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 --2- 172.21.15.179:0/1916704730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f85400cadb0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f854006ef00 tx=0x7f8530009bf0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 -- 172.21.15.179:0/1916704730 shutdown_connections 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 --2- 172.21.15.179:0/1916704730 >> v2:172.21.15.195:3300/0 conn(0x7f85400b19e0 0x7f85400cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 --2- 172.21.15.179:0/1916704730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f85400cadb0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 -- 172.21.15.179:0/1916704730 >> 172.21.15.179:0/1916704730 conn(0x7f85400cea30 msgr2=0x7f85400cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 -- 172.21.15.179:0/1916704730 shutdown_connections 2024-05-08T22:40:20.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 -- 172.21.15.179:0/1916704730 wait complete. 2024-05-08T22:40:20.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.157+0000 7f853ffff700 1 Processor -- start 2024-05-08T22:40:20.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- start start 2024-05-08T22:40:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85400b19e0 0x7f854015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 0x7f85401577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400ca6f0 con 0x7f8540159320 2024-05-08T22:40:20.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854015a580 con 0x7f8540159700 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854015a800 con 0x7f85400b19e0 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35298/0 (socket says 172.21.15.179:35298) 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 -- 172.21.15.179:0/3239137534 learned_addr learned my addr 172.21.15.179:0/3239137534 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853effd700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85400b19e0 0x7f854015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853f7fe700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 0x7f85401577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 -- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85400b19e0 msgr2=0x7f854015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85400b19e0 0x7f854015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 -- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 msgr2=0x7f85401577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 0x7f85401577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 -- 172.21.15.179:0/3239137534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009570 con 0x7f8540159320 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853f7fe700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 0x7f85401577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:20.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f85367fc700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f852800e8e0 tx=0x7f852800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528010820 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528010980 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854015aae0 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528010d30 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854015b0a0 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.161+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854004e700 con 0x7f8540159320 2024-05-08T22:40:20.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.165+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85280113f0 con 0x7f8540159320 2024-05-08T22:40:20.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.165+0000 7f8537fff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 0x7f8520068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.165+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8528083e60 con 0x7f8540159320 2024-05-08T22:40:20.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.165+0000 7f853effd700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 0x7f8520068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.165+0000 7f853effd700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 0x7f8520068bf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f8530008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.169+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8528050fb0 con 0x7f8540159320 2024-05-08T22:40:20.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.373+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f85400ca830 con 0x7f8540159320 2024-05-08T22:40:20.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.837+0000 7f8537fff700 1 -- 172.21.15.179:0/3239137534 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v57) v1 ==== 162+0+0 (secure 0 0 0) 0x7f85280160b0 con 0x7f8540159320 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 msgr2=0x7f8520068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 0x7f8520068bf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f8530008030 comp rx=0 tx=0).stop 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 msgr2=0x7f8540151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f852800e8e0 tx=0x7f852800ebf0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 shutdown_connections 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8520066730 0x7f8520068bf0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8540159700 0x7f85401577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540159320 0x7f8540151520 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 --2- 172.21.15.179:0/3239137534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85400b19e0 0x7f854015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 >> 172.21.15.179:0/3239137534 conn(0x7f85400cea30 msgr2=0x7f854006d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:20.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 shutdown_connections 2024-05-08T22:40:20.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.841+0000 7f853ffff700 1 -- 172.21.15.179:0/3239137534 wait complete. 2024-05-08T22:40:20.848 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:40:20.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:20.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 Processor -- start 2024-05-08T22:40:20.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 -- start start 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f792006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 0x7f79200b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f792015d970 0x7f792015de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792015e340 con 0x7f792015d970 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792006ce80 con 0x7f79200cea40 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920143a20 con 0x7f7920143640 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f792015d970 0x7f792015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f792015d970 0x7f792015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35308/0 (socket says 172.21.15.179:35308) 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 -- 172.21.15.179:0/307746966 learned_addr learned my addr 172.21.15.179:0/307746966 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:20.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791e59c700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f792006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791dd9b700 1 --2- 172.21.15.179:0/307746966 >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 0x7f79200b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 -- 172.21.15.179:0/307746966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 msgr2=0x7f792006eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f792006eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 -- 172.21.15.179:0/307746966 >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 msgr2=0x7f79200b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 --2- 172.21.15.179:0/307746966 >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 0x7f79200b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 -- 172.21.15.179:0/307746966 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200cf220 con 0x7f792015d970 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791dd9b700 1 --2- 172.21.15.179:0/307746966 >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 0x7f79200b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:20.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791ed9d700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f792015de00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7920070980 tx=0x7f7910009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc6646140bb321 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791d59a700 1 -- 172.21.15.179:0/307746966 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910011830 con 0x7f792015d970 2024-05-08T22:40:20.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791d59a700 1 -- 172.21.15.179:0/307746966 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7910011990 con 0x7f792015d970 2024-05-08T22:40:20.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.973+0000 7f791d59a700 1 -- 172.21.15.179:0/307746966 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910011d20 con 0x7f792015d970 2024-05-08T22:40:20.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- 172.21.15.179:0/307746966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 msgr2=0x7f792015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f792015de00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7920070980 tx=0x7f7910009bf0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- 172.21.15.179:0/307746966 shutdown_connections 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f792015de00 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 --2- 172.21.15.179:0/307746966 >> v2:172.21.15.195:3300/0 conn(0x7f79200cea40 0x7f79200b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 --2- 172.21.15.179:0/307746966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f792006eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- 172.21.15.179:0/307746966 >> 172.21.15.179:0/307746966 conn(0x7f79200cfae0 msgr2=0x7f79200cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- 172.21.15.179:0/307746966 shutdown_connections 2024-05-08T22:40:20.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- 172.21.15.179:0/307746966 wait complete. 2024-05-08T22:40:20.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 Processor -- start 2024-05-08T22:40:20.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.977+0000 7f791f59e700 1 -- start start 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79200cea40 0x7f7920154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f7920137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920155020 con 0x7f792015d970 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920155300 con 0x7f79200cea40 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79201555e0 con 0x7f7920143640 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35324/0 (socket says 172.21.15.179:35324) 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 -- 172.21.15.179:0/3909492748 learned_addr learned my addr 172.21.15.179:0/3909492748 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791dd9b700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f7920137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791e59c700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79200cea40 0x7f7920154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 -- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 msgr2=0x7f7920137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f7920137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 -- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79200cea40 msgr2=0x7f7920154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79200cea40 0x7f7920154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 -- 172.21.15.179:0/3909492748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009a70 con 0x7f792015d970 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791ed9d700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f792006b390 tx=0x7f7910004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910004450 con 0x7f792015d970 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79100045b0 con 0x7f792015d970 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7910009570 con 0x7f792015d970 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100038e0 con 0x7f792015d970 2024-05-08T22:40:20.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.981+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7920155ba0 con 0x7f792015d970 2024-05-08T22:40:20.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792004e700 con 0x7f792015d970 2024-05-08T22:40:20.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7910021070 con 0x7f792015d970 2024-05-08T22:40:20.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f7916ffd700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 0x7f7900068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:20.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f79100856e0 con 0x7f792015d970 2024-05-08T22:40:20.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f791e59c700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 0x7f7900068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:20.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.985+0000 7f791e59c700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 0x7f7900068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ad0 tx=0x7f7908009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:20.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:20.989+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f79100528d0 con 0x7f792015d970 2024-05-08T22:40:21.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79200b4b90 con 0x7f792015d970 2024-05-08T22:40:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f7916ffd700 1 -- 172.21.15.179:0/3909492748 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f7910055ef0 con 0x7f792015d970 2024-05-08T22:40:21.199 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:21.200 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:20.842431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 msgr2=0x7f7900068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 0x7f7900068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ad0 tx=0x7f7908009240 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 msgr2=0x7f79201560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f792006b390 tx=0x7f7910004770 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 shutdown_connections 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7900066730 0x7f7900068bf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792015d970 0x7f79201560c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7920143640 0x7f7920137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 --2- 172.21.15.179:0/3909492748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79200cea40 0x7f7920154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 >> 172.21.15.179:0/3909492748 conn(0x7f79200cfae0 msgr2=0x7f7920141220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:21.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 shutdown_connections 2024-05-08T22:40:21.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:21.197+0000 7f791f59e700 1 -- 172.21.15.179:0/3909492748 wait complete. 2024-05-08T22:40:21.203 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 58 2024-05-08T22:40:21.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:40:20.842431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8154}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8154': {'gid': 8154, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/55939533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 55939533}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 55939533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:40:21.621 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8154:up:replay 2024-05-08T22:40:22.443 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:40:22.446 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:40:22.622 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 Processor -- start 2024-05-08T22:40:22.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 -- start start 2024-05-08T22:40:22.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f37b415d5b0 0x7f37b40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37b40ce3b0 0x7f37b4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40ce840 con 0x7f37b40ce3b0 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b406afc0 con 0x7f37b415d5b0 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b9e1c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b415d990 con 0x7f37b40b2120 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37b40ce3b0 0x7f37b4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37b40ce3b0 0x7f37b4161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35340/0 (socket says 172.21.15.179:35340) 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 -- 172.21.15.179:0/274565749 learned_addr learned my addr 172.21.15.179:0/274565749 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:22.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b8e1a700 1 --2- 172.21.15.179:0/274565749 >> v2:172.21.15.195:3300/0 conn(0x7f37b415d5b0 0x7f37b40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 -- 172.21.15.179:0/274565749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 msgr2=0x7f37b40cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 --2- 172.21.15.179:0/274565749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b40cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 -- 172.21.15.179:0/274565749 >> v2:172.21.15.195:3300/0 conn(0x7f37b415d5b0 msgr2=0x7f37b40b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 --2- 172.21.15.179:0/274565749 >> v2:172.21.15.195:3300/0 conn(0x7f37b415d5b0 0x7f37b40b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 -- 172.21.15.179:0/274565749 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40d26f0 con 0x7f37b40ce3b0 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b961b700 1 --2- 172.21.15.179:0/274565749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b40ce3b0 0x7f37b4161090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f37b408a630 tx=0x7f37ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cb1cf16a8862c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b37fe700 1 -- 172.21.15.179:0/274565749 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00e070 con 0x7f37b40ce3b0 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b37fe700 1 -- 172.21.15.179:0/274565749 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00b910 con 0x7f37b40ce3b0 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.741+0000 7f37b37fe700 1 -- 172.21.15.179:0/274565749 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00bcc0 con 0x7f37b40ce3b0 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- 172.21.15.179:0/274565749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b40ce3b0 msgr2=0x7f37b4161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/274565749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b40ce3b0 0x7f37b4161090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f37b408a630 tx=0x7f37ac00b330 comp rx=0 tx=0).stop 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- 172.21.15.179:0/274565749 shutdown_connections 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/274565749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b40ce3b0 0x7f37b4161090 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/274565749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/274565749 >> v2:172.21.15.195:3300/0 conn(0x7f37b415d5b0 0x7f37b40b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- 172.21.15.179:0/274565749 >> 172.21.15.179:0/274565749 conn(0x7f37b40b4940 msgr2=0x7f37b40b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:22.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- 172.21.15.179:0/274565749 shutdown_connections 2024-05-08T22:40:22.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- 172.21.15.179:0/274565749 wait complete. 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 Processor -- start 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- start start 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b41f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37b40ce3b0 0x7f37b40ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41f41d0 con 0x7f37b415d5b0 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41f43d0 con 0x7f37b40ce3b0 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b9e1c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41f4670 con 0x7f37b40b2120 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b8e1a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b41f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37b40ce3b0 0x7f37b40ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35348/0 (socket says 172.21.15.179:35348) 2024-05-08T22:40:22.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 -- 172.21.15.179:0/1462033510 learned_addr learned my addr 172.21.15.179:0/1462033510 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:22.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 -- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 msgr2=0x7f37b41f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:22.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b41f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 -- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37b40ce3b0 msgr2=0x7f37b40ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:22.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37b40ce3b0 0x7f37b40ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 -- 172.21.15.179:0/1462033510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009b30 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.745+0000 7f37b961b700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f37b408b480 tx=0x7f37ac004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0099b0 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00ce80 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b41f4950 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004470 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b41f4f10 con 0x7f37b415d5b0 2024-05-08T22:40:22.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b406c470 con 0x7f37b415d5b0 2024-05-08T22:40:22.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37ac0045d0 con 0x7f37b415d5b0 2024-05-08T22:40:22.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 0x7f3798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:22.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b8e1a700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 0x7f3798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:22.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.749+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f37ac085d40 con 0x7f37b415d5b0 2024-05-08T22:40:22.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.753+0000 7f37b8e1a700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 0x7f3798068bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009bf0 tx=0x7f37a8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:22.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.757+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f37ac052fb0 con 0x7f37b415d5b0 2024-05-08T22:40:22.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.965+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b404e700 con 0x7f37b415d5b0 2024-05-08T22:40:22.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b17fa700 1 -- 172.21.15.179:0/1462033510 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f37ac0565d0 con 0x7f37b415d5b0 2024-05-08T22:40:22.970 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:22.971 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:22.853622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:22.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 msgr2=0x7f3798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 0x7f3798068bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009bf0 tx=0x7f37a8009240 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 msgr2=0x7f37b40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f37b408b480 tx=0x7f37ac004af0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 shutdown_connections 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3798066730 0x7f3798068bf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37b40ce3b0 0x7f37b40ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37b415d5b0 0x7f37b40c8cd0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 --2- 172.21.15.179:0/1462033510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37b40b2120 0x7f37b41f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 >> 172.21.15.179:0/1462033510 conn(0x7f37b40b4940 msgr2=0x7f37b40d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 shutdown_connections 2024-05-08T22:40:22.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:22.969+0000 7f37b9e1c700 1 -- 172.21.15.179:0/1462033510 wait complete. 2024-05-08T22:40:22.975 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 60 2024-05-08T22:40:23.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:40:22.853622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8154}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8154': {'gid': 8154, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.179:6835/55939533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 55939533}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 55939533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:40:23.388 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8154:up:rejoin 2024-05-08T22:40:24.389 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.513+0000 7ff3c255f700 1 Processor -- start 2024-05-08T22:40:24.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.513+0000 7ff3c255f700 1 -- start start 2024-05-08T22:40:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3bc13de70 0x7ff3bc15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3bc06c450 0x7ff3bc06c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc15c360 con 0x7ff3bc15b900 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc06cda0 con 0x7ff3bc06c450 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc15bda0 con 0x7ff3bc13de70 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3bc06c450 0x7ff3bc06c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3bc06c450 0x7ff3bc06c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35356/0 (socket says 172.21.15.179:35356) 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 -- 172.21.15.179:0/1545446951 learned_addr learned my addr 172.21.15.179:0/1545446951 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c155d700 1 --2- 172.21.15.179:0/1545446951 >> v2:172.21.15.195:3300/0 conn(0x7ff3bc13de70 0x7ff3bc15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 -- 172.21.15.179:0/1545446951 >> v2:172.21.15.195:3300/0 conn(0x7ff3bc13de70 msgr2=0x7ff3bc15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 --2- 172.21.15.179:0/1545446951 >> v2:172.21.15.195:3300/0 conn(0x7ff3bc13de70 0x7ff3bc15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 -- 172.21.15.179:0/1545446951 >> v1:172.21.15.195:6789/0 conn(0x7ff3bc15b900 legacy=0x7ff3bc13e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 -- 172.21.15.179:0/1545446951 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc15c080 con 0x7ff3bc06c450 2024-05-08T22:40:24.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c1d5e700 1 --2- 172.21.15.179:0/1545446951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc06c860 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc071a60 tx=0x7ff3b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6221f241be5c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3affff700 1 -- 172.21.15.179:0/1545446951 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400e070 con 0x7ff3bc06c450 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3affff700 1 -- 172.21.15.179:0/1545446951 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400b9c0 con 0x7ff3bc06c450 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3affff700 1 -- 172.21.15.179:0/1545446951 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400bd70 con 0x7ff3bc06c450 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- 172.21.15.179:0/1545446951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 msgr2=0x7ff3bc06c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 --2- 172.21.15.179:0/1545446951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc06c860 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc071a60 tx=0x7ff3b400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- 172.21.15.179:0/1545446951 shutdown_connections 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 --2- 172.21.15.179:0/1545446951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc06c860 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 --2- 172.21.15.179:0/1545446951 >> v2:172.21.15.195:3300/0 conn(0x7ff3bc13de70 0x7ff3bc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- 172.21.15.179:0/1545446951 >> 172.21.15.179:0/1545446951 conn(0x7ff3bc0cfae0 msgr2=0x7ff3bc0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:24.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- 172.21.15.179:0/1545446951 shutdown_connections 2024-05-08T22:40:24.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.517+0000 7ff3c255f700 1 -- 172.21.15.179:0/1545446951 wait complete. 2024-05-08T22:40:24.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 Processor -- start 2024-05-08T22:40:24.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 -- start start 2024-05-08T22:40:24.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 0x7ff3bc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 0x7ff3bc0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc13fe40 con 0x7ff3bc06c450 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b2b50 con 0x7ff3bc13de70 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c255f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc13cae0 con 0x7ff3bc15b900 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35362/0 (socket says 172.21.15.179:35362) 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 -- 172.21.15.179:0/121224325 learned_addr learned my addr 172.21.15.179:0/121224325 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c0d5c700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 0x7ff3bc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c1d5e700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 0x7ff3bc0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 -- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 msgr2=0x7ff3bc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 0x7ff3bc0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 -- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 msgr2=0x7ff3bc06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 0x7ff3bc06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 -- 172.21.15.179:0/121224325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b4009b30 con 0x7ff3bc06c450 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c0d5c700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 0x7ff3bc06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c1d5e700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 0x7ff3bc0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3c155d700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e760 tx=0x7ff3b000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:24.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.521+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0013610 con 0x7ff3bc06c450 2024-05-08T22:40:24.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0013770 con 0x7ff3bc06c450 2024-05-08T22:40:24.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc13fb60 con 0x7ff3bc06c450 2024-05-08T22:40:24.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0013ae0 con 0x7ff3bc06c450 2024-05-08T22:40:24.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc0b3080 con 0x7ff3bc06c450 2024-05-08T22:40:24.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc06ef10 con 0x7ff3bc06c450 2024-05-08T22:40:24.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3b0013c40 con 0x7ff3bc06c450 2024-05-08T22:40:24.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3adffb700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 0x7ff3a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:24.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7ff3b0084130 con 0x7ff3bc06c450 2024-05-08T22:40:24.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.525+0000 7ff3c0d5c700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 0x7ff3a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:24.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.529+0000 7ff3c0d5c700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 0x7ff3a0068bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0707e0 tx=0x7ff3b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:24.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.533+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff3b00511f0 con 0x7ff3bc06c450 2024-05-08T22:40:24.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3bc04e700 con 0x7ff3bc06c450 2024-05-08T22:40:24.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3adffb700 1 -- 172.21.15.179:0/121224325 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4743 (secure 0 0 0) 0x7ff3b0054810 con 0x7ff3bc06c450 2024-05-08T22:40:24.743 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:24.744 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:23.862086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 msgr2=0x7ff3a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 0x7ff3a0068bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0707e0 tx=0x7ff3b8009040 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 msgr2=0x7ff3bc13c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e760 tx=0x7ff3b000eb20 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 shutdown_connections 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a0066730 0x7ff3a0068bf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3bc13de70 0x7ff3bc06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3bc15b900 0x7ff3bc0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 --2- 172.21.15.179:0/121224325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3bc06c450 0x7ff3bc13c5a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 >> 172.21.15.179:0/121224325 conn(0x7ff3bc0cfae0 msgr2=0x7ff3bc0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:24.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 shutdown_connections 2024-05-08T22:40:24.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:24.741+0000 7ff3c255f700 1 -- 172.21.15.179:0/121224325 wait complete. 2024-05-08T22:40:24.747 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 61 2024-05-08T22:40:25.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:40:23.862086+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8154}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8154': {'gid': 8154, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.179:6835/55939533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 55939533}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 55939533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:40:25.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:40:25.161 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:40:25.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.277+0000 7f88c170d700 1 Processor -- start 2024-05-08T22:40:25.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.277+0000 7f88c170d700 1 -- start start 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88bc06bb50 0x7f88bc163f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc15ccb0 con 0x7f88bc06eba0 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc0b2e30 con 0x7f88bc15c8d0 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0cb320 con 0x7f88bc06bb50 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88bb7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f88bc06eba0 0x7f88bc06b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42340/0 (socket says 172.21.15.179:42340) 2024-05-08T22:40:25.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88bb7fe700 1 -- 172.21.15.179:0/3616886021 learned_addr learned my addr 172.21.15.179:0/3616886021 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:25.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3616886021 >> v2:172.21.15.179:3300/0 conn(0x7f88bc06bb50 0x7f88bc163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3616886021 >> v1:172.21.15.195:6789/0 conn(0x7f88bc15c8d0 legacy=0x7f88bc0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3616886021 >> v1:172.21.15.179:6789/0 conn(0x7f88bc06eba0 legacy=0x7f88bc06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3616886021 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0b30d0 con 0x7f88bc06bb50 2024-05-08T22:40:25.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3616886021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc06bb50 0x7f88bc163f30 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f88b400aea0 tx=0x7f88b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42892bc4ed0d7f25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88baffd700 1 -- 172.21.15.179:0/3616886021 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400f070 con 0x7f88bc06bb50 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88baffd700 1 -- 172.21.15.179:0/3616886021 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400c9c0 con 0x7f88bc06bb50 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88baffd700 1 -- 172.21.15.179:0/3616886021 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400cd70 con 0x7f88bc06bb50 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- 172.21.15.179:0/3616886021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc06bb50 msgr2=0x7f88bc163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 --2- 172.21.15.179:0/3616886021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc06bb50 0x7f88bc163f30 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f88b400aea0 tx=0x7f88b400c330 comp rx=0 tx=0).stop 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- 172.21.15.179:0/3616886021 shutdown_connections 2024-05-08T22:40:25.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 --2- 172.21.15.179:0/3616886021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc06bb50 0x7f88bc163f30 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- 172.21.15.179:0/3616886021 >> 172.21.15.179:0/3616886021 conn(0x7f88bc0ce960 msgr2=0x7f88bc0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- 172.21.15.179:0/3616886021 shutdown_connections 2024-05-08T22:40:25.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.281+0000 7f88c170d700 1 -- 172.21.15.179:0/3616886021 wait complete. 2024-05-08T22:40:25.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 Processor -- start 2024-05-08T22:40:25.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 -- start start 2024-05-08T22:40:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc06bb50 0x7f88bc14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc06eba0 0x7f88bc153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc14d6c0 con 0x7f88bc15c8d0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0ca410 con 0x7f88bc06bb50 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c170d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc14d960 con 0x7f88bc06eba0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c0f0c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c0f0c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51144/0 (socket says 172.21.15.179:51144) 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88c0f0c700 1 -- 172.21.15.179:0/2042642337 learned_addr learned my addr 172.21.15.179:0/2042642337 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88bbfff700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc06bb50 0x7f88bc14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.285+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc06eba0 0x7f88bc153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 -- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc06eba0 msgr2=0x7f88bc153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc06eba0 0x7f88bc153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 -- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc06bb50 msgr2=0x7f88bc14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc06bb50 0x7f88bc14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 -- 172.21.15.179:0/2042642337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b400ab80 con 0x7f88bc15c8d0 2024-05-08T22:40:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0cf5e0 tx=0x7f88b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88b8ff9700 1 -- 172.21.15.179:0/2042642337 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004800 con 0x7f88bc15c8d0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88b8ff9700 1 -- 172.21.15.179:0/2042642337 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400a330 con 0x7f88bc15c8d0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc14dc40 con 0x7f88bc15c8d0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88b8ff9700 1 -- 172.21.15.179:0/2042642337 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4008660 con 0x7f88bc15c8d0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0ca6f0 con 0x7f88bc15c8d0 2024-05-08T22:40:25.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.289+0000 7f88b8ff9700 1 -- 172.21.15.179:0/2042642337 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88b40087c0 con 0x7f88bc15c8d0 2024-05-08T22:40:25.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88b8ff9700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 0x7f889c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88b8ff9700 1 -- 172.21.15.179:0/2042642337 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f88b4085db0 con 0x7f88bc15c8d0 2024-05-08T22:40:25.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88bbfff700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 0x7f889c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88bbfff700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 0x7f889c068bf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f88ac000f60 tx=0x7f88ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 Processor -- start 2024-05-08T22:40:25.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 -- start start 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24c8be0 0x2499470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x24999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x23f8d80 con 0x24d1b00 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23f6510 con 0x24c8be0 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.293+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x23f1e00 con 0x24d2420 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51154/0 (socket says 172.21.15.179:51154) 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 -- 172.21.15.179:0/3447631726 learned_addr learned my addr 172.21.15.179:0/3447631726 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x24999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24c8be0 0x2499470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 -- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 msgr2=0x24999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x24999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 -- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24c8be0 msgr2=0x2499470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24c8be0 0x2499470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 -- 172.21.15.179:0/3447631726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2264960 con 0x24d1b00 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f6f0 tx=0x7f88ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa7dece3b8a8099 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88b1ffb700 1 -- 172.21.15.179:0/3447631726 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00fe70 con 0x24d1b00 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88b1ffb700 1 -- 172.21.15.179:0/3447631726 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac014450 con 0x24d1b00 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88b1ffb700 1 -- 172.21.15.179:0/3447631726 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0147c0 con 0x24d1b00 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3447631726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 msgr2=0x24d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f6f0 tx=0x7f88ac012000 comp rx=0 tx=0).stop 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3447631726 shutdown_connections 2024-05-08T22:40:25.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24c8be0 0x2499470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x24999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3447631726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24d1ee0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3447631726 >> 172.21.15.179:0/3447631726 conn(0x22194d0 msgr2=0x24b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3447631726 shutdown_connections 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3447631726 wait complete. 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 -- start start 2024-05-08T22:40:25.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.297+0000 7f88c5c1e740 1 Processor -- start 2024-05-08T22:40:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x248b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x248bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x23af730 con 0x24c8be0 2024-05-08T22:40:25.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23b55b0 con 0x24d1b00 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x229daa0 con 0x24d2420 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 10 client.0 _ll_register_callbacks cb 0x23eb610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51166/0 (socket says 172.21.15.179:51166) 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 -- 172.21.15.179:0/249728069 learned_addr learned my addr 172.21.15.179:0/249728069 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x248bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x248b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 -- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 msgr2=0x248bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x248bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 -- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 msgr2=0x248b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x248b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 -- 172.21.15.179:0/249728069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0b2e30 con 0x24c8be0 2024-05-08T22:40:25.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bbfff700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00fab0 tx=0x7f88ac001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad135936ee41ad9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02b860 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f88ac01c040 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22261e0 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88ac014e30 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 10 client.8220 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x23b8220 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22261e0 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f88ac02b9c0 con 0x24c8be0 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 10 client.8220 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 10 client.8220 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 20 client.8220 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:25.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24dad30 0x24e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 4 client.8220 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2270460 con 0x24dad30 2024-05-08T22:40:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.301+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24dad30 0x24e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f889affd700 10 client.8220 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:25.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24dad30 0x24e53f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc5644fb164515e server_cookie=3c978b020ff137ad in_seq=0 out_seq=0 2024-05-08T22:40:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f889affd700 1 -- 172.21.15.179:0/249728069 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2270460 con 0x24dad30 2024-05-08T22:40:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f889affd700 10 client.8220 handle_command_reply: tid=0 2024-05-08T22:40:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 client.8220 shutdown 2024-05-08T22:40:25.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 msgr2=0x24d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00fab0 tx=0x7f88ac001080 comp rx=0 tx=0).stop 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 shutdown_connections 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24dad30 0x24e53f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x248b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24d2420 0x248bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/249728069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24c8be0 0x24d7cb0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 >> 172.21.15.179:0/249728069 conn(0x22194d0 msgr2=0x24b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f889affd700 0 client.8220 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 shutdown_connections 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 1 -- 172.21.15.179:0/249728069 wait complete. 2024-05-08T22:40:25.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.305+0000 7f88c5c1e740 20 client.8220 trim_cache size 0 max 16384 2024-05-08T22:40:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.325+0000 7f88c5c1e740 1 Processor -- start 2024-05-08T22:40:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.325+0000 7f88c5c1e740 1 -- start start 2024-05-08T22:40:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x24e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x24e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cbaa0 con 0x24d2420 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23bc4f0 con 0x24d1b00 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2420ec0 con 0x24c8be0 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51180/0 (socket says 172.21.15.179:51180) 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3230366357 learned_addr learned my addr 172.21.15.179:0/3230366357 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x24e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x24e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 msgr2=0x24e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x24e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 msgr2=0x24e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x24e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 -- 172.21.15.179:0/3230366357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x239e450 con 0x24d2420 2024-05-08T22:40:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x24e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f88b4008410 tx=0x7f88b4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1125c2e2c7a4d1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88b27fc700 1 -- 172.21.15.179:0/3230366357 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40572c0 con 0x24d2420 2024-05-08T22:40:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88b27fc700 1 -- 172.21.15.179:0/3230366357 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4056bc0 con 0x24d2420 2024-05-08T22:40:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88b27fc700 1 -- 172.21.15.179:0/3230366357 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b405a9f0 con 0x24d2420 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3230366357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 msgr2=0x2489730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f88b4008410 tx=0x7f88b4012000 comp rx=0 tx=0).stop 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3230366357 shutdown_connections 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d1b00 0x24e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d2420 0x2489730 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3230366357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x24e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3230366357 >> 172.21.15.179:0/3230366357 conn(0x22194d0 msgr2=0x24b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3230366357 shutdown_connections 2024-05-08T22:40:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.329+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3230366357 wait complete. 2024-05-08T22:40:25.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 -- start start 2024-05-08T22:40:25.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 Processor -- start 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x2534800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 0x24f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2390470 con 0x24d1b00 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d7ef0 con 0x24d2420 2024-05-08T22:40:25.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d7c90 con 0x24c8be0 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51184/0 (socket says 172.21.15.179:51184) 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c5c1e740 10 client.0 _ll_register_callbacks cb 0x7f88ac017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 -- 172.21.15.179:0/3173487103 learned_addr learned my addr 172.21.15.179:0/3173487103 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x2534800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 0x24f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 -- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 msgr2=0x2534800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x2534800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 -- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 msgr2=0x24f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 0x24f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 -- 172.21.15.179:0/3173487103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0ca6f0 con 0x24d1b00 2024-05-08T22:40:25.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88c0f0c700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 0x24f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x2534800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f88bb7fe700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f88bc06e310 tx=0x7f88a4007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987acd5f6a99a40e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.333+0000 7f889affd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004aa0 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f88a4004c00 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88ac014450 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a4004dc0 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 10 client.8226 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2392720 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac014450 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f88a4005030 con 0x24d1b00 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 10 client.8226 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:25.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 10 client.8226 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:25.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 20 client.8226 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:25.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24f0bb0 0x2487710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 4 client.8226 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:40:25.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "status"}) v1 -- 0x22951e0 con 0x24f0bb0 2024-05-08T22:40:25.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24f0bb0 0x2487710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 10 client.8226 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:25.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88bbfff700 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24f0bb0 0x2487710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14e254a1daf5ea2 server_cookie=4b244ab34ab8846a in_seq=0 out_seq=0 2024-05-08T22:40:25.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 1 -- 172.21.15.179:0/3173487103 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x22951e0 con 0x24f0bb0 2024-05-08T22:40:25.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 10 client.8226 handle_command_reply: tid=0 2024-05-08T22:40:25.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 client.8226 shutdown 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 msgr2=0x24efdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f88bc06e310 tx=0x7f88a4007490 comp rx=0 tx=0).stop 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 shutdown_connections 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24f0bb0 0x2487710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x24d2420 0x24f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f889affd700 0 client.8226 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x24d1b00 0x24efdd0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 --2- 172.21.15.179:0/3173487103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x24c8be0 0x2534800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 >> 172.21.15.179:0/3173487103 conn(0x22194d0 msgr2=0x24e9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 shutdown_connections 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 1 -- 172.21.15.179:0/3173487103 wait complete. 2024-05-08T22:40:25.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c5c1e740 20 client.8226 trim_cache size 0 max 16384 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 8154, 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.4899698580000003, 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 61, 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 99, 2024-05-08T22:40:25.345 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 99, 2024-05-08T22:40:25.346 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 5.4619691589999997 2024-05-08T22:40:25.346 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:25.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 msgr2=0x7f889c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 0x7f889c068bf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f88ac000f60 tx=0x7f88ac009040 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 msgr2=0x7f88bc1f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0cf5e0 tx=0x7f88b4004af0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 shutdown_connections 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f889c066730 0x7f889c068bf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88bc15c8d0 0x7f88bc1f3040 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88bc06eba0 0x7f88bc153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 --2- 172.21.15.179:0/2042642337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88bc06bb50 0x7f88bc14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.337+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 >> 172.21.15.179:0/2042642337 conn(0x7f88bc0ce960 msgr2=0x7f88bc14fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.341+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 shutdown_connections 2024-05-08T22:40:25.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.341+0000 7f88c170d700 1 -- 172.21.15.179:0/2042642337 wait complete. 2024-05-08T22:40:25.356 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:40:25.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 Processor -- start 2024-05-08T22:40:25.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 -- start start 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffba81434d0 0x7ffba806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80cd200 con 0x7ffba806ed50 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80cd4e0 con 0x7ffba806dc20 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbaf00a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80cd7c0 con 0x7ffba81434d0 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae008700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffba81434d0 0x7ffba806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae809700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffba806ed50 0x7ffba80b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42354/0 (socket says 172.21.15.179:42354) 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae809700 1 -- 172.21.15.179:0/1845520764 learned_addr learned my addr 172.21.15.179:0/1845520764 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbad006700 1 -- 172.21.15.179:0/1845520764 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27574841 0 0) 0x7ffba80cd200 con 0x7ffba806ed50 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbad006700 1 -- 172.21.15.179:0/1845520764 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb940035d0 con 0x7ffba806ed50 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae008700 1 -- 172.21.15.179:0/1845520764 >> v1:172.21.15.195:6789/0 conn(0x7ffba806dc20 legacy=0x7ffba806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae008700 1 -- 172.21.15.179:0/1845520764 >> v1:172.21.15.179:6789/0 conn(0x7ffba806ed50 legacy=0x7ffba80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae008700 1 -- 172.21.15.179:0/1845520764 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80cdaa0 con 0x7ffba81434d0 2024-05-08T22:40:25.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbae008700 1 --2- 172.21.15.179:0/1845520764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba81434d0 0x7ffba806d6e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009850 tx=0x7ffba4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22ade52502aed3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbad006700 1 -- 172.21.15.179:0/1845520764 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011830 con 0x7ffba81434d0 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbad006700 1 -- 172.21.15.179:0/1845520764 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4011990 con 0x7ffba81434d0 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.469+0000 7ffbad006700 1 -- 172.21.15.179:0/1845520764 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011d40 con 0x7ffba81434d0 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- 172.21.15.179:0/1845520764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba81434d0 msgr2=0x7ffba806d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/1845520764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba81434d0 0x7ffba806d6e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009850 tx=0x7ffba4009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- 172.21.15.179:0/1845520764 shutdown_connections 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/1845520764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba81434d0 0x7ffba806d6e0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- 172.21.15.179:0/1845520764 >> 172.21.15.179:0/1845520764 conn(0x7ffba80ce9d0 msgr2=0x7ffba80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- 172.21.15.179:0/1845520764 shutdown_connections 2024-05-08T22:40:25.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- 172.21.15.179:0/1845520764 wait complete. 2024-05-08T22:40:25.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 Processor -- start 2024-05-08T22:40:25.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- start start 2024-05-08T22:40:25.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffba81434d0 0x7ffba806bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 0x7ffba80b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba806c3e0 con 0x7ffba806ed50 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80ca420 con 0x7ffba81564d0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbaf00a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80ca700 con 0x7ffba81434d0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbae008700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbae008700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51196/0 (socket says 172.21.15.179:51196) 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.473+0000 7ffbae008700 1 -- 172.21.15.179:0/2599757932 learned_addr learned my addr 172.21.15.179:0/2599757932 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbad807700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffba81434d0 0x7ffba806bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae809700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 0x7ffba80b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 -- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffba81434d0 msgr2=0x7ffba806bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffba81434d0 0x7ffba806bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 -- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 msgr2=0x7ffba80b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 0x7ffba80b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 -- 172.21.15.179:0/2599757932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40094b0 con 0x7ffba806ed50 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae809700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 0x7ffba80b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbae008700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffba4014040 tx=0x7ffba4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4019a00 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba401e070 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba80ca9e0 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4019da0 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80cafa0 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba8067740 con 0x7ffba806ed50 2024-05-08T22:40:25.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.477+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffba402d450 con 0x7ffba806ed50 2024-05-08T22:40:25.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.481+0000 7ffb9effd700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 0x7ffb8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:25.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.481+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7ffba4085320 con 0x7ffba806ed50 2024-05-08T22:40:25.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.481+0000 7ffbad807700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 0x7ffb8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:25.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.481+0000 7ffbad807700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 0x7ffb8c068bf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffba80c8e20 tx=0x7ffb98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:25.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.485+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffba4052510 con 0x7ffba806ed50 2024-05-08T22:40:25.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.689+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffba804e700 con 0x7ffba806ed50 2024-05-08T22:40:25.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.689+0000 7ffb9effd700 1 -- 172.21.15.179:0/2599757932 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffba4055b30 con 0x7ffba806ed50 2024-05-08T22:40:25.693 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:40:25.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 msgr2=0x7ffb8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 0x7ffb8c068bf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffba80c8e20 tx=0x7ffb98009040 comp rx=0 tx=0).stop 2024-05-08T22:40:25.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 msgr2=0x7ffba80c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:25.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffba4014040 tx=0x7ffba4004770 comp rx=0 tx=0).stop 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 shutdown_connections 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb8c066730 0x7ffb8c068bf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffba81564d0 0x7ffba80b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffba81434d0 0x7ffba806bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 --2- 172.21.15.179:0/2599757932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba806ed50 0x7ffba80c88e0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 >> 172.21.15.179:0/2599757932 conn(0x7ffba80ce9d0 msgr2=0x7ffba815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 shutdown_connections 2024-05-08T22:40:25.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:25.693+0000 7ffbaf00a700 1 -- 172.21.15.179:0/2599757932 wait complete. 2024-05-08T22:40:25.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:40:25.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:40:25.708 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:25.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.708 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:40:25.716 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:40:25.716 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:40:25.716 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:40:25.716 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:40:25.717 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.718 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:40:25.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.719 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:40:25.733 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:40:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.734 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:40:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:40:25.743 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:40:25.743 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:40:25.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:40:25.743 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:40:25.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.829 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:40:25.837 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:40:25.837 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:40:25.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:25.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.837 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:40:25.852 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:40:25.849+0000 7f7f2b8ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:40:25.854 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:40:25.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:40:25.854 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:25.904 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:40:25.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:40:25.905 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:25.911 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:25.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:25.926 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:25.985 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:25.986 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:25.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:25.987 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:25.994 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:40:25.994 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:40:25.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:26.012 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:26.012 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:26.069 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:26.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:26.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:26.070 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:26.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:40:26.085+0000 7f9ba6a76640 -1 init, newargv = 0x55ea8f2f2a80 newargc=15 2024-05-08T22:40:26.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[46654]: starting ceph client 2024-05-08T22:40:26.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[46654]: starting fuse 2024-05-08T22:40:27.176 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:40:27.194 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:40:27.195 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:40:27.254 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:40:27.256 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:40:27.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:27.257 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:40:27.264 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:40:27.264 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:27.264 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:40:27.265 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:40:27.266 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:40:27.266 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:40:27.266 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:40:27.266 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:40:27.352 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.46654.asok 2024-05-08T22:40:27.536 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46654.asok 2024-05-08T22:40:27.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:27.537 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46654.asok status 2024-05-08T22:40:27.682 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:27.682 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:40:27.682 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:40:27.682 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:40:27.682 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "46654", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 8241, 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 8241 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:40:27.683 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 641111157 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 641111157 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.8241 192.168.0.1:0/641111157", 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/641111157", 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 61, 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 99, 2024-05-08T22:40:27.684 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:40:27.685 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:40:27.685 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:40:27.685 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:27.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:27.692 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:40:27.692 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:27.697 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:40:27.697 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:27.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:27.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:40:27.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:40:27.708 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-08T22:40:27.721 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:40:27.717+0000 7f7f280c3700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-08T22:40:27.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:40:27.726 INFO:teuthology.orchestra.run.smithi179.stderr:stat: cannot stat 'testdir/hardlink': Input/output error 2024-05-08T22:40:27.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-08T22:40:27.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 Processor -- start 2024-05-08T22:40:27.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 -- start start 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcd000ca140 0x7fcd0006e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0015b250 con 0x7fcd0015e1d0 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0015a640 con 0x7fcd0006e670 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fcd0506f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0015a920 con 0x7fcd000ca140 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcd0015e1d0 0x7fcd0006c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42356/0 (socket says 172.21.15.179:42356) 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfffff700 1 -- 172.21.15.179:0/462412829 learned_addr learned my addr 172.21.15.179:0/462412829 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> v2:172.21.15.195:3300/0 conn(0x7fcd000ca140 0x7fcd0006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399513426 0 0) 0x7fcd0015b250 con 0x7fcd0015e1d0 2024-05-08T22:40:27.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce80035d0 con 0x7fcd0015e1d0 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603853813 0 0) 0x7fcd0015a640 con 0x7fcd0006e670 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0015b250 con 0x7fcd0006e670 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 896392341 0 0) 0x7fcce80035d0 con 0x7fcd0015e1d0 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 >> v2:172.21.15.195:3300/0 conn(0x7fcd000ca140 msgr2=0x7fcd0006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 --2- 172.21.15.179:0/462412829 >> v2:172.21.15.195:3300/0 conn(0x7fcd000ca140 0x7fcd0006e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 >> v1:172.21.15.195:6789/0 conn(0x7fcd0006e670 legacy=0x7fcd000cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0015ac00 con 0x7fcd0015e1d0 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fccf0001670 con 0x7fcd0015e1d0 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 >> v1:172.21.15.179:6789/0 conn(0x7fcd0015e1d0 legacy=0x7fcd0006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.849+0000 7fccfeffd700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8003bb0 0x7fcce8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 0x7fcce8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce8005230 0x7fcce800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0015a920 con 0x7fcce8004870 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce800c160 con 0x7fcce8005230 2024-05-08T22:40:27.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce800c320 con 0x7fcce8003bb0 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> v2:172.21.15.195:3300/0 conn(0x7fcd000ca140 0x7fcd0006e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfffff700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8003bb0 0x7fcce8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 0x7fcce8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0486e700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce8005230 0x7fcce800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 -- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8003bb0 msgr2=0x7fcce8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8003bb0 0x7fcce8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 -- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce8005230 msgr2=0x7fcce800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce8005230 0x7fcce800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 -- 172.21.15.179:0/462412829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80035d0 con 0x7fcce8004870 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0486e700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcce8005230 0x7fcce800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0486e700 1 -- 172.21.15.179:0/462412829 reap_dead start 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccff7fe700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 0x7fcce8004cf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009d20 tx=0x7fcce80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2756ae2ebcd719 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4009e70 con 0x7fcce8004870 2024-05-08T22:40:27.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf400e9a0 con 0x7fcce8004870 2024-05-08T22:40:27.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fccfeffd700 1 -- 172.21.15.179:0/462412829 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400ed10 con 0x7fcce8004870 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 -- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 msgr2=0x7fcce8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 0x7fcce8004cf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009d20 tx=0x7fcce80058a0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 -- 172.21.15.179:0/462412829 shutdown_connections 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 --2- 172.21.15.179:0/462412829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcce8004870 0x7fcce8004cf0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 -- 172.21.15.179:0/462412829 >> 172.21.15.179:0/462412829 conn(0x7fcd000b17c0 msgr2=0x7fcd000b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 -- 172.21.15.179:0/462412829 shutdown_connections 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.853+0000 7fcd0506f700 1 -- 172.21.15.179:0/462412829 wait complete. 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 Processor -- start 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- start start 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8004870 0x7fcd00148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd00148590 0x7fcd0014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001501b0 con 0x7fcd000b0780 2024-05-08T22:40:27.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000b1140 con 0x7fcd00148590 2024-05-08T22:40:27.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0014e0d0 con 0x7fcce8004870 2024-05-08T22:40:27.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51208/0 (socket says 172.21.15.179:51208) 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 -- 172.21.15.179:0/809270279 learned_addr learned my addr 172.21.15.179:0/809270279 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fccfffff700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8004870 0x7fcd00148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fccff7fe700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd00148590 0x7fcd0014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 -- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8004870 msgr2=0x7fcd00148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8004870 0x7fcd00148010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 -- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd00148590 msgr2=0x7fcd0014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd00148590 0x7fcd0014f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 -- 172.21.15.179:0/809270279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0014e3b0 con 0x7fcd000b0780 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0486e700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fcd000cd1b0 tx=0x7fccf800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fccfcff9700 1 -- 172.21.15.179:0/809270279 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8003c30 con 0x7fcd000b0780 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fccfcff9700 1 -- 172.21.15.179:0/809270279 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8003d90 con 0x7fcd000b0780 2024-05-08T22:40:27.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0014e690 con 0x7fcd000b0780 2024-05-08T22:40:27.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fccfcff9700 1 -- 172.21.15.179:0/809270279 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8005310 con 0x7fcd000b0780 2024-05-08T22:40:27.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.857+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0014fea0 con 0x7fcd000b0780 2024-05-08T22:40:27.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.861+0000 7fccfcff9700 1 -- 172.21.15.179:0/809270279 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccf8005470 con 0x7fcd000b0780 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.861+0000 7fccfcff9700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 0x7fccec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.861+0000 7fccfcff9700 1 -- 172.21.15.179:0/809270279 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fccf80842b0 con 0x7fcd000b0780 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.861+0000 7fccfffff700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 0x7fccec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 Processor -- start 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fccfffff700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 0x7fccec068c20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001200 tx=0x7fccf0001e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- start start 2024-05-08T22:40:27.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x249f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x249f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x23cceb0 con 0x249fc40 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23ca640 con 0x249cef0 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x23c5f30 con 0x249ede0 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51220/0 (socket says 172.21.15.179:51220) 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fccff7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x249f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fccfffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x249f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 -- 172.21.15.179:0/742473989 learned_addr learned my addr 172.21.15.179:0/742473989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 -- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 msgr2=0x249f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x249f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 -- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 msgr2=0x249f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x249f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 -- 172.21.15.179:0/742473989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23ec950 con 0x249fc40 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd0486e700 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fccf8082600 tx=0x7fccf8054b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4c554071c61e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcce5ffb700 1 -- 172.21.15.179:0/742473989 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8055160 con 0x249fc40 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcce5ffb700 1 -- 172.21.15.179:0/742473989 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8058f90 con 0x249fc40 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcce5ffb700 1 -- 172.21.15.179:0/742473989 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf805d550 con 0x249fc40 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- 172.21.15.179:0/742473989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 msgr2=0x24a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fccf8082600 tx=0x7fccf8054b90 comp rx=0 tx=0).stop 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- 172.21.15.179:0/742473989 shutdown_connections 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x249f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24a0020 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 --2- 172.21.15.179:0/742473989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x249f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- 172.21.15.179:0/742473989 >> 172.21.15.179:0/742473989 conn(0x21ed700 msgr2=0x2487890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- 172.21.15.179:0/742473989 shutdown_connections 2024-05-08T22:40:27.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.865+0000 7fcd09580740 1 -- 172.21.15.179:0/742473989 wait complete. 2024-05-08T22:40:27.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- start start 2024-05-08T22:40:27.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 Processor -- start 2024-05-08T22:40:27.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x245d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 0x24842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2389740 con 0x249ede0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23893e0 con 0x249fc40 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2271cd0 con 0x249cef0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 10 client.0 _ll_register_callbacks cb 0x2417ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51232/0 (socket says 172.21.15.179:51232) 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccfffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x245d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 -- 172.21.15.179:0/2986160415 learned_addr learned my addr 172.21.15.179:0/2986160415 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd0486e700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 0x24842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 -- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 msgr2=0x245d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x245d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 -- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 msgr2=0x24842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 0x24842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 -- 172.21.15.179:0/2986160415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0014fea0 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd0486e700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 0x24842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccff7fe700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fcce800c130 tx=0x7fccf4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cb7c81018b6e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4021040 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21fa410 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fccf4009e70 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccf4006c80 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 10 client.8253 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2238b90 con 0x249ede0 2024-05-08T22:40:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21fa410 con 0x249ede0 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fccf4011070 con 0x249ede0 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 10 client.8253 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 10 client.8253 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 20 client.8253 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24605c0 0x24b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 4 client.8253 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:27.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.869+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2244690 con 0x24605c0 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccfffff700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24605c0 0x24b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccdeffd700 10 client.8253 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccfffff700 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24605c0 0x24b8b20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e987a635eb5b2e4 server_cookie=2ddcf57b537dda8b in_seq=0 out_seq=0 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccdeffd700 1 -- 172.21.15.179:0/2986160415 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2244690 con 0x24605c0 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccdeffd700 10 client.8253 handle_command_reply: tid=0 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 client.8253 shutdown 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 msgr2=0x245dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fcce800c130 tx=0x7fccf4004770 comp rx=0 tx=0).stop 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 shutdown_connections 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x24605c0 0x24b8b20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249fc40 0x24842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249ede0 0x245dd40 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 --2- 172.21.15.179:0/2986160415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x245d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 >> 172.21.15.179:0/2986160415 conn(0x21ed700 msgr2=0x24a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fccdeffd700 0 client.8253 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 shutdown_connections 2024-05-08T22:40:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 1 -- 172.21.15.179:0/2986160415 wait complete. 2024-05-08T22:40:27.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.873+0000 7fcd09580740 20 client.8253 trim_cache size 0 max 16384 2024-05-08T22:40:27.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 Processor -- start 2024-05-08T22:40:27.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 -- start start 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x24bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249ede0 0x24bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x245a800 con 0x249fc40 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x249fa90 con 0x249ede0 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2387030 con 0x249cef0 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51242/0 (socket says 172.21.15.179:51242) 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 -- 172.21.15.179:0/2079831900 learned_addr learned my addr 172.21.15.179:0/2079831900 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fccff7fe700 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249ede0 0x24bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fccfffff700 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x24bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 -- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 msgr2=0x24bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x24bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 -- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249ede0 msgr2=0x24bce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249ede0 0x24bce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 -- 172.21.15.179:0/2079831900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x238c350 con 0x249fc40 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcd0486e700 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fccf8051610 tx=0x7fccf801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1bb0aa8b6a1855 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcce67fc700 1 -- 172.21.15.179:0/2079831900 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8054ac0 con 0x249fc40 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcce67fc700 1 -- 172.21.15.179:0/2079831900 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8054f30 con 0x249fc40 2024-05-08T22:40:27.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.897+0000 7fcce67fc700 1 -- 172.21.15.179:0/2079831900 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf80840c0 con 0x249fc40 2024-05-08T22:40:27.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/2079831900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 msgr2=0x249d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fccf8051610 tx=0x7fccf801f000 comp rx=0 tx=0).stop 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/2079831900 shutdown_connections 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249ede0 0x24bce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x249d2d0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- 172.21.15.179:0/2079831900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249cef0 0x24bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/2079831900 >> 172.21.15.179:0/2079831900 conn(0x21ed700 msgr2=0x2487d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/2079831900 shutdown_connections 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/2079831900 wait complete. 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- start start 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 Processor -- start 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x25065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x2507630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c4c40 con 0x249fc40 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x24bd150 con 0x249cef0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x243c2c0 con 0x249ede0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 10 client.0 _ll_register_callbacks cb 0x237fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51252/0 (socket says 172.21.15.179:51252) 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 -- 172.21.15.179:0/1243754924 learned_addr learned my addr 172.21.15.179:0/1243754924 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccfffff700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x25065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccff7fe700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x2507630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 -- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 msgr2=0x2507630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x2507630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 -- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 msgr2=0x25065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x25065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 -- 172.21.15.179:0/1243754924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23ec950 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd0486e700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fccf80550a0 tx=0x7fccf8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75500d37dfe93ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8054ac0 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fccf805d550 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccf8058f90 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccf8055160 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 10 client.8259 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2372580 con 0x249fc40 2024-05-08T22:40:27.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.901+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8058f90 con 0x249fc40 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fccf807a070 con 0x249fc40 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 10 client.8259 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 10 client.8259 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 20 client.8259 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2507b70 0x24c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 4 client.8259 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x2269410 con 0x2507b70 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccfffff700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2507b70 0x24c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:27.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccdeffd700 10 client.8259 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccfffff700 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2507b70 0x24c2ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7179e32a2ec038 server_cookie=23e2c47aea182304 in_seq=0 out_seq=0 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccdeffd700 1 -- 172.21.15.179:0/1243754924 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x2269410 con 0x2507b70 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccdeffd700 10 client.8259 handle_command_reply: tid=0 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 client.8259 shutdown 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 msgr2=0x24c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fccf80550a0 tx=0x7fccf8001650 comp rx=0 tx=0).stop 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 shutdown_connections 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2507b70 0x24c2ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x249fc40 0x24c25a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x249ede0 0x2507630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 --2- 172.21.15.179:0/1243754924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x249cef0 0x25065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 >> 172.21.15.179:0/1243754924 conn(0x21ed700 msgr2=0x245d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fccdeffd700 0 client.8259 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:27.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 shutdown_connections 2024-05-08T22:40:27.912 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:40:27.905+0000 7f7f280c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0) n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5633b5711b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-08T22:40:27.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 1 -- 172.21.15.179:0/1243754924 wait complete. 2024-05-08T22:40:27.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd09580740 20 client.8259 trim_cache size 0 max 16384 2024-05-08T22:40:27.912 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:27.913 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:40:27.913 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "cc285fad-4691-4cfa-bda3-ac50372190ea", 2024-05-08T22:40:27.913 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:40:27.913 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 msgr2=0x7fccec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 0x7fccec068c20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001200 tx=0x7fccf0001e20 comp rx=0 tx=0).stop 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 msgr2=0x7fcd000b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fcd000cd1b0 tx=0x7fccf800ee50 comp rx=0 tx=0).stop 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 shutdown_connections 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccec066760 0x7fccec068c20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd00148590 0x7fcd0014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd000b0780 0x7fcd000b0c00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 --2- 172.21.15.179:0/809270279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcce8004870 0x7fcd00148010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:27.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 >> 172.21.15.179:0/809270279 conn(0x7fcd000b17c0 msgr2=0x7fcd0006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:27.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 shutdown_connections 2024-05-08T22:40:27.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:27.905+0000 7fcd0506f700 1 -- 172.21.15.179:0/809270279 wait complete. 2024-05-08T22:40:27.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:40:28.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 Processor -- start 2024-05-08T22:40:28.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 -- start start 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10700d29a0 0x7f10700b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 0x7f10700cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700ce4a0 con 0x7f107015d0e0 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107015d4c0 con 0x7f10700d29a0 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700b6f80 con 0x7f10700cdb10 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10700d29a0 0x7f10700b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10700d29a0 0x7f10700b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51254/0 (socket says 172.21.15.179:51254) 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 -- 172.21.15.179:0/1341944469 learned_addr learned my addr 172.21.15.179:0/1341944469 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106ed9d700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 0x7f10700cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 -- 172.21.15.179:0/1341944469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 msgr2=0x7f10700cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 0x7f10700cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 -- 172.21.15.179:0/1341944469 >> v1:172.21.15.179:6789/0 conn(0x7f107015d0e0 legacy=0x7f10700d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 -- 172.21.15.179:0/1341944469 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700d1900 con 0x7f10700d29a0 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106ed9d700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 0x7f10700cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106dd9b700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700d29a0 0x7f10700b89f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1064009910 tx=0x7f1064009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c6b6d56501cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106d59a700 1 -- 172.21.15.179:0/1341944469 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400ba60 con 0x7f10700d29a0 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106d59a700 1 -- 172.21.15.179:0/1341944469 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106400bbc0 con 0x7f10700d29a0 2024-05-08T22:40:28.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.037+0000 7f106d59a700 1 -- 172.21.15.179:0/1341944469 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064009e60 con 0x7f10700d29a0 2024-05-08T22:40:28.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- 172.21.15.179:0/1341944469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700d29a0 msgr2=0x7f10700b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700d29a0 0x7f10700b89f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1064009910 tx=0x7f1064009c20 comp rx=0 tx=0).stop 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- 172.21.15.179:0/1341944469 shutdown_connections 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10700cdb10 0x7f10700cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- 172.21.15.179:0/1341944469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700d29a0 0x7f10700b89f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- 172.21.15.179:0/1341944469 >> 172.21.15.179:0/1341944469 conn(0x7f10700d3260 msgr2=0x7f10700d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- 172.21.15.179:0/1341944469 shutdown_connections 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- 172.21.15.179:0/1341944469 wait complete. 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 Processor -- start 2024-05-08T22:40:28.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- start start 2024-05-08T22:40:28.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 0x7f10700c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 0x7f10701f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700c87c0 con 0x7f10700cdb10 2024-05-08T22:40:28.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700c8aa0 con 0x7f10700d29a0 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700c8d80 con 0x7f107015d0e0 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51266/0 (socket says 172.21.15.179:51266) 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106e59c700 1 -- 172.21.15.179:0/763166590 learned_addr learned my addr 172.21.15.179:0/763166590 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106dd9b700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 0x7f10700c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106ed9d700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 0x7f10701f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.041+0000 7f106e59c700 1 -- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 msgr2=0x7f10701f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106e59c700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 0x7f10701f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106e59c700 1 -- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 msgr2=0x7f10700c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106e59c700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 0x7f10700c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106e59c700 1 -- 172.21.15.179:0/763166590 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009570 con 0x7f10700cdb10 2024-05-08T22:40:28.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106dd9b700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 0x7f10700c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106ed9d700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 0x7f10701f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106e59c700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f106000c500 tx=0x7f106000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 -- 172.21.15.179:0/763166590 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600052d0 con 0x7f10700cdb10 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 -- 172.21.15.179:0/763166590 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060005430 con 0x7f10700cdb10 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700c9fb0 con 0x7f10700cdb10 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 -- 172.21.15.179:0/763166590 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600055a0 con 0x7f10700cdb10 2024-05-08T22:40:28.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700c9060 con 0x7f10700cdb10 2024-05-08T22:40:28.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 -- 172.21.15.179:0/763166590 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10600038e0 con 0x7f10700cdb10 2024-05-08T22:40:28.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 0x7f1050068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.045+0000 7f105effd700 1 -- 172.21.15.179:0/763166590 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f10600835b0 con 0x7f10700cdb10 2024-05-08T22:40:28.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106dd9b700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 0x7f1050068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 Processor -- start 2024-05-08T22:40:28.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- start start 2024-05-08T22:40:28.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106dd9b700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 0x7f1050068c20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f10700d2da0 tx=0x7f1064008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x2012c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203f5d0 0x2013810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa5010 con 0x2012840 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f32f40 con 0x203c6f0 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f85e30 con 0x203f5d0 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51278/0 (socket says 172.21.15.179:51278) 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 -- 172.21.15.179:0/4072278656 learned_addr learned my addr 172.21.15.179:0/4072278656 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106ed9d700 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203f5d0 0x2013810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106dd9b700 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x2012c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 -- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203f5d0 msgr2=0x2013810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203f5d0 0x2013810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 -- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 msgr2=0x2012c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x2012c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 -- 172.21.15.179:0/4072278656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1df1a40 con 0x2012840 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f106e59c700 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f106000a040 tx=0x7f1060053e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1231760091d943a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f104ffff700 1 -- 172.21.15.179:0/4072278656 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060058290 con 0x2012840 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f104ffff700 1 -- 172.21.15.179:0/4072278656 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106005c850 con 0x2012840 2024-05-08T22:40:28.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f104ffff700 1 -- 172.21.15.179:0/4072278656 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060053950 con 0x2012840 2024-05-08T22:40:28.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- 172.21.15.179:0/4072278656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 msgr2=0x2017d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f106000a040 tx=0x7f1060053e40 comp rx=0 tx=0).stop 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- 172.21.15.179:0/4072278656 shutdown_connections 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x2012c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203f5d0 0x2013810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 --2- 172.21.15.179:0/4072278656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2012840 0x2017d90 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- 172.21.15.179:0/4072278656 >> 172.21.15.179:0/4072278656 conn(0x1da65b0 msgr2=0x203eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- 172.21.15.179:0/4072278656 shutdown_connections 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.049+0000 7f1077dfe740 1 -- 172.21.15.179:0/4072278656 wait complete. 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 -- start start 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 Processor -- start 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 0x2067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203c6f0 0x2041010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f81150 con 0x203f5d0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f7eeb0 con 0x2012840 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f3c8e0 con 0x203c6f0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f1077dfe740 10 client.0 _ll_register_callbacks cb 0x1f1b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106e59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 0x2067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51284/0 (socket says 172.21.15.179:51284) 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 -- 172.21.15.179:0/3582047650 learned_addr learned my addr 172.21.15.179:0/3582047650 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106dd9b700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203c6f0 0x2041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 -- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203c6f0 msgr2=0x2041010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203c6f0 0x2041010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 -- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 msgr2=0x2067420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 0x2067420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106ed9d700 1 -- 172.21.15.179:0/3582047650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700c9060 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.053+0000 7f106e59c700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 0x2067420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f106ed9d700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f106800df00 tx=0x7f106800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa18e8ad6a46c652 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068013040 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f106800cd20 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1db32c0 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f106800cee0 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 10 client.8274 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1f45290 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1db32c0 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1068018440 con 0x203f5d0 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 10 client.8274 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 10 client.8274 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 20 client.8274 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:28.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015980 0x2015d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 4 client.8274 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1dfd540 con 0x2015980 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f106e59c700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015980 0x2015d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 10 client.8274 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f106e59c700 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015980 0x2015d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f5bc4d6e64c01d server_cookie=a6a198b1dbdbafeb in_seq=0 out_seq=0 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 1 -- 172.21.15.179:0/3582047650 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1dfd540 con 0x2015980 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 10 client.8274 handle_command_reply: tid=0 2024-05-08T22:40:28.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 client.8274 shutdown 2024-05-08T22:40:28.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 msgr2=0x2042ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f106800df00 tx=0x7f106800df30 comp rx=0 tx=0).stop 2024-05-08T22:40:28.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 shutdown_connections 2024-05-08T22:40:28.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015980 0x2015d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x2042ea0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x203c6f0 0x2041010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 --2- 172.21.15.179:0/3582047650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2012840 0x2067420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 >> 172.21.15.179:0/3582047650 conn(0x1da65b0 msgr2=0x203ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f104d7fa700 0 client.8274 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 shutdown_connections 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 1 -- 172.21.15.179:0/3582047650 wait complete. 2024-05-08T22:40:28.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.057+0000 7f1077dfe740 20 client.8274 trim_cache size 0 max 16384 2024-05-08T22:40:28.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 Processor -- start 2024-05-08T22:40:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 -- start start 2024-05-08T22:40:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x2017d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x206db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e2ab80 con 0x203f5d0 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x203f160 con 0x203c6f0 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f1d860 con 0x2012840 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106dd9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x206db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51292/0 (socket says 172.21.15.179:51292) 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 -- 172.21.15.179:0/2925155738 learned_addr learned my addr 172.21.15.179:0/2925155738 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106e59c700 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x2017d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 -- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 msgr2=0x2017d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x2017d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 -- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 msgr2=0x206db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x206db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 -- 172.21.15.179:0/2925155738 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f2b4c0 con 0x203f5d0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106dd9b700 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x206db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f106ed9d700 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1068007730 tx=0x7f1068011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6025b7c997f4e79d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f105cff9700 1 -- 172.21.15.179:0/2925155738 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068013020 con 0x203f5d0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f105cff9700 1 -- 172.21.15.179:0/2925155738 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068007bc0 con 0x203f5d0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.081+0000 7f105cff9700 1 -- 172.21.15.179:0/2925155738 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068018dd0 con 0x203f5d0 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- 172.21.15.179:0/2925155738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 msgr2=0x206e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1068007730 tx=0x7f1068011000 comp rx=0 tx=0).stop 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- 172.21.15.179:0/2925155738 shutdown_connections 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203c6f0 0x206db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203f5d0 0x206e040 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 --2- 172.21.15.179:0/2925155738 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x2017d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- 172.21.15.179:0/2925155738 >> 172.21.15.179:0/2925155738 conn(0x1da65b0 msgr2=0x2063670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- 172.21.15.179:0/2925155738 shutdown_connections 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- 172.21.15.179:0/2925155738 wait complete. 2024-05-08T22:40:28.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 -- start start 2024-05-08T22:40:28.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.085+0000 7f1077dfe740 1 Processor -- start 2024-05-08T22:40:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x20c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203f5d0 0x207b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x20174e0 con 0x203c6f0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20c2610 con 0x203f5d0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x207c090 con 0x2012840 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 10 client.0 _ll_register_callbacks cb 0x1f38d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51306/0 (socket says 172.21.15.179:51306) 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 -- 172.21.15.179:0/1176950676 learned_addr learned my addr 172.21.15.179:0/1176950676 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106e59c700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x20c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106ed9d700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203f5d0 0x207b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 -- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 msgr2=0x20c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x20c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 -- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203f5d0 msgr2=0x207b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203f5d0 0x207b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 -- 172.21.15.179:0/1176950676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700d1900 con 0x203c6f0 2024-05-08T22:40:28.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106dd9b700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f10640044a0 tx=0x7f10640047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b4af0c1dce74c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106402c040 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f106401c640 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f106005c850 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1064003ef0 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 10 client.8280 fetch_fsmap learned FSMap version 61 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1f45290 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106005c850 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f106402f070 con 0x203c6f0 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 10 client.8280 fetch_fsmap finished waiting for FSMap version 61 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 10 client.8280 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 20 client.8280 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015610 0x2015a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 4 client.8280 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:40:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1e222c0 con 0x2015610 2024-05-08T22:40:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106e59c700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015610 0x2015a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f106e59c700 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015610 0x2015a20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85db5fc0736b83fe server_cookie=493b326bf5eb28ef in_seq=0 out_seq=0 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 10 client.8280 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 1 -- 172.21.15.179:0/1176950676 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x1e222c0 con 0x2015610 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 10 client.8280 handle_command_reply: tid=0 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 client.8280 shutdown 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 msgr2=0x207b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f10640044a0 tx=0x7f10640047e0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 shutdown_connections 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2015610 0x2015a20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x203f5d0 0x207b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x203c6f0 0x207b250 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f104d7fa700 0 client.8280 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 --2- 172.21.15.179:0/1176950676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2012840 0x20c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 >> 172.21.15.179:0/1176950676 conn(0x1da65b0 msgr2=0x206db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.089+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 shutdown_connections 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f1077dfe740 1 -- 172.21.15.179:0/1176950676 wait complete. 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f1077dfe740 20 client.8280 trim_cache size 0 max 16384 2024-05-08T22:40:28.097 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "cc285fad-4691-4cfa-bda3-ac50372190ea": { 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir", 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "cc285fad-4691-4cfa-bda3-ac50372190ea", 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive" 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:28.098 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 msgr2=0x7f1050068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 0x7f1050068c20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f10700d2da0 tx=0x7f1064008030 comp rx=0 tx=0).stop 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 msgr2=0x7f10701f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f106000c500 tx=0x7f106000c8c0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 shutdown_connections 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1050066760 0x7f1050068c20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10700d29a0 0x7f10700c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f107015d0e0 0x7f10701f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 --2- 172.21.15.179:0/763166590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10700cdb10 0x7f10701f6ba0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:28.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 >> 172.21.15.179:0/763166590 conn(0x7f10700d3260 msgr2=0x7f10700b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:28.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 shutdown_connections 2024-05-08T22:40:28.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:28.093+0000 7f106f59e700 1 -- 172.21.15.179:0/763166590 wait complete. 2024-05-08T22:40:28.110 INFO:tasks.cephfs.filesystem:scrub status for tag:cc285fad-4691-4cfa-bda3-ac50372190ea - {'path': '/dir', 'tag': 'cc285fad-4691-4cfa-bda3-ac50372190ea', 'options': 'recursive'} 2024-05-08T22:40:30.889 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:40:30.885+0000 7fce8f998700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T22:40:52.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:40:52.509 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:40:58.110 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:40:58.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.232+0000 7f8bcb59e700 1 Processor -- start 2024-05-08T22:40:58.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.232+0000 7f8bcb59e700 1 -- start start 2024-05-08T22:40:58.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8bcc15c0b0 0x7f8bcc0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0cbfd0 con 0x7f8bcc1448b0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0cd970 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0cb3c0 con 0x7f8bcc15c0b0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57832/0 (socket says 172.21.15.179:57832) 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 -- 172.21.15.179:0/91953128 learned_addr learned my addr 172.21.15.179:0/91953128 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/91953128 >> v2:172.21.15.195:3300/0 conn(0x7f8bcc15c0b0 0x7f8bcc0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 -- 172.21.15.179:0/91953128 >> v2:172.21.15.195:3300/0 conn(0x7f8bcc15c0b0 msgr2=0x7f8bcc0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/91953128 >> v2:172.21.15.195:3300/0 conn(0x7f8bcc15c0b0 0x7f8bcc0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 -- 172.21.15.179:0/91953128 >> v1:172.21.15.195:6789/0 conn(0x7f8bcc1448b0 legacy=0x7f8bcc0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 -- 172.21.15.179:0/91953128 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0cb6a0 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/91953128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc072600 tx=0x7f8bc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8430836a8947cf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bc959a700 1 -- 172.21.15.179:0/91953128 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400e070 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bc959a700 1 -- 172.21.15.179:0/91953128 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc400b910 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bc959a700 1 -- 172.21.15.179:0/91953128 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400bcc0 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- 172.21.15.179:0/91953128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc0cc9a0 msgr2=0x7f8bcc06bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/91953128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc072600 tx=0x7f8bc400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- 172.21.15.179:0/91953128 shutdown_connections 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/91953128 >> v2:172.21.15.195:3300/0 conn(0x7f8bcc15c0b0 0x7f8bcc0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/91953128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc06bbc0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- 172.21.15.179:0/91953128 >> 172.21.15.179:0/91953128 conn(0x7f8bcc070d80 msgr2=0x7f8bcc071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- 172.21.15.179:0/91953128 shutdown_connections 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.236+0000 7f8bcb59e700 1 -- 172.21.15.179:0/91953128 wait complete. 2024-05-08T22:40:58.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 Processor -- start 2024-05-08T22:40:58.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 -- start start 2024-05-08T22:40:58.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc1f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bcc15c0b0 0x7f8bcc142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc141bb0 con 0x7f8bcc1448b0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc141e90 con 0x7f8bcc0cc9a0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc142170 con 0x7f8bcc15c0b0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57840/0 (socket says 172.21.15.179:57840) 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/233717586 learned_addr learned my addr 172.21.15.179:0/233717586 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bcc15c0b0 0x7f8bcc142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bca59c700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc1f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bcc15c0b0 msgr2=0x7f8bcc142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bcc15c0b0 0x7f8bcc142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 msgr2=0x7f8bcc1f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc1f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/233717586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009b30 con 0x7f8bcc1448b0 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bca59c700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc1f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:58.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.240+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e840 tx=0x7f8bbc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 -- 172.21.15.179:0/233717586 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0095a0 con 0x7f8bcc1448b0 2024-05-08T22:40:58.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 -- 172.21.15.179:0/233717586 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc013680 con 0x7f8bcc1448b0 2024-05-08T22:40:58.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc142450 con 0x7f8bcc1448b0 2024-05-08T22:40:58.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 -- 172.21.15.179:0/233717586 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0139f0 con 0x7f8bcc1448b0 2024-05-08T22:40:58.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1f05c0 con 0x7f8bcc1448b0 2024-05-08T22:40:58.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 -- 172.21.15.179:0/233717586 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bbc013b50 con 0x7f8bcc1448b0 2024-05-08T22:40:58.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 0x7f8bac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.244+0000 7f8bbaffd700 1 -- 172.21.15.179:0/233717586 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8bbc084300 con 0x7f8bcc1448b0 2024-05-08T22:40:58.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 0x7f8bac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 Processor -- start 2024-05-08T22:40:58.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- start start 2024-05-08T22:40:58.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 0x7f8bac068bf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000f60 tx=0x7f8bc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 0x292cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292d460 0x2929810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x28bb010 con 0x2928840 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2848f40 con 0x29555d0 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x289be30 con 0x292d460 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57850/0 (socket says 172.21.15.179:57850) 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 -- 172.21.15.179:0/3027927273 learned_addr learned my addr 172.21.15.179:0/3027927273 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292d460 0x2929810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 0x292cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 -- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292d460 msgr2=0x2929810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292d460 0x2929810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 -- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 msgr2=0x292cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 0x292cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 -- 172.21.15.179:0/3027927273 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2707a40 con 0x2928840 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 0x292cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bca59c700 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005180 tx=0x7f8bc0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf90a1856278d2c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8babfff700 1 -- 172.21.15.179:0/3027927273 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003b30 con 0x2928840 2024-05-08T22:40:58.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8babfff700 1 -- 172.21.15.179:0/3027927273 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0003c90 con 0x2928840 2024-05-08T22:40:58.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8babfff700 1 -- 172.21.15.179:0/3027927273 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000e650 con 0x2928840 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- 172.21.15.179:0/3027927273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 msgr2=0x2928c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005180 tx=0x7f8bc0011000 comp rx=0 tx=0).stop 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.248+0000 7f8bd41cc740 1 -- 172.21.15.179:0/3027927273 shutdown_connections 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x29555d0 0x292cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292d460 0x2929810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/3027927273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2928840 0x2928c20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- 172.21.15.179:0/3027927273 >> 172.21.15.179:0/3027927273 conn(0x26bc5b0 msgr2=0x2954ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- 172.21.15.179:0/3027927273 shutdown_connections 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- 172.21.15.179:0/3027927273 wait complete. 2024-05-08T22:40:58.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- start start 2024-05-08T22:40:58.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 Processor -- start 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x293c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x293a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2897150 con 0x29555d0 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2894eb0 con 0x2928840 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x28528e0 con 0x292c160 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 10 client.0 _ll_register_callbacks cb 0x2831670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57858/0 (socket says 172.21.15.179:57858) 2024-05-08T22:40:58.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/926590600 learned_addr learned my addr 172.21.15.179:0/926590600 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x293a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bca59c700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x293c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 msgr2=0x293a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x293a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 msgr2=0x293c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x293c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 -- 172.21.15.179:0/926590600 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc000f040 con 0x29555d0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc051580 tx=0x7f8bbc055130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec385b269ef2d111 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8ba97fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0840b0 con 0x29555d0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8bbc05b0b0 con 0x29555d0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.252+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26c92c0 con 0x29555d0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bbc0822a0 con 0x29555d0 2024-05-08T22:40:58.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 10 client.8292 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x285b290 con 0x29555d0 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26c92c0 con 0x29555d0 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8bbc054570 con 0x29555d0 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 10 client.8292 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 10 client.8292 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 20 client.8292 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x293ff00 0x2940310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 4 client.8292 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2713540 con 0x293ff00 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bca59c700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x293ff00 0x2940310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 10 client.8292 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bca59c700 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x293ff00 0x2940310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682b2c28b415304d server_cookie=dd0722900ac320d5 in_seq=0 out_seq=0 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 1 -- 172.21.15.179:0/926590600 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2713540 con 0x293ff00 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 10 client.8292 handle_command_reply: tid=0 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 client.8292 shutdown 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 msgr2=0x292bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc051580 tx=0x7f8bbc055130 comp rx=0 tx=0).stop 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 shutdown_connections 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x293ff00 0x2940310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x293a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x292bc20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/926590600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x293c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 >> 172.21.15.179:0/926590600 conn(0x26bc5b0 msgr2=0x2927640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8ba97fa700 0 client.8292 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 shutdown_connections 2024-05-08T22:40:58.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 1 -- 172.21.15.179:0/926590600 wait complete. 2024-05-08T22:40:58.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.256+0000 7f8bd41cc740 20 client.8292 trim_cache size 0 max 16384 2024-05-08T22:40:58.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 Processor -- start 2024-05-08T22:40:58.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 -- start start 2024-05-08T22:40:58.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2928840 0x2981610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x292c160 0x2981b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2740b80 con 0x29555d0 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2833620 con 0x292c160 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x281ea50 con 0x2928840 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57866/0 (socket says 172.21.15.179:57866) 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 -- 172.21.15.179:0/643892768 learned_addr learned my addr 172.21.15.179:0/643892768 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bca59c700 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2928840 0x2981610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x292c160 0x2981b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 -- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2928840 msgr2=0x2981610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2928840 0x2981610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 -- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x292c160 msgr2=0x2981b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x292c160 0x2981b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.276+0000 7f8bcad9d700 1 -- 172.21.15.179:0/643892768 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28414c0 con 0x29555d0 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06cd30 tx=0x7f8bc4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d1133a5780d96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bb8ff9700 1 -- 172.21.15.179:0/643892768 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003a40 con 0x29555d0 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bb8ff9700 1 -- 172.21.15.179:0/643892768 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4023040 con 0x29555d0 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bb8ff9700 1 -- 172.21.15.179:0/643892768 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003de0 con 0x29555d0 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- 172.21.15.179:0/643892768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 msgr2=0x2977110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06cd30 tx=0x7f8bc4010000 comp rx=0 tx=0).stop 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- 172.21.15.179:0/643892768 shutdown_connections 2024-05-08T22:40:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x292c160 0x2981b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x2977110 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/643892768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2928840 0x2981610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- 172.21.15.179:0/643892768 >> 172.21.15.179:0/643892768 conn(0x26bc5b0 msgr2=0x293e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- 172.21.15.179:0/643892768 shutdown_connections 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- 172.21.15.179:0/643892768 wait complete. 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- start start 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 Processor -- start 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x2991060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x298f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x28926c0 con 0x29555d0 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2977c80 con 0x2928840 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x29915a0 con 0x292c160 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bd41cc740 10 client.0 _ll_register_callbacks cb 0x283c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57874/0 (socket says 172.21.15.179:57874) 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 -- 172.21.15.179:0/1386555404 learned_addr learned my addr 172.21.15.179:0/1386555404 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x2991060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bc9d9b700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x298f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 -- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 msgr2=0x298f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x298f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 -- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 msgr2=0x2991060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x2991060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 -- 172.21.15.179:0/1386555404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc400be70 con 0x29555d0 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8bcad9d700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400d010 tx=0x7f8bc4004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f77d49eadcc27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8ba97fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.280+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4004450 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bc0003c90 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8bc4003e30 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc40045b0 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 10 client.8298 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x2835790 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0003c90 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8bc402e030 con 0x29555d0 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 10 client.8298 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 10 client.8298 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 20 client.8298 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2990590 0x298dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 4 client.8298 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:40:58.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x27382c0 con 0x2990590 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bca59c700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2990590 0x298dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bca59c700 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2990590 0x298dc90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b928254e90b4bf4e server_cookie=25cdf0e8cfcf46c8 in_seq=0 out_seq=0 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 10 client.8298 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 1 -- 172.21.15.179:0/1386555404 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x27382c0 con 0x2990590 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 10 client.8298 handle_command_reply: tid=0 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 client.8298 shutdown 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 msgr2=0x298d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400d010 tx=0x7f8bc4004b60 comp rx=0 tx=0).stop 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 shutdown_connections 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x2990590 0x298dc90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x29555d0 0x298d750 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8ba97fa700 0 client.8298 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x292c160 0x298f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 --2- 172.21.15.179:0/1386555404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2928840 0x2991060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 >> 172.21.15.179:0/1386555404 conn(0x26bc5b0 msgr2=0x2977110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 shutdown_connections 2024-05-08T22:40:58.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 1 -- 172.21.15.179:0/1386555404 wait complete. 2024-05-08T22:40:58.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bd41cc740 20 client.8298 trim_cache size 0 max 16384 2024-05-08T22:40:58.293 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:58.293 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:40:58.294 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:40:58.294 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 msgr2=0x7f8bac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 0x7f8bac068bf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000f60 tx=0x7f8bc0009040 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 msgr2=0x7f8bcc1312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e840 tx=0x7f8bbc00eb50 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 shutdown_connections 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8bac066730 0x7f8bac068bf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bcc15c0b0 0x7f8bcc142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc1448b0 0x7f8bcc1312f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 --2- 172.21.15.179:0/233717586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bcc0cc9a0 0x7f8bcc1f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 >> 172.21.15.179:0/233717586 conn(0x7f8bcc070d80 msgr2=0x7f8bcc15d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 shutdown_connections 2024-05-08T22:40:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.284+0000 7f8bcb59e700 1 -- 172.21.15.179:0/233717586 wait complete. 2024-05-08T22:40:58.304 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:40:58.305 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:40:58.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 Processor -- start 2024-05-08T22:40:58.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 -- start start 2024-05-08T22:40:58.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb815b1e0 0x7f8eb80cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8eb80d0310 0x7f8eb80d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80d0ca0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80cc6f0 con 0x7f8eb80d0310 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80b01b0 con 0x7f8eb815b1e0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8eb80d0310 0x7f8eb80d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8eb80cbea0 0x7f8eb80b1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57252/0 (socket says 172.21.15.179:57252) 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb77fe700 1 -- 172.21.15.179:0/141188789 learned_addr learned my addr 172.21.15.179:0/141188789 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb7fff700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb815b1e0 0x7f8eb80cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb6ffd700 1 -- 172.21.15.179:0/141188789 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137759337 0 0) 0x7f8eb0001490 con 0x7f8eb80cbea0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb6ffd700 1 -- 172.21.15.179:0/141188789 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea00035d0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 -- 172.21.15.179:0/141188789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb815b1e0 msgr2=0x7f8eb80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb815b1e0 0x7f8eb80cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 -- 172.21.15.179:0/141188789 >> v1:172.21.15.179:6789/0 conn(0x7f8eb80cbea0 legacy=0x7f8eb80b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 -- 172.21.15.179:0/141188789 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80cc410 con 0x7f8eb80d0310 2024-05-08T22:40:58.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebc9ad700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80d0310 0x7f8eb80d0760 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009850 tx=0x7f8ea8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d39f16b7989305 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb6ffd700 1 -- 172.21.15.179:0/141188789 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011830 con 0x7f8eb80d0310 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb6ffd700 1 -- 172.21.15.179:0/141188789 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8011990 con 0x7f8eb80d0310 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8eb6ffd700 1 -- 172.21.15.179:0/141188789 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8011d00 con 0x7f8eb80d0310 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 -- 172.21.15.179:0/141188789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80d0310 msgr2=0x7f8eb80d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.428+0000 7f8ebe410700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80d0310 0x7f8eb80d0760 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009850 tx=0x7f8ea8009b60 comp rx=0 tx=0).stop 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 -- 172.21.15.179:0/141188789 shutdown_connections 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80d0310 0x7f8eb80d0760 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 --2- 172.21.15.179:0/141188789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb815b1e0 0x7f8eb80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 -- 172.21.15.179:0/141188789 >> 172.21.15.179:0/141188789 conn(0x7f8eb80cd8c0 msgr2=0x7f8eb80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 -- 172.21.15.179:0/141188789 shutdown_connections 2024-05-08T22:40:58.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 -- 172.21.15.179:0/141188789 wait complete. 2024-05-08T22:40:58.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 Processor -- start 2024-05-08T22:40:58.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.432+0000 7f8ebe410700 1 -- start start 2024-05-08T22:40:58.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 0x7f8eb8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 0x7f8eb80a8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81f38c0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81517b0 con 0x7f8eb815b1e0 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c7fb0 con 0x7f8eb80d0310 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57888/0 (socket says 172.21.15.179:57888) 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 -- 172.21.15.179:0/291747674 learned_addr learned my addr 172.21.15.179:0/291747674 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebc9ad700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 0x7f8eb80a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb77fe700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 0x7f8eb8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 -- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 msgr2=0x7f8eb8151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 0x7f8eb8151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 -- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 msgr2=0x7f8eb80a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 0x7f8eb80a8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 -- 172.21.15.179:0/291747674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80094b0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebc9ad700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 0x7f8eb80a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb77fe700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 0x7f8eb8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:58.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb7fff700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00e810 tx=0x7f8eac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac013610 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac013770 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb81f3ab0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac013b20 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80c8250 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.436+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb80c8470 con 0x7f8eb80cbea0 2024-05-08T22:40:58.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.440+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8eac013c80 con 0x7f8eb80cbea0 2024-05-08T22:40:58.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.440+0000 7f8eb4ff9700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 0x7f8ea4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.440+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8eac084140 con 0x7f8eb80cbea0 2024-05-08T22:40:58.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.440+0000 7f8eb77fe700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 0x7f8ea4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.440+0000 7f8eb77fe700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 0x7f8ea4068c20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00015f0 tx=0x7f8eb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.448+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8eac051180 con 0x7f8eb80cbea0 2024-05-08T22:40:58.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eb8073290 con 0x7f8eb80cbea0 2024-05-08T22:40:58.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8eb4ff9700 1 -- 172.21.15.179:0/291747674 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f8eac0547a0 con 0x7f8eb80cbea0 2024-05-08T22:40:58.659 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:58.659 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:30.890207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 msgr2=0x7f8ea4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 0x7f8ea4068c20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00015f0 tx=0x7f8eb0009040 comp rx=0 tx=0).stop 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 msgr2=0x7f8eb81f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00e810 tx=0x7f8eac00ebd0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 shutdown_connections 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ea4066760 0x7f8ea4068c20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb815b1e0 0x7f8eb80a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb80d0310 0x7f8eb8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 --2- 172.21.15.179:0/291747674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8eb80cbea0 0x7f8eb81f3380 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 >> 172.21.15.179:0/291747674 conn(0x7f8eb80cd8c0 msgr2=0x7f8eb806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 shutdown_connections 2024-05-08T22:40:58.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.656+0000 7f8ebe410700 1 -- 172.21.15.179:0/291747674 wait complete. 2024-05-08T22:40:58.663 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 62 2024-05-08T22:40:58.674 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T22:40:58.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 Processor -- start 2024-05-08T22:40:58.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 -- start start 2024-05-08T22:40:58.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85f00ce230 0x7f85f00d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f00d16c0 con 0x7f85f01605e0 2024-05-08T22:40:58.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f00d19a0 con 0x7f85f00d1190 2024-05-08T22:40:58.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f5ddd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f00d2ca0 con 0x7f85f00ce230 2024-05-08T22:40:58.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f4ddb700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f85f01605e0 0x7f85f00b87b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57268/0 (socket says 172.21.15.179:57268) 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f4ddb700 1 -- 172.21.15.179:0/885859997 learned_addr learned my addr 172.21.15.179:0/885859997 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f55dc700 1 --2- 172.21.15.179:0/885859997 >> v2:172.21.15.179:3300/0 conn(0x7f85f00ce230 0x7f85f00d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85ef7fe700 1 -- 172.21.15.179:0/885859997 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783201991 0 0) 0x7f85f00d16c0 con 0x7f85f01605e0 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85ef7fe700 1 -- 172.21.15.179:0/885859997 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85dc0035d0 con 0x7f85f01605e0 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f55dc700 1 -- 172.21.15.179:0/885859997 >> v1:172.21.15.195:6789/0 conn(0x7f85f00d1190 legacy=0x7f85f00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f55dc700 1 -- 172.21.15.179:0/885859997 >> v1:172.21.15.179:6789/0 conn(0x7f85f01605e0 legacy=0x7f85f00b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f55dc700 1 -- 172.21.15.179:0/885859997 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00d1c80 con 0x7f85f00ce230 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85f55dc700 1 --2- 172.21.15.179:0/885859997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00ce230 0x7f85f00d2760 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009d90 tx=0x7f85e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16334bd62aa307ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85ef7fe700 1 -- 172.21.15.179:0/885859997 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800e070 con 0x7f85f00ce230 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85ef7fe700 1 -- 172.21.15.179:0/885859997 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800bb20 con 0x7f85f00ce230 2024-05-08T22:40:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.788+0000 7f85ef7fe700 1 -- 172.21.15.179:0/885859997 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800be90 con 0x7f85f00ce230 2024-05-08T22:40:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- 172.21.15.179:0/885859997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00ce230 msgr2=0x7f85f00d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/885859997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00ce230 0x7f85f00d2760 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009d90 tx=0x7f85e800b330 comp rx=0 tx=0).stop 2024-05-08T22:40:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- 172.21.15.179:0/885859997 shutdown_connections 2024-05-08T22:40:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/885859997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00ce230 0x7f85f00d2760 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- 172.21.15.179:0/885859997 >> 172.21.15.179:0/885859997 conn(0x7f85f00b4940 msgr2=0x7f85f00b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- 172.21.15.179:0/885859997 shutdown_connections 2024-05-08T22:40:58.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- 172.21.15.179:0/885859997 wait complete. 2024-05-08T22:40:58.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 Processor -- start 2024-05-08T22:40:58.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- start start 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 0x7f85f01f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 0x7f85f0157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01f3110 con 0x7f85f00d1190 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01f3a70 con 0x7f85f00ce230 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f5ddd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01588c0 con 0x7f85f01605e0 2024-05-08T22:40:58.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57894/0 (socket says 172.21.15.179:57894) 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85effff700 1 -- 172.21.15.179:0/1161224676 learned_addr learned my addr 172.21.15.179:0/1161224676 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 0x7f85f01f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.792+0000 7f85f55dc700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 0x7f85f0157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 -- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 msgr2=0x7f85f0157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 0x7f85f0157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 -- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 msgr2=0x7f85f01f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 0x7f85f01f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 -- 172.21.15.179:0/1161224676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8009a70 con 0x7f85f00d1190 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 0x7f85f01f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85f55dc700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 0x7f85f0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85effff700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c530 tx=0x7f85e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 -- 172.21.15.179:0/1161224676 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00052d0 con 0x7f85f00d1190 2024-05-08T22:40:58.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 -- 172.21.15.179:0/1161224676 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0005430 con 0x7f85f00d1190 2024-05-08T22:40:58.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f0158ba0 con 0x7f85f00d1190 2024-05-08T22:40:58.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 -- 172.21.15.179:0/1161224676 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00055a0 con 0x7f85f00d1190 2024-05-08T22:40:58.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f0158590 con 0x7f85f00d1190 2024-05-08T22:40:58.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 -- 172.21.15.179:0/1161224676 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85e00038e0 con 0x7f85f00d1190 2024-05-08T22:40:58.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 0x7f85d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85ed7fa700 1 -- 172.21.15.179:0/1161224676 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f85e00837a0 con 0x7f85f00d1190 2024-05-08T22:40:58.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.796+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 0x7f85d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 Processor -- start 2024-05-08T22:40:58.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- start start 2024-05-08T22:40:58.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 0x7f85d0068bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85d80011e0 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1b45d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6d810 con 0x1b462a0 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6afa0 con 0x1a8e520 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a66890 con 0x1b46680 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85f4ddb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1b45d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57902/0 (socket says 172.21.15.179:57902) 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 -- 172.21.15.179:0/647808050 learned_addr learned my addr 172.21.15.179:0/647808050 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85f55dc700 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 -- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 msgr2=0x1b0f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b0f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 -- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 msgr2=0x1b45d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1b45d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 -- 172.21.15.179:0/647808050 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18d8ba0 con 0x1b462a0 2024-05-08T22:40:58.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85effff700 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f85e0081af0 tx=0x7f85e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560d4dbd2ea1d025 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85e67fc700 1 -- 172.21.15.179:0/647808050 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0054650 con 0x1b462a0 2024-05-08T22:40:58.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85e67fc700 1 -- 172.21.15.179:0/647808050 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0058480 con 0x1b462a0 2024-05-08T22:40:58.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85e67fc700 1 -- 172.21.15.179:0/647808050 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e005ca40 con 0x1b462a0 2024-05-08T22:40:58.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- 172.21.15.179:0/647808050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 msgr2=0x1b0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f85e0081af0 tx=0x7f85e001a000 comp rx=0 tx=0).stop 2024-05-08T22:40:58.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- 172.21.15.179:0/647808050 shutdown_connections 2024-05-08T22:40:58.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b0ef30 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/647808050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1b45d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- 172.21.15.179:0/647808050 >> 172.21.15.179:0/647808050 conn(0x188d710 msgr2=0x1b27d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- 172.21.15.179:0/647808050 shutdown_connections 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- 172.21.15.179:0/647808050 wait complete. 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.800+0000 7f85fa2ee740 1 -- start start 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 Processor -- start 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b46680 0x1b471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a29d80 con 0x1b462a0 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a29a20 con 0x1b46680 2024-05-08T22:40:58.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1959770 con 0x1a8e520 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 10 client.0 _ll_register_callbacks cb 0x1a5ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57910/0 (socket says 172.21.15.179:57910) 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85f55dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b46680 0x1b471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 -- 172.21.15.179:0/4211743542 learned_addr learned my addr 172.21.15.179:0/4211743542 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 -- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 msgr2=0x1b452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 -- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b46680 msgr2=0x1b471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b46680 0x1b471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 -- 172.21.15.179:0/4211743542 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f0158590 con 0x1b462a0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:58.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85effff700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f85e0081ba0 tx=0x7f85e0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ee554d6c8e084c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0054650 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f85e0053b40 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x189a420 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e0053d00 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 10 client.8322 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x1a2c910 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x189a420 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f85e006e030 con 0x1b462a0 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 10 client.8322 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 10 client.8322 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 20 client.8322 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b47c00 0x1affe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 4 client.8322 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:58.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18e46a0 con 0x1b47c00 2024-05-08T22:40:58.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.804+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b47c00 0x1affe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85cf7fe700 10 client.8322 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b47c00 0x1affe70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b37fb7383dc2e0d server_cookie=e0a0d8113d12e076 in_seq=0 out_seq=0 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85cf7fe700 1 -- 172.21.15.179:0/4211743542 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x18e46a0 con 0x1b47c00 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85cf7fe700 10 client.8322 handle_command_reply: tid=0 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 client.8322 shutdown 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 msgr2=0x1b46c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f85e0081ba0 tx=0x7f85e0014070 comp rx=0 tx=0).stop 2024-05-08T22:40:58.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 shutdown_connections 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b47c00 0x1affe70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b46680 0x1b471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b46c90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85cf7fe700 0 client.8322 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/4211743542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 >> 172.21.15.179:0/4211743542 conn(0x188d710 msgr2=0x1b252a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 shutdown_connections 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 1 -- 172.21.15.179:0/4211743542 wait complete. 2024-05-08T22:40:58.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.808+0000 7f85fa2ee740 20 client.8322 trim_cache size 0 max 16384 2024-05-08T22:40:58.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 Processor -- start 2024-05-08T22:40:58.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 -- start start 2024-05-08T22:40:58.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 0x1b23ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a240e0 con 0x1b46680 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a62080 con 0x1b462a0 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a6eaa0 con 0x1a8e520 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57924/0 (socket says 172.21.15.179:57924) 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 -- 172.21.15.179:0/2170929633 learned_addr learned my addr 172.21.15.179:0/2170929633 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85effff700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 0x1b23ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 -- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 msgr2=0x1b25360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b25360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 -- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 msgr2=0x1b23ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 0x1b23ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f55dc700 1 -- 172.21.15.179:0/2170929633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a12b40 con 0x1b46680 2024-05-08T22:40:58.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85effff700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 0x1b23ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.824+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85f55dc700 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b370 tx=0x7f85e8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a25c932050618c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85e6ffd700 1 -- 172.21.15.179:0/2170929633 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e802b070 con 0x1b46680 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85e6ffd700 1 -- 172.21.15.179:0/2170929633 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8004040 con 0x1b46680 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85e6ffd700 1 -- 172.21.15.179:0/2170929633 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8020040 con 0x1b46680 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2170929633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 msgr2=0x1b24530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b370 tx=0x7f85e8011000 comp rx=0 tx=0).stop 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2170929633 shutdown_connections 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b462a0 0x1b23ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b46680 0x1b24530 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2170929633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1a8e520 0x1b25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2170929633 >> 172.21.15.179:0/2170929633 conn(0x188d710 msgr2=0x1b27af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2170929633 shutdown_connections 2024-05-08T22:40:58.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2170929633 wait complete. 2024-05-08T22:40:58.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- start start 2024-05-08T22:40:58.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 Processor -- start 2024-05-08T22:40:58.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1ba78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a640a0 con 0x1b462a0 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a693f0 con 0x1a8e520 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b25460 con 0x1b46680 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85fa2ee740 10 client.0 _ll_register_callbacks cb 0x1a65460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57938/0 (socket says 172.21.15.179:57938) 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 -- 172.21.15.179:0/2635468662 learned_addr learned my addr 172.21.15.179:0/2635468662 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1ba78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85f55dc700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 -- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 msgr2=0x1b446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 -- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 msgr2=0x1ba78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1ba78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 -- 172.21.15.179:0/2635468662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f00d1c80 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85effff700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c4a0 tx=0x7f85e0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29829b3f5532cb16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.828+0000 7f85cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0054650 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e0058480 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f85e005a410 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e0053fb0 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 10 client.8328 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x1a06c20 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0058480 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f85e00548b0 con 0x1b462a0 2024-05-08T22:40:58.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 10 client.8328 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 10 client.8328 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 20 client.8328 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1afd9e0 0x1afddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 4 client.8328 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x1909420 con 0x1afd9e0 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1afd9e0 0x1afddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f4ddb700 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1afd9e0 0x1afddc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8298a4535c384a7e server_cookie=4b56c06c2b8c3718 in_seq=0 out_seq=0 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 10 client.8328 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 1 -- 172.21.15.179:0/2635468662 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+418 (crc 0 0 0) 0x1909420 con 0x1afd9e0 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 10 client.8328 handle_command_reply: tid=0 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 client.8328 shutdown 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 msgr2=0x1b63b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c4a0 tx=0x7f85e0000ed0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 shutdown_connections 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1afd9e0 0x1afddc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b46680 0x1b446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85cf7fe700 0 client.8328 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b462a0 0x1b63b70 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 --2- 172.21.15.179:0/2635468662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a8e520 0x1ba78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 >> 172.21.15.179:0/2635468662 conn(0x188d710 msgr2=0x1affe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 shutdown_connections 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 1 -- 172.21.15.179:0/2635468662 wait complete. 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85fa2ee740 20 client.8328 trim_cache size 0 max 16384 2024-05-08T22:40:58.843 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "dir_frag", 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 16108948, 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627776, 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "frag": "*", 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir" 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "backtrace", 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 1118347355, 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:40:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/testdir/hardlink" 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "backtrace", 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 3401610050, 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627776, 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir" 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:58.845 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 msgr2=0x7f85d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 0x7f85d0068bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85d80011e0 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 msgr2=0x7f85f0157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c530 tx=0x7f85e000c8f0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 shutdown_connections 2024-05-08T22:40:58.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f85d0066730 0x7f85d0068bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85f01605e0 0x7f85f0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85f00d1190 0x7f85f0157380 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 --2- 172.21.15.179:0/1161224676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f00ce230 0x7f85f01f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.832+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 >> 172.21.15.179:0/1161224676 conn(0x7f85f00b4940 msgr2=0x7f85f015f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.836+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 shutdown_connections 2024-05-08T22:40:58.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.836+0000 7f85f5ddd700 1 -- 172.21.15.179:0/1161224676 wait complete. 2024-05-08T22:40:58.853 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:40:58.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 Processor -- start 2024-05-08T22:40:58.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 -- start start 2024-05-08T22:40:58.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f963406de70 0x7f9634161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9634164db0 0x7f963406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9634165d50 con 0x7f963415ef80 2024-05-08T22:40:58.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963406ad70 con 0x7f963406de70 2024-05-08T22:40:58.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9634165260 con 0x7f9634164db0 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963914e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f963415ef80 0x7f963406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57280/0 (socket says 172.21.15.179:57280) 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963914e700 1 -- 172.21.15.179:0/936670434 learned_addr learned my addr 172.21.15.179:0/936670434 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 --2- 172.21.15.179:0/936670434 >> v2:172.21.15.195:3300/0 conn(0x7f963406de70 0x7f9634161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963994f700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9634164db0 0x7f963406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f9627fff700 1 -- 172.21.15.179:0/936670434 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234314609 0 0) 0x7f9634165d50 con 0x7f963415ef80 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f9627fff700 1 -- 172.21.15.179:0/936670434 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96200035d0 con 0x7f963415ef80 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 -- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9634164db0 msgr2=0x7f963406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9634164db0 0x7f963406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 -- 172.21.15.179:0/936670434 >> v1:172.21.15.179:6789/0 conn(0x7f963415ef80 legacy=0x7f963406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 -- 172.21.15.179:0/936670434 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963406e4b0 con 0x7f963406de70 2024-05-08T22:40:58.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963894d700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963406de70 0x7f9634161790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f962c009d90 tx=0x7f962c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3525204d5db3b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f9627fff700 1 -- 172.21.15.179:0/936670434 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c00e070 con 0x7f963406de70 2024-05-08T22:40:58.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f9627fff700 1 -- 172.21.15.179:0/936670434 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c00bb20 con 0x7f963406de70 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f9627fff700 1 -- 172.21.15.179:0/936670434 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c00be90 con 0x7f963406de70 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.964+0000 7f963a150700 1 -- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963406de70 msgr2=0x7f9634161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963406de70 0x7f9634161790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f962c009d90 tx=0x7f962c00b330 comp rx=0 tx=0).stop 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 -- 172.21.15.179:0/936670434 shutdown_connections 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9634164db0 0x7f963406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 --2- 172.21.15.179:0/936670434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963406de70 0x7f9634161790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 -- 172.21.15.179:0/936670434 >> 172.21.15.179:0/936670434 conn(0x7f96340ce9d0 msgr2=0x7f96340cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 -- 172.21.15.179:0/936670434 shutdown_connections 2024-05-08T22:40:58.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 -- 172.21.15.179:0/936670434 wait complete. 2024-05-08T22:40:58.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 Processor -- start 2024-05-08T22:40:58.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.968+0000 7f963a150700 1 -- start start 2024-05-08T22:40:58.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f963406de70 0x7f96341eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 0x7f9634147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634153200 con 0x7f9634164db0 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634152650 con 0x7f963415ef80 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9634152860 con 0x7f963406de70 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57944/0 (socket says 172.21.15.179:57944) 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 -- 172.21.15.179:0/1065858460 learned_addr learned my addr 172.21.15.179:0/1065858460 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963894d700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 0x7f9634147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963914e700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f963406de70 0x7f96341eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 -- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f963406de70 msgr2=0x7f96341eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f963406de70 0x7f96341eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 -- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 msgr2=0x7f9634147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:58.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 0x7f9634147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 -- 172.21.15.179:0/1065858460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c009a70 con 0x7f9634164db0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963894d700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 0x7f9634147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963994f700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f963000e7f0 tx=0x7f963000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300095c0 con 0x7f9634164db0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630013680 con 0x7f9634164db0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9634152b40 con 0x7f9634164db0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96341480a0 con 0x7f9634164db0 2024-05-08T22:40:58.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630005c40 con 0x7f9634164db0 2024-05-08T22:40:58.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96300137f0 con 0x7f9634164db0 2024-05-08T22:40:58.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9634153340 con 0x7f9634164db0 2024-05-08T22:40:58.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.972+0000 7f9625ffb700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 0x7f9618068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:58.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.976+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f9630084240 con 0x7f9634164db0 2024-05-08T22:40:58.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.976+0000 7f963914e700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 0x7f9618068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:58.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.976+0000 7f963914e700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 0x7f9618068c20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9628007fd0 tx=0x7f9628009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:58.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:58.988+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9634153340 con 0x7f9634164db0 2024-05-08T22:40:59.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.196+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9634153340 con 0x7f9634164db0 2024-05-08T22:40:59.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.196+0000 7f9625ffb700 1 -- 172.21.15.179:0/1065858460 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9634153340 con 0x7f9634164db0 2024-05-08T22:40:59.200 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:40:59.200 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 msgr2=0x7f9618068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 0x7f9618068c20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9628007fd0 tx=0x7f9628009040 comp rx=0 tx=0).stop 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 msgr2=0x7f963415b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f963000e7f0 tx=0x7f963000eb00 comp rx=0 tx=0).stop 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 shutdown_connections 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9618066760 0x7f9618068c20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f963415ef80 0x7f9634147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9634164db0 0x7f963415b760 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 --2- 172.21.15.179:0/1065858460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f963406de70 0x7f96341eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 >> 172.21.15.179:0/1065858460 conn(0x7f96340ce9d0 msgr2=0x7f96340ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 shutdown_connections 2024-05-08T22:40:59.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.200+0000 7f963a150700 1 -- 172.21.15.179:0/1065858460 wait complete. 2024-05-08T22:40:59.215 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:40:59.215 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-08T22:40:59.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 Processor -- start 2024-05-08T22:40:59.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 -- start start 2024-05-08T22:40:59.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f862413cb80 0x7f862413f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f86240cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86240ce910 con 0x7f86240ce480 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862413fce0 con 0x7f862413cb80 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86240b2eb0 con 0x7f862413f860 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f8621d9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f86240cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f862259c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f862413cb80 0x7f862413f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f8622d9d700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f86240ce480 0x7f862415d750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:55148/0 (socket says 172.21.15.179:55148) 2024-05-08T22:40:59.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.328+0000 7f8622d9d700 1 -- 172.21.15.179:0/1601002313 learned_addr learned my addr 172.21.15.179:0/1601002313 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862259c700 1 -- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 msgr2=0x7f86240cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862259c700 1 --2- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f86240cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862259c700 1 -- 172.21.15.179:0/1601002313 >> v1:172.21.15.195:6789/0 conn(0x7f86240ce480 legacy=0x7f862415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862159a700 1 -- 172.21.15.179:0/1601002313 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274542548 0 0) 0x7f86240ce910 con 0x7f86240ce480 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862259c700 1 -- 172.21.15.179:0/1601002313 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862413cf60 con 0x7f862413cb80 2024-05-08T22:40:59.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862259c700 1 --2- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413f320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8614009d90 tx=0x7f861400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e8305195d7be6d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862159a700 1 -- 172.21.15.179:0/1601002313 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861400e070 con 0x7f862413cb80 2024-05-08T22:40:59.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862159a700 1 -- 172.21.15.179:0/1601002313 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861400bb20 con 0x7f862413cb80 2024-05-08T22:40:59.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862159a700 1 -- 172.21.15.179:0/1601002313 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614012430 con 0x7f862413cb80 2024-05-08T22:40:59.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 -- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 msgr2=0x7f862413f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 --2- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413f320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8614009d90 tx=0x7f861400b330 comp rx=0 tx=0).stop 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 -- 172.21.15.179:0/1601002313 shutdown_connections 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 --2- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f86240cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 --2- 172.21.15.179:0/1601002313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413f320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 -- 172.21.15.179:0/1601002313 >> 172.21.15.179:0/1601002313 conn(0x7f862406f790 msgr2=0x7f862406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 -- 172.21.15.179:0/1601002313 shutdown_connections 2024-05-08T22:40:59.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.332+0000 7f862359e700 1 -- 172.21.15.179:0/1601002313 wait complete. 2024-05-08T22:40:59.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 Processor -- start 2024-05-08T22:40:59.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- start start 2024-05-08T22:40:59.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f8624158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86241f35f0 con 0x7f86240ce480 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862413a120 con 0x7f862413cb80 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86241f2ce0 con 0x7f862413f860 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57960/0 (socket says 172.21.15.179:57960) 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 -- 172.21.15.179:0/558605983 learned_addr learned my addr 172.21.15.179:0/558605983 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f8622d9d700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f8624158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f8621d9b700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 -- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 msgr2=0x7f8624158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f8624158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 -- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 msgr2=0x7f862413b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 -- 172.21.15.179:0/558605983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614009a70 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862259c700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8614014040 tx=0x7f8614004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f861affd700 1 -- 172.21.15.179:0/558605983 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614020070 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f861affd700 1 -- 172.21.15.179:0/558605983 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86140098f0 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862413bf90 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f861affd700 1 -- 172.21.15.179:0/558605983 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614012430 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.336+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86241f3210 con 0x7f86240ce480 2024-05-08T22:40:59.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.340+0000 7f861affd700 1 -- 172.21.15.179:0/558605983 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8614024070 con 0x7f86240ce480 2024-05-08T22:40:59.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.340+0000 7f861affd700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 0x7f8604068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.340+0000 7f861affd700 1 -- 172.21.15.179:0/558605983 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f86140874d0 con 0x7f86240ce480 2024-05-08T22:40:59.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.340+0000 7f8621d9b700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 0x7f8604068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.340+0000 7f8621d9b700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 0x7f8604068bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f86240b5e10 tx=0x7f860c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 Processor -- start 2024-05-08T22:40:59.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 -- start start 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 0x21611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21a2dd0 0x2161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x20cf500 con 0x21acab0 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ccc90 con 0x21a2dd0 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20c8580 con 0x21ad3d0 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57966/0 (socket says 172.21.15.179:57966) 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 -- 172.21.15.179:0/2069397156 learned_addr learned my addr 172.21.15.179:0/2069397156 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8621d9b700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 0x21611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8622d9d700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21a2dd0 0x2161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 -- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 msgr2=0x21611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 0x21611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 -- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21a2dd0 msgr2=0x2161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21a2dd0 0x2161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 -- 172.21.15.179:0/2069397156 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1fbaee0 con 0x21acab0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8621d9b700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 0x21611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f862259c700 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8614085820 tx=0x7f8614000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a21b3a16cb4400 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8603fff700 1 -- 172.21.15.179:0/2069397156 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614060810 con 0x21acab0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8603fff700 1 -- 172.21.15.179:0/2069397156 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86140567c0 con 0x21acab0 2024-05-08T22:40:59.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.344+0000 7f8603fff700 1 -- 172.21.15.179:0/2069397156 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86140872b0 con 0x21acab0 2024-05-08T22:40:59.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- 172.21.15.179:0/2069397156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 msgr2=0x21ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8614085820 tx=0x7f8614000f40 comp rx=0 tx=0).stop 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- 172.21.15.179:0/2069397156 shutdown_connections 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21a2dd0 0x2161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21ad3d0 0x21611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 --2- 172.21.15.179:0/2069397156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x21ace90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- 172.21.15.179:0/2069397156 >> 172.21.15.179:0/2069397156 conn(0x1eef780 msgr2=0x218a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- 172.21.15.179:0/2069397156 shutdown_connections 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- 172.21.15.179:0/2069397156 wait complete. 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 -- start start 2024-05-08T22:40:59.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.348+0000 7f862bd17740 1 Processor -- start 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 0x2182b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x21801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x208b920 con 0x21acab0 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x208b5c0 con 0x21ad3d0 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x205de10 con 0x21a2dd0 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 10 client.0 _ll_register_callbacks cb 0x20c1bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57970/0 (socket says 172.21.15.179:57970) 2024-05-08T22:40:59.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 -- 172.21.15.179:0/591059456 learned_addr learned my addr 172.21.15.179:0/591059456 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862259c700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 0x2182b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8622d9d700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x21801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 -- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 msgr2=0x2182b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 0x2182b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 -- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 msgr2=0x21801a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x21801a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 -- 172.21.15.179:0/591059456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86241f3210 con 0x21acab0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862259c700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 0x2182b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f8621d9b700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f860c015440 tx=0x7f860c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2cfa0f46fa0edd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c022420 con 0x21acab0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f860c022580 con 0x21acab0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1efc490 con 0x21acab0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f860c022740 con 0x21acab0 2024-05-08T22:40:59.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 10 client.8343 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:59.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x208e600 con 0x21acab0 2024-05-08T22:40:59.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1efc490 con 0x21acab0 2024-05-08T22:40:59.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f860c0228a0 con 0x21acab0 2024-05-08T22:40:59.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 10 client.8343 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:59.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 10 client.8343 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:59.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 20 client.8343 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:59.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x21c0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 4 client.8343 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:59.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f46710 con 0x21b3710 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862259c700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x21c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 10 client.8343 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862259c700 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x21c0690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9323f58d6f0ec5e0 server_cookie=ad62314edda06ce0 in_seq=0 out_seq=0 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 1 -- 172.21.15.179:0/591059456 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1f46710 con 0x21b3710 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 10 client.8343 handle_command_reply: tid=0 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 client.8343 shutdown 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 msgr2=0x217fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f860c015440 tx=0x7f860c004770 comp rx=0 tx=0).stop 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 shutdown_connections 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x21c0690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x21801a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f86017fa700 0 client.8343 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21acab0 0x217fc60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 --2- 172.21.15.179:0/591059456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21a2dd0 0x2182b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 >> 172.21.15.179:0/591059456 conn(0x1eef780 msgr2=0x2187a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 shutdown_connections 2024-05-08T22:40:59.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.352+0000 7f862bd17740 1 -- 172.21.15.179:0/591059456 wait complete. 2024-05-08T22:40:59.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.356+0000 7f862bd17740 20 client.8343 trim_cache size 0 max 16384 2024-05-08T22:40:59.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.372+0000 7f862bd17740 1 Processor -- start 2024-05-08T22:40:59.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.372+0000 7f862bd17740 1 -- start start 2024-05-08T22:40:59.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x2189a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2189fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2082010 con 0x21a2dd0 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2087760 con 0x21ad3d0 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20c3d70 con 0x21acab0 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57972/0 (socket says 172.21.15.179:57972) 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4263174347 learned_addr learned my addr 172.21.15.179:0/4263174347 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8621d9b700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x2189a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8622d9d700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2189fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 msgr2=0x2189a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x2189a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 msgr2=0x2189fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2189fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4263174347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2074830 con 0x21a2dd0 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8622d9d700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2189fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8614025000 tx=0x7f861402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e831ab868a6c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8618ff9700 1 -- 172.21.15.179:0/4263174347 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614060810 con 0x21a2dd0 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8618ff9700 1 -- 172.21.15.179:0/4263174347 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86140870b0 con 0x21a2dd0 2024-05-08T22:40:59.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8618ff9700 1 -- 172.21.15.179:0/4263174347 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614058680 con 0x21a2dd0 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- 172.21.15.179:0/4263174347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 msgr2=0x21a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8614025000 tx=0x7f861402b000 comp rx=0 tx=0).stop 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- 172.21.15.179:0/4263174347 shutdown_connections 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2189fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x2189a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- 172.21.15.179:0/4263174347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x21a1560 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- 172.21.15.179:0/4263174347 >> 172.21.15.179:0/4263174347 conn(0x1eef780 msgr2=0x21b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- 172.21.15.179:0/4263174347 shutdown_connections 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- 172.21.15.179:0/4263174347 wait complete. 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- start start 2024-05-08T22:40:59.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 Processor -- start 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x21b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2212000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x20c5d90 con 0x21a2dd0 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20665a0 con 0x21ad3d0 2024-05-08T22:40:59.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2160a80 con 0x21acab0 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862bd17740 10 client.0 _ll_register_callbacks cb 0x20c7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57974/0 (socket says 172.21.15.179:57974) 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4281422713 learned_addr learned my addr 172.21.15.179:0/4281422713 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8621d9b700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x21b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f8622d9d700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2212000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 msgr2=0x21b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x21b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 -- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 msgr2=0x2212000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.376+0000 7f862259c700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2212000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862259c700 1 -- 172.21.15.179:0/4281422713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1fbaee0 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f8622d9d700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2212000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862259c700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8614084040 tx=0x7f861400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec8fbe05efe6a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614057910 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8614057a70 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86140567c0 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8614058680 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 10 client.8349 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x1f3ac10 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614068020 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8614060380 con 0x21a2dd0 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 10 client.8349 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 10 client.8349 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:59.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 20 client.8349 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x2212540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 4 client.8349 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x1f6d5f0 con 0x21b3710 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f8621d9b700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x2212540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 10 client.8349 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f8621d9b700 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x2212540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf8fe395614ac48 server_cookie=1661e3838fb8671f in_seq=0 out_seq=0 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 1 -- 172.21.15.179:0/4281422713 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1f6d5f0 con 0x21b3710 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 10 client.8349 handle_command_reply: tid=0 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 client.8349 shutdown 2024-05-08T22:40:59.388 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:59.389 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:40:59.389 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "7dc97a27-cf8e-4c8c-9efd-12e91993c4c4", 2024-05-08T22:40:59.389 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:40:59.389 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 msgr2=0x2160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8614084040 tx=0x7f861400b680 comp rx=0 tx=0).stop 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 shutdown_connections 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x21b3710 0x2212540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21ad3d0 0x2212000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f86017fa700 0 client.8349 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21acab0 0x21b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 --2- 172.21.15.179:0/4281422713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21a2dd0 0x2160540 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 >> 172.21.15.179:0/4281422713 conn(0x1eef780 msgr2=0x21bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 shutdown_connections 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 1 -- 172.21.15.179:0/4281422713 wait complete. 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862bd17740 20 client.8349 trim_cache size 0 max 16384 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 msgr2=0x7f8604068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.380+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 0x7f8604068bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f86240b5e10 tx=0x7f860c008040 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 msgr2=0x7f8624139be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8614014040 tx=0x7f8614004af0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 shutdown_connections 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8604066730 0x7f8604068bf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f862413cb80 0x7f862413b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f862413f860 0x7f8624158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 --2- 172.21.15.179:0/558605983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86240ce480 0x7f8624139be0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 >> 172.21.15.179:0/558605983 conn(0x7f862406f790 msgr2=0x7f8624153020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 shutdown_connections 2024-05-08T22:40:59.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f862359e700 1 -- 172.21.15.179:0/558605983 wait complete. 2024-05-08T22:40:59.392 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:40:59.384+0000 7f7f280c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)->f(v1 m2024-05-08T22:40:09.291554+0000) n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)->n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5633b5711b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-08T22:40:59.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:40:59.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 Processor -- start 2024-05-08T22:40:59.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 -- start start 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d9c0cdca0 0x7f8d9c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0ce5c0 con 0x7f8d9c0b65b0 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b8f00 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.516+0000 7f8da2f1f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cf640 con 0x7f8d9c06c240 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d9c0cdca0 0x7f8d9c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da271e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8d9c0b65b0 0x7f8d9c0b6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57284/0 (socket says 172.21.15.179:57284) 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da271e700 1 -- 172.21.15.179:0/2312354395 learned_addr learned my addr 172.21.15.179:0/2312354395 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da171c700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da0f1b700 1 -- 172.21.15.179:0/2312354395 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577600273 0 0) 0x7f8d9c0ce5c0 con 0x7f8d9c0b65b0 2024-05-08T22:40:59.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da0f1b700 1 -- 172.21.15.179:0/2312354395 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d880035d0 con 0x7f8d9c0b65b0 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 -- 172.21.15.179:0/2312354395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 msgr2=0x7f8d9c0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 -- 172.21.15.179:0/2312354395 >> v1:172.21.15.179:6789/0 conn(0x7f8d9c0b65b0 legacy=0x7f8d9c0b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 -- 172.21.15.179:0/2312354395 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0b8c20 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c0ce080 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009880 tx=0x7f8d8c009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71948567d537570 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da0f1b700 1 -- 172.21.15.179:0/2312354395 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c011830 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da0f1b700 1 -- 172.21.15.179:0/2312354395 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c011990 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da0f1b700 1 -- 172.21.15.179:0/2312354395 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c011d00 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2312354395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 msgr2=0x7f8d9c0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c0ce080 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009880 tx=0x7f8d8c009b90 comp rx=0 tx=0).stop 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2312354395 shutdown_connections 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2312354395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c0ce080 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2312354395 >> 172.21.15.179:0/2312354395 conn(0x7f8d9c0d2ed0 msgr2=0x7f8d9c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2312354395 shutdown_connections 2024-05-08T22:40:59.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.520+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2312354395 wait complete. 2024-05-08T22:40:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.524+0000 7f8da2f1f700 1 Processor -- start 2024-05-08T22:40:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.524+0000 7f8da2f1f700 1 -- start start 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 0x7f8d9c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b51f0 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b4570 con 0x7f8d9c0b65b0 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b4850 con 0x7f8d9c06c240 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57992/0 (socket says 172.21.15.179:57992) 2024-05-08T22:40:59.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 -- 172.21.15.179:0/2853997819 learned_addr learned my addr 172.21.15.179:0/2853997819 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da171c700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 0x7f8d9c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 -- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 msgr2=0x7f8d9c0a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0a9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 -- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 msgr2=0x7f8d9c1506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 0x7f8d9c1506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 -- 172.21.15.179:0/2853997819 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0094b0 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da171c700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 0x7f8d9c1506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da271e700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ca20 tx=0x7f8d9400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8d927fc700 1 -- 172.21.15.179:0/2853997819 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400a770 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8d927fc700 1 -- 172.21.15.179:0/2853997819 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9400a8d0 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c0b4b30 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8d927fc700 1 -- 172.21.15.179:0/2853997819 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ac60 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.528+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c0a9d60 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8d927fc700 1 -- 172.21.15.179:0/2853997819 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d9401b3b0 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8d927fc700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 0x7f8d80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8d927fc700 1 -- 172.21.15.179:0/2853997819 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8d94083cb0 con 0x7f8d9c0cdca0 2024-05-08T22:40:59.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 0x7f8d80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 0x7f8d80068bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c000c00 tx=0x7f8d8c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 Processor -- start 2024-05-08T22:40:59.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 -- start start 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c5840 0x17cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1758010 con 0x17ef6f0 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16e5f40 con 0x17c5840 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1738e30 con 0x17f25d0 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da171c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da171c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57998/0 (socket says 172.21.15.179:57998) 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.532+0000 7f8da171c700 1 -- 172.21.15.179:0/3110346241 learned_addr learned my addr 172.21.15.179:0/3110346241 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c5840 0x17cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da271e700 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 -- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 msgr2=0x17c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 -- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c5840 msgr2=0x17cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c5840 0x17cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 -- 172.21.15.179:0/3110346241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15a4a40 con 0x17ef6f0 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009bc0 tx=0x7f8d98009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7007f9d6f728f84f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8d7f7fe700 1 -- 172.21.15.179:0/3110346241 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004220 con 0x17ef6f0 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8d7f7fe700 1 -- 172.21.15.179:0/3110346241 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98004380 con 0x17ef6f0 2024-05-08T22:40:59.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8d7f7fe700 1 -- 172.21.15.179:0/3110346241 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98011750 con 0x17ef6f0 2024-05-08T22:40:59.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- 172.21.15.179:0/3110346241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 msgr2=0x17c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009bc0 tx=0x7f8d98009240 comp rx=0 tx=0).stop 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- 172.21.15.179:0/3110346241 shutdown_connections 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x17c5c20 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- 172.21.15.179:0/3110346241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c5840 0x17cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- 172.21.15.179:0/3110346241 >> 172.21.15.179:0/3110346241 conn(0x15595b0 msgr2=0x17f1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- 172.21.15.179:0/3110346241 shutdown_connections 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- 172.21.15.179:0/3110346241 wait complete. 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- start start 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 Processor -- start 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ef6f0 0x17c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1734150 con 0x17c5840 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1731eb0 con 0x17ef6f0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x16ef8e0 con 0x17f25d0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da7430740 10 client.0 _ll_register_callbacks cb 0x16ce670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da1f1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da1f1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58002/0 (socket says 172.21.15.179:58002) 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da1f1d700 1 -- 172.21.15.179:0/3321070974 learned_addr learned my addr 172.21.15.179:0/3321070974 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da171c700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ef6f0 0x17c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.536+0000 7f8da271e700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 -- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 msgr2=0x17c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 -- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ef6f0 msgr2=0x17c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ef6f0 0x17c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 -- 172.21.15.179:0/3321070974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d98009820 con 0x17c5840 2024-05-08T22:40:59.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:59.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009ee0 tx=0x7f8d8c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d60f888ffec2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c02b070 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8d8c003ce0 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15662c0 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d8c003ea0 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 10 client.8364 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x16f8290 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15662c0 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8d8c01c440 con 0x17c5840 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 10 client.8364 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 10 client.8364 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 20 client.8364 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:59.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17dacd0 0x1824ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 4 client.8364 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15b0540 con 0x17dacd0 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da171c700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17dacd0 0x1824ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 10 client.8364 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da171c700 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17dacd0 0x1824ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9f2769ab2fc56d server_cookie=f21c13ac5a32bfd8 in_seq=0 out_seq=0 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 1 -- 172.21.15.179:0/3321070974 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x15b0540 con 0x17dacd0 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8d7cff9700 10 client.8364 handle_command_reply: tid=0 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 client.8364 shutdown 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 msgr2=0x17de510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.540+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009ee0 tx=0x7f8d8c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 shutdown_connections 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17dacd0 0x1824ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ef6f0 0x17c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8d7cff9700 0 client.8364 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17f25d0 0x17c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 --2- 172.21.15.179:0/3321070974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17de510 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 >> 172.21.15.179:0/3321070974 conn(0x15595b0 msgr2=0x17c3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 shutdown_connections 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 1 -- 172.21.15.179:0/3321070974 wait complete. 2024-05-08T22:40:59.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.544+0000 7f8da7430740 20 client.8364 trim_cache size 0 max 16384 2024-05-08T22:40:59.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 Processor -- start 2024-05-08T22:40:59.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- start start 2024-05-08T22:40:59.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x17c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ddb80 con 0x17c5840 2024-05-08T22:40:59.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bba50 con 0x15595b0 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1759610 con 0x17dacd0 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58014/0 (socket says 172.21.15.179:58014) 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 -- 172.21.15.179:0/2688988119 learned_addr learned my addr 172.21.15.179:0/2688988119 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da271e700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x17c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 -- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 msgr2=0x17c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x17c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 -- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 msgr2=0x17d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 -- 172.21.15.179:0/2688988119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16de4c0 con 0x17c5840 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:40:59.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da171c700 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x16bc7b0 tx=0x7f8d98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bfc892f913db76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8d7ffff700 1 -- 172.21.15.179:0/2688988119 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98005a40 con 0x17c5840 2024-05-08T22:40:59.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8d7ffff700 1 -- 172.21.15.179:0/2688988119 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98023040 con 0x17c5840 2024-05-08T22:40:59.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8d7ffff700 1 -- 172.21.15.179:0/2688988119 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98005dc0 con 0x17c5840 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- 172.21.15.179:0/2688988119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 msgr2=0x17c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x16bc7b0 tx=0x7f8d98016000 comp rx=0 tx=0).stop 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- 172.21.15.179:0/2688988119 shutdown_connections 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x17c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c5840 0x17c8420 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 --2- 172.21.15.179:0/2688988119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- 172.21.15.179:0/2688988119 >> 172.21.15.179:0/2688988119 conn(0x17dc030 msgr2=0x1829550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- 172.21.15.179:0/2688988119 shutdown_connections 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- 172.21.15.179:0/2688988119 wait complete. 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 -- start start 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.564+0000 7f8da7430740 1 Processor -- start 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x1875460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d0620 con 0x17ef6f0 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1702c00 con 0x15595b0 2024-05-08T22:40:59.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18151a0 con 0x17dacd0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 10 client.0 _ll_register_callbacks cb 0x16ebd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58030/0 (socket says 172.21.15.179:58030) 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 -- 172.21.15.179:0/1293358061 learned_addr learned my addr 172.21.15.179:0/1293358061 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da171c700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x1875460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 -- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 msgr2=0x1875460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x1875460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 -- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 msgr2=0x17d7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 -- 172.21.15.179:0/1293358061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15a4a40 con 0x17ef6f0 2024-05-08T22:40:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da271e700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8d94051060 tx=0x7f8d94001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8dfd27dd97fcbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ae80 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8d9405cac0 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d98004380 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d9405cc80 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 10 client.8370 fetch_fsmap learned FSMap version 62 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x16d2790 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98004380 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8d94055b70 con 0x17ef6f0 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 10 client.8370 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 10 client.8370 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 20 client.8370 populate_metadata read hostname 'smithi179' 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17f25d0 0x1813b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:40:59.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 4 client.8370 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:40:59.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x15d52c0 con 0x17f25d0 2024-05-08T22:40:59.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17f25d0 0x1813b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8d7cff9700 10 client.8370 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.568+0000 7f8da1f1d700 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17f25d0 0x1813b90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3207c87af1d79b2c server_cookie=f47b2e5d34590c71 in_seq=0 out_seq=0 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8d7cff9700 1 -- 172.21.15.179:0/1293358061 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x15d52c0 con 0x17f25d0 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8d7cff9700 10 client.8370 handle_command_reply: tid=0 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 client.8370 shutdown 2024-05-08T22:40:59.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 msgr2=0x18759a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8d94051060 tx=0x7f8d94001250 comp rx=0 tx=0).stop 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 shutdown_connections 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x17f25d0 0x1813b90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ef6f0 0x18759a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8d7cff9700 0 client.8370 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17dacd0 0x1875460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 --2- 172.21.15.179:0/1293358061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x15595b0 0x17d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 >> 172.21.15.179:0/1293358061 conn(0x17dc030 msgr2=0x17c3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 shutdown_connections 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 1 -- 172.21.15.179:0/1293358061 wait complete. 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da7430740 20 client.8370 trim_cache size 0 max 16384 2024-05-08T22:40:59.579 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "7dc97a27-cf8e-4c8c-9efd-12e91993c4c4": { 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir", 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "7dc97a27-cf8e-4c8c-9efd-12e91993c4c4", 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:40:59.580 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:40:59.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 msgr2=0x7f8d80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 0x7f8d80068bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c000c00 tx=0x7f8d8c008030 comp rx=0 tx=0).stop 2024-05-08T22:40:59.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 msgr2=0x7f8d9c14de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:40:59.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ca20 tx=0x7f8d9400cb00 comp rx=0 tx=0).stop 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 shutdown_connections 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d80066730 0x7f8d80068bf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d9c0b65b0 0x7f8d9c1506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c0cdca0 0x7f8d9c14de60 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 --2- 172.21.15.179:0/2853997819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d9c06c240 0x7f8d9c0a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 >> 172.21.15.179:0/2853997819 conn(0x7f8d9c0d2ed0 msgr2=0x7f8d9c0cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 shutdown_connections 2024-05-08T22:40:59.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:40:59.572+0000 7f8da2f1f700 1 -- 172.21.15.179:0/2853997819 wait complete. 2024-05-08T22:40:59.590 INFO:tasks.cephfs.filesystem:scrub status for tag:7dc97a27-cf8e-4c8c-9efd-12e91993c4c4 - {'path': '/dir', 'tag': '7dc97a27-cf8e-4c8c-9efd-12e91993c4c4', 'options': 'recursive,repair'} 2024-05-08T22:41:22.561 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:41:22.565 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:41:29.592 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:41:29.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.715+0000 7f5b5e7d2700 1 Processor -- start 2024-05-08T22:41:29.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.715+0000 7f5b5e7d2700 1 -- start start 2024-05-08T22:41:29.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5e7d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b580b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5e7d2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580b46b0 con 0x7f5b581607f0 2024-05-08T22:41:29.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5e7d2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580b4910 con 0x7f5b580af530 2024-05-08T22:41:29.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5e7d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580b4bf0 con 0x7f5b580afa70 2024-05-08T22:41:29.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5b581607f0 0x7f5b580aee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37746/0 (socket says 172.21.15.179:37746) 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 -- 172.21.15.179:0/3764367096 learned_addr learned my addr 172.21.15.179:0/3764367096 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b580b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231426803 0 0) 0x7f5b580b46b0 con 0x7f5b581607f0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b440035d0 con 0x7f5b581607f0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673107030 0 0) 0x7f5b440035d0 con 0x7f5b581607f0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 msgr2=0x7f5b580b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b580b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 >> v1:172.21.15.195:6789/0 conn(0x7f5b580af530 legacy=0x7f5b5806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580b4ed0 con 0x7f5b581607f0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5b48001670 con 0x7f5b581607f0 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 >> v1:172.21.15.179:6789/0 conn(0x7f5b581607f0 legacy=0x7f5b580aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b44003a50 0x7f5b44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b44004400 0x7f5b440048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580b4bf0 con 0x7f5b44003a50 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4400bae0 con 0x7f5b44004400 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4400bca0 con 0x7f5b580afa70 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b44003a50 0x7f5b44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b44004400 0x7f5b440048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 -- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 msgr2=0x7f5b440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 -- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b44004400 msgr2=0x7f5b440048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b44004400 0x7f5b440048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 -- 172.21.15.179:0/3764367096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440035d0 con 0x7f5b44003a50 2024-05-08T22:41:29.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b580afa70 0x7f5b440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b44004400 0x7f5b440048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b44003a50 0x7f5b44003e30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c2f0 tx=0x7f5b440058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8025b6905d2ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800ce30 con 0x7f5b44003a50 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48005980 con 0x7f5b44003a50 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.719+0000 7f5b577fe700 1 -- 172.21.15.179:0/3764367096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48005d10 con 0x7f5b44003a50 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3764367096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b44003a50 msgr2=0x7f5b44003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3764367096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b44003a50 0x7f5b44003e30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c2f0 tx=0x7f5b440058a0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3764367096 reap_dead start 2024-05-08T22:41:29.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3764367096 shutdown_connections 2024-05-08T22:41:29.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3764367096 >> 172.21.15.179:0/3764367096 conn(0x7f5b58098a70 msgr2=0x7f5b580b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3764367096 shutdown_connections 2024-05-08T22:41:29.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3764367096 wait complete. 2024-05-08T22:41:29.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 Processor -- start 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- start start 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b58155d90 0x7f5b58158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b58156170 0x7f5b580c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5815bcc0 con 0x7f5b581588b0 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5815bf10 con 0x7f5b58156170 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.723+0000 7f5b5e7d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5815c1f0 con 0x7f5b58155d90 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55472/0 (socket says 172.21.15.179:55472) 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b58155d90 0x7f5b58158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3030185950 learned_addr learned my addr 172.21.15.179:0/3030185950 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b58156170 0x7f5b580c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b58155d90 msgr2=0x7f5b58158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b58155d90 0x7f5b58158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b58156170 msgr2=0x7f5b580c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b58156170 0x7f5b580c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3030185950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5815c4d0 con 0x7f5b581588b0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5b580cd930 tx=0x7f5b5000f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b557fa700 1 -- 172.21.15.179:0/3030185950 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000fbf0 con 0x7f5b581588b0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b557fa700 1 -- 172.21.15.179:0/3030185950 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50011040 con 0x7f5b581588b0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b557fa700 1 -- 172.21.15.179:0/3030185950 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50017400 con 0x7f5b581588b0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5815c7b0 con 0x7f5b581588b0 2024-05-08T22:41:29.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580ad390 con 0x7f5b581588b0 2024-05-08T22:41:29.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.727+0000 7f5b557fa700 1 -- 172.21.15.179:0/3030185950 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b500203b0 con 0x7f5b581588b0 2024-05-08T22:41:29.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b557fa700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 0x7f5b38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b557fa700 1 -- 172.21.15.179:0/3030185950 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f5b50084110 con 0x7f5b581588b0 2024-05-08T22:41:29.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 0x7f5b38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 0x7f5b38068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5b5806cf00 tx=0x7f5b48007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b61a81740 1 Processor -- start 2024-05-08T22:41:29.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.731+0000 7f5b61a81740 1 -- start start 2024-05-08T22:41:29.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xc55c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xca1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xbe8010 con 0xc829b0 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xb75f40 con 0xc55840 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xbc8e30 con 0xc825d0 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55478/0 (socket says 172.21.15.179:55478) 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/2548633251 learned_addr learned my addr 172.21.15.179:0/2548633251 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b57fff700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xc55c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xca1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 msgr2=0xca1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xca1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 msgr2=0xc55c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xc55c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/2548633251 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xa34a40 con 0xc829b0 2024-05-08T22:41:29.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b57fff700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xc55c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:29.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002230 tx=0x7f5b50054890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256c070098cff154 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b4dffb700 1 -- 172.21.15.179:0/2548633251 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5005b070 con 0xc829b0 2024-05-08T22:41:29.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b4dffb700 1 -- 172.21.15.179:0/2548633251 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50058d60 con 0xc829b0 2024-05-08T22:41:29.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b4dffb700 1 -- 172.21.15.179:0/2548633251 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5005d320 con 0xc829b0 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- 172.21.15.179:0/2548633251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 msgr2=0xca1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002230 tx=0x7f5b50054890 comp rx=0 tx=0).stop 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- 172.21.15.179:0/2548633251 shutdown_connections 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xca1780 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xca1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 --2- 172.21.15.179:0/2548633251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xc55c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- 172.21.15.179:0/2548633251 >> 172.21.15.179:0/2548633251 conn(0x9e95b0 msgr2=0xc81ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- 172.21.15.179:0/2548633251 shutdown_connections 2024-05-08T22:41:29.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- 172.21.15.179:0/2548633251 wait complete. 2024-05-08T22:41:29.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 -- start start 2024-05-08T22:41:29.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.735+0000 7f5b61a81740 1 Processor -- start 2024-05-08T22:41:29.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xc84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xbc4150 con 0xc829b0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xbc1eb0 con 0xc825d0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xb7f8e0 con 0xc55840 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 10 client.0 _ll_register_callbacks cb 0xb5e670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55490/0 (socket says 172.21.15.179:55490) 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/478409475 learned_addr learned my addr 172.21.15.179:0/478409475 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b57fff700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xc84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 msgr2=0xc84b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xc84b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 msgr2=0xc541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/478409475 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580ad390 con 0xc829b0 2024-05-08T22:41:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b57fff700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xc84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009fd0 tx=0x7f5b5000d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169762ba7f48523c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50054620 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f5b50082120 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x9f62c0 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b5005d320 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 10 client.8382 fetch_fsmap learned FSMap version 62 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0xb88290 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x9f62c0 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5b5005ce90 con 0xc829b0 2024-05-08T22:41:29.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 10 client.8382 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 10 client.8382 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 20 client.8382 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc86a20 0xc86e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 4 client.8382 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xa40540 con 0xc86a20 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b57fff700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc86a20 0xc86e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b36ffd700 10 client.8382 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.739+0000 7f5b57fff700 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc86a20 0xc86e00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e0c8b74db30769 server_cookie=c1150af094b19190 in_seq=0 out_seq=0 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b36ffd700 1 -- 172.21.15.179:0/478409475 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xa40540 con 0xc86a20 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b36ffd700 10 client.8382 handle_command_reply: tid=0 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 client.8382 shutdown 2024-05-08T22:41:29.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 msgr2=0xc87720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009fd0 tx=0x7f5b5000d070 comp rx=0 tx=0).stop 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 shutdown_connections 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc86a20 0xc86e00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b36ffd700 0 client.8382 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc87720 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 --2- 172.21.15.179:0/478409475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xc84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 >> 172.21.15.179:0/478409475 conn(0x9e95b0 msgr2=0xc81cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 shutdown_connections 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 1 -- 172.21.15.179:0/478409475 wait complete. 2024-05-08T22:41:29.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.743+0000 7f5b61a81740 20 client.8382 trim_cache size 0 max 16384 2024-05-08T22:41:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 Processor -- start 2024-05-08T22:41:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 -- start start 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xcb1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc7d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xc85c80 con 0xc829b0 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xa6db80 con 0xc825d0 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xb67230 con 0xc55840 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55502/0 (socket says 172.21.15.179:55502) 2024-05-08T22:41:29.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3646607182 learned_addr learned my addr 172.21.15.179:0/3646607182 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc7d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b57fff700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xcb1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 msgr2=0xcb1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xcb1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 msgr2=0xc7d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc7d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/3646607182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xb6e4c0 con 0xc829b0 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.763+0000 7f5b4ffff700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc7d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000f80 tx=0x7f5b50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59acb363cbb9697d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b4e7fc700 1 -- 172.21.15.179:0/3646607182 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50055630 con 0xc829b0 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b4e7fc700 1 -- 172.21.15.179:0/3646607182 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50054a70 con 0xc829b0 2024-05-08T22:41:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b4e7fc700 1 -- 172.21.15.179:0/3646607182 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50054640 con 0xc829b0 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- 172.21.15.179:0/3646607182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 msgr2=0xc7dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000f80 tx=0x7f5b50019000 comp rx=0 tx=0).stop 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- 172.21.15.179:0/3646607182 shutdown_connections 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc825d0 0xc7d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xc7dc50 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- 172.21.15.179:0/3646607182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc55840 0xcb1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- 172.21.15.179:0/3646607182 >> 172.21.15.179:0/3646607182 conn(0x9e95b0 msgr2=0xcb0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- 172.21.15.179:0/3646607182 shutdown_connections 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- 172.21.15.179:0/3646607182 wait complete. 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- start start 2024-05-08T22:41:29.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 Processor -- start 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xcb7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xd05780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb9100 con 0xc829b0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb9590 con 0xc55840 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xd06200 con 0xc825d0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b61a81740 10 client.0 _ll_register_callbacks cb 0xc5b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b4ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xd05780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55514/0 (socket says 172.21.15.179:55514) 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xcb7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/1467190176 learned_addr learned my addr 172.21.15.179:0/1467190176 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 msgr2=0xd05780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xd05780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 msgr2=0xcb7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xcb7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b5cd6f700 1 -- 172.21.15.179:0/1467190176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b40005200 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.767+0000 7f5b57fff700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xcb7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b5cd6f700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009fd0 tx=0x7f5b50007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb798487e017d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50055630 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f5b50063060 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b50058d60 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b50082120 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 10 client.8388 fetch_fsmap learned FSMap version 62 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0xb88290 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50058d60 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5b5006cb40 con 0xc829b0 2024-05-08T22:41:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 10 client.8388 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 10 client.8388 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 20 client.8388 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc58e60 0xd01d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 4 client.8388 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0xa652c0 con 0xc58e60 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b57fff700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc58e60 0xd01d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 10 client.8388 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b57fff700 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc58e60 0xd01d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76e7b4eaf0c2d3f server_cookie=8596951fabd88e2d in_seq=0 out_seq=0 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 1 -- 172.21.15.179:0/1467190176 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xa652c0 con 0xc58e60 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 10 client.8388 handle_command_reply: tid=0 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 client.8388 shutdown 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 msgr2=0xd05cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009fd0 tx=0x7f5b50007f90 comp rx=0 tx=0).stop 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 shutdown_connections 2024-05-08T22:41:29.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0xc58e60 0xd01d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xc829b0 0xd05cc0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b36ffd700 0 client.8388 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xc825d0 0xd05780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 --2- 172.21.15.179:0/1467190176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xc55840 0xcb7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 >> 172.21.15.179:0/1467190176 conn(0x9e95b0 msgr2=0xc7d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 shutdown_connections 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 1 -- 172.21.15.179:0/1467190176 wait complete. 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b61a81740 20 client.8388 trim_cache size 0 max 16384 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:41:29.781 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:41:29.782 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:41:29.782 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:41:29.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 msgr2=0x7f5b38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 0x7f5b38068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5b5806cf00 tx=0x7f5b48007000 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 msgr2=0x7f5b580c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.771+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5b580cd930 tx=0x7f5b5000f3b0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 shutdown_connections 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5b38066730 0x7f5b38068bf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b58156170 0x7f5b580c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b581588b0 0x7f5b580c8d00 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 --2- 172.21.15.179:0/3030185950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b58155d90 0x7f5b58158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 >> 172.21.15.179:0/3030185950 conn(0x7f5b58098a70 msgr2=0x7f5b5806e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 shutdown_connections 2024-05-08T22:41:29.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.775+0000 7f5b5e7d2700 1 -- 172.21.15.179:0/3030185950 wait complete. 2024-05-08T22:41:29.795 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:41:29.796 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:41:29.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.911+0000 7f21048dc700 1 Processor -- start 2024-05-08T22:41:29.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.911+0000 7f21048dc700 1 -- start start 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.911+0000 7f21048dc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f21001434d0 0x7f210006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f210006ed50 0x7f21000b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000cd200 con 0x7f210006dc20 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000cd4e0 con 0x7f21001434d0 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000cd7c0 con 0x7f210006ed50 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fe59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f210006dc20 0x7f210006e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37762/0 (socket says 172.21.15.179:37762) 2024-05-08T22:41:29.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fe59c700 1 -- 172.21.15.179:0/1440995622 learned_addr learned my addr 172.21.15.179:0/1440995622 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 --2- 172.21.15.179:0/1440995622 >> v2:172.21.15.179:3300/0 conn(0x7f21001434d0 0x7f210006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20ff59e700 1 --2- 172.21.15.179:0/1440995622 >> v2:172.21.15.195:3300/0 conn(0x7f210006ed50 0x7f21000b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919434793 0 0) 0x7f21000cd200 con 0x7f210006dc20 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e80035d0 con 0x7f210006dc20 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 733837498 0 0) 0x7f20e80035d0 con 0x7f210006dc20 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 -- 172.21.15.179:0/1440995622 >> v2:172.21.15.195:3300/0 conn(0x7f210006ed50 msgr2=0x7f21000b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 --2- 172.21.15.179:0/1440995622 >> v2:172.21.15.195:3300/0 conn(0x7f210006ed50 0x7f21000b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 -- 172.21.15.179:0/1440995622 >> v1:172.21.15.179:6789/0 conn(0x7f210006dc20 legacy=0x7f210006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 -- 172.21.15.179:0/1440995622 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21000cdaa0 con 0x7f21001434d0 2024-05-08T22:41:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fed9d700 1 --2- 172.21.15.179:0/1440995622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21001434d0 0x7f210006d6e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f210006aec0 tx=0x7f20f4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685d1ad8cfa83f60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0001670 con 0x7f21001434d0 2024-05-08T22:41:29.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f4004030 con 0x7f21001434d0 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f20fdd9b700 1 -- 172.21.15.179:0/1440995622 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40043c0 con 0x7f21001434d0 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- 172.21.15.179:0/1440995622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21001434d0 msgr2=0x7f210006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 --2- 172.21.15.179:0/1440995622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21001434d0 0x7f210006d6e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f210006aec0 tx=0x7f20f4009b30 comp rx=0 tx=0).stop 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- 172.21.15.179:0/1440995622 shutdown_connections 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 --2- 172.21.15.179:0/1440995622 >> v2:172.21.15.195:3300/0 conn(0x7f210006ed50 0x7f21000b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 --2- 172.21.15.179:0/1440995622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21001434d0 0x7f210006d6e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- 172.21.15.179:0/1440995622 >> 172.21.15.179:0/1440995622 conn(0x7f21000ce9d0 msgr2=0x7f21000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- 172.21.15.179:0/1440995622 shutdown_connections 2024-05-08T22:41:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.915+0000 7f21048dc700 1 -- 172.21.15.179:0/1440995622 wait complete. 2024-05-08T22:41:29.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 Processor -- start 2024-05-08T22:41:29.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- start start 2024-05-08T22:41:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21001434d0 0x7f210013b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f210006bd60 0x7f210013b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210013da00 con 0x7f210006ed50 2024-05-08T22:41:29.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000ca760 con 0x7f21001434d0 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000caa40 con 0x7f210006bd60 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55536/0 (socket says 172.21.15.179:55536) 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 -- 172.21.15.179:0/902100169 learned_addr learned my addr 172.21.15.179:0/902100169 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fe59c700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21001434d0 0x7f210013b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20ff59e700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f210006bd60 0x7f210013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 -- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f210006bd60 msgr2=0x7f210013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f210006bd60 0x7f210013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 -- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21001434d0 msgr2=0x7f210013b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21001434d0 0x7f210013b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 -- 172.21.15.179:0/902100169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40094b0 con 0x7f210006ed50 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20fed9d700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000c00 tx=0x7f20f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4003af0 con 0x7f210006ed50 2024-05-08T22:41:29.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f401f070 con 0x7f210006ed50 2024-05-08T22:41:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21000cad20 con 0x7f210006ed50 2024-05-08T22:41:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4003e70 con 0x7f210006ed50 2024-05-08T22:41:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.919+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100153e60 con 0x7f210006ed50 2024-05-08T22:41:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000cb240 con 0x7f210006ed50 2024-05-08T22:41:29.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20f4021030 con 0x7f210006ed50 2024-05-08T22:41:29.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f20e77fe700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 0x7f20ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:29.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f20f4085720 con 0x7f210006ed50 2024-05-08T22:41:29.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f20fe59c700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 0x7f20ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:29.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.923+0000 7f20fe59c700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 0x7f20ec068bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001820 tx=0x7f20f0001230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:29.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:29.927+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f21000cb240 con 0x7f210006ed50 2024-05-08T22:41:30.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21000cb240 con 0x7f210006ed50 2024-05-08T22:41:30.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f20e77fe700 1 -- 172.21.15.179:0/902100169 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f21000cb240 con 0x7f210006ed50 2024-05-08T22:41:30.147 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:30.148 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:30.890207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:41:30.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 msgr2=0x7f20ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 0x7f20ec068bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001820 tx=0x7f20f0001230 comp rx=0 tx=0).stop 2024-05-08T22:41:30.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 msgr2=0x7f21000c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000c00 tx=0x7f20f4004770 comp rx=0 tx=0).stop 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 shutdown_connections 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20ec066730 0x7f20ec068bf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21001434d0 0x7f210013b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f210006bd60 0x7f210013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 --2- 172.21.15.179:0/902100169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f210006ed50 0x7f21000c9df0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.143+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 >> 172.21.15.179:0/902100169 conn(0x7f21000ce9d0 msgr2=0x7f21000cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.147+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 shutdown_connections 2024-05-08T22:41:30.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.147+0000 7f21048dc700 1 -- 172.21.15.179:0/902100169 wait complete. 2024-05-08T22:41:30.152 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 62 2024-05-08T22:41:30.162 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T22:41:30.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.271+0000 7f6b6b59e700 1 Processor -- start 2024-05-08T22:41:30.276 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.271+0000 7f6b6b59e700 1 -- start start 2024-05-08T22:41:30.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.271+0000 7f6b6b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 0x7f6b6c0d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.271+0000 7f6b6b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.279 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.271+0000 7f6b6b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0d16c0 con 0x7f6b6c1605e0 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d2ca0 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ce4d0 con 0x7f6b6c0d1190 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55540/0 (socket says 172.21.15.179:55540) 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/935323546 learned_addr learned my addr 172.21.15.179:0/935323546 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/935323546 >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 0x7f6b6c0d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/935323546 >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 msgr2=0x7f6b6c0d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/935323546 >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 0x7f6b6c0d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/935323546 >> v1:172.21.15.195:6789/0 conn(0x7f6b6c1605e0 legacy=0x7f6b6c0b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/935323546 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0d19a0 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/935323546 >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 0x7f6b6c0d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/935323546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c08b2d0 tx=0x7f6b6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600eb523caf33189 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6959a700 1 -- 172.21.15.179:0/935323546 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400e070 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6959a700 1 -- 172.21.15.179:0/935323546 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6400b980 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6959a700 1 -- 172.21.15.179:0/935323546 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400bd30 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- 172.21.15.179:0/935323546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0cdb20 msgr2=0x7f6b6c0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/935323546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c08b2d0 tx=0x7f6b6400b330 comp rx=0 tx=0).stop 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- 172.21.15.179:0/935323546 shutdown_connections 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/935323546 >> v2:172.21.15.195:3300/0 conn(0x7f6b6c0d1190 0x7f6b6c0d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/935323546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c0cdf90 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- 172.21.15.179:0/935323546 >> 172.21.15.179:0/935323546 conn(0x7f6b6c0b4940 msgr2=0x7f6b6c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- 172.21.15.179:0/935323546 shutdown_connections 2024-05-08T22:41:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.275+0000 7f6b6b59e700 1 -- 172.21.15.179:0/935323546 wait complete. 2024-05-08T22:41:30.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 Processor -- start 2024-05-08T22:41:30.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- start start 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b6c1605e0 0x7f6b6c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ab070 con 0x7f6b6c0d1190 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15b460 con 0x7f6b6c0cdb20 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1f2b00 con 0x7f6b6c1605e0 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55544/0 (socket says 172.21.15.179:55544) 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 -- 172.21.15.179:0/1093201879 learned_addr learned my addr 172.21.15.179:0/1093201879 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b6c1605e0 0x7f6b6c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 -- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b6c1605e0 msgr2=0x7f6b6c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b6c1605e0 0x7f6b6c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 -- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c0cdb20 msgr2=0x7f6b6c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 -- 172.21.15.179:0/1093201879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64009b30 con 0x7f6b6c0d1190 2024-05-08T22:41:30.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e760 tx=0x7f6b5c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b62ffd700 1 -- 172.21.15.179:0/1093201879 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00c070 con 0x7f6b6c0d1190 2024-05-08T22:41:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b62ffd700 1 -- 172.21.15.179:0/1093201879 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c004250 con 0x7f6b6c0d1190 2024-05-08T22:41:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b62ffd700 1 -- 172.21.15.179:0/1093201879 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0036c0 con 0x7f6b6c0d1190 2024-05-08T22:41:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c1f2de0 con 0x7f6b6c0d1190 2024-05-08T22:41:30.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.279+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c1f33a0 con 0x7f6b6c0d1190 2024-05-08T22:41:30.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b62ffd700 1 -- 172.21.15.179:0/1093201879 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b5c003820 con 0x7f6b6c0d1190 2024-05-08T22:41:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b62ffd700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 0x7f6b4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b62ffd700 1 -- 172.21.15.179:0/1093201879 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f6b5c083f40 con 0x7f6b6c0d1190 2024-05-08T22:41:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 0x7f6b4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b73ee0740 1 Processor -- start 2024-05-08T22:41:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.283+0000 7f6b73ee0740 1 -- start start 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 0x7f6b4c068bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001800 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b532a0 0x1b1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b1c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7a810 con 0x1a9b520 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a77fa0 con 0x1b532a0 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a73890 con 0x1b53680 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55550/0 (socket says 172.21.15.179:55550) 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 -- 172.21.15.179:0/854339661 learned_addr learned my addr 172.21.15.179:0/854339661 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b1c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b532a0 0x1b1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 -- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 msgr2=0x1b1c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b1c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 -- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b532a0 msgr2=0x1b1bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b532a0 0x1b1bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 -- 172.21.15.179:0/854339661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18e5ba0 con 0x1a9b520 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400f770 tx=0x7f6b54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaa5ca74caafa8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b4bfff700 1 -- 172.21.15.179:0/854339661 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400fe40 con 0x1a9b520 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b4bfff700 1 -- 172.21.15.179:0/854339661 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54014450 con 0x1a9b520 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b4bfff700 1 -- 172.21.15.179:0/854339661 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540147c0 con 0x1a9b520 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- 172.21.15.179:0/854339661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 msgr2=0x1b52d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400f770 tx=0x7f6b54000ed0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- 172.21.15.179:0/854339661 shutdown_connections 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1b532a0 0x1b1bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b1c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/854339661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1a9b520 0x1b52d60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- 172.21.15.179:0/854339661 >> 172.21.15.179:0/854339661 conn(0x189a710 msgr2=0x1b34d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- 172.21.15.179:0/854339661 shutdown_connections 2024-05-08T22:41:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- 172.21.15.179:0/854339661 wait complete. 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 -- start start 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.287+0000 7f6b73ee0740 1 Processor -- start 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a36d80 con 0x1b532a0 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a36a20 con 0x1a9b520 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1966770 con 0x1b53680 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 10 client.0 _ll_register_callbacks cb 0x1a6cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55564/0 (socket says 172.21.15.179:55564) 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 -- 172.21.15.179:0/505573385 learned_addr learned my addr 172.21.15.179:0/505573385 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 -- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 msgr2=0x1b0ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 -- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 msgr2=0x1b52350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b52350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 -- 172.21.15.179:0/505573385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0d16c0 con 0x1b532a0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b52350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00efd0 tx=0x7f6b5c0555c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a30df23b4515fee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c058c20 con 0x1b532a0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f6b5c05d1e0 con 0x1b532a0 2024-05-08T22:41:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18a7420 con 0x1b532a0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b5c0542e0 con 0x1b532a0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 10 client.8409 fetch_fsmap learned FSMap version 62 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x1a39910 con 0x1b532a0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18a7420 con 0x1b532a0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6b5c05cd50 con 0x1b532a0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 10 client.8409 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 10 client.8409 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 20 client.8409 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b0c380 0x1b0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 4 client.8409 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18f16a0 con 0x1b0c380 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b0c380 0x1b0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b497fa700 10 client.8409 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:30.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.291+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b0c380 0x1b0c760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99e8bce09624342 server_cookie=49cd90318636551c in_seq=0 out_seq=0 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b497fa700 1 -- 172.21.15.179:0/505573385 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x18f16a0 con 0x1b0c380 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b497fa700 10 client.8409 handle_command_reply: tid=0 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 client.8409 shutdown 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 msgr2=0x1b0ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00efd0 tx=0x7f6b5c0555c0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 shutdown_connections 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b0c380 0x1b0c760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b497fa700 0 client.8409 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b0ea40 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/505573385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b52350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 >> 172.21.15.179:0/505573385 conn(0x189a710 msgr2=0x1b322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 shutdown_connections 2024-05-08T22:41:30.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 1 -- 172.21.15.179:0/505573385 wait complete. 2024-05-08T22:41:30.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.295+0000 7f6b73ee0740 20 client.8409 trim_cache size 0 max 16384 2024-05-08T22:41:30.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 Processor -- start 2024-05-08T22:41:30.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- start start 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b10070 con 0x1b532a0 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a9a9a0 con 0x1a9b520 2024-05-08T22:41:30.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a34660 con 0x1b53680 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55574/0 (socket says 172.21.15.179:55574) 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 -- 172.21.15.179:0/4198218735 learned_addr learned my addr 172.21.15.179:0/4198218735 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 -- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 msgr2=0x1b0afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 -- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 msgr2=0x1b34af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b34af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 -- 172.21.15.179:0/4198218735 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a1fb40 con 0x1b532a0 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:30.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c054d30 tx=0x7f6b5c069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e591247ba1c3cb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b60ff9700 1 -- 172.21.15.179:0/4198218735 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c075020 con 0x1b532a0 2024-05-08T22:41:30.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b60ff9700 1 -- 172.21.15.179:0/4198218735 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c051130 con 0x1b532a0 2024-05-08T22:41:30.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b60ff9700 1 -- 172.21.15.179:0/4198218735 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0554f0 con 0x1b532a0 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- 172.21.15.179:0/4198218735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 msgr2=0x1b35030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c054d30 tx=0x7f6b5c069000 comp rx=0 tx=0).stop 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- 172.21.15.179:0/4198218735 shutdown_connections 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b53680 0x1b0afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b532a0 0x1b35030 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/4198218735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1b34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- 172.21.15.179:0/4198218735 >> 172.21.15.179:0/4198218735 conn(0x189a710 msgr2=0x1b0c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- 172.21.15.179:0/4198218735 shutdown_connections 2024-05-08T22:41:30.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.311+0000 7f6b73ee0740 1 -- 172.21.15.179:0/4198218735 wait complete. 2024-05-08T22:41:30.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 -- start start 2024-05-08T22:41:30.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 Processor -- start 2024-05-08T22:41:30.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1bb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b532a0 0x1b0cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a190a0 con 0x1b53680 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a72720 con 0x1a9b520 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a763f0 con 0x1b532a0 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b73ee0740 10 client.0 _ll_register_callbacks cb 0x1a72460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55582/0 (socket says 172.21.15.179:55582) 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/1087599923 learned_addr learned my addr 172.21.15.179:0/1087599923 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1bb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b69d9b700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b532a0 0x1b0cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b532a0 msgr2=0x1b0cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b532a0 0x1b0cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 msgr2=0x1bb3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.315+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1bb3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b6ad9d700 1 -- 172.21.15.179:0/1087599923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a39910 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b6ad9d700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c08b560 tx=0x7f6b64004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e458ca5b713b0d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64004700 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f6b64004860 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1b1e410 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b6401d420 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 10 client.8415 fetch_fsmap learned FSMap version 62 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x1a13c20 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1b1e410 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6b6400b050 con 0x1b53680 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 10 client.8415 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T22:41:30.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 10 client.8415 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 20 client.8415 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b57d10 0x1b580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 4 client.8415 mds_command: new command op to 8154 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 --> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x7f6b54014450 con 0x1b57d10 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b57d10 0x1b580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 10 client.8415 ms_handle_connect on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:30.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b6a59c700 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b57d10 0x1b580f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b412efdfbe5148 server_cookie=88b66957dca49191 in_seq=0 out_seq=0 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 1 -- 172.21.15.179:0/1087599923 <== mds.0 v2:172.21.15.179:6834/55939533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x7f6b54014450 con 0x1b57d10 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 10 client.8415 handle_command_reply: tid=0 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 client.8415 shutdown 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 msgr2=0x1b0d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c08b560 tx=0x7f6b64004b60 comp rx=0 tx=0).stop 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 shutdown_connections 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:6834/55939533,v1:172.21.15.179:6835/55939533] conn(0x1b57d10 0x1b580f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1b53680 0x1b0d100 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b497fa700 0 client.8415 ms_handle_reset on v2:172.21.15.179:6834/55939533 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1b532a0 0x1b0cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 --2- 172.21.15.179:0/1087599923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1a9b520 0x1bb3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 >> 172.21.15.179:0/1087599923 conn(0x189a710 msgr2=0x1b0fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 shutdown_connections 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 1 -- 172.21.15.179:0/1087599923 wait complete. 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.319+0000 7f6b73ee0740 20 client.8415 trim_cache size 0 max 16384 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: "damage_type": "backtrace", 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 1118347355, 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1099511627777, 2024-05-08T22:41:30.329 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/testdir/hardlink" 2024-05-08T22:41:30.330 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:41:30.330 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:41:30.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 msgr2=0x7f6b4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 0x7f6b4c068bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001800 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2024-05-08T22:41:30.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 msgr2=0x7f6b6c0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e760 tx=0x7f6b5c00ea70 comp rx=0 tx=0).stop 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 shutdown_connections 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b4c066730 0x7f6b4c068bf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b6c1605e0 0x7f6b6c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b6c0d1190 0x7f6b6c0b1010 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 --2- 172.21.15.179:0/1093201879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c0cdb20 0x7f6b6c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 >> 172.21.15.179:0/1093201879 conn(0x7f6b6c0b4940 msgr2=0x7f6b6c0b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 shutdown_connections 2024-05-08T22:41:30.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.323+0000 7f6b6b59e700 1 -- 172.21.15.179:0/1093201879 wait complete. 2024-05-08T22:41:30.344 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:41:30.345 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:41:30.352 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:41:30.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 Processor -- start 2024-05-08T22:41:30.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 -- start start 2024-05-08T22:41:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdbe8157110 0x7fdbe806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe806bc90 con 0x7fdbe81574f0 2024-05-08T22:41:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe806bf70 con 0x7fdbe80ae730 2024-05-08T22:41:30.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbee2b7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe815ad70 con 0x7fdbe8157110 2024-05-08T22:41:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbecab4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdbe81574f0 0x7fdbe815a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37774/0 (socket says 172.21.15.179:37774) 2024-05-08T22:41:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbecab4700 1 -- 172.21.15.179:0/2657550105 learned_addr learned my addr 172.21.15.179:0/2657550105 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:30.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbed2b5700 1 --2- 172.21.15.179:0/2657550105 >> v2:172.21.15.195:3300/0 conn(0x7fdbe8157110 0x7fdbe806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbe7fff700 1 -- 172.21.15.179:0/2657550105 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419919723 0 0) 0x7fdbe806bc90 con 0x7fdbe81574f0 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbe7fff700 1 -- 172.21.15.179:0/2657550105 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd00035d0 con 0x7fdbe81574f0 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbed2b5700 1 -- 172.21.15.179:0/2657550105 >> v1:172.21.15.195:6789/0 conn(0x7fdbe80ae730 legacy=0x7fdbe80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbed2b5700 1 -- 172.21.15.179:0/2657550105 >> v1:172.21.15.179:6789/0 conn(0x7fdbe81574f0 legacy=0x7fdbe815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbed2b5700 1 -- 172.21.15.179:0/2657550105 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe806c250 con 0x7fdbe8157110 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbed2b5700 1 --2- 172.21.15.179:0/2657550105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe8157110 0x7fdbe806b440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009850 tx=0x7fdbdc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736c523b4b65f657 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbe7fff700 1 -- 172.21.15.179:0/2657550105 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc011830 con 0x7fdbe8157110 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbe7fff700 1 -- 172.21.15.179:0/2657550105 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc011990 con 0x7fdbe8157110 2024-05-08T22:41:30.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.467+0000 7fdbe7fff700 1 -- 172.21.15.179:0/2657550105 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc011d00 con 0x7fdbe8157110 2024-05-08T22:41:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2657550105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe8157110 msgr2=0x7fdbe806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2657550105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe8157110 0x7fdbe806b440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009850 tx=0x7fdbdc009b60 comp rx=0 tx=0).stop 2024-05-08T22:41:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2657550105 shutdown_connections 2024-05-08T22:41:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2657550105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe8157110 0x7fdbe806b440 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2657550105 >> 172.21.15.179:0/2657550105 conn(0x7fdbe8098a70 msgr2=0x7fdbe80b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2657550105 shutdown_connections 2024-05-08T22:41:30.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2657550105 wait complete. 2024-05-08T22:41:30.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 Processor -- start 2024-05-08T22:41:30.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.471+0000 7fdbee2b7700 1 -- start start 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe80ae730 0x7fdbe8145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbe8157110 0x7fdbe80acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81356b0 con 0x7fdbe81574f0 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80ad640 con 0x7fdbe80ae730 2024-05-08T22:41:30.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80ad920 con 0x7fdbe8157110 2024-05-08T22:41:30.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55590/0 (socket says 172.21.15.179:55590) 2024-05-08T22:41:30.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 -- 172.21.15.179:0/2680669973 learned_addr learned my addr 172.21.15.179:0/2680669973 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbed2b5700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe80ae730 0x7fdbe8145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbecab4700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbe8157110 0x7fdbe80acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 -- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbe8157110 msgr2=0x7fdbe80acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbe8157110 0x7fdbe80acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 -- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe80ae730 msgr2=0x7fdbe8145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe80ae730 0x7fdbe8145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 -- 172.21.15.179:0/2680669973 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0094b0 con 0x7fdbe81574f0 2024-05-08T22:41:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbedab6700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c480 tx=0x7fdbd800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80052d0 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800ce90 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe80adc00 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80036a0 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe80ae1c0 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.475+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe806bc90 con 0x7fdbe81574f0 2024-05-08T22:41:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.479+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbd8003800 con 0x7fdbe81574f0 2024-05-08T22:41:30.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.479+0000 7fdbe5ffb700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.479+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fdbd8083710 con 0x7fdbe81574f0 2024-05-08T22:41:30.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.479+0000 7fdbed2b5700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.479+0000 7fdbed2b5700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009ba0 tx=0x7fdbdc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.487+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdbd80507f0 con 0x7fdbe81574f0 2024-05-08T22:41:30.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.659+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbe80672c0 con 0x7fdbe81574f0 2024-05-08T22:41:30.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.663+0000 7fdbe5ffb700 1 -- 172.21.15.179:0/2680669973 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16622 (secure 0 0 0) 0x7fdbd8053e10 con 0x7fdbe81574f0 2024-05-08T22:41:30.668 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:30.668 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":99,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:19.823666+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 msgr2=0x7fdbd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009ba0 tx=0x7fdbdc008030 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 msgr2=0x7fdbe8157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c480 tx=0x7fdbd800c790 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 shutdown_connections 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbe81574f0 0x7fdbe8157b30 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbe8157110 0x7fdbe80acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 --2- 172.21.15.179:0/2680669973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbe80ae730 0x7fdbe8145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 >> 172.21.15.179:0/2680669973 conn(0x7fdbe8098a70 msgr2=0x7fdbe814a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 shutdown_connections 2024-05-08T22:41:30.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.667+0000 7fdbee2b7700 1 -- 172.21.15.179:0/2680669973 wait complete. 2024-05-08T22:41:30.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:41:30.683 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:41:30.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 Processor -- start 2024-05-08T22:41:30.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 -- start start 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1da015d1f0 0x7f1da015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00ad590 con 0x7f1da00b1b30 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da015dcf0 con 0x7f1da00b2720 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1da562c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da015db10 con 0x7f1da015d1f0 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1da015d1f0 0x7f1da015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9f7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1da00b1b30 0x7f1da00ac180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37784/0 (socket says 172.21.15.179:37784) 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9f7fe700 1 -- 172.21.15.179:0/815498594 learned_addr learned my addr 172.21.15.179:0/815498594 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9ffff700 1 -- 172.21.15.179:0/815498594 >> v1:172.21.15.195:6789/0 conn(0x7f1da00b2720 legacy=0x7f1da00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9ffff700 1 -- 172.21.15.179:0/815498594 >> v1:172.21.15.179:6789/0 conn(0x7f1da00b1b30 legacy=0x7f1da00ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9ffff700 1 -- 172.21.15.179:0/815498594 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da015dfd0 con 0x7f1da015d1f0 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9ffff700 1 --2- 172.21.15.179:0/815498594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da015d1f0 0x7f1da015d5d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d90 tx=0x7f1d9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a541cc107096c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9effd700 1 -- 172.21.15.179:0/815498594 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000e070 con 0x7f1da015d1f0 2024-05-08T22:41:30.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9effd700 1 -- 172.21.15.179:0/815498594 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9000bb90 con 0x7f1da015d1f0 2024-05-08T22:41:30.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.799+0000 7f1d9effd700 1 -- 172.21.15.179:0/815498594 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90003a40 con 0x7f1da015d1f0 2024-05-08T22:41:30.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- 172.21.15.179:0/815498594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da015d1f0 msgr2=0x7f1da015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 --2- 172.21.15.179:0/815498594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da015d1f0 0x7f1da015d5d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d90 tx=0x7f1d9000b330 comp rx=0 tx=0).stop 2024-05-08T22:41:30.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- 172.21.15.179:0/815498594 shutdown_connections 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 --2- 172.21.15.179:0/815498594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da015d1f0 0x7f1da015d5d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- 172.21.15.179:0/815498594 >> 172.21.15.179:0/815498594 conn(0x7f1da00cea30 msgr2=0x7f1da00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- 172.21.15.179:0/815498594 shutdown_connections 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- 172.21.15.179:0/815498594 wait complete. 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 Processor -- start 2024-05-08T22:41:30.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.803+0000 7f1da562c700 1 -- start start 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1da00b1b30 0x7f1da01f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1da015d1f0 0x7f1da015f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00ce2d0 con 0x7f1da00b2720 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da015b130 con 0x7f1da00b1b30 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01f6e70 con 0x7f1da015d1f0 2024-05-08T22:41:30.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55602/0 (socket says 172.21.15.179:55602) 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 -- 172.21.15.179:0/1148660580 learned_addr learned my addr 172.21.15.179:0/1148660580 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9ffff700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1da00b1b30 0x7f1da01f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da4e2b700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1da015d1f0 0x7f1da015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 -- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1da015d1f0 msgr2=0x7f1da015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1da015d1f0 0x7f1da015f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 -- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1da00b1b30 msgr2=0x7f1da01f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1da00b1b30 0x7f1da01f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 -- 172.21.15.179:0/1148660580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009a70 con 0x7f1da00b2720 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9f7fe700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c500 tx=0x7f1d8800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88005d90 con 0x7f1da00b2720 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d880070d0 con 0x7f1da00b2720 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88007480 con 0x7f1da00b2720 2024-05-08T22:41:30.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da01f65d0 con 0x7f1da00b2720 2024-05-08T22:41:30.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da01f6b90 con 0x7f1da00b2720 2024-05-08T22:41:30.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.807+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da004e700 con 0x7f1da00b2720 2024-05-08T22:41:30.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.811+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d880075e0 con 0x7f1da00b2720 2024-05-08T22:41:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.811+0000 7f1d9cff9700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 0x7f1d80068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.811+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f1d88083780 con 0x7f1da00b2720 2024-05-08T22:41:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.811+0000 7f1d9ffff700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 0x7f1d80068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:30.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.811+0000 7f1d9ffff700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 0x7f1d80068c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f80 tx=0x7f1d90011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:30.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.815+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1d880508d0 con 0x7f1da00b2720 2024-05-08T22:41:30.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1da006d460 con 0x7f1da00b2720 2024-05-08T22:41:30.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1d9cff9700 1 -- 172.21.15.179:0/1148660580 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d88053ef0 con 0x7f1da00b2720 2024-05-08T22:41:30.991 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:41:30.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 msgr2=0x7f1d80068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 0x7f1d80068c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f80 tx=0x7f1d90011000 comp rx=0 tx=0).stop 2024-05-08T22:41:30.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 msgr2=0x7f1da015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:30.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c500 tx=0x7f1d8800c810 comp rx=0 tx=0).stop 2024-05-08T22:41:30.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 shutdown_connections 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1d80066760 0x7f1d80068c20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1da015d1f0 0x7f1da015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da00b2720 0x7f1da015abf0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 --2- 172.21.15.179:0/1148660580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1da00b1b30 0x7f1da01f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 >> 172.21.15.179:0/1148660580 conn(0x7f1da00cea30 msgr2=0x7f1da006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 shutdown_connections 2024-05-08T22:41:30.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:30.987+0000 7f1da562c700 1 -- 172.21.15.179:0/1148660580 wait complete. 2024-05-08T22:41:31.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:41:31.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 Processor -- start 2024-05-08T22:41:31.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 -- start start 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff298159320 0x7ff2980cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff2980b19e0 0x7ff2980cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29806cda0 con 0x7ff298159700 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2980b1f10 con 0x7ff298159320 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff29cb1d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2980b21f0 con 0x7ff2980b19e0 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff2977fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff298159320 0x7ff2980cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff297fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ff298159700 0x7ff29806b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:55686/0 (socket says 172.21.15.179:55686) 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff297fff700 1 -- 172.21.15.179:0/3888887055 learned_addr learned my addr 172.21.15.179:0/3888887055 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff296ffd700 1 --2- 172.21.15.179:0/3888887055 >> v2:172.21.15.195:3300/0 conn(0x7ff2980b19e0 0x7ff2980cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.119+0000 7ff2977fe700 1 --2- 172.21.15.179:0/3888887055 >> v2:172.21.15.179:3300/0 conn(0x7ff298159320 0x7ff2980cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55618/0 (socket says 172.21.15.179:55618) 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2967fc700 1 -- 172.21.15.179:0/3888887055 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049949760 0 0) 0x7ff29806cda0 con 0x7ff298159700 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2967fc700 1 -- 172.21.15.179:0/3888887055 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2800035d0 con 0x7ff298159700 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2977fe700 1 -- 172.21.15.179:0/3888887055 >> v2:172.21.15.195:3300/0 conn(0x7ff2980b19e0 msgr2=0x7ff2980cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2977fe700 1 --2- 172.21.15.179:0/3888887055 >> v2:172.21.15.195:3300/0 conn(0x7ff2980b19e0 0x7ff2980cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2977fe700 1 -- 172.21.15.179:0/3888887055 >> v1:172.21.15.195:6789/0 conn(0x7ff298159700 legacy=0x7ff29806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2977fe700 1 -- 172.21.15.179:0/3888887055 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980b24d0 con 0x7ff298159320 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2977fe700 1 --2- 172.21.15.179:0/3888887055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff298159320 0x7ff2980cadb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff29806ef00 tx=0x7ff28c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc328982430862f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2967fc700 1 -- 172.21.15.179:0/3888887055 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011830 con 0x7ff298159320 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2967fc700 1 -- 172.21.15.179:0/3888887055 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c011990 con 0x7ff298159320 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff2967fc700 1 -- 172.21.15.179:0/3888887055 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011d20 con 0x7ff298159320 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- 172.21.15.179:0/3888887055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff298159320 msgr2=0x7ff2980cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/3888887055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff298159320 0x7ff2980cadb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff29806ef00 tx=0x7ff28c009b30 comp rx=0 tx=0).stop 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- 172.21.15.179:0/3888887055 shutdown_connections 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/3888887055 >> v2:172.21.15.195:3300/0 conn(0x7ff2980b19e0 0x7ff2980cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/3888887055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff298159320 0x7ff2980cadb0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- 172.21.15.179:0/3888887055 >> 172.21.15.179:0/3888887055 conn(0x7ff2980cea30 msgr2=0x7ff2980cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- 172.21.15.179:0/3888887055 shutdown_connections 2024-05-08T22:41:31.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- 172.21.15.179:0/3888887055 wait complete. 2024-05-08T22:41:31.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 Processor -- start 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.123+0000 7ff29cb1d700 1 -- start start 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298159320 0x7ff2981633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 0x7ff298163900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298144a80 con 0x7ff2980b19e0 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298140560 con 0x7ff298159700 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff298146fa0 con 0x7ff298159320 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55624/0 (socket says 172.21.15.179:55624) 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 -- 172.21.15.179:0/276008195 learned_addr learned my addr 172.21.15.179:0/276008195 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff296ffd700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298159320 0x7ff2981633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff297fff700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 0x7ff298163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 -- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298159320 msgr2=0x7ff2981633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298159320 0x7ff2981633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 -- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 msgr2=0x7ff298163900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 0x7ff298163900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 -- 172.21.15.179:0/276008195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0094b0 con 0x7ff2980b19e0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff297fff700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 0x7ff298163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff2977fe700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009b70 tx=0x7ff28c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004480 con 0x7ff2980b19e0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c01f070 con 0x7ff2980b19e0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0038c0 con 0x7ff2980b19e0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29815a760 con 0x7ff2980b19e0 2024-05-08T22:41:31.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29815ad20 con 0x7ff2980b19e0 2024-05-08T22:41:31.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29804e700 con 0x7ff2980b19e0 2024-05-08T22:41:31.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.127+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff28c021030 con 0x7ff2980b19e0 2024-05-08T22:41:31.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.131+0000 7ff27ffff700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 0x7ff284068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.131+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7ff28c085690 con 0x7ff2980b19e0 2024-05-08T22:41:31.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.131+0000 7ff296ffd700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 0x7ff284068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.131+0000 7ff296ffd700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 0x7ff284068c20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff29815cc10 tx=0x7ff288008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.135+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff28c052880 con 0x7ff2980b19e0 2024-05-08T22:41:31.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.303+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff29806dc00 con 0x7ff2980b19e0 2024-05-08T22:41:31.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.303+0000 7ff27ffff700 1 -- 172.21.15.179:0/276008195 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff28c055ea0 con 0x7ff2980b19e0 2024-05-08T22:41:31.309 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:41:31.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 msgr2=0x7ff284068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 0x7ff284068c20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff29815cc10 tx=0x7ff288008040 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 msgr2=0x7ff298140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009b70 tx=0x7ff28c004770 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 shutdown_connections 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff284066760 0x7ff284068c20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff298159700 0x7ff298163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298159320 0x7ff2981633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 --2- 172.21.15.179:0/276008195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2980b19e0 0x7ff298140020 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 >> 172.21.15.179:0/276008195 conn(0x7ff2980cea30 msgr2=0x7ff29806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 shutdown_connections 2024-05-08T22:41:31.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.307+0000 7ff29cb1d700 1 -- 172.21.15.179:0/276008195 wait complete. 2024-05-08T22:41:31.323 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:41:31.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.439+0000 7fd0b2b88700 1 Processor -- start 2024-05-08T22:41:31.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.439+0000 7fd0b2b88700 1 -- start start 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0ac15db00 0x7fd0ac06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 0x7fd0ac0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac06b110 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b1d30 con 0x7fd0ac15db00 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b19e0 con 0x7fd0ac15dee0 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0ac15db00 0x7fd0ac06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0ac15db00 0x7fd0ac06cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55636/0 (socket says 172.21.15.179:55636) 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3939827957 learned_addr learned my addr 172.21.15.179:0/3939827957 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 0x7fd0ac0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3939827957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 msgr2=0x7fd0ac0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 0x7fd0ac0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3939827957 >> v1:172.21.15.195:6789/0 conn(0x7fd0ac0cd3c0 legacy=0x7fd0ac0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3939827957 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac157cf0 con 0x7fd0ac15db00 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 0x7fd0ac0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b1b86700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15db00 0x7fd0ac06cda0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009910 tx=0x7fd0a0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6771eec3360704 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b0b84700 1 -- 172.21.15.179:0/3939827957 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a000ba60 con 0x7fd0ac15db00 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b0b84700 1 -- 172.21.15.179:0/3939827957 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a000bbc0 con 0x7fd0ac15db00 2024-05-08T22:41:31.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b0b84700 1 -- 172.21.15.179:0/3939827957 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0009e60 con 0x7fd0ac15db00 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3939827957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15db00 msgr2=0x7fd0ac06cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15db00 0x7fd0ac06cda0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009910 tx=0x7fd0a0009c20 comp rx=0 tx=0).stop 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3939827957 shutdown_connections 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15dee0 0x7fd0ac0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3939827957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac15db00 0x7fd0ac06cda0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3939827957 >> 172.21.15.179:0/3939827957 conn(0x7fd0ac0cea30 msgr2=0x7fd0ac0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3939827957 shutdown_connections 2024-05-08T22:41:31.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.443+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3939827957 wait complete. 2024-05-08T22:41:31.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 Processor -- start 2024-05-08T22:41:31.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- start start 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 0x7fd0ac166410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 0x7fd0ac1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac166df0 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1670d0 con 0x7fd0ac15dee0 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1673b0 con 0x7fd0ac15db00 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55650/0 (socket says 172.21.15.179:55650) 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3859332938 learned_addr learned my addr 172.21.15.179:0/3859332938 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 0x7fd0ac166410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 msgr2=0x7fd0ac166410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2387700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 0x7fd0ac1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 0x7fd0ac166410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 msgr2=0x7fd0ac1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 0x7fd0ac1507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 -- 172.21.15.179:0/3859332938 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0009570 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 0x7fd0ac166410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2387700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 0x7fd0ac1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b1b86700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06ef30 tx=0x7fd0a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004610 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a001f070 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac167690 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0003900 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.447+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac15a210 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac167bb0 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0a0003a60 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd09e7fc700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fd0a0021070 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.451+0000 7fd0b1385700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009980 tx=0x7fd0a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.459+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd0ac167bb0 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.631+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0ac167bb0 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.631+0000 7fd09e7fc700 1 -- 172.21.15.179:0/3859332938 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0ac167bb0 con 0x7fd0ac0cd3c0 2024-05-08T22:41:31.637 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:41:31.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 msgr2=0x7fd090068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009980 tx=0x7fd0a8008040 comp rx=0 tx=0).stop 2024-05-08T22:41:31.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 msgr2=0x7fd0ac159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06ef30 tx=0x7fd0a0004770 comp rx=0 tx=0).stop 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 shutdown_connections 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd090066730 0x7fd090068bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0ac15dee0 0x7fd0ac1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0ac15db00 0x7fd0ac166410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 --2- 172.21.15.179:0/3859332938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0ac0cd3c0 0x7fd0ac159cd0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 >> 172.21.15.179:0/3859332938 conn(0x7fd0ac0cea30 msgr2=0x7fd0ac06b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 shutdown_connections 2024-05-08T22:41:31.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.635+0000 7fd0b2b88700 1 -- 172.21.15.179:0/3859332938 wait complete. 2024-05-08T22:41:31.651 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:41:31.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 Processor -- start 2024-05-08T22:41:31.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- start start 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3ec06dda0 0x7fa3ec0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3ec06e180 0x7fa3ec06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec06b540 con 0x7fa3ec0b25d0 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b3110 con 0x7fa3ec06e180 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec06ac50 con 0x7fa3ec06dda0 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3ec06e180 0x7fa3ec06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3ec06e180 0x7fa3ec06e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55652/0 (socket says 172.21.15.179:55652) 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f0d0c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa3ec0b25d0 0x7fa3ec0b2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37794/0 (socket says 172.21.15.179:37794) 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/1870556910 learned_addr learned my addr 172.21.15.179:0/1870556910 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3ebfff700 1 --2- 172.21.15.179:0/1870556910 >> v2:172.21.15.195:3300/0 conn(0x7fa3ec06dda0 0x7fa3ec0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/1870556910 >> v2:172.21.15.195:3300/0 conn(0x7fa3ec06dda0 msgr2=0x7fa3ec0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 --2- 172.21.15.179:0/1870556910 >> v2:172.21.15.195:3300/0 conn(0x7fa3ec06dda0 0x7fa3ec0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/1870556910 >> v1:172.21.15.179:6789/0 conn(0x7fa3ec0b25d0 legacy=0x7fa3ec0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/1870556910 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec06aef0 con 0x7fa3ec06e180 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eb7fe700 1 --2- 172.21.15.179:0/1870556910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec06e5e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec072000 tx=0x7fa3dc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f110006adee3a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eaffd700 1 -- 172.21.15.179:0/1870556910 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00b8a0 con 0x7fa3ec06e180 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eaffd700 1 -- 172.21.15.179:0/1870556910 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc00ba00 con 0x7fa3ec06e180 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3eaffd700 1 -- 172.21.15.179:0/1870556910 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00bd90 con 0x7fa3ec06e180 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- 172.21.15.179:0/1870556910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 msgr2=0x7fa3ec06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 --2- 172.21.15.179:0/1870556910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec06e5e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec072000 tx=0x7fa3dc009bf0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- 172.21.15.179:0/1870556910 shutdown_connections 2024-05-08T22:41:31.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 --2- 172.21.15.179:0/1870556910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec06e5e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 --2- 172.21.15.179:0/1870556910 >> v2:172.21.15.195:3300/0 conn(0x7fa3ec06dda0 0x7fa3ec0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- 172.21.15.179:0/1870556910 >> 172.21.15.179:0/1870556910 conn(0x7fa3ec0ce960 msgr2=0x7fa3ec0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- 172.21.15.179:0/1870556910 shutdown_connections 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.759+0000 7fa3f150d700 1 -- 172.21.15.179:0/1870556910 wait complete. 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 Processor -- start 2024-05-08T22:41:31.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 -- start start 2024-05-08T22:41:31.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3ec06dda0 0x7fa3ec159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3ec0b25d0 0x7fa3ec15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec15a190 con 0x7fa3ec06e180 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec1582f0 con 0x7fa3ec0b25d0 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f150d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0a7570 con 0x7fa3ec06dda0 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55654/0 (socket says 172.21.15.179:55654) 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/2857475983 learned_addr learned my addr 172.21.15.179:0/2857475983 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3ebfff700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3ec06dda0 0x7fa3ec159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3f0d0c700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3ec0b25d0 0x7fa3ec15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3ec06dda0 msgr2=0x7fa3ec159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3ec06dda0 0x7fa3ec159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3ec0b25d0 msgr2=0x7fa3ec15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3ec0b25d0 0x7fa3ec15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 -- 172.21.15.179:0/2857475983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc009570 con 0x7fa3ec06e180 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.763+0000 7fa3eb7fe700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0cf5e0 tx=0x7fa3dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0036c0 con 0x7fa3ec06e180 2024-05-08T22:41:31.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc01f070 con 0x7fa3ec06e180 2024-05-08T22:41:31.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc003a80 con 0x7fa3ec06e180 2024-05-08T22:41:31.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec0a7850 con 0x7fa3ec06e180 2024-05-08T22:41:31.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec0a7e10 con 0x7fa3ec06e180 2024-05-08T22:41:31.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec04e700 con 0x7fa3ec06e180 2024-05-08T22:41:31.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.767+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3dc021030 con 0x7fa3ec06e180 2024-05-08T22:41:31.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.771+0000 7fa3e8ff9700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 0x7fa3d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:31.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.771+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa3dc085960 con 0x7fa3ec06e180 2024-05-08T22:41:31.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.771+0000 7fa3ebfff700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 0x7fa3d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:31.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.771+0000 7fa3ebfff700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 0x7fa3d8068c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009a00 tx=0x7fa3e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:31.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.775+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa3dc052b50 con 0x7fa3ec06e180 2024-05-08T22:41:31.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.959+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3ec15c630 con 0x7fa3ec06e180 2024-05-08T22:41:31.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.959+0000 7fa3e8ff9700 1 -- 172.21.15.179:0/2857475983 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v99) v1 ==== 80+0+124 (secure 0 0 0) 0x7fa3dc056170 con 0x7fa3ec06e180 2024-05-08T22:41:31.965 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6835/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:41:31.966 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6834/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 msgr2=0x7fa3d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 0x7fa3d8068c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009a00 tx=0x7fa3e0008040 comp rx=0 tx=0).stop 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 msgr2=0x7fa3ec157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0cf5e0 tx=0x7fa3dc004770 comp rx=0 tx=0).stop 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 shutdown_connections 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3d8066760 0x7fa3d8068c20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3ec0b25d0 0x7fa3ec15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3ec06e180 0x7fa3ec157db0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 --2- 172.21.15.179:0/2857475983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3ec06dda0 0x7fa3ec159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 >> 172.21.15.179:0/2857475983 conn(0x7fa3ec0ce960 msgr2=0x7fa3ec0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 shutdown_connections 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:31.963+0000 7fa3f150d700 1 -- 172.21.15.179:0/2857475983 wait complete. 2024-05-08T22:41:31.969 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:41:31.980 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:41:31.980 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:41:31.989 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:41:32.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 Processor -- start 2024-05-08T22:41:32.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 -- start start 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a600add40 0x7f3a6006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a60161ac0 con 0x7f3a600cc310 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a600cdaa0 con 0x7f3a600ce060 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a680b8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600ae550 con 0x7f3a600add40 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65653700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3a600cc310 0x7f3a6006b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37798/0 (socket says 172.21.15.179:37798) 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65653700 1 -- 172.21.15.179:0/3497778609 learned_addr learned my addr 172.21.15.179:0/3497778609 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a66655700 1 --2- 172.21.15.179:0/3497778609 >> v2:172.21.15.195:3300/0 conn(0x7f3a600add40 0x7f3a6006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848133730 0 0) 0x7f3a60161ac0 con 0x7f3a600cc310 2024-05-08T22:41:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a4c0035d0 con 0x7f3a600cc310 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4007796994 0 0) 0x7f3a4c0035d0 con 0x7f3a600cc310 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 >> v2:172.21.15.195:3300/0 conn(0x7f3a600add40 msgr2=0x7f3a6006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 --2- 172.21.15.179:0/3497778609 >> v2:172.21.15.195:3300/0 conn(0x7f3a600add40 0x7f3a6006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 >> v1:172.21.15.195:6789/0 conn(0x7f3a600ce060 legacy=0x7f3a600ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6006adc0 con 0x7f3a600cc310 2024-05-08T22:41:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f3a580019f0 con 0x7f3a600cc310 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a66655700 1 --2- 172.21.15.179:0/3497778609 >> v2:172.21.15.195:3300/0 conn(0x7f3a600add40 0x7f3a6006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 >> v1:172.21.15.179:6789/0 conn(0x7f3a600cc310 legacy=0x7f3a6006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a4c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c004870 0x7f3a4c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c005230 0x7f3a4c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600ae550 con 0x7f3a4c003a20 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c00c0d0 con 0x7f3a4c004870 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c00c290 con 0x7f3a4c005230 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a4c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a66655700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c005230 0x7f3a4c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65653700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c004870 0x7f3a4c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 -- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c005230 msgr2=0x7f3a4c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:32.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c005230 0x7f3a4c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 -- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c004870 msgr2=0x7f3a4c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c004870 0x7f3a4c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a66655700 1 -- 172.21.15.179:0/3497778609 reap_dead start 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.099+0000 7f3a65e54700 1 -- 172.21.15.179:0/3497778609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0035d0 con 0x7f3a4c003a20 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a65e54700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a4c003e40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a500017e0 tx=0x7f3a4c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74e4f33ef440c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500052d0 con 0x7f3a4c003a20 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50005430 con 0x7f3a4c003a20 2024-05-08T22:41:32.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a64e52700 1 -- 172.21.15.179:0/3497778609 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500055a0 con 0x7f3a4c003a20 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 -- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 msgr2=0x7f3a4c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a4c003e40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a500017e0 tx=0x7f3a4c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 -- 172.21.15.179:0/3497778609 shutdown_connections 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 --2- 172.21.15.179:0/3497778609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a4c003e40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 -- 172.21.15.179:0/3497778609 >> 172.21.15.179:0/3497778609 conn(0x7f3a600cb660 msgr2=0x7f3a600cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 -- 172.21.15.179:0/3497778609 shutdown_connections 2024-05-08T22:41:32.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.103+0000 7f3a680b8700 1 -- 172.21.15.179:0/3497778609 wait complete. 2024-05-08T22:41:32.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 Processor -- start 2024-05-08T22:41:32.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- start start 2024-05-08T22:41:32.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a600b2fc0 0x7f3a600c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a60155a00 0x7f3a600b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600c3f90 con 0x7f3a4c003a20 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600b1f20 con 0x7f3a60155a00 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600b2200 con 0x7f3a600b2fc0 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55684/0 (socket says 172.21.15.179:55684) 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 -- 172.21.15.179:0/2763184862 learned_addr learned my addr 172.21.15.179:0/2763184862 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65653700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a600b2fc0 0x7f3a600c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a66655700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a60155a00 0x7f3a600b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 -- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a600b2fc0 msgr2=0x7f3a600c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a600b2fc0 0x7f3a600c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 -- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a60155a00 msgr2=0x7f3a600b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a60155a00 0x7f3a600b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 -- 172.21.15.179:0/2763184862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600b24e0 con 0x7f3a4c003a20 2024-05-08T22:41:32.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a65e54700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005040 tx=0x7f3a50001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500055a0 con 0x7f3a4c003a20 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50001290 con 0x7f3a4c003a20 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a600b27c0 con 0x7f3a4c003a20 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007260 con 0x7f3a4c003a20 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.107+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60155e80 con 0x7f3a4c003a20 2024-05-08T22:41:32.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.111+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60067770 con 0x7f3a4c003a20 2024-05-08T22:41:32.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.111+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a500073c0 con 0x7f3a4c003a20 2024-05-08T22:41:32.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.111+0000 7f3a567fc700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 0x7f3a44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.111+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f3a50084c60 con 0x7f3a4c003a20 2024-05-08T22:41:32.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.111+0000 7f3a65653700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 0x7f3a44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.115+0000 7f3a65653700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 0x7f3a44068bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3a6015ef80 tx=0x7f3a58006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.119+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3a50051f20 con 0x7f3a4c003a20 2024-05-08T22:41:32.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.295+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a60073290 con 0x7f3a4c003a20 2024-05-08T22:41:32.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.299+0000 7f3a567fc700 1 -- 172.21.15.179:0/2763184862 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16622 (secure 0 0 0) 0x7f3a50055540 con 0x7f3a4c003a20 2024-05-08T22:41:32.305 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:32.305 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":99,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:40:19.823666+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 msgr2=0x7f3a44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 0x7f3a44068bf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3a6015ef80 tx=0x7f3a58006040 comp rx=0 tx=0).stop 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 msgr2=0x7f3a600b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005040 tx=0x7f3a50001120 comp rx=0 tx=0).stop 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 shutdown_connections 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a44066730 0x7f3a44068bf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a60155a00 0x7f3a600b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a600b2fc0 0x7f3a600c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 --2- 172.21.15.179:0/2763184862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c003a20 0x7f3a600b1840 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 >> 172.21.15.179:0/2763184862 conn(0x7f3a600cb660 msgr2=0x7f3a600ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 shutdown_connections 2024-05-08T22:41:32.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.303+0000 7f3a680b8700 1 -- 172.21.15.179:0/2763184862 wait complete. 2024-05-08T22:41:32.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:41:32.319 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:41:32.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 Processor -- start 2024-05-08T22:41:32.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 -- start start 2024-05-08T22:41:32.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 0x7fd5c015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 0x7fd5c00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00cb370 con 0x7fd5c0158850 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c015f350 con 0x7fd5c015eec0 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c6f36700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c006e5c0 con 0x7fd5c006e090 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c5f34700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd5c0158850 0x7fd5c00ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37808/0 (socket says 172.21.15.179:37808) 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.427+0000 7fd5c5f34700 1 -- 172.21.15.179:0/3469895846 learned_addr learned my addr 172.21.15.179:0/3469895846 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6735700 1 --2- 172.21.15.179:0/3469895846 >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 0x7fd5c00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 --2- 172.21.15.179:0/3469895846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 0x7fd5c015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479618608 0 0) 0x7fd5c00cb370 con 0x7fd5c0158850 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a80035d0 con 0x7fd5c0158850 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 -- 172.21.15.179:0/3469895846 >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 msgr2=0x7fd5c00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:32.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 --2- 172.21.15.179:0/3469895846 >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 0x7fd5c00cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 -- 172.21.15.179:0/3469895846 >> v1:172.21.15.179:6789/0 conn(0x7fd5c0158850 legacy=0x7fd5c00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2941632639 0 0) 0x7fd5a80035d0 con 0x7fd5c0158850 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 -- 172.21.15.179:0/3469895846 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c006e8a0 con 0x7fd5c006e090 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6735700 1 --2- 172.21.15.179:0/3469895846 >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 0x7fd5c00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c5733700 1 --2- 172.21.15.179:0/3469895846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 0x7fd5c015e980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009d90 tx=0x7fd5b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b5ff987c67458e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000e070 con 0x7fd5c006e090 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000bb20 con 0x7fd5c006e090 2024-05-08T22:41:32.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c4f32700 1 -- 172.21.15.179:0/3469895846 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000be90 con 0x7fd5c006e090 2024-05-08T22:41:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3469895846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 msgr2=0x7fd5c015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3469895846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 0x7fd5c015e980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009d90 tx=0x7fd5b000b330 comp rx=0 tx=0).stop 2024-05-08T22:41:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3469895846 shutdown_connections 2024-05-08T22:41:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3469895846 >> v2:172.21.15.195:3300/0 conn(0x7fd5c015eec0 0x7fd5c00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3469895846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c006e090 0x7fd5c015e980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3469895846 >> 172.21.15.179:0/3469895846 conn(0x7fd5c00cea30 msgr2=0x7fd5c00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3469895846 shutdown_connections 2024-05-08T22:41:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.431+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3469895846 wait complete. 2024-05-08T22:41:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 Processor -- start 2024-05-08T22:41:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- start start 2024-05-08T22:41:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5c0158850 0x7fd5c015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 0x7fd5c00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0138770 con 0x7fd5c006e090 2024-05-08T22:41:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00abdc0 con 0x7fd5c0158850 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00ac0a0 con 0x7fd5c015eec0 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c5f34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 0x7fd5c00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c5f34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55694/0 (socket says 172.21.15.179:55694) 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c5733700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5c0158850 0x7fd5c015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c5f34700 1 -- 172.21.15.179:0/3077819587 learned_addr learned my addr 172.21.15.179:0/3077819587 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 -- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5c0158850 msgr2=0x7fd5c015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5c0158850 0x7fd5c015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 -- 172.21.15.179:0/3077819587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 msgr2=0x7fd5c0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 -- 172.21.15.179:0/3077819587 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b80094b0 con 0x7fd5c015eec0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c5f34700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6735700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 0x7fd5c00cbed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e930 tx=0x7fd5b800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004020 con 0x7fd5c015eec0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b80041a0 con 0x7fd5c015eec0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b0009a70 con 0x7fd5c015eec0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004550 con 0x7fd5c015eec0 2024-05-08T22:41:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.435+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c00ac660 con 0x7fd5c015eec0 2024-05-08T22:41:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c004e700 con 0x7fd5c015eec0 2024-05-08T22:41:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5b80036c0 con 0x7fd5c015eec0 2024-05-08T22:41:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5b67fc700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 0x7fd5ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fd5b8083f70 con 0x7fd5c015eec0 2024-05-08T22:41:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5c5f34700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 0x7fd5ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.439+0000 7fd5c5f34700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 0x7fd5ac068bf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc001d90 tx=0x7fd5bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.451+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd5b80b5cf0 con 0x7fd5c015eec0 2024-05-08T22:41:32.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.619+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5c00ce810 con 0x7fd5c015eec0 2024-05-08T22:41:32.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.619+0000 7fd5b67fc700 1 -- 172.21.15.179:0/3077819587 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5b80510e0 con 0x7fd5c015eec0 2024-05-08T22:41:32.626 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:41:32.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 msgr2=0x7fd5ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 0x7fd5ac068bf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc001d90 tx=0x7fd5bc009040 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 msgr2=0x7fd5c00cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 0x7fd5c00cbed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e930 tx=0x7fd5b800ec40 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 shutdown_connections 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd5ac066730 0x7fd5ac068bf0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5c0158850 0x7fd5c015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5c015eec0 0x7fd5c00cbed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 --2- 172.21.15.179:0/3077819587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5c006e090 0x7fd5c0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 >> 172.21.15.179:0/3077819587 conn(0x7fd5c00cea30 msgr2=0x7fd5c006b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 shutdown_connections 2024-05-08T22:41:32.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.623+0000 7fd5c6f36700 1 -- 172.21.15.179:0/3077819587 wait complete. 2024-05-08T22:41:32.641 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:41:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.759+0000 7f08f8e84700 1 Processor -- start 2024-05-08T22:41:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.759+0000 7f08f8e84700 1 -- start start 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 0x7f08f40b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 0x7f08f406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f40b22d0 con 0x7f08f4142fd0 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40b19e0 con 0x7f08f4142890 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415e640 con 0x7f08f40b2ca0 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f3fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f08f4142fd0 0x7f08f415d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37814/0 (socket says 172.21.15.179:37814) 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f3fff700 1 -- 172.21.15.179:0/431478549 learned_addr learned my addr 172.21.15.179:0/431478549 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f37fe700 1 --2- 172.21.15.179:0/431478549 >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 0x7f08f40b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/431478549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 0x7f08f406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396578845 0 0) 0x7f08f40b22d0 con 0x7f08f4142fd0 2024-05-08T22:41:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc0035d0 con 0x7f08f4142fd0 2024-05-08T22:41:32.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 -- 172.21.15.179:0/431478549 >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 msgr2=0x7f08f40b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:32.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/431478549 >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 0x7f08f40b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173981195 0 0) 0x7f08dc0035d0 con 0x7f08f4142fd0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 -- 172.21.15.179:0/431478549 >> v1:172.21.15.179:6789/0 conn(0x7f08f4142fd0 legacy=0x7f08f415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 -- 172.21.15.179:0/431478549 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40b1c80 con 0x7f08f40b2ca0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f37fe700 1 --2- 172.21.15.179:0/431478549 >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 0x7f08f40b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/431478549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 0x7f08f406cda0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009850 tx=0x7f08e4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d13ddb6923421f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0019f0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4004030 con 0x7f08f40b2ca0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f27fc700 1 -- 172.21.15.179:0/431478549 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40043e0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- 172.21.15.179:0/431478549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 msgr2=0x7f08f406cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 --2- 172.21.15.179:0/431478549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 0x7f08f406cda0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009850 tx=0x7f08e4009b60 comp rx=0 tx=0).stop 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- 172.21.15.179:0/431478549 shutdown_connections 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 --2- 172.21.15.179:0/431478549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f40b2ca0 0x7f08f406cda0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 --2- 172.21.15.179:0/431478549 >> v2:172.21.15.195:3300/0 conn(0x7f08f4142890 0x7f08f40b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- 172.21.15.179:0/431478549 >> 172.21.15.179:0/431478549 conn(0x7f08f40cea30 msgr2=0x7f08f40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- 172.21.15.179:0/431478549 shutdown_connections 2024-05-08T22:41:32.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.763+0000 7f08f8e84700 1 -- 172.21.15.179:0/431478549 wait complete. 2024-05-08T22:41:32.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 Processor -- start 2024-05-08T22:41:32.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- start start 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f4142890 0x7f08f4133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f4142fd0 0x7f08f4134320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f41f2a40 con 0x7f08f40b2ca0 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f406c0c0 con 0x7f08f4142fd0 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40aba10 con 0x7f08f4142890 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55704/0 (socket says 172.21.15.179:55704) 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 -- 172.21.15.179:0/2656313864 learned_addr learned my addr 172.21.15.179:0/2656313864 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f3fff700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f4142fd0 0x7f08f4134320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f4142890 0x7f08f4133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 -- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f4142890 msgr2=0x7f08f4133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f4142890 0x7f08f4133de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 -- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f4142fd0 msgr2=0x7f08f4134320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f4142fd0 0x7f08f4134320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 -- 172.21.15.179:0/2656313864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80094b0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f37fe700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f08e800ebe0 tx=0x7f08e800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800c850 con 0x7f08f40b2ca0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800c9b0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800cd40 con 0x7f08f40b2ca0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40094b0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.767+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40abfd0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.771+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404e700 con 0x7f08f40b2ca0 2024-05-08T22:41:32.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.771+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08e8013470 con 0x7f08f40b2ca0 2024-05-08T22:41:32.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.771+0000 7f08dbfff700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:32.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.771+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f08e80843c0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.771+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:32.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.775+0000 7f08f2ffd700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 0x7f08e0068bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f08f406b410 tx=0x7f08e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:32.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.783+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f08e80515b0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.947+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08f40aeb80 con 0x7f08f40b2ca0 2024-05-08T22:41:32.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.947+0000 7f08dbfff700 1 -- 172.21.15.179:0/2656313864 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08e800cea0 con 0x7f08f40b2ca0 2024-05-08T22:41:32.952 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 msgr2=0x7f08e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 0x7f08e0068bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f08f406b410 tx=0x7f08e4008030 comp rx=0 tx=0).stop 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 msgr2=0x7f08f4158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f08e800ebe0 tx=0x7f08e800eef0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 shutdown_connections 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08e0066730 0x7f08e0068bf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08f4142fd0 0x7f08f4134320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08f4142890 0x7f08f4133de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 --2- 172.21.15.179:0/2656313864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08f40b2ca0 0x7f08f4158dd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:32.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 >> 172.21.15.179:0/2656313864 conn(0x7f08f40cea30 msgr2=0x7f08f406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:32.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 shutdown_connections 2024-05-08T22:41:32.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:32.951+0000 7f08f8e84700 1 -- 172.21.15.179:0/2656313864 wait complete. 2024-05-08T22:41:32.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:41:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 Processor -- start 2024-05-08T22:41:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 -- start start 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57a80b8a80 0x7f57a80cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80b8ee0 con 0x7f57a815d0c0 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a815d4a0 con 0x7f57a80ce0e0 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57aef50700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80ce570 con 0x7f57a80b8a80 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57ad74d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57a80b8a80 0x7f57a80cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57adf4e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f57a815d0c0 0x7f57a80d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37830/0 (socket says 172.21.15.179:37830) 2024-05-08T22:41:33.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.087+0000 7f57adf4e700 1 -- 172.21.15.179:0/2825540803 learned_addr learned my addr 172.21.15.179:0/2825540803 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57ad74d700 1 -- 172.21.15.179:0/2825540803 >> v1:172.21.15.195:6789/0 conn(0x7f57a80ce0e0 legacy=0x7f57a806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57ad74d700 1 -- 172.21.15.179:0/2825540803 >> v1:172.21.15.179:6789/0 conn(0x7f57a815d0c0 legacy=0x7f57a80d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57ad74d700 1 -- 172.21.15.179:0/2825540803 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a806abb0 con 0x7f57a80b8a80 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57acf4c700 1 -- 172.21.15.179:0/2825540803 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321865390 0 0) 0x7f57a80b8ee0 con 0x7f57a815d0c0 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57ad74d700 1 --2- 172.21.15.179:0/2825540803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a80cdba0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009850 tx=0x7f57a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e1b6ebc1dbb251 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57acf4c700 1 -- 172.21.15.179:0/2825540803 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4011830 con 0x7f57a80b8a80 2024-05-08T22:41:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57acf4c700 1 -- 172.21.15.179:0/2825540803 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4011990 con 0x7f57a80b8a80 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57acf4c700 1 -- 172.21.15.179:0/2825540803 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4011d20 con 0x7f57a80b8a80 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 -- 172.21.15.179:0/2825540803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 msgr2=0x7f57a80cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 --2- 172.21.15.179:0/2825540803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a80cdba0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009850 tx=0x7f57a4009b60 comp rx=0 tx=0).stop 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 -- 172.21.15.179:0/2825540803 shutdown_connections 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 --2- 172.21.15.179:0/2825540803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a80cdba0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 -- 172.21.15.179:0/2825540803 >> 172.21.15.179:0/2825540803 conn(0x7f57a80d2ed0 msgr2=0x7f57a80d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 -- 172.21.15.179:0/2825540803 shutdown_connections 2024-05-08T22:41:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 -- 172.21.15.179:0/2825540803 wait complete. 2024-05-08T22:41:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.091+0000 7f57aef50700 1 Processor -- start 2024-05-08T22:41:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- start start 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80ce0e0 0x7f57a80c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a815d0c0 0x7f57a80c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8154cf0 con 0x7f57a80b8a80 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81540b0 con 0x7f57a80ce0e0 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8154350 con 0x7f57a815d0c0 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55720/0 (socket says 172.21.15.179:55720) 2024-05-08T22:41:33.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 -- 172.21.15.179:0/2727249879 learned_addr learned my addr 172.21.15.179:0/2727249879 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57ad74d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80ce0e0 0x7f57a80c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57ae74f700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a815d0c0 0x7f57a80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 -- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a815d0c0 msgr2=0x7f57a80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a815d0c0 0x7f57a80c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 -- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80ce0e0 msgr2=0x7f57a80c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80ce0e0 0x7f57a80c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 -- 172.21.15.179:0/2727249879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40094b0 con 0x7f57a80b8a80 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57adf4e700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f579800c4a0 tx=0x7f579800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798004ca0 con 0x7f57a80b8a80 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798004e00 con 0x7f57a80b8a80 2024-05-08T22:41:33.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8154630 con 0x7f57a80b8a80 2024-05-08T22:41:33.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798007310 con 0x7f57a80b8a80 2024-05-08T22:41:33.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.095+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8144590 con 0x7f57a80b8a80 2024-05-08T22:41:33.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a806daa0 con 0x7f57a80b8a80 2024-05-08T22:41:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5798007470 con 0x7f57a80b8a80 2024-05-08T22:41:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f579e7fc700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 0x7f578c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f5798083f10 con 0x7f57a80b8a80 2024-05-08T22:41:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f57ad74d700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 0x7f578c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.099+0000 7f57ad74d700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 0x7f578c068bf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.107+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5798051080 con 0x7f57a80b8a80 2024-05-08T22:41:33.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.275+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57a804e700 con 0x7f57a80b8a80 2024-05-08T22:41:33.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.275+0000 7f579e7fc700 1 -- 172.21.15.179:0/2727249879 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5798007720 con 0x7f57a80b8a80 2024-05-08T22:41:33.281 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 msgr2=0x7f578c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 0x7f578c068bf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4008030 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 msgr2=0x7f57a8144050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f579800c4a0 tx=0x7f579800c7b0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 shutdown_connections 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f578c066730 0x7f578c068bf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80ce0e0 0x7f57a80c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a815d0c0 0x7f57a80c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 --2- 172.21.15.179:0/2727249879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57a80b8a80 0x7f57a8144050 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 >> 172.21.15.179:0/2727249879 conn(0x7f57a80d2ed0 msgr2=0x7f57a806bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:33.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 shutdown_connections 2024-05-08T22:41:33.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.279+0000 7f57aef50700 1 -- 172.21.15.179:0/2727249879 wait complete. 2024-05-08T22:41:33.296 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:41:33.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 Processor -- start 2024-05-08T22:41:33.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- start start 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0cf00d1670 0x7f0cf00d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cf00b2510 0x7f0cf00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf00d20f0 con 0x7f0cf015d370 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00d2e40 con 0x7f0cf00b2510 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf006c4f0 con 0x7f0cf00d1670 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cf00b2510 0x7f0cf00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0cf00b2510 0x7f0cf00b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55736/0 (socket says 172.21.15.179:55736) 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 -- 172.21.15.179:0/4279937509 learned_addr learned my addr 172.21.15.179:0/4279937509 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:33.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cef7fe700 1 --2- 172.21.15.179:0/4279937509 >> v2:172.21.15.195:3300/0 conn(0x7f0cf00d1670 0x7f0cf00d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 -- 172.21.15.179:0/4279937509 >> v2:172.21.15.195:3300/0 conn(0x7f0cf00d1670 msgr2=0x7f0cf00d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 --2- 172.21.15.179:0/4279937509 >> v2:172.21.15.195:3300/0 conn(0x7f0cf00d1670 0x7f0cf00d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 -- 172.21.15.179:0/4279937509 >> v1:172.21.15.195:6789/0 conn(0x7f0cf015d370 legacy=0x7f0cf00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 -- 172.21.15.179:0/4279937509 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00d1b30 con 0x7f0cf00b2510 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf4806700 1 --2- 172.21.15.179:0/4279937509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf00b6300 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0cf008a7c0 tx=0x7f0ce800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4ac293f930ca23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0ceeffd700 1 -- 172.21.15.179:0/4279937509 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800e070 con 0x7f0cf00b2510 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0ceeffd700 1 -- 172.21.15.179:0/4279937509 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800b910 con 0x7f0cf00b2510 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0ceeffd700 1 -- 172.21.15.179:0/4279937509 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800bcc0 con 0x7f0cf00b2510 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- 172.21.15.179:0/4279937509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 msgr2=0x7f0cf00b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 --2- 172.21.15.179:0/4279937509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf00b6300 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0cf008a7c0 tx=0x7f0ce800b330 comp rx=0 tx=0).stop 2024-05-08T22:41:33.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- 172.21.15.179:0/4279937509 shutdown_connections 2024-05-08T22:41:33.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 --2- 172.21.15.179:0/4279937509 >> v2:172.21.15.195:3300/0 conn(0x7f0cf00d1670 0x7f0cf00d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 --2- 172.21.15.179:0/4279937509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf00b6300 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- 172.21.15.179:0/4279937509 >> 172.21.15.179:0/4279937509 conn(0x7f0cf00cfae0 msgr2=0x7f0cf00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:33.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.403+0000 7f0cf6269700 1 -- 172.21.15.179:0/4279937509 shutdown_connections 2024-05-08T22:41:33.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 -- 172.21.15.179:0/4279937509 wait complete. 2024-05-08T22:41:33.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 Processor -- start 2024-05-08T22:41:33.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 -- start start 2024-05-08T22:41:33.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cf00d1670 0x7f0cf01f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 0x7f0cf0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015eb60 con 0x7f0cf00b2510 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015ee40 con 0x7f0cf015d370 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cf6269700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015f120 con 0x7f0cf00d1670 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55750/0 (socket says 172.21.15.179:55750) 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0ceffff700 1 -- 172.21.15.179:0/848876789 learned_addr learned my addr 172.21.15.179:0/848876789 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.407+0000 7f0cef7fe700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cf00d1670 0x7f0cf01f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cf4806700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 0x7f0cf0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 -- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cf00d1670 msgr2=0x7f0cf01f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cf00d1670 0x7f0cf01f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 -- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 msgr2=0x7f0cf0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 0x7f0cf0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 -- 172.21.15.179:0/848876789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8009b30 con 0x7f0cf00b2510 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cf4806700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 0x7f0cf0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0ceffff700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c500 tx=0x7f0ce000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00052d0 con 0x7f0cf00b2510 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf015f400 con 0x7f0cf00b2510 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0005430 con 0x7f0cf00b2510 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00055a0 con 0x7f0cf00b2510 2024-05-08T22:41:33.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01508c0 con 0x7f0cf00b2510 2024-05-08T22:41:33.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf015f920 con 0x7f0cf00b2510 2024-05-08T22:41:33.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ce00036a0 con 0x7f0cf00b2510 2024-05-08T22:41:33.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 0x7f0cdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:33.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.411+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0ce0083690 con 0x7f0cf00b2510 2024-05-08T22:41:33.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.415+0000 7f0cef7fe700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 0x7f0cdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:33.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.415+0000 7f0cef7fe700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 0x7f0cdc068bf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00b6cf0 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:33.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.423+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0ce0050800 con 0x7f0cf00b2510 2024-05-08T22:41:33.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.607+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0cf0073290 con 0x7f0cf00b2510 2024-05-08T22:41:33.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.607+0000 7f0cecff9700 1 -- 172.21.15.179:0/848876789 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v99) v1 ==== 80+0+124 (secure 0 0 0) 0x7f0ce0053e20 con 0x7f0cf00b2510 2024-05-08T22:41:33.614 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6835/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:41:33.615 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6834/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 msgr2=0x7f0cdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 0x7f0cdc068bf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00b6cf0 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 msgr2=0x7f0cf015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c500 tx=0x7f0ce000c8c0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 shutdown_connections 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0cdc066730 0x7f0cdc068bf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cf015d370 0x7f0cf0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0cf00d1670 0x7f0cf01f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 --2- 172.21.15.179:0/848876789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf00b2510 0x7f0cf015e460 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:33.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 >> 172.21.15.179:0/848876789 conn(0x7f0cf00cfae0 msgr2=0x7f0cf00d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:33.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 shutdown_connections 2024-05-08T22:41:33.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:33.615+0000 7f0cf6269700 1 -- 172.21.15.179:0/848876789 wait complete. 2024-05-08T22:41:33.621 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:41:33.633 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:41:33.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:33.634 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:33.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[46654]: fuse finished with error 0 and tester_r 0 2024-05-08T22:41:33.649 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:41:39.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:41:39.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:39.751 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:41:39.760 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:41:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 Processor -- start 2024-05-08T22:41:39.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 -- start start 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1fc06ac90 0x7fa1fc0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa1fc0ca990 0x7fa1fc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc06eb20 con 0x7fa1fc15df60 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cae40 con 0x7fa1fc06ac90 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa203404700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc06df10 con 0x7fa1fc0ca990 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1fc06ac90 0x7fa1fc0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1fc06ac90 0x7fa1fc0ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59750/0 (socket says 172.21.15.179:59750) 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 -- 172.21.15.179:0/1351846885 learned_addr learned my addr 172.21.15.179:0/1351846885 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa2019a1700 1 --2- 172.21.15.179:0/1351846885 >> v2:172.21.15.195:3300/0 conn(0x7fa1fc0ca990 0x7fa1fc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 -- 172.21.15.179:0/1351846885 >> v2:172.21.15.195:3300/0 conn(0x7fa1fc0ca990 msgr2=0x7fa1fc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 --2- 172.21.15.179:0/1351846885 >> v2:172.21.15.195:3300/0 conn(0x7fa1fc0ca990 0x7fa1fc0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 -- 172.21.15.179:0/1351846885 >> v1:172.21.15.195:6789/0 conn(0x7fa1fc15df60 legacy=0x7fa1fc15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.923+0000 7fa20099f700 1 -- 172.21.15.179:0/1351846885 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc06e1f0 con 0x7fa1fc06ac90 2024-05-08T22:41:39.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa20099f700 1 --2- 172.21.15.179:0/1351846885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0ca450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0722a0 tx=0x7fa1ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea531b70433c43b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa1f3fff700 1 -- 172.21.15.179:0/1351846885 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00b8a0 con 0x7fa1fc06ac90 2024-05-08T22:41:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa1f3fff700 1 -- 172.21.15.179:0/1351846885 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00ba00 con 0x7fa1fc06ac90 2024-05-08T22:41:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa1f3fff700 1 -- 172.21.15.179:0/1351846885 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00bd90 con 0x7fa1fc06ac90 2024-05-08T22:41:39.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- 172.21.15.179:0/1351846885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc06ac90 msgr2=0x7fa1fc0ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 --2- 172.21.15.179:0/1351846885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0ca450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0722a0 tx=0x7fa1ec009bf0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- 172.21.15.179:0/1351846885 shutdown_connections 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 --2- 172.21.15.179:0/1351846885 >> v2:172.21.15.195:3300/0 conn(0x7fa1fc0ca990 0x7fa1fc0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 --2- 172.21.15.179:0/1351846885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0ca450 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- 172.21.15.179:0/1351846885 >> 172.21.15.179:0/1351846885 conn(0x7fa1fc0cb660 msgr2=0x7fa1fc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- 172.21.15.179:0/1351846885 shutdown_connections 2024-05-08T22:41:39.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- 172.21.15.179:0/1351846885 wait complete. 2024-05-08T22:41:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 Processor -- start 2024-05-08T22:41:39.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.927+0000 7fa203404700 1 -- start start 2024-05-08T22:41:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0c0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1fc15df60 0x7fa1fc0c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1535b0 con 0x7fa1fc0ca990 2024-05-08T22:41:39.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0b18e0 con 0x7fa1fc06ac90 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c4dc0 con 0x7fa1fc15df60 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59764/0 (socket says 172.21.15.179:59764) 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 -- 172.21.15.179:0/592507593 learned_addr learned my addr 172.21.15.179:0/592507593 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa2011a0700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa2019a1700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1fc15df60 0x7fa1fc0c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 -- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1fc15df60 msgr2=0x7fa1fc0c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1fc15df60 0x7fa1fc0c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 -- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1fc06ac90 msgr2=0x7fa1fc0c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 -- 172.21.15.179:0/592507593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009570 con 0x7fa1fc0ca990 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa20099f700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0038c0 con 0x7fa1fc0ca990 2024-05-08T22:41:39.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec01e070 con 0x7fa1fc0ca990 2024-05-08T22:41:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc0c50a0 con 0x7fa1fc0ca990 2024-05-08T22:41:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec003c60 con 0x7fa1fc0ca990 2024-05-08T22:41:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc0c5380 con 0x7fa1fc0ca990 2024-05-08T22:41:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.931+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc067770 con 0x7fa1fc0ca990 2024-05-08T22:41:39.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.935+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1ec011420 con 0x7fa1fc0ca990 2024-05-08T22:41:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.935+0000 7fa1f1ffb700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 0x7fa1e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.935+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa1ec020070 con 0x7fa1fc0ca990 2024-05-08T22:41:39.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.935+0000 7fa2011a0700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 0x7fa1e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:39.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.935+0000 7fa2011a0700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 0x7fa1e0068bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc153e00 tx=0x7fa1f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:39.939+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1ec052cd0 con 0x7fa1fc0ca990 2024-05-08T22:41:40.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.151+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1fc0cea50 con 0x7fa1fc0ca990 2024-05-08T22:41:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.151+0000 7fa1f1ffb700 1 -- 172.21.15.179:0/592507593 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fa1ec0562f0 con 0x7fa1fc0ca990 2024-05-08T22:41:40.153 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:40.154 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:40:30.890207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8154},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8154":{"gid":8154,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6835/55939533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":55939533},{"type":"v1","addr":"172.21.15.179:6835","nonce":55939533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 msgr2=0x7fa1e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 0x7fa1e0068bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc153e00 tx=0x7fa1f8009040 comp rx=0 tx=0).stop 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 msgr2=0x7fa1fc0c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec004770 comp rx=0 tx=0).stop 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 shutdown_connections 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1e0066730 0x7fa1e0068bf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1fc15df60 0x7fa1fc0c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc0ca990 0x7fa1fc0c1af0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 --2- 172.21.15.179:0/592507593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1fc06ac90 0x7fa1fc0c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 >> 172.21.15.179:0/592507593 conn(0x7fa1fc0cb660 msgr2=0x7fa1fc0afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 shutdown_connections 2024-05-08T22:41:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.155+0000 7fa203404700 1 -- 172.21.15.179:0/592507593 wait complete. 2024-05-08T22:41:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 62 2024-05-08T22:41:40.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T22:41:40.301 INFO:teuthology.orchestra.run.smithi179.stdout:{"message":"","return_code":0} 2024-05-08T22:41:40.302 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:41:40.302 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:41:40.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.411+0000 7fb19f2aa700 1 Processor -- start 2024-05-08T22:41:40.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.411+0000 7fb19f2aa700 1 -- start start 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb1980c2170 0x7fb1980cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1980cda40 0x7fb19806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1980cdea0 con 0x7fb19806c100 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb198161af0 con 0x7fb1980c2170 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980ca670 con 0x7fb1980cda40 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb1980c2170 0x7fb1980cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d847700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb19806c100 0x7fb198165190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36202/0 (socket says 172.21.15.179:36202) 2024-05-08T22:41:40.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d847700 1 -- 172.21.15.179:0/138461295 learned_addr learned my addr 172.21.15.179:0/138461295 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:40.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19c845700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1980cda40 0x7fb19806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 -- 172.21.15.179:0/138461295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1980cda40 msgr2=0x7fb19806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1980cda40 0x7fb19806bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 -- 172.21.15.179:0/138461295 >> v1:172.21.15.179:6789/0 conn(0x7fb19806c100 legacy=0x7fb198165190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb197fff700 1 -- 172.21.15.179:0/138461295 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345832552 0 0) 0x7fb1980cdea0 con 0x7fb19806c100 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 -- 172.21.15.179:0/138461295 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980ca950 con 0x7fb1980c2170 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19d046700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980c2170 0x7fb1980cd500 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009850 tx=0x7fb18c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7677fe8fa648e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb197fff700 1 -- 172.21.15.179:0/138461295 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011830 con 0x7fb1980c2170 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb197fff700 1 -- 172.21.15.179:0/138461295 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c011990 con 0x7fb1980c2170 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb197fff700 1 -- 172.21.15.179:0/138461295 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011d40 con 0x7fb1980c2170 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- 172.21.15.179:0/138461295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980c2170 msgr2=0x7fb1980cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980c2170 0x7fb1980cd500 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009850 tx=0x7fb18c009b60 comp rx=0 tx=0).stop 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- 172.21.15.179:0/138461295 shutdown_connections 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1980cda40 0x7fb19806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/138461295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980c2170 0x7fb1980cd500 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- 172.21.15.179:0/138461295 >> 172.21.15.179:0/138461295 conn(0x7fb198070e50 msgr2=0x7fb198071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- 172.21.15.179:0/138461295 shutdown_connections 2024-05-08T22:41:40.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.415+0000 7fb19f2aa700 1 -- 172.21.15.179:0/138461295 wait complete. 2024-05-08T22:41:40.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 Processor -- start 2024-05-08T22:41:40.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 -- start start 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb19806c100 0x7fb19815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1980c2170 0x7fb1980c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1981fb4d0 con 0x7fb1980cda40 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1981635f0 con 0x7fb1980c2170 2024-05-08T22:41:40.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19f2aa700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980c16f0 con 0x7fb19806c100 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59794/0 (socket says 172.21.15.179:59794) 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 -- 172.21.15.179:0/3574295890 learned_addr learned my addr 172.21.15.179:0/3574295890 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19c845700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1980c2170 0x7fb1980c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d046700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb19806c100 0x7fb19815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 -- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb19806c100 msgr2=0x7fb19815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb19806c100 0x7fb19815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 -- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1980c2170 msgr2=0x7fb1980c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.419+0000 7fb19d847700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1980c2170 0x7fb1980c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19d847700 1 -- 172.21.15.179:0/3574295890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0094b0 con 0x7fb1980cda40 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19d847700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb19000c460 tx=0x7fb19000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190004ca0 con 0x7fb1980cda40 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190004e20 con 0x7fb1980cda40 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1981fa710 con 0x7fb1980cda40 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190007310 con 0x7fb1980cda40 2024-05-08T22:41:40.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1981facd0 con 0x7fb1980cda40 2024-05-08T22:41:40.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb198067770 con 0x7fb1980cda40 2024-05-08T22:41:40.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb190007470 con 0x7fb1980cda40 2024-05-08T22:41:40.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 0x7fb184068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fb190083e70 con 0x7fb1980cda40 2024-05-08T22:41:40.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.423+0000 7fb19d046700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 0x7fb184068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.427+0000 7fb19d046700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 0x7fb184068bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0090b0 tx=0x7fb18c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.435+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb190050fe0 con 0x7fb1980cda40 2024-05-08T22:41:40.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.639+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1980c8260 con 0x7fb1980cda40 2024-05-08T22:41:40.642 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:41:40.643+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:41:40.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.655+0000 7fb195ffb700 1 -- 172.21.15.179:0/3574295890 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb190007720 con 0x7fb1980cda40 2024-05-08T22:41:40.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 msgr2=0x7fb184068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 0x7fb184068bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0090b0 tx=0x7fb18c01c040 comp rx=0 tx=0).stop 2024-05-08T22:41:40.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 msgr2=0x7fb1980c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb19000c460 tx=0x7fb19000c770 comp rx=0 tx=0).stop 2024-05-08T22:41:40.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 shutdown_connections 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb184066730 0x7fb184068bf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1980c2170 0x7fb1980c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1980cda40 0x7fb1980c42d0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 --2- 172.21.15.179:0/3574295890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb19806c100 0x7fb19815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 >> 172.21.15.179:0/3574295890 conn(0x7fb198070e50 msgr2=0x7fb1980b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 shutdown_connections 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.659+0000 7fb19f2aa700 1 -- 172.21.15.179:0/3574295890 wait complete. 2024-05-08T22:41:40.662 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:41:40.663 INFO:tasks.ceph.mds.a.smithi179.stderr: -4414> 2024-05-08T22:40:25.849+0000 7f7f2b8ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:41:40.664 INFO:tasks.ceph.mds.a.smithi179.stderr: -4065> 2024-05-08T22:40:27.717+0000 7f7f280c3700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-08T22:41:40.664 INFO:tasks.ceph.mds.a.smithi179.stderr: -3881> 2024-05-08T22:40:27.905+0000 7f7f280c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0) n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5633b5711b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-08T22:41:40.667 INFO:tasks.ceph.mds.a.smithi179.stderr: -2301> 2024-05-08T22:40:59.384+0000 7f7f280c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)->f(v1 m2024-05-08T22:40:09.291554+0000) n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)->n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5633b5711b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T22:40:09.291554+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T22:40:09.322265+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-08T22:41:40.675 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-08T22:41:40.697 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:41:40.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- start start 2024-05-08T22:41:40.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 Processor -- start 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 0x55714e88e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714e7e9400 0x55714e88eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e86aa00 con 0x55714e7e9000 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a780 con 0x55714e7e9400 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a640 con 0x55714e7e9800 2024-05-08T22:41:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59810/0 (socket says 172.21.15.179:59810) 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 -- 172.21.15.179:0/4199667148 learned_addr learned my addr 172.21.15.179:0/4199667148 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff72557f700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 0x55714e88e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff724d7e700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714e7e9400 0x55714e88eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 -- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 msgr2=0x55714e88e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 0x55714e88e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 -- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714e7e9400 msgr2=0x55714e88eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714e7e9400 0x55714e88eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 -- 172.21.15.179:0/4199667148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714e80a9c0 con 0x55714e7e9000 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff72557f700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 0x55714e88e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff725d80700 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55714e8f78c0 tx=0x55714f66e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bede260ad9d2c78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff722d7a700 1 -- 172.21.15.179:0/4199667148 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e819600 con 0x55714e7e9000 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff722d7a700 1 -- 172.21.15.179:0/4199667148 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55714e834180 con 0x55714e7e9000 2024-05-08T22:41:40.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff722d7a700 1 -- 172.21.15.179:0/4199667148 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e8191e0 con 0x55714e7e9000 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55714e80ab60 con 0x55714e7e9000 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55714e80a340 con 0x55714e7e9000 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.711+0000 7ff726dc59c0 4 waiting for MDS map... 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff722d7a700 1 -- 172.21.15.179:0/4199667148 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55714e822c40 con 0x55714e7e9000 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff722d7a700 1 -- 172.21.15.179:0/4199667148 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 63) v1 ==== 2573+0+0 (secure 0 0 0) 0x55714f5d2000 con 0x55714e7e9000 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 4 Got MDS map 63 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 4 datascan.main: Using metadata pool output 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 Processor -- start 2024-05-08T22:41:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 -- start start 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 0x55714e88fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672c00 0x55714e890100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e81fe00 con 0x55714f672400 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a780 con 0x55714f672800 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a500 con 0x55714f672c00 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59826/0 (socket says 172.21.15.179:59826) 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 -- 172.21.15.179:0/3074324097 learned_addr learned my addr 172.21.15.179:0/3074324097 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff72557f700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 0x55714e88fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672c00 0x55714e890100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 -- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672c00 msgr2=0x55714e890100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672c00 0x55714e890100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 -- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 msgr2=0x55714e88fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 0x55714e88fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 -- 172.21.15.179:0/3074324097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714e80a000 con 0x55714f672400 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff72557f700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 0x55714e88fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff724d7e700 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x55714eaea750 tx=0x55714eaf0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41daa669ec5d6124 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff720d76700 1 -- 172.21.15.179:0/3074324097 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e819600 con 0x55714f672400 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff720d76700 1 -- 172.21.15.179:0/3074324097 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55714f5d6480 con 0x55714f672400 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff720d76700 1 -- 172.21.15.179:0/3074324097 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e8191e0 con 0x55714f672400 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 -- 172.21.15.179:0/3074324097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 msgr2=0x55714e88f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.715+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x55714eaea750 tx=0x55714eaf0c60 comp rx=0 tx=0).stop 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/3074324097 shutdown_connections 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672800 0x55714e88fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672c00 0x55714e890100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/3074324097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672400 0x55714e88f600 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/3074324097 >> 172.21.15.179:0/3074324097 conn(0x55714e7e8c00 msgr2=0x55714e8fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/3074324097 shutdown_connections 2024-05-08T22:41:40.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/3074324097 wait complete. 2024-05-08T22:41:40.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 Processor -- start 2024-05-08T22:41:40.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- start start 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672400 0x55714e891180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672800 0x55714e890c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a3c0 con 0x55714f672c00 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a280 con 0x55714f672400 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714e86a140 con 0x55714f672800 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59834/0 (socket says 172.21.15.179:59834) 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 learned_addr learned my addr 172.21.15.179:0/1767847829 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672800 0x55714e890c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672400 0x55714e891180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672800 msgr2=0x55714e890c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714f672800 0x55714e890c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672400 msgr2=0x55714e891180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714f672400 0x55714e891180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714e80a9c0 con 0x55714f672c00 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x55714eaeb8f0 tx=0x55714e8bf7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff71f573700 1 -- 172.21.15.179:0/1767847829 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e818dc0 con 0x55714f672c00 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff71f573700 1 -- 172.21.15.179:0/1767847829 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55714f5d6c00 con 0x55714f672c00 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55714e80ab60 con 0x55714f672c00 2024-05-08T22:41:40.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff71f573700 1 -- 172.21.15.179:0/1767847829 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55714e819600 con 0x55714f672c00 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55714e80a340 con 0x55714f672c00 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.719+0000 7ff726dc59c0 4 datascan.init: resolving metadata pool 13 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff71f573700 1 -- 172.21.15.179:0/1767847829 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55714e848840 con 0x55714f672c00 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff71f573700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55714ec40000 0x55714e891700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff71f573700 1 -- 172.21.15.179:0/1767847829 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55714e822a80 con 0x55714f672c00 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff726dc59c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff726dc59c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff726dc59c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55714f673000 0x55714e890100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714f673400 con 0x55714f673000 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55714ec40000 0x55714e891700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55714f673000 0x55714e890100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55714f673000 0x55714e890100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.723+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55714ec40000 0x55714e891700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55714e8f63c0 tx=0x55714e8aff20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.727+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(1 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.727+0000 7ff720575700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55714ec40400 0x55714ec4e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.727+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40800 con 0x55714ec40400 2024-05-08T22:41:40.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.727+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55714ec40400 0x55714ec4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.727+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55714ec40400 0x55714ec4e000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.731+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(2 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.731+0000 7ff720d76700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55714ec41000 0x55714ec4e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.731+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41400 con 0x55714ec41000 2024-05-08T22:41:40.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.731+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55714ec41000 0x55714ec4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.731+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55714ec41000 0x55714ec4e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(3 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714ec40400 2024-05-08T22:41:40.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(4 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41800 con 0x55714ec40400 2024-05-08T22:41:40.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(5 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff720575700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55714ec40800 0x55714ec4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714ec40800 2024-05-08T22:41:40.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55714ec40800 0x55714ec4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.735+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55714ec40800 0x55714ec4eb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(6 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41c00 con 0x55714ec41000 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(7 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff720575700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55714ec41800 0x55714ec4f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41400 con 0x55714ec41800 2024-05-08T22:41:40.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55714ec41800 0x55714ec4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55714ec41800 0x55714ec4f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(8 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff720d76700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55714ec70000 0x55714ec4f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70400 con 0x55714ec70000 2024-05-08T22:41:40.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55714ec70000 0x55714ec4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.739+0000 7ff72557f700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55714ec70000 0x55714ec4f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.743+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(9 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec70000 2024-05-08T22:41:40.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.743+0000 7ff720575700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55714ec41c00 0x55714ec50100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.743+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714ec41c00 2024-05-08T22:41:40.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.743+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55714ec41c00 0x55714ec50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.743+0000 7ff724d7e700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55714ec41c00 0x55714ec50100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.747+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(10 [pgnls start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.747+0000 7ff720d76700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55714ec70800 0x55714ec4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.747+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70c00 con 0x55714ec70800 2024-05-08T22:41:40.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.747+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55714ec70800 0x55714ec4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.747+0000 7ff725d80700 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55714ec70800 0x55714ec4fb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(11 [pgnls start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71400 con 0x55714f673000 2024-05-08T22:41:40.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(12 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714ec40400 2024-05-08T22:41:40.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(13 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71800 con 0x55714ec41000 2024-05-08T22:41:40.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(14 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714ec40400 2024-05-08T22:41:40.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(15 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71c00 con 0x55714ec41000 2024-05-08T22:41:40.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.755+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(16 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714f673400 con 0x55714ec70000 2024-05-08T22:41:40.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(17 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec70000 2024-05-08T22:41:40.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41400 con 0x55714ec70800 2024-05-08T22:41:40.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(18 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714f673000 2024-05-08T22:41:40.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(19 [pgnls start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70400 con 0x55714ec70800 2024-05-08T22:41:40.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(20 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a000 con 0x55714ec41800 2024-05-08T22:41:40.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(21 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.759+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71800 con 0x55714ec70800 2024-05-08T22:41:40.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(22 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a400 con 0x55714ec40400 2024-05-08T22:41:40.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(23 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71c00 con 0x55714ec40400 2024-05-08T22:41:40.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(24 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a800 con 0x55714ec40400 2024-05-08T22:41:40.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(25 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70c00 con 0x55714ec70800 2024-05-08T22:41:40.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(26 [pgnls start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.763+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7ac00 con 0x55714ec40800 2024-05-08T22:41:40.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(27 [pgnls start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714f673000 2024-05-08T22:41:40.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(28 [pgnls start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b000 con 0x55714ec70800 2024-05-08T22:41:40.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(29 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7ac00 con 0x55714ec40800 2024-05-08T22:41:40.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(30 [pgnls start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b400 con 0x55714ec41c00 2024-05-08T22:41:40.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(31 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71400 con 0x55714ec41800 2024-05-08T22:41:40.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(32 [pgnls start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.767+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b800 con 0x55714ec40800 2024-05-08T22:41:40.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(33 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714ec41800 2024-05-08T22:41:40.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(34 [pgnls start_epoch 100 out=49b] v98'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a800 con 0x55714ec70800 2024-05-08T22:41:40.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 7 ==== osd_op_reply(35 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70400 con 0x55714ec41c00 2024-05-08T22:41:40.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(36 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b000 con 0x55714ec41800 2024-05-08T22:41:40.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(37 [pgnls start_epoch 100 out=49b] v98'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.771+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71800 con 0x55714ec70000 2024-05-08T22:41:40.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(38 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec70000 2024-05-08T22:41:40.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a000 con 0x55714ec70000 2024-05-08T22:41:40.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(39 [pgnls start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec70000 2024-05-08T22:41:40.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714ec41c00 2024-05-08T22:41:40.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(40 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a400 con 0x55714ec40400 2024-05-08T22:41:40.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 9 ==== osd_op_reply(41 [pgnls start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec41400 con 0x55714f673000 2024-05-08T22:41:40.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 5 ==== osd_op_reply(42 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71c00 con 0x55714ec41c00 2024-05-08T22:41:40.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(43 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.775+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7ac00 con 0x55714ec70800 2024-05-08T22:41:40.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 8 ==== osd_op_reply(44 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70c00 con 0x55714ec70000 2024-05-08T22:41:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.6 v2:172.21.15.195:6824/26291 5 ==== osd_op_reply(45 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec70000 2024-05-08T22:41:40.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b400 con 0x55714ec70800 2024-05-08T22:41:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 9 ==== osd_op_reply(46 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71400 con 0x55714ec40400 2024-05-08T22:41:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 10 ==== osd_op_reply(47 [pgnls start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b800 con 0x55714ec40800 2024-05-08T22:41:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 5 ==== osd_op_reply(48 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.779+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714ec40400 2024-05-08T22:41:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 11 ==== osd_op_reply(49 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7bc00 con 0x55714ec41c00 2024-05-08T22:41:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(50 [pgnls start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71400 con 0x55714f673000 2024-05-08T22:41:40.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 6 ==== osd_op_reply(51 [pgnls start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec40c00 con 0x55714ec41000 2024-05-08T22:41:40.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(52 [pgnls start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71000 con 0x55714ec40400 2024-05-08T22:41:40.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 12 ==== osd_op_reply(53 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70400 con 0x55714ec70800 2024-05-08T22:41:40.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.783+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 10 ==== osd_op_reply(54 [pgnls start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a800 con 0x55714ec41000 2024-05-08T22:41:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(55 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668fc0 con 0x55714ec41000 2024-05-08T22:41:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec71800 con 0x55714ec70800 2024-05-08T22:41:40.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 11 ==== osd_op_reply(56 [pgnls start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b000 con 0x55714ec41800 2024-05-08T22:41:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(57 [pgnls start_epoch 100 out=81b] v98'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714f673800 con 0x55714ec41800 2024-05-08T22:41:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.787+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.791+0000 7ff726dc59c0 10 datascan.operator(): handling object 1099511627777.0 2024-05-08T22:41:40.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.791+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec84000 con 0x55714ec41800 2024-05-08T22:41:40.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.791+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(59 10000000001.00000000 [call] v100'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.791+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec84400 con 0x55714ec41800 2024-05-08T22:41:40.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.5 v2:172.21.15.195:6802/26288 9 ==== osd_op_reply(60 [pgnls start_epoch 100 out=49b] v98'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec41800 2024-05-08T22:41:40.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7a400 con 0x55714f673000 2024-05-08T22:41:40.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.2 v2:172.21.15.179:6824/26078 7 ==== osd_op_reply(61 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714f673000 2024-05-08T22:41:40.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec70c00 con 0x55714ec70800 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.4 v2:172.21.15.195:6800/26287 12 ==== osd_op_reply(62 [pgnls start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec70800 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7ac00 con 0x55714ec41c00 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(63 [pgnls start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec41c00 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b000 con 0x55714ec40400 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 13 ==== osd_op_reply(64 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.795+0000 7ff720d76700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b400 con 0x55714ec40400 2024-05-08T22:41:40.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff725d80700 1 -- 172.21.15.179:0/1767847829 <== osd.0 v2:172.21.15.179:6800/26067 14 ==== osd_op_reply(65 [pgnls start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668d80 con 0x55714ec40400 2024-05-08T22:41:40.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff720575700 1 -- 172.21.15.179:0/1767847829 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55714ec7b800 con 0x55714ec40800 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff724d7e700 1 -- 172.21.15.179:0/1767847829 <== osd.3 v2:172.21.15.179:6816/26068 6 ==== osd_op_reply(66 [pgnls start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55714f668b40 con 0x55714ec40800 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55714ec40400 msgr2=0x55714ec4e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55714ec40400 0x55714ec4e000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55714ec41000 msgr2=0x55714ec4e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55714ec41000 0x55714ec4e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55714f673000 msgr2=0x55714e890100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55714f673000 0x55714e890100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55714ec40800 msgr2=0x55714ec4eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55714ec40800 0x55714ec4eb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55714ec70800 msgr2=0x55714ec4fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55714ec70800 0x55714ec4fb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 reap_dead start 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55714ec41800 msgr2=0x55714ec4f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55714ec41800 0x55714ec4f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55714ec70000 msgr2=0x55714ec4f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55714ec70000 0x55714ec4f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55714ec41c00 msgr2=0x55714ec50100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55714ec41c00 0x55714ec50100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55714ec40000 msgr2=0x55714e891700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55714ec40000 0x55714e891700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55714e8f63c0 tx=0x55714e8aff20 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 msgr2=0x55714e890680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x55714eaeb8f0 tx=0x55714e8bf7a0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff72557f700 1 -- 172.21.15.179:0/1767847829 reap_dead start 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 shutdown_connections 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/1767847829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714f672c00 0x55714e890680 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 >> 172.21.15.179:0/1767847829 conn(0x55714e7e8c00 msgr2=0x55714e8fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 shutdown_connections 2024-05-08T22:41:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/1767847829 wait complete. 2024-05-08T22:41:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 msgr2=0x55714e88f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55714e8f78c0 tx=0x55714f66e120 comp rx=0 tx=0).stop 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 shutdown_connections 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55714e7e9400 0x55714e88eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55714e7e9000 0x55714e88f080 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 --2- 172.21.15.179:0/4199667148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55714e7e9800 0x55714e88e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 >> 172.21.15.179:0/4199667148 conn(0x55714e7e9c00 msgr2=0x55714e8fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 shutdown_connections 2024-05-08T22:41:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.799+0000 7ff726dc59c0 1 -- 172.21.15.179:0/4199667148 wait complete. 2024-05-08T22:41:40.807 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.131578, result: 2024-05-08T22:41:40.807 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-08T22:41:40.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 -- start start 2024-05-08T22:41:40.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 Processor -- start 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 0x55ae67920580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 0x55ae67920b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678b1e00 con 0x55ae6787b800 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fca00 con 0x55ae6787b000 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc640 con 0x55ae6787b400 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7bd84700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 0x55ae67920580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7b583700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 0x55ae67920b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7c585700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7b583700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 0x55ae67920b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:54466/0 (socket says 172.21.15.179:54466) 2024-05-08T22:41:40.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7bd84700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 0x55ae67920580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59836/0 (socket says 172.21.15.179:59836) 2024-05-08T22:41:40.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7c585700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:48180/0 (socket says 172.21.15.179:48180) 2024-05-08T22:41:40.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.847+0000 7fad7b583700 1 -- 172.21.15.179:0/2020676368 learned_addr learned my addr 172.21.15.179:0/2020676368 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 -- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 msgr2=0x55ae67920b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 0x55ae67920b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 -- 172.21.15.179:0/2020676368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 msgr2=0x55ae67920580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 0x55ae67920580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 -- 172.21.15.179:0/2020676368 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6789c9c0 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7c585700 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ae679838f0 tx=0x55ae686f4240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb182e7ff4efe69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7957f700 1 -- 172.21.15.179:0/2020676368 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678ab600 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae6789cb60 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ae6789c340 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 4 waiting for MDS map... 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7957f700 1 -- 172.21.15.179:0/2020676368 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae678c6480 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7957f700 1 -- 172.21.15.179:0/2020676368 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678ab1e0 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7957f700 1 -- 172.21.15.179:0/2020676368 <== mon.1 v2:172.21.15.195:3300/0 4 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55ae678b4c40 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7957f700 1 -- 172.21.15.179:0/2020676368 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 63) v1 ==== 2573+0+0 (secure 0 0 0) 0x55ae68702900 con 0x55ae6787b000 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 4 Got MDS map 63 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 4 datascan.main: Using metadata pool output 2024-05-08T22:41:40.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:41:40.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 Processor -- start 2024-05-08T22:41:40.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- start start 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74400 0x55ae67921600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc500 con 0x55ae67b74c00 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc3c0 con 0x55ae67b74800 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc640 con 0x55ae67b74400 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7b583700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7b583700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74400 0x55ae67921600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:48182/0 (socket says 172.21.15.179:48182) 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 -- 172.21.15.179:0/3876835647 learned_addr learned my addr 172.21.15.179:0/3876835647 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 -- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74400 msgr2=0x55ae67921600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74400 0x55ae67921600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 -- 172.21.15.179:0/3876835647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 msgr2=0x55ae67922100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7bd84700 1 -- 172.21.15.179:0/3876835647 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6789c000 con 0x55ae67b74800 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.851+0000 7fad7b583700 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55ae67b68870 tx=0x55ae68662c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7033d031f2857fca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7757b700 1 -- 172.21.15.179:0/3876835647 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678ab600 con 0x55ae67b74800 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7757b700 1 -- 172.21.15.179:0/3876835647 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae68665380 con 0x55ae67b74800 2024-05-08T22:41:40.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7757b700 1 -- 172.21.15.179:0/3876835647 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678ab1e0 con 0x55ae67b74800 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 msgr2=0x55ae67921b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55ae67b68870 tx=0x55ae68662c00 comp rx=0 tx=0).stop 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/3876835647 shutdown_connections 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74800 0x55ae67921b80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/3876835647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74400 0x55ae67921600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/3876835647 >> 172.21.15.179:0/3876835647 conn(0x55ae6787ac00 msgr2=0x55ae67937800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/3876835647 shutdown_connections 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/3876835647 wait complete. 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 Processor -- start 2024-05-08T22:41:40.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- start start 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74400 0x55ae67923180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74800 0x55ae67922c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc280 con 0x55ae67b74c00 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc140 con 0x55ae67b74400 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7d5ca9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae678fc000 con 0x55ae67b74800 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7b583700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74800 0x55ae67922c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7b583700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59848/0 (socket says 172.21.15.179:59848) 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 learned_addr learned my addr 172.21.15.179:0/1797372006 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74400 0x55ae67923180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74400 msgr2=0x55ae67923180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae67b74400 0x55ae67923180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 msgr2=0x55ae67922680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6789c9c0 con 0x55ae67b74800 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.855+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae67b74c00 0x55ae67922680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74800 0x55ae67922c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55ae67b69bc0 tx=0x55ae686f41e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 -- 172.21.15.179:0/1797372006 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678aadc0 con 0x55ae67b74800 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 -- 172.21.15.179:0/1797372006 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae68665500 con 0x55ae67b74800 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 -- 172.21.15.179:0/1797372006 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ae678ab600 con 0x55ae67b74800 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae6789cb60 con 0x55ae67b74800 2024-05-08T22:41:40.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae6789c340 con 0x55ae67b74800 2024-05-08T22:41:40.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 4 datascan.init: resolving metadata pool 13 2024-05-08T22:41:40.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 -- 172.21.15.179:0/1797372006 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55ae678da840 con 0x55ae67b74800 2024-05-08T22:41:40.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ae67bfe400 0x55ae67923700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad75d78700 1 -- 172.21.15.179:0/1797372006 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55ae678b4a80 con 0x55ae67b74800 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ae67bfe400 0x55ae67923700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ae67b75000 0x55ae67922100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.859+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67b75400 con 0x55ae67b75000 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ae67b75000 0x55ae67922100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ae67b75000 0x55ae67922100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ae67bfe400 0x55ae67923700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55ae67c8d800 tx=0x55ae679501e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ae67b75800 0x55ae67921b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67b75800 2024-05-08T22:41:40.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ae67b75800 0x55ae67921b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ae67b75800 0x55ae67921b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ae67bfec00 0x55ae67921600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67bfec00 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ae67bfec00 0x55ae67921600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.863+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ae67bfec00 0x55ae67921600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2000 con 0x55ae67b75000 2024-05-08T22:41:40.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7757b700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ae67cf2400 0x55ae67cf8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2800 con 0x55ae67cf2400 2024-05-08T22:41:40.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ae67cf2400 0x55ae67cf8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ae67cf2400 0x55ae67cf8000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2000 con 0x55ae67b75000 2024-05-08T22:41:40.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67b75000 2024-05-08T22:41:40.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad76d7a700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ae67b75400 0x55ae67cf8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.867+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2800 con 0x55ae67b75400 2024-05-08T22:41:40.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ae67b75400 0x55ae67cf8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ae67b75400 0x55ae67cf8580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67cf2400 2024-05-08T22:41:40.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad76d7a700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ae67cf3000 0x55ae67cf8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67cf3000 2024-05-08T22:41:40.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ae67cf3000 0x55ae67cf8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7bd84700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ae67cf3000 0x55ae67cf8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.871+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67b75800 2024-05-08T22:41:40.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad76d7a700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ae67cf2800 0x55ae67cf9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67cf2800 2024-05-08T22:41:40.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ae67cf2800 0x55ae67cf9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7b583700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ae67cf2800 0x55ae67cf9080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7757b700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ae67cf2000 0x55ae67cf9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:40.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67cf2000 2024-05-08T22:41:40.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ae67cf2000 0x55ae67cf9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:40.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7c585700 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ae67cf2000 0x55ae67cf9600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:40.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.875+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67bfec00 2024-05-08T22:41:40.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67b75000 2024-05-08T22:41:40.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67cf2400 2024-05-08T22:41:40.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67b75000 2024-05-08T22:41:40.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67cf2400 2024-05-08T22:41:40.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12000 con 0x55ae67b75800 2024-05-08T22:41:40.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.879+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67cf2000 2024-05-08T22:41:40.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67bfec00 2024-05-08T22:41:40.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67cf2000 2024-05-08T22:41:40.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12800 con 0x55ae67cf3000 2024-05-08T22:41:40.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67cf2000 2024-05-08T22:41:40.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67b75000 2024-05-08T22:41:40.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.883+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13c00 con 0x55ae67b75000 2024-05-08T22:41:40.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13000 con 0x55ae67b75000 2024-05-08T22:41:40.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67cf2000 2024-05-08T22:41:40.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12000 con 0x55ae67b75400 2024-05-08T22:41:40.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67bfec00 2024-05-08T22:41:40.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67cf2000 2024-05-08T22:41:40.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.887+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67b75400 2024-05-08T22:41:40.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67cf2800 2024-05-08T22:41:40.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(33 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67cf3000 2024-05-08T22:41:40.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67b75400 2024-05-08T22:41:40.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67cf3000 2024-05-08T22:41:40.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 100 out=49b] v98'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67cf2000 2024-05-08T22:41:40.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.891+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67cf2800 2024-05-08T22:41:40.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12000 con 0x55ae67cf3000 2024-05-08T22:41:40.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 100 out=49b] v98'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67b75800 2024-05-08T22:41:40.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67b75800 2024-05-08T22:41:40.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67cf2800 2024-05-08T22:41:40.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.895+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(42 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16000 con 0x55ae67b75000 2024-05-08T22:41:40.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67bfec00 2024-05-08T22:41:40.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67cf2800 2024-05-08T22:41:40.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67cf2000 2024-05-08T22:41:40.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16800 con 0x55ae67b75800 2024-05-08T22:41:40.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.899+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12800 con 0x55ae67cf2000 2024-05-08T22:41:40.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16c00 con 0x55ae67b75000 2024-05-08T22:41:40.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67b75400 2024-05-08T22:41:40.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67b75000 2024-05-08T22:41:40.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67cf2800 2024-05-08T22:41:40.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.903+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17400 con 0x55ae67bfec00 2024-05-08T22:41:40.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67cf2400 2024-05-08T22:41:40.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(54 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17800 con 0x55ae67b75000 2024-05-08T22:41:40.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67cf2000 2024-05-08T22:41:40.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17c00 con 0x55ae67cf2400 2024-05-08T22:41:40.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(57 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.907+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67cf2000 2024-05-08T22:41:40.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 100 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67cf3000 2024-05-08T22:41:40.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 100 out=81b] v100'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff400 con 0x55ae67bfec00 2024-05-08T22:41:40.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16c00 con 0x55ae67b75000 2024-05-08T22:41:40.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67cf2400 2024-05-08T22:41:40.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.911+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67b75000 2024-05-08T22:41:40.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67b75000 2024-05-08T22:41:40.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17800 con 0x55ae67b75400 2024-05-08T22:41:40.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13c00 con 0x55ae67cf2400 2024-05-08T22:41:40.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17c00 con 0x55ae67cf3000 2024-05-08T22:41:40.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.915+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13000 con 0x55ae67b75800 2024-05-08T22:41:40.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16800 con 0x55ae67cf2800 2024-05-08T22:41:40.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(69 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17400 con 0x55ae67cf2000 2024-05-08T22:41:40.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67bfec00 2024-05-08T22:41:40.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67b75000 2024-05-08T22:41:40.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.919+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67cf2400 2024-05-08T22:41:40.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 9 ==== osd_op_reply(73 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16000 con 0x55ae67b75000 2024-05-08T22:41:40.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67cf2400 2024-05-08T22:41:40.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67b75800 2024-05-08T22:41:40.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16c00 con 0x55ae67cf2000 2024-05-08T22:41:40.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67bfec00 2024-05-08T22:41:40.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.923+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67cf2000 2024-05-08T22:41:40.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12000 con 0x55ae67cf3000 2024-05-08T22:41:40.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67cf2000 2024-05-08T22:41:40.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67b75000 2024-05-08T22:41:40.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13c00 con 0x55ae67b75000 2024-05-08T22:41:40.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.927+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16800 con 0x55ae67b75000 2024-05-08T22:41:40.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12800 con 0x55ae67cf2000 2024-05-08T22:41:40.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67b75400 2024-05-08T22:41:40.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67bfec00 2024-05-08T22:41:40.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17800 con 0x55ae67cf2000 2024-05-08T22:41:40.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67b75400 2024-05-08T22:41:40.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.931+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17400 con 0x55ae67cf2800 2024-05-08T22:41:40.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 8 ==== osd_op_reply(90 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13000 con 0x55ae67cf3000 2024-05-08T22:41:40.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67b75400 2024-05-08T22:41:40.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67cf3000 2024-05-08T22:41:40.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 100 out=49b] v98'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67cf2000 2024-05-08T22:41:40.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.935+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67cf2800 2024-05-08T22:41:40.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 9 ==== osd_op_reply(95 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12800 con 0x55ae67cf3000 2024-05-08T22:41:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 11 ==== osd_op_reply(96 [pgnls-filter start_epoch 100 out=49b] v98'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12000 con 0x55ae67b75800 2024-05-08T22:41:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12c00 con 0x55ae67b75800 2024-05-08T22:41:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67cf2800 2024-05-08T22:41:40.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 10 ==== osd_op_reply(99 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67b75000 2024-05-08T22:41:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 100 out=49b] v98'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.939+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67bfec00 2024-05-08T22:41:40.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3800 con 0x55ae67cf2800 2024-05-08T22:41:40.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 11 ==== osd_op_reply(102 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13c00 con 0x55ae67cf2000 2024-05-08T22:41:40.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf2c00 con 0x55ae67b75800 2024-05-08T22:41:40.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13800 con 0x55ae67cf2000 2024-05-08T22:41:40.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.943+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16c00 con 0x55ae67b75000 2024-05-08T22:41:40.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67b75400 2024-05-08T22:41:40.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67b75000 2024-05-08T22:41:40.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bfe800 con 0x55ae67cf2800 2024-05-08T22:41:40.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 12 ==== osd_op_reply(109 [pgnls-filter start_epoch 100 out=49b] v98'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17800 con 0x55ae67bfec00 2024-05-08T22:41:40.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 100 out=49b] v98'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.947+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16000 con 0x55ae67cf2400 2024-05-08T22:41:40.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 11 ==== osd_op_reply(111 [pgnls-filter start_epoch 100 out=49b] v98'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17c00 con 0x55ae67b75000 2024-05-08T22:41:40.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67cf2000 2024-05-08T22:41:40.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff400 con 0x55ae67cf2400 2024-05-08T22:41:40.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 12 ==== osd_op_reply(114 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16800 con 0x55ae67cf2000 2024-05-08T22:41:40.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 100 out=49b] v98'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.951+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67cf3000 2024-05-08T22:41:40.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 100 out=81b] v100'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-08T22:41:40.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 10 datascan.operator(): handling object 10000000001.0 2024-05-08T22:41:40.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff800 con 0x55ae67cf3000 2024-05-08T22:41:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-08T22:41:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-08T22:41:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-08T22:41:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bffc00 con 0x55ae67cf2400 2024-05-08T22:41:40.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 13 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-08T22:41:40.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff800 con 0x55ae67b75800 2024-05-08T22:41:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.6 v2:172.21.15.195:6824/26291 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75800 2024-05-08T22:41:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-08T22:41:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-05-08T22:41:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 20 datascan.get_frag_of: frag is * 2024-05-08T22:41:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.955+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17800 con 0x55ae67cf2400 2024-05-08T22:41:40.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 14 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-08T22:41:40.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-08T22:41:40.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17000 con 0x55ae67cf2400 2024-05-08T22:41:40.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 15 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T22:41:40.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 20 datascan.read_dentry: file_head not found in result 2024-05-08T22:41:40.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-08T22:41:40.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.959+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3c00 con 0x55ae67cf2400 2024-05-08T22:41:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.1 v2:172.21.15.179:6808/26069 16 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v100'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2400 2024-05-08T22:41:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7d5ca9c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T22:41:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67cf3400 con 0x55ae67cf3000 2024-05-08T22:41:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.5 v2:172.21.15.195:6802/26288 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 100 out=49b] v98'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67cf3000 2024-05-08T22:41:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67bff000 con 0x55ae67bfec00 2024-05-08T22:41:40.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 <== osd.2 v2:172.21.15.179:6824/26078 13 ==== osd_op_reply(124 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8fc0 con 0x55ae67bfec00 2024-05-08T22:41:40.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d17400 con 0x55ae67cf2000 2024-05-08T22:41:40.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.4 v2:172.21.15.195:6800/26287 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 100 out=49b] v98'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67cf2000 2024-05-08T22:41:40.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d13400 con 0x55ae67cf2800 2024-05-08T22:41:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.7 v2:172.21.15.195:6816/26289 13 ==== osd_op_reply(126 [pgnls-filter start_epoch 100 out=49b] v98'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67cf2800 2024-05-08T22:41:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.963+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16c00 con 0x55ae67b75000 2024-05-08T22:41:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad76d7a700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d12400 con 0x55ae67b75000 2024-05-08T22:41:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7b583700 1 -- 172.21.15.179:0/1797372006 <== osd.0 v2:172.21.15.179:6800/26067 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 100 out=49b] v98'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8b40 con 0x55ae67b75000 2024-05-08T22:41:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7757b700 1 -- 172.21.15.179:0/1797372006 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 100 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55ae67d16400 con 0x55ae67b75400 2024-05-08T22:41:40.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7c585700 1 -- 172.21.15.179:0/1797372006 <== osd.3 v2:172.21.15.179:6816/26068 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 100 out=49b] v98'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae685e8d80 con 0x55ae67b75400 2024-05-08T22:41:40.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ae67b75000 msgr2=0x55ae67922100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ae67b75000 0x55ae67922100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ae67cf2400 msgr2=0x55ae67cf8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ae67cf2400 0x55ae67cf8000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ae67bfec00 msgr2=0x55ae67921600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ae67bfec00 0x55ae67921600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ae67b75400 msgr2=0x55ae67cf8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ae67b75400 0x55ae67cf8580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 reap_dead start 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ae67cf2000 msgr2=0x55ae67cf9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ae67cf2000 0x55ae67cf9600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ae67cf3000 msgr2=0x55ae67cf8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ae67cf3000 0x55ae67cf8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ae67b75800 msgr2=0x55ae67921b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ae67b75800 0x55ae67921b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ae67cf2800 msgr2=0x55ae67cf9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ae67cf2800 0x55ae67cf9080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ae67bfe400 msgr2=0x55ae67923700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ae67bfe400 0x55ae67923700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55ae67c8d800 tx=0x55ae679501e0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74800 msgr2=0x55ae67922c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/1797372006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae67b74800 0x55ae67922c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55ae67b69bc0 tx=0x55ae686f41e0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 reap_dead start 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 shutdown_connections 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 >> 172.21.15.179:0/1797372006 conn(0x55ae6787ac00 msgr2=0x55ae67937800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 reap_dead start 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7bd84700 1 -- 172.21.15.179:0/1797372006 reap_dead start 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 shutdown_connections 2024-05-08T22:41:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/1797372006 wait complete. 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 msgr2=0x55ae67921080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ae679838f0 tx=0x55ae686f4240 comp rx=0 tx=0).stop 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 shutdown_connections 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ae6787b000 0x55ae67921080 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ae6787b400 0x55ae67920b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 --2- 172.21.15.179:0/2020676368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae6787b800 0x55ae67920580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 >> 172.21.15.179:0/2020676368 conn(0x55ae6787bc00 msgr2=0x55ae67937000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 shutdown_connections 2024-05-08T22:41:40.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:40.967+0000 7fad7d5ca9c0 1 -- 172.21.15.179:0/2020676368 wait complete. 2024-05-08T22:41:40.974 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.167230, result: 2024-05-08T22:41:40.975 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-08T22:41:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- start start 2024-05-08T22:41:41.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 Processor -- start 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 0x55c393248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 0x55c393249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393224a00 con 0x55c3931a3800 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393224780 con 0x55c3931a3000 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c393224640 con 0x55c3931a3400 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59868/0 (socket says 172.21.15.179:59868) 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 -- 172.21.15.179:0/3220057809 learned_addr learned my addr 172.21.15.179:0/3220057809 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 0x55c393248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 0x55c393249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 -- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 msgr2=0x55c393248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 0x55c393248b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 -- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 msgr2=0x55c393249080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 0x55c393249080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 -- 172.21.15.179:0/3220057809 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3931c49c0 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 0x55c393248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 0x55c393249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x55c3932ab800 tx=0x55c3932797a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee328c00b86659c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51c3ca700 1 -- 172.21.15.179:0/3220057809 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d3080 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51c3ca700 1 -- 172.21.15.179:0/3220057809 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3931ee180 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51c3ca700 1 -- 172.21.15.179:0/3220057809 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d3600 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3931c41a0 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c3931c49c0 con 0x55c3931a3800 2024-05-08T22:41:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 4 waiting for MDS map... 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51c3ca700 1 -- 172.21.15.179:0/3220057809 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55c3931dcc40 con 0x55c3931a3800 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff51c3ca700 1 -- 172.21.15.179:0/3220057809 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 63) v1 ==== 2573+0+0 (secure 0 0 0) 0x55c3931e4f00 con 0x55c3931a3800 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.015+0000 7ff5204159c0 4 Got MDS map 63 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 4 datascan.main: Using metadata pool output 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:41:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 Processor -- start 2024-05-08T22:41:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 -- start start 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 0x55c393249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 0x55c393249b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 0x55c39324a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3931d9e00 con 0x55c393498400 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393224780 con 0x55c393498c00 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c393224a00 con 0x55c393498800 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff51f3d0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 0x55c393249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff51f3d0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 0x55c393249600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59878/0 (socket says 172.21.15.179:59878) 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff51f3d0700 1 -- 172.21.15.179:0/2124423539 learned_addr learned my addr 172.21.15.179:0/2124423539 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 0x55c393249b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.019+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 0x55c39324a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 -- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 msgr2=0x55c39324a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 0x55c39324a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 -- 172.21.15.179:0/2124423539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 msgr2=0x55c393249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 0x55c393249600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 -- 172.21.15.179:0/2124423539 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3931c4b60 con 0x55c393498800 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 0x55c39324a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 0x55c393249b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55c393f96570 tx=0x55c3932798c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a42988b0b508bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51a3c6700 1 -- 172.21.15.179:0/2124423539 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d31e0 con 0x55c393498800 2024-05-08T22:41:41.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51a3c6700 1 -- 172.21.15.179:0/2124423539 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3931ee900 con 0x55c393498800 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51a3c6700 1 -- 172.21.15.179:0/2124423539 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d2f20 con 0x55c393498800 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 msgr2=0x55c393249b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 0x55c393249b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55c393f96570 tx=0x55c3932798c0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- 172.21.15.179:0/2124423539 shutdown_connections 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498c00 0x55c39324a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498800 0x55c393249b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- 172.21.15.179:0/2124423539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498400 0x55c393249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- 172.21.15.179:0/2124423539 >> 172.21.15.179:0/2124423539 conn(0x55c3931a2c00 msgr2=0x55c39325f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- 172.21.15.179:0/2124423539 shutdown_connections 2024-05-08T22:41:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- 172.21.15.179:0/2124423539 wait complete. 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 Processor -- start 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- start start 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498400 0x55c39324b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 0x55c39324ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498c00 0x55c39324a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393224500 con 0x55c393498800 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3932243c0 con 0x55c393498400 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff5204159c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c393224280 con 0x55c393498c00 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 0x55c39324ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 0x55c39324ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59880/0 (socket says 172.21.15.179:59880) 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 learned_addr learned my addr 172.21.15.179:0/281697903 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498400 0x55c39324b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.023+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498c00 0x55c39324a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498c00 msgr2=0x55c39324a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c393498c00 0x55c39324a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498400 msgr2=0x55c39324b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498400 0x55c39324b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3931c41a0 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c393498400 0x55c39324b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 0x55c39324ac00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55c3932ab9e0 tx=0x55c3934b2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 -- 172.21.15.179:0/281697903 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d31e0 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 -- 172.21.15.179:0/281697903 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3931ee480 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3931c49c0 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 -- 172.21.15.179:0/281697903 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3931d2f20 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3931c4b60 con 0x55c393498800 2024-05-08T22:41:41.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 4 datascan.init: resolving metadata pool 13 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 -- 172.21.15.179:0/281697903 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55c393202840 con 0x55c393498800 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55c3935e4000 0x55c39324b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff518bc3700 1 -- 172.21.15.179:0/281697903 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x55c3931dca80 con 0x55c393498800 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 4 datascan.main: resolving metadata pool 13 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55c393499000 0x55c39324a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393499400 con 0x55c393499000 2024-05-08T22:41:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.027+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55c3935e4000 0x55c39324b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55c393499000 0x55c39324a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55c393499000 0x55c39324a100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55c3935e4000 0x55c39324b700 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55c3932abbc0 tx=0x55c3934b3c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(1 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c393499400 0x55c3935fc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e4400 con 0x55c393499400 2024-05-08T22:41:41.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c393499400 0x55c3935fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c393499400 0x55c3935fc000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(2 [pgnls start_epoch 100 out=177b] v100'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55c3935e4400 0x55c3935fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e4800 con 0x55c3935e4400 2024-05-08T22:41:41.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55c3935e4400 0x55c3935fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.031+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55c3935e4400 0x55c3935fc580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(3 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e4800 con 0x55c393499000 2024-05-08T22:41:41.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(4 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55c3935e4800 0x55c3935fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e4c00 con 0x55c3935e4800 2024-05-08T22:41:41.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55c3935e4800 0x55c3935fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55c3935e4800 0x55c3935fcb00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(5 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55c3935e4c00 0x55c3935fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5000 con 0x55c3935e4c00 2024-05-08T22:41:41.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55c3935e4c00 0x55c3935fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.035+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55c3935e4c00 0x55c3935fd080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(6 [pgnls start_epoch 100 out=149b] v99'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5000 con 0x55c393499400 2024-05-08T22:41:41.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(7 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5000 con 0x55c3935e4800 2024-05-08T22:41:41.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(8 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55c3935e5000 0x55c3935fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5400 con 0x55c3935e5000 2024-05-08T22:41:41.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55c3935e5000 0x55c3935fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51ebcf700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55c3935e5000 0x55c3935fd600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(9 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5400 con 0x55c3935e4c00 2024-05-08T22:41:41.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(10 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.039+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5400 con 0x55c3935e4400 2024-05-08T22:41:41.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(11 [pgnls start_epoch 100 out=81b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5400 con 0x55c393499400 2024-05-08T22:41:41.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(12 [pgnls start_epoch 100 out=49b] v99'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55c3935e5400 0x55c3935fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a000 con 0x55c3935e5400 2024-05-08T22:41:41.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55c3935e5400 0x55c3935fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55c3935e5400 0x55c3935fe680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(13 [pgnls start_epoch 100 out=77b] v99'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51e3ce700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55c39361a000 0x55c3935fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c39361a000 2024-05-08T22:41:41.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55c39361a000 0x55c3935fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.043+0000 7ff51f3d0700 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55c39361a000 0x55c3935fdb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(14 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c39361a000 2024-05-08T22:41:41.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499000 2024-05-08T22:41:41.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(15 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5400 2024-05-08T22:41:41.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(16 [pgnls start_epoch 100 out=49b] v99'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e4800 2024-05-08T22:41:41.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(17 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499400 2024-05-08T22:41:41.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(18 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.047+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5000 2024-05-08T22:41:41.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(19 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c39361a000 2024-05-08T22:41:41.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(20 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c39361a000 2024-05-08T22:41:41.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5400 2024-05-08T22:41:41.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(21 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5400 2024-05-08T22:41:41.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(22 [pgnls start_epoch 100 out=97b] v99'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5400 2024-05-08T22:41:41.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(23 [pgnls start_epoch 100 out=74b] v95'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499000 2024-05-08T22:41:41.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(24 [pgnls start_epoch 100 out=73b] v99'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.051+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e4c00 2024-05-08T22:41:41.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(25 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499000 2024-05-08T22:41:41.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(26 [pgnls start_epoch 100 out=98b] v99'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5000 2024-05-08T22:41:41.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(27 [pgnls start_epoch 100 out=76b] v99'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5400 2024-05-08T22:41:41.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(28 [pgnls start_epoch 100 out=73b] v99'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499000 2024-05-08T22:41:41.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(29 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.055+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e5000 2024-05-08T22:41:41.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(30 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e4800 2024-05-08T22:41:41.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(31 [pgnls start_epoch 100 out=73b] v95'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c3935e4800 2024-05-08T22:41:41.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 5 ==== osd_op_reply(32 [pgnls start_epoch 100 out=101b] v99'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-08T22:41:41.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393499800 con 0x55c393499400 2024-05-08T22:41:41.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-08T22:41:41.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.059+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a800 con 0x55c393499400 2024-05-08T22:41:41.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-08T22:41:41.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361ac00 con 0x55c393499400 2024-05-08T22:41:41.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 20 datascan.scan_links: injected ino detected 2024-05-08T22:41:41.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-08T22:41:41.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b000 con 0x55c393499400 2024-05-08T22:41:41.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-08T22:41:41.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b400 con 0x55c393499400 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-08T22:41:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b800 con 0x55c3935e4c00 2024-05-08T22:41:41.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-08T22:41:41.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.063+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361bc00 con 0x55c3935e4800 2024-05-08T22:41:41.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-08T22:41:41.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a400 con 0x55c3935e4400 2024-05-08T22:41:41.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a800 con 0x55c393499000 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-08T22:41:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361ac00 con 0x55c3935e5400 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-08T22:41:41.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.067+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393499800 con 0x55c393499000 2024-05-08T22:41:41.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-08T22:41:41.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T22:41:41.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-08T22:41:41.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361bc00 con 0x55c393499000 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-08T22:41:41.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a800 con 0x55c393499000 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-08T22:41:41.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361ac00 con 0x55c3935e4800 2024-05-08T22:41:41.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.071+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 11 ==== osd_op_reply(47 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499400 2024-05-08T22:41:41.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 10 ==== osd_op_reply(48 [pgnls start_epoch 100 out=177b] v100'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4400 2024-05-08T22:41:41.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(49 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 12 ==== osd_op_reply(50 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4800 2024-05-08T22:41:41.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 8 ==== osd_op_reply(51 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.075+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4c00 2024-05-08T22:41:41.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(52 [pgnls start_epoch 100 out=149b] v99'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499400 2024-05-08T22:41:41.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 11 ==== osd_op_reply(53 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4800 2024-05-08T22:41:41.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 9 ==== osd_op_reply(54 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5000 2024-05-08T22:41:41.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(55 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.079+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4c00 2024-05-08T22:41:41.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(56 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4400 2024-05-08T22:41:41.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 5 ==== osd_op_reply(57 [pgnls start_epoch 100 out=81b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499400 2024-05-08T22:41:41.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 12 ==== osd_op_reply(58 [pgnls start_epoch 100 out=49b] v99'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 8 ==== osd_op_reply(59 [pgnls start_epoch 100 out=77b] v99'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c39361a000 2024-05-08T22:41:41.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(60 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c39361a000 2024-05-08T22:41:41.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.083+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 13 ==== osd_op_reply(61 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 9 ==== osd_op_reply(62 [pgnls start_epoch 100 out=49b] v99'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4800 2024-05-08T22:41:41.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 10 ==== osd_op_reply(63 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499400 2024-05-08T22:41:41.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 13 ==== osd_op_reply(64 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5000 2024-05-08T22:41:41.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(65 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.087+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c39361a000 2024-05-08T22:41:41.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(66 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c39361a000 2024-05-08T22:41:41.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 10 ==== osd_op_reply(67 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 11 ==== osd_op_reply(68 [pgnls start_epoch 100 out=97b] v99'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 12 ==== osd_op_reply(69 [pgnls start_epoch 100 out=74b] v95'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 14 ==== osd_op_reply(70 [pgnls start_epoch 100 out=73b] v99'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4c00 2024-05-08T22:41:41.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(71 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.091+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 15 ==== osd_op_reply(72 [pgnls start_epoch 100 out=98b] v99'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5000 2024-05-08T22:41:41.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(73 [pgnls start_epoch 100 out=76b] v99'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5400 2024-05-08T22:41:41.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 13 ==== osd_op_reply(74 [pgnls start_epoch 100 out=73b] v99'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 16 ==== osd_op_reply(75 [pgnls start_epoch 100 out=73b] v99'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.095+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e5000 2024-05-08T22:41:41.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(76 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e5000 2024-05-08T22:41:41.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4800 2024-05-08T22:41:41.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 11 ==== osd_op_reply(77 [pgnls start_epoch 100 out=73b] v95'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c3935e4800 2024-05-08T22:41:41.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 12 ==== osd_op_reply(78 [pgnls start_epoch 100 out=101b] v99'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-08T22:41:41.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499400 2024-05-08T22:41:41.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-08T22:41:41.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393630000 con 0x55c393499400 2024-05-08T22:41:41.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-08T22:41:41.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.099+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393630400 con 0x55c393499400 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 20 datascan.scan_links: injected ino detected 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-08T22:41:41.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393630800 con 0x55c393499400 2024-05-08T22:41:41.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-08T22:41:41.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393630c00 con 0x55c393499400 2024-05-08T22:41:41.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393631000 con 0x55c3935e4c00 2024-05-08T22:41:41.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10d80 con 0x55c3935e4c00 2024-05-08T22:41:41.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-08T22:41:41.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393631400 con 0x55c3935e4800 2024-05-08T22:41:41.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-08T22:41:41.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.103+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393631800 con 0x55c3935e4400 2024-05-08T22:41:41.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff51ebcf700 1 -- 172.21.15.179:0/281697903 <== osd.2 v2:172.21.15.179:6824/26078 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55c393f10fc0 con 0x55c3935e4400 2024-05-08T22:41:41.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-08T22:41:41.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T22:41:41.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-08T22:41:41.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393631c00 con 0x55c393499000 2024-05-08T22:41:41.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-08T22:41:41.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393631400 con 0x55c3935e5400 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-08T22:41:41.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393630000 con 0x55c393499000 2024-05-08T22:41:41.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-08T22:41:41.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T22:41:41.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-08T22:41:41.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.107+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5c00 con 0x55c393499000 2024-05-08T22:41:41.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c3935e5800 con 0x55c393499000 2024-05-08T22:41:41.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-08T22:41:41.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361bc00 con 0x55c3935e4800 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.6 v2:172.21.15.195:6824/26291 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e4800 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-08T22:41:41.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.111+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361a800 con 0x55c393499400 2024-05-08T22:41:41.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T22:41:41.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361ac00 con 0x55c393499400 2024-05-08T22:41:41.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v100'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.115+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b000 con 0x55c393499400 2024-05-08T22:41:41.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T22:41:41.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 20 datascan.scan_links: first is now 2 2024-05-08T22:41:41.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b400 con 0x55c393499400 2024-05-08T22:41:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff51f3d0700 1 -- 172.21.15.179:0/281697903 <== osd.1 v2:172.21.15.179:6808/26069 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v100'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c393f10d80 con 0x55c393499400 2024-05-08T22:41:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T22:41:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 10 datascan.scan_links: updating inotable 2024-05-08T22:41:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.119+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361b800 con 0x55c393499000 2024-05-08T22:41:41.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff5204159c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-08T22:41:41.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff5204159c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-08T22:41:41.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c393499800 con 0x55c393499000 2024-05-08T22:41:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.7 v2:172.21.15.195:6816/26289 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v100'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55c393f10b40 con 0x55c393499000 2024-05-08T22:41:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff5204159c0 10 datascan.scan_links: updating snaptable 2024-05-08T22:41:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.123+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v8 -- 0x55c39361bc00 con 0x55c3935e5400 2024-05-08T22:41:41.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 <== osd.4 v2:172.21.15.195:6800/26287 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55c393f10b40 con 0x55c3935e5400 2024-05-08T22:41:41.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55c3935e4c00 msgr2=0x55c3935fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55c3935e4c00 0x55c3935fd080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c393499400 msgr2=0x55c3935fc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55c393499400 0x55c3935fc000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55c3935e4400 msgr2=0x55c3935fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55c3935e4400 0x55c3935fc580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55c39361a000 msgr2=0x55c3935fdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55c39361a000 0x55c3935fdb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 reap_dead start 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55c3935e5400 msgr2=0x55c3935fe680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55c3935e5400 0x55c3935fe680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 reap_dead start 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55c3935e5000 msgr2=0x55c3935fd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55c3935e5000 0x55c3935fd600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55c3935e4800 msgr2=0x55c3935fcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55c3935e4800 0x55c3935fcb00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55c393499000 msgr2=0x55c39324a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55c393499000 0x55c39324a100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55c3935e4000 msgr2=0x55c39324b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55c3935e4000 0x55c39324b700 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55c3932abbc0 tx=0x55c3934b3c20 comp rx=0 tx=0).stop 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 msgr2=0x55c39324ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/281697903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c393498800 0x55c39324ac00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55c3932ab9e0 tx=0x55c3934b2c00 comp rx=0 tx=0).stop 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff51e3ce700 1 -- 172.21.15.179:0/281697903 reap_dead start 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 shutdown_connections 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 >> 172.21.15.179:0/281697903 conn(0x55c3931a2c00 msgr2=0x55c39325f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 shutdown_connections 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/281697903 wait complete. 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 msgr2=0x55c393248580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x55c3932ab800 tx=0x55c3932797a0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 shutdown_connections 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3931a3000 0x55c393249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c3931a3400 0x55c393248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 --2- 172.21.15.179:0/3220057809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3931a3800 0x55c393248580 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 >> 172.21.15.179:0/3220057809 conn(0x55c3931a3c00 msgr2=0x55c39325f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 shutdown_connections 2024-05-08T22:41:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.127+0000 7ff5204159c0 1 -- 172.21.15.179:0/3220057809 wait complete. 2024-05-08T22:41:41.132 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.157765, result: 2024-05-08T22:41:41.133 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:41:41.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 Processor -- start 2024-05-08T22:41:41.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 -- start start 2024-05-08T22:41:41.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 0x7fe50c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe50c06d4b0 0x7fe50c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c0b22d0 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50c0b19e0 con 0x7fe50c06d4b0 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe512fad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c15e640 con 0x7fe50c142890 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe5127ac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe50c06d4b0 0x7fe50c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe5127ac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe50c06d4b0 0x7fe50c15d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59886/0 (socket says 172.21.15.179:59886) 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe5117aa700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe50c0b2ca0 0x7fe50c06cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36250/0 (socket says 172.21.15.179:36250) 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.259+0000 7fe5127ac700 1 -- 172.21.15.179:0/2807451827 learned_addr learned my addr 172.21.15.179:0/2807451827 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe5127ac700 1 -- 172.21.15.179:0/2807451827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 msgr2=0x7fe50c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe511fab700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 0x7fe50c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe510fa9700 1 -- 172.21.15.179:0/2807451827 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938974497 0 0) 0x7fe50c0b22d0 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe5127ac700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 0x7fe50c0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe5127ac700 1 -- 172.21.15.179:0/2807451827 >> v1:172.21.15.179:6789/0 conn(0x7fe50c0b2ca0 legacy=0x7fe50c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe5127ac700 1 -- 172.21.15.179:0/2807451827 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0b1c80 con 0x7fe50c06d4b0 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe511fab700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 0x7fe50c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:41.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe5127ac700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c06d4b0 0x7fe50c15d830 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe504009e50 tx=0x7fe50400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fe07d14370ede5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe510fa9700 1 -- 172.21.15.179:0/2807451827 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400e070 con 0x7fe50c06d4b0 2024-05-08T22:41:41.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe510fa9700 1 -- 172.21.15.179:0/2807451827 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50400b9c0 con 0x7fe50c06d4b0 2024-05-08T22:41:41.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe510fa9700 1 -- 172.21.15.179:0/2807451827 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400bd70 con 0x7fe50c06d4b0 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 -- 172.21.15.179:0/2807451827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c06d4b0 msgr2=0x7fe50c15d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c06d4b0 0x7fe50c15d830 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe504009e50 tx=0x7fe50400b330 comp rx=0 tx=0).stop 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 -- 172.21.15.179:0/2807451827 shutdown_connections 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c06d4b0 0x7fe50c15d830 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 --2- 172.21.15.179:0/2807451827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c142890 0x7fe50c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 -- 172.21.15.179:0/2807451827 >> 172.21.15.179:0/2807451827 conn(0x7fe50c0cea30 msgr2=0x7fe50c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 -- 172.21.15.179:0/2807451827 shutdown_connections 2024-05-08T22:41:41.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.263+0000 7fe512fad700 1 -- 172.21.15.179:0/2807451827 wait complete. 2024-05-08T22:41:41.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 Processor -- start 2024-05-08T22:41:41.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- start start 2024-05-08T22:41:41.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 0x7fe50c135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 0x7fe50c14fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c12fad0 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1540c0 con 0x7fe50c142890 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1543a0 con 0x7fe50c06d4b0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59892/0 (socket says 172.21.15.179:59892) 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 -- 172.21.15.179:0/520822998 learned_addr learned my addr 172.21.15.179:0/520822998 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe511fab700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 0x7fe50c135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5127ac700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 0x7fe50c14fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 -- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 msgr2=0x7fe50c135000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 0x7fe50c135000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 -- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 msgr2=0x7fe50c14fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 0x7fe50c14fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 -- 172.21.15.179:0/520822998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009b30 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe511fab700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 0x7fe50c135000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5127ac700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 0x7fe50c14fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe5117aa700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c750 tx=0x7fe50800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508004ca0 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508004e20 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c154680 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508007310 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.267+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c154c40 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c154e30 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe508007470 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe4fe7fc700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 0x7fe4f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe508084130 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe511fab700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 0x7fe4f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.271+0000 7fe511fab700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 0x7fe4f0068bf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe500006fd0 tx=0x7fe500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.279+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe5080512d0 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.491+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe50c04e700 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe4fe7fc700 1 -- 172.21.15.179:0/520822998 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe508007720 con 0x7fe50c0b2ca0 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 msgr2=0x7fe4f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 0x7fe4f0068bf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe500006fd0 tx=0x7fe500008040 comp rx=0 tx=0).stop 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 msgr2=0x7fe50c131c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c750 tx=0x7fe50800cb10 comp rx=0 tx=0).stop 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 shutdown_connections 2024-05-08T22:41:41.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe4f0066730 0x7fe4f0068bf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe50c142890 0x7fe50c14fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe50c0b2ca0 0x7fe50c131c80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 --2- 172.21.15.179:0/520822998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe50c06d4b0 0x7fe50c135000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.659+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 >> 172.21.15.179:0/520822998 conn(0x7fe50c0cea30 msgr2=0x7fe50c06b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.663+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 shutdown_connections 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.663+0000 7fe512fad700 1 -- 172.21.15.179:0/520822998 wait complete. 2024-05-08T22:41:41.663 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:41:41.678 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:41:41.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 Processor -- start 2024-05-08T22:41:41.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- start start 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e8006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e800add40 0x7f1e8006b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e80161ac0 con 0x7f1e800ce060 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e800ae550 con 0x7f1e800add40 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8006abb0 con 0x7f1e800cc310 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1e800ce060 0x7f1e800ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36254/0 (socket says 172.21.15.179:36254) 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7ffff700 1 -- 172.21.15.179:0/3476438259 learned_addr learned my addr 172.21.15.179:0/3476438259 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:41.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 --2- 172.21.15.179:0/3476438259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e8006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e84906700 1 --2- 172.21.15.179:0/3476438259 >> v2:172.21.15.195:3300/0 conn(0x7f1e800add40 0x7f1e8006b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443929866 0 0) 0x7f1e80161ac0 con 0x7f1e800ce060 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e6c0035d0 con 0x7f1e800ce060 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 -- 172.21.15.179:0/3476438259 >> v2:172.21.15.195:3300/0 conn(0x7f1e800add40 msgr2=0x7f1e8006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 --2- 172.21.15.179:0/3476438259 >> v2:172.21.15.195:3300/0 conn(0x7f1e800add40 0x7f1e8006b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 -- 172.21.15.179:0/3476438259 >> v1:172.21.15.179:6789/0 conn(0x7f1e800ce060 legacy=0x7f1e800ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 -- 172.21.15.179:0/3476438259 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800cdaa0 con 0x7f1e800cc310 2024-05-08T22:41:41.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3860187596 0 0) 0x7f1e6c0035d0 con 0x7f1e800ce060 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e77fff700 1 --2- 172.21.15.179:0/3476438259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e8006ce50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009850 tx=0x7f1e68009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c94f27c3b248eee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70001670 con 0x7f1e800cc310 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68004030 con 0x7f1e800cc310 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e7f7fe700 1 -- 172.21.15.179:0/3476438259 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680043e0 con 0x7f1e800cc310 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- 172.21.15.179:0/3476438259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 msgr2=0x7f1e8006ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 --2- 172.21.15.179:0/3476438259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e8006ce50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009850 tx=0x7f1e68009b60 comp rx=0 tx=0).stop 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- 172.21.15.179:0/3476438259 shutdown_connections 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 --2- 172.21.15.179:0/3476438259 >> v2:172.21.15.195:3300/0 conn(0x7f1e800add40 0x7f1e8006b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 --2- 172.21.15.179:0/3476438259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e8006ce50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- 172.21.15.179:0/3476438259 >> 172.21.15.179:0/3476438259 conn(0x7f1e800cb660 msgr2=0x7f1e800cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- 172.21.15.179:0/3476438259 shutdown_connections 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.791+0000 7f1e86369700 1 -- 172.21.15.179:0/3476438259 wait complete. 2024-05-08T22:41:41.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 Processor -- start 2024-05-08T22:41:41.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- start start 2024-05-08T22:41:41.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e80071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 0x7f1e800b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800c4600 con 0x7f1e800add40 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800c48e0 con 0x7f1e800ce060 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800c4bc0 con 0x7f1e800cc310 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59904/0 (socket says 172.21.15.179:59904) 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 -- 172.21.15.179:0/2682643961 learned_addr learned my addr 172.21.15.179:0/2682643961 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e77fff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e80071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e84906700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 0x7f1e800b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 -- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 msgr2=0x7f1e80071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e80071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 -- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 msgr2=0x7f1e800b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:41.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 0x7f1e800b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 -- 172.21.15.179:0/2682643961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680094b0 con 0x7f1e800add40 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e84906700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 0x7f1e800b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7ffff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1e700011e0 tx=0x7f1e7000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70004ca0 con 0x7f1e800add40 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e70004e20 con 0x7f1e800add40 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e800c4ea0 con 0x7f1e800add40 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70007310 con 0x7f1e800add40 2024-05-08T22:41:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.795+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e80071f70 con 0x7f1e800add40 2024-05-08T22:41:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e80067770 con 0x7f1e800add40 2024-05-08T22:41:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e70007470 con 0x7f1e800add40 2024-05-08T22:41:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e7d7fa700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 0x7f1e60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f1e700841b0 con 0x7f1e800add40 2024-05-08T22:41:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e77fff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 0x7f1e60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.799+0000 7f1e77fff700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 0x7f1e60068bf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:41.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:41.803+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e70051230 con 0x7f1e800add40 2024-05-08T22:41:42.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.011+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e80073290 con 0x7f1e800add40 2024-05-08T22:41:42.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.011+0000 7f1e7d7fa700 1 -- 172.21.15.179:0/2682643961 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f1e70007720 con 0x7f1e800add40 2024-05-08T22:41:42.013 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:42.013 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:41.660005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:41:42.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 msgr2=0x7f1e60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:42.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 0x7f1e60068bf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68008030 comp rx=0 tx=0).stop 2024-05-08T22:41:42.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 msgr2=0x7f1e8015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1e700011e0 tx=0x7f1e7000c9e0 comp rx=0 tx=0).stop 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 shutdown_connections 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1e60066730 0x7f1e60068bf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e800ce060 0x7f1e800b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e800cc310 0x7f1e80071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 --2- 172.21.15.179:0/2682643961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e800add40 0x7f1e8015eb70 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 >> 172.21.15.179:0/2682643961 conn(0x7f1e800cb660 msgr2=0x7f1e800af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 shutdown_connections 2024-05-08T22:41:42.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:42.015+0000 7f1e86369700 1 -- 172.21.15.179:0/2682643961 wait complete. 2024-05-08T22:41:42.017 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 65 2024-05-08T22:41:42.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:41:41.660005+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/249337448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 249337448}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 249337448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:41:42.431 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8499:up:replay 2024-05-08T22:41:43.432 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:41:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 Processor -- start 2024-05-08T22:41:43.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 -- start start 2024-05-08T22:41:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d70161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2d7006de70 0x7f2d7006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d70165d50 con 0x7f2d70164db0 2024-05-08T22:41:43.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7006ad70 con 0x7f2d7006de70 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70165230 con 0x7f2d7015ef80 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6e7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2d70164db0 0x7f2d7006d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36264/0 (socket says 172.21.15.179:36264) 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6e7fc700 1 -- 172.21.15.179:0/3681526555 learned_addr learned my addr 172.21.15.179:0/3681526555 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6f7fe700 1 --2- 172.21.15.179:0/3681526555 >> v2:172.21.15.195:3300/0 conn(0x7f2d7006de70 0x7f2d7006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6effd700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d70161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214403416 0 0) 0x7f2d70165d50 con 0x7f2d70164db0 2024-05-08T22:41:43.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d5c0035d0 con 0x7f2d70164db0 2024-05-08T22:41:43.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953535720 0 0) 0x7f2d5c0035d0 con 0x7f2d70164db0 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 msgr2=0x7f2d70161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d70161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 >> v2:172.21.15.195:3300/0 conn(0x7f2d7006de70 msgr2=0x7f2d7006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 --2- 172.21.15.179:0/3681526555 >> v2:172.21.15.195:3300/0 conn(0x7f2d7006de70 0x7f2d7006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7006e4b0 con 0x7f2d70164db0 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f2d580019f0 con 0x7f2d70164db0 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 >> v1:172.21.15.179:6789/0 conn(0x7f2d70164db0 legacy=0x7f2d7006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5c003a20 0x7f2d5c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.555+0000 7f2d6dffb700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d5c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c004db0 0x7f2d5c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7006ad70 con 0x7f2d5c003a20 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70165230 con 0x7f2d5c004db0 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00ba30 con 0x7f2d7015ef80 2024-05-08T22:41:43.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5c003a20 0x7f2d5c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6e7fc700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d5c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6f7fe700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c004db0 0x7f2d5c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 -- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 msgr2=0x7f2d5c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d7015ef80 0x7f2d5c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 -- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c004db0 msgr2=0x7f2d5c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c004db0 0x7f2d5c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 -- 172.21.15.179:0/3681526555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0035d0 con 0x7f2d5c003a20 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6effd700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5c003a20 0x7f2d5c003e40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009b70 tx=0x7f2d5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403c10004d8de87c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000e6c0 con 0x7f2d5c003a20 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000e820 con 0x7f2d5c003a20 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6dffb700 1 -- 172.21.15.179:0/3681526555 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000eb90 con 0x7f2d5c003a20 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 -- 172.21.15.179:0/3681526555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5c003a20 msgr2=0x7f2d5c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/3681526555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5c003a20 0x7f2d5c003e40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009b70 tx=0x7f2d5c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6f7fe700 1 -- 172.21.15.179:0/3681526555 reap_dead start 2024-05-08T22:41:43.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 -- 172.21.15.179:0/3681526555 shutdown_connections 2024-05-08T22:41:43.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 -- 172.21.15.179:0/3681526555 >> 172.21.15.179:0/3681526555 conn(0x7f2d700ce9d0 msgr2=0x7f2d700cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:43.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 -- 172.21.15.179:0/3681526555 shutdown_connections 2024-05-08T22:41:43.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 -- 172.21.15.179:0/3681526555 wait complete. 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.559+0000 7f2d6ffff700 1 Processor -- start 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- start start 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c003a20 0x7f2d7014cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d700cd5f0 0x7f2d700cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700ce170 con 0x7f2d70163810 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701641d0 con 0x7f2d5c003a20 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701f78a0 con 0x7f2d700cd5f0 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59928/0 (socket says 172.21.15.179:59928) 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 -- 172.21.15.179:0/2486238469 learned_addr learned my addr 172.21.15.179:0/2486238469 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6effd700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c003a20 0x7f2d7014cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6e7fc700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d700cd5f0 0x7f2d700cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 -- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d700cd5f0 msgr2=0x7f2d700cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d700cd5f0 0x7f2d700cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 -- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c003a20 msgr2=0x7f2d7014cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c003a20 0x7f2d7014cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 -- 172.21.15.179:0/2486238469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d701f7d30 con 0x7f2d70163810 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6f7fe700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2d700b2970 tx=0x7f2d68010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68004300 con 0x7f2d70163810 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d70157ac0 con 0x7f2d70163810 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d68004460 con 0x7f2d70163810 2024-05-08T22:41:43.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680045d0 con 0x7f2d70163810 2024-05-08T22:41:43.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.563+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d701f7250 con 0x7f2d70163810 2024-05-08T22:41:43.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70067740 con 0x7f2d70163810 2024-05-08T22:41:43.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d68003900 con 0x7f2d70163810 2024-05-08T22:41:43.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d677fe700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 0x7f2d50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:43.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f2d68084570 con 0x7f2d70163810 2024-05-08T22:41:43.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d6effd700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 0x7f2d50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:43.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.567+0000 7f2d6effd700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 0x7f2d50068bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d600090b0 tx=0x7f2d60008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:43.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.571+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d68051600 con 0x7f2d70163810 2024-05-08T22:41:43.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d7004e700 con 0x7f2d70163810 2024-05-08T22:41:43.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d677fe700 1 -- 172.21.15.179:0/2486238469 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f2d68054c20 con 0x7f2d70163810 2024-05-08T22:41:43.784 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:43.784 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:43.665771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:41:43.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 msgr2=0x7f2d50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 0x7f2d50068bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d600090b0 tx=0x7f2d60008030 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 msgr2=0x7f2d70163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.783+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2d700b2970 tx=0x7f2d68010b80 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 shutdown_connections 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2d50066730 0x7f2d50068bf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70163810 0x7f2d70163c90 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d700cd5f0 0x7f2d700cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 --2- 172.21.15.179:0/2486238469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d5c003a20 0x7f2d7014cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 >> 172.21.15.179:0/2486238469 conn(0x7f2d700ce9d0 msgr2=0x7f2d700af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 shutdown_connections 2024-05-08T22:41:43.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:43.787+0000 7f2d6ffff700 1 -- 172.21.15.179:0/2486238469 wait complete. 2024-05-08T22:41:43.788 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 67 2024-05-08T22:41:44.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:41:43.665771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.179:6835/249337448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 249337448}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 249337448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:41:44.202 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8499:up:rejoin 2024-05-08T22:41:45.203 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:41:45.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 Processor -- start 2024-05-08T22:41:45.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 -- start start 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5e700cde70 0x7f5e700ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e7006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e700ce810 con 0x7f5e7015d5b0 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7006afc0 con 0x7f5e700cde70 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e750eb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015d990 con 0x7f5e7006b450 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5e7015d5b0 0x7f5e700b1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57998/0 (socket says 172.21.15.179:57998) 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6ffff700 1 -- 172.21.15.179:0/2661708489 learned_addr learned my addr 172.21.15.179:0/2661708489 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e748ea700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e7006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/2661708489 >> v2:172.21.15.195:3300/0 conn(0x7f5e700cde70 0x7f5e700ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105108851 0 0) 0x7f5e700ce810 con 0x7f5e7015d5b0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e5c0035d0 con 0x7f5e7015d5b0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088395951 0 0) 0x7f5e5c0035d0 con 0x7f5e7015d5b0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 msgr2=0x7f5e7006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e7006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 >> v2:172.21.15.195:3300/0 conn(0x7f5e700cde70 msgr2=0x7f5e700ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 --2- 172.21.15.179:0/2661708489 >> v2:172.21.15.195:3300/0 conn(0x7f5e700cde70 0x7f5e700ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700d26f0 con 0x7f5e7015d5b0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f5e60001670 con 0x7f5e7015d5b0 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 >> v1:172.21.15.179:6789/0 conn(0x7f5e7015d5b0 legacy=0x7f5e700b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e5c003a20 0x7f5e5c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c004870 0x7f5e5c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e5c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7006afc0 con 0x7f5e5c004870 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015d990 con 0x7f5e5c003a20 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.327+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c00ba30 con 0x7f5e7006b450 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c004870 0x7f5e5c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6ffff700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e5c003a20 0x7f5e5c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e748ea700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e5c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 -- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 msgr2=0x7f5e5c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e7006b450 0x7f5e5c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 -- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e5c003a20 msgr2=0x7f5e5c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e5c003a20 0x7f5e5c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 -- 172.21.15.179:0/2661708489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0035d0 con 0x7f5e5c004870 2024-05-08T22:41:45.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c004870 0x7f5e5c004cf0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009b70 tx=0x7f5e5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c15859b21989681 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:45.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800e6c0 con 0x7f5e5c004870 2024-05-08T22:41:45.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5800e820 con 0x7f5e5c004870 2024-05-08T22:41:45.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e6effd700 1 -- 172.21.15.179:0/2661708489 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800eb90 con 0x7f5e5c004870 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 -- 172.21.15.179:0/2661708489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c004870 msgr2=0x7f5e5c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 --2- 172.21.15.179:0/2661708489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c004870 0x7f5e5c004cf0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009b70 tx=0x7f5e5c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e748ea700 1 -- 172.21.15.179:0/2661708489 reap_dead start 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 -- 172.21.15.179:0/2661708489 shutdown_connections 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 -- 172.21.15.179:0/2661708489 >> 172.21.15.179:0/2661708489 conn(0x7f5e700b4940 msgr2=0x7f5e700b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 -- 172.21.15.179:0/2661708489 shutdown_connections 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.331+0000 7f5e750eb700 1 -- 172.21.15.179:0/2661708489 wait complete. 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 Processor -- start 2024-05-08T22:41:45.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- start start 2024-05-08T22:41:45.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e701f5e30 0x7f5e7014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e701f6210 0x7f5e70160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e701f6690 con 0x7f5e7014ba80 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7014c470 con 0x7f5e701f6210 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e700a97b0 con 0x7f5e701f5e30 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50686/0 (socket says 172.21.15.179:50686) 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 -- 172.21.15.179:0/1903675797 learned_addr learned my addr 172.21.15.179:0/1903675797 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e6ffff700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e701f5e30 0x7f5e7014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e6f7fe700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e701f6210 0x7f5e70160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 -- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e701f5e30 msgr2=0x7f5e7014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e701f5e30 0x7f5e7014c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 -- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e701f6210 msgr2=0x7f5e70160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e701f6210 0x7f5e70160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 -- 172.21.15.179:0/1903675797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70149bc0 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e748ea700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5e700d0990 tx=0x7f5e68011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800e040 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800f040 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800fc40 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e700a9ce0 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.335+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e70149e60 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e68022460 con 0x7f5e7014ba80 2024-05-08T22:41:45.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e70149a20 con 0x7f5e7014ba80 2024-05-08T22:41:45.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e6cff9700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 0x7f5e50068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:45.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f5e68084780 con 0x7f5e7014ba80 2024-05-08T22:41:45.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e6ffff700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 0x7f5e50068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:45.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.339+0000 7f5e6ffff700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 0x7f5e50068c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e60001cf0 tx=0x7f5e600079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:45.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.347+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5e70149a20 con 0x7f5e7014ba80 2024-05-08T22:41:45.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.551+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e70149a20 con 0x7f5e7014ba80 2024-05-08T22:41:45.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e6cff9700 1 -- 172.21.15.179:0/1903675797 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f5e70149a20 con 0x7f5e7014ba80 2024-05-08T22:41:45.554 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:41:45.554 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 msgr2=0x7f5e50068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 0x7f5e50068c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e60001cf0 tx=0x7f5e600079a0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 msgr2=0x7f5e7014bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5e700d0990 tx=0x7f5e68011ed0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 shutdown_connections 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e50066760 0x7f5e50068c20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e701f6210 0x7f5e70160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e7014ba80 0x7f5e7014bf30 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 --2- 172.21.15.179:0/1903675797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e701f5e30 0x7f5e7014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 >> 172.21.15.179:0/1903675797 conn(0x7f5e700b4940 msgr2=0x7f5e700ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 shutdown_connections 2024-05-08T22:41:45.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:45.555+0000 7f5e750eb700 1 -- 172.21.15.179:0/1903675797 wait complete. 2024-05-08T22:41:45.558 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:41:45.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:40:01.140896+0000', 'modified': '2024-05-08T22:41:44.668492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.179:6835/249337448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 249337448}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 249337448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:41:45.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:41:45.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:41:46.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 Processor -- start 2024-05-08T22:41:46.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 -- start start 2024-05-08T22:41:46.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47bc150bc0 0x7f47bc15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47bc0ad740 0x7f47bc0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc0ae0d0 con 0x7f47bc15d840 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0b2930 con 0x7f47bc150bc0 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47c0ffd700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0b1cb0 con 0x7f47bc0ad740 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47bc150bc0 0x7f47bc15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47bc150bc0 0x7f47bc15ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50696/0 (socket says 172.21.15.179:50696) 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 -- 172.21.15.179:0/859935926 learned_addr learned my addr 172.21.15.179:0/859935926 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47baffd700 1 --2- 172.21.15.179:0/859935926 >> v2:172.21.15.195:3300/0 conn(0x7f47bc0ad740 0x7f47bc0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 -- 172.21.15.179:0/859935926 >> v2:172.21.15.195:3300/0 conn(0x7f47bc0ad740 msgr2=0x7f47bc0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/859935926 >> v2:172.21.15.195:3300/0 conn(0x7f47bc0ad740 0x7f47bc0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 -- 172.21.15.179:0/859935926 >> v1:172.21.15.195:6789/0 conn(0x7f47bc15d840 legacy=0x7f47bc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 -- 172.21.15.179:0/859935926 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0b1f90 con 0x7f47bc150bc0 2024-05-08T22:41:46.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.099+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/859935926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc15ac70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009910 tx=0x7f47b0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f3d1ad44f3e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47ba7fc700 1 -- 172.21.15.179:0/859935926 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ba20 con 0x7f47bc150bc0 2024-05-08T22:41:46.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47ba7fc700 1 -- 172.21.15.179:0/859935926 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000bb80 con 0x7f47bc150bc0 2024-05-08T22:41:46.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47ba7fc700 1 -- 172.21.15.179:0/859935926 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0009e60 con 0x7f47bc150bc0 2024-05-08T22:41:46.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- 172.21.15.179:0/859935926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 msgr2=0x7f47bc15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/859935926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc15ac70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009910 tx=0x7f47b0009c20 comp rx=0 tx=0).stop 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- 172.21.15.179:0/859935926 shutdown_connections 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/859935926 >> v2:172.21.15.195:3300/0 conn(0x7f47bc0ad740 0x7f47bc0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/859935926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc15ac70 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- 172.21.15.179:0/859935926 >> 172.21.15.179:0/859935926 conn(0x7f47bc0ce960 msgr2=0x7f47bc0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- 172.21.15.179:0/859935926 shutdown_connections 2024-05-08T22:41:46.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- 172.21.15.179:0/859935926 wait complete. 2024-05-08T22:41:46.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 Processor -- start 2024-05-08T22:41:46.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.103+0000 7f47c0ffd700 1 -- start start 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47bc0ad740 0x7f47bc1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 0x7f47bc158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc147110 con 0x7f47bc150bc0 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc1473f0 con 0x7f47bc15d840 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc152090 con 0x7f47bc0ad740 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50708/0 (socket says 172.21.15.179:50708) 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 -- 172.21.15.179:0/2454739161 learned_addr learned my addr 172.21.15.179:0/2454739161 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47bc0ad740 0x7f47bc1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47bbfff700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 0x7f47bc158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 -- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47bc0ad740 msgr2=0x7f47bc1468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47bc0ad740 0x7f47bc1468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 -- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 msgr2=0x7f47bc158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 0x7f47bc158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 -- 172.21.15.179:0/2454739161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0009570 con 0x7f47bc150bc0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47bbfff700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 0x7f47bc158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47baffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f47ac007ec0 tx=0x7f47ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47a3fff700 1 -- 172.21.15.179:0/2454739161 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00eea0 con 0x7f47bc150bc0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47a3fff700 1 -- 172.21.15.179:0/2454739161 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac014420 con 0x7f47bc150bc0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc152370 con 0x7f47bc150bc0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47a3fff700 1 -- 172.21.15.179:0/2454739161 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0147b0 con 0x7f47bc150bc0 2024-05-08T22:41:46.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.107+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc144680 con 0x7f47bc150bc0 2024-05-08T22:41:46.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.111+0000 7f47a3fff700 1 -- 172.21.15.179:0/2454739161 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47ac014910 con 0x7f47bc150bc0 2024-05-08T22:41:46.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.111+0000 7f47a3fff700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 0x7f47a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.111+0000 7f47a3fff700 1 -- 172.21.15.179:0/2454739161 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f47ac084cf0 con 0x7f47bc150bc0 2024-05-08T22:41:46.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.111+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 0x7f47a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 0x7f47a8068bf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 Processor -- start 2024-05-08T22:41:46.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- start start 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2ba06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 0x2b681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2abed80 con 0x2b973f0 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2abc510 con 0x2ba0310 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ab7e00 con 0x2ba0c30 2024-05-08T22:41:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50724/0 (socket says 172.21.15.179:50724) 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 -- 172.21.15.179:0/2123485451 learned_addr learned my addr 172.21.15.179:0/2123485451 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2ba06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47bbfff700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 0x2b681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 -- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 msgr2=0x2b681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 0x2b681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 -- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 msgr2=0x2ba06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2ba06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 -- 172.21.15.179:0/2123485451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x292a960 con 0x2b973f0 2024-05-08T22:41:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47bbfff700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 0x2b681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:46.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47baffd700 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f47ac055690 tx=0x7f47ac056160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939738d58bce1126 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47a17fa700 1 -- 172.21.15.179:0/2123485451 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0597c0 con 0x2b973f0 2024-05-08T22:41:46.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47a17fa700 1 -- 172.21.15.179:0/2123485451 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac05de10 con 0x2b973f0 2024-05-08T22:41:46.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47a17fa700 1 -- 172.21.15.179:0/2123485451 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac06d1c0 con 0x2b973f0 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- 172.21.15.179:0/2123485451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 msgr2=0x2b67c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f47ac055690 tx=0x7f47ac056160 comp rx=0 tx=0).stop 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- 172.21.15.179:0/2123485451 shutdown_connections 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0c30 0x2b681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b973f0 0x2b67c80 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 --2- 172.21.15.179:0/2123485451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2ba06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- 172.21.15.179:0/2123485451 >> 172.21.15.179:0/2123485451 conn(0x28df4d0 msgr2=0x2b81a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- 172.21.15.179:0/2123485451 shutdown_connections 2024-05-08T22:41:46.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.115+0000 7f47c550e740 1 -- 172.21.15.179:0/2123485451 wait complete. 2024-05-08T22:41:46.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- start start 2024-05-08T22:41:46.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 Processor -- start 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2b96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b4f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a75730 con 0x2ba0c30 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a7b5b0 con 0x2ba0310 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2963aa0 con 0x2b973f0 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 10 client.0 _ll_register_callbacks cb 0x2ab1610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50734/0 (socket says 172.21.15.179:50734) 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 -- 172.21.15.179:0/3867287203 learned_addr learned my addr 172.21.15.179:0/3867287203 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2b96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47baffd700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 -- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 msgr2=0x2b96b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2b96b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 -- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 msgr2=0x2b4f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b4f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 -- 172.21.15.179:0/3867287203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc144680 con 0x2ba0c30 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2b96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005490 tx=0x7f47b40104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1400e1243f18f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f479a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4010ee0 con 0x2ba0c30 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f47b4015460 con 0x2ba0c30 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28ec1e0 con 0x2ba0c30 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4015620 con 0x2ba0c30 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 10 client.8580 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:46.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.119+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x2a7e220 con 0x2ba0c30 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28ec1e0 con 0x2ba0c30 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f47b4015780 con 0x2ba0c30 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 10 client.8580 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 10 client.8580 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 20 client.8580 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b51d20 0x2b98c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 4 client.8580 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2936460 con 0x2b51d20 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b51d20 0x2b98c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b51d20 0x2b98c10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e769f0279a87616d server_cookie=113871bd5c852c98 in_seq=0 out_seq=0 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 10 client.8580 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 1 -- 172.21.15.179:0/3867287203 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2936460 con 0x2b51d20 2024-05-08T22:41:46.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 10 client.8580 handle_command_reply: tid=0 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 client.8580 shutdown 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 msgr2=0x2b512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005490 tx=0x7f47b40104c0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 shutdown_connections 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b4f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b51d20 0x2b98c10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b512f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 --2- 172.21.15.179:0/3867287203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2b96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f479a7fc700 0 client.8580 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 >> 172.21.15.179:0/3867287203 conn(0x28df4d0 msgr2=0x2b81210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 shutdown_connections 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 1 -- 172.21.15.179:0/3867287203 wait complete. 2024-05-08T22:41:46.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.123+0000 7f47c550e740 20 client.8580 trim_cache size 0 max 16384 2024-05-08T22:41:46.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 Processor -- start 2024-05-08T22:41:46.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 -- start start 2024-05-08T22:41:46.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b973f0 0x2bfcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0310 0x2bfd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a818d0 con 0x2ba0c30 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a824f0 con 0x2b973f0 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.143+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ae6ec0 con 0x2ba0310 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50748/0 (socket says 172.21.15.179:50748) 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 -- 172.21.15.179:0/3146212505 learned_addr learned my addr 172.21.15.179:0/3146212505 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b973f0 0x2bfcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47baffd700 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0310 0x2bfd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 -- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0310 msgr2=0x2bfd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0310 0x2bfd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 -- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b973f0 msgr2=0x2bfcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b973f0 0x2bfcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 -- 172.21.15.179:0/3146212505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2a64450 con 0x2ba0c30 2024-05-08T22:41:46.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000fb0 tx=0x7f47b400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec34340efdbe7dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47a1ffb700 1 -- 172.21.15.179:0/3146212505 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4015c90 con 0x2ba0c30 2024-05-08T22:41:46.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47a1ffb700 1 -- 172.21.15.179:0/3146212505 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4010ce0 con 0x2ba0c30 2024-05-08T22:41:46.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47a1ffb700 1 -- 172.21.15.179:0/3146212505 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4014600 con 0x2ba0c30 2024-05-08T22:41:46.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- 172.21.15.179:0/3146212505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 msgr2=0x2b69a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000fb0 tx=0x7f47b400e000 comp rx=0 tx=0).stop 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- 172.21.15.179:0/3146212505 shutdown_connections 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b69a20 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2ba0310 0x2bfd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- 172.21.15.179:0/3146212505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b973f0 0x2bfcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- 172.21.15.179:0/3146212505 >> 172.21.15.179:0/3146212505 conn(0x28df4d0 msgr2=0x2baffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- 172.21.15.179:0/3146212505 shutdown_connections 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- 172.21.15.179:0/3146212505 wait complete. 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- start start 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 Processor -- start 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2bbf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b9f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bbe010 con 0x2ba0c30 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b6b430 con 0x2ba0310 2024-05-08T22:41:46.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b99c90 con 0x2b973f0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47c550e740 10 client.0 _ll_register_callbacks cb 0x7f47ac051cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.147+0000 7f47bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50762/0 (socket says 172.21.15.179:50762) 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 -- 172.21.15.179:0/430544182 learned_addr learned my addr 172.21.15.179:0/430544182 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2bbf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47baffd700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b9f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 -- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 msgr2=0x2bbf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2bbf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 -- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 msgr2=0x2b9f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b9f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 -- 172.21.15.179:0/430544182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a7e220 con 0x2ba0c30 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2bbf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bbfff700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003c20 tx=0x7f47b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e755368df9902e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:46.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4010e50 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f47b4015670 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47ac05de10 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4015830 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 10 client.8586 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x2a58720 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac05de10 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f47b401d070 con 0x2ba0c30 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 10 client.8586 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 10 client.8586 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 20 client.8586 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2bfd9a0 0x2bfbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 4 client.8586 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:41:46.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "status"}) v1 -- 0x295b1e0 con 0x2bfd9a0 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2bfd9a0 0x2bfbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 10 client.8586 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47bb7fe700 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2bfd9a0 0x2bfbbd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8103862f18b12677 server_cookie=f5b44aa1c85246ff in_seq=0 out_seq=0 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 1 -- 172.21.15.179:0/430544182 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x295b1e0 con 0x2bfd9a0 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 10 client.8586 handle_command_reply: tid=0 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 client.8586 shutdown 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 msgr2=0x2b9e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003c20 tx=0x7f47b4000ed0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 shutdown_connections 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2ba0310 0x2b9f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2bfd9a0 0x2bfbbd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2ba0c30 0x2b9e380 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 --2- 172.21.15.179:0/430544182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b973f0 0x2bbf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 >> 172.21.15.179:0/430544182 conn(0x28df4d0 msgr2=0x2b94d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f479a7fc700 0 client.8586 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 shutdown_connections 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 1 -- 172.21.15.179:0/430544182 wait complete. 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.151+0000 7f47c550e740 20 client.8586 trim_cache size 0 max 16384 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 8499, 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:41:46.159 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.4864600330000002, 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 68, 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 100, 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 100, 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 5.4533454969999999 2024-05-08T22:41:46.160 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:41:46.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 msgr2=0x7f47a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 0x7f47a8068bf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b0008030 comp rx=0 tx=0).stop 2024-05-08T22:41:46.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 msgr2=0x7f47bc144140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f47ac007ec0 tx=0x7f47ac00e3c0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 shutdown_connections 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47a8066730 0x7f47a8068bf0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47bc15d840 0x7f47bc158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47bc150bc0 0x7f47bc144140 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 --2- 172.21.15.179:0/2454739161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47bc0ad740 0x7f47bc1468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 >> 172.21.15.179:0/2454739161 conn(0x7f47bc0ce960 msgr2=0x7f47bc0cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 shutdown_connections 2024-05-08T22:41:46.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.155+0000 7f47c0ffd700 1 -- 172.21.15.179:0/2454739161 wait complete. 2024-05-08T22:41:46.170 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:41:46.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 Processor -- start 2024-05-08T22:41:46.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 -- start start 2024-05-08T22:41:46.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f00c9940 0x7fa9f015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f0158f60 con 0x7fa9f00c8d70 2024-05-08T22:41:46.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f006b860 con 0x7fa9f0158b80 2024-05-08T22:41:46.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f62e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f006ac50 con 0x7fa9f00c9940 2024-05-08T22:41:46.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9ef7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa9f00c8d70 0x7fa9f00c9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:58008/0 (socket says 172.21.15.179:58008) 2024-05-08T22:41:46.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9ef7fe700 1 -- 172.21.15.179:0/1022652842 learned_addr learned my addr 172.21.15.179:0/1022652842 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:41:46.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f00c9940 0x7fa9f015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872168010 0 0) 0x7fa9f0158f60 con 0x7fa9f00c8d70 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d80035d0 con 0x7fa9f00c8d70 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270765689 0 0) 0x7fa9f006b860 con 0x7fa9f0158b80 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f0158f60 con 0x7fa9f0158b80 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3247264461 0 0) 0x7fa9d80035d0 con 0x7fa9f00c8d70 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f00c9940 msgr2=0x7fa9f015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.287+0000 7fa9eeffd700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f00c9940 0x7fa9f015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 >> v1:172.21.15.195:6789/0 conn(0x7fa9f0158b80 legacy=0x7fa9f006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f006af30 con 0x7fa9f00c8d70 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fa9e00019f0 con 0x7fa9f00c8d70 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f00c9940 0x7fa9f015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:46.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 >> v1:172.21.15.179:6789/0 conn(0x7fa9f00c8d70 legacy=0x7fa9f00c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9d8003be0 0x7fa9d8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d8004870 0x7fa9d8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9d800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f006ac50 con 0x7fa9d8005230 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800bf70 con 0x7fa9d8004870 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800c0e0 con 0x7fa9d8003be0 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9d800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9effff700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9d8003be0 0x7fa9d8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9ef7fe700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d8004870 0x7fa9d8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 -- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9d8003be0 msgr2=0x7fa9d8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9d8003be0 0x7fa9d8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 -- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d8004870 msgr2=0x7fa9d8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d8004870 0x7fa9d8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 -- 172.21.15.179:0/1022652842 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80094b0 con 0x7fa9d8005230 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9d800ba30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009cf0 tx=0x7fa9d80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cd26c5f4274cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8004030 con 0x7fa9d8005230 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8004190 con 0x7fa9d8005230 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9eeffd700 1 -- 172.21.15.179:0/1022652842 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8004520 con 0x7fa9d8005230 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1022652842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 msgr2=0x7fa9d800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1022652842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9d800ba30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009cf0 tx=0x7fa9d80058a0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f4886700 1 -- 172.21.15.179:0/1022652842 reap_dead start 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1022652842 shutdown_connections 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1022652842 >> 172.21.15.179:0/1022652842 conn(0x7fa9f00cb660 msgr2=0x7fa9f00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1022652842 shutdown_connections 2024-05-08T22:41:46.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.291+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1022652842 wait complete. 2024-05-08T22:41:46.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 Processor -- start 2024-05-08T22:41:46.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 -- start start 2024-05-08T22:41:46.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9f00c26b0 0x7fa9f01f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f009ebf0 0x7fa9f009f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f009f590 con 0x7fa9d8005230 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0156850 con 0x7fa9f00c26b0 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f62e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01f37b0 con 0x7fa9f009ebf0 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50782/0 (socket says 172.21.15.179:50782) 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 -- 172.21.15.179:0/1421345919 learned_addr learned my addr 172.21.15.179:0/1421345919 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9ef7fe700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9f00c26b0 0x7fa9f01f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9f4886700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f009ebf0 0x7fa9f009f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 -- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f009ebf0 msgr2=0x7fa9f009f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f009ebf0 0x7fa9f009f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 -- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9f00c26b0 msgr2=0x7fa9f01f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9f00c26b0 0x7fa9f01f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 -- 172.21.15.179:0/1421345919 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f009cf20 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.295+0000 7fa9effff700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400f7a0 tx=0x7fa9e400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4005320 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e40054a0 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f009d200 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4006ac0 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f009d7c0 con 0x7fa9d8005230 2024-05-08T22:41:46.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f0067770 con 0x7fa9d8005230 2024-05-08T22:41:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa9e4006c20 con 0x7fa9d8005230 2024-05-08T22:41:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 0x7fa9dc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fa9e4083ab0 con 0x7fa9d8005230 2024-05-08T22:41:46.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.299+0000 7fa9ef7fe700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 0x7fa9dc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:46.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.303+0000 7fa9ef7fe700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 0x7fa9dc068c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0157cb0 tx=0x7fa9e0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:46.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.307+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa9e4050c50 con 0x7fa9d8005230 2024-05-08T22:41:46.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.515+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9f006f000 con 0x7fa9d8005230 2024-05-08T22:41:46.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.515+0000 7fa9ecff9700 1 -- 172.21.15.179:0/1421345919 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9e401a070 con 0x7fa9d8005230 2024-05-08T22:41:46.517 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 msgr2=0x7fa9dc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 0x7fa9dc068c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0157cb0 tx=0x7fa9e0006040 comp rx=0 tx=0).stop 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 msgr2=0x7fa9f009dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400f7a0 tx=0x7fa9e400fab0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 shutdown_connections 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa9dc066760 0x7fa9dc068c20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9f00c26b0 0x7fa9f01f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9f009ebf0 0x7fa9f009f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 --2- 172.21.15.179:0/1421345919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9d8005230 0x7fa9f009dfc0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:46.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 >> 172.21.15.179:0/1421345919 conn(0x7fa9f00cb660 msgr2=0x7fa9f015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:46.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 shutdown_connections 2024-05-08T22:41:46.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:46.519+0000 7fa9f62e9700 1 -- 172.21.15.179:0/1421345919 wait complete. 2024-05-08T22:41:46.532 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:41:46.532 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:41:46.532 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:41:46.533 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:41:46.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.533 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:41:46.539 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:41:46.539 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:41:46.540 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42065sec preferred_lft 42065sec 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:41:46.541 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:41:46.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.542 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:41:46.596 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:41:46.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.597 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:41:46.604 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:41:46.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:41:46.605 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:41:46.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.645 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:41:46.693 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:41:46.693 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:41:46.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:41:46.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.694 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:41:46.706 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:41:46.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:41:46.706 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:46.713 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:41:46.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:41:46.714 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:46.764 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:41:46.780 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:41:46.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:41:46.790 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:41:46.791 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:41:46.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.791 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:41:46.798 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:41:46.798 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:41:46.801 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:41:46.817 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:41:46.817 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:41:46.873 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:41:46.874 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:41:46.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:46.875 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:41:46.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:41:46.891+0000 7f2250406640 -1 init, newargv = 0x55592748a990 newargc=15 2024-05-08T22:41:46.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[47869]: starting ceph client 2024-05-08T22:41:46.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[47869]: starting fuse 2024-05-08T22:41:47.983 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:41:48.001 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:41:48.002 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:41:48.061 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:41:48.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:41:48.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:48.063 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:41:48.068 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:41:48.068 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:41:48.068 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:41:48.069 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:41:48.070 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:41:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.47869.asok 2024-05-08T22:41:48.337 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47869.asok 2024-05-08T22:41:48.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:48.337 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47869.asok status 2024-05-08T22:41:48.485 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "47869", 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:41:48.486 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 8598, 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 8598 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 874098999 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:41:48.487 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 874098999 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.8598 192.168.0.1:0/874098999", 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/874098999", 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 68, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 100, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:41:48.488 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:41:48.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:48.496 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:41:48.496 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:48.501 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:41:48.501 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:48.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:41:48.502 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:41:48.512 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-08T22:41:48.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.635+0000 7f7269eb9700 1 Processor -- start 2024-05-08T22:41:48.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.635+0000 7f7269eb9700 1 -- start start 2024-05-08T22:41:48.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72640c8dd0 0x7f72640c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f72640b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640c9770 con 0x7f7264160860 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7264160c40 con 0x7f72640c8dd0 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72640cd730 con 0x7f72640b38b0 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72640c8dd0 0x7f72640c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72640c8dd0 0x7f72640c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50786/0 (socket says 172.21.15.179:50786) 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 -- 172.21.15.179:0/2448013011 learned_addr learned my addr 172.21.15.179:0/2448013011 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f72696b8700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f72640b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 -- 172.21.15.179:0/2448013011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 msgr2=0x7f72640b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f72640b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 -- 172.21.15.179:0/2448013011 >> v1:172.21.15.195:6789/0 conn(0x7f7264160860 legacy=0x7f72640b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 -- 172.21.15.179:0/2448013011 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640cda10 con 0x7f72640c8dd0 2024-05-08T22:41:48.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7263fff700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72640c8dd0 0x7f72640c9230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f724c009910 tx=0x7f724c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ae8d1b02abfdd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f72637fe700 1 -- 172.21.15.179:0/2448013011 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011830 con 0x7f72640c8dd0 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f72637fe700 1 -- 172.21.15.179:0/2448013011 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c011990 con 0x7f72640c8dd0 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f72637fe700 1 -- 172.21.15.179:0/2448013011 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011d40 con 0x7f72640c8dd0 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.639+0000 7f7269eb9700 1 -- 172.21.15.179:0/2448013011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72640c8dd0 msgr2=0x7f72640c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72640c8dd0 0x7f72640c9230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f724c009910 tx=0x7f724c009c20 comp rx=0 tx=0).stop 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- 172.21.15.179:0/2448013011 shutdown_connections 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f72640b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- 172.21.15.179:0/2448013011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72640c8dd0 0x7f72640c9230 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- 172.21.15.179:0/2448013011 >> 172.21.15.179:0/2448013011 conn(0x7f72640cc540 msgr2=0x7f72640cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- 172.21.15.179:0/2448013011 shutdown_connections 2024-05-08T22:41:48.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- 172.21.15.179:0/2448013011 wait complete. 2024-05-08T22:41:48.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 Processor -- start 2024-05-08T22:41:48.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- start start 2024-05-08T22:41:48.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f726413a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 0x7f7264150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7264151780 con 0x7f7264160860 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726413b1c0 con 0x7f72640c8dd0 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f7269eb9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7264150ee0 con 0x7f72640b38b0 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50798/0 (socket says 172.21.15.179:50798) 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.643+0000 7f72696b8700 1 -- 172.21.15.179:0/817453581 learned_addr learned my addr 172.21.15.179:0/817453581 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 -- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 msgr2=0x7f726413a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f726413a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 -- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 msgr2=0x7f7264150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f7263fff700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 0x7f7264150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 0x7f7264150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 -- 172.21.15.179:0/817453581 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009570 con 0x7f7264160860 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f7263fff700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 0x7f7264150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72696b8700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f725c009d30 tx=0x7f725c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 -- 172.21.15.179:0/817453581 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c009ba0 con 0x7f7264160860 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72641511c0 con 0x7f7264160860 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 -- 172.21.15.179:0/817453581 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00f040 con 0x7f7264160860 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 -- 172.21.15.179:0/817453581 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c014480 con 0x7f7264160860 2024-05-08T22:41:48.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264150ad0 con 0x7f7264160860 2024-05-08T22:41:48.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 -- 172.21.15.179:0/817453581 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f725c0145e0 con 0x7f7264160860 2024-05-08T22:41:48.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 0x7f7244068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f72617fa700 1 -- 172.21.15.179:0/817453581 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f725c084cc0 con 0x7f7264160860 2024-05-08T22:41:48.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.647+0000 7f7268eb7700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 0x7f7244068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f7268eb7700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 0x7f7244068bf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f726406f070 tx=0x7f7254007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 Processor -- start 2024-05-08T22:41:48.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 -- start start 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1eeead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1eb7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e15500 con 0x1ea9670 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e12c90 con 0x1eee6f0 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e0e580 con 0x1eef010 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f7263fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f7268eb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1eeead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f7263fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50806/0 (socket says 172.21.15.179:50806) 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f72696b8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1eb7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.651+0000 7f7263fff700 1 -- 172.21.15.179:0/3092532055 learned_addr learned my addr 172.21.15.179:0/3092532055 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 -- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 msgr2=0x1eb7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1eb7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 -- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 msgr2=0x1eeead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1eeead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 -- 172.21.15.179:0/3092532055 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d00ee0 con 0x1ea9670 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f7263fff700 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f724c013000 tx=0x7f724c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c997504733d4e8cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f725a7fc700 1 -- 172.21.15.179:0/3092532055 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011b00 con 0x1ea9670 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f725a7fc700 1 -- 172.21.15.179:0/3092532055 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c011c60 con 0x1ea9670 2024-05-08T22:41:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f725a7fc700 1 -- 172.21.15.179:0/3092532055 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c004240 con 0x1ea9670 2024-05-08T22:41:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- 172.21.15.179:0/3092532055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 msgr2=0x1eb72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f724c013000 tx=0x7f724c010000 comp rx=0 tx=0).stop 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- 172.21.15.179:0/3092532055 shutdown_connections 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1eb7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ea9670 0x1eb72c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3092532055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1eeead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- 172.21.15.179:0/3092532055 >> 172.21.15.179:0/3092532055 conn(0x1c35780 msgr2=0x1ed0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- 172.21.15.179:0/3092532055 shutdown_connections 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- 172.21.15.179:0/3092532055 wait complete. 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 -- start start 2024-05-08T22:41:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.655+0000 7f726e3ca740 1 Processor -- start 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 0x1ee8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1ea78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd1920 con 0x1eef010 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd15c0 con 0x1eee6f0 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da3e10 con 0x1ea9670 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 10 client.0 _ll_register_callbacks cb 0x1e07bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50810/0 (socket says 172.21.15.179:50810) 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 -- 172.21.15.179:0/3183995386 learned_addr learned my addr 172.21.15.179:0/3183995386 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f7263fff700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1ea78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 0x1ee8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 -- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 msgr2=0x1ee8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 0x1ee8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 -- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 msgr2=0x1ea78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1ea78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 -- 172.21.15.179:0/3183995386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72640cda10 con 0x1eef010 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 0x1ee8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72696b8700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f725c082f90 tx=0x7f725c055fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620161f3898797a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:48.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c05dc00 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f725c06d1c0 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c42490 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f725c06d380 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 10 client.8610 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x1dd4600 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c42490 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f725c072200 con 0x1eef010 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 10 client.8610 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 10 client.8610 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 20 client.8610 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1ef4f80 0x1ea4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 4 client.8610 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c8c710 con 0x1ef4f80 2024-05-08T22:41:48.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1ef4f80 0x1ea4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f72437fe700 10 client.8610 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.659+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1ef4f80 0x1ea4e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f98e86261381e3 server_cookie=5bd3290c190ca7e7 in_seq=0 out_seq=0 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f72437fe700 1 -- 172.21.15.179:0/3183995386 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1c8c710 con 0x1ef4f80 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f72437fe700 10 client.8610 handle_command_reply: tid=0 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 client.8610 shutdown 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 msgr2=0x1ed0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f725c082f90 tx=0x7f725c055fe0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 shutdown_connections 2024-05-08T22:41:48.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1eee6f0 0x1ea78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1ef4f80 0x1ea4e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ed0820 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f72437fe700 0 client.8610 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3183995386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ea9670 0x1ee8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 >> 172.21.15.179:0/3183995386 conn(0x1c35780 msgr2=0x1eebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 shutdown_connections 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 1 -- 172.21.15.179:0/3183995386 wait complete. 2024-05-08T22:41:48.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.663+0000 7f726e3ca740 20 client.8610 trim_cache size 0 max 16384 2024-05-08T22:41:48.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 Processor -- start 2024-05-08T22:41:48.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 -- start start 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 0x1ed08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ee88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e08980 con 0x1eef010 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dc8010 con 0x1ea9670 2024-05-08T22:41:48.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e11e60 con 0x1eee6f0 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ee88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7263fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 0x1ed08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:42422/0 (socket says 172.21.15.179:42422) 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7263fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 0x1ed08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:41844/0 (socket says 172.21.15.179:41844) 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f72696b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ee88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50812/0 (socket says 172.21.15.179:50812) 2024-05-08T22:41:48.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 -- 172.21.15.179:0/2516742077 learned_addr learned my addr 172.21.15.179:0/2516742077 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 -- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 msgr2=0x1ed08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 0x1ed08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 -- 172.21.15.179:0/2516742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 msgr2=0x1ee88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ee88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 -- 172.21.15.179:0/2516742077 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1dba830 con 0x1ea9670 2024-05-08T22:41:48.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.683+0000 7f7268eb7700 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7254000f60 tx=0x7f7254012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7f49527f511a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.687+0000 7f725affd700 1 -- 172.21.15.179:0/2516742077 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401d040 con 0x1ea9670 2024-05-08T22:41:48.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.687+0000 7f725affd700 1 -- 172.21.15.179:0/2516742077 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254023380 con 0x1ea9670 2024-05-08T22:41:48.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.687+0000 7f725affd700 1 -- 172.21.15.179:0/2516742077 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540236f0 con 0x1ea9670 2024-05-08T22:41:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.687+0000 7f726e3ca740 1 -- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 msgr2=0x1ed03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.687+0000 7f726e3ca740 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7254000f60 tx=0x7f7254012000 comp rx=0 tx=0).stop 2024-05-08T22:41:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- 172.21.15.179:0/2516742077 shutdown_connections 2024-05-08T22:41:48.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eef010 0x1ee88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eee6f0 0x1ed08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- 172.21.15.179:0/2516742077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1ed03a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- 172.21.15.179:0/2516742077 >> 172.21.15.179:0/2516742077 conn(0x1c35780 msgr2=0x1efe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- 172.21.15.179:0/2516742077 shutdown_connections 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- 172.21.15.179:0/2516742077 wait complete. 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- start start 2024-05-08T22:41:48.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 Processor -- start 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1f0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1f4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e09d70 con 0x1eee6f0 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dac5a0 con 0x1ea9670 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e0bd90 con 0x1eef010 2024-05-08T22:41:48.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f726e3ca740 10 client.0 _ll_register_callbacks cb 0x1e0d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7268eb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1f0ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50818/0 (socket says 172.21.15.179:50818) 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 -- 172.21.15.179:0/3664822926 learned_addr learned my addr 172.21.15.179:0/3664822926 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f72696b8700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1f4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 -- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 msgr2=0x1f4fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1f4fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 -- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 msgr2=0x1f0ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1f0ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f7263fff700 1 -- 172.21.15.179:0/3664822926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72640c9770 con 0x1eee6f0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.691+0000 7f72696b8700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1f4fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f7263fff700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7264066670 tx=0x7f724c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04e138c8a95e1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:48.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c004450 con 0x1eee6f0 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f724c0045b0 con 0x1eee6f0 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f724c011c60 con 0x1eee6f0 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f724c006b10 con 0x1eee6f0 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 10 client.8616 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x1c80c10 con 0x1eee6f0 2024-05-08T22:41:48.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c011c60 con 0x1eee6f0 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f724c038350 con 0x1eee6f0 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 10 client.8616 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 10 client.8616 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 20 client.8616 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1f501a0 0x1eeb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 4 client.8616 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-08T22:41:48.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x1cb35f0 con 0x1f501a0 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1f501a0 0x1eeb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f7268eb7700 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1f501a0 0x1eeb050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b959e476940b1e4 server_cookie=44532df7d0923cb9 in_seq=0 out_seq=0 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 10 client.8616 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 1 -- 172.21.15.179:0/3664822926 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1cb35f0 con 0x1f501a0 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 10 client.8616 handle_command_reply: tid=0 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 client.8616 shutdown 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 msgr2=0x1f0d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7264066670 tx=0x7f724c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 shutdown_connections 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1f501a0 0x1eeb050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eef010 0x1f4fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f72437fe700 0 client.8616 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eee6f0 0x1f0d1f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 --2- 172.21.15.179:0/3664822926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ea9670 0x1f0ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 >> 172.21.15.179:0/3664822926 conn(0x1c35780 msgr2=0x1ed03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 shutdown_connections 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.695+0000 7f726e3ca740 1 -- 172.21.15.179:0/3664822926 wait complete. 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f726e3ca740 20 client.8616 trim_cache size 0 max 16384 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "cdcae56f-1a92-4d6b-8f06-867dd5929654", 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:41:48.701 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:41:48.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 msgr2=0x7f7244068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 0x7f7244068bf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f726406f070 tx=0x7f7254007a00 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 msgr2=0x7f72640af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f725c009d30 tx=0x7f725c00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 shutdown_connections 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7244066730 0x7f7244068bf0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72640c8dd0 0x7f7264150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264160860 0x7f72640af810 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 --2- 172.21.15.179:0/817453581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72640b38b0 0x7f726413a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 >> 172.21.15.179:0/817453581 conn(0x7f72640cc540 msgr2=0x7f72640b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 shutdown_connections 2024-05-08T22:41:48.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.699+0000 7f7269eb9700 1 -- 172.21.15.179:0/817453581 wait complete. 2024-05-08T22:41:48.703 INFO:tasks.ceph.mds.a.smithi179.stderr:2024-05-08T22:41:48.699+0000 7fede1a52700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x563966cd8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:41:48.715 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:41:48.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 Processor -- start 2024-05-08T22:41:48.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 -- start start 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b74161360 0x7f9b7406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b740c3b30 0x7f9b740cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b740c43d0 con 0x7f9b74161740 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b740cf2e0 con 0x7f9b740c3b30 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b73fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7406e230 con 0x7f9b74161360 2024-05-08T22:41:48.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b740c3b30 0x7f9b740cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b740c3b30 0x7f9b740cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50822/0 (socket says 172.21.15.179:50822) 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b72ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b74161360 0x7f9b7406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 -- 172.21.15.179:0/614867907 learned_addr learned my addr 172.21.15.179:0/614867907 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b727fc700 1 --1- 172.21.15.179:0/614867907 >> v1:172.21.15.195:6789/0 conn(0x7f9b74161740 0x7f9b740ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43466/0 (socket says 172.21.15.179:43466) 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 -- 172.21.15.179:0/614867907 >> v2:172.21.15.195:3300/0 conn(0x7f9b74161360 msgr2=0x7f9b7406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 --2- 172.21.15.179:0/614867907 >> v2:172.21.15.195:3300/0 conn(0x7f9b74161360 0x7f9b7406cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 -- 172.21.15.179:0/614867907 >> v1:172.21.15.195:6789/0 conn(0x7f9b74161740 legacy=0x7f9b740ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 -- 172.21.15.179:0/614867907 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740cd100 con 0x7f9b740c3b30 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b737fe700 1 --2- 172.21.15.179:0/614867907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b740cb560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9b74067b90 tx=0x7f9b6c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b487d173af8f5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b71ffb700 1 -- 172.21.15.179:0/614867907 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c011830 con 0x7f9b740c3b30 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b71ffb700 1 -- 172.21.15.179:0/614867907 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c011990 con 0x7f9b740c3b30 2024-05-08T22:41:48.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.839+0000 7f9b71ffb700 1 -- 172.21.15.179:0/614867907 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c011d20 con 0x7f9b740c3b30 2024-05-08T22:41:48.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- 172.21.15.179:0/614867907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 msgr2=0x7f9b740cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- 172.21.15.179:0/614867907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b740cb560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9b74067b90 tx=0x7f9b6c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- 172.21.15.179:0/614867907 shutdown_connections 2024-05-08T22:41:48.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- 172.21.15.179:0/614867907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b740cb560 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- 172.21.15.179:0/614867907 >> v2:172.21.15.195:3300/0 conn(0x7f9b74161360 0x7f9b7406cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- 172.21.15.179:0/614867907 >> 172.21.15.179:0/614867907 conn(0x7f9b740b2820 msgr2=0x7f9b740b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- 172.21.15.179:0/614867907 shutdown_connections 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- 172.21.15.179:0/614867907 wait complete. 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 Processor -- start 2024-05-08T22:41:48.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- start start 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b74161360 0x7f9b740c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 0x7f9b7406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740c9560 con 0x7f9b740c3b30 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740c7f80 con 0x7f9b74161360 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b73fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740c8260 con 0x7f9b74161740 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50826/0 (socket says 172.21.15.179:50826) 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b72ffd700 1 -- 172.21.15.179:0/3097721108 learned_addr learned my addr 172.21.15.179:0/3097721108 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.843+0000 7f9b727fc700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b74161360 0x7f9b740c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b737fe700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 0x7f9b7406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 -- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 msgr2=0x7f9b7406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 0x7f9b7406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 -- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b74161360 msgr2=0x7f9b740c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b74161360 0x7f9b740c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 -- 172.21.15.179:0/3097721108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c009570 con 0x7f9b740c3b30 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b737fe700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 0x7f9b7406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9d0 tx=0x7f9b6800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b5bfff700 1 -- 172.21.15.179:0/3097721108 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68009720 con 0x7f9b740c3b30 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b5bfff700 1 -- 172.21.15.179:0/3097721108 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6800c850 con 0x7f9b740c3b30 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b740c8540 con 0x7f9b740c3b30 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b5bfff700 1 -- 172.21.15.179:0/3097721108 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800cbc0 con 0x7f9b740c3b30 2024-05-08T22:41:48.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b740c8b00 con 0x7f9b740c3b30 2024-05-08T22:41:48.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b5bfff700 1 -- 172.21.15.179:0/3097721108 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b6800cd20 con 0x7f9b740c3b30 2024-05-08T22:41:48.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.847+0000 7f9b5bfff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 0x7f9b60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b5bfff700 1 -- 172.21.15.179:0/3097721108 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f9b680844f0 con 0x7f9b740c3b30 2024-05-08T22:41:48.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b727fc700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 0x7f9b60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b727fc700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 0x7f9b60068bf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9b7406fbb0 tx=0x7f9b64001940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 Processor -- start 2024-05-08T22:41:48.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 -- start start 2024-05-08T22:41:48.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 0x21b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2145010 con 0x21b2840 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d2f40 con 0x21df5d0 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.851+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2125e30 con 0x21dc6f0 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50842/0 (socket says 172.21.15.179:50842) 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 -- 172.21.15.179:0/2713584816 learned_addr learned my addr 172.21.15.179:0/2713584816 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b737fe700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 0x21b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b727fc700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 -- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 msgr2=0x21b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 -- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 msgr2=0x21b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 0x21b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 -- 172.21.15.179:0/2713584816 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f91a40 con 0x21b2840 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b737fe700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 0x21b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9b6805d180 tx=0x7f9b680552c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a43c745c692d604 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b597fa700 1 -- 172.21.15.179:0/2713584816 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68053660 con 0x21b2840 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b597fa700 1 -- 172.21.15.179:0/2713584816 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b68082be0 con 0x21b2840 2024-05-08T22:41:48.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b597fa700 1 -- 172.21.15.179:0/2713584816 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68055520 con 0x21b2840 2024-05-08T22:41:48.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- 172.21.15.179:0/2713584816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 msgr2=0x21b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9b6805d180 tx=0x7f9b680552c0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- 172.21.15.179:0/2713584816 shutdown_connections 2024-05-08T22:41:48.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21df5d0 0x21b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/2713584816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21b2840 0x21b7d90 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- 172.21.15.179:0/2713584816 >> 172.21.15.179:0/2713584816 conn(0x1f465b0 msgr2=0x21deec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- 172.21.15.179:0/2713584816 shutdown_connections 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- 172.21.15.179:0/2713584816 wait complete. 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 -- start start 2024-05-08T22:41:48.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.855+0000 7f9b7cba7740 1 Processor -- start 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x2207500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2121150 con 0x21df5d0 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x211eeb0 con 0x21b2840 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20dc8e0 con 0x21dc6f0 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 10 client.0 _ll_register_callbacks cb 0x20bb670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:48.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50858/0 (socket says 172.21.15.179:50858) 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 -- 172.21.15.179:0/3572549561 learned_addr learned my addr 172.21.15.179:0/3572549561 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x2207500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b727fc700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 -- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 msgr2=0x21e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 -- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 msgr2=0x2207500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x2207500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 -- 172.21.15.179:0/3572549561 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b740c8b00 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x2207500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b737fe700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009150 tx=0x7f9b6c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56455726eba5855d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c004450 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f9b6c0045b0 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f532c0 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b6c011d30 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 10 client.8628 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x20e5290 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f532c0 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9b6c0036c0 con 0x21df5d0 2024-05-08T22:41:48.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 10 client.8628 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 10 client.8628 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 20 client.8628 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b61d0 0x2211d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 4 client.8628 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f9d540 con 0x21b61d0 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b61d0 0x2211d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b527fc700 10 client.8628 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.859+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b61d0 0x2211d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=477401fa848d11cd server_cookie=1f42ed6ac8d6c3a6 in_seq=0 out_seq=0 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b527fc700 1 -- 172.21.15.179:0/3572549561 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1f9d540 con 0x21b61d0 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b527fc700 10 client.8628 handle_command_reply: tid=0 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 client.8628 shutdown 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 msgr2=0x21e1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009150 tx=0x7f9b6c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 shutdown_connections 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b61d0 0x2211d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21e1500 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b527fc700 0 client.8628 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/3572549561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x2207500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 >> 172.21.15.179:0/3572549561 conn(0x1f465b0 msgr2=0x21decb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 shutdown_connections 2024-05-08T22:41:48.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 1 -- 172.21.15.179:0/3572549561 wait complete. 2024-05-08T22:41:48.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.863+0000 7f9b7cba7740 20 client.8628 trim_cache size 0 max 16384 2024-05-08T22:41:48.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.879+0000 7f9b7cba7740 1 Processor -- start 2024-05-08T22:41:48.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.879+0000 7f9b7cba7740 1 -- start start 2024-05-08T22:41:48.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x21b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21df5d0 0x2201c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fcab80 con 0x21dc6f0 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x211c6c0 con 0x21b2840 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2146610 con 0x21df5d0 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50872/0 (socket says 172.21.15.179:50872) 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 -- 172.21.15.179:0/64135991 learned_addr learned my addr 172.21.15.179:0/64135991 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b737fe700 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21df5d0 0x2201c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x21b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 -- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21df5d0 msgr2=0x2201c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21df5d0 0x2201c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 -- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 msgr2=0x21b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x21b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 -- 172.21.15.179:0/64135991 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20cb4c0 con 0x21dc6f0 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b727fc700 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9b640134d0 tx=0x7f9b64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8789a9109f7ab53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b59ffb700 1 -- 172.21.15.179:0/64135991 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64023410 con 0x21dc6f0 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b59ffb700 1 -- 172.21.15.179:0/64135991 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b64023570 con 0x21dc6f0 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b59ffb700 1 -- 172.21.15.179:0/64135991 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640236e0 con 0x21dc6f0 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- 172.21.15.179:0/64135991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 msgr2=0x22016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9b640134d0 tx=0x7f9b64012000 comp rx=0 tx=0).stop 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- 172.21.15.179:0/64135991 shutdown_connections 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21df5d0 0x2201c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21dc6f0 0x22016e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/64135991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x21b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- 172.21.15.179:0/64135991 >> 172.21.15.179:0/64135991 conn(0x1f465b0 msgr2=0x2215da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- 172.21.15.179:0/64135991 shutdown_connections 2024-05-08T22:41:48.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.883+0000 7f9b7cba7740 1 -- 172.21.15.179:0/64135991 wait complete. 2024-05-08T22:41:48.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- start start 2024-05-08T22:41:48.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 Processor -- start 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x225f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x21dfd50 con 0x21df5d0 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20bd620 con 0x21b2840 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2202030 con 0x21dc6f0 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 10 client.0 _ll_register_callbacks cb 0x20d8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:41:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50882/0 (socket says 172.21.15.179:50882) 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 -- 172.21.15.179:0/785605667 learned_addr learned my addr 172.21.15.179:0/785605667 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b727fc700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x225f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 -- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 msgr2=0x21ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 -- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 msgr2=0x225f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x225f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 -- 172.21.15.179:0/785605667 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6401d040 con 0x21df5d0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b737fe700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c006ce0 tx=0x7f9b6c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babd4c8cd8e6914 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b527fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c004450 con 0x21df5d0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f9b6c000e40 con 0x21df5d0 2024-05-08T22:41:48.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b68082be0 con 0x21df5d0 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b6c0045b0 con 0x21df5d0 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 10 client.8634 fetch_fsmap learned FSMap version 68 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x20bf790 con 0x21df5d0 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.887+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b68082be0 con 0x21df5d0 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9b6c02f680 con 0x21df5d0 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 10 client.8634 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 10 client.8634 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 20 client.8634 populate_metadata read hostname 'smithi179' 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b65b0 0x21b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 4 client.8634 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:41:48.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1fc22c0 con 0x21b65b0 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b65b0 0x21b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b527fc700 10 client.8634 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b72ffd700 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b65b0 0x21b69c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e2c641df0273c6 server_cookie=16b70815ba0bf150 in_seq=0 out_seq=0 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b527fc700 1 -- 172.21.15.179:0/785605667 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x1fc22c0 con 0x21b65b0 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b527fc700 10 client.8634 handle_command_reply: tid=0 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 client.8634 shutdown 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 msgr2=0x21ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c006ce0 tx=0x7f9b6c011000 comp rx=0 tx=0).stop 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 shutdown_connections 2024-05-08T22:41:48.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x21b65b0 0x21b69c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21df5d0 0x21ffb70 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21dc6f0 0x21ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b527fc700 0 client.8634 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 --2- 172.21.15.179:0/785605667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21b2840 0x225f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 >> 172.21.15.179:0/785605667 conn(0x1f465b0 msgr2=0x21e0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 shutdown_connections 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 1 -- 172.21.15.179:0/785605667 wait complete. 2024-05-08T22:41:48.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b7cba7740 20 client.8634 trim_cache size 0 max 16384 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "cdcae56f-1a92-4d6b-8f06-867dd5929654": { 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/dir", 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "cdcae56f-1a92-4d6b-8f06-867dd5929654", 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:41:48.897 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:41:48.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 msgr2=0x7f9b60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 0x7f9b60068bf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9b7406fbb0 tx=0x7f9b64001940 comp rx=0 tx=0).stop 2024-05-08T22:41:48.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 msgr2=0x7f9b741f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9d0 tx=0x7f9b6800ed90 comp rx=0 tx=0).stop 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 shutdown_connections 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9b60066730 0x7f9b60068bf0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b74161360 0x7f9b740c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b74161740 0x7f9b7406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 --2- 172.21.15.179:0/3097721108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b740c3b30 0x7f9b741f66e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 >> 172.21.15.179:0/3097721108 conn(0x7f9b740b2820 msgr2=0x7f9b7406e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 shutdown_connections 2024-05-08T22:41:48.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:41:48.891+0000 7f9b73fff700 1 -- 172.21.15.179:0/3097721108 wait complete. 2024-05-08T22:41:48.909 INFO:tasks.cephfs.filesystem:scrub status for tag:cdcae56f-1a92-4d6b-8f06-867dd5929654 - {'path': '/dir', 'tag': 'cdcae56f-1a92-4d6b-8f06-867dd5929654', 'options': 'recursive,repair'} 2024-05-08T22:41:52.617 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:41:52.620 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:42:18.911 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T22:42:19.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.038+0000 7f2dfbfff700 1 Processor -- start 2024-05-08T22:42:19.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.038+0000 7f2dfbfff700 1 -- start start 2024-05-08T22:42:19.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc06bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 0x7f2dfc06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dfc0b3640 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dfc165b30 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc165fc0 con 0x7f2dfc1648b0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43946/0 (socket says 172.21.15.179:43946) 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/4148080890 learned_addr learned my addr 172.21.15.179:0/4148080890 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc06bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/4148080890 >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 0x7f2dfc06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/4148080890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 msgr2=0x7f2dfc06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc06bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/4148080890 >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 msgr2=0x7f2dfc06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/4148080890 >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 0x7f2dfc06c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/4148080890 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc165200 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/4148080890 >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 0x7f2dfc06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc072790 tx=0x7f2dec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2932036e0036f55a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2df9ffb700 1 -- 172.21.15.179:0/4148080890 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00e070 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2df9ffb700 1 -- 172.21.15.179:0/4148080890 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00b9c0 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2df9ffb700 1 -- 172.21.15.179:0/4148080890 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00bd70 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- 172.21.15.179:0/4148080890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b37a0 msgr2=0x7f2dfc0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc072790 tx=0x7f2dec00b330 comp rx=0 tx=0).stop 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- 172.21.15.179:0/4148080890 shutdown_connections 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc0b1ac0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/4148080890 >> v2:172.21.15.195:3300/0 conn(0x7f2dfc0b30a0 0x7f2dfc06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/4148080890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc06bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- 172.21.15.179:0/4148080890 >> 172.21.15.179:0/4148080890 conn(0x7f2dfc070d80 msgr2=0x7f2dfc071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- 172.21.15.179:0/4148080890 shutdown_connections 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.042+0000 7f2dfbfff700 1 -- 172.21.15.179:0/4148080890 wait complete. 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 Processor -- start 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- start start 2024-05-08T22:42:19.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15a7c0 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc14b2c0 con 0x7f2dfc0b37a0 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc159b40 con 0x7f2dfc1648b0 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43952/0 (socket says 172.21.15.179:43952) 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 -- 172.21.15.179:0/718693211 learned_addr learned my addr 172.21.15.179:0/718693211 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 -- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 msgr2=0x7f2dfc15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 -- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dfc0b37a0 msgr2=0x7f2dfc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 -- 172.21.15.179:0/718693211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec009b30 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d800 tx=0x7f2de400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2df37fe700 1 -- 172.21.15.179:0/718693211 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40125f0 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2df37fe700 1 -- 172.21.15.179:0/718693211 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4012750 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc159e20 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2df37fe700 1 -- 172.21.15.179:0/718693211 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4012ac0 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.046+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc15a3e0 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2df37fe700 1 -- 172.21.15.179:0/718693211 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2de4012c20 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2df37fe700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 0x7f2ddc068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2df37fe700 1 -- 172.21.15.179:0/718693211 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f2de4084400 con 0x7f2dfc0b30a0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 0x7f2ddc068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 0x7f2ddc068c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc154bb0 tx=0x7f2df4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2e0457a740 1 Processor -- start 2024-05-08T22:42:19.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.050+0000 7f2e0457a740 1 -- start start 2024-05-08T22:42:19.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b3fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2acd010 con 0x2b675d0 2024-05-08T22:42:19.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a5af40 con 0x2b3a840 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2aade30 con 0x2b646f0 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b3fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfa7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43956/0 (socket says 172.21.15.179:43956) 2024-05-08T22:42:19.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/265514366 learned_addr learned my addr 172.21.15.179:0/265514366 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 msgr2=0x2b3ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 msgr2=0x2b3fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b3fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/265514366 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2919a40 con 0x2b675d0 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b3fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2dec012000 tx=0x7f2dec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879e5a35bbf1ca9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2df0ff9700 1 -- 172.21.15.179:0/265514366 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00bd30 con 0x2b675d0 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2df0ff9700 1 -- 172.21.15.179:0/265514366 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00be90 con 0x2b675d0 2024-05-08T22:42:19.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2df0ff9700 1 -- 172.21.15.179:0/265514366 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec003c40 con 0x2b675d0 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- 172.21.15.179:0/265514366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 msgr2=0x2b3b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2dec012000 tx=0x7f2dec010000 comp rx=0 tx=0).stop 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- 172.21.15.179:0/265514366 shutdown_connections 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3b810 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 --2- 172.21.15.179:0/265514366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b3fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- 172.21.15.179:0/265514366 >> 172.21.15.179:0/265514366 conn(0x28ce5b0 msgr2=0x2b66ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- 172.21.15.179:0/265514366 shutdown_connections 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.054+0000 7f2e0457a740 1 -- 172.21.15.179:0/265514366 wait complete. 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- start start 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 Processor -- start 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aa9150 con 0x2b675d0 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aa6eb0 con 0x2b3a840 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a648e0 con 0x2b646f0 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 10 client.0 _ll_register_callbacks cb 0x2a43670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43964/0 (socket says 172.21.15.179:43964) 2024-05-08T22:42:19.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/1101201182 learned_addr learned my addr 172.21.15.179:0/1101201182 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 -- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 msgr2=0x2b3df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 -- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 msgr2=0x2b3e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 -- 172.21.15.179:0/1101201182 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc15a3e0 con 0x2b3a840 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6b980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2de40551d0 tx=0x7f2de4054c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6139749301449777 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40556a0 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f2de4059070 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28db2c0 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de405d630 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 10 client.8377 fetch_fsmap learned FSMap version 68 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.058+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x7f2df400e040 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28db2c0 con 0x2b3a840 2024-05-08T22:42:19.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2de4082250 con 0x2b3a840 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 10 client.8377 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 10 client.8377 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 20 client.8377 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b4f750 0x2b4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 4 client.8377 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2925540 con 0x2b4f750 2024-05-08T22:42:19.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b4f750 0x2b4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 10 client.8377 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b4f750 0x2b4fb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2fddf31a69b9d99 server_cookie=3d4577d442686b6 in_seq=0 out_seq=0 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1101201182 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2925540 con 0x2b4f750 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 10 client.8377 handle_command_reply: tid=0 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 client.8377 shutdown 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 msgr2=0x2b6b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6b980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2de40551d0 tx=0x7f2de4054c70 comp rx=0 tx=0).stop 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 shutdown_connections 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b4f750 0x2b4fb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b3e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1101201182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6b980 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2dd9ffb700 0 client.8377 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 >> 172.21.15.179:0/1101201182 conn(0x28ce5b0 msgr2=0x2b66cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 shutdown_connections 2024-05-08T22:42:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 1 -- 172.21.15.179:0/1101201182 wait complete. 2024-05-08T22:42:19.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.062+0000 7f2e0457a740 20 client.8377 trim_cache size 0 max 16384 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 Processor -- start 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 -- start start 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b675d0 0x2b3ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2952b80 con 0x2b3a840 2024-05-08T22:42:19.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a77c00 con 0x2b675d0 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2aa46c0 con 0x2b646f0 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43976/0 (socket says 172.21.15.179:43976) 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2dfaffd700 1 -- 172.21.15.179:0/747725709 learned_addr learned my addr 172.21.15.179:0/747725709 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b675d0 0x2b3ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.082+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 -- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 msgr2=0x2b3ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 -- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b675d0 msgr2=0x2b3ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b675d0 0x2b3ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 -- 172.21.15.179:0/747725709 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc165200 con 0x2b3a840 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d7d0 tx=0x7f2de4058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf52a25b372c210 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2df17fa700 1 -- 172.21.15.179:0/747725709 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de405d630 con 0x2b3a840 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2df17fa700 1 -- 172.21.15.179:0/747725709 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4055940 con 0x2b3a840 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2df17fa700 1 -- 172.21.15.179:0/747725709 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40556e0 con 0x2b3a840 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- 172.21.15.179:0/747725709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 msgr2=0x2b4d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d7d0 tx=0x7f2de4058000 comp rx=0 tx=0).stop 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- 172.21.15.179:0/747725709 shutdown_connections 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b675d0 0x2b3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b3ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- 172.21.15.179:0/747725709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b3a840 0x2b4d670 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- 172.21.15.179:0/747725709 >> 172.21.15.179:0/747725709 conn(0x28ce5b0 msgr2=0x2b52230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- 172.21.15.179:0/747725709 shutdown_connections 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- 172.21.15.179:0/747725709 wait complete. 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- start start 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 Processor -- start 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b3e580 con 0x2b675d0 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b3e7e0 con 0x2b3a840 2024-05-08T22:42:19.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b6aa40 con 0x2b646f0 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2e0457a740 10 client.0 _ll_register_callbacks cb 0x2a60d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:60804/0 (socket says 172.21.15.179:60804) 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfaffd700 1 -- 172.21.15.179:0/1565134881 learned_addr learned my addr 172.21.15.179:0/1565134881 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.086+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 msgr2=0x2b8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 msgr2=0x2b6a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 -- 172.21.15.179:0/1565134881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df400e040 con 0x2b675d0 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfa7fc700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:19.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfb7fe700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b8dfc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000f80 tx=0x7f2dec00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c698bff995ce24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec02a070 con 0x2b675d0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f2dec02fd20 con 0x2b675d0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x294a2c0 con 0x2b675d0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dec02ff00 con 0x2b675d0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 10 client.8652 fetch_fsmap learned FSMap version 68 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x2a6d290 con 0x2b675d0 2024-05-08T22:42:19.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x294a2c0 con 0x2b675d0 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2dec035020 con 0x2b675d0 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 10 client.8652 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 10 client.8652 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 20 client.8652 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b8e500 0x2bf14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 4 client.8652 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x2b4d950 con 0x2b8e500 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b8e500 0x2bf14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 10 client.8652 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dfaffd700 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b8e500 0x2bf14c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b550bd37ba2d215d server_cookie=c761bbba2553cec5 in_seq=0 out_seq=0 2024-05-08T22:42:19.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 1 -- 172.21.15.179:0/1565134881 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x2b4d950 con 0x2b8e500 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 10 client.8652 handle_command_reply: tid=0 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 client.8652 shutdown 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 msgr2=0x2b8dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b8dfc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000f80 tx=0x7f2dec00e000 comp rx=0 tx=0).stop 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 shutdown_connections 2024-05-08T22:42:19.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x2b8e500 0x2bf14c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2b675d0 0x2b8dfc0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2dd9ffb700 0 client.8652 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2b646f0 0x2b8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 --2- 172.21.15.179:0/1565134881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2b3a840 0x2b6a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 >> 172.21.15.179:0/1565134881 conn(0x28ce5b0 msgr2=0x2b3ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 shutdown_connections 2024-05-08T22:42:19.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.090+0000 7f2e0457a740 1 -- 172.21.15.179:0/1565134881 wait complete. 2024-05-08T22:42:19.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2e0457a740 20 client.8652 trim_cache size 0 max 16384 2024-05-08T22:42:19.112 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:42:19.112 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:42:19.112 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:42:19.112 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:42:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 msgr2=0x7f2ddc068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 0x7f2ddc068c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc154bb0 tx=0x7f2df4008040 comp rx=0 tx=0).stop 2024-05-08T22:42:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 msgr2=0x7f2dfc14ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d800 tx=0x7f2de400db10 comp rx=0 tx=0).stop 2024-05-08T22:42:19.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 shutdown_connections 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ddc066780 0x7f2ddc068c40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dfc0b37a0 0x7f2dfc159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dfc1648b0 0x7f2dfc15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 --2- 172.21.15.179:0/718693211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dfc0b30a0 0x7f2dfc14ad80 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 >> 172.21.15.179:0/718693211 conn(0x7f2dfc070d80 msgr2=0x7f2dfc0cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 shutdown_connections 2024-05-08T22:42:19.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.094+0000 7f2dfbfff700 1 -- 172.21.15.179:0/718693211 wait complete. 2024-05-08T22:42:19.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:42:19.116 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:19.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 Processor -- start 2024-05-08T22:42:19.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 -- start start 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc38161790 0x7fbc38164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc38165d50 con 0x7fbc3806de70 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806ad70 con 0x7fbc3815ef80 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3806e4b0 con 0x7fbc38161790 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3cd95700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbc3806de70 0x7fbc3806b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55882/0 (socket says 172.21.15.179:55882) 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3cd95700 1 -- 172.21.15.179:0/2986371848 learned_addr learned my addr 172.21.15.179:0/2986371848 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2986371848 >> v2:172.21.15.195:3300/0 conn(0x7fbc38161790 0x7fbc38164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176927964 0 0) 0x7fbc38165d50 con 0x7fbc3806de70 2024-05-08T22:42:19.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc240035d0 con 0x7fbc3806de70 2024-05-08T22:42:19.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3617070860 0 0) 0x7fbc240035d0 con 0x7fbc3806de70 2024-05-08T22:42:19.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 >> v2:172.21.15.195:3300/0 conn(0x7fbc38161790 msgr2=0x7fbc38164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 --2- 172.21.15.179:0/2986371848 >> v2:172.21.15.195:3300/0 conn(0x7fbc38161790 0x7fbc38164db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 >> v1:172.21.15.195:6789/0 conn(0x7fbc3815ef80 legacy=0x7fbc3806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc381654b0 con 0x7fbc3806de70 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fbc340019f0 con 0x7fbc3806de70 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 >> v1:172.21.15.179:6789/0 conn(0x7fbc3806de70 legacy=0x7fbc3806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24003a20 0x7fbc24003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc24004870 0x7fbc24004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 0x7fbc2400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806e4b0 con 0x7fbc24005230 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2400c1d0 con 0x7fbc24004870 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2400c390 con 0x7fbc24003a20 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 0x7fbc2400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3cd95700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc24004870 0x7fbc24004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3d596700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24003a20 0x7fbc24003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24003a20 msgr2=0x7fbc24003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24003a20 0x7fbc24003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc24004870 msgr2=0x7fbc24004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc24004870 0x7fbc24004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2986371848 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240035d0 con 0x7fbc24005230 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3cd95700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc24004870 0x7fbc24004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2986371848 reap_dead start 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 0x7fbc2400ba30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009b70 tx=0x7fbc240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ae202912ce06a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004020 con 0x7fbc24005230 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc30004180 con 0x7fbc24005230 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc2ffff700 1 -- 172.21.15.179:0/2986371848 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc300044f0 con 0x7fbc24005230 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 -- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 msgr2=0x7fbc2400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.250+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 0x7fbc2400ba30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009b70 tx=0x7fbc240058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- 172.21.15.179:0/2986371848 shutdown_connections 2024-05-08T22:42:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2986371848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc24005230 0x7fbc2400ba30 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- 172.21.15.179:0/2986371848 >> 172.21.15.179:0/2986371848 conn(0x7fbc380ce9d0 msgr2=0x7fbc380cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- 172.21.15.179:0/2986371848 shutdown_connections 2024-05-08T22:42:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- 172.21.15.179:0/2986371848 wait complete. 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 Processor -- start 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- start start 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24005230 0x7fbc38151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc380cd780 0x7fbc380cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380ce140 con 0x7fbc381fa3a0 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc381f7650 con 0x7fbc380cd780 2024-05-08T22:42:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3e598700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc381f6d40 con 0x7fbc24005230 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3d596700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24005230 0x7fbc38151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43998/0 (socket says 172.21.15.179:43998) 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2341956024 learned_addr learned my addr 172.21.15.179:0/2341956024 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3cd95700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc380cd780 0x7fbc380cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24005230 msgr2=0x7fbc38151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24005230 0x7fbc38151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc380cd780 msgr2=0x7fbc380cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc380cd780 0x7fbc380cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.254+0000 7fbc3dd97700 1 -- 172.21.15.179:0/2341956024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc381fb0f0 con 0x7fbc381fa3a0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc3dd97700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b2970 tx=0x7fbc300047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc300038c0 con 0x7fbc381fa3a0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc30003a40 con 0x7fbc381fa3a0 2024-05-08T22:42:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc381f7000 con 0x7fbc381fa3a0 2024-05-08T22:42:19.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30003df0 con 0x7fbc381fa3a0 2024-05-08T22:42:19.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc381fad70 con 0x7fbc381fa3a0 2024-05-08T22:42:19.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc381f7800 con 0x7fbc381fa3a0 2024-05-08T22:42:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc30013470 con 0x7fbc381fa3a0 2024-05-08T22:42:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.258+0000 7fbc2dffb700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 0x7fbc1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.262+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fbc30085790 con 0x7fbc381fa3a0 2024-05-08T22:42:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.262+0000 7fbc3d596700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 0x7fbc1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.262+0000 7fbc3d596700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 0x7fbc1c068c20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbc380cab80 tx=0x7fbc28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.266+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc381f7800 con 0x7fbc381fa3a0 2024-05-08T22:42:19.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.474+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc381f77c0 con 0x7fbc381fa3a0 2024-05-08T22:42:19.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.474+0000 7fbc2dffb700 1 -- 172.21.15.179:0/2341956024 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fbc30056440 con 0x7fbc381fa3a0 2024-05-08T22:42:19.478 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:19.478 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:42:19.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 msgr2=0x7fbc1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 0x7fbc1c068c20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbc380cab80 tx=0x7fbc28009040 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 msgr2=0x7fbc381fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b2970 tx=0x7fbc300047e0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 shutdown_connections 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbc1c066760 0x7fbc1c068c20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc380cd780 0x7fbc380cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc381fa3a0 0x7fbc381fa830 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 --2- 172.21.15.179:0/2341956024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc24005230 0x7fbc38151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 >> 172.21.15.179:0/2341956024 conn(0x7fbc380ce9d0 msgr2=0x7fbc3815cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 shutdown_connections 2024-05-08T22:42:19.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.478+0000 7fbc3e598700 1 -- 172.21.15.179:0/2341956024 wait complete. 2024-05-08T22:42:19.482 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:42:19.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T22:42:19.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 Processor -- start 2024-05-08T22:42:19.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- start start 2024-05-08T22:42:19.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 0x7fed740cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fed740ce450 0x7fed74161c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7406bb30 con 0x7fed740d15e0 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7406af20 con 0x7fed740ce450 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7406b200 con 0x7fed740d19c0 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fed740ce450 0x7fed74161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fed740ce450 0x7fed74161c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44006/0 (socket says 172.21.15.179:44006) 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 -- 172.21.15.179:0/2441279901 learned_addr learned my addr 172.21.15.179:0/2441279901 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7991f700 1 --2- 172.21.15.179:0/2441279901 >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 0x7fed740cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 -- 172.21.15.179:0/2441279901 >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 msgr2=0x7fed740cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 --2- 172.21.15.179:0/2441279901 >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 0x7fed740cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 -- 172.21.15.179:0/2441279901 >> v1:172.21.15.195:6789/0 conn(0x7fed740d15e0 legacy=0x7fed7406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 -- 172.21.15.179:0/2441279901 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7406b4e0 con 0x7fed740ce450 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7991f700 1 --2- 172.21.15.179:0/2441279901 >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 0x7fed740cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:19.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7a921700 1 --2- 172.21.15.179:0/2441279901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740ce450 0x7fed74161c50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fed70009910 tx=0x7fed70009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d9dd0d0f56846 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7911e700 1 -- 172.21.15.179:0/2441279901 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70004230 con 0x7fed740ce450 2024-05-08T22:42:19.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7911e700 1 -- 172.21.15.179:0/2441279901 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed70004390 con 0x7fed740ce450 2024-05-08T22:42:19.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7911e700 1 -- 172.21.15.179:0/2441279901 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70009e60 con 0x7fed740ce450 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- 172.21.15.179:0/2441279901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740ce450 msgr2=0x7fed74161c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 --2- 172.21.15.179:0/2441279901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740ce450 0x7fed74161c50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fed70009910 tx=0x7fed70009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- 172.21.15.179:0/2441279901 shutdown_connections 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 --2- 172.21.15.179:0/2441279901 >> v2:172.21.15.195:3300/0 conn(0x7fed740d19c0 0x7fed740cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 --2- 172.21.15.179:0/2441279901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740ce450 0x7fed74161c50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- 172.21.15.179:0/2441279901 >> 172.21.15.179:0/2441279901 conn(0x7fed740b49a0 msgr2=0x7fed740b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- 172.21.15.179:0/2441279901 shutdown_connections 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.614+0000 7fed7b122700 1 -- 172.21.15.179:0/2441279901 wait complete. 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 Processor -- start 2024-05-08T22:42:19.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- start start 2024-05-08T22:42:19.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed740ce450 0x7fed74143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed740d15e0 0x7fed741f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c96e0 con 0x7fed740d19c0 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c6980 con 0x7fed740d15e0 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c8d80 con 0x7fed740ce450 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44008/0 (socket says 172.21.15.179:44008) 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 -- 172.21.15.179:0/3499822579 learned_addr learned my addr 172.21.15.179:0/3499822579 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7991f700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed740d15e0 0x7fed741f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a120700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed740ce450 0x7fed74143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 -- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed740ce450 msgr2=0x7fed74143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed740ce450 0x7fed74143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 -- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed740d15e0 msgr2=0x7fed741f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed740d15e0 0x7fed741f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 -- 172.21.15.179:0/3499822579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009570 con 0x7fed740d19c0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7a921700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fed70008000 tx=0x7fed70004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed66ffd700 1 -- 172.21.15.179:0/3499822579 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000beb0 con 0x7fed740d19c0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed66ffd700 1 -- 172.21.15.179:0/3499822579 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7001f070 con 0x7fed740d19c0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed740c6bd0 con 0x7fed740d19c0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed66ffd700 1 -- 172.21.15.179:0/3499822579 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70003900 con 0x7fed740d19c0 2024-05-08T22:42:19.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.618+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed740c9020 con 0x7fed740d19c0 2024-05-08T22:42:19.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.622+0000 7fed66ffd700 1 -- 172.21.15.179:0/3499822579 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed70021030 con 0x7fed740d19c0 2024-05-08T22:42:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.622+0000 7fed66ffd700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 0x7fed58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.622+0000 7fed66ffd700 1 -- 172.21.15.179:0/3499822579 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fed70085b40 con 0x7fed740d19c0 2024-05-08T22:42:19.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.622+0000 7fed7a120700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 0x7fed58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 Processor -- start 2024-05-08T22:42:19.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- start start 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a120700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 0x7fed58068bf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed740b28c0 tx=0x7fed68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x13d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1336810 con 0x140f680 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1333fa0 con 0x1357520 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x132f890 con 0x140f2a0 2024-05-08T22:42:19.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44010/0 (socket says 172.21.15.179:44010) 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 -- 172.21.15.179:0/3251193170 learned_addr learned my addr 172.21.15.179:0/3251193170 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7991f700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x13d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a120700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 -- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 msgr2=0x13d7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x13d7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 -- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 msgr2=0x140ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 -- 172.21.15.179:0/3251193170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11a1ba0 con 0x140f680 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7991f700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x13d7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7a921700 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fed70056890 tx=0x7fed70056510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c136689db1462cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed57fff700 1 -- 172.21.15.179:0/3251193170 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70056900 con 0x140f680 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed57fff700 1 -- 172.21.15.179:0/3251193170 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7005a730 con 0x140f680 2024-05-08T22:42:19.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed57fff700 1 -- 172.21.15.179:0/3251193170 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7005ecf0 con 0x140f680 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- 172.21.15.179:0/3251193170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 msgr2=0x13d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fed70056890 tx=0x7fed70056510 comp rx=0 tx=0).stop 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- 172.21.15.179:0/3251193170 shutdown_connections 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x13d8470 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x13d7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 --2- 172.21.15.179:0/3251193170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- 172.21.15.179:0/3251193170 >> 172.21.15.179:0/3251193170 conn(0x1156710 msgr2=0x13f0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- 172.21.15.179:0/3251193170 shutdown_connections 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.626+0000 7fed7f633740 1 -- 172.21.15.179:0/3251193170 wait complete. 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 -- start start 2024-05-08T22:42:19.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 Processor -- start 2024-05-08T22:42:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x1414f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x12f2d80 con 0x140f680 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12f2a20 con 0x1357520 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1222770 con 0x140f2a0 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44014/0 (socket says 172.21.15.179:44014) 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 -- 172.21.15.179:0/434885859 learned_addr learned my addr 172.21.15.179:0/434885859 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7f633740 10 client.0 _ll_register_callbacks cb 0x1328f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a120700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7991f700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x1414f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 -- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 msgr2=0x1414f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x1414f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 -- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 msgr2=0x140e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 -- 172.21.15.179:0/434885859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed740c9020 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed7a921700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fed70056890 tx=0x7fed70009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb95f9dff863b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.630+0000 7fed557fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70056900 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1163420 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fed70053010 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed700531d0 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 10 client.8670 fetch_fsmap learned FSMap version 68 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x12f5910 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1163420 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fed70056b60 con 0x140f680 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 10 client.8670 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 10 client.8670 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:42:19.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 20 client.8670 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x13c9d00 0x13c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 4 client.8670 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x11ad6a0 con 0x13c9d00 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7a120700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x13c9d00 0x13c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 10 client.8670 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7a120700 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x13c9d00 0x13c8e70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f1f8512205edb8 server_cookie=582ce0ac73f29868 in_seq=0 out_seq=0 2024-05-08T22:42:19.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 1 -- 172.21.15.179:0/434885859 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x11ad6a0 con 0x13c9d00 2024-05-08T22:42:19.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 10 client.8670 handle_command_reply: tid=0 2024-05-08T22:42:19.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 client.8670 shutdown 2024-05-08T22:42:19.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 msgr2=0x1415480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fed70056890 tx=0x7fed70009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 shutdown_connections 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x13c9d00 0x13c8e70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed557fa700 0 client.8670 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f680 0x1415480 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x1414f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 --2- 172.21.15.179:0/434885859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1357520 0x140e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 >> 172.21.15.179:0/434885859 conn(0x1156710 msgr2=0x13ee1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 shutdown_connections 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 1 -- 172.21.15.179:0/434885859 wait complete. 2024-05-08T22:42:19.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.634+0000 7fed7f633740 20 client.8670 trim_cache size 0 max 16384 2024-05-08T22:42:19.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 Processor -- start 2024-05-08T22:42:19.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 -- start start 2024-05-08T22:42:19.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1357520 0x141e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x1414ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x13569a0 con 0x140f2a0 2024-05-08T22:42:19.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12f0660 con 0x140f680 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12d4730 con 0x1357520 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44020/0 (socket says 172.21.15.179:44020) 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 -- 172.21.15.179:0/1685537600 learned_addr learned my addr 172.21.15.179:0/1685537600 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7a921700 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x1414ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7a120700 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1357520 0x141e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 -- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1357520 msgr2=0x141e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1357520 0x141e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 -- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 msgr2=0x1414ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x1414ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 -- 172.21.15.179:0/1685537600 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12dbb40 con 0x140f2a0 2024-05-08T22:42:19.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.650+0000 7fed7991f700 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fed741f2f80 tx=0x7fed6c0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331625649989f440 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed64dae700 1 -- 172.21.15.179:0/1685537600 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c015070 con 0x140f2a0 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed64dae700 1 -- 172.21.15.179:0/1685537600 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c00e5f0 con 0x140f2a0 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed64dae700 1 -- 172.21.15.179:0/1685537600 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00e960 con 0x140f2a0 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/1685537600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 msgr2=0x141eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fed741f2f80 tx=0x7fed6c0106e0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/1685537600 shutdown_connections 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x1414ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x140f2a0 0x141eb40 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- 172.21.15.179:0/1685537600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1357520 0x141e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/1685537600 >> 172.21.15.179:0/1685537600 conn(0x1156710 msgr2=0x141def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/1685537600 shutdown_connections 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/1685537600 wait complete. 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- start start 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 Processor -- start 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x140d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x142e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x132b080 con 0x1357520 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x13323f0 con 0x140f680 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12ed0e0 con 0x140f2a0 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 10 client.0 _ll_register_callbacks cb 0x132e460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44026/0 (socket says 172.21.15.179:44026) 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 -- 172.21.15.179:0/2116910373 learned_addr learned my addr 172.21.15.179:0/2116910373 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a921700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x142e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7991f700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x140d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 -- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 msgr2=0x140d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x140d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 -- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 msgr2=0x142e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x142e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 -- 172.21.15.179:0/2116910373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c007bc0 con 0x1357520 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed557fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7a120700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fed68009010 tx=0x7fed680037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f5fed926881fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68016e50 con 0x1357520 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fed6802fd00 con 0x1357520 2024-05-08T22:42:19.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.654+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed7005a730 con 0x1357520 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed68005020 con 0x1357520 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 10 client.8676 fetch_fsmap learned FSMap version 68 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x12cfc20 con 0x1357520 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7005a730 con 0x1357520 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fed68005180 con 0x1357520 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 10 client.8676 fetch_fsmap finished waiting for FSMap version 68 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 10 client.8676 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 20 client.8676 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1472100 0x14724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 4 client.8676 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 --> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x11d2420 con 0x1472100 2024-05-08T22:42:19.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7991f700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1472100 0x14724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 10 client.8676 ms_handle_connect on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7991f700 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1472100 0x14724e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c376f03a5ddc6f server_cookie=f45d61a16bf7a9fa in_seq=0 out_seq=0 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 1 -- 172.21.15.179:0/2116910373 <== mds.0 v2:172.21.15.179:6834/249337448 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x11d2420 con 0x1472100 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 10 client.8676 handle_command_reply: tid=0 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 client.8676 shutdown 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 msgr2=0x13c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fed68009010 tx=0x7fed680037a0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 shutdown_connections 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x140f680 0x142e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:6834/249337448,v1:172.21.15.179:6835/249337448] conn(0x1472100 0x14724e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x140f2a0 0x140d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 --2- 172.21.15.179:0/2116910373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1357520 0x13c7450 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 >> 172.21.15.179:0/2116910373 conn(0x1156710 msgr2=0x141e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed557fa700 0 client.8676 ms_handle_reset on v2:172.21.15.179:6834/249337448 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 shutdown_connections 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 1 -- 172.21.15.179:0/2116910373 wait complete. 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7f633740 20 client.8676 trim_cache size 0 max 16384 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:19.667 INFO:teuthology.orchestra.run.smithi179.stdout:[] 2024-05-08T22:42:19.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 msgr2=0x7fed58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 0x7fed58068bf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed740b28c0 tx=0x7fed68009040 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 msgr2=0x7fed740cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fed70008000 tx=0x7fed70004770 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 shutdown_connections 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fed58066730 0x7fed58068bf0 secure :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed740b28c0 tx=0x7fed68009040 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed740d15e0 0x7fed741f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed740d19c0 0x7fed740cc950 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 --2- 172.21.15.179:0/3499822579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed740ce450 0x7fed74143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 >> 172.21.15.179:0/3499822579 conn(0x7fed740b49a0 msgr2=0x7fed7406c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 shutdown_connections 2024-05-08T22:42:19.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.658+0000 7fed7b122700 1 -- 172.21.15.179:0/3499822579 wait complete. 2024-05-08T22:42:19.674 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T22:42:19.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.782+0000 7fa1e5a08700 1 Processor -- start 2024-05-08T22:42:19.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.782+0000 7fa1e5a08700 1 -- start start 2024-05-08T22:42:19.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015d580 0x7fa1e00b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1e0161640 0x7fa1e00d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00d2640 con 0x7fa1e00d18a0 2024-05-08T22:42:19.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e006b3a0 con 0x7fa1e0161640 2024-05-08T22:42:19.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00b6750 con 0x7fa1e015d580 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1e0161640 0x7fa1e00d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1dffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa1e00d18a0 0x7fa1e0160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55884/0 (socket says 172.21.15.179:55884) 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1dffff700 1 -- 172.21.15.179:0/1592208733 learned_addr learned my addr 172.21.15.179:0/1592208733 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e4a06700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015d580 0x7fa1e00b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1df7fe700 1 -- 172.21.15.179:0/1592208733 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646458376 0 0) 0x7fa1e00d2640 con 0x7fa1e00d18a0 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1df7fe700 1 -- 172.21.15.179:0/1592208733 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c80035d0 con 0x7fa1e00d18a0 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 -- 172.21.15.179:0/1592208733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015d580 msgr2=0x7fa1e00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015d580 0x7fa1e00b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 -- 172.21.15.179:0/1592208733 >> v1:172.21.15.179:6789/0 conn(0x7fa1e00d18a0 legacy=0x7fa1e0160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 -- 172.21.15.179:0/1592208733 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00d2360 con 0x7fa1e0161640 2024-05-08T22:42:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5207700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00d1d00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009d90 tx=0x7fa1d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d021328b26792328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1df7fe700 1 -- 172.21.15.179:0/1592208733 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800e070 con 0x7fa1e0161640 2024-05-08T22:42:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1df7fe700 1 -- 172.21.15.179:0/1592208733 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d800bb20 con 0x7fa1e0161640 2024-05-08T22:42:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1df7fe700 1 -- 172.21.15.179:0/1592208733 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800be90 con 0x7fa1e0161640 2024-05-08T22:42:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- 172.21.15.179:0/1592208733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 msgr2=0x7fa1e00d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00d1d00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009d90 tx=0x7fa1d800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- 172.21.15.179:0/1592208733 shutdown_connections 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00d1d00 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/1592208733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015d580 0x7fa1e00b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.786+0000 7fa1e5a08700 1 -- 172.21.15.179:0/1592208733 >> 172.21.15.179:0/1592208733 conn(0x7fa1e00d2d90 msgr2=0x7fa1e00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- 172.21.15.179:0/1592208733 shutdown_connections 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- 172.21.15.179:0/1592208733 wait complete. 2024-05-08T22:42:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 Processor -- start 2024-05-08T22:42:19.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- start start 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 0x7fa1e01600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e015d580 0x7fa1e0156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0159c60 con 0x7fa1e0161640 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01f3940 con 0x7fa1e015d580 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5a08700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01f2dc0 con 0x7fa1e00d18a0 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44042/0 (socket says 172.21.15.179:44042) 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 -- 172.21.15.179:0/882558615 learned_addr learned my addr 172.21.15.179:0/882558615 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:19.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e4a06700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 0x7fa1e01600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1dffff700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e015d580 0x7fa1e0156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 -- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 msgr2=0x7fa1e01600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 0x7fa1e01600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 -- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e015d580 msgr2=0x7fa1e0156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e015d580 0x7fa1e0156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e5207700 1 -- 172.21.15.179:0/882558615 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009a70 con 0x7fa1e0161640 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.790+0000 7fa1e4a06700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 0x7fa1e01600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1e5207700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8014040 tx=0x7fa1d8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8007000 con 0x7fa1e0161640 2024-05-08T22:42:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8012860 con 0x7fa1e0161640 2024-05-08T22:42:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8012c10 con 0x7fa1e0161640 2024-05-08T22:42:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e01f30a0 con 0x7fa1e0161640 2024-05-08T22:42:19.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01f3660 con 0x7fa1e0161640 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e015ea20 con 0x7fa1e0161640 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.794+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1d801c400 con 0x7fa1e0161640 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.798+0000 7fa1dd7fa700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 0x7fa1cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.798+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fa1d8086110 con 0x7fa1e0161640 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.798+0000 7fa1e4a06700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 0x7fa1cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:19.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.798+0000 7fa1e4a06700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 0x7fa1cc068bf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0160030 tx=0x7fa1d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:19.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:19.806+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1d8053360 con 0x7fa1e0161640 2024-05-08T22:42:20.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa1e004e700 con 0x7fa1e0161640 2024-05-08T22:42:20.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1dd7fa700 1 -- 172.21.15.179:0/882558615 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa1d8056980 con 0x7fa1e0161640 2024-05-08T22:42:20.015 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:20.015 INFO:teuthology.orchestra.run.smithi179.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:42:20.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 msgr2=0x7fa1cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 0x7fa1cc068bf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0160030 tx=0x7fa1d4008040 comp rx=0 tx=0).stop 2024-05-08T22:42:20.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 msgr2=0x7fa1e00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8014040 tx=0x7fa1d8004af0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 shutdown_connections 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa1cc066730 0x7fa1cc068bf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e015d580 0x7fa1e0156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e0161640 0x7fa1e00c9e60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 --2- 172.21.15.179:0/882558615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e00d18a0 0x7fa1e01600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 >> 172.21.15.179:0/882558615 conn(0x7fa1e00d2d90 msgr2=0x7fa1e00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 shutdown_connections 2024-05-08T22:42:20.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.014+0000 7fa1e5a08700 1 -- 172.21.15.179:0/882558615 wait complete. 2024-05-08T22:42:20.029 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T22:42:20.030 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:42:20.030 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:42:20.045 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:42:20.067 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:42:20.067 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:20.067 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:20.074 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:20.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 Processor -- start 2024-05-08T22:42:20.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 -- start start 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5804143640 0x7f580406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58040cea40 0x7f58040b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f580415d970 0x7f580415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580415e340 con 0x7f58040cea40 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580406ce80 con 0x7f580415d970 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580909c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5804143a20 con 0x7f5804143640 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f58037fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58040cea40 0x7f58040b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f58037fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58040cea40 0x7f58040b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44052/0 (socket says 172.21.15.179:44052) 2024-05-08T22:42:20.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f58037fe700 1 -- 172.21.15.179:0/2870160982 learned_addr learned my addr 172.21.15.179:0/2870160982 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f5803fff700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5804143640 0x7f580406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.234+0000 7f580889b700 1 --2- 172.21.15.179:0/2870160982 >> v2:172.21.15.195:3300/0 conn(0x7f580415d970 0x7f580415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 -- 172.21.15.179:0/2870160982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5804143640 msgr2=0x7f580406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5804143640 0x7f580406eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 -- 172.21.15.179:0/2870160982 >> v2:172.21.15.195:3300/0 conn(0x7f580415d970 msgr2=0x7f580415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 --2- 172.21.15.179:0/2870160982 >> v2:172.21.15.195:3300/0 conn(0x7f580415d970 0x7f580415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 -- 172.21.15.179:0/2870160982 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040cf220 con 0x7f58040cea40 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f58037fe700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58040cea40 0x7f58040b2a50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f580415d8b0 tx=0x7f57ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd483af8ad114653 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f5802ffd700 1 -- 172.21.15.179:0/2870160982 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec011830 con 0x7f58040cea40 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f5802ffd700 1 -- 172.21.15.179:0/2870160982 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec011990 con 0x7f58040cea40 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f5802ffd700 1 -- 172.21.15.179:0/2870160982 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec011d20 con 0x7f58040cea40 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- 172.21.15.179:0/2870160982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58040cea40 msgr2=0x7f58040b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58040cea40 0x7f58040b2a50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f580415d8b0 tx=0x7f57ec009bf0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- 172.21.15.179:0/2870160982 shutdown_connections 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 --2- 172.21.15.179:0/2870160982 >> v2:172.21.15.195:3300/0 conn(0x7f580415d970 0x7f580415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58040cea40 0x7f58040b2a50 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 --2- 172.21.15.179:0/2870160982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5804143640 0x7f580406eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- 172.21.15.179:0/2870160982 >> 172.21.15.179:0/2870160982 conn(0x7f58040cfae0 msgr2=0x7f58040cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- 172.21.15.179:0/2870160982 shutdown_connections 2024-05-08T22:42:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- 172.21.15.179:0/2870160982 wait complete. 2024-05-08T22:42:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 Processor -- start 2024-05-08T22:42:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.238+0000 7f580909c700 1 -- start start 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58040cea40 0x7f58040aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f580415d970 0x7f58040b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041f61d0 con 0x7f5804143640 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041f6420 con 0x7f580415d970 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58041f6700 con 0x7f58040cea40 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44054/0 (socket says 172.21.15.179:44054) 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 -- 172.21.15.179:0/2671608009 learned_addr learned my addr 172.21.15.179:0/2671608009 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f5803fff700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58040cea40 0x7f58040aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580889b700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f580415d970 0x7f58040b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 -- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58040cea40 msgr2=0x7f58040aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58040cea40 0x7f58040aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 -- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f580415d970 msgr2=0x7f58040b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f580415d970 0x7f58040b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 -- 172.21.15.179:0/2671608009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009570 con 0x7f5804143640 2024-05-08T22:42:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f58037fe700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f57ec014040 tx=0x7f57ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec004450 con 0x7f5804143640 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec01e070 con 0x7f5804143640 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58041f69e0 con 0x7f5804143640 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003900 con 0x7f5804143640 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58040af2d0 con 0x7f5804143640 2024-05-08T22:42:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.242+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58040af4c0 con 0x7f5804143640 2024-05-08T22:42:20.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.246+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57ec003a60 con 0x7f5804143640 2024-05-08T22:42:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.246+0000 7f5800ff9700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 0x7f57e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.246+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f57ec084df0 con 0x7f5804143640 2024-05-08T22:42:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.246+0000 7f5803fff700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 0x7f57e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.246+0000 7f5803fff700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 0x7f57e8068bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f57f4011340 tx=0x7f57f400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.250+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57ec052010 con 0x7f5804143640 2024-05-08T22:42:20.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.422+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5804072c50 con 0x7f5804143640 2024-05-08T22:42:20.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f5800ff9700 1 -- 172.21.15.179:0/2671608009 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16751 (secure 0 0 0) 0x7f57ec015090 con 0x7f5804143640 2024-05-08T22:42:20.428 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:20.428 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":100,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:41:40.642772+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6834/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:20.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 msgr2=0x7f57e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 0x7f57e8068bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f57f4011340 tx=0x7f57f400b040 comp rx=0 tx=0).stop 2024-05-08T22:42:20.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 msgr2=0x7f58040b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f57ec014040 tx=0x7f57ec004770 comp rx=0 tx=0).stop 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 shutdown_connections 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f57e8066730 0x7f57e8068bf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f580415d970 0x7f58040b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5804143640 0x7f58040b0840 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.426+0000 7f580909c700 1 --2- 172.21.15.179:0/2671608009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58040cea40 0x7f58040aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.430+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 >> 172.21.15.179:0/2671608009 conn(0x7f58040cfae0 msgr2=0x7f58040cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.430+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 shutdown_connections 2024-05-08T22:42:20.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.430+0000 7f580909c700 1 -- 172.21.15.179:0/2671608009 wait complete. 2024-05-08T22:42:20.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:20.443 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:20.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 Processor -- start 2024-05-08T22:42:20.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 -- start start 2024-05-08T22:42:20.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52815d050 0x7fd5280b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd5280b4aa0 0x7fd528160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280cb640 con 0x7fd5281611b0 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd528161b50 con 0x7fd5280b4aa0 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd52d15a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd528161640 con 0x7fd52815d050 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd5277fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd5280b4aa0 0x7fd528160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd5277fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd5280b4aa0 0x7fd528160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44056/0 (socket says 172.21.15.179:44056) 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd5277fe700 1 -- 172.21.15.179:0/388436000 learned_addr learned my addr 172.21.15.179:0/388436000 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.558+0000 7fd527fff700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52815d050 0x7fd5280b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd5277fe700 1 -- 172.21.15.179:0/388436000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52815d050 msgr2=0x7fd5280b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd5277fe700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52815d050 0x7fd5280b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd5277fe700 1 -- 172.21.15.179:0/388436000 >> v1:172.21.15.195:6789/0 conn(0x7fd5281611b0 legacy=0x7fd5280cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd5277fe700 1 -- 172.21.15.179:0/388436000 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280cb310 con 0x7fd5280b4aa0 2024-05-08T22:42:20.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd526ffd700 1 -- 172.21.15.179:0/388436000 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516207499 0 0) 0x7fd5280cb640 con 0x7fd5281611b0 2024-05-08T22:42:20.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd5277fe700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5280b4aa0 0x7fd528160c70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd518009910 tx=0x7fd518009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8962f1ab59346d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd526ffd700 1 -- 172.21.15.179:0/388436000 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011830 con 0x7fd5280b4aa0 2024-05-08T22:42:20.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd526ffd700 1 -- 172.21.15.179:0/388436000 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518011990 con 0x7fd5280b4aa0 2024-05-08T22:42:20.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd526ffd700 1 -- 172.21.15.179:0/388436000 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011d40 con 0x7fd5280b4aa0 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- 172.21.15.179:0/388436000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5280b4aa0 msgr2=0x7fd528160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5280b4aa0 0x7fd528160c70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd518009910 tx=0x7fd518009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- 172.21.15.179:0/388436000 shutdown_connections 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5280b4aa0 0x7fd528160c70 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 --2- 172.21.15.179:0/388436000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd52815d050 0x7fd5280b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- 172.21.15.179:0/388436000 >> 172.21.15.179:0/388436000 conn(0x7fd5280b2710 msgr2=0x7fd5280b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- 172.21.15.179:0/388436000 shutdown_connections 2024-05-08T22:42:20.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- 172.21.15.179:0/388436000 wait complete. 2024-05-08T22:42:20.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 Processor -- start 2024-05-08T22:42:20.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.562+0000 7fd52d15a700 1 -- start start 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5280b4aa0 0x7fd5280ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5281611b0 0x7fd528159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280c5410 con 0x7fd52815d050 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280c56f0 con 0x7fd5280b4aa0 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280c7cf0 con 0x7fd5281611b0 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44068/0 (socket says 172.21.15.179:44068) 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 -- 172.21.15.179:0/3003294527 learned_addr learned my addr 172.21.15.179:0/3003294527 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd527fff700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5280b4aa0 0x7fd5280ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52c959700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5281611b0 0x7fd528159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 -- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5281611b0 msgr2=0x7fd528159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5281611b0 0x7fd528159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 -- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5280b4aa0 msgr2=0x7fd5280ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5280b4aa0 0x7fd5280ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 -- 172.21.15.179:0/3003294527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518009570 con 0x7fd52815d050 2024-05-08T22:42:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd5277fe700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd518009f70 tx=0x7fd518004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801a900 con 0x7fd52815d050 2024-05-08T22:42:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51801f070 con 0x7fd52815d050 2024-05-08T22:42:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801acc0 con 0x7fd52815d050 2024-05-08T22:42:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5280c7fd0 con 0x7fd52815d050 2024-05-08T22:42:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.566+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5281f2890 con 0x7fd52815d050 2024-05-08T22:42:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5281f2a20 con 0x7fd52815d050 2024-05-08T22:42:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd51801b430 con 0x7fd52815d050 2024-05-08T22:42:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd524ff9700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fd518021070 con 0x7fd52815d050 2024-05-08T22:42:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd527fff700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.570+0000 7fd527fff700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 0x7fd514068bf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009bf0 tx=0x7fd51c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.574+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd5281f2a20 con 0x7fd52815d050 2024-05-08T22:42:20.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.742+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5280c84f0 con 0x7fd52815d050 2024-05-08T22:42:20.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd524ff9700 1 -- 172.21.15.179:0/3003294527 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5280c84f0 con 0x7fd52815d050 2024-05-08T22:42:20.747 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:20.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 msgr2=0x7fd514068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 0x7fd514068bf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009bf0 tx=0x7fd51c009240 comp rx=0 tx=0).stop 2024-05-08T22:42:20.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 msgr2=0x7fd528158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd518009f70 tx=0x7fd518004770 comp rx=0 tx=0).stop 2024-05-08T22:42:20.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 shutdown_connections 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd514066730 0x7fd514068bf0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5281611b0 0x7fd528159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52815d050 0x7fd528158c30 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 --2- 172.21.15.179:0/3003294527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5280b4aa0 0x7fd5280ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 >> 172.21.15.179:0/3003294527 conn(0x7fd5280b2710 msgr2=0x7fd52815d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 shutdown_connections 2024-05-08T22:42:20.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.746+0000 7fd52d15a700 1 -- 172.21.15.179:0/3003294527 wait complete. 2024-05-08T22:42:20.762 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:20.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 Processor -- start 2024-05-08T22:42:20.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 -- start start 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f585c0aec70 0x7f585c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f585c0ccea0 0x7f585c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c0b4000 con 0x7f585c0cbb50 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585c0b15b0 con 0x7f585c0ccea0 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586217d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585c0b1850 con 0x7f585c0aec70 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586197c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f585c0ccea0 0x7f585c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.874+0000 7f586197c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f585c0ccea0 0x7f585c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44074/0 (socket says 172.21.15.179:44074) 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 -- 172.21.15.179:0/3295879906 learned_addr learned my addr 172.21.15.179:0/3295879906 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586097a700 1 --2- 172.21.15.179:0/3295879906 >> v2:172.21.15.195:3300/0 conn(0x7f585c0aec70 0x7f585c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 -- 172.21.15.179:0/3295879906 >> v2:172.21.15.195:3300/0 conn(0x7f585c0aec70 msgr2=0x7f585c0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 --2- 172.21.15.179:0/3295879906 >> v2:172.21.15.195:3300/0 conn(0x7f585c0aec70 0x7f585c0b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 -- 172.21.15.179:0/3295879906 >> v1:172.21.15.195:6789/0 conn(0x7f585c0cbb50 legacy=0x7f585c0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 -- 172.21.15.179:0/3295879906 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0b1b30 con 0x7f585c0ccea0 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586197c700 1 --2- 172.21.15.179:0/3295879906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0ccea0 0x7f585c0b4490 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f585c0b5440 tx=0x7f585400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2e822dbe124987 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f5853fff700 1 -- 172.21.15.179:0/3295879906 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854010070 con 0x7f585c0ccea0 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f5853fff700 1 -- 172.21.15.179:0/3295879906 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585400dad0 con 0x7f585c0ccea0 2024-05-08T22:42:20.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f5853fff700 1 -- 172.21.15.179:0/3295879906 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585400de80 con 0x7f585c0ccea0 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 -- 172.21.15.179:0/3295879906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0ccea0 msgr2=0x7f585c0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 --2- 172.21.15.179:0/3295879906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0ccea0 0x7f585c0b4490 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f585c0b5440 tx=0x7f585400d430 comp rx=0 tx=0).stop 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 -- 172.21.15.179:0/3295879906 shutdown_connections 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 --2- 172.21.15.179:0/3295879906 >> v2:172.21.15.195:3300/0 conn(0x7f585c0aec70 0x7f585c0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 --2- 172.21.15.179:0/3295879906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0ccea0 0x7f585c0b4490 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 -- 172.21.15.179:0/3295879906 >> 172.21.15.179:0/3295879906 conn(0x7f585c0b2ca0 msgr2=0x7f585c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 -- 172.21.15.179:0/3295879906 shutdown_connections 2024-05-08T22:42:20.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.878+0000 7f586217d700 1 -- 172.21.15.179:0/3295879906 wait complete. 2024-05-08T22:42:20.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 Processor -- start 2024-05-08T22:42:20.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 -- start start 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f585c0aec70 0x7f585c0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f585c0ccea0 0x7f585c1ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c159430 con 0x7f585c0cbb50 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0c4620 con 0x7f585c0aec70 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586217d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0cf290 con 0x7f585c0ccea0 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586097a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586097a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44090/0 (socket says 172.21.15.179:44090) 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586097a700 1 -- 172.21.15.179:0/2942893153 learned_addr learned my addr 172.21.15.179:0/2942893153 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586117b700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f585c0aec70 0x7f585c0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.882+0000 7f586197c700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f585c0ccea0 0x7f585c1ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 -- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f585c0ccea0 msgr2=0x7f585c1ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f585c0ccea0 0x7f585c1ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 -- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f585c0aec70 msgr2=0x7f585c0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f585c0aec70 0x7f585c0c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 -- 172.21.15.179:0/2942893153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585400bd40 con 0x7f585c0cbb50 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586097a700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e870 tx=0x7f584c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0095c0 con 0x7f585c0cbb50 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c013680 con 0x7f585c0cbb50 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585c0d0000 con 0x7f585c0cbb50 2024-05-08T22:42:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c013a10 con 0x7f585c0cbb50 2024-05-08T22:42:20.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c0c4ba0 con 0x7f585c0cbb50 2024-05-08T22:42:20.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585c04e700 con 0x7f585c0cbb50 2024-05-08T22:42:20.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f584c013b70 con 0x7f585c0cbb50 2024-05-08T22:42:20.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 0x7f5840068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:20.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f584c084270 con 0x7f585c0cbb50 2024-05-08T22:42:20.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.886+0000 7f586117b700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 0x7f5840068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:20.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.890+0000 7f586117b700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 0x7f5840068bf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:20.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:20.898+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f584c051300 con 0x7f585c0cbb50 2024-05-08T22:42:21.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.058+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f585c0ccc60 con 0x7f585c0cbb50 2024-05-08T22:42:21.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.058+0000 7f5851ffb700 1 -- 172.21.15.179:0/2942893153 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f584c054920 con 0x7f585c0cbb50 2024-05-08T22:42:21.062 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:21.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 msgr2=0x7f5840068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 0x7f5840068bf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 msgr2=0x7f585c0ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e870 tx=0x7f584c00eb80 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 shutdown_connections 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840066730 0x7f5840068bf0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f585c0ccea0 0x7f585c1ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f585c0cbb50 0x7f585c0ced50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 --2- 172.21.15.179:0/2942893153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f585c0aec70 0x7f585c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 >> 172.21.15.179:0/2942893153 conn(0x7f585c0b2ca0 msgr2=0x7f585c15ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 shutdown_connections 2024-05-08T22:42:21.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.062+0000 7f586217d700 1 -- 172.21.15.179:0/2942893153 wait complete. 2024-05-08T22:42:21.076 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:21.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 Processor -- start 2024-05-08T22:42:21.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 -- start start 2024-05-08T22:42:21.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f952815d0c0 0x7f95280b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95280ce2b0 0x7f952806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952815d4a0 con 0x7f95280d1d20 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952806abb0 con 0x7f95280ce2b0 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.194+0000 7f952d523700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280d2500 con 0x7f952815d0c0 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95280ce2b0 0x7f952806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95280ce2b0 0x7f952806b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44106/0 (socket says 172.21.15.179:44106) 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 -- 172.21.15.179:0/762858096 learned_addr learned my addr 172.21.15.179:0/762858096 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f951f7fe700 1 --1- 172.21.15.179:0/762858096 >> v1:172.21.15.179:6789/0 conn(0x7f95280d1d20 0x7f95280cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55894/0 (socket says 172.21.15.179:55894) 2024-05-08T22:42:21.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f9527fff700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f952815d0c0 0x7f95280b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f95277fe700 1 -- 172.21.15.179:0/762858096 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289817092 0 0) 0x7f952815d4a0 con 0x7f95280d1d20 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 -- 172.21.15.179:0/762858096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f952815d0c0 msgr2=0x7f95280b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f952815d0c0 0x7f95280b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 -- 172.21.15.179:0/762858096 >> v1:172.21.15.179:6789/0 conn(0x7f95280d1d20 legacy=0x7f95280cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 -- 172.21.15.179:0/762858096 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280d27e0 con 0x7f95280ce2b0 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952cd22700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95280ce2b0 0x7f952806b370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9520009e50 tx=0x7f952000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693b6b1252f8dd63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f95277fe700 1 -- 172.21.15.179:0/762858096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000e070 con 0x7f95280ce2b0 2024-05-08T22:42:21.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f95277fe700 1 -- 172.21.15.179:0/762858096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000b910 con 0x7f95280ce2b0 2024-05-08T22:42:21.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f95277fe700 1 -- 172.21.15.179:0/762858096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000bcc0 con 0x7f95280ce2b0 2024-05-08T22:42:21.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 -- 172.21.15.179:0/762858096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95280ce2b0 msgr2=0x7f952806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95280ce2b0 0x7f952806b370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9520009e50 tx=0x7f952000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:21.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 -- 172.21.15.179:0/762858096 shutdown_connections 2024-05-08T22:42:21.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95280ce2b0 0x7f952806b370 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 --2- 172.21.15.179:0/762858096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f952815d0c0 0x7f95280b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 -- 172.21.15.179:0/762858096 >> 172.21.15.179:0/762858096 conn(0x7f95280d2ed0 msgr2=0x7f95280d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 -- 172.21.15.179:0/762858096 shutdown_connections 2024-05-08T22:42:21.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.198+0000 7f952d523700 1 -- 172.21.15.179:0/762858096 wait complete. 2024-05-08T22:42:21.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 Processor -- start 2024-05-08T22:42:21.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- start start 2024-05-08T22:42:21.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95280d1d20 0x7f9528157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952815d0c0 0x7f9528157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280cb090 con 0x7f952815d0c0 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280ca6f0 con 0x7f95280d1d20 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280ca9d0 con 0x7f95280ce2b0 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952cd22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952815d0c0 0x7f9528157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:41082/0 (socket says 172.21.15.179:41082) 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 -- 172.21.15.179:0/3471883392 learned_addr learned my addr 172.21.15.179:0/3471883392 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:21.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f951f7fe700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95280d1d20 0x7f9528157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 -- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95280d1d20 msgr2=0x7f9528157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95280d1d20 0x7f9528157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 -- 172.21.15.179:0/3471883392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952815d0c0 msgr2=0x7f9528157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952815d0c0 0x7f9528157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 -- 172.21.15.179:0/3471883392 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9520009b30 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f9527fff700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f951000e840 tx=0x7f951000eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100095a0 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510013680 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100139f0 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95280cacb0 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.202+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9528155610 con 0x7f95280ce2b0 2024-05-08T22:42:21.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.206+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95280b7cf0 con 0x7f95280ce2b0 2024-05-08T22:42:21.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.206+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9510013b50 con 0x7f95280ce2b0 2024-05-08T22:42:21.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.206+0000 7f95257fa700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 0x7f950c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.206+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f9510084240 con 0x7f95280ce2b0 2024-05-08T22:42:21.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.206+0000 7f951f7fe700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 0x7f950c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.210+0000 7f951f7fe700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 0x7f950c068bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95280c8b90 tx=0x7f9518007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.218+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f95100512c0 con 0x7f95280ce2b0 2024-05-08T22:42:21.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.390+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f952804e700 con 0x7f95280ce2b0 2024-05-08T22:42:21.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.390+0000 7f95257fa700 1 -- 172.21.15.179:0/3471883392 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95100548e0 con 0x7f95280ce2b0 2024-05-08T22:42:21.391 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:21.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 msgr2=0x7f950c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 0x7f950c068bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95280c8b90 tx=0x7f9518007e00 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 msgr2=0x7f95280c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f951000e840 tx=0x7f951000eb50 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 shutdown_connections 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f950c066730 0x7f950c068bf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95280d1d20 0x7f9528157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952815d0c0 0x7f9528157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 --2- 172.21.15.179:0/3471883392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95280ce2b0 0x7f95280c83d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 >> 172.21.15.179:0/3471883392 conn(0x7f95280d2ed0 msgr2=0x7f95280b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 shutdown_connections 2024-05-08T22:42:21.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.394+0000 7f952d523700 1 -- 172.21.15.179:0/3471883392 wait complete. 2024-05-08T22:42:21.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:42:21.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 Processor -- start 2024-05-08T22:42:21.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- start start 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37c406dda0 0x7f37c40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 0x7f37c40b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c406b540 con 0x7f37c406e180 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c406ac50 con 0x7f37c406dda0 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c406aef0 con 0x7f37c40b25d0 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37c406dda0 0x7f37c40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f37c406dda0 0x7f37c40cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44112/0 (socket says 172.21.15.179:44112) 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 -- 172.21.15.179:0/2662367194 learned_addr learned my addr 172.21.15.179:0/2662367194 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c8e96700 1 --2- 172.21.15.179:0/2662367194 >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 0x7f37c40b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 -- 172.21.15.179:0/2662367194 >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 msgr2=0x7f37c40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 --2- 172.21.15.179:0/2662367194 >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 0x7f37c40b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 -- 172.21.15.179:0/2662367194 >> v1:172.21.15.195:6789/0 conn(0x7f37c406e180 legacy=0x7f37c406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 -- 172.21.15.179:0/2662367194 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c406b1d0 con 0x7f37c406dda0 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c8e96700 1 --2- 172.21.15.179:0/2662367194 >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 0x7f37c40b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c3fff700 1 --2- 172.21.15.179:0/2662367194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406dda0 0x7f37c40cbc30 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f37c4070af0 tx=0x7f37b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91246b5f97af159 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c2ffd700 1 -- 172.21.15.179:0/2662367194 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800e070 con 0x7f37c406dda0 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c2ffd700 1 -- 172.21.15.179:0/2662367194 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800b9c0 con 0x7f37c406dda0 2024-05-08T22:42:21.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c2ffd700 1 -- 172.21.15.179:0/2662367194 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800bd70 con 0x7f37c406dda0 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- 172.21.15.179:0/2662367194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406dda0 msgr2=0x7f37c40cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 --2- 172.21.15.179:0/2662367194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406dda0 0x7f37c40cbc30 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f37c4070af0 tx=0x7f37b800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- 172.21.15.179:0/2662367194 shutdown_connections 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 --2- 172.21.15.179:0/2662367194 >> v2:172.21.15.195:3300/0 conn(0x7f37c40b25d0 0x7f37c40b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 --2- 172.21.15.179:0/2662367194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406dda0 0x7f37c40cbc30 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- 172.21.15.179:0/2662367194 >> 172.21.15.179:0/2662367194 conn(0x7f37c40ce960 msgr2=0x7f37c40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- 172.21.15.179:0/2662367194 shutdown_connections 2024-05-08T22:42:21.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.522+0000 7f37c9697700 1 -- 172.21.15.179:0/2662367194 wait complete. 2024-05-08T22:42:21.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 Processor -- start 2024-05-08T22:42:21.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- start start 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37c406dda0 0x7f37c41fade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37c40b25d0 0x7f37c40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40ac9a0 con 0x7f37c406e180 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40abb10 con 0x7f37c406dda0 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41fb320 con 0x7f37c40b25d0 2024-05-08T22:42:21.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44118/0 (socket says 172.21.15.179:44118) 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 -- 172.21.15.179:0/1911742782 learned_addr learned my addr 172.21.15.179:0/1911742782 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c8e96700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37c40b25d0 0x7f37c40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c3fff700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37c406dda0 0x7f37c41fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 -- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37c40b25d0 msgr2=0x7f37c40acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37c40b25d0 0x7f37c40acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 -- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37c406dda0 msgr2=0x7f37c41fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37c406dda0 0x7f37c41fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 -- 172.21.15.179:0/1911742782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009b30 con 0x7f37c406e180 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c37fe700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c500 tx=0x7f37bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0052d0 con 0x7f37c406e180 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc005430 con 0x7f37c406e180 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c414cab0 con 0x7f37c406e180 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0055a0 con 0x7f37c406e180 2024-05-08T22:42:21.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.526+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c40ac000 con 0x7f37c406e180 2024-05-08T22:42:21.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.530+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c40acae0 con 0x7f37c406e180 2024-05-08T22:42:21.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.530+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37bc0038e0 con 0x7f37c406e180 2024-05-08T22:42:21.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.530+0000 7f37c0ff9700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 0x7f37b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:21.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.534+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f37bc0837e0 con 0x7f37c406e180 2024-05-08T22:42:21.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.534+0000 7f37c3fff700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 0x7f37b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:21.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.534+0000 7f37c3fff700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 0x7f37b0068bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000f80 tx=0x7f37b8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:21.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.534+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f37c40acae0 con 0x7f37c406e180 2024-05-08T22:42:21.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.718+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37c40acae0 con 0x7f37c406e180 2024-05-08T22:42:21.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.718+0000 7f37c0ff9700 1 -- 172.21.15.179:0/1911742782 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v100) v1 ==== 80+0+244 (secure 0 0 0) 0x7f37c40acae0 con 0x7f37c406e180 2024-05-08T22:42:21.720 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6835/55939533 2024-05-09T22:41:40.642711+0000 2024-05-08T22:42:21.720 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6834/55939533 2024-05-09T22:41:40.642711+0000 2024-05-08T22:42:21.720 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6835/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:42:21.720 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179:6834/1693330449 2024-05-09T22:40:19.823587+0000 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.718+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 msgr2=0x7f37b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.718+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 0x7f37b0068bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000f80 tx=0x7f37b8009000 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 msgr2=0x7f37c414c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c500 tx=0x7f37bc00c8c0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 shutdown_connections 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f37b0066730 0x7f37b0068bf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37c40b25d0 0x7f37c40acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37c406e180 0x7f37c414c570 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 --2- 172.21.15.179:0/1911742782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37c406dda0 0x7f37c41fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:21.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 >> 172.21.15.179:0/1911742782 conn(0x7f37c40ce960 msgr2=0x7f37c40cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:21.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 shutdown_connections 2024-05-08T22:42:21.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:21.722+0000 7f37c9697700 1 -- 172.21.15.179:0/1911742782 wait complete. 2024-05-08T22:42:21.724 INFO:teuthology.orchestra.run.smithi179.stderr:listed 4 entries 2024-05-08T22:42:21.735 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:42:21.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:21.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:21.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[47869]: fuse finished with error 0 and tester_r 0 2024-05-08T22:42:21.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:42:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:21.746 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:42:21.915 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:42:21.915 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:21.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:42:21.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:42:21.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:21.923 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:42:21.972 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:42:21.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:42:21.973 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:21.973 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:21.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:42:21.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:42:21.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:21.982 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:42:22.028 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:42:22.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:42:22.030 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:42:22.030 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:22.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:42:22.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:42:22.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:22.038 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:42:22.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:42:22.045 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:42:22.045 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:22.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 Processor -- start 2024-05-08T22:42:22.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 -- start start 2024-05-08T22:42:22.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f222c0b3550 0x7f222c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0b39a0 con 0x7f222c1539c0 2024-05-08T22:42:22.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c159b60 con 0x7f222c0b3550 2024-05-08T22:42:22.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f2230889700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c159e40 con 0x7f222c0b19e0 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f222b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f222c0b3550 0x7f222c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f222a59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f222b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f222c0b3550 0x7f222c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44126/0 (socket says 172.21.15.179:44126) 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.162+0000 7f222b59e700 1 -- 172.21.15.179:0/4194755276 learned_addr learned my addr 172.21.15.179:0/4194755276 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f222b59e700 1 -- 172.21.15.179:0/4194755276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 msgr2=0x7f222c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f222b59e700 1 --2- 172.21.15.179:0/4194755276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f222b59e700 1 -- 172.21.15.179:0/4194755276 >> v1:172.21.15.195:6789/0 conn(0x7f222c1539c0 legacy=0x7f222c06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f222b59e700 1 -- 172.21.15.179:0/4194755276 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c15a120 con 0x7f222c0b3550 2024-05-08T22:42:22.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f222b59e700 1 --2- 172.21.15.179:0/4194755276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c0b3550 0x7f222c15d120 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f2224009910 tx=0x7f2224009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce28888a55dba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2229d9b700 1 -- 172.21.15.179:0/4194755276 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011830 con 0x7f222c0b3550 2024-05-08T22:42:22.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2229d9b700 1 -- 172.21.15.179:0/4194755276 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224011990 con 0x7f222c0b3550 2024-05-08T22:42:22.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2229d9b700 1 -- 172.21.15.179:0/4194755276 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011d40 con 0x7f222c0b3550 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 -- 172.21.15.179:0/4194755276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c0b3550 msgr2=0x7f222c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 --2- 172.21.15.179:0/4194755276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c0b3550 0x7f222c15d120 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f2224009910 tx=0x7f2224009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 -- 172.21.15.179:0/4194755276 shutdown_connections 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 --2- 172.21.15.179:0/4194755276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 --2- 172.21.15.179:0/4194755276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c0b3550 0x7f222c15d120 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 -- 172.21.15.179:0/4194755276 >> 172.21.15.179:0/4194755276 conn(0x7f222c0ce9d0 msgr2=0x7f222c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 -- 172.21.15.179:0/4194755276 shutdown_connections 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.166+0000 7f2230889700 1 -- 172.21.15.179:0/4194755276 wait complete. 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 Processor -- start 2024-05-08T22:42:22.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 -- start start 2024-05-08T22:42:22.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c1449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f222c0b3550 0x7f222c141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c144f80 con 0x7f222c1539c0 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c145260 con 0x7f222c0b3550 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f2230889700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c145540 con 0x7f222c0b19e0 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.170+0000 7f222b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222ad9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c1449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44138/0 (socket says 172.21.15.179:44138) 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 -- 172.21.15.179:0/1683695185 learned_addr learned my addr 172.21.15.179:0/1683695185 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f222c0b3550 0x7f222c141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 -- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 msgr2=0x7f222c1449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c1449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 -- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f222c0b3550 msgr2=0x7f222c141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f222c0b3550 0x7f222c141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 -- 172.21.15.179:0/1683695185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009570 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222ad9d700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c1449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f222b59e700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2224009150 tx=0x7f2224004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224004450 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22240045b0 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1517f0 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224003900 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1426e0 con 0x7f222c1539c0 2024-05-08T22:42:22.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.174+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c067740 con 0x7f222c1539c0 2024-05-08T22:42:22.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.178+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2224003a60 con 0x7f222c1539c0 2024-05-08T22:42:22.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.178+0000 7f22237fe700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 0x7f220c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.178+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f22240874b0 con 0x7f222c1539c0 2024-05-08T22:42:22.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.178+0000 7f222ad9d700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 0x7f220c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.178+0000 7f222ad9d700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 0x7f220c068bf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f222c153170 tx=0x7f2214009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.186+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2224054650 con 0x7f222c1539c0 2024-05-08T22:42:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.394+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f222c04e700 con 0x7f222c1539c0 2024-05-08T22:42:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.394+0000 7f22237fe700 1 -- 172.21.15.179:0/1683695185 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f2224057c70 con 0x7f222c1539c0 2024-05-08T22:42:22.396 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:22.396 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 msgr2=0x7f220c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 0x7f220c068bf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f222c153170 tx=0x7f2214009040 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 msgr2=0x7f222c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2224009150 tx=0x7f2224004770 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 shutdown_connections 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f220c066730 0x7f220c068bf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f222c0b3550 0x7f222c141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f222c1539c0 0x7f222c0cd750 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 --2- 172.21.15.179:0/1683695185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f222c0b19e0 0x7f222c1449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 >> 172.21.15.179:0/1683695185 conn(0x7f222c0ce9d0 msgr2=0x7f222c15ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:22.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 shutdown_connections 2024-05-08T22:42:22.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.398+0000 7f2230889700 1 -- 172.21.15.179:0/1683695185 wait complete. 2024-05-08T22:42:22.400 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:42:22.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:22.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.518+0000 7fda88d73700 1 Processor -- start 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.518+0000 7fda88d73700 1 -- start start 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 0x7fda840cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8406abb0 con 0x7fda840cc700 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda840cb9a0 con 0x7fda840ce380 2024-05-08T22:42:22.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8406ae90 con 0x7fda840cb510 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda82ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 0x7fda840cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:41132/0 (socket says 172.21.15.179:41132) 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda837fe700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fda840cc700 0x7fda8415b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:41732/0 (socket says 172.21.15.179:41732) 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda82ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 0x7fda840cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:60994/0 (socket says 172.21.15.179:60994) 2024-05-08T22:42:22.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 -- 172.21.15.179:0/1410619204 learned_addr learned my addr 172.21.15.179:0/1410619204 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:22.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 -- 172.21.15.179:0/1410619204 >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 msgr2=0x7fda840cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:22.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 --2- 172.21.15.179:0/1410619204 >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 0x7fda840cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 -- 172.21.15.179:0/1410619204 >> v1:172.21.15.195:6789/0 conn(0x7fda840cc700 legacy=0x7fda8415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 -- 172.21.15.179:0/1410619204 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840ce800 con 0x7fda840cb510 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda83fff700 1 --2- 172.21.15.179:0/1410619204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009910 tx=0x7fda7c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b17a7935f9706ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda827fc700 1 -- 172.21.15.179:0/1410619204 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011830 con 0x7fda840cb510 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda827fc700 1 -- 172.21.15.179:0/1410619204 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c011990 con 0x7fda840cb510 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda827fc700 1 -- 172.21.15.179:0/1410619204 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011d40 con 0x7fda840cb510 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- 172.21.15.179:0/1410619204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 msgr2=0x7fda8406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 --2- 172.21.15.179:0/1410619204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009910 tx=0x7fda7c009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- 172.21.15.179:0/1410619204 shutdown_connections 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 --2- 172.21.15.179:0/1410619204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cb510 0x7fda8406b320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 --2- 172.21.15.179:0/1410619204 >> v2:172.21.15.195:3300/0 conn(0x7fda840ce380 0x7fda840cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- 172.21.15.179:0/1410619204 >> 172.21.15.179:0/1410619204 conn(0x7fda840ce9d0 msgr2=0x7fda840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:22.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- 172.21.15.179:0/1410619204 shutdown_connections 2024-05-08T22:42:22.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.522+0000 7fda88d73700 1 -- 172.21.15.179:0/1410619204 wait complete. 2024-05-08T22:42:22.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 Processor -- start 2024-05-08T22:42:22.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- start start 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda840cb510 0x7fda841568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cc700 0x7fda84158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841537d0 con 0x7fda840ce380 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84153a20 con 0x7fda840cb510 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda84153d00 con 0x7fda840cc700 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44150/0 (socket says 172.21.15.179:44150) 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 -- 172.21.15.179:0/3482597136 learned_addr learned my addr 172.21.15.179:0/3482597136 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda82ffd700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cc700 0x7fda84158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda837fe700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda840cb510 0x7fda841568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 -- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cc700 msgr2=0x7fda84158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cc700 0x7fda84158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 -- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda840cb510 msgr2=0x7fda841568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda840cb510 0x7fda841568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 -- 172.21.15.179:0/3482597136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c009570 con 0x7fda840ce380 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda83fff700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fda8406afd0 tx=0x7fda7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004300 con 0x7fda840ce380 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c01f070 con 0x7fda840ce380 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0036c0 con 0x7fda840ce380 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda84153fe0 con 0x7fda840ce380 2024-05-08T22:42:22.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.526+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841586b0 con 0x7fda840ce380 2024-05-08T22:42:22.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.530+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda84154500 con 0x7fda840ce380 2024-05-08T22:42:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.530+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda7c021030 con 0x7fda840ce380 2024-05-08T22:42:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.530+0000 7fda7bfff700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 0x7fda64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.530+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fda7c085700 con 0x7fda840ce380 2024-05-08T22:42:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.530+0000 7fda837fe700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 0x7fda64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.534+0000 7fda837fe700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 0x7fda64068bf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.538+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda84154500 con 0x7fda840ce380 2024-05-08T22:42:22.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:42:22.684 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:42:22.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.706+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda84154500 con 0x7fda840ce380 2024-05-08T22:42:22.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.710+0000 7fda7bfff700 1 -- 172.21.15.179:0/3482597136 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16751 (secure 0 0 0) 0x7fda84154500 con 0x7fda840ce380 2024-05-08T22:42:22.713 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:22.713 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":100,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:41:40.642772+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6834/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:22.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 msgr2=0x7fda64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 0x7fda64068bf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c007f30 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 msgr2=0x7fda841547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fda8406afd0 tx=0x7fda7c004770 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 shutdown_connections 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fda64066730 0x7fda64068bf0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda840ce380 0x7fda841547e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fda840cc700 0x7fda84158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 --2- 172.21.15.179:0/3482597136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda840cb510 0x7fda841568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 >> 172.21.15.179:0/3482597136 conn(0x7fda840ce9d0 msgr2=0x7fda8406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 shutdown_connections 2024-05-08T22:42:22.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.714+0000 7fda88d73700 1 -- 172.21.15.179:0/3482597136 wait complete. 2024-05-08T22:42:22.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:39:59.457728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:40:00.136943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:22.729 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:22.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.846+0000 7f5cf5470700 1 Processor -- start 2024-05-08T22:42:22.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.846+0000 7f5cf5470700 1 -- start start 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf5470700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 0x7f5cf00cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf5470700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5cf00cb560 0x7f5cf00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf5470700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf00cd860 con 0x7f5cf00cd450 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf5470700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf00ca3a0 con 0x7f5cf00cb560 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf5470700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf00ca680 con 0x7f5cf00ae760 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf4c6f700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5cf00cd450 0x7f5cf00b4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55902/0 (socket says 172.21.15.179:55902) 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5cf00cb560 0x7f5cf00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cf4c6f700 1 -- 172.21.15.179:0/2958072149 learned_addr learned my addr 172.21.15.179:0/2958072149 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceffff700 1 --2- 172.21.15.179:0/2958072149 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 0x7f5cf00cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceeffd700 1 -- 172.21.15.179:0/2958072149 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267853548 0 0) 0x7f5cf00cd860 con 0x7f5cf00cd450 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 -- 172.21.15.179:0/2958072149 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 msgr2=0x7f5cf00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 --2- 172.21.15.179:0/2958072149 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 0x7f5cf00cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 -- 172.21.15.179:0/2958072149 >> v1:172.21.15.179:6789/0 conn(0x7f5cf00cd450 legacy=0x7f5cf00b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 -- 172.21.15.179:0/2958072149 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf00ca960 con 0x7f5cf00cb560 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceffff700 1 --2- 172.21.15.179:0/2958072149 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 0x7f5cf00cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5cef7fe700 1 --2- 172.21.15.179:0/2958072149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cb560 0x7f5cf00ccf10 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5cf006fea0 tx=0x7f5ce4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b23c24d274a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceeffd700 1 -- 172.21.15.179:0/2958072149 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011830 con 0x7f5cf00cb560 2024-05-08T22:42:22.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceeffd700 1 -- 172.21.15.179:0/2958072149 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4011990 con 0x7f5cf00cb560 2024-05-08T22:42:22.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.850+0000 7f5ceeffd700 1 -- 172.21.15.179:0/2958072149 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011d40 con 0x7f5cf00cb560 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- 172.21.15.179:0/2958072149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cb560 msgr2=0x7f5cf00ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 --2- 172.21.15.179:0/2958072149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cb560 0x7f5cf00ccf10 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5cf006fea0 tx=0x7f5ce4009b30 comp rx=0 tx=0).stop 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- 172.21.15.179:0/2958072149 shutdown_connections 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 --2- 172.21.15.179:0/2958072149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cb560 0x7f5cf00ccf10 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 --2- 172.21.15.179:0/2958072149 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00ae760 0x7f5cf00cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- 172.21.15.179:0/2958072149 >> 172.21.15.179:0/2958072149 conn(0x7f5cf0098a70 msgr2=0x7f5cf00b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:22.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- 172.21.15.179:0/2958072149 shutdown_connections 2024-05-08T22:42:22.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- 172.21.15.179:0/2958072149 wait complete. 2024-05-08T22:42:22.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 Processor -- start 2024-05-08T22:42:22.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.854+0000 7f5cf5470700 1 -- start start 2024-05-08T22:42:22.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 0x7f5cf0099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 0x7f5cf014f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf0158ea0 con 0x7f5cf00cd450 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf00c7830 con 0x7f5cf00cb560 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf00c7b10 con 0x7f5cf00ae760 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44166/0 (socket says 172.21.15.179:44166) 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 -- 172.21.15.179:0/1811040330 learned_addr learned my addr 172.21.15.179:0/1811040330 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5ceffff700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 0x7f5cf0099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cef7fe700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 0x7f5cf014f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 -- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 msgr2=0x7f5cf0099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 0x7f5cf0099300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 -- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 msgr2=0x7f5cf014f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 0x7f5cf014f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 -- 172.21.15.179:0/1811040330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40094b0 con 0x7f5cf00cd450 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cef7fe700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 0x7f5cf014f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5ceffff700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 0x7f5cf0099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf4c6f700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8002230 tx=0x7f5ce800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800ac20 con 0x7f5cf00cd450 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce800ada0 con 0x7f5cf00cd450 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf00c7df0 con 0x7f5cf00cd450 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8007310 con 0x7f5cf00cd450 2024-05-08T22:42:22.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.858+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf00c83b0 con 0x7f5cf00cd450 2024-05-08T22:42:22.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf00c85a0 con 0x7f5cf00cd450 2024-05-08T22:42:22.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5ce8007470 con 0x7f5cf00cd450 2024-05-08T22:42:22.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5cecff9700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 0x7f5cdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:22.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f5ce8084030 con 0x7f5cf00cd450 2024-05-08T22:42:22.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5ceffff700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 0x7f5cdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:22.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.862+0000 7f5ceffff700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 0x7f5cdc068bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c9a0 tx=0x7f5ce0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:22.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:22.866+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5ce80510c0 con 0x7f5cf00cd450 2024-05-08T22:42:23.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cf0099900 con 0x7f5cf00cd450 2024-05-08T22:42:23.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cecff9700 1 -- 172.21.15.179:0/1811040330 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ce80546e0 con 0x7f5cf00cd450 2024-05-08T22:42:23.039 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:23.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 msgr2=0x7f5cdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 0x7f5cdc068bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c9a0 tx=0x7f5ce0008040 comp rx=0 tx=0).stop 2024-05-08T22:42:23.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 msgr2=0x7f5cf009ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8002230 tx=0x7f5ce800a060 comp rx=0 tx=0).stop 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 shutdown_connections 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5cdc066730 0x7f5cdc068bf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf00cb560 0x7f5cf014f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cf00cd450 0x7f5cf009ad80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 --2- 172.21.15.179:0/1811040330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00ae760 0x7f5cf0099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 >> 172.21.15.179:0/1811040330 conn(0x7f5cf0098a70 msgr2=0x7f5cf006abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 shutdown_connections 2024-05-08T22:42:23.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.038+0000 7f5cf5470700 1 -- 172.21.15.179:0/1811040330 wait complete. 2024-05-08T22:42:23.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:23.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 Processor -- start 2024-05-08T22:42:23.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 -- start start 2024-05-08T22:42:23.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 0x7f28c40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 0x7f28c40cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c415d4a0 con 0x7f28c40ce0e0 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c40ce570 con 0x7f28c40d1d20 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.174+0000 7f28c8b44700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c406abb0 con 0x7f28c415d0c0 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 0x7f28c40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c2ffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 0x7f28c40cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c3fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f28c40ce0e0 0x7f28c4160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:41740/0 (socket says 172.21.15.179:41740) 2024-05-08T22:42:23.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c3fff700 1 -- 172.21.15.179:0/1980952379 learned_addr learned my addr 172.21.15.179:0/1980952379 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:23.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c27fc700 1 -- 172.21.15.179:0/1980952379 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641241318 0 0) 0x7f28c415d4a0 con 0x7f28c40ce0e0 2024-05-08T22:42:23.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c27fc700 1 -- 172.21.15.179:0/1980952379 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b00035d0 con 0x7f28c40ce0e0 2024-05-08T22:42:23.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 -- 172.21.15.179:0/1980952379 >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 msgr2=0x7f28c40cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 --2- 172.21.15.179:0/1980952379 >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 0x7f28c40cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 -- 172.21.15.179:0/1980952379 >> v1:172.21.15.195:6789/0 conn(0x7f28c40ce0e0 legacy=0x7f28c4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 -- 172.21.15.179:0/1980952379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40d2500 con 0x7f28c415d0c0 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c2ffd700 1 --2- 172.21.15.179:0/1980952379 >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 0x7f28c40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c37fe700 1 --2- 172.21.15.179:0/1980952379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 0x7f28c40b8a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009d90 tx=0x7f28b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8438a7eb36501e19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c27fc700 1 -- 172.21.15.179:0/1980952379 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400e070 con 0x7f28c415d0c0 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c27fc700 1 -- 172.21.15.179:0/1980952379 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400bb20 con 0x7f28c415d0c0 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c27fc700 1 -- 172.21.15.179:0/1980952379 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4012430 con 0x7f28c415d0c0 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 -- 172.21.15.179:0/1980952379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 msgr2=0x7f28c40b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 --2- 172.21.15.179:0/1980952379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 0x7f28c40b8a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009d90 tx=0x7f28b400b330 comp rx=0 tx=0).stop 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 -- 172.21.15.179:0/1980952379 shutdown_connections 2024-05-08T22:42:23.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 --2- 172.21.15.179:0/1980952379 >> v2:172.21.15.195:3300/0 conn(0x7f28c40d1d20 0x7f28c40cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 --2- 172.21.15.179:0/1980952379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c415d0c0 0x7f28c40b8a80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 -- 172.21.15.179:0/1980952379 >> 172.21.15.179:0/1980952379 conn(0x7f28c40d2ed0 msgr2=0x7f28c40d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 -- 172.21.15.179:0/1980952379 shutdown_connections 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.178+0000 7f28c8b44700 1 -- 172.21.15.179:0/1980952379 wait complete. 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 Processor -- start 2024-05-08T22:42:23.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- start start 2024-05-08T22:42:23.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c40d1d20 0x7f28c414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 0x7f28c414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414bbf0 con 0x7f28c40ce0e0 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414bec0 con 0x7f28c415d0c0 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414c1a0 con 0x7f28c40d1d20 2024-05-08T22:42:23.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44170/0 (socket says 172.21.15.179:44170) 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c37fe700 1 -- 172.21.15.179:0/591962534 learned_addr learned my addr 172.21.15.179:0/591962534 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 0x7f28c414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c2ffd700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c40d1d20 0x7f28c414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 -- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c40d1d20 msgr2=0x7f28c414b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c40d1d20 0x7f28c414b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 -- 172.21.15.179:0/591962534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 msgr2=0x7f28c41f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 -- 172.21.15.179:0/591962534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac0094b0 con 0x7f28c415d0c0 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c37fe700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c3fff700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 0x7f28c414fc50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c430 tx=0x7f28bc00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc005d90 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b4009a70 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc00ce40 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0072d0 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.182+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c414c760 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c404e700 con 0x7f28c415d0c0 2024-05-08T22:42:23.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28bc007430 con 0x7f28c415d0c0 2024-05-08T22:42:23.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28bbfff700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 0x7f28a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f28bc083290 con 0x7f28c415d0c0 2024-05-08T22:42:23.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28c37fe700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 0x7f28a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.186+0000 7f28c37fe700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 0x7f28a4068bf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000f80 tx=0x7f28b4011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.198+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f28bc0503d0 con 0x7f28c415d0c0 2024-05-08T22:42:23.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.370+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28c40676e0 con 0x7f28c415d0c0 2024-05-08T22:42:23.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28bbfff700 1 -- 172.21.15.179:0/591962534 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28bc0539f0 con 0x7f28c415d0c0 2024-05-08T22:42:23.374 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:23.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 msgr2=0x7f28a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 0x7f28a4068bf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000f80 tx=0x7f28b4011000 comp rx=0 tx=0).stop 2024-05-08T22:42:23.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 msgr2=0x7f28c414fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 0x7f28c414fc50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c430 tx=0x7f28bc00c740 comp rx=0 tx=0).stop 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 shutdown_connections 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f28a4066730 0x7f28a4068bf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28c415d0c0 0x7f28c414fc50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c40d1d20 0x7f28c414b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 --2- 172.21.15.179:0/591962534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28c40ce0e0 0x7f28c41f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 >> 172.21.15.179:0/591962534 conn(0x7f28c40d2ed0 msgr2=0x7f28c41531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 shutdown_connections 2024-05-08T22:42:23.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.374+0000 7f28c8b44700 1 -- 172.21.15.179:0/591962534 wait complete. 2024-05-08T22:42:23.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:23.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 Processor -- start 2024-05-08T22:42:23.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 -- start start 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0cb640 con 0x7ff3ec160c70 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0cb310 con 0x7ff3ec15d050 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec161b50 con 0x7ff3ec1610d0 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ea7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff3ec160c70 0x7ff3ec0cbad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55912/0 (socket says 172.21.15.179:55912) 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3ea7fc700 1 -- 172.21.15.179:0/505048275 learned_addr learned my addr 172.21.15.179:0/505048275 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:23.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.502+0000 7ff3eb7fe700 1 --2- 172.21.15.179:0/505048275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60521418 0 0) 0x7ff3ec0cb640 con 0x7ff3ec160c70 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d40035d0 con 0x7ff3ec160c70 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3eb7fe700 1 -- 172.21.15.179:0/505048275 >> v1:172.21.15.195:6789/0 conn(0x7ff3ec15d050 legacy=0x7ff3ec0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441084879 0 0) 0x7ff3ec0cb310 con 0x7ff3ec15d050 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3eb7fe700 1 -- 172.21.15.179:0/505048275 >> v1:172.21.15.179:6789/0 conn(0x7ff3ec160c70 legacy=0x7ff3ec0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3eb7fe700 1 -- 172.21.15.179:0/505048275 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0b65a0 con 0x7ff3ec1610d0 2024-05-08T22:42:23.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3eb7fe700 1 --2- 172.21.15.179:0/505048275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec0cc1e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009dc0 tx=0x7ff3e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c824ea6dbea529f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400e070 con 0x7ff3ec1610d0 2024-05-08T22:42:23.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e400bb20 con 0x7ff3ec1610d0 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3e9ffb700 1 -- 172.21.15.179:0/505048275 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400bed0 con 0x7ff3ec1610d0 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 -- 172.21.15.179:0/505048275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 msgr2=0x7ff3ec0cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/505048275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec0cc1e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009dc0 tx=0x7ff3e400b330 comp rx=0 tx=0).stop 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 -- 172.21.15.179:0/505048275 shutdown_connections 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/505048275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec0cc1e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 -- 172.21.15.179:0/505048275 >> 172.21.15.179:0/505048275 conn(0x7ff3ec0b2710 msgr2=0x7ff3ec0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 -- 172.21.15.179:0/505048275 shutdown_connections 2024-05-08T22:42:23.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.506+0000 7ff3ebfff700 1 -- 172.21.15.179:0/505048275 wait complete. 2024-05-08T22:42:23.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 Processor -- start 2024-05-08T22:42:23.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- start start 2024-05-08T22:42:23.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3ec160c70 0x7ff3ec1f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec1f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1f2d10 con 0x7ff3ec15d050 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1f66c0 con 0x7ff3ec160c70 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1f69a0 con 0x7ff3ec1610d0 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44178/0 (socket says 172.21.15.179:44178) 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 -- 172.21.15.179:0/94940478 learned_addr learned my addr 172.21.15.179:0/94940478 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eb7fe700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec1f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ea7fc700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3ec160c70 0x7ff3ec1f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 -- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 msgr2=0x7ff3ec1f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec1f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 -- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3ec160c70 msgr2=0x7ff3ec1f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3ec160c70 0x7ff3ec1f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 -- 172.21.15.179:0/94940478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4009a70 con 0x7ff3ec15d050 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eb7fe700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec1f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3eaffd700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00011e0 tx=0x7ff3e000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00054e0 con 0x7ff3ec15d050 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e00036a0 con 0x7ff3ec15d050 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0003a50 con 0x7ff3ec15d050 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec1f6c80 con 0x7ff3ec15d050 2024-05-08T22:42:23.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.510+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec1f38c0 con 0x7ff3ec15d050 2024-05-08T22:42:23.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec1f6e70 con 0x7ff3ec15d050 2024-05-08T22:42:23.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3e0003bb0 con 0x7ff3ec15d050 2024-05-08T22:42:23.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3d37fe700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7ff3e0083900 con 0x7ff3ec15d050 2024-05-08T22:42:23.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3ea7fc700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.514+0000 7ff3ea7fc700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0c6fe0 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.522+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff3e0050a10 con 0x7ff3ec15d050 2024-05-08T22:42:23.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3ec04e700 con 0x7ff3ec15d050 2024-05-08T22:42:23.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3d37fe700 1 -- 172.21.15.179:0/94940478 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3e0017070 con 0x7ff3ec15d050 2024-05-08T22:42:23.691 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 msgr2=0x7ff3d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0c6fe0 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 msgr2=0x7ff3ec06f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00011e0 tx=0x7ff3e000cae0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 shutdown_connections 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3d8066730 0x7ff3d8068bf0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3ec160c70 0x7ff3ec1f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3ec1610d0 0x7ff3ec1f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 --2- 172.21.15.179:0/94940478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3ec15d050 0x7ff3ec06f630 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 >> 172.21.15.179:0/94940478 conn(0x7ff3ec0b2710 msgr2=0x7ff3ec0b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 shutdown_connections 2024-05-08T22:42:23.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.690+0000 7ff3ebfff700 1 -- 172.21.15.179:0/94940478 wait complete. 2024-05-08T22:42:23.707 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:23.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 Processor -- start 2024-05-08T22:42:23.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 -- start start 2024-05-08T22:42:23.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f33200ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200caff0 con 0x7f33200d0180 2024-05-08T22:42:23.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200cf0e0 con 0x7f3320160cf0 2024-05-08T22:42:23.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f3324ebe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200cfd60 con 0x7f332015d140 2024-05-08T22:42:23.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331effd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f33200d0180 0x7f33200b6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55924/0 (socket says 172.21.15.179:55924) 2024-05-08T22:42:23.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331effd700 1 -- 172.21.15.179:0/1209027113 learned_addr learned my addr 172.21.15.179:0/1209027113 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331f7fe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f33200ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93204335 0 0) 0x7f33200caff0 con 0x7f33200d0180 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33000035d0 con 0x7f33200d0180 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314231059 0 0) 0x7f33200cf0e0 con 0x7f3320160cf0 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33200caff0 con 0x7f3320160cf0 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525133465 0 0) 0x7f33000035d0 con 0x7f33200d0180 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 msgr2=0x7f33200ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f33200ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 >> v1:172.21.15.195:6789/0 conn(0x7f3320160cf0 legacy=0x7f3320167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200cf3c0 con 0x7f33200d0180 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.814+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f3308001670 con 0x7f33200d0180 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 >> v1:172.21.15.179:6789/0 conn(0x7f33200d0180 legacy=0x7f33200b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3300003be0 0x7f3300003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300004870 0x7f3300004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f3300005330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200cfd60 con 0x7f3300004870 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330000bc30 con 0x7f3300003be0 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330000bdf0 con 0x7f332015d140 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331effd700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300004870 0x7f3300004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3300003be0 0x7f3300003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331ffff700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f3300005330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 -- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 msgr2=0x7f3300005330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332015d140 0x7f3300005330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 -- 172.21.15.179:0/1209027113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300004870 msgr2=0x7f3300004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300004870 0x7f3300004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 -- 172.21.15.179:0/1209027113 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000035d0 con 0x7f3300003be0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331f7fe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3300003be0 0x7f3300003fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3310009b70 tx=0x7f33000058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea28427049a646b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000e6c0 con 0x7f3300003be0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000e820 con 0x7f3300003be0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331e7fc700 1 -- 172.21.15.179:0/1209027113 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000eb90 con 0x7f3300003be0 2024-05-08T22:42:23.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 -- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3300003be0 msgr2=0x7f3300003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 --2- 172.21.15.179:0/1209027113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3300003be0 0x7f3300003fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3310009b70 tx=0x7f33000058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f331ffff700 1 -- 172.21.15.179:0/1209027113 reap_dead start 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 -- 172.21.15.179:0/1209027113 shutdown_connections 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 -- 172.21.15.179:0/1209027113 >> 172.21.15.179:0/1209027113 conn(0x7f33200d0e10 msgr2=0x7f33200d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 -- 172.21.15.179:0/1209027113 shutdown_connections 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.818+0000 7f3324ebe700 1 -- 172.21.15.179:0/1209027113 wait complete. 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.822+0000 7f3324ebe700 1 Processor -- start 2024-05-08T22:42:23.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.822+0000 7f3324ebe700 1 -- start start 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f332006c510 0x7f332006c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332006e5b0 0x7f332006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332006fc20 con 0x7f3300003be0 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332006ceb0 con 0x7f332006c510 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332006f380 con 0x7f332006e5b0 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44196/0 (socket says 172.21.15.179:44196) 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 -- 172.21.15.179:0/2038471596 learned_addr learned my addr 172.21.15.179:0/2038471596 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331effd700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f332006c510 0x7f332006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331ffff700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332006e5b0 0x7f332006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 -- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332006e5b0 msgr2=0x7f332006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332006e5b0 0x7f332006ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 -- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f332006c510 msgr2=0x7f332006c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f332006c510 0x7f332006c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 -- 172.21.15.179:0/2038471596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332006f660 con 0x7f3300003be0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f331f7fe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f330000bdc0 tx=0x7f33100047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000e6c0 con 0x7f3300003be0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331001c670 con 0x7f3300003be0 2024-05-08T22:42:23.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331001ca20 con 0x7f3300003be0 2024-05-08T22:42:23.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332006f940 con 0x7f3300003be0 2024-05-08T22:42:23.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3320150590 con 0x7f3300003be0 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33200b0660 con 0x7f3300003be0 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f331001cb80 con 0x7f3300003be0 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 0x7f3304068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.826+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f3310085e80 con 0x7f3300003be0 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.830+0000 7f331effd700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 0x7f3304068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:23.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.830+0000 7f331effd700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 0x7f3304068bf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f33200b05e0 tx=0x7f3308000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:23.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:23.834+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3310056bd0 con 0x7f3300003be0 2024-05-08T22:42:24.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.038+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332015de90 con 0x7f3300003be0 2024-05-08T22:42:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.038+0000 7f3317fff700 1 -- 172.21.15.179:0/2038471596 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f332015de90 con 0x7f3300003be0 2024-05-08T22:42:24.040 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:24.041 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 msgr2=0x7f3304068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 0x7f3304068bf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f33200b05e0 tx=0x7f3308000f80 comp rx=0 tx=0).stop 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 msgr2=0x7f3320150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f330000bdc0 tx=0x7f33100047e0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 shutdown_connections 2024-05-08T22:42:24.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3304066730 0x7f3304068bf0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f332006c510 0x7f332006c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f332006e5b0 0x7f332006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 --2- 172.21.15.179:0/2038471596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3300003be0 0x7f3320150050 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 >> 172.21.15.179:0/2038471596 conn(0x7f33200d0e10 msgr2=0x7f33200b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 shutdown_connections 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.042+0000 7f3324ebe700 1 -- 172.21.15.179:0/2038471596 wait complete. 2024-05-08T22:42:24.044 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:42:24.055 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:24.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 Processor -- start 2024-05-08T22:42:24.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 -- start start 2024-05-08T22:42:24.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f234406c550 0x7f234406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f234415a3e0 0x7f234415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2344157de0 con 0x7f23440cee90 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23440cad20 con 0x7f234406c550 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234406cf20 con 0x7f234415a3e0 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2342ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f23440cee90 0x7f23440cb140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55928/0 (socket says 172.21.15.179:55928) 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2342ffd700 1 -- 172.21.15.179:0/2346232776 learned_addr learned my addr 172.21.15.179:0/2346232776 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 --2- 172.21.15.179:0/2346232776 >> v2:172.21.15.179:3300/0 conn(0x7f234406c550 0x7f234406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23437fe700 1 --2- 172.21.15.179:0/2346232776 >> v2:172.21.15.195:3300/0 conn(0x7f234415a3e0 0x7f234415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2341ffb700 1 -- 172.21.15.179:0/2346232776 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342186783 0 0) 0x7f2344157de0 con 0x7f23440cee90 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2341ffb700 1 -- 172.21.15.179:0/2346232776 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23300035d0 con 0x7f23440cee90 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 -- 172.21.15.179:0/2346232776 >> v2:172.21.15.195:3300/0 conn(0x7f234415a3e0 msgr2=0x7f234415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 --2- 172.21.15.179:0/2346232776 >> v2:172.21.15.195:3300/0 conn(0x7f234415a3e0 0x7f234415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 -- 172.21.15.179:0/2346232776 >> v1:172.21.15.179:6789/0 conn(0x7f23440cee90 legacy=0x7f23440cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 -- 172.21.15.179:0/2346232776 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234415ad30 con 0x7f234406c550 2024-05-08T22:42:24.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f23427fc700 1 --2- 172.21.15.179:0/2346232776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234406c550 0x7f234406c9e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f23440cf970 tx=0x7f233c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e151f0100f9e064 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2341ffb700 1 -- 172.21.15.179:0/2346232776 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c00e070 con 0x7f234406c550 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2341ffb700 1 -- 172.21.15.179:0/2346232776 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c00bb20 con 0x7f234406c550 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f2341ffb700 1 -- 172.21.15.179:0/2346232776 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c012430 con 0x7f234406c550 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 -- 172.21.15.179:0/2346232776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234406c550 msgr2=0x7f234406c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.170+0000 7f234931c700 1 --2- 172.21.15.179:0/2346232776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234406c550 0x7f234406c9e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f23440cf970 tx=0x7f233c00b330 comp rx=0 tx=0).stop 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- 172.21.15.179:0/2346232776 shutdown_connections 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 --2- 172.21.15.179:0/2346232776 >> v2:172.21.15.195:3300/0 conn(0x7f234415a3e0 0x7f234415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 --2- 172.21.15.179:0/2346232776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234406c550 0x7f234406c9e0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- 172.21.15.179:0/2346232776 >> 172.21.15.179:0/2346232776 conn(0x7f23440cfd60 msgr2=0x7f23440d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- 172.21.15.179:0/2346232776 shutdown_connections 2024-05-08T22:42:24.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- 172.21.15.179:0/2346232776 wait complete. 2024-05-08T22:42:24.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 Processor -- start 2024-05-08T22:42:24.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- start start 2024-05-08T22:42:24.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234406c550 0x7f234414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 0x7f23440b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234414e940 con 0x7f234415a3e0 2024-05-08T22:42:24.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344156400 con 0x7f234406c550 2024-05-08T22:42:24.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f234931c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23441566e0 con 0x7f23440cee90 2024-05-08T22:42:24.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.174+0000 7f23437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44212/0 (socket says 172.21.15.179:44212) 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 -- 172.21.15.179:0/2834565447 learned_addr learned my addr 172.21.15.179:0/2834565447 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f2342ffd700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234406c550 0x7f234414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23427fc700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 0x7f23440b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 -- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 msgr2=0x7f23440b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 0x7f23440b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 -- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234406c550 msgr2=0x7f234414e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234406c550 0x7f234414e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 -- 172.21.15.179:0/2834565447 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c009a70 con 0x7f234415a3e0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23427fc700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 0x7f23440b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f23437fe700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f233400eb60 tx=0x7f233400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334009740 con 0x7f234415a3e0 2024-05-08T22:42:24.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23440b27c0 con 0x7f234415a3e0 2024-05-08T22:42:24.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400c850 con 0x7f234415a3e0 2024-05-08T22:42:24.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400cbe0 con 0x7f234415a3e0 2024-05-08T22:42:24.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23440b2d80 con 0x7f234415a3e0 2024-05-08T22:42:24.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2344067770 con 0x7f234415a3e0 2024-05-08T22:42:24.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.178+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f233400cd40 con 0x7f234415a3e0 2024-05-08T22:42:24.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.182+0000 7f233b7fe700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 0x7f2324068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.182+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f2334084460 con 0x7f234415a3e0 2024-05-08T22:42:24.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.182+0000 7f2342ffd700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 0x7f2324068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.182+0000 7f2342ffd700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 0x7f2324068bf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f232c007fd0 tx=0x7f232c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.190+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f23340514f0 con 0x7f234415a3e0 2024-05-08T22:42:24.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.358+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23440b2fa0 con 0x7f234415a3e0 2024-05-08T22:42:24.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.362+0000 7f233b7fe700 1 -- 172.21.15.179:0/2834565447 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16751 (secure 0 0 0) 0x7f2334054b10 con 0x7f234415a3e0 2024-05-08T22:42:24.365 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:24.365 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":100,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:41:40.642772+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6834/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 msgr2=0x7f2324068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 0x7f2324068bf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f232c007fd0 tx=0x7f232c007de0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 msgr2=0x7f23440b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f233400eb60 tx=0x7f233400ee70 comp rx=0 tx=0).stop 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 shutdown_connections 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2324066730 0x7f2324068bf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f234415a3e0 0x7f23440b4f50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23440cee90 0x7f23440b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 --2- 172.21.15.179:0/2834565447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234406c550 0x7f234414e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 >> 172.21.15.179:0/2834565447 conn(0x7f23440cfd60 msgr2=0x7f234415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 shutdown_connections 2024-05-08T22:42:24.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.366+0000 7f234931c700 1 -- 172.21.15.179:0/2834565447 wait complete. 2024-05-08T22:42:24.380 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:42:24.380 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:24.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 Processor -- start 2024-05-08T22:42:24.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- start start 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7e8c0ce060 0x7f7e8c06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e8c0add40 0x7f7e8c06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8c161ac0 con 0x7f7e8c0cc310 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8c0ae550 con 0x7f7e8c0add40 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8c06abb0 con 0x7f7e8c0ce060 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e8c0add40 0x7f7e8c06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7e8c0add40 0x7f7e8c06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44220/0 (socket says 172.21.15.179:44220) 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 -- 172.21.15.179:0/2856523211 learned_addr learned my addr 172.21.15.179:0/2856523211 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9215a700 1 --2- 172.21.15.179:0/2856523211 >> v2:172.21.15.195:3300/0 conn(0x7f7e8c0ce060 0x7f7e8c06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 -- 172.21.15.179:0/2856523211 >> v2:172.21.15.195:3300/0 conn(0x7f7e8c0ce060 msgr2=0x7f7e8c06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 --2- 172.21.15.179:0/2856523211 >> v2:172.21.15.195:3300/0 conn(0x7f7e8c0ce060 0x7f7e8c06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 -- 172.21.15.179:0/2856523211 >> v1:172.21.15.195:6789/0 conn(0x7f7e8c0cc310 legacy=0x7f7e8c0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 -- 172.21.15.179:0/2856523211 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0cdaa0 con 0x7f7e8c0add40 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e9295b700 1 --2- 172.21.15.179:0/2856523211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0add40 0x7f7e8c06b8a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c074410 tx=0x7f7e7c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67419f1ced65878b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e91158700 1 -- 172.21.15.179:0/2856523211 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011830 con 0x7f7e8c0add40 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e91158700 1 -- 172.21.15.179:0/2856523211 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c011990 con 0x7f7e8c0add40 2024-05-08T22:42:24.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e91158700 1 -- 172.21.15.179:0/2856523211 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011d20 con 0x7f7e8c0add40 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- 172.21.15.179:0/2856523211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0add40 msgr2=0x7f7e8c06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 --2- 172.21.15.179:0/2856523211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0add40 0x7f7e8c06b8a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c074410 tx=0x7f7e7c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- 172.21.15.179:0/2856523211 shutdown_connections 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 --2- 172.21.15.179:0/2856523211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0add40 0x7f7e8c06b8a0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 --2- 172.21.15.179:0/2856523211 >> v2:172.21.15.195:3300/0 conn(0x7f7e8c0ce060 0x7f7e8c06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- 172.21.15.179:0/2856523211 >> 172.21.15.179:0/2856523211 conn(0x7f7e8c0cb660 msgr2=0x7f7e8c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- 172.21.15.179:0/2856523211 shutdown_connections 2024-05-08T22:42:24.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.494+0000 7f7e943be700 1 -- 172.21.15.179:0/2856523211 wait complete. 2024-05-08T22:42:24.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 Processor -- start 2024-05-08T22:42:24.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 -- start start 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e8c0add40 0x7f7e8c160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e8c0cc310 0x7f7e8c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c0b2da0 con 0x7f7e8c0ce060 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c0b29e0 con 0x7f7e8c0add40 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e943be700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c0ace10 con 0x7f7e8c0cc310 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44226/0 (socket says 172.21.15.179:44226) 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 -- 172.21.15.179:0/2015262913 learned_addr learned my addr 172.21.15.179:0/2015262913 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9215a700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e8c0add40 0x7f7e8c160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e91959700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e8c0cc310 0x7f7e8c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 -- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e8c0cc310 msgr2=0x7f7e8c0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e8c0cc310 0x7f7e8c0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 -- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e8c0add40 msgr2=0x7f7e8c160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e8c0add40 0x7f7e8c160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.498+0000 7f7e9295b700 1 -- 172.21.15.179:0/2015262913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c009570 con 0x7f7e8c0ce060 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e9295b700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c014040 tx=0x7f7e7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c004450 con 0x7f7e8c0ce060 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c01e070 con 0x7f7e8c0ce060 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8c0ad0f0 con 0x7f7e8c0ce060 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c003900 con 0x7f7e8c0ce060 2024-05-08T22:42:24.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8c160850 con 0x7f7e8c0ce060 2024-05-08T22:42:24.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8c067770 con 0x7f7e8c0ce060 2024-05-08T22:42:24.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e7c003a60 con 0x7f7e8c0ce060 2024-05-08T22:42:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.502+0000 7f7e8affd700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.506+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f7e7c020070 con 0x7f7e8c0ce060 2024-05-08T22:42:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.506+0000 7f7e9215a700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.506+0000 7f7e9215a700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7e840113e0 tx=0x7f7e8400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.510+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e7c052840 con 0x7f7e8c0ce060 2024-05-08T22:42:24.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.718+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e8c0cd7a0 con 0x7f7e8c0ce060 2024-05-08T22:42:24.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e8affd700 1 -- 172.21.15.179:0/2015262913 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f7e8c0cd7a0 con 0x7f7e8c0ce060 2024-05-08T22:42:24.723 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:24.723 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 msgr2=0x7f7e78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7e840113e0 tx=0x7f7e8400b040 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 msgr2=0x7f7e8c0c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c014040 tx=0x7f7e7c004770 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 shutdown_connections 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e78066730 0x7f7e78068bf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e8c0ce060 0x7f7e8c0c8460 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e8c0cc310 0x7f7e8c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 --2- 172.21.15.179:0/2015262913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e8c0add40 0x7f7e8c160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 >> 172.21.15.179:0/2015262913 conn(0x7f7e8c0cb660 msgr2=0x7f7e8c071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 shutdown_connections 2024-05-08T22:42:24.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.722+0000 7f7e943be700 1 -- 172.21.15.179:0/2015262913 wait complete. 2024-05-08T22:42:24.727 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:42:24.738 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:24.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 Processor -- start 2024-05-08T22:42:24.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- start start 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f902415ef80 0x7f9024161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9024164db0 0x7f902406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9024165d50 con 0x7f902406dca0 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902406e130 con 0x7f902415ef80 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902406ad70 con 0x7f9024164db0 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f9029d0b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f902406dca0 0x7f90240b2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:55932/0 (socket says 172.21.15.179:55932) 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f9029d0b700 1 -- 172.21.15.179:0/3380282994 learned_addr learned my addr 172.21.15.179:0/3380282994 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 --2- 172.21.15.179:0/3380282994 >> v2:172.21.15.195:3300/0 conn(0x7f902415ef80 0x7f9024161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f9028d09700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9024164db0 0x7f902406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f901bfff700 1 -- 172.21.15.179:0/3380282994 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468655975 0 0) 0x7f9024165d50 con 0x7f902406dca0 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f901bfff700 1 -- 172.21.15.179:0/3380282994 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90100035d0 con 0x7f902406dca0 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 -- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9024164db0 msgr2=0x7f902406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9024164db0 0x7f902406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 -- 172.21.15.179:0/3380282994 >> v1:172.21.15.179:6789/0 conn(0x7f902406dca0 legacy=0x7f90240b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 -- 172.21.15.179:0/3380282994 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9024165230 con 0x7f902415ef80 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902950a700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902415ef80 0x7f9024161790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9020009850 tx=0x7f9020009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66541d849fa3e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f901bfff700 1 -- 172.21.15.179:0/3380282994 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020011830 con 0x7f902415ef80 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f901bfff700 1 -- 172.21.15.179:0/3380282994 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9020011990 con 0x7f902415ef80 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f901bfff700 1 -- 172.21.15.179:0/3380282994 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020011d00 con 0x7f902415ef80 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902415ef80 msgr2=0x7f9024161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902415ef80 0x7f9024161790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9020009850 tx=0x7f9020009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- 172.21.15.179:0/3380282994 shutdown_connections 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9024164db0 0x7f902406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 --2- 172.21.15.179:0/3380282994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902415ef80 0x7f9024161790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- 172.21.15.179:0/3380282994 >> 172.21.15.179:0/3380282994 conn(0x7f90240ce9d0 msgr2=0x7f90240cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- 172.21.15.179:0/3380282994 shutdown_connections 2024-05-08T22:42:24.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.854+0000 7f902a50c700 1 -- 172.21.15.179:0/3380282994 wait complete. 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 Processor -- start 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- start start 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f902406dca0 0x7f90241634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 0x7f90240cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024163a30 con 0x7f902415ef80 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024143500 con 0x7f9024164db0 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9024145cc0 con 0x7f902406dca0 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902950a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f902406dca0 0x7f90241634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9029d0b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 0x7f90240cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44238/0 (socket says 172.21.15.179:44238) 2024-05-08T22:42:24.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 -- 172.21.15.179:0/2701812835 learned_addr learned my addr 172.21.15.179:0/2701812835 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 -- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f902406dca0 msgr2=0x7f90241634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f902406dca0 0x7f90241634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 -- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 msgr2=0x7f90240cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 0x7f90240cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 -- 172.21.15.179:0/2701812835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200094b0 con 0x7f902415ef80 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9029d0b700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 0x7f90240cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9028d09700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f901400e840 tx=0x7f901400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140095a0 con 0x7f902415ef80 2024-05-08T22:42:24.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9024145fa0 con 0x7f902415ef80 2024-05-08T22:42:24.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9014013680 con 0x7f902415ef80 2024-05-08T22:42:24.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140139f0 con 0x7f902415ef80 2024-05-08T22:42:24.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.858+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9024146560 con 0x7f902415ef80 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9024067740 con 0x7f902415ef80 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9014013b50 con 0x7f902415ef80 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f9019ffb700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 0x7f9008068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f9014084230 con 0x7f902415ef80 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f902950a700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 0x7f9008068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:24.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.862+0000 7f902950a700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 0x7f9008068bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f90200090b0 tx=0x7f9020008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:24.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:24.870+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90140512c0 con 0x7f902415ef80 2024-05-08T22:42:25.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f902404e700 con 0x7f902415ef80 2024-05-08T22:42:25.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f9019ffb700 1 -- 172.21.15.179:0/2701812835 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f90140548e0 con 0x7f902415ef80 2024-05-08T22:42:25.091 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:25.092 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:40:01.140896+0000","modified":"2024-05-08T22:41:44.668492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.179:6835/249337448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":249337448},{"type":"v1","addr":"172.21.15.179:6835","nonce":249337448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 msgr2=0x7f9008068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 0x7f9008068bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f90200090b0 tx=0x7f9020008030 comp rx=0 tx=0).stop 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 msgr2=0x7f90241628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f901400e840 tx=0x7f901400eb50 comp rx=0 tx=0).stop 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 shutdown_connections 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9008066730 0x7f9008068bf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9024164db0 0x7f90240cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f902415ef80 0x7f90241628d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 --2- 172.21.15.179:0/2701812835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f902406dca0 0x7f90241634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 >> 172.21.15.179:0/2701812835 conn(0x7f90240ce9d0 msgr2=0x7f90240b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 shutdown_connections 2024-05-08T22:42:25.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.090+0000 7f902a50c700 1 -- 172.21.15.179:0/2701812835 wait complete. 2024-05-08T22:42:25.095 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 68 2024-05-08T22:42:25.110 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:25.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 Processor -- start 2024-05-08T22:42:25.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 -- start start 2024-05-08T22:42:25.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f71b80af2c0 0x7f71b80af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80af750 con 0x7f71b80ae740 2024-05-08T22:42:25.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80acbc0 con 0x7f71b80af2c0 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.222+0000 7f71bcd5c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80ac290 con 0x7f71b815b150 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44254/0 (socket says 172.21.15.179:44254) 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:41246/0 (socket says 172.21.15.179:41246) 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b6ffd700 1 -- 172.21.15.179:0/4127366030 learned_addr learned my addr 172.21.15.179:0/4127366030 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b7fff700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.195:3300/0 conn(0x7f71b80af2c0 0x7f71b80af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 -- 172.21.15.179:0/4127366030 >> v2:172.21.15.195:3300/0 conn(0x7f71b80af2c0 msgr2=0x7f71b80af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.195:3300/0 conn(0x7f71b80af2c0 0x7f71b80af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 -- 172.21.15.179:0/4127366030 >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 msgr2=0x7f71b80aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:25.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 -- 172.21.15.179:0/4127366030 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80ac570 con 0x7f71b815b150 2024-05-08T22:42:25.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b6ffd700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:25.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b77fe700 1 --2- 172.21.15.179:0/4127366030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71b8071260 tx=0x7f71a8009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182800f8740fcbc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b67fc700 1 -- 172.21.15.179:0/4127366030 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8011830 con 0x7f71b815b150 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b67fc700 1 -- 172.21.15.179:0/4127366030 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8011990 con 0x7f71b815b150 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71b67fc700 1 -- 172.21.15.179:0/4127366030 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8011d20 con 0x7f71b815b150 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 -- 172.21.15.179:0/4127366030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 msgr2=0x7f71b80b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/4127366030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71b8071260 tx=0x7f71a8009b30 comp rx=0 tx=0).stop 2024-05-08T22:42:25.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 -- 172.21.15.179:0/4127366030 shutdown_connections 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.195:3300/0 conn(0x7f71b80af2c0 0x7f71b80af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/4127366030 >> v2:172.21.15.179:3300/0 conn(0x7f71b80ae740 0x7f71b80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/4127366030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b80b4a30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 -- 172.21.15.179:0/4127366030 >> 172.21.15.179:0/4127366030 conn(0x7f71b8098a70 msgr2=0x7f71b80b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 -- 172.21.15.179:0/4127366030 shutdown_connections 2024-05-08T22:42:25.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.226+0000 7f71bcd5c700 1 -- 172.21.15.179:0/4127366030 wait complete. 2024-05-08T22:42:25.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 Processor -- start 2024-05-08T22:42:25.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- start start 2024-05-08T22:42:25.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 0x7f71b815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b8156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b815b660 con 0x7f71b80ae740 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b815da20 con 0x7f71b80af2c0 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8099920 con 0x7f71b815b150 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44258/0 (socket says 172.21.15.179:44258) 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 -- 172.21.15.179:0/1391932786 learned_addr learned my addr 172.21.15.179:0/1391932786 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b7fff700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b8156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b6ffd700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 0x7f71b815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 -- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 msgr2=0x7f71b8156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b8156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 -- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 msgr2=0x7f71b815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 0x7f71b815d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 -- 172.21.15.179:0/1391932786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac009570 con 0x7f71b80ae740 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b6ffd700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 0x7f71b815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71b77fe700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f71a80090b0 tx=0x7f71a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004300 con 0x7f71b80ae740 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a801f070 con 0x7f71b80ae740 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a80094b0 con 0x7f71b80ae740 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a80036c0 con 0x7f71b80ae740 2024-05-08T22:42:25.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.230+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b8099ee0 con 0x7f71b80ae740 2024-05-08T22:42:25.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.234+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b804e700 con 0x7f71b80ae740 2024-05-08T22:42:25.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.234+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71a8021030 con 0x7f71b80ae740 2024-05-08T22:42:25.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.234+0000 7f719ffff700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 0x7f71a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.234+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f71a8084950 con 0x7f71b80ae740 2024-05-08T22:42:25.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.234+0000 7f71b6ffd700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 0x7f71a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.238+0000 7f71b6ffd700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 0x7f71a4068bf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f71b815c8b0 tx=0x7f71ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.242+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f71a8051a60 con 0x7f71b80ae740 2024-05-08T22:42:25.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.414+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b80cfe20 con 0x7f71b80ae740 2024-05-08T22:42:25.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.414+0000 7f719ffff700 1 -- 172.21.15.179:0/1391932786 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16751 (secure 0 0 0) 0x7f71a8055080 con 0x7f71b80ae740 2024-05-08T22:42:25.418 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:25.418 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":100,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:41:40.642772+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:39:59.457728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T22:40:00.136943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6834/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:25.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 msgr2=0x7f71a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 0x7f71a4068bf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f71b815c8b0 tx=0x7f71ac008040 comp rx=0 tx=0).stop 2024-05-08T22:42:25.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 msgr2=0x7f71b8099300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f71a80090b0 tx=0x7f71a8004770 comp rx=0 tx=0).stop 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 shutdown_connections 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f71a4066730 0x7f71a4068bf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b80af2c0 0x7f71b815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b815b150 0x7f71b8156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 --2- 172.21.15.179:0/1391932786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71b80ae740 0x7f71b8099300 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.418+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 >> 172.21.15.179:0/1391932786 conn(0x7f71b8098a70 msgr2=0x7f71b8159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.422+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 shutdown_connections 2024-05-08T22:42:25.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.422+0000 7f71bcd5c700 1 -- 172.21.15.179:0/1391932786 wait complete. 2024-05-08T22:42:25.437 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:42:25.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 Processor -- start 2024-05-08T22:42:25.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 -- start start 2024-05-08T22:42:25.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6c406dda0 0x7fb6c40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c406b540 con 0x7fb6c406e180 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c406ac50 con 0x7fb6c40b25d0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c40b3110 con 0x7fb6c406dda0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c2ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6c406dda0 0x7fb6c40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6ba7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb6c406e180 0x7fb6c406e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52320/0 (socket says 172.21.15.179:52320) 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6ba7fc700 1 -- 172.21.15.179:0/3228656522 learned_addr learned my addr 172.21.15.179:0/3228656522 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c2ffd700 1 -- 172.21.15.179:0/3228656522 >> v1:172.21.15.195:6789/0 conn(0x7fb6c40b25d0 legacy=0x7fb6c40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c2ffd700 1 -- 172.21.15.179:0/3228656522 >> v1:172.21.15.179:6789/0 conn(0x7fb6c406e180 legacy=0x7fb6c406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c2ffd700 1 -- 172.21.15.179:0/3228656522 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c406aef0 con 0x7fb6c406dda0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c27fc700 1 -- 172.21.15.179:0/3228656522 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211040775 0 0) 0x7fb6c406b540 con 0x7fb6c406e180 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c2ffd700 1 --2- 172.21.15.179:0/3228656522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406dda0 0x7fb6c40cbc30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009850 tx=0x7fb6b4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e136d5fe309e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c27fc700 1 -- 172.21.15.179:0/3228656522 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4011830 con 0x7fb6c406dda0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c27fc700 1 -- 172.21.15.179:0/3228656522 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4011990 con 0x7fb6c406dda0 2024-05-08T22:42:25.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.550+0000 7fb6c27fc700 1 -- 172.21.15.179:0/3228656522 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4011d20 con 0x7fb6c406dda0 2024-05-08T22:42:25.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- 172.21.15.179:0/3228656522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406dda0 msgr2=0x7fb6c40cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/3228656522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406dda0 0x7fb6c40cbc30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009850 tx=0x7fb6b4009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:25.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- 172.21.15.179:0/3228656522 shutdown_connections 2024-05-08T22:42:25.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/3228656522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406dda0 0x7fb6c40cbc30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- 172.21.15.179:0/3228656522 >> 172.21.15.179:0/3228656522 conn(0x7fb6c40ce960 msgr2=0x7fb6c40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- 172.21.15.179:0/3228656522 shutdown_connections 2024-05-08T22:42:25.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- 172.21.15.179:0/3228656522 wait complete. 2024-05-08T22:42:25.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 Processor -- start 2024-05-08T22:42:25.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 -- start start 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.554+0000 7fb6c3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6c406dda0 0x7fb6c4150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6c40b25d0 0x7fb6c40cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c4163e20 con 0x7fb6c406e180 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c4164100 con 0x7fb6c406dda0 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41643e0 con 0x7fb6c40b25d0 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40292/0 (socket says 172.21.15.179:40292) 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 -- 172.21.15.179:0/2783330839 learned_addr learned my addr 172.21.15.179:0/2783330839 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c2ffd700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6c406dda0 0x7fb6c4150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c37fe700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6c40b25d0 0x7fb6c40cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 -- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6c40b25d0 msgr2=0x7fb6c40cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6c40b25d0 0x7fb6c40cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 -- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6c406dda0 msgr2=0x7fb6c4150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6c406dda0 0x7fb6c4150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 -- 172.21.15.179:0/2783330839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40094b0 con 0x7fb6c406e180 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6ba7fc700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00c550 tx=0x7fb6ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac004ca0 con 0x7fb6c406e180 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac004e00 con 0x7fb6c406e180 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c41646c0 con 0x7fb6c406e180 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41f7020 con 0x7fb6c406e180 2024-05-08T22:42:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0072d0 con 0x7fb6c406e180 2024-05-08T22:42:25.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a8005310 con 0x7fb6c406e180 2024-05-08T22:42:25.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.558+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6ac007430 con 0x7fb6c406e180 2024-05-08T22:42:25.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.562+0000 7fb6c0ff9700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 0x7fb6a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.562+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fb6ac084130 con 0x7fb6c406e180 2024-05-08T22:42:25.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.562+0000 7fb6c2ffd700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 0x7fb6a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.562+0000 7fb6c2ffd700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 0x7fb6a4068bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40090b0 tx=0x7fb6b4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.570+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6ac0511c0 con 0x7fb6c406e180 2024-05-08T22:42:25.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.786+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6a8005890 con 0x7fb6c406e180 2024-05-08T22:42:25.788 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:42:25.786+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:42:25.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.798+0000 7fb6c0ff9700 1 -- 172.21.15.179:0/2783330839 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6ac0547e0 con 0x7fb6c406e180 2024-05-08T22:42:25.804 INFO:tasks.ceph.mds.a.smithi179.stderr: -1744> 2024-05-08T22:41:48.699+0000 7fede1a52700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x563966cd8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T22:42:25.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 msgr2=0x7fb6a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 0x7fb6a4068bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40090b0 tx=0x7fb6b4008030 comp rx=0 tx=0).stop 2024-05-08T22:42:25.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 msgr2=0x7fb6c40cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00c550 tx=0x7fb6ac00c910 comp rx=0 tx=0).stop 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 shutdown_connections 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb6a4066730 0x7fb6a4068bf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6c40b25d0 0x7fb6c40cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c406e180 0x7fb6c40cd120 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 --2- 172.21.15.179:0/2783330839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6c406dda0 0x7fb6c4150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 >> 172.21.15.179:0/2783330839 conn(0x7fb6c40ce960 msgr2=0x7fb6c40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 shutdown_connections 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.806+0000 7fb6c3fff700 1 -- 172.21.15.179:0/2783330839 wait complete. 2024-05-08T22:42:25.808 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:42:25.821 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:42:25.833 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:42:25.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 Processor -- start 2024-05-08T22:42:25.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 -- start start 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffbcc159860 0x7ffbcc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffbcc0b1700 0x7ffbcc06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbcc0cc720 con 0x7ffbcc06d960 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbcc06c1c0 con 0x7ffbcc0b1700 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbd1da8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbcc0cc980 con 0x7ffbcc159860 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffbcc0b1700 0x7ffbcc06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffbcc0b1700 0x7ffbcc06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40300/0 (socket says 172.21.15.179:40300) 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 -- 172.21.15.179:0/376949455 learned_addr learned my addr 172.21.15.179:0/376949455 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcaffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffbcc06d960 0x7ffbcc06f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52334/0 (socket says 172.21.15.179:52334) 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcb7fe700 1 --2- 172.21.15.179:0/376949455 >> v2:172.21.15.195:3300/0 conn(0x7ffbcc159860 0x7ffbcc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 -- 172.21.15.179:0/376949455 >> v2:172.21.15.195:3300/0 conn(0x7ffbcc159860 msgr2=0x7ffbcc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:25.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 --2- 172.21.15.179:0/376949455 >> v2:172.21.15.195:3300/0 conn(0x7ffbcc159860 0x7ffbcc06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 -- 172.21.15.179:0/376949455 >> v1:172.21.15.179:6789/0 conn(0x7ffbcc06d960 legacy=0x7ffbcc06f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 -- 172.21.15.179:0/376949455 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc0ccc60 con 0x7ffbcc0b1700 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbcbfff700 1 --2- 172.21.15.179:0/376949455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc06b440 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc072960 tx=0x7ffbc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158ea726bac59e0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbca7fc700 1 -- 172.21.15.179:0/376949455 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400e070 con 0x7ffbcc0b1700 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbca7fc700 1 -- 172.21.15.179:0/376949455 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400b8d0 con 0x7ffbcc0b1700 2024-05-08T22:42:25.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.930+0000 7ffbca7fc700 1 -- 172.21.15.179:0/376949455 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400bc80 con 0x7ffbcc0b1700 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- 172.21.15.179:0/376949455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc0b1700 msgr2=0x7ffbcc06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/376949455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc06b440 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc072960 tx=0x7ffbc400b330 comp rx=0 tx=0).stop 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- 172.21.15.179:0/376949455 shutdown_connections 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/376949455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc06b440 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/376949455 >> v2:172.21.15.195:3300/0 conn(0x7ffbcc159860 0x7ffbcc06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- 172.21.15.179:0/376949455 >> 172.21.15.179:0/376949455 conn(0x7ffbcc098a70 msgr2=0x7ffbcc0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- 172.21.15.179:0/376949455 shutdown_connections 2024-05-08T22:42:25.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- 172.21.15.179:0/376949455 wait complete. 2024-05-08T22:42:25.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 Processor -- start 2024-05-08T22:42:25.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- start start 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbcc159860 0x7ffbcc0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc14e310 con 0x7ffbcc06d960 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc14eb20 con 0x7ffbcc0b1700 2024-05-08T22:42:25.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbd1da8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1525a0 con 0x7ffbcc159860 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbcb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.934+0000 7ffbcb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40316/0 (socket says 172.21.15.179:40316) 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 -- 172.21.15.179:0/2426014886 learned_addr learned my addr 172.21.15.179:0/2426014886 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcbfff700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbcc159860 0x7ffbcc0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcaffd700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 -- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbcc159860 msgr2=0x7ffbcc0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbcc159860 0x7ffbcc0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 -- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbcc0b1700 msgr2=0x7ffbcc14cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc14cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 -- 172.21.15.179:0/2426014886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4009b30 con 0x7ffbcc06d960 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbcb7fe700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00e870 tx=0x7ffbbc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0095a0 con 0x7ffbcc06d960 2024-05-08T22:42:25.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc152880 con 0x7ffbcc06d960 2024-05-08T22:42:25.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc00ee70 con 0x7ffbcc06d960 2024-05-08T22:42:25.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc152e40 con 0x7ffbcc06d960 2024-05-08T22:42:25.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00ca50 con 0x7ffbcc06d960 2024-05-08T22:42:25.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc073290 con 0x7ffbcc06d960 2024-05-08T22:42:25.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.938+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffbbc00cc30 con 0x7ffbcc06d960 2024-05-08T22:42:25.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.942+0000 7ffbc3fff700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 0x7ffbac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:25.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.942+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6403+0+0 (secure 0 0 0) 0x7ffbbc084760 con 0x7ffbcc06d960 2024-05-08T22:42:25.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.942+0000 7ffbcaffd700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 0x7ffbac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:25.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.942+0000 7ffbcaffd700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 0x7ffbac068bf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ac340 tx=0x7ffbb4007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:25.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:25.946+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffbbc051790 con 0x7ffbcc06d960 2024-05-08T22:42:26.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.154+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffbcc0b0d60 con 0x7ffbcc06d960 2024-05-08T22:42:26.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.802+0000 7ffbc3fff700 1 -- 172.21.15.179:0/2426014886 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffbbc054db0 con 0x7ffbcc06d960 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 msgr2=0x7ffbac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 0x7ffbac068bf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ac340 tx=0x7ffbb4007cf0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 msgr2=0x7ffbcc14a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00e870 tx=0x7ffbbc00ec30 comp rx=0 tx=0).stop 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 shutdown_connections 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbac066730 0x7ffbac068bf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbcc0b1700 0x7ffbcc14cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbcc159860 0x7ffbcc0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 --2- 172.21.15.179:0/2426014886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbcc06d960 0x7ffbcc14a040 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 >> 172.21.15.179:0/2426014886 conn(0x7ffbcc098a70 msgr2=0x7ffbcc0acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:26.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 shutdown_connections 2024-05-08T22:42:26.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.806+0000 7ffbd1da8700 1 -- 172.21.15.179:0/2426014886 wait complete. 2024-05-08T22:42:26.825 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:42:26.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 Processor -- start 2024-05-08T22:42:26.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 -- start start 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06e406c2e0 0x7f06e406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06e40ce5a0 0x7f06e40d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e415d870 con 0x7f06e415d490 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e40d2740 con 0x7f06e406c2e0 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eb307700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e40b7300 con 0x7f06e40ce5a0 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06ea305700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f06e415d490 0x7f06e40cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52340/0 (socket says 172.21.15.179:52340) 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06ea305700 1 -- 172.21.15.179:0/922383171 learned_addr learned my addr 172.21.15.179:0/922383171 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 --2- 172.21.15.179:0/922383171 >> v2:172.21.15.179:3300/0 conn(0x7f06e406c2e0 0x7f06e406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06eab06700 1 --2- 172.21.15.179:0/922383171 >> v2:172.21.15.195:3300/0 conn(0x7f06e40ce5a0 0x7f06e40d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 -- 172.21.15.179:0/922383171 >> v2:172.21.15.195:3300/0 conn(0x7f06e40ce5a0 msgr2=0x7f06e40d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 --2- 172.21.15.179:0/922383171 >> v2:172.21.15.195:3300/0 conn(0x7f06e40ce5a0 0x7f06e40d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 -- 172.21.15.179:0/922383171 >> v1:172.21.15.179:6789/0 conn(0x7f06e415d490 legacy=0x7f06e40cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9303700 1 -- 172.21.15.179:0/922383171 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587211112 0 0) 0x7f06e415d870 con 0x7f06e415d490 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 -- 172.21.15.179:0/922383171 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e406b240 con 0x7f06e406c2e0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9b04700 1 --2- 172.21.15.179:0/922383171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e406c2e0 0x7f06e406abb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f06e40d0960 tx=0x7f06d4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea678716bc23c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9303700 1 -- 172.21.15.179:0/922383171 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4011830 con 0x7f06e406c2e0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9303700 1 -- 172.21.15.179:0/922383171 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4011990 con 0x7f06e406c2e0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.990+0000 7f06e9303700 1 -- 172.21.15.179:0/922383171 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4011d20 con 0x7f06e406c2e0 2024-05-08T22:42:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- 172.21.15.179:0/922383171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e406c2e0 msgr2=0x7f06e406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- 172.21.15.179:0/922383171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e406c2e0 0x7f06e406abb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f06e40d0960 tx=0x7f06d4009b30 comp rx=0 tx=0).stop 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- 172.21.15.179:0/922383171 shutdown_connections 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- 172.21.15.179:0/922383171 >> v2:172.21.15.195:3300/0 conn(0x7f06e40ce5a0 0x7f06e40d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- 172.21.15.179:0/922383171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e406c2e0 0x7f06e406abb0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- 172.21.15.179:0/922383171 >> 172.21.15.179:0/922383171 conn(0x7f06e40b4940 msgr2=0x7f06e40b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- 172.21.15.179:0/922383171 shutdown_connections 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- 172.21.15.179:0/922383171 wait complete. 2024-05-08T22:42:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 Processor -- start 2024-05-08T22:42:26.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- start start 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 0x7f06e40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 0x7f06e40b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41e7630 con 0x7f06e40ce5a0 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41e7910 con 0x7f06e415d490 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06eb307700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41e7bf0 con 0x7f06e406c2e0 2024-05-08T22:42:26.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06e9b04700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06e9b04700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40322/0 (socket says 172.21.15.179:40322) 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.994+0000 7f06e9b04700 1 -- 172.21.15.179:0/2861961818 learned_addr learned my addr 172.21.15.179:0/2861961818 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06eab06700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 0x7f06e40b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06ea305700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 0x7f06e40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 -- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 msgr2=0x7f06e40cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 0x7f06e40cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 -- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 msgr2=0x7f06e40b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 0x7f06e40b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 -- 172.21.15.179:0/2861961818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40094b0 con 0x7f06e40ce5a0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06eab06700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 0x7f06e40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06ea305700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 0x7f06e40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06e9b04700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f06d40090b0 tx=0x7f06d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40036c0 con 0x7f06e40ce5a0 2024-05-08T22:42:27.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d401e070 con 0x7f06e40ce5a0 2024-05-08T22:42:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e41e7ed0 con 0x7f06e40ce5a0 2024-05-08T22:42:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4003a40 con 0x7f06e40ce5a0 2024-05-08T22:42:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e41f3270 con 0x7f06e40ce5a0 2024-05-08T22:42:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e41e7d30 con 0x7f06e40ce5a0 2024-05-08T22:42:27.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:26.998+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06d4003ba0 con 0x7f06e40ce5a0 2024-05-08T22:42:27.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.002+0000 7f06daffd700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.002+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6403+0+0 (secure 0 0 0) 0x7f06d40852e0 con 0x7f06e40ce5a0 2024-05-08T22:42:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.002+0000 7f06ea305700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.002+0000 7f06ea305700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 0x7f06c8068bf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f06e0000f60 tx=0x7f06e0007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:27.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.006+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f06e41e7d30 con 0x7f06e40ce5a0 2024-05-08T22:42:27.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.170+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06e41e7d30 con 0x7f06e40ce5a0 2024-05-08T22:42:27.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.818+0000 7f06daffd700 1 -- 172.21.15.179:0/2861961818 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f06e41e7d30 con 0x7f06e40ce5a0 2024-05-08T22:42:27.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 msgr2=0x7f06c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 0x7f06c8068bf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f06e0000f60 tx=0x7f06e0007cc0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 msgr2=0x7f06e40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f06d40090b0 tx=0x7f06d4004770 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 shutdown_connections 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f06c8066730 0x7f06c8068bf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06e415d490 0x7f06e40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e40ce5a0 0x7f06e40b3d40 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 --2- 172.21.15.179:0/2861961818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e406c2e0 0x7f06e40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 >> 172.21.15.179:0/2861961818 conn(0x7f06e40b4940 msgr2=0x7f06e40b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 shutdown_connections 2024-05-08T22:42:27.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.822+0000 7f06eb307700 1 -- 172.21.15.179:0/2861961818 wait complete. 2024-05-08T22:42:27.827 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:42:27.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:42:27.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 Processor -- start 2024-05-08T22:42:27.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 -- start start 2024-05-08T22:42:27.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f66040d2ab0 0x7f66040d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040d1a10 con 0x7f66040d00a0 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040d1cf0 con 0x7f660415d3c0 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.962+0000 7f660b41d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040d1fd0 con 0x7f66040d2ab0 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66089b8700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f66040d00a0 0x7f66040b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52344/0 (socket says 172.21.15.179:52344) 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66089b8700 1 -- 172.21.15.179:0/2099697671 learned_addr learned my addr 172.21.15.179:0/2099697671 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66099ba700 1 --2- 172.21.15.179:0/2099697671 >> v2:172.21.15.195:3300/0 conn(0x7f66040d2ab0 0x7f66040d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427693879 0 0) 0x7f66040d1a10 con 0x7f66040d00a0 2024-05-08T22:42:27.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f00035d0 con 0x7f66040d00a0 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949243401 0 0) 0x7f66040d1cf0 con 0x7f660415d3c0 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66040d1a10 con 0x7f660415d3c0 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452481779 0 0) 0x7f65f00035d0 con 0x7f66040d00a0 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 >> v2:172.21.15.195:3300/0 conn(0x7f66040d2ab0 msgr2=0x7f66040d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 --2- 172.21.15.179:0/2099697671 >> v2:172.21.15.195:3300/0 conn(0x7f66040d2ab0 0x7f66040d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 >> v1:172.21.15.195:6789/0 conn(0x7f660415d3c0 legacy=0x7f66040cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040d22b0 con 0x7f66040d00a0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f65f4001670 con 0x7f66040d00a0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 >> v1:172.21.15.179:6789/0 conn(0x7f66040d00a0 legacy=0x7f66040b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 0x7f65f0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0004870 0x7f65f0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f0005230 0x7f65f000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040d1fd0 con 0x7f65f0003bb0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f000c0b0 con 0x7f65f0005230 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f000c270 con 0x7f65f0004870 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 0x7f65f0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66099ba700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f0005230 0x7f65f000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66089b8700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0004870 0x7f65f0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 -- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0004870 msgr2=0x7f65f0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0004870 0x7f65f0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 -- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f0005230 msgr2=0x7f65f000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f0005230 0x7f65f000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 -- 172.21.15.179:0/2099697671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00035d0 con 0x7f65f0003bb0 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66099ba700 1 -- 172.21.15.179:0/2099697671 reap_dead start 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f66091b9700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 0x7f65f0003fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6600001960 tx=0x7f65f00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fe2b46219dab6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600001450 con 0x7f65f0003bb0 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6600006aa0 con 0x7f65f0003bb0 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f65fbfff700 1 -- 172.21.15.179:0/2099697671 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600006e50 con 0x7f65f0003bb0 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 -- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 msgr2=0x7f65f0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 0x7f65f0003fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6600001960 tx=0x7f65f00058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 -- 172.21.15.179:0/2099697671 shutdown_connections 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 --2- 172.21.15.179:0/2099697671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65f0003bb0 0x7f65f0003fb0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 -- 172.21.15.179:0/2099697671 >> 172.21.15.179:0/2099697671 conn(0x7f66040b4ca0 msgr2=0x7f66040b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:27.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 -- 172.21.15.179:0/2099697671 shutdown_connections 2024-05-08T22:42:27.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.966+0000 7f660b41d700 1 -- 172.21.15.179:0/2099697671 wait complete. 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 Processor -- start 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- start start 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0003bb0 0x7f6604160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66040cb380 0x7f66040c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660415d820 con 0x7f660414e390 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c8030 con 0x7f66040cb380 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c8260 con 0x7f65f0003bb0 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66091b9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0003bb0 0x7f6604160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40340/0 (socket says 172.21.15.179:40340) 2024-05-08T22:42:27.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 -- 172.21.15.179:0/4072611116 learned_addr learned my addr 172.21.15.179:0/4072611116 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66089b8700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66040cb380 0x7f66040c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 -- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0003bb0 msgr2=0x7f6604160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0003bb0 0x7f6604160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 -- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66040cb380 msgr2=0x7f66040c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66040cb380 0x7f66040c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 -- 172.21.15.179:0/4072611116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040c8540 con 0x7f660414e390 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f66099ba700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f660408c270 tx=0x7f65fc0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00e070 con 0x7f660414e390 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc003d10 con 0x7f660414e390 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040c8820 con 0x7f660414e390 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc011790 con 0x7f660414e390 2024-05-08T22:42:27.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.970+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660414e820 con 0x7f660414e390 2024-05-08T22:42:27.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65fc0118f0 con 0x7f660414e390 2024-05-08T22:42:27.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660415d9d0 con 0x7f660414e390 2024-05-08T22:42:27.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f65f9ffb700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 0x7f65e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:27.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5965+0+0 (secure 0 0 0) 0x7f65fc0846c0 con 0x7f660414e390 2024-05-08T22:42:27.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f66091b9700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 0x7f65e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:27.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.974+0000 7f66091b9700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 0x7f65e8068bf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6600000c00 tx=0x7f6600005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:27.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:27.982+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f660415d9d0 con 0x7f660414e390 2024-05-08T22:42:28.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.162+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f660415d9d0 con 0x7f660414e390 2024-05-08T22:42:28.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.822+0000 7f65f9ffb700 1 -- 172.21.15.179:0/4072611116 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f660415d9d0 con 0x7f660414e390 2024-05-08T22:42:28.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 msgr2=0x7f65e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 0x7f65e8068bf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6600000c00 tx=0x7f6600005040 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 msgr2=0x7f6604151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f660408c270 tx=0x7f65fc0111d0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 shutdown_connections 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65e8066730 0x7f65e8068bf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66040cb380 0x7f66040c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f660414e390 0x7f6604151630 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 --2- 172.21.15.179:0/4072611116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f0003bb0 0x7f6604160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 >> 172.21.15.179:0/4072611116 conn(0x7f66040b4ca0 msgr2=0x7f66040b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 shutdown_connections 2024-05-08T22:42:28.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.826+0000 7f660b41d700 1 -- 172.21.15.179:0/4072611116 wait complete. 2024-05-08T22:42:28.830 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:42:28.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:42:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.962+0000 7f820ebd1700 1 Processor -- start 2024-05-08T22:42:28.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.962+0000 7f820ebd1700 1 -- start start 2024-05-08T22:42:28.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f82080cd0f0 0x7f82080cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080b11f0 con 0x7f820815d3b0 2024-05-08T22:42:28.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080cfa20 con 0x7f82080cd0f0 2024-05-08T22:42:28.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080cd9b0 con 0x7f82080cf590 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f8207fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f82080cd0f0 0x7f82080cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:46564/0 (socket says 172.21.15.179:46564) 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 -- 172.21.15.179:0/288209764 learned_addr learned my addr 172.21.15.179:0/288209764 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 -- 172.21.15.179:0/288209764 >> v2:172.21.15.195:3300/0 conn(0x7f82080cd0f0 msgr2=0x7f82080cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 --2- 172.21.15.179:0/288209764 >> v2:172.21.15.195:3300/0 conn(0x7f82080cd0f0 0x7f82080cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 -- 172.21.15.179:0/288209764 >> v1:172.21.15.195:6789/0 conn(0x7f820815d3b0 legacy=0x7f82080cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 -- 172.21.15.179:0/288209764 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080cd6d0 con 0x7f82080cf590 2024-05-08T22:42:28.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820d16e700 1 --2- 172.21.15.179:0/288209764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8200009e50 tx=0x7f820000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c006d2c2ffd5c92e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:28.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f82077fe700 1 -- 172.21.15.179:0/288209764 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000e070 con 0x7f82080cf590 2024-05-08T22:42:28.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f82077fe700 1 -- 172.21.15.179:0/288209764 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820000b910 con 0x7f82080cf590 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f82077fe700 1 -- 172.21.15.179:0/288209764 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000bcc0 con 0x7f82080cf590 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- 172.21.15.179:0/288209764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 msgr2=0x7f82080b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 --2- 172.21.15.179:0/288209764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8200009e50 tx=0x7f820000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- 172.21.15.179:0/288209764 shutdown_connections 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 --2- 172.21.15.179:0/288209764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f82080b1680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 --2- 172.21.15.179:0/288209764 >> v2:172.21.15.195:3300/0 conn(0x7f82080cd0f0 0x7f82080cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- 172.21.15.179:0/288209764 >> 172.21.15.179:0/288209764 conn(0x7f82080d1180 msgr2=0x7f82080d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- 172.21.15.179:0/288209764 shutdown_connections 2024-05-08T22:42:28.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.966+0000 7f820ebd1700 1 -- 172.21.15.179:0/288209764 wait complete. 2024-05-08T22:42:28.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 Processor -- start 2024-05-08T22:42:28.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- start start 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82080cd0f0 0x7f82081f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f8208144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208157cf0 con 0x7f820815d3b0 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b4a70 con 0x7f82080cd0f0 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080c93d0 con 0x7f82080cf590 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40350/0 (socket says 172.21.15.179:40350) 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 -- 172.21.15.179:0/3932835348 learned_addr learned my addr 172.21.15.179:0/3932835348 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820c96d700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82080cd0f0 0x7f82081f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f8207fff700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f8208144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 -- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 msgr2=0x7f8208144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f8208144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 -- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82080cd0f0 msgr2=0x7f82081f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82080cd0f0 0x7f82081f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 -- 172.21.15.179:0/3932835348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009b30 con 0x7f820815d3b0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f8207fff700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f8208144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820d16e700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82080cdaf0 tx=0x7f8200004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000099b0 con 0x7f820815d3b0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200004970 con 0x7f820815d3b0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080c96b0 con 0x7f820815d3b0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820001b670 con 0x7f820815d3b0 2024-05-08T22:42:28.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.970+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82080c9c70 con 0x7f820815d3b0 2024-05-08T22:42:28.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f820001b7d0 con 0x7f820815d3b0 2024-05-08T22:42:28.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820806a9d0 con 0x7f820815d3b0 2024-05-08T22:42:28.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f82057fa700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 0x7f81e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:28.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8200085980 con 0x7f820815d3b0 2024-05-08T22:42:28.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f820c96d700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 0x7f81e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:28.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.974+0000 7f820c96d700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 0x7f81e8068bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81f00018a0 tx=0x7f81f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:28.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:28.986+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8200052df0 con 0x7f820815d3b0 2024-05-08T22:42:29.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.166+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8208073290 con 0x7f820815d3b0 2024-05-08T22:42:29.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f82057fa700 1 -- 172.21.15.179:0/3932835348 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8200056410 con 0x7f820815d3b0 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 msgr2=0x7f81e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 0x7f81e8068bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81f00018a0 tx=0x7f81f0009040 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 msgr2=0x7f82081468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82080cdaf0 tx=0x7f8200004af0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 shutdown_connections 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f81e8066730 0x7f81e8068bf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820815d3b0 0x7f82081468d0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82080cf590 0x7f8208144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 --2- 172.21.15.179:0/3932835348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82080cd0f0 0x7f82081f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 >> 172.21.15.179:0/3932835348 conn(0x7f82080d1180 msgr2=0x7f820806f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:29.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 shutdown_connections 2024-05-08T22:42:29.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.174+0000 7f820ebd1700 1 -- 172.21.15.179:0/3932835348 wait complete. 2024-05-08T22:42:29.190 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-08T22:42:29.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 Processor -- start 2024-05-08T22:42:29.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 -- start start 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f89b4159e20 0x7f89b40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b40cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40ca310 con 0x7f89b40b1620 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b415a200 con 0x7f89b4159e20 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b8e6f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40b4e20 con 0x7f89b40b50b0 2024-05-08T22:42:29.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f89b4159e20 0x7f89b40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f89b4159e20 0x7f89b40cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40362/0 (socket says 172.21.15.179:40362) 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 -- 172.21.15.179:0/3198067996 learned_addr learned my addr 172.21.15.179:0/3198067996 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89aaffd700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b40cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 -- 172.21.15.179:0/3198067996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 msgr2=0x7f89b40cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b40cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 -- 172.21.15.179:0/3198067996 >> v1:172.21.15.195:6789/0 conn(0x7f89b40b1620 legacy=0x7f89b415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 -- 172.21.15.179:0/3198067996 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40c99e0 con 0x7f89b4159e20 2024-05-08T22:42:29.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b37fe700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b4159e20 0x7f89b40cce30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bbb0 tx=0x7f89a400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5caf44d67b10e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b2ffd700 1 -- 172.21.15.179:0/3198067996 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a400dd90 con 0x7f89b4159e20 2024-05-08T22:42:29.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b2ffd700 1 -- 172.21.15.179:0/3198067996 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4010470 con 0x7f89b4159e20 2024-05-08T22:42:29.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.302+0000 7f89b2ffd700 1 -- 172.21.15.179:0/3198067996 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4010820 con 0x7f89b4159e20 2024-05-08T22:42:29.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3198067996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b4159e20 msgr2=0x7f89b40cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b4159e20 0x7f89b40cce30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bbb0 tx=0x7f89a400bec0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3198067996 shutdown_connections 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b40cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3198067996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b4159e20 0x7f89b40cce30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3198067996 >> 172.21.15.179:0/3198067996 conn(0x7f89b4098a70 msgr2=0x7f89b40b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3198067996 shutdown_connections 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3198067996 wait complete. 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 Processor -- start 2024-05-08T22:42:29.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- start start 2024-05-08T22:42:29.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89b4159e20 0x7f89b415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40ae5b0 con 0x7f89b40b1620 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40ae890 con 0x7f89b4159e20 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.306+0000 7f89b8e6f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40aeb70 con 0x7f89b40b50b0 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40372/0 (socket says 172.21.15.179:40372) 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 -- 172.21.15.179:0/3118018203 learned_addr learned my addr 172.21.15.179:0/3118018203 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89aaffd700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b3fff700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89b4159e20 0x7f89b415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 -- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 msgr2=0x7f89b415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b415a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 -- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89b4159e20 msgr2=0x7f89b415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89b4159e20 0x7f89b415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 -- 172.21.15.179:0/3118018203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a400b810 con 0x7f89b40b1620 2024-05-08T22:42:29.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b37fe700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bf00 tx=0x7f89a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a401f070 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a40038d0 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40aee50 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4003c60 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b40ae210 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b40af370 con 0x7f89b40b1620 2024-05-08T22:42:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.310+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89a4019400 con 0x7f89b40b1620 2024-05-08T22:42:29.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.314+0000 7f89b0ff9700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 0x7f8994068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.314+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f89a4087360 con 0x7f89b40b1620 2024-05-08T22:42:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.314+0000 7f89aaffd700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 0x7f8994068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.314+0000 7f89aaffd700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 0x7f8994068bf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89b40adcc0 tx=0x7f899c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.318+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f89a40547d0 con 0x7f89b40b1620 2024-05-08T22:42:29.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.550+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f89b404e700 con 0x7f89b40b1620 2024-05-08T22:42:29.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.838+0000 7f89b0ff9700 1 -- 172.21.15.179:0/3118018203 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v420) v1 ==== 167+0+0 (secure 0 0 0) 0x7f89a4057df0 con 0x7f89b40b1620 2024-05-08T22:42:29.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 msgr2=0x7f8994068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 0x7f8994068bf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89b40adcc0 tx=0x7f899c008040 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 msgr2=0x7f89b4152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bf00 tx=0x7f89a4004770 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 shutdown_connections 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8994066730 0x7f8994068bf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89b4159e20 0x7f89b415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89b40b50b0 0x7f89b415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 --2- 172.21.15.179:0/3118018203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b40b1620 0x7f89b4152110 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 >> 172.21.15.179:0/3118018203 conn(0x7f89b4098a70 msgr2=0x7f89b40cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 shutdown_connections 2024-05-08T22:42:29.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.842+0000 7f89b8e6f700 1 -- 172.21.15.179:0/3118018203 wait complete. 2024-05-08T22:42:29.860 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T22:42:29.860 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:42:29.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:29.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.970+0000 7f792759e700 1 Processor -- start 2024-05-08T22:42:29.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.970+0000 7f792759e700 1 -- start start 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f792806de70 0x7f7928161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7928164db0 0x7f792806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7928165d50 con 0x7f792815ef80 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792806ad70 con 0x7f7928164db0 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7928165220 con 0x7f792806de70 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7928164db0 0x7f792806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7928164db0 0x7f792806b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40380/0 (socket says 172.21.15.179:40380) 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 -- 172.21.15.179:0/578696256 learned_addr learned my addr 172.21.15.179:0/578696256 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7925d9b700 1 --2- 172.21.15.179:0/578696256 >> v2:172.21.15.195:3300/0 conn(0x7f792806de70 0x7f7928161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 -- 172.21.15.179:0/578696256 >> v2:172.21.15.195:3300/0 conn(0x7f792806de70 msgr2=0x7f7928161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 --2- 172.21.15.179:0/578696256 >> v2:172.21.15.195:3300/0 conn(0x7f792806de70 0x7f7928161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 -- 172.21.15.179:0/578696256 >> v1:172.21.15.195:6789/0 conn(0x7f792815ef80 legacy=0x7f792806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 -- 172.21.15.179:0/578696256 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792806e4b0 con 0x7f7928164db0 2024-05-08T22:42:29.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f7926d9d700 1 --2- 172.21.15.179:0/578696256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7928164db0 0x7f792806b200 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f79280b2020 tx=0x7f792000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa92807c461b53ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792559a700 1 -- 172.21.15.179:0/578696256 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792000e070 con 0x7f7928164db0 2024-05-08T22:42:29.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792559a700 1 -- 172.21.15.179:0/578696256 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792000b910 con 0x7f7928164db0 2024-05-08T22:42:29.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792559a700 1 -- 172.21.15.179:0/578696256 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792000bcc0 con 0x7f7928164db0 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- 172.21.15.179:0/578696256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7928164db0 msgr2=0x7f792806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 --2- 172.21.15.179:0/578696256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7928164db0 0x7f792806b200 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f79280b2020 tx=0x7f792000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- 172.21.15.179:0/578696256 shutdown_connections 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 --2- 172.21.15.179:0/578696256 >> v2:172.21.15.195:3300/0 conn(0x7f792806de70 0x7f7928161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 --2- 172.21.15.179:0/578696256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7928164db0 0x7f792806b200 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- 172.21.15.179:0/578696256 >> 172.21.15.179:0/578696256 conn(0x7f79280ce9d0 msgr2=0x7f79280cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- 172.21.15.179:0/578696256 shutdown_connections 2024-05-08T22:42:29.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.974+0000 7f792759e700 1 -- 172.21.15.179:0/578696256 wait complete. 2024-05-08T22:42:29.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 Processor -- start 2024-05-08T22:42:29.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- start start 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792815ef80 0x7f79280c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 0x7f792806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792814aa40 con 0x7f792806de70 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792815a270 con 0x7f792815ef80 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7928159660 con 0x7f7928164db0 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40386/0 (socket says 172.21.15.179:40386) 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 -- 172.21.15.179:0/980817609 learned_addr learned my addr 172.21.15.179:0/980817609 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7925d9b700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792815ef80 0x7f79280c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7926d9d700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 0x7f792806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 -- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 msgr2=0x7f792806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 0x7f792806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 -- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792815ef80 msgr2=0x7f79280c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792815ef80 0x7f79280c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 -- 172.21.15.179:0/980817609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920009b30 con 0x7f792806de70 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7926d9d700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 0x7f792806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792659c700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c500 tx=0x7f791c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0052d0 con 0x7f792806de70 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928159940 con 0x7f792806de70 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c005430 con 0x7f792806de70 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928159f00 con 0x7f792806de70 2024-05-08T22:42:29.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0055a0 con 0x7f792806de70 2024-05-08T22:42:29.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.978+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928067740 con 0x7f792806de70 2024-05-08T22:42:29.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.982+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f791c006a80 con 0x7f792806de70 2024-05-08T22:42:29.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.982+0000 7f7916ffd700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 0x7f7908068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:29.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.982+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f791c083440 con 0x7f792806de70 2024-05-08T22:42:29.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.982+0000 7f7925d9b700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 0x7f7908068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:29.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.982+0000 7f7925d9b700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 0x7f7908068bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79281584e0 tx=0x7f7918008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:29.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:29.990+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f791c0507a0 con 0x7f792806de70 2024-05-08T22:42:30.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.154+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f792804e700 con 0x7f792806de70 2024-05-08T22:42:30.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.154+0000 7f7916ffd700 1 -- 172.21.15.179:0/980817609 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13033 (secure 0 0 0) 0x7f791c053dc0 con 0x7f792806de70 2024-05-08T22:42:30.157 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:30.157 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:28.817518+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6835/249337448":"2024-05-09T22:42:25.789618+0000","172.21.15.179:6834/249337448":"2024-05-09T22:42:25.789618+0000","172.21.15.179:6835/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6834/55939533":"2024-05-09T22:41:40.642711+0000","172.21.15.179:6835/1693330449":"2024-05-09T22:40:19.823587+0000","172.21.15.179:6834/1693330449":"2024-05-09T22:40:19.823587+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 msgr2=0x7f7908068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 0x7f7908068bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79281584e0 tx=0x7f7918008040 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 msgr2=0x7f7928149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c500 tx=0x7f791c00c8c0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 shutdown_connections 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7908066730 0x7f7908068bf0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792815ef80 0x7f79280c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7928164db0 0x7f792806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 --2- 172.21.15.179:0/980817609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792806de70 0x7f7928149f30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 >> 172.21.15.179:0/980817609 conn(0x7f79280ce9d0 msgr2=0x7f792815fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 shutdown_connections 2024-05-08T22:42:30.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.158+0000 7f792759e700 1 -- 172.21.15.179:0/980817609 wait complete. 2024-05-08T22:42:30.178 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:30.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.290+0000 7f27babc4700 1 Processor -- start 2024-05-08T22:42:30.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.290+0000 7f27babc4700 1 -- start start 2024-05-08T22:42:30.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f27b40b1a50 0x7f27b40b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27b40b2c10 0x7f27b406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b406dd50 con 0x7f27b414ae30 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b406e9d0 con 0x7f27b40b2c10 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b40b3080 con 0x7f27b40b1a50 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27b40b2c10 0x7f27b406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27b40b2c10 0x7f27b406abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40392/0 (socket says 172.21.15.179:40392) 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 -- 172.21.15.179:0/410653237 learned_addr learned my addr 172.21.15.179:0/410653237 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27b93c1700 1 --2- 172.21.15.179:0/410653237 >> v2:172.21.15.195:3300/0 conn(0x7f27b40b1a50 0x7f27b40b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 -- 172.21.15.179:0/410653237 >> v2:172.21.15.195:3300/0 conn(0x7f27b40b1a50 msgr2=0x7f27b40b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 --2- 172.21.15.179:0/410653237 >> v2:172.21.15.195:3300/0 conn(0x7f27b40b1a50 0x7f27b40b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 -- 172.21.15.179:0/410653237 >> v1:172.21.15.179:6789/0 conn(0x7f27b414ae30 legacy=0x7f27b4165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 -- 172.21.15.179:0/410653237 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b406e030 con 0x7f27b40b2c10 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27ba3c3700 1 --2- 172.21.15.179:0/410653237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b2c10 0x7f27b406abb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f27b406d430 tx=0x7f27ac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc69299446d689d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27b8bc0700 1 -- 172.21.15.179:0/410653237 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00ba20 con 0x7f27b40b2c10 2024-05-08T22:42:30.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27b8bc0700 1 -- 172.21.15.179:0/410653237 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00bb80 con 0x7f27b40b2c10 2024-05-08T22:42:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27b8bc0700 1 -- 172.21.15.179:0/410653237 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac009e60 con 0x7f27b40b2c10 2024-05-08T22:42:30.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- 172.21.15.179:0/410653237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b2c10 msgr2=0x7f27b406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 --2- 172.21.15.179:0/410653237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b2c10 0x7f27b406abb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f27b406d430 tx=0x7f27ac009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- 172.21.15.179:0/410653237 shutdown_connections 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 --2- 172.21.15.179:0/410653237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b2c10 0x7f27b406abb0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 --2- 172.21.15.179:0/410653237 >> v2:172.21.15.195:3300/0 conn(0x7f27b40b1a50 0x7f27b40b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- 172.21.15.179:0/410653237 >> 172.21.15.179:0/410653237 conn(0x7f27b40cea30 msgr2=0x7f27b40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- 172.21.15.179:0/410653237 shutdown_connections 2024-05-08T22:42:30.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.294+0000 7f27babc4700 1 -- 172.21.15.179:0/410653237 wait complete. 2024-05-08T22:42:30.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.298+0000 7f27babc4700 1 Processor -- start 2024-05-08T22:42:30.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.298+0000 7f27babc4700 1 -- start start 2024-05-08T22:42:30.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 0x7f27b4140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27b414ae30 0x7f27b4141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b41fabf0 con 0x7f27b40b1a50 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b41faed0 con 0x7f27b40b2c10 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b41fb1b0 con 0x7f27b414ae30 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40402/0 (socket says 172.21.15.179:40402) 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 -- 172.21.15.179:0/1976085276 learned_addr learned my addr 172.21.15.179:0/1976085276 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b93c1700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 0x7f27b4140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27ba3c3700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27b414ae30 0x7f27b4141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 -- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27b414ae30 msgr2=0x7f27b4141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27b414ae30 0x7f27b4141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 -- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 msgr2=0x7f27b4140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 0x7f27b4140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 -- 172.21.15.179:0/1976085276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac009570 con 0x7f27b40b1a50 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b93c1700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 0x7f27b4140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27b9bc2700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f27a400c410 tx=0x7f27a400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a40052d0 con 0x7f27b40b1a50 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a4005430 con 0x7f27b40b1a50 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b41fb490 con 0x7f27b40b1a50 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a40055a0 con 0x7f27b40b1a50 2024-05-08T22:42:30.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.302+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b4163740 con 0x7f27b40b1a50 2024-05-08T22:42:30.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b41638d0 con 0x7f27b40b1a50 2024-05-08T22:42:30.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27a40036a0 con 0x7f27b40b1a50 2024-05-08T22:42:30.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27aa7fc700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 0x7f2798068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f27a4083280 con 0x7f27b40b1a50 2024-05-08T22:42:30.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27b93c1700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 0x7f2798068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.306+0000 7f27b93c1700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 0x7f2798068bf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f27b41f7590 tx=0x7f27b0003a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.314+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f27b41638d0 con 0x7f27b40b1a50 2024-05-08T22:42:30.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.478+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27b41638d0 con 0x7f27b40b1a50 2024-05-08T22:42:30.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.478+0000 7f27aa7fc700 1 -- 172.21.15.179:0/1976085276 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27b41638d0 con 0x7f27b40b1a50 2024-05-08T22:42:30.482 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 msgr2=0x7f2798068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 0x7f2798068bf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f27b41f7590 tx=0x7f27b0003a80 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 msgr2=0x7f27b41f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f27a400c410 tx=0x7f27a400c7d0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 shutdown_connections 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2798066730 0x7f2798068bf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27b40b2c10 0x7f27b4140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27b414ae30 0x7f27b4141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 --2- 172.21.15.179:0/1976085276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27b40b1a50 0x7f27b41f7050 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 >> 172.21.15.179:0/1976085276 conn(0x7f27b40cea30 msgr2=0x7f27b4148520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 shutdown_connections 2024-05-08T22:42:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.482+0000 7f27babc4700 1 -- 172.21.15.179:0/1976085276 wait complete. 2024-05-08T22:42:30.497 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-08T22:42:30.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 Processor -- start 2024-05-08T22:42:30.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 -- start start 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9fb80b4ad0 0x7f9fb806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb815d9c0 con 0x7f9fb815d590 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb806d370 con 0x7f9fb80d07d0 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.606+0000 7f9fb7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb806d650 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9fb80b4ad0 0x7f9fb806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9fb80b4ad0 0x7f9fb806cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40416/0 (socket says 172.21.15.179:40416) 2024-05-08T22:42:30.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 -- 172.21.15.179:0/3579912332 learned_addr learned my addr 172.21.15.179:0/3579912332 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:30.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 -- 172.21.15.179:0/3579912332 >> v1:172.21.15.195:6789/0 conn(0x7f9fb80d07d0 legacy=0x7f9fb80b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 -- 172.21.15.179:0/3579912332 >> v1:172.21.15.179:6789/0 conn(0x7f9fb815d590 legacy=0x7f9fb815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 -- 172.21.15.179:0/3579912332 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb806d930 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb67fc700 1 --2- 172.21.15.179:0/3579912332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb806cd00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009910 tx=0x7f9fa8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d3cf3be66e48b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb5ffb700 1 -- 172.21.15.179:0/3579912332 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800ba60 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb5ffb700 1 -- 172.21.15.179:0/3579912332 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa800bbc0 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb5ffb700 1 -- 172.21.15.179:0/3579912332 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8009e60 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 -- 172.21.15.179:0/3579912332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80b4ad0 msgr2=0x7f9fb806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/3579912332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb806cd00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009910 tx=0x7f9fa8009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 -- 172.21.15.179:0/3579912332 shutdown_connections 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/3579912332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb806cd00 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 -- 172.21.15.179:0/3579912332 >> 172.21.15.179:0/3579912332 conn(0x7f9fb80b2840 msgr2=0x7f9fb80b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 -- 172.21.15.179:0/3579912332 shutdown_connections 2024-05-08T22:42:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.610+0000 7f9fb7fff700 1 -- 172.21.15.179:0/3579912332 wait complete. 2024-05-08T22:42:30.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 Processor -- start 2024-05-08T22:42:30.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- start start 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fb815d590 0x7f9fb8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8154b60 con 0x7f9fb80d07d0 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb814bbb0 con 0x7f9fb80b4ad0 2024-05-08T22:42:30.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8155dc0 con 0x7f9fb815d590 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40426/0 (socket says 172.21.15.179:40426) 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 -- 172.21.15.179:0/1219825906 learned_addr learned my addr 172.21.15.179:0/1219825906 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb6ffd700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb77fe700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fb815d590 0x7f9fb8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 -- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fb815d590 msgr2=0x7f9fb8157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fb815d590 0x7f9fb8157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 -- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fb80b4ad0 msgr2=0x7f9fb814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 -- 172.21.15.179:0/1219825906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa8009570 con 0x7f9fb80d07d0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb67fc700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8000c00 tx=0x7f9fa8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80036c0 con 0x7f9fb80d07d0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa801f070 con 0x7f9fb80d07d0 2024-05-08T22:42:30.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb81560a0 con 0x7f9fb80d07d0 2024-05-08T22:42:30.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8003a80 con 0x7f9fb80d07d0 2024-05-08T22:42:30.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.614+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb8156660 con 0x7f9fb80d07d0 2024-05-08T22:42:30.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb804e700 con 0x7f9fb80d07d0 2024-05-08T22:42:30.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9fa8003be0 con 0x7f9fb80d07d0 2024-05-08T22:42:30.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9f9f7fe700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 0x7f9fa4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:30.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9fa8021070 con 0x7f9fb80d07d0 2024-05-08T22:42:30.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9fb6ffd700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 0x7f9fa4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:30.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.618+0000 7f9fb6ffd700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 0x7f9fa4068bf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000f60 tx=0x7f9fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:30.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.626+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9fa8052f80 con 0x7f9fb80d07d0 2024-05-08T22:42:30.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:30.858+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f9fb80676e0 con 0x7f9fb80d07d0 2024-05-08T22:42:31.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.846+0000 7f9f9f7fe700 1 -- 172.21.15.179:0/1219825906 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v422) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9fa8015020 con 0x7f9fb80d07d0 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 msgr2=0x7f9fa4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 0x7f9fa4068bf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000f60 tx=0x7f9fac009040 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 msgr2=0x7f9fb8156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8000c00 tx=0x7f9fa8004770 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 shutdown_connections 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9fa4066730 0x7f9fa4068bf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fb815d590 0x7f9fb8157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb80d07d0 0x7f9fb8156f30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 --2- 172.21.15.179:0/1219825906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fb80b4ad0 0x7f9fb814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 >> 172.21.15.179:0/1219825906 conn(0x7f9fb80b2840 msgr2=0x7f9fb80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 shutdown_connections 2024-05-08T22:42:31.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.850+0000 7f9fb7fff700 1 -- 172.21.15.179:0/1219825906 wait complete. 2024-05-08T22:42:31.869 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:42:31.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.990+0000 7f6daf55e700 1 Processor -- start 2024-05-08T22:42:31.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.990+0000 7f6daf55e700 1 -- start start 2024-05-08T22:42:31.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6da8142890 0x7f6da80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:31.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:31.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80b22d0 con 0x7f6da8142fd0 2024-05-08T22:42:31.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80b19e0 con 0x7f6da8142890 2024-05-08T22:42:31.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815e640 con 0x7f6da80b2ca0 2024-05-08T22:42:31.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6da8142890 0x7f6da80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:31.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6da8142890 0x7f6da80b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:51588/0 (socket says 172.21.15.179:51588) 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daed5d700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6da8142fd0 0x7f6da815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:44890/0 (socket says 172.21.15.179:44890) 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dadd5b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 -- 172.21.15.179:0/1947160924 learned_addr learned my addr 172.21.15.179:0/1947160924 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 -- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 msgr2=0x7f6da806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da806cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dad55a700 1 -- 172.21.15.179:0/1947160924 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267471053 0 0) 0x7f6da80b22d0 con 0x7f6da8142fd0 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 -- 172.21.15.179:0/1947160924 >> v1:172.21.15.195:6789/0 conn(0x7f6da8142fd0 legacy=0x7f6da815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 -- 172.21.15.179:0/1947160924 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80b1c80 con 0x7f6da8142890 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dadd5b700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da806cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:31.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dae55c700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da8142890 0x7f6da80b2760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009910 tx=0x7f6da4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690d89e84b7d4910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:31.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dad55a700 1 -- 172.21.15.179:0/1947160924 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4011830 con 0x7f6da8142890 2024-05-08T22:42:31.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dad55a700 1 -- 172.21.15.179:0/1947160924 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4011990 con 0x7f6da8142890 2024-05-08T22:42:31.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6dad55a700 1 -- 172.21.15.179:0/1947160924 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4011d40 con 0x7f6da8142890 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 -- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da8142890 msgr2=0x7f6da80b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.994+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da8142890 0x7f6da80b2760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009910 tx=0x7f6da4009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- 172.21.15.179:0/1947160924 shutdown_connections 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da806cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1947160924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da8142890 0x7f6da80b2760 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- 172.21.15.179:0/1947160924 >> 172.21.15.179:0/1947160924 conn(0x7f6da80cea30 msgr2=0x7f6da80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- 172.21.15.179:0/1947160924 shutdown_connections 2024-05-08T22:42:31.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- 172.21.15.179:0/1947160924 wait complete. 2024-05-08T22:42:32.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 Processor -- start 2024-05-08T22:42:32.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- start start 2024-05-08T22:42:32.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da8139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da813b400 0x7f6da81f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81597a0 con 0x7f6da8142fd0 2024-05-08T22:42:32.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8139be0 con 0x7f6da813b400 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daf55e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da813af70 con 0x7f6da80b2ca0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6dadd5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6dadd5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40438/0 (socket says 172.21.15.179:40438) 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6dadd5b700 1 -- 172.21.15.179:0/1645238982 learned_addr learned my addr 172.21.15.179:0/1645238982 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6daed5d700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da813b400 0x7f6da81f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:31.998+0000 7f6dae55c700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da8139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 -- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 msgr2=0x7f6da8139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da8139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 -- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da813b400 msgr2=0x7f6da81f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da813b400 0x7f6da81f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 -- 172.21.15.179:0/1645238982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0094b0 con 0x7f6da8142fd0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dadd5b700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00ebe0 tx=0x7f6d9c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00c850 con 0x7f6da8142fd0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c00c9b0 con 0x7f6da8142fd0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da4009570 con 0x7f6da8142fd0 2024-05-08T22:42:32.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00cd40 con 0x7f6da8142fd0 2024-05-08T22:42:32.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da813a640 con 0x7f6da8142fd0 2024-05-08T22:42:32.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da813b0e0 con 0x7f6da8142fd0 2024-05-08T22:42:32.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d9c013470 con 0x7f6da8142fd0 2024-05-08T22:42:32.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 0x7f6d8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6d9c084100 con 0x7f6da8142fd0 2024-05-08T22:42:32.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.002+0000 7f6dae55c700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 0x7f6d8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.006+0000 7f6dae55c700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 0x7f6d8c068bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6da806c610 tx=0x7f6da4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.014+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6d9c051580 con 0x7f6da8142fd0 2024-05-08T22:42:32.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.186+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6da804e700 con 0x7f6da8142fd0 2024-05-08T22:42:32.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.194+0000 7f6d9affd700 1 -- 172.21.15.179:0/1645238982 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d9c00cea0 con 0x7f6da8142fd0 2024-05-08T22:42:32.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 msgr2=0x7f6d8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 0x7f6d8c068bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6da806c610 tx=0x7f6da4008030 comp rx=0 tx=0).stop 2024-05-08T22:42:32.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 msgr2=0x7f6da81396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00ebe0 tx=0x7f6d9c00eef0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 shutdown_connections 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d8c066730 0x7f6d8c068bf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6da813b400 0x7f6da81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8142fd0 0x7f6da81396a0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 --2- 172.21.15.179:0/1645238982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6da80b2ca0 0x7f6da8139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 >> 172.21.15.179:0/1645238982 conn(0x7f6da80cea30 msgr2=0x7f6da80cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 shutdown_connections 2024-05-08T22:42:32.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.198+0000 7f6daf55e700 1 -- 172.21.15.179:0/1645238982 wait complete. 2024-05-08T22:42:32.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:42:32.214 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:42:32.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:42:32.254 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:42:32.254 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:32.254 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:32.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:42:32.261 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:42:32.262 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:32.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:42:32.304 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:32.304 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:32.311 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:42:32.311 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:42:32.311 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:32.317 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:42:32.317 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:42:32.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 Processor -- start 2024-05-08T22:42:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- start start 2024-05-08T22:42:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6c040cc310 0x7f6c0406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c04161ac0 con 0x7f6c040add40 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040cdaa0 con 0x7f6c040ce060 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c040ae550 con 0x7f6c040cc310 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0359e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6c040add40 0x7f6c0406b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52364/0 (socket says 172.21.15.179:52364) 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6bfa59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6c040cc310 0x7f6c0406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0359e700 1 -- 172.21.15.179:0/3435897932 learned_addr learned my addr 172.21.15.179:0/3435897932 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0259c700 1 -- 172.21.15.179:0/3435897932 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147887980 0 0) 0x7f6c04161ac0 con 0x7f6c040add40 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0259c700 1 -- 172.21.15.179:0/3435897932 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf00035d0 con 0x7f6c040add40 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6bfa59c700 1 -- 172.21.15.179:0/3435897932 >> v1:172.21.15.195:6789/0 conn(0x7f6c040ce060 legacy=0x7f6c040ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6bfa59c700 1 -- 172.21.15.179:0/3435897932 >> v1:172.21.15.179:6789/0 conn(0x7f6c040add40 legacy=0x7f6c0406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6bfa59c700 1 -- 172.21.15.179:0/3435897932 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0406adc0 con 0x7f6c040cc310 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6bfa59c700 1 --2- 172.21.15.179:0/3435897932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040cc310 0x7f6c0406ce50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009850 tx=0x7f6bec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f1df431b2c8cda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0259c700 1 -- 172.21.15.179:0/3435897932 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec011830 con 0x7f6c040cc310 2024-05-08T22:42:32.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0259c700 1 -- 172.21.15.179:0/3435897932 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec011990 con 0x7f6c040cc310 2024-05-08T22:42:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c0259c700 1 -- 172.21.15.179:0/3435897932 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec011d40 con 0x7f6c040cc310 2024-05-08T22:42:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- 172.21.15.179:0/3435897932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040cc310 msgr2=0x7f6c0406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 --2- 172.21.15.179:0/3435897932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040cc310 0x7f6c0406ce50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009850 tx=0x7f6bec009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- 172.21.15.179:0/3435897932 shutdown_connections 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 --2- 172.21.15.179:0/3435897932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040cc310 0x7f6c0406ce50 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- 172.21.15.179:0/3435897932 >> 172.21.15.179:0/3435897932 conn(0x7f6c040cb660 msgr2=0x7f6c040cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- 172.21.15.179:0/3435897932 shutdown_connections 2024-05-08T22:42:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.438+0000 7f6c09112700 1 -- 172.21.15.179:0/3435897932 wait complete. 2024-05-08T22:42:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 Processor -- start 2024-05-08T22:42:32.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 -- start start 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 0x7f6c041f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c040ce060 0x7f6c041f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c041f6ef0 con 0x7f6c040add40 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040acd00 con 0x7f6c040cc310 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c09112700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040acfe0 con 0x7f6c040ce060 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40448/0 (socket says 172.21.15.179:40448) 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6bfa59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 0x7f6c041f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c02d9d700 1 -- 172.21.15.179:0/3009452770 learned_addr learned my addr 172.21.15.179:0/3009452770 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:32.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.442+0000 7f6c0359e700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c040ce060 0x7f6c041f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 -- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c040ce060 msgr2=0x7f6c041f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c040ce060 0x7f6c041f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 -- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 msgr2=0x7f6c041f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 0x7f6c041f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 -- 172.21.15.179:0/3009452770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0094b0 con 0x7f6c040add40 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6bfa59c700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 0x7f6c041f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c02d9d700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400c500 tx=0x7f6bf400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40052d0 con 0x7f6c040add40 2024-05-08T22:42:32.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4005430 con 0x7f6c040add40 2024-05-08T22:42:32.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c040ad2c0 con 0x7f6c040add40 2024-05-08T22:42:32.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40055a0 con 0x7f6c040add40 2024-05-08T22:42:32.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c040b0090 con 0x7f6c040add40 2024-05-08T22:42:32.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c04067770 con 0x7f6c040add40 2024-05-08T22:42:32.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.446+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6bf40038e0 con 0x7f6c040add40 2024-05-08T22:42:32.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.450+0000 7f6bfbfff700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 0x7f6be4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.450+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6bf4083350 con 0x7f6c040add40 2024-05-08T22:42:32.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.450+0000 7f6bfa59c700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 0x7f6be4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.450+0000 7f6bfa59c700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 0x7f6be4068c20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0090b0 tx=0x7f6bec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.458+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6bf40507d0 con 0x7f6c040add40 2024-05-08T22:42:32.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.666+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c0406c370 con 0x7f6c040add40 2024-05-08T22:42:32.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6bfbfff700 1 -- 172.21.15.179:0/3009452770 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f6bf4017df0 con 0x7f6c040add40 2024-05-08T22:42:32.671 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:32.671 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1183792994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1183792994},{"type":"v1","addr":"172.21.15.195:6835","nonce":1183792994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7705,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2579091797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2579091797},{"type":"v1","addr":"172.21.15.195:6837","nonce":2579091797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1978021264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1978021264},{"type":"v1","addr":"172.21.15.179:6833","nonce":1978021264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":8793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2522536463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2522536463},{"type":"v1","addr":"172.21.15.179:6835","nonce":2522536463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[]} 2024-05-08T22:42:32.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 msgr2=0x7f6be4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 0x7f6be4068c20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0090b0 tx=0x7f6bec008030 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 msgr2=0x7f6c040afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400c500 tx=0x7f6bf400c810 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 shutdown_connections 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6be4066760 0x7f6be4068c20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c040cc310 0x7f6c041f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c040ce060 0x7f6c041f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 --2- 172.21.15.179:0/3009452770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c040add40 0x7f6c040afb50 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 >> 172.21.15.179:0/3009452770 conn(0x7f6c040cb660 msgr2=0x7f6c040ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 shutdown_connections 2024-05-08T22:42:32.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.670+0000 7f6c09112700 1 -- 172.21.15.179:0/3009452770 wait complete. 2024-05-08T22:42:32.675 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 70 2024-05-08T22:42:32.687 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:42:32.687 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:42:32.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:42:32.687 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:42:32.687 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:42:32.687 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:42:32.688 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:42:32.688 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:42:32.688 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:42:32.688 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:42:32.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:42:32.688 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:42:32.688 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:42:32.688 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:42:32.689 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:42:32.689 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:42:32.701 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:42:32.702 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:42:32.703 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:42:32.703 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:42:32.705 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:42:32.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:42:32.706 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:42:32.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:42:32.747 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:42:32.748 DEBUG:teuthology.parallel:result is None 2024-05-08T22:42:32.749 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:42:32.749 DEBUG:teuthology.parallel:result is None 2024-05-08T22:42:32.752 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:42:32.752 DEBUG:teuthology.parallel:result is None 2024-05-08T22:42:32.753 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:42:32.753 DEBUG:teuthology.parallel:result is None 2024-05-08T22:42:32.753 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:42:32.821 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:42:32.830 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:42:32.836 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:42:32.836 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:42:32.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 Processor -- start 2024-05-08T22:42:32.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 -- start start 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9c4815d400 0x7f9c4806adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c480ce2f0 con 0x7f9c480ccd90 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4806e8d0 con 0x7f9c4815d400 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.858+0000 7f9c4f6fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480ceb00 con 0x7f9c480c92d0 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4d49a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9c4815d400 0x7f9c4806adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:46718/0 (socket says 172.21.15.179:46718) 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 -- 172.21.15.179:0/3595252494 learned_addr learned my addr 172.21.15.179:0/3595252494 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4dc9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9c480ccd90 0x7f9c4815dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52390/0 (socket says 172.21.15.179:52390) 2024-05-08T22:42:32.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 -- 172.21.15.179:0/3595252494 >> v2:172.21.15.195:3300/0 conn(0x7f9c4815d400 msgr2=0x7f9c4806adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:32.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/3595252494 >> v2:172.21.15.195:3300/0 conn(0x7f9c4815d400 0x7f9c4806adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 -- 172.21.15.179:0/3595252494 >> v1:172.21.15.179:6789/0 conn(0x7f9c480ccd90 legacy=0x7f9c4815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 -- 172.21.15.179:0/3595252494 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480cac10 con 0x7f9c480c92d0 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/3595252494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800bbb0 tx=0x7f9c3800bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9636ab8965ca1e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c3ffff700 1 -- 172.21.15.179:0/3595252494 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800dd30 con 0x7f9c480c92d0 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c3ffff700 1 -- 172.21.15.179:0/3595252494 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800de90 con 0x7f9c480c92d0 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c3ffff700 1 -- 172.21.15.179:0/3595252494 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38010690 con 0x7f9c480c92d0 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/3595252494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 msgr2=0x7f9c480cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/3595252494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800bbb0 tx=0x7f9c3800bec0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/3595252494 shutdown_connections 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/3595252494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480cc850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/3595252494 >> v2:172.21.15.195:3300/0 conn(0x7f9c4815d400 0x7f9c4806adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/3595252494 >> 172.21.15.179:0/3595252494 conn(0x7f9c480cb700 msgr2=0x7f9c480cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/3595252494 shutdown_connections 2024-05-08T22:42:32.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.862+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/3595252494 wait complete. 2024-05-08T22:42:32.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 Processor -- start 2024-05-08T22:42:32.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 -- start start 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 0x7f9c4814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 0x7f9c4814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480ac4e0 con 0x7f9c480ccd90 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480ab8f0 con 0x7f9c4815d400 2024-05-08T22:42:32.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4f6fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480abb40 con 0x7f9c480c92d0 2024-05-08T22:42:32.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:46724/0 (socket says 172.21.15.179:46724) 2024-05-08T22:42:32.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 -- 172.21.15.179:0/1171051928 learned_addr learned my addr 172.21.15.179:0/1171051928 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:32.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4dc9b700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 0x7f9c4814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 0x7f9c4814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 -- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 msgr2=0x7f9c4814f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 0x7f9c4814f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 -- 172.21.15.179:0/1171051928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 msgr2=0x7f9c4814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 0x7f9c4814eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 -- 172.21.15.179:0/1171051928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3800b810 con 0x7f9c480c92d0 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4dc9b700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 0x7f9c4814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 0x7f9c4814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:32.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.866+0000 7f9c4d49a700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e840 tx=0x7f9c4400eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c440095a0 con 0x7f9c480c92d0 2024-05-08T22:42:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c44013680 con 0x7f9c480c92d0 2024-05-08T22:42:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c480abe20 con 0x7f9c480c92d0 2024-05-08T22:42:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c440139f0 con 0x7f9c480c92d0 2024-05-08T22:42:32.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c480ae400 con 0x7f9c480c92d0 2024-05-08T22:42:32.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c48067770 con 0x7f9c480c92d0 2024-05-08T22:42:32.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.870+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c44013b50 con 0x7f9c480c92d0 2024-05-08T22:42:32.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.874+0000 7f9c3dffb700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 0x7f9c2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:32.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.874+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9c44083e50 con 0x7f9c480c92d0 2024-05-08T22:42:32.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.874+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 0x7f9c2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:32.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.874+0000 7f9c4cc99700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 0x7f9c2c068bf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c38000c00 tx=0x7f9c3801b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:32.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:32.882+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9c44051360 con 0x7f9c480c92d0 2024-05-08T22:42:33.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.058+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9c480ac620 con 0x7f9c480c92d0 2024-05-08T22:42:33.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.874+0000 7f9c3dffb700 1 -- 172.21.15.179:0/1171051928 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9c480ac620 con 0x7f9c480c92d0 2024-05-08T22:42:33.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.878+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 msgr2=0x7f9c2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.878+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 0x7f9c2c068bf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c38000c00 tx=0x7f9c3801b040 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.878+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 msgr2=0x7f9c480adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.878+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e840 tx=0x7f9c4400eb50 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 shutdown_connections 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c2c066730 0x7f9c2c068bf0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c4815d400 0x7f9c4814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c480ccd90 0x7f9c4814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 --2- 172.21.15.179:0/1171051928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c480c92d0 0x7f9c480adec0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 >> 172.21.15.179:0/1171051928 conn(0x7f9c480cb700 msgr2=0x7f9c4806e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 shutdown_connections 2024-05-08T22:42:33.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:33.882+0000 7f9c4f6fe700 1 -- 172.21.15.179:0/1171051928 wait complete. 2024-05-08T22:42:33.884 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:42:33.900 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:42:34.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.014+0000 7f67f181f700 1 Processor -- start 2024-05-08T22:42:34.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.014+0000 7f67f181f700 1 -- start start 2024-05-08T22:42:34.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.014+0000 7f67f181f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67ec0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f181f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0b3560 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f181f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0b3840 con 0x7f67ec0cf600 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f181f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec15d4d0 con 0x7f67ec15d0f0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67ebfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f67ec0b2ac0 0x7f67ec0ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52400/0 (socket says 172.21.15.179:52400) 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67ebfff700 1 -- 172.21.15.179:0/2964714540 learned_addr learned my addr 172.21.15.179:0/2964714540 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67ec0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216435180 0 0) 0x7f67ec0b3560 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d40035d0 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963717495 0 0) 0x7f67d40035d0 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 msgr2=0x7f67ec0b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67ec0b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 >> v1:172.21.15.195:6789/0 conn(0x7f67ec0cf600 legacy=0x7f67ec0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0b3ff0 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f67dc001670 con 0x7f67ec0b2ac0 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 >> v1:172.21.15.179:6789/0 conn(0x7f67ec0b2ac0 legacy=0x7f67ec0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d4003a50 0x7f67d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67d4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67d4004db0 0x7f67d4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec15d4d0 con 0x7f67d4003a50 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d400bbc0 con 0x7f67d4004db0 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d400bd50 con 0x7f67ec15d0f0 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d4003a50 0x7f67d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f101e700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67d4004db0 0x7f67d4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67ebfff700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67d4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 -- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 msgr2=0x7f67d4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec15d0f0 0x7f67d4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 -- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67d4004db0 msgr2=0x7f67d4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67d4004db0 0x7f67d4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 -- 172.21.15.179:0/2964714540 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d40035d0 con 0x7f67d4003a50 2024-05-08T22:42:34.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f081d700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d4003a50 0x7f67d4003e30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009ba0 tx=0x7f67d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef16a8d5b508cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000e6c0 con 0x7f67d4003a50 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e000e820 con 0x7f67d4003a50 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67eb7fe700 1 -- 172.21.15.179:0/2964714540 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000eb90 con 0x7f67d4003a50 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.018+0000 7f67f181f700 1 -- 172.21.15.179:0/2964714540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d4003a50 msgr2=0x7f67d4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 --2- 172.21.15.179:0/2964714540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d4003a50 0x7f67d4003e30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009ba0 tx=0x7f67d40058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f101e700 1 -- 172.21.15.179:0/2964714540 reap_dead start 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 -- 172.21.15.179:0/2964714540 shutdown_connections 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 -- 172.21.15.179:0/2964714540 >> 172.21.15.179:0/2964714540 conn(0x7f67ec0cfad0 msgr2=0x7f67ec0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 -- 172.21.15.179:0/2964714540 shutdown_connections 2024-05-08T22:42:34.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 -- 172.21.15.179:0/2964714540 wait complete. 2024-05-08T22:42:34.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 Processor -- start 2024-05-08T22:42:34.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.022+0000 7f67f181f700 1 -- start start 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec144760 0x7f67ec1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 0x7f67ec153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec150ce0 con 0x7f67ec06bfa0 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec153b90 con 0x7f67ec1531d0 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec152b50 con 0x7f67ec144760 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40504/0 (socket says 172.21.15.179:40504) 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 -- 172.21.15.179:0/1410781874 learned_addr learned my addr 172.21.15.179:0/1410781874 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:34.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67ebfff700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 0x7f67ec153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f081d700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec144760 0x7f67ec1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 -- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec144760 msgr2=0x7f67ec1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec144760 0x7f67ec1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 -- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 msgr2=0x7f67ec153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 0x7f67ec153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 -- 172.21.15.179:0/1410781874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec157a30 con 0x7f67ec06bfa0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67ebfff700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 0x7f67ec153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f101e700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0d0b90 tx=0x7f67e400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400fed0 con 0x7f67ec06bfa0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4003c30 con 0x7f67ec06bfa0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400d990 con 0x7f67ec06bfa0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec155f80 con 0x7f67ec06bfa0 2024-05-08T22:42:34.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.026+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1554a0 con 0x7f67ec06bfa0 2024-05-08T22:42:34.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec04e700 con 0x7f67ec06bfa0 2024-05-08T22:42:34.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f67e400daf0 con 0x7f67ec06bfa0 2024-05-08T22:42:34.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67e97fa700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 0x7f67d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f67e4084120 con 0x7f67ec06bfa0 2024-05-08T22:42:34.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67f081d700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 0x7f67d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.030+0000 7f67f081d700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 0x7f67d8068bf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0b2790 tx=0x7f67e001f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.038+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f67e4051730 con 0x7f67ec06bfa0 2024-05-08T22:42:34.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.266+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67ec0676e0 con 0x7f67ec06bfa0 2024-05-08T22:42:34.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.266+0000 7f67e97fa700 1 -- 172.21.15.179:0/1410781874 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f67e4054d50 con 0x7f67ec06bfa0 2024-05-08T22:42:34.270 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:34.270 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:42:34.270 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:42:34.270 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.271 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.272 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:42:34.273 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.274 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.275 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.276 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.277 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:42:34.278 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:42:34.280 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:42:34.281 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:42:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:42:34.283 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:42:34.284 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 msgr2=0x7f67d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 0x7f67d8068bf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0b2790 tx=0x7f67e001f040 comp rx=0 tx=0).stop 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 msgr2=0x7f67ec06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0d0b90 tx=0x7f67e400d430 comp rx=0 tx=0).stop 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 shutdown_connections 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f67d8066730 0x7f67d8068bf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67ec1531d0 0x7f67ec153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ec06bfa0 0x7f67ec06c420 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 --2- 172.21.15.179:0/1410781874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67ec144760 0x7f67ec1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 >> 172.21.15.179:0/1410781874 conn(0x7f67ec0cfad0 msgr2=0x7f67ec0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 shutdown_connections 2024-05-08T22:42:34.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.274+0000 7f67f181f700 1 -- 172.21.15.179:0/1410781874 wait complete. 2024-05-08T22:42:34.290 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:34.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 Processor -- start 2024-05-08T22:42:34.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 -- start start 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10681539c0 0x7f106806b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f106806bc50 0x7f10680b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1068159b60 con 0x7f10680b19e0 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106806b2c0 con 0x7f106806bc50 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106c992700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1068159e40 con 0x7f10681539c0 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106759e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f10680b19e0 0x7f106806abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52416/0 (socket says 172.21.15.179:52416) 2024-05-08T22:42:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f106806bc50 0x7f10680b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106759e700 1 -- 172.21.15.179:0/2722519846 learned_addr learned my addr 172.21.15.179:0/2722519846 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1066d9d700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10681539c0 0x7f106806b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1065d9b700 1 -- 172.21.15.179:0/2722519846 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929541947 0 0) 0x7f1068159b60 con 0x7f10680b19e0 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1065d9b700 1 -- 172.21.15.179:0/2722519846 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10540035d0 con 0x7f10680b19e0 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 -- 172.21.15.179:0/2722519846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10681539c0 msgr2=0x7f106806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10681539c0 0x7f106806b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 -- 172.21.15.179:0/2722519846 >> v1:172.21.15.179:6789/0 conn(0x7f10680b19e0 legacy=0x7f106806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 -- 172.21.15.179:0/2722519846 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106815a120 con 0x7f106806bc50 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f106659c700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806bc50 0x7f10680b3010 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1050009d90 tx=0x7f105000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9625b718b39b1dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1065d9b700 1 -- 172.21.15.179:0/2722519846 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000e070 con 0x7f106806bc50 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1065d9b700 1 -- 172.21.15.179:0/2722519846 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105000bb20 con 0x7f106806bc50 2024-05-08T22:42:34.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.402+0000 7f1065d9b700 1 -- 172.21.15.179:0/2722519846 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000be90 con 0x7f106806bc50 2024-05-08T22:42:34.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/2722519846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806bc50 msgr2=0x7f10680b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806bc50 0x7f10680b3010 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1050009d90 tx=0x7f105000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/2722519846 shutdown_connections 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806bc50 0x7f10680b3010 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- 172.21.15.179:0/2722519846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10681539c0 0x7f106806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/2722519846 >> 172.21.15.179:0/2722519846 conn(0x7f10680ce9d0 msgr2=0x7f10680cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/2722519846 shutdown_connections 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/2722519846 wait complete. 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 Processor -- start 2024-05-08T22:42:34.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- start start 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806bc50 0x7f1068143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680b19e0 0x7f10681484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106814f250 con 0x7f10681539c0 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106814f530 con 0x7f10680b19e0 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106814f810 con 0x7f106806bc50 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40532/0 (socket says 172.21.15.179:40532) 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 -- 172.21.15.179:0/755781924 learned_addr learned my addr 172.21.15.179:0/755781924 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106659c700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680b19e0 0x7f10681484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f1066d9d700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806bc50 0x7f1068143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 -- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806bc50 msgr2=0x7f1068143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806bc50 0x7f1068143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 -- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680b19e0 msgr2=0x7f10681484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680b19e0 0x7f10681484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 -- 172.21.15.179:0/755781924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050009a70 con 0x7f10681539c0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106759e700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f106000c430 tx=0x7f106000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060005d90 con 0x7f10681539c0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10600070d0 con 0x7f10681539c0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106814faf0 con 0x7f10681539c0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060007460 con 0x7f10681539c0 2024-05-08T22:42:34.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.406+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1068143ca0 con 0x7f10681539c0 2024-05-08T22:42:34.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.410+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1068067740 con 0x7f10681539c0 2024-05-08T22:42:34.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.410+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10600075c0 con 0x7f10681539c0 2024-05-08T22:42:34.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.410+0000 7f105f7fe700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 0x7f1048068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.410+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1060083410 con 0x7f10681539c0 2024-05-08T22:42:34.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.410+0000 7f1066d9d700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 0x7f1048068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.414+0000 7f1066d9d700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 0x7f1048068bf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f105800b9d0 tx=0x7f1058008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.418+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1060050a50 con 0x7f10681539c0 2024-05-08T22:42:34.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.590+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106804e700 con 0x7f10681539c0 2024-05-08T22:42:34.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.594+0000 7f105f7fe700 1 -- 172.21.15.179:0/755781924 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+12644 (secure 0 0 0) 0x7f1060054070 con 0x7f10681539c0 2024-05-08T22:42:34.596 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:34.596 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":108,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:33.870568+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:34.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.594+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 msgr2=0x7f1048068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.594+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 0x7f1048068bf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f105800b9d0 tx=0x7f1058008040 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.594+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 msgr2=0x7f1068158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.594+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f106000c430 tx=0x7f106000c7f0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 shutdown_connections 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1048066730 0x7f1048068bf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680b19e0 0x7f10681484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10681539c0 0x7f1068158330 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 --2- 172.21.15.179:0/755781924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806bc50 0x7f1068143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 >> 172.21.15.179:0/755781924 conn(0x7f10680ce9d0 msgr2=0x7f1068147630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 shutdown_connections 2024-05-08T22:42:34.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.598+0000 7f106c992700 1 -- 172.21.15.179:0/755781924 wait complete. 2024-05-08T22:42:34.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:34.611 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 Processor -- start 2024-05-08T22:42:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 -- start start 2024-05-08T22:42:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde6015d2e0 0x7fde600ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde600cd8d0 0x7fde600cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6015d6c0 con 0x7fde600cc630 2024-05-08T22:42:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600ce270 con 0x7fde6015d2e0 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde66d38700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6006cb60 con 0x7fde600cd8d0 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde64ad4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde6015d2e0 0x7fde600ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde64ad4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde6015d2e0 0x7fde600ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40548/0 (socket says 172.21.15.179:40548) 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde64ad4700 1 -- 172.21.15.179:0/3246135220 learned_addr learned my addr 172.21.15.179:0/3246135220 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.734+0000 7fde5ffff700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde600cd8d0 0x7fde600cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde64ad4700 1 -- 172.21.15.179:0/3246135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde600cd8d0 msgr2=0x7fde600cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde64ad4700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde600cd8d0 0x7fde600cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde64ad4700 1 -- 172.21.15.179:0/3246135220 >> v1:172.21.15.195:6789/0 conn(0x7fde600cc630 legacy=0x7fde600ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde64ad4700 1 -- 172.21.15.179:0/3246135220 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6006c580 con 0x7fde6015d2e0 2024-05-08T22:42:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde64ad4700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde6015d2e0 0x7fde600ca350 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fde54009910 tx=0x7fde54009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eabf9013e22d495 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde5f7fe700 1 -- 172.21.15.179:0/3246135220 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54011830 con 0x7fde6015d2e0 2024-05-08T22:42:34.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde5f7fe700 1 -- 172.21.15.179:0/3246135220 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54011990 con 0x7fde6015d2e0 2024-05-08T22:42:34.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde5f7fe700 1 -- 172.21.15.179:0/3246135220 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54011d40 con 0x7fde6015d2e0 2024-05-08T22:42:34.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 -- 172.21.15.179:0/3246135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde6015d2e0 msgr2=0x7fde600ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde6015d2e0 0x7fde600ca350 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fde54009910 tx=0x7fde54009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 -- 172.21.15.179:0/3246135220 shutdown_connections 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde600cd8d0 0x7fde600cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 --2- 172.21.15.179:0/3246135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde6015d2e0 0x7fde600ca350 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 -- 172.21.15.179:0/3246135220 >> 172.21.15.179:0/3246135220 conn(0x7fde600b18d0 msgr2=0x7fde600b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 -- 172.21.15.179:0/3246135220 shutdown_connections 2024-05-08T22:42:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.738+0000 7fde66d38700 1 -- 172.21.15.179:0/3246135220 wait complete. 2024-05-08T22:42:34.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 Processor -- start 2024-05-08T22:42:34.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- start start 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde600cd8d0 0x7fde6015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde6015d2e0 0x7fde6015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6015a6c0 con 0x7fde600cc630 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde601507c0 con 0x7fde600cd8d0 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde600adbb0 con 0x7fde6015d2e0 2024-05-08T22:42:34.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde5ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde600cd8d0 0x7fde6015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40556/0 (socket says 172.21.15.179:40556) 2024-05-08T22:42:34.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde652d5700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde6015d2e0 0x7fde6015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 -- 172.21.15.179:0/3189199035 learned_addr learned my addr 172.21.15.179:0/3189199035 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 -- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde6015d2e0 msgr2=0x7fde6015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde6015d2e0 0x7fde6015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 -- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde600cd8d0 msgr2=0x7fde6015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde600cd8d0 0x7fde6015af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 -- 172.21.15.179:0/3189199035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde54009570 con 0x7fde600cc630 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde64ad4700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fde600d0e00 tx=0x7fde54004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54004510 con 0x7fde600cc630 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5401e070 con 0x7fde600cc630 2024-05-08T22:42:34.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde600ade90 con 0x7fde600cc630 2024-05-08T22:42:34.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54003900 con 0x7fde600cc630 2024-05-08T22:42:34.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.742+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde600ae450 con 0x7fde600cc630 2024-05-08T22:42:34.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.746+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60073290 con 0x7fde600cc630 2024-05-08T22:42:34.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.746+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde54003a60 con 0x7fde600cc630 2024-05-08T22:42:34.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.746+0000 7fde5d7fa700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 0x7fde4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:34.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.746+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fde54084f90 con 0x7fde600cc630 2024-05-08T22:42:34.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.746+0000 7fde5ffff700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 0x7fde4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:34.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.750+0000 7fde5ffff700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 0x7fde4c068bf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde50009bf0 tx=0x7fde50009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:34.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.754+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fde54052580 con 0x7fde600cc630 2024-05-08T22:42:34.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.922+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde600afaa0 con 0x7fde600cc630 2024-05-08T22:42:34.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.922+0000 7fde5d7fa700 1 -- 172.21.15.179:0/3189199035 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde54020020 con 0x7fde600cc630 2024-05-08T22:42:34.924 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:34.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 msgr2=0x7fde4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 0x7fde4c068bf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde50009bf0 tx=0x7fde50009240 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 msgr2=0x7fde60150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fde600d0e00 tx=0x7fde54004770 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 shutdown_connections 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fde4c066730 0x7fde4c068bf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde600cd8d0 0x7fde6015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde6015d2e0 0x7fde6015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 --2- 172.21.15.179:0/3189199035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde600cc630 0x7fde60150280 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 >> 172.21.15.179:0/3189199035 conn(0x7fde600b18d0 msgr2=0x7fde6015c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 shutdown_connections 2024-05-08T22:42:34.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:34.926+0000 7fde66d38700 1 -- 172.21.15.179:0/3189199035 wait complete. 2024-05-08T22:42:34.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:42:35.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 Processor -- start 2024-05-08T22:42:35.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 -- start start 2024-05-08T22:42:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f85080cc310 0x7f850806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080add40 0x7f850806b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8508161ac0 con 0x7f85080ce060 2024-05-08T22:42:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85080ae550 con 0x7f85080cc310 2024-05-08T22:42:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850cfad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850806abb0 con 0x7f85080add40 2024-05-08T22:42:35.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f85080ce060 0x7f85080ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:52420/0 (socket says 172.21.15.179:52420) 2024-05-08T22:42:35.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850659c700 1 -- 172.21.15.179:0/2710917598 learned_addr learned my addr 172.21.15.179:0/2710917598 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:35.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8505d9b700 1 --2- 172.21.15.179:0/2710917598 >> v2:172.21.15.195:3300/0 conn(0x7f85080cc310 0x7f850806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8506d9d700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080add40 0x7f850806b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850559a700 1 -- 172.21.15.179:0/2710917598 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775211866 0 0) 0x7f8508161ac0 con 0x7f85080ce060 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f850559a700 1 -- 172.21.15.179:0/2710917598 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f00035d0 con 0x7f85080ce060 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8505d9b700 1 -- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080add40 msgr2=0x7f850806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8505d9b700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080add40 0x7f850806b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8505d9b700 1 -- 172.21.15.179:0/2710917598 >> v1:172.21.15.179:6789/0 conn(0x7f85080ce060 legacy=0x7f85080ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.050+0000 7f8505d9b700 1 -- 172.21.15.179:0/2710917598 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080cdaa0 con 0x7f85080cc310 2024-05-08T22:42:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f8505d9b700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f850806ce50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905be87842893bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850559a700 1 -- 172.21.15.179:0/2710917598 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011830 con 0x7f85080cc310 2024-05-08T22:42:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850559a700 1 -- 172.21.15.179:0/2710917598 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8011990 con 0x7f85080cc310 2024-05-08T22:42:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850559a700 1 -- 172.21.15.179:0/2710917598 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011d40 con 0x7f85080cc310 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 msgr2=0x7f850806ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f850806ce50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f8009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- 172.21.15.179:0/2710917598 shutdown_connections 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080add40 0x7f850806b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 --2- 172.21.15.179:0/2710917598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f850806ce50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- 172.21.15.179:0/2710917598 >> 172.21.15.179:0/2710917598 conn(0x7f85080cb660 msgr2=0x7f85080cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- 172.21.15.179:0/2710917598 shutdown_connections 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- 172.21.15.179:0/2710917598 wait complete. 2024-05-08T22:42:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 Processor -- start 2024-05-08T22:42:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.054+0000 7f850cfad700 1 -- start start 2024-05-08T22:42:35.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 0x7f850815e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080ce060 0x7f85080b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85080ac390 con 0x7f85080add40 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850815ec80 con 0x7f85080cc310 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850815f180 con 0x7f85080ce060 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 0x7f850815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f8505d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f8505d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:51722/0 (socket says 172.21.15.179:51722) 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f8505d9b700 1 -- 172.21.15.179:0/1603946374 learned_addr learned my addr 172.21.15.179:0/1603946374 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f8506d9d700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080ce060 0x7f85080b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 -- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080ce060 msgr2=0x7f85080b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080ce060 0x7f85080b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 -- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 msgr2=0x7f85080b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 -- 172.21.15.179:0/1603946374 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80094b0 con 0x7f85080add40 2024-05-08T22:42:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f8505d9b700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850659c700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 0x7f850815e740 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c500 tx=0x7f84fc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc005d90 con 0x7f85080add40 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc0070d0 con 0x7f85080add40 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85080b05a0 con 0x7f85080add40 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc007440 con 0x7f85080add40 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85080ab9f0 con 0x7f85080add40 2024-05-08T22:42:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.058+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508067770 con 0x7f85080add40 2024-05-08T22:42:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.062+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84fc0075a0 con 0x7f85080add40 2024-05-08T22:42:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.062+0000 7f84f6ffd700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 0x7f84e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.062+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f84fc083300 con 0x7f85080add40 2024-05-08T22:42:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.062+0000 7f8505d9b700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 0x7f84e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.062+0000 7f8505d9b700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 0x7f84e8068c20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000c00 tx=0x7f84f801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.070+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f84fc050870 con 0x7f85080add40 2024-05-08T22:42:35.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.242+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8508073290 con 0x7f85080add40 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f84f6ffd700 1 -- 172.21.15.179:0/1603946374 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+65 (secure 0 0 0) 0x7f84fc053e90 con 0x7f85080add40 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:35.247 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:42:35.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 msgr2=0x7f84e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 0x7f84e8068c20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000c00 tx=0x7f84f801c040 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 msgr2=0x7f850815e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 0x7f850815e740 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c500 tx=0x7f84fc00c810 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 shutdown_connections 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f84e8066760 0x7f84e8068c20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85080cc310 0x7f85080b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85080ce060 0x7f85080b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 --2- 172.21.15.179:0/1603946374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85080add40 0x7f850815e740 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 >> 172.21.15.179:0/1603946374 conn(0x7f85080cb660 msgr2=0x7f85080713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 shutdown_connections 2024-05-08T22:42:35.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.246+0000 7f850cfad700 1 -- 172.21.15.179:0/1603946374 wait complete. 2024-05-08T22:42:35.261 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:42:35.261 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:42:35.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 Processor -- start 2024-05-08T22:42:35.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 -- start start 2024-05-08T22:42:35.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8cfc159770 0x7f8cfc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0ce7c0 con 0x7f8cfc06d290 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ccd60 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.378+0000 7f8d01503700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0cd040 con 0x7f8cfc159770 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37292/0 (socket says 172.21.15.179:37292) 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 -- 172.21.15.179:0/3528987321 learned_addr learned my addr 172.21.15.179:0/3528987321 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:35.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfbfff700 1 --2- 172.21.15.179:0/3528987321 >> v2:172.21.15.195:3300/0 conn(0x7f8cfc159770 0x7f8cfc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 -- 172.21.15.179:0/3528987321 >> v2:172.21.15.195:3300/0 conn(0x7f8cfc159770 msgr2=0x7f8cfc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:35.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 --2- 172.21.15.179:0/3528987321 >> v2:172.21.15.195:3300/0 conn(0x7f8cfc159770 0x7f8cfc0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 -- 172.21.15.179:0/3528987321 >> v1:172.21.15.195:6789/0 conn(0x7f8cfc06d290 legacy=0x7f8cfc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 -- 172.21.15.179:0/3528987321 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0cd320 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfaffd700 1 -- 172.21.15.179:0/3528987321 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923207488 0 0) 0x7f8cfc0ce7c0 con 0x7f8cfc06d290 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfb7fe700 1 --2- 172.21.15.179:0/3528987321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06de80 tx=0x7f8cec00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3b3527e15238b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfaffd700 1 -- 172.21.15.179:0/3528987321 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec010070 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfaffd700 1 -- 172.21.15.179:0/3528987321 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec00dce0 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8cfaffd700 1 -- 172.21.15.179:0/3528987321 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00b310 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 -- 172.21.15.179:0/3528987321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc15dbb0 msgr2=0x7f8cfc06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 --2- 172.21.15.179:0/3528987321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06de80 tx=0x7f8cec00d430 comp rx=0 tx=0).stop 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 -- 172.21.15.179:0/3528987321 shutdown_connections 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 --2- 172.21.15.179:0/3528987321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc06cd50 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 --2- 172.21.15.179:0/3528987321 >> v2:172.21.15.195:3300/0 conn(0x7f8cfc159770 0x7f8cfc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 -- 172.21.15.179:0/3528987321 >> 172.21.15.179:0/3528987321 conn(0x7f8cfc0ce9a0 msgr2=0x7f8cfc0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:35.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 -- 172.21.15.179:0/3528987321 shutdown_connections 2024-05-08T22:42:35.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 -- 172.21.15.179:0/3528987321 wait complete. 2024-05-08T22:42:35.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.382+0000 7f8d01503700 1 Processor -- start 2024-05-08T22:42:35.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- start start 2024-05-08T22:42:35.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc159770 0x7f8cfc1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc14bc90 con 0x7f8cfc06d290 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1512b0 con 0x7f8cfc15dbb0 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ad4c0 con 0x7f8cfc159770 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d00d02700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37304/0 (socket says 172.21.15.179:37304) 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 -- 172.21.15.179:0/3456598554 learned_addr learned my addr 172.21.15.179:0/3456598554 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfb7fe700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc159770 0x7f8cfc1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 -- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc159770 msgr2=0x7f8cfc1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc159770 0x7f8cfc1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 -- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc15dbb0 msgr2=0x7f8cfc0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 -- 172.21.15.179:0/3456598554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec00bd40 con 0x7f8cfc06d290 2024-05-08T22:42:35.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cfbfff700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9d0 tx=0x7f8ce400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4009720 con 0x7f8cfc06d290 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400ee70 con 0x7f8cfc06d290 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc151500 con 0x7f8cfc06d290 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400ca50 con 0x7f8cfc06d290 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc151ac0 con 0x7f8cfc06d290 2024-05-08T22:42:35.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.386+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc04e770 con 0x7f8cfc06d290 2024-05-08T22:42:35.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.390+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ce400cbb0 con 0x7f8cfc06d290 2024-05-08T22:42:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.390+0000 7f8cf8ff9700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 0x7f8cdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.390+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8ce4083f60 con 0x7f8cfc06d290 2024-05-08T22:42:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.390+0000 7f8cfb7fe700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 0x7f8cdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.390+0000 7f8cfb7fe700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 0x7f8cdc068bf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f80 tx=0x7f8cec00b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:35.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.398+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ce40514d0 con 0x7f8cfc06d290 2024-05-08T22:42:35.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.566+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8cfc06e320 con 0x7f8cfc06d290 2024-05-08T22:42:35.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.890+0000 7f8cf8ff9700 1 -- 172.21.15.179:0/3456598554 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ce400ce60 con 0x7f8cfc06d290 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 msgr2=0x7f8cdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 0x7f8cdc068bf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f80 tx=0x7f8cec00b000 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 msgr2=0x7f8cfc0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9d0 tx=0x7f8ce400ece0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 shutdown_connections 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8cdc066730 0x7f8cdc068bf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cfc15dbb0 0x7f8cfc0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cfc159770 0x7f8cfc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 --2- 172.21.15.179:0/3456598554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc06d290 0x7f8cfc0acee0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 >> 172.21.15.179:0/3456598554 conn(0x7f8cfc0ce9a0 msgr2=0x7f8cfc1e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 shutdown_connections 2024-05-08T22:42:35.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:35.902+0000 7f8d01503700 1 -- 172.21.15.179:0/3456598554 wait complete. 2024-05-08T22:42:35.905 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:42:35.916 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:42:36.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.034+0000 7fb68ffff700 1 Processor -- start 2024-05-08T22:42:36.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.034+0000 7fb68ffff700 1 -- start start 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb69015d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68ffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb69013e030 0x7fb69013e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68ffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69006e470 con 0x7fb69006df70 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68ffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69006e750 con 0x7fb69013e030 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69013d550 con 0x7fb690159d00 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68effd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb69015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb69013e030 0x7fb69013e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68e7fc700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb69006df70 0x7fb69015deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:57810/0 (socket says 172.21.15.179:57810) 2024-05-08T22:42:36.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68e7fc700 1 -- 172.21.15.179:0/4249673321 learned_addr learned my addr 172.21.15.179:0/4249673321 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:36.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 -- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 msgr2=0x7fb69015d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 --2- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb69015d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 -- 172.21.15.179:0/4249673321 >> v1:172.21.15.195:6789/0 conn(0x7fb69006df70 legacy=0x7fb69015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 -- 172.21.15.179:0/4249673321 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69013d830 con 0x7fb69013e030 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68f7fe700 1 --2- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69013e030 0x7fb69013e480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb688009d90 tx=0x7fb68800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0755c5c704802e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68dffb700 1 -- 172.21.15.179:0/4249673321 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800bd20 con 0x7fb69013e030 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68dffb700 1 -- 172.21.15.179:0/4249673321 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800be80 con 0x7fb69013e030 2024-05-08T22:42:36.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.038+0000 7fb68dffb700 1 -- 172.21.15.179:0/4249673321 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6880116a0 con 0x7fb69013e030 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69013e030 msgr2=0x7fb69013e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 --2- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69013e030 0x7fb69013e480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb688009d90 tx=0x7fb68800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- 172.21.15.179:0/4249673321 shutdown_connections 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 --2- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69013e030 0x7fb69013e480 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 --2- 172.21.15.179:0/4249673321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb69015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- 172.21.15.179:0/4249673321 >> 172.21.15.179:0/4249673321 conn(0x7fb6900cfb50 msgr2=0x7fb6900cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- 172.21.15.179:0/4249673321 shutdown_connections 2024-05-08T22:42:36.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- 172.21.15.179:0/4249673321 wait complete. 2024-05-08T22:42:36.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 Processor -- start 2024-05-08T22:42:36.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.042+0000 7fb68ffff700 1 -- start start 2024-05-08T22:42:36.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69006df70 0x7fb6901514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb690142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb690133ee0 con 0x7fb69013e030 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb690151a30 con 0x7fb69006df70 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901332d0 con 0x7fb690159d00 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37318/0 (socket says 172.21.15.179:37318) 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 -- 172.21.15.179:0/3968886325 learned_addr learned my addr 172.21.15.179:0/3968886325 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68f7fe700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb690142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68effd700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69006df70 0x7fb6901514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 -- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 msgr2=0x7fb690142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb690142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 -- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69006df70 msgr2=0x7fb6901514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69006df70 0x7fb6901514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 -- 172.21.15.179:0/3968886325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688009a70 con 0x7fb69013e030 2024-05-08T22:42:36.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68e7fc700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c500 tx=0x7fb68400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684005d90 con 0x7fb69013e030 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840070d0 con 0x7fb69013e030 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69015a260 con 0x7fb69013e030 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684007460 con 0x7fb69013e030 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.046+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb690133890 con 0x7fb69013e030 2024-05-08T22:42:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6840075c0 con 0x7fb69013e030 2024-05-08T22:42:36.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690138040 con 0x7fb69013e030 2024-05-08T22:42:36.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb6777fe700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 0x7fb67c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:36.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fb684083520 con 0x7fb69013e030 2024-05-08T22:42:36.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb68effd700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 0x7fb67c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:36.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.050+0000 7fb68effd700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 0x7fb67c068bf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b50 tx=0x7fb680009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:36.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.058+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb690138040 con 0x7fb69013e030 2024-05-08T22:42:36.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.230+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb690138040 con 0x7fb69013e030 2024-05-08T22:42:36.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.906+0000 7fb6777fe700 1 -- 172.21.15.179:0/3968886325 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb690138040 con 0x7fb69013e030 2024-05-08T22:42:36.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.922+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 msgr2=0x7fb67c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.922+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 0x7fb67c068bf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b50 tx=0x7fb680009240 comp rx=0 tx=0).stop 2024-05-08T22:42:36.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.922+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 msgr2=0x7fb690132860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:36.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.922+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c500 tx=0x7fb68400c810 comp rx=0 tx=0).stop 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 shutdown_connections 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb67c066730 0x7fb67c068bf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb690159d00 0x7fb690142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb69013e030 0x7fb690132860 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 --2- 172.21.15.179:0/3968886325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb69006df70 0x7fb6901514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 >> 172.21.15.179:0/3968886325 conn(0x7fb6900cfb50 msgr2=0x7fb6900b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 shutdown_connections 2024-05-08T22:42:36.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:36.926+0000 7fb68ffff700 1 -- 172.21.15.179:0/3968886325 wait complete. 2024-05-08T22:42:36.928 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:42:36.939 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:42:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 Processor -- start 2024-05-08T22:42:37.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 -- start start 2024-05-08T22:42:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 0x7f29840b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 0x7f29840cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298415d4a0 con 0x7f29840ce0e0 2024-05-08T22:42:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29840ce570 con 0x7f298415d0c0 2024-05-08T22:42:37.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298406abb0 con 0x7f29840d1d20 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f2982d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f29840ce0e0 0x7f2984160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35952/0 (socket says 172.21.15.179:35952) 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f2982d9d700 1 -- 172.21.15.179:0/1989764749 learned_addr learned my addr 172.21.15.179:0/1989764749 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f2981d9b700 1 --2- 172.21.15.179:0/1989764749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 0x7f29840cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.054+0000 7f298259c700 1 --2- 172.21.15.179:0/1989764749 >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 0x7f29840b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359047490 0 0) 0x7f298415d4a0 con 0x7f29840ce0e0 2024-05-08T22:42:37.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29700035d0 con 0x7f29840ce0e0 2024-05-08T22:42:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638564928 0 0) 0x7f29700035d0 con 0x7f29840ce0e0 2024-05-08T22:42:37.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f2981d9b700 1 -- 172.21.15.179:0/1989764749 >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 msgr2=0x7f29840b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f2981d9b700 1 --2- 172.21.15.179:0/1989764749 >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 0x7f29840b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f2981d9b700 1 -- 172.21.15.179:0/1989764749 >> v1:172.21.15.179:6789/0 conn(0x7f29840ce0e0 legacy=0x7f2984160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f2981d9b700 1 -- 172.21.15.179:0/1989764749 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840d2500 con 0x7f29840d1d20 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298259c700 1 --2- 172.21.15.179:0/1989764749 >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 0x7f29840b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f2981d9b700 1 --2- 172.21.15.179:0/1989764749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 0x7f29840cdba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f296c009850 tx=0x7f296c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa19278c0a84ce23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0019f0 con 0x7f29840d1d20 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c004030 con 0x7f29840d1d20 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298159a700 1 -- 172.21.15.179:0/1989764749 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0043a0 con 0x7f29840d1d20 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- 172.21.15.179:0/1989764749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 msgr2=0x7f29840cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 --2- 172.21.15.179:0/1989764749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 0x7f29840cdba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f296c009850 tx=0x7f296c009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- 172.21.15.179:0/1989764749 shutdown_connections 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 --2- 172.21.15.179:0/1989764749 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840d1d20 0x7f29840cdba0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 --2- 172.21.15.179:0/1989764749 >> v2:172.21.15.195:3300/0 conn(0x7f298415d0c0 0x7f29840b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- 172.21.15.179:0/1989764749 >> 172.21.15.179:0/1989764749 conn(0x7f29840d2ed0 msgr2=0x7f29840d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- 172.21.15.179:0/1989764749 shutdown_connections 2024-05-08T22:42:37.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- 172.21.15.179:0/1989764749 wait complete. 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 Processor -- start 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.058+0000 7f298359e700 1 -- start start 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840ce0e0 0x7f2984155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 0x7f29841503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cc4f0 con 0x7f29840d1d20 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cbf40 con 0x7f298415d0c0 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cc210 con 0x7f29840ce0e0 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37320/0 (socket says 172.21.15.179:37320) 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 -- 172.21.15.179:0/575845764 learned_addr learned my addr 172.21.15.179:0/575845764 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298259c700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840ce0e0 0x7f2984155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2982d9d700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 0x7f29841503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 -- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840ce0e0 msgr2=0x7f2984155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840ce0e0 0x7f2984155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 -- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 msgr2=0x7f29841503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 0x7f29841503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 -- 172.21.15.179:0/575845764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740094b0 con 0x7f29840d1d20 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2982d9d700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 0x7f29841503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f2981d9b700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f296c0090b0 tx=0x7f296c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c003d30 con 0x7f29840d1d20 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c0094b0 con 0x7f29840d1d20 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c01f070 con 0x7f29840d1d20 2024-05-08T22:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2984153fc0 con 0x7f29840d1d20 2024-05-08T22:42:37.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c011830 con 0x7f29840d1d20 2024-05-08T22:42:37.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2984158260 con 0x7f29840d1d20 2024-05-08T22:42:37.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.062+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f296c011990 con 0x7f29840d1d20 2024-05-08T22:42:37.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.066+0000 7f297affd700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 0x7f2964068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.066+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f296c021070 con 0x7f29840d1d20 2024-05-08T22:42:37.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.066+0000 7f298259c700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 0x7f2964068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.066+0000 7f298259c700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 0x7f2964068bf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f29841f6030 tx=0x7f2974009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.074+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f296c052a10 con 0x7f29840d1d20 2024-05-08T22:42:37.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.278+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f298404e700 con 0x7f29840d1d20 2024-05-08T22:42:37.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.286+0000 7f297affd700 1 -- 172.21.15.179:0/575845764 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7f296c056030 con 0x7f29840d1d20 2024-05-08T22:42:37.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 msgr2=0x7f2964068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 0x7f2964068bf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f29841f6030 tx=0x7f2974009240 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 msgr2=0x7f29840cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f296c0090b0 tx=0x7f296c004770 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 shutdown_connections 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2964066730 0x7f2964068bf0 secure :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f29841f6030 tx=0x7f2974009240 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415d0c0 0x7f29841503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29840d1d20 0x7f29840cba00 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 --2- 172.21.15.179:0/575845764 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29840ce0e0 0x7f2984155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 >> 172.21.15.179:0/575845764 conn(0x7f29840d2ed0 msgr2=0x7f29840b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 shutdown_connections 2024-05-08T22:42:37.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.310+0000 7f298359e700 1 -- 172.21.15.179:0/575845764 wait complete. 2024-05-08T22:42:37.314 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 15 and data pool 16 2024-05-08T22:42:37.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:37.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 Processor -- start 2024-05-08T22:42:37.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 -- start start 2024-05-08T22:42:37.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f30a415ef80 0x7f30a4161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 0x7f30a406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a4165d50 con 0x7f30a4164db0 2024-05-08T22:42:37.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a406ad70 con 0x7f30a415ef80 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a4165260 con 0x7f30a406de70 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f30a415ef80 0x7f30a4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f30a415ef80 0x7f30a4161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37330/0 (socket says 172.21.15.179:37330) 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 -- 172.21.15.179:0/1053260073 learned_addr learned my addr 172.21.15.179:0/1053260073 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 -- 172.21.15.179:0/1053260073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 msgr2=0x7f30a406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a8fcc700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 0x7f30a406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 0x7f30a406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 -- 172.21.15.179:0/1053260073 >> v1:172.21.15.195:6789/0 conn(0x7f30a4164db0 legacy=0x7f30a406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 -- 172.21.15.179:0/1053260073 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a406e4b0 con 0x7f30a415ef80 2024-05-08T22:42:37.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a8fcc700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 0x7f30a406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a3fff700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a4161790 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f308c009e50 tx=0x7f308c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7178c36818e870b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a2ffd700 1 -- 172.21.15.179:0/1053260073 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00e070 con 0x7f30a415ef80 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a2ffd700 1 -- 172.21.15.179:0/1053260073 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c00b9c0 con 0x7f30a415ef80 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a2ffd700 1 -- 172.21.15.179:0/1053260073 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00bd70 con 0x7f30a415ef80 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 -- 172.21.15.179:0/1053260073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 msgr2=0x7f30a4161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.430+0000 7f30a97cd700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a4161790 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f308c009e50 tx=0x7f308c00b330 comp rx=0 tx=0).stop 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 -- 172.21.15.179:0/1053260073 shutdown_connections 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a406de70 0x7f30a406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 --2- 172.21.15.179:0/1053260073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a4161790 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 -- 172.21.15.179:0/1053260073 >> 172.21.15.179:0/1053260073 conn(0x7f30a40ce9d0 msgr2=0x7f30a40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 -- 172.21.15.179:0/1053260073 shutdown_connections 2024-05-08T22:42:37.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 -- 172.21.15.179:0/1053260073 wait complete. 2024-05-08T22:42:37.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 Processor -- start 2024-05-08T22:42:37.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.434+0000 7f30a97cd700 1 -- start start 2024-05-08T22:42:37.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a406de70 0x7f30a41f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a4164db0 0x7f30a41f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a41f8400 con 0x7f30a415ef80 2024-05-08T22:42:37.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a414d380 con 0x7f30a406de70 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a414d850 con 0x7f30a4164db0 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37336/0 (socket says 172.21.15.179:37336) 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 -- 172.21.15.179:0/810119158 learned_addr learned my addr 172.21.15.179:0/810119158 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a8fcc700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a4164db0 0x7f30a41f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a3fff700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a406de70 0x7f30a41f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 -- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a4164db0 msgr2=0x7f30a41f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a4164db0 0x7f30a41f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 -- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a406de70 msgr2=0x7f30a41f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a406de70 0x7f30a41f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 -- 172.21.15.179:0/810119158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c009b30 con 0x7f30a415ef80 2024-05-08T22:42:37.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a37fe700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f309400c500 tx=0x7f309400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940052d0 con 0x7f30a415ef80 2024-05-08T22:42:37.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094005430 con 0x7f30a415ef80 2024-05-08T22:42:37.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a414db30 con 0x7f30a415ef80 2024-05-08T22:42:37.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940055a0 con 0x7f30a415ef80 2024-05-08T22:42:37.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a414e0f0 con 0x7f30a415ef80 2024-05-08T22:42:37.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a4067740 con 0x7f30a415ef80 2024-05-08T22:42:37.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.438+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f30940038e0 con 0x7f30a415ef80 2024-05-08T22:42:37.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.442+0000 7f30a0ff9700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 0x7f3088068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.442+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3094083510 con 0x7f30a415ef80 2024-05-08T22:42:37.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.442+0000 7f30a3fff700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 0x7f3088068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.442+0000 7f30a3fff700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 0x7f3088068bf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b370 tx=0x7f308c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.446+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f30940507d0 con 0x7f30a415ef80 2024-05-08T22:42:37.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.622+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a404e700 con 0x7f30a415ef80 2024-05-08T22:42:37.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.622+0000 7f30a0ff9700 1 -- 172.21.15.179:0/810119158 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f3094053df0 con 0x7f30a415ef80 2024-05-08T22:42:37.626 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:37.626 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":111,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:37.284026+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 msgr2=0x7f3088068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 0x7f3088068bf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b370 tx=0x7f308c009000 comp rx=0 tx=0).stop 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 msgr2=0x7f30a414ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f309400c500 tx=0x7f309400c810 comp rx=0 tx=0).stop 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 shutdown_connections 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3088066730 0x7f3088068bf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30a4164db0 0x7f30a41f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a415ef80 0x7f30a414ce40 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 --2- 172.21.15.179:0/810119158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30a406de70 0x7f30a41f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 >> 172.21.15.179:0/810119158 conn(0x7f30a40ce9d0 msgr2=0x7f30a4160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 shutdown_connections 2024-05-08T22:42:37.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.626+0000 7f30a97cd700 1 -- 172.21.15.179:0/810119158 wait complete. 2024-05-08T22:42:37.640 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:37.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.750+0000 7fa3b9a99700 1 Processor -- start 2024-05-08T22:42:37.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.750+0000 7fa3b9a99700 1 -- start start 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3b41434d0 0x7fa3b406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3b406ed50 0x7fa3b40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b40cd200 con 0x7fa3b406dc20 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b40cd4e0 con 0x7fa3b406ed50 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b40cd7c0 con 0x7fa3b41434d0 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b3fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa3b406dc20 0x7fa3b406e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35956/0 (socket says 172.21.15.179:35956) 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3b406ed50 0x7fa3b40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b3fff700 1 -- 172.21.15.179:0/1223539320 learned_addr learned my addr 172.21.15.179:0/1223539320 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:37.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b8a97700 1 --2- 172.21.15.179:0/1223539320 >> v2:172.21.15.195:3300/0 conn(0x7fa3b41434d0 0x7fa3b406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b37fe700 1 -- 172.21.15.179:0/1223539320 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113140614 0 0) 0x7fa3b40cd200 con 0x7fa3b406dc20 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 -- 172.21.15.179:0/1223539320 >> v2:172.21.15.195:3300/0 conn(0x7fa3b41434d0 msgr2=0x7fa3b406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 --2- 172.21.15.179:0/1223539320 >> v2:172.21.15.195:3300/0 conn(0x7fa3b41434d0 0x7fa3b406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 -- 172.21.15.179:0/1223539320 >> v1:172.21.15.179:6789/0 conn(0x7fa3b406dc20 legacy=0x7fa3b406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 -- 172.21.15.179:0/1223539320 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b40cdaa0 con 0x7fa3b406ed50 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9298700 1 --2- 172.21.15.179:0/1223539320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b40b19e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4070800 tx=0x7fa3ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca53e9886cc5bafc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b37fe700 1 -- 172.21.15.179:0/1223539320 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00e070 con 0x7fa3b406ed50 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b37fe700 1 -- 172.21.15.179:0/1223539320 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac00bb20 con 0x7fa3b406ed50 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b37fe700 1 -- 172.21.15.179:0/1223539320 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac012430 con 0x7fa3b406ed50 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- 172.21.15.179:0/1223539320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 msgr2=0x7fa3b40b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/1223539320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b40b19e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4070800 tx=0x7fa3ac00b330 comp rx=0 tx=0).stop 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- 172.21.15.179:0/1223539320 shutdown_connections 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/1223539320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b40b19e0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/1223539320 >> v2:172.21.15.195:3300/0 conn(0x7fa3b41434d0 0x7fa3b406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- 172.21.15.179:0/1223539320 >> 172.21.15.179:0/1223539320 conn(0x7fa3b40ce9d0 msgr2=0x7fa3b40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- 172.21.15.179:0/1223539320 shutdown_connections 2024-05-08T22:42:37.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.754+0000 7fa3b9a99700 1 -- 172.21.15.179:0/1223539320 wait complete. 2024-05-08T22:42:37.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 Processor -- start 2024-05-08T22:42:37.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 -- start start 2024-05-08T22:42:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b41434d0 0x7fa3b40c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 0x7fa3b40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40ca960 con 0x7fa3b406ed50 2024-05-08T22:42:37.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b41f5ca0 con 0x7fa3b40c9f70 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9a99700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b41f5f80 con 0x7fa3b41434d0 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37366/0 (socket says 172.21.15.179:37366) 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 -- 172.21.15.179:0/2901865754 learned_addr learned my addr 172.21.15.179:0/2901865754 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9298700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 0x7fa3b40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b3fff700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b41434d0 0x7fa3b40c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 -- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b41434d0 msgr2=0x7fa3b40c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b41434d0 0x7fa3b40c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 -- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 msgr2=0x7fa3b40ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 0x7fa3b40ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b8a97700 1 -- 172.21.15.179:0/2901865754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009a70 con 0x7fa3b406ed50 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.758+0000 7fa3b9298700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 0x7fa3b40ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b8a97700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e840 tx=0x7fa3a400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40095a0 con 0x7fa3b406ed50 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4013680 con 0x7fa3b406ed50 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40139f0 con 0x7fa3b406ed50 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b41f6260 con 0x7fa3b406ed50 2024-05-08T22:42:37.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b41f6820 con 0x7fa3b406ed50 2024-05-08T22:42:37.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.762+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b4067740 con 0x7fa3b406ed50 2024-05-08T22:42:37.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.766+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3a4013b50 con 0x7fa3b406ed50 2024-05-08T22:42:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.766+0000 7fa3b17fa700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 0x7fa3a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.766+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa3a4084030 con 0x7fa3b406ed50 2024-05-08T22:42:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.766+0000 7fa3b3fff700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 0x7fa3a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:37.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.766+0000 7fa3b3fff700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 0x7fa3a0068c20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40c5b30 tx=0x7fa3a8007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:37.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.774+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa3a40512f0 con 0x7fa3b406ed50 2024-05-08T22:42:37.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.946+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3b404e700 con 0x7fa3b406ed50 2024-05-08T22:42:37.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b17fa700 1 -- 172.21.15.179:0/2901865754 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16309 (secure 0 0 0) 0x7fa3a4054910 con 0x7fa3b406ed50 2024-05-08T22:42:37.952 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:37.952 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":111,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:37.284026+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 msgr2=0x7fa3a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 0x7fa3a0068c20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40c5b30 tx=0x7fa3a8007e00 comp rx=0 tx=0).stop 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 msgr2=0x7fa3b4159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e840 tx=0x7fa3a400eb50 comp rx=0 tx=0).stop 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 shutdown_connections 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3a0066760 0x7fa3a0068c20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b40c9f70 0x7fa3b40ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b41434d0 0x7fa3b40c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 --2- 172.21.15.179:0/2901865754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3b406ed50 0x7fa3b4159700 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 >> 172.21.15.179:0/2901865754 conn(0x7fa3b40ce9d0 msgr2=0x7fa3b415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 shutdown_connections 2024-05-08T22:42:37.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:37.950+0000 7fa3b9a99700 1 -- 172.21.15.179:0/2901865754 wait complete. 2024-05-08T22:42:37.967 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:42:38.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 Processor -- start 2024-05-08T22:42:38.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- start start 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb3c15d050 0x7fdb3c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3c0cb640 con 0x7fdb3c1611b0 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c161b50 con 0x7fdb3c0b4aa0 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c161640 con 0x7fdb3c15d050 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37382/0 (socket says 172.21.15.179:37382) 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 -- 172.21.15.179:0/2381527329 learned_addr learned my addr 172.21.15.179:0/2381527329 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb41184700 1 --2- 172.21.15.179:0/2381527329 >> v2:172.21.15.195:3300/0 conn(0x7fdb3c15d050 0x7fdb3c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 -- 172.21.15.179:0/2381527329 >> v2:172.21.15.195:3300/0 conn(0x7fdb3c15d050 msgr2=0x7fdb3c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:38.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 --2- 172.21.15.179:0/2381527329 >> v2:172.21.15.195:3300/0 conn(0x7fdb3c15d050 0x7fdb3c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 -- 172.21.15.179:0/2381527329 >> v1:172.21.15.195:6789/0 conn(0x7fdb3c1611b0 legacy=0x7fdb3c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 -- 172.21.15.179:0/2381527329 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0cb310 con 0x7fdb3c0b4aa0 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb40983700 1 --2- 172.21.15.179:0/2381527329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c074520 tx=0x7fdb38009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108490401c22c518 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb33fff700 1 -- 172.21.15.179:0/2381527329 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800b8a0 con 0x7fdb3c0b4aa0 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb33fff700 1 -- 172.21.15.179:0/2381527329 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3800ba00 con 0x7fdb3c0b4aa0 2024-05-08T22:42:38.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb33fff700 1 -- 172.21.15.179:0/2381527329 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800bd90 con 0x7fdb3c0b4aa0 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- 172.21.15.179:0/2381527329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c0b4aa0 msgr2=0x7fdb3c160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 --2- 172.21.15.179:0/2381527329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c074520 tx=0x7fdb38009bf0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- 172.21.15.179:0/2381527329 shutdown_connections 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 --2- 172.21.15.179:0/2381527329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c0b4aa0 0x7fdb3c160c70 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 --2- 172.21.15.179:0/2381527329 >> v2:172.21.15.195:3300/0 conn(0x7fdb3c15d050 0x7fdb3c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- 172.21.15.179:0/2381527329 >> 172.21.15.179:0/2381527329 conn(0x7fdb3c0b2710 msgr2=0x7fdb3c0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- 172.21.15.179:0/2381527329 shutdown_connections 2024-05-08T22:42:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.078+0000 7fdb42186700 1 -- 172.21.15.179:0/2381527329 wait complete. 2024-05-08T22:42:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 Processor -- start 2024-05-08T22:42:38.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 -- start start 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb3c15d050 0x7fdb3c0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb3c15faf0 0x7fdb3c15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c1604e0 con 0x7fdb3c1611b0 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c1f3ac0 con 0x7fdb3c15d050 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb42186700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c1f37e0 con 0x7fdb3c15faf0 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb40983700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb41184700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb3c15d050 0x7fdb3c0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb40983700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37390/0 (socket says 172.21.15.179:37390) 2024-05-08T22:42:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb40983700 1 -- 172.21.15.179:0/406451399 learned_addr learned my addr 172.21.15.179:0/406451399 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:38.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.082+0000 7fdb41985700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb3c15faf0 0x7fdb3c15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 -- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb3c15faf0 msgr2=0x7fdb3c15ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb3c15faf0 0x7fdb3c15ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 -- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb3c15d050 msgr2=0x7fdb3c0c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb3c15d050 0x7fdb3c0c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 -- 172.21.15.179:0/406451399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38009570 con 0x7fdb3c1611b0 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb40983700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c067b10 tx=0x7fdb38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380036c0 con 0x7fdb3c1611b0 2024-05-08T22:42:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3801f070 con 0x7fdb3c1611b0 2024-05-08T22:42:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c0c58b0 con 0x7fdb3c1611b0 2024-05-08T22:42:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38003a20 con 0x7fdb3c1611b0 2024-05-08T22:42:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c0c5e70 con 0x7fdb3c1611b0 2024-05-08T22:42:38.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.086+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c0c6090 con 0x7fdb3c1611b0 2024-05-08T22:42:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.090+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb38021030 con 0x7fdb3c1611b0 2024-05-08T22:42:38.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.090+0000 7fdb31ffb700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.090+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdb38085540 con 0x7fdb3c1611b0 2024-05-08T22:42:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.090+0000 7fdb41184700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.090+0000 7fdb41184700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 0x7fdb20068bf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0c6ff0 tx=0x7fdb2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:38.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.094+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdb38052800 con 0x7fdb3c1611b0 2024-05-08T22:42:38.207 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:42:38.202+0000 7f3d336ae700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:42:38.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:38.298+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdb3c04e700 con 0x7fdb3c1611b0 2024-05-08T22:42:39.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.302+0000 7fdb31ffb700 1 -- 172.21.15.179:0/406451399 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdb38055e20 con 0x7fdb3c1611b0 2024-05-08T22:42:39.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 msgr2=0x7fdb20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 0x7fdb20068bf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0c6ff0 tx=0x7fdb2c008040 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 msgr2=0x7fdb3c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c067b10 tx=0x7fdb38004770 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 shutdown_connections 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb3c15faf0 0x7fdb3c15ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c1611b0 0x7fdb3c0ca360 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 --2- 172.21.15.179:0/406451399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb3c15d050 0x7fdb3c0c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 >> 172.21.15.179:0/406451399 conn(0x7fdb3c0b2710 msgr2=0x7fdb3c0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:39.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 shutdown_connections 2024-05-08T22:42:39.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.306+0000 7fdb42186700 1 -- 172.21.15.179:0/406451399 wait complete. 2024-05-08T22:42:39.327 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:42:39.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:42:39.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.442+0000 7f9e8e340700 1 Processor -- start 2024-05-08T22:42:39.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.442+0000 7f9e8e340700 1 -- start start 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9e880b3480 0x7f9e8815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 0x7f9e880b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8815dce0 con 0x7f9e880b24d0 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8806b5e0 con 0x7f9e880b3480 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815dfc0 con 0x7f9e880b3860 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8c8dd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9e880b24d0 0x7f9e8806ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35968/0 (socket says 172.21.15.179:35968) 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8c8dd700 1 -- 172.21.15.179:0/1641476295 learned_addr learned my addr 172.21.15.179:0/1641476295 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 --2- 172.21.15.179:0/1641476295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 0x7f9e880b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e87fff700 1 --2- 172.21.15.179:0/1641476295 >> v2:172.21.15.195:3300/0 conn(0x7f9e880b3480 0x7f9e8815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e86ffd700 1 -- 172.21.15.179:0/1641476295 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160211363 0 0) 0x7f9e8815dce0 con 0x7f9e880b24d0 2024-05-08T22:42:39.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e86ffd700 1 -- 172.21.15.179:0/1641476295 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e740035d0 con 0x7f9e880b24d0 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 -- 172.21.15.179:0/1641476295 >> v2:172.21.15.195:3300/0 conn(0x7f9e880b3480 msgr2=0x7f9e8815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 --2- 172.21.15.179:0/1641476295 >> v2:172.21.15.195:3300/0 conn(0x7f9e880b3480 0x7f9e8815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 -- 172.21.15.179:0/1641476295 >> v1:172.21.15.179:6789/0 conn(0x7f9e880b24d0 legacy=0x7f9e8806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 -- 172.21.15.179:0/1641476295 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815e2a0 con 0x7f9e880b3860 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e877fe700 1 --2- 172.21.15.179:0/1641476295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 0x7f9e880b1f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e78009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30570cc884e056f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e86ffd700 1 -- 172.21.15.179:0/1641476295 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78011830 con 0x7f9e880b3860 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e86ffd700 1 -- 172.21.15.179:0/1641476295 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78011990 con 0x7f9e880b3860 2024-05-08T22:42:39.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e86ffd700 1 -- 172.21.15.179:0/1641476295 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78011d00 con 0x7f9e880b3860 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- 172.21.15.179:0/1641476295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 msgr2=0x7f9e880b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 --2- 172.21.15.179:0/1641476295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 0x7f9e880b1f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e78009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- 172.21.15.179:0/1641476295 shutdown_connections 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 --2- 172.21.15.179:0/1641476295 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b3860 0x7f9e880b1f90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 --2- 172.21.15.179:0/1641476295 >> v2:172.21.15.195:3300/0 conn(0x7f9e880b3480 0x7f9e8815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- 172.21.15.179:0/1641476295 >> 172.21.15.179:0/1641476295 conn(0x7f9e880ceca0 msgr2=0x7f9e880cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:39.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- 172.21.15.179:0/1641476295 shutdown_connections 2024-05-08T22:42:39.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.446+0000 7f9e8e340700 1 -- 172.21.15.179:0/1641476295 wait complete. 2024-05-08T22:42:39.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 Processor -- start 2024-05-08T22:42:39.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- start start 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b24d0 0x7f9e881409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 0x7f9e8815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88145020 con 0x7f9e880b3860 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88145880 con 0x7f9e880b3480 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e881452c0 con 0x7f9e880b24d0 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37394/0 (socket says 172.21.15.179:37394) 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 -- 172.21.15.179:0/3059959283 learned_addr learned my addr 172.21.15.179:0/3059959283 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e87fff700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b24d0 0x7f9e881409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e877fe700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 0x7f9e8815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 -- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b24d0 msgr2=0x7f9e881409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b24d0 0x7f9e881409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 -- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 msgr2=0x7f9e8815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 0x7f9e8815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 -- 172.21.15.179:0/3059959283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780094b0 con 0x7f9e880b3860 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e877fe700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 0x7f9e8815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8c8dd700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000cb40 tx=0x7f9e8000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80007500 con 0x7f9e880b3860 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80007660 con 0x7f9e880b3860 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e881455a0 con 0x7f9e880b3860 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800079f0 con 0x7f9e880b3860 2024-05-08T22:42:39.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.450+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8815d040 con 0x7f9e880b3860 2024-05-08T22:42:39.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.454+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e88146500 con 0x7f9e880b3860 2024-05-08T22:42:39.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.454+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e8001b3b0 con 0x7f9e880b3860 2024-05-08T22:42:39.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.454+0000 7f9e84ff9700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 0x7f9e68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:39.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.454+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9e80083ca0 con 0x7f9e880b3860 2024-05-08T22:42:39.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.454+0000 7f9e87fff700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 0x7f9e68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:39.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.458+0000 7f9e87fff700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 0x7f9e68068bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009bf0 tx=0x7f9e70009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:39.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.462+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e88146500 con 0x7f9e880b3860 2024-05-08T22:42:39.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:39.674+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9e88146500 con 0x7f9e880b3860 2024-05-08T22:42:40.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.306+0000 7f9e84ff9700 1 -- 172.21.15.179:0/3059959283 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e88146500 con 0x7f9e880b3860 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 msgr2=0x7f9e68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 0x7f9e68068bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009bf0 tx=0x7f9e70009240 comp rx=0 tx=0).stop 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 msgr2=0x7f9e8814dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000cb40 tx=0x7f9e8000cb70 comp rx=0 tx=0).stop 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 shutdown_connections 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9e68066730 0x7f9e68068bf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e880b3480 0x7f9e8815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e880b3860 0x7f9e8814dea0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 --2- 172.21.15.179:0/3059959283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e880b24d0 0x7f9e881409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 >> 172.21.15.179:0/3059959283 conn(0x7f9e880ceca0 msgr2=0x7f9e8806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:40.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 shutdown_connections 2024-05-08T22:42:40.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.310+0000 7f9e8e340700 1 -- 172.21.15.179:0/3059959283 wait complete. 2024-05-08T22:42:40.327 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:42:40.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:40.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.438+0000 7f7db359e700 1 Processor -- start 2024-05-08T22:42:40.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.438+0000 7f7db359e700 1 -- start start 2024-05-08T22:42:40.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7db40ce380 0x7f7db40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db406abb0 con 0x7f7db40cb510 2024-05-08T22:42:40.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db406ae90 con 0x7f7db40cc700 2024-05-08T22:42:40.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40cb980 con 0x7f7db40ce380 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db2d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7db40cb510 0x7f7db406b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35976/0 (socket says 172.21.15.179:35976) 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db2d9d700 1 -- 172.21.15.179:0/3767741366 learned_addr learned my addr 172.21.15.179:0/3767741366 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7da9d9b700 1 --2- 172.21.15.179:0/3767741366 >> v2:172.21.15.195:3300/0 conn(0x7f7db40ce380 0x7f7db40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3767741366 >> v1:172.21.15.195:6789/0 conn(0x7f7db40cc700 legacy=0x7f7db415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db1d9b700 1 -- 172.21.15.179:0/3767741366 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882486651 0 0) 0x7f7db406abb0 con 0x7f7db40cb510 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3767741366 >> v1:172.21.15.179:6789/0 conn(0x7f7db40cb510 legacy=0x7f7db406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3767741366 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40ce800 con 0x7f7db40ce380 2024-05-08T22:42:40.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7da9d9b700 1 --2- 172.21.15.179:0/3767741366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db40cafd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009850 tx=0x7f7d9c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694357136307da5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db1d9b700 1 -- 172.21.15.179:0/3767741366 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011830 con 0x7f7db40ce380 2024-05-08T22:42:40.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db1d9b700 1 -- 172.21.15.179:0/3767741366 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c011990 con 0x7f7db40ce380 2024-05-08T22:42:40.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db1d9b700 1 -- 172.21.15.179:0/3767741366 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011d20 con 0x7f7db40ce380 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- 172.21.15.179:0/3767741366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 msgr2=0x7f7db40cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 --2- 172.21.15.179:0/3767741366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db40cafd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009850 tx=0x7f7d9c009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- 172.21.15.179:0/3767741366 shutdown_connections 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 --2- 172.21.15.179:0/3767741366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db40cafd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- 172.21.15.179:0/3767741366 >> 172.21.15.179:0/3767741366 conn(0x7f7db40ce9d0 msgr2=0x7f7db40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- 172.21.15.179:0/3767741366 shutdown_connections 2024-05-08T22:42:40.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.442+0000 7f7db359e700 1 -- 172.21.15.179:0/3767741366 wait complete. 2024-05-08T22:42:40.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.446+0000 7f7db359e700 1 Processor -- start 2024-05-08T22:42:40.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.446+0000 7f7db359e700 1 -- start start 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.446+0000 7f7db359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7db40cb510 0x7f7db414a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.446+0000 7f7db359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.446+0000 7f7db359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db41e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41e7620 con 0x7f7db40cc700 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41e6b50 con 0x7f7db40ce380 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db414e7f0 con 0x7f7db40cb510 2024-05-08T22:42:40.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37404/0 (socket says 172.21.15.179:37404) 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3685734386 learned_addr learned my addr 172.21.15.179:0/3685734386 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db2d9d700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db41e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db259c700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7db40cb510 0x7f7db414a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7db40cb510 msgr2=0x7f7db414a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7db40cb510 0x7f7db414a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 msgr2=0x7f7db41e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db41e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 -- 172.21.15.179:0/3685734386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0094b0 con 0x7f7db40cc700 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7da9d9b700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c000c00 tx=0x7f7d9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0036c0 con 0x7f7db40cc700 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c01e070 con 0x7f7db40cc700 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db41e6d80 con 0x7f7db40cc700 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c003a60 con 0x7f7db40cc700 2024-05-08T22:42:40.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db41e7340 con 0x7f7db40cc700 2024-05-08T22:42:40.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.450+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db4067740 con 0x7f7db40cc700 2024-05-08T22:42:40.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.454+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d9c003bc0 con 0x7f7db40cc700 2024-05-08T22:42:40.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.454+0000 7f7dab7fe700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.454+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d9c085580 con 0x7f7db40cc700 2024-05-08T22:42:40.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.454+0000 7f7db259c700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.454+0000 7f7db259c700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 0x7f7d94068bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7da4000f60 tx=0x7f7da4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.458+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7d9c0527a0 con 0x7f7db40cc700 2024-05-08T22:42:40.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.666+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7db404e700 con 0x7f7db40cc700 2024-05-08T22:42:40.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.666+0000 7f7dab7fe700 1 -- 172.21.15.179:0/3685734386 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f7d9c055dc0 con 0x7f7db40cc700 2024-05-08T22:42:40.670 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:40.670 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:42:40.300797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57425":{"gid":57425,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/4059323811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4059323811},{"type":"v1","addr":"172.21.15.195:6837","nonce":4059323811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 msgr2=0x7f7d94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 0x7f7d94068bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7da4000f60 tx=0x7f7da4009040 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 msgr2=0x7f7db40b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c000c00 tx=0x7f7d9c004770 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 shutdown_connections 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d94066730 0x7f7d94068bf0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7db40ce380 0x7f7db41e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7db40cc700 0x7f7db40b2dc0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 --2- 172.21.15.179:0/3685734386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7db40cb510 0x7f7db414a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 >> 172.21.15.179:0/3685734386 conn(0x7f7db40ce9d0 msgr2=0x7f7db40ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 shutdown_connections 2024-05-08T22:42:40.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.670+0000 7f7db359e700 1 -- 172.21.15.179:0/3685734386 wait complete. 2024-05-08T22:42:40.674 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 75 2024-05-08T22:42:40.685 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:40.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.794+0000 7f0256d31700 1 Processor -- start 2024-05-08T22:42:40.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.794+0000 7f0256d31700 1 -- start start 2024-05-08T22:42:40.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f025006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02500cd200 con 0x7f025006ed50 2024-05-08T22:42:40.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02500cd4e0 con 0x7f025006dc20 2024-05-08T22:42:40.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02500cd7c0 con 0x7f02501434d0 2024-05-08T22:42:40.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256530700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f025006ed50 0x7f02500b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:35990/0 (socket says 172.21.15.179:35990) 2024-05-08T22:42:40.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256530700 1 -- 172.21.15.179:0/2927965145 learned_addr learned my addr 172.21.15.179:0/2927965145 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:40.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0255d2f700 1 --2- 172.21.15.179:0/2927965145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f025006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206467633 0 0) 0x7f02500cd200 con 0x7f025006ed50 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f023c0035d0 con 0x7f025006ed50 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527575581 0 0) 0x7f02500cd4e0 con 0x7f025006dc20 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02500cd200 con 0x7f025006dc20 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0255d2f700 1 -- 172.21.15.179:0/2927965145 >> v1:172.21.15.195:6789/0 conn(0x7f025006dc20 legacy=0x7f025006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0255d2f700 1 -- 172.21.15.179:0/2927965145 >> v1:172.21.15.179:6789/0 conn(0x7f025006ed50 legacy=0x7f02500b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2593550231 0 0) 0x7f023c0035d0 con 0x7f025006ed50 2024-05-08T22:42:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0255d2f700 1 -- 172.21.15.179:0/2927965145 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500cdaa0 con 0x7f02501434d0 2024-05-08T22:42:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0255d2f700 1 --2- 172.21.15.179:0/2927965145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f025006d6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0240009880 tx=0x7f0240009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1482ea32ece940 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480019f0 con 0x7f02501434d0 2024-05-08T22:42:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240004030 con 0x7f02501434d0 2024-05-08T22:42:40.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0254d2d700 1 -- 172.21.15.179:0/2927965145 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400043c0 con 0x7f02501434d0 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 -- 172.21.15.179:0/2927965145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 msgr2=0x7f025006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.798+0000 7f0256d31700 1 --2- 172.21.15.179:0/2927965145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f025006d6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0240009880 tx=0x7f0240009b90 comp rx=0 tx=0).stop 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 -- 172.21.15.179:0/2927965145 shutdown_connections 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 --2- 172.21.15.179:0/2927965145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f025006d6e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 -- 172.21.15.179:0/2927965145 >> 172.21.15.179:0/2927965145 conn(0x7f02500ce9d0 msgr2=0x7f02500cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:40.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 -- 172.21.15.179:0/2927965145 shutdown_connections 2024-05-08T22:42:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 -- 172.21.15.179:0/2927965145 wait complete. 2024-05-08T22:42:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 Processor -- start 2024-05-08T22:42:40.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.802+0000 7f0256d31700 1 -- start start 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f02501e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 0x7f02501e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501e58b0 con 0x7f025006ed50 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501e8010 con 0x7f02501e5420 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02501e82f0 con 0x7f02501434d0 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37408/0 (socket says 172.21.15.179:37408) 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 -- 172.21.15.179:0/3930427606 learned_addr learned my addr 172.21.15.179:0/3930427606 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256530700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 0x7f02501e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f025552e700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f02501e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 -- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 msgr2=0x7f02501e4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f02501e4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 -- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 msgr2=0x7f02501e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 0x7f02501e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 -- 172.21.15.179:0/3930427606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400094b0 con 0x7f025006ed50 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256530700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 0x7f02501e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f025552e700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f02501e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0255d2f700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f02400097d0 tx=0x7f0240004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240003d40 con 0x7f025006ed50 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240011630 con 0x7f025006ed50 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240003ea0 con 0x7f025006ed50 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02501e85d0 con 0x7f025006ed50 2024-05-08T22:42:40.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02501e8b90 con 0x7f025006ed50 2024-05-08T22:42:40.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.806+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0250067740 con 0x7f025006ed50 2024-05-08T22:42:40.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.810+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0240018050 con 0x7f025006ed50 2024-05-08T22:42:40.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.810+0000 7f02467fc700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 0x7f0234068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:40.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.810+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f02400851d0 con 0x7f025006ed50 2024-05-08T22:42:40.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.810+0000 7f025552e700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 0x7f0234068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:40.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.810+0000 7f025552e700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 0x7f0234068c20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02501e24a0 tx=0x7f024c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:40.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.814+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0240052470 con 0x7f025006ed50 2024-05-08T22:42:40.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.986+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02501e8d80 con 0x7f025006ed50 2024-05-08T22:42:40.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.986+0000 7f02467fc700 1 -- 172.21.15.179:0/3930427606 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16296 (secure 0 0 0) 0x7f0240055a90 con 0x7f025006ed50 2024-05-08T22:42:40.990 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:40.990 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":112,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:38.288356+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 msgr2=0x7f0234068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 0x7f0234068c20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02501e24a0 tx=0x7f024c009040 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 msgr2=0x7f0250159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f02400097d0 tx=0x7f0240004770 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 shutdown_connections 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0234066760 0x7f0234068c20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02501e5420 0x7f02501e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02501434d0 0x7f02501e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 --2- 172.21.15.179:0/3930427606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f025006ed50 0x7f0250159670 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 >> 172.21.15.179:0/3930427606 conn(0x7f02500ce9d0 msgr2=0x7f0250141b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 shutdown_connections 2024-05-08T22:42:40.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:40.990+0000 7f0256d31700 1 -- 172.21.15.179:0/3930427606 wait complete. 2024-05-08T22:42:41.006 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:42:41.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:42:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.118+0000 7f0e61285700 1 Processor -- start 2024-05-08T22:42:41.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.118+0000 7f0e61285700 1 -- start start 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.118+0000 7f0e61285700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e5c15ad70 0x7f0e5c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e5c15f550 0x7f0e5c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c06cb00 con 0x7f0e5c0ceac0 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c15d6f0 con 0x7f0e5c15ad70 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c06bda0 con 0x7f0e5c15f550 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e537fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0e5c0ceac0 0x7f0e5c15ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36000/0 (socket says 172.21.15.179:36000) 2024-05-08T22:42:41.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e537fe700 1 -- 172.21.15.179:0/1335424436 learned_addr learned my addr 172.21.15.179:0/1335424436 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 --2- 172.21.15.179:0/1335424436 >> v2:172.21.15.179:3300/0 conn(0x7f0e5c15ad70 0x7f0e5c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e60a84700 1 --2- 172.21.15.179:0/1335424436 >> v2:172.21.15.195:3300/0 conn(0x7f0e5c15f550 0x7f0e5c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902285596 0 0) 0x7f0e5c06cb00 con 0x7f0e5c0ceac0 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e480035d0 con 0x7f0e5c0ceac0 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2775781185 0 0) 0x7f0e480035d0 con 0x7f0e5c0ceac0 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 -- 172.21.15.179:0/1335424436 >> v2:172.21.15.195:3300/0 conn(0x7f0e5c15f550 msgr2=0x7f0e5c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 --2- 172.21.15.179:0/1335424436 >> v2:172.21.15.195:3300/0 conn(0x7f0e5c15f550 0x7f0e5c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 -- 172.21.15.179:0/1335424436 >> v1:172.21.15.179:6789/0 conn(0x7f0e5c0ceac0 legacy=0x7f0e5c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 -- 172.21.15.179:0/1335424436 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c06c1d0 con 0x7f0e5c15ad70 2024-05-08T22:42:41.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5bfff700 1 --2- 172.21.15.179:0/1335424436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c15e900 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009850 tx=0x7f0e4c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1edd106226406 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44001670 con 0x7f0e5c15ad70 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c004030 con 0x7f0e5c15ad70 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e5b7fe700 1 -- 172.21.15.179:0/1335424436 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0043c0 con 0x7f0e5c15ad70 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- 172.21.15.179:0/1335424436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 msgr2=0x7f0e5c15e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 --2- 172.21.15.179:0/1335424436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c15e900 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009850 tx=0x7f0e4c009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- 172.21.15.179:0/1335424436 shutdown_connections 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 --2- 172.21.15.179:0/1335424436 >> v2:172.21.15.195:3300/0 conn(0x7f0e5c15f550 0x7f0e5c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 --2- 172.21.15.179:0/1335424436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c15e900 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- 172.21.15.179:0/1335424436 >> 172.21.15.179:0/1335424436 conn(0x7f0e5c0cfb50 msgr2=0x7f0e5c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:41.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- 172.21.15.179:0/1335424436 shutdown_connections 2024-05-08T22:42:41.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 -- 172.21.15.179:0/1335424436 wait complete. 2024-05-08T22:42:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.122+0000 7f0e61285700 1 Processor -- start 2024-05-08T22:42:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- start start 2024-05-08T22:42:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 0x7f0e5c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 0x7f0e5c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c14ef60 con 0x7f0e5c15ad70 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0c4a30 con 0x7f0e5c15f550 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c151cb0 con 0x7f0e5c0ceac0 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 0x7f0e5c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 0x7f0e5c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 0x7f0e5c144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:41700/0 (socket says 172.21.15.179:41700) 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37426/0 (socket says 172.21.15.179:37426) 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e5bfff700 1 -- 172.21.15.179:0/1497372914 learned_addr learned my addr 172.21.15.179:0/1497372914 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 -- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 msgr2=0x7f0e5c144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 0x7f0e5c144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 -- 172.21.15.179:0/1497372914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 msgr2=0x7f0e5c151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 -- 172.21.15.179:0/1497372914 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0094b0 con 0x7f0e5c15f550 2024-05-08T22:42:41.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e537fe700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e60a84700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 0x7f0e5c159920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400eba0 tx=0x7f0e5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54009a80 con 0x7f0e5c15f550 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5400f040 con 0x7f0e5c15f550 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c0c4c80 con 0x7f0e5c15f550 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400cc40 con 0x7f0e5c15f550 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.126+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c0c5240 con 0x7f0e5c15f550 2024-05-08T22:42:41.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.130+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c04e700 con 0x7f0e5c15f550 2024-05-08T22:42:41.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.130+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e54014480 con 0x7f0e5c15f550 2024-05-08T22:42:41.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.130+0000 7f0e597fa700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 0x7f0e3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.130+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0e54084b20 con 0x7f0e5c15f550 2024-05-08T22:42:41.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.130+0000 7f0e5bfff700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 0x7f0e3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.134+0000 7f0e5bfff700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 0x7f0e3c068bf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009ba0 tx=0x7f0e4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.142+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e54051de0 con 0x7f0e5c15f550 2024-05-08T22:42:41.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.310+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e5c06c790 con 0x7f0e3c066730 2024-05-08T22:42:41.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.326+0000 7f0e597fa700 1 -- 172.21.15.179:0/1497372914 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288265 (secure 0 0 0) 0x7f0e4c0036c0 con 0x7f0e3c066730 2024-05-08T22:42:41.333 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 msgr2=0x7f0e3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 0x7f0e3c068bf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009ba0 tx=0x7f0e4c008030 comp rx=0 tx=0).stop 2024-05-08T22:42:41.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 msgr2=0x7f0e5c159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 0x7f0e5c159920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400eba0 tx=0x7f0e5400eeb0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 shutdown_connections 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0e3c066730 0x7f0e3c068bf0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e5c15f550 0x7f0e5c159920 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e5c15ad70 0x7f0e5c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 --2- 172.21.15.179:0/1497372914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e5c0ceac0 0x7f0e5c144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.330+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 >> 172.21.15.179:0/1497372914 conn(0x7f0e5c0cfb50 msgr2=0x7f0e5c155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.334+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 shutdown_connections 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.334+0000 7f0e61285700 1 -- 172.21.15.179:0/1497372914 wait complete. 2024-05-08T22:42:41.336 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:42:41.349 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-05-08T22:42:39.580276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265000,"kb_used_data":11916,"kb_used_omap":12,"kb_used_meta":252915,"kb_avail":754709720,"statfs":{"total":773094113280,"available":772822753280,"internally_reserved":0,"allocated":12201984,"data_stored":6230521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":258985528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001228"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442557+0000","last_change":"2024-05-08T22:42:37.308637+0000","last_active":"2024-05-08T22:42:38.442557+0000","last_peered":"2024-05-08T22:42:38.442557+0000","last_clean":"2024-05-08T22:42:38.442557+0000","last_became_active":"2024-05-08T22:42:37.308328+0000","last_became_peered":"2024-05-08T22:42:37.308328+0000","last_unstale":"2024-05-08T22:42:38.442557+0000","last_undegraded":"2024-05-08T22:42:38.442557+0000","last_fullsized":"2024-05-08T22:42:38.442557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:51.017187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301805+0000","last_change":"2024-05-08T22:42:37.300831+0000","last_active":"2024-05-08T22:42:38.301805+0000","last_peered":"2024-05-08T22:42:38.301805+0000","last_clean":"2024-05-08T22:42:38.301805+0000","last_became_active":"2024-05-08T22:42:37.300574+0000","last_became_peered":"2024-05-08T22:42:37.300574+0000","last_unstale":"2024-05-08T22:42:38.301805+0000","last_undegraded":"2024-05-08T22:42:38.301805+0000","last_fullsized":"2024-05-08T22:42:38.301805+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:39.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442193+0000","last_change":"2024-05-08T22:42:37.322626+0000","last_active":"2024-05-08T22:42:38.442193+0000","last_peered":"2024-05-08T22:42:38.442193+0000","last_clean":"2024-05-08T22:42:38.442193+0000","last_became_active":"2024-05-08T22:42:37.322444+0000","last_became_peered":"2024-05-08T22:42:37.322444+0000","last_unstale":"2024-05-08T22:42:38.442193+0000","last_undegraded":"2024-05-08T22:42:38.442193+0000","last_fullsized":"2024-05-08T22:42:38.442193+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:45.034279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302043+0000","last_change":"2024-05-08T22:42:37.308018+0000","last_active":"2024-05-08T22:42:38.302043+0000","last_peered":"2024-05-08T22:42:38.302043+0000","last_clean":"2024-05-08T22:42:38.302043+0000","last_became_active":"2024-05-08T22:42:37.307706+0000","last_became_peered":"2024-05-08T22:42:37.307706+0000","last_unstale":"2024-05-08T22:42:38.302043+0000","last_undegraded":"2024-05-08T22:42:38.302043+0000","last_fullsized":"2024-05-08T22:42:38.302043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:01.975004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440656+0000","last_change":"2024-05-08T22:42:37.305140+0000","last_active":"2024-05-08T22:42:38.440656+0000","last_peered":"2024-05-08T22:42:38.440656+0000","last_clean":"2024-05-08T22:42:38.440656+0000","last_became_active":"2024-05-08T22:42:37.304877+0000","last_became_peered":"2024-05-08T22:42:37.304877+0000","last_unstale":"2024-05-08T22:42:38.440656+0000","last_undegraded":"2024-05-08T22:42:38.440656+0000","last_fullsized":"2024-05-08T22:42:38.440656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:20.843199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300721+0000","last_change":"2024-05-08T22:42:37.311688+0000","last_active":"2024-05-08T22:42:38.300721+0000","last_peered":"2024-05-08T22:42:38.300721+0000","last_clean":"2024-05-08T22:42:38.300721+0000","last_became_active":"2024-05-08T22:42:37.311444+0000","last_became_peered":"2024-05-08T22:42:37.311444+0000","last_unstale":"2024-05-08T22:42:38.300721+0000","last_undegraded":"2024-05-08T22:42:38.300721+0000","last_fullsized":"2024-05-08T22:42:38.300721+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:40.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300144+0000","last_change":"2024-05-08T22:42:37.312196+0000","last_active":"2024-05-08T22:42:38.300144+0000","last_peered":"2024-05-08T22:42:38.300144+0000","last_clean":"2024-05-08T22:42:38.300144+0000","last_became_active":"2024-05-08T22:42:37.311997+0000","last_became_peered":"2024-05-08T22:42:37.311997+0000","last_unstale":"2024-05-08T22:42:38.300144+0000","last_undegraded":"2024-05-08T22:42:38.300144+0000","last_fullsized":"2024-05-08T22:42:38.300144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:03.199565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440591+0000","last_change":"2024-05-08T22:42:37.306347+0000","last_active":"2024-05-08T22:42:38.440591+0000","last_peered":"2024-05-08T22:42:38.440591+0000","last_clean":"2024-05-08T22:42:38.440591+0000","last_became_active":"2024-05-08T22:42:37.305982+0000","last_became_peered":"2024-05-08T22:42:37.305982+0000","last_unstale":"2024-05-08T22:42:38.440591+0000","last_undegraded":"2024-05-08T22:42:38.440591+0000","last_fullsized":"2024-05-08T22:42:38.440591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:19.944179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302293+0000","last_change":"2024-05-08T22:42:37.306845+0000","last_active":"2024-05-08T22:42:38.302293+0000","last_peered":"2024-05-08T22:42:38.302293+0000","last_clean":"2024-05-08T22:42:38.302293+0000","last_became_active":"2024-05-08T22:42:37.306551+0000","last_became_peered":"2024-05-08T22:42:37.306551+0000","last_unstale":"2024-05-08T22:42:38.302293+0000","last_undegraded":"2024-05-08T22:42:38.302293+0000","last_fullsized":"2024-05-08T22:42:38.302293+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:54:44.808075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440938+0000","last_change":"2024-05-08T22:42:37.305144+0000","last_active":"2024-05-08T22:42:38.440938+0000","last_peered":"2024-05-08T22:42:38.440938+0000","last_clean":"2024-05-08T22:42:38.440938+0000","last_became_active":"2024-05-08T22:42:37.304900+0000","last_became_peered":"2024-05-08T22:42:37.304900+0000","last_unstale":"2024-05-08T22:42:38.440938+0000","last_undegraded":"2024-05-08T22:42:38.440938+0000","last_fullsized":"2024-05-08T22:42:38.440938+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:50.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301675+0000","last_change":"2024-05-08T22:42:37.317444+0000","last_active":"2024-05-08T22:42:38.301675+0000","last_peered":"2024-05-08T22:42:38.301675+0000","last_clean":"2024-05-08T22:42:38.301675+0000","last_became_active":"2024-05-08T22:42:37.317192+0000","last_became_peered":"2024-05-08T22:42:37.317192+0000","last_unstale":"2024-05-08T22:42:38.301675+0000","last_undegraded":"2024-05-08T22:42:38.301675+0000","last_fullsized":"2024-05-08T22:42:38.301675+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:15.105556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.319962+0000","last_change":"2024-05-08T22:42:37.316083+0000","last_active":"2024-05-08T22:42:37.319962+0000","last_peered":"2024-05-08T22:42:37.319962+0000","last_clean":"2024-05-08T22:42:37.319962+0000","last_became_active":"2024-05-08T22:42:37.315828+0000","last_became_peered":"2024-05-08T22:42:37.315828+0000","last_unstale":"2024-05-08T22:42:37.319962+0000","last_undegraded":"2024-05-08T22:42:37.319962+0000","last_fullsized":"2024-05-08T22:42:37.319962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:38.308703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301387+0000","last_change":"2024-05-08T22:42:37.319590+0000","last_active":"2024-05-08T22:42:38.301387+0000","last_peered":"2024-05-08T22:42:38.301387+0000","last_clean":"2024-05-08T22:42:38.301387+0000","last_became_active":"2024-05-08T22:42:37.319333+0000","last_became_peered":"2024-05-08T22:42:37.319333+0000","last_unstale":"2024-05-08T22:42:38.301387+0000","last_undegraded":"2024-05-08T22:42:38.301387+0000","last_fullsized":"2024-05-08T22:42:38.301387+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:13.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438133+0000","last_change":"2024-05-08T22:42:37.303124+0000","last_active":"2024-05-08T22:42:38.438133+0000","last_peered":"2024-05-08T22:42:38.438133+0000","last_clean":"2024-05-08T22:42:38.438133+0000","last_became_active":"2024-05-08T22:42:37.302857+0000","last_became_peered":"2024-05-08T22:42:37.302857+0000","last_unstale":"2024-05-08T22:42:38.438133+0000","last_undegraded":"2024-05-08T22:42:38.438133+0000","last_fullsized":"2024-05-08T22:42:38.438133+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:37.906276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438067+0000","last_change":"2024-05-08T22:42:37.318212+0000","last_active":"2024-05-08T22:42:38.438067+0000","last_peered":"2024-05-08T22:42:38.438067+0000","last_clean":"2024-05-08T22:42:38.438067+0000","last_became_active":"2024-05-08T22:42:37.317970+0000","last_became_peered":"2024-05-08T22:42:37.317970+0000","last_unstale":"2024-05-08T22:42:38.438067+0000","last_undegraded":"2024-05-08T22:42:38.438067+0000","last_fullsized":"2024-05-08T22:42:38.438067+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:29.864562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438077+0000","last_change":"2024-05-08T22:42:37.295570+0000","last_active":"2024-05-08T22:42:38.438077+0000","last_peered":"2024-05-08T22:42:38.438077+0000","last_clean":"2024-05-08T22:42:38.438077+0000","last_became_active":"2024-05-08T22:42:37.295318+0000","last_became_peered":"2024-05-08T22:42:37.295318+0000","last_unstale":"2024-05-08T22:42:38.438077+0000","last_undegraded":"2024-05-08T22:42:38.438077+0000","last_fullsized":"2024-05-08T22:42:38.438077+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:52.333211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442785+0000","last_change":"2024-05-08T22:42:37.321577+0000","last_active":"2024-05-08T22:42:38.442785+0000","last_peered":"2024-05-08T22:42:38.442785+0000","last_clean":"2024-05-08T22:42:38.442785+0000","last_became_active":"2024-05-08T22:42:37.321319+0000","last_became_peered":"2024-05-08T22:42:37.321319+0000","last_unstale":"2024-05-08T22:42:38.442785+0000","last_undegraded":"2024-05-08T22:42:38.442785+0000","last_fullsized":"2024-05-08T22:42:38.442785+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:53.288401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302075+0000","last_change":"2024-05-08T22:42:37.310885+0000","last_active":"2024-05-08T22:42:38.302075+0000","last_peered":"2024-05-08T22:42:38.302075+0000","last_clean":"2024-05-08T22:42:38.302075+0000","last_became_active":"2024-05-08T22:42:37.310617+0000","last_became_peered":"2024-05-08T22:42:37.310617+0000","last_unstale":"2024-05-08T22:42:38.302075+0000","last_undegraded":"2024-05-08T22:42:38.302075+0000","last_fullsized":"2024-05-08T22:42:38.302075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:47.244030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442331+0000","last_change":"2024-05-08T22:42:37.308217+0000","last_active":"2024-05-08T22:42:38.442331+0000","last_peered":"2024-05-08T22:42:38.442331+0000","last_clean":"2024-05-08T22:42:38.442331+0000","last_became_active":"2024-05-08T22:42:37.307091+0000","last_became_peered":"2024-05-08T22:42:37.307091+0000","last_unstale":"2024-05-08T22:42:38.442331+0000","last_undegraded":"2024-05-08T22:42:38.442331+0000","last_fullsized":"2024-05-08T22:42:38.442331+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:59.154570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.301072+0000","last_change":"2024-05-08T22:42:37.298817+0000","last_active":"2024-05-08T22:42:37.301072+0000","last_peered":"2024-05-08T22:42:37.301072+0000","last_clean":"2024-05-08T22:42:37.301072+0000","last_became_active":"2024-05-08T22:42:37.298566+0000","last_became_peered":"2024-05-08T22:42:37.298566+0000","last_unstale":"2024-05-08T22:42:37.301072+0000","last_undegraded":"2024-05-08T22:42:37.301072+0000","last_fullsized":"2024-05-08T22:42:37.301072+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:10.363342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442780+0000","last_change":"2024-05-08T22:42:37.324175+0000","last_active":"2024-05-08T22:42:38.442780+0000","last_peered":"2024-05-08T22:42:38.442780+0000","last_clean":"2024-05-08T22:42:38.442780+0000","last_became_active":"2024-05-08T22:42:37.323899+0000","last_became_peered":"2024-05-08T22:42:37.323899+0000","last_unstale":"2024-05-08T22:42:38.442780+0000","last_undegraded":"2024-05-08T22:42:38.442780+0000","last_fullsized":"2024-05-08T22:42:38.442780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:19.934128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440203+0000","last_change":"2024-05-08T22:42:37.319201+0000","last_active":"2024-05-08T22:42:38.440203+0000","last_peered":"2024-05-08T22:42:38.440203+0000","last_clean":"2024-05-08T22:42:38.440203+0000","last_became_active":"2024-05-08T22:42:37.318913+0000","last_became_peered":"2024-05-08T22:42:37.318913+0000","last_unstale":"2024-05-08T22:42:38.440203+0000","last_undegraded":"2024-05-08T22:42:38.440203+0000","last_fullsized":"2024-05-08T22:42:38.440203+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:51.098236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442302+0000","last_change":"2024-05-08T22:42:37.319606+0000","last_active":"2024-05-08T22:42:38.442302+0000","last_peered":"2024-05-08T22:42:38.442302+0000","last_clean":"2024-05-08T22:42:38.442302+0000","last_became_active":"2024-05-08T22:42:37.319365+0000","last_became_peered":"2024-05-08T22:42:37.319365+0000","last_unstale":"2024-05-08T22:42:38.442302+0000","last_undegraded":"2024-05-08T22:42:38.442302+0000","last_fullsized":"2024-05-08T22:42:38.442302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:33.045019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440050+0000","last_change":"2024-05-08T22:42:37.305749+0000","last_active":"2024-05-08T22:42:38.440050+0000","last_peered":"2024-05-08T22:42:38.440050+0000","last_clean":"2024-05-08T22:42:38.440050+0000","last_became_active":"2024-05-08T22:42:37.305548+0000","last_became_peered":"2024-05-08T22:42:37.305548+0000","last_unstale":"2024-05-08T22:42:38.440050+0000","last_undegraded":"2024-05-08T22:42:38.440050+0000","last_fullsized":"2024-05-08T22:42:38.440050+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:36:47.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.320816+0000","last_change":"2024-05-08T22:42:37.319936+0000","last_active":"2024-05-08T22:42:37.320816+0000","last_peered":"2024-05-08T22:42:37.320816+0000","last_clean":"2024-05-08T22:42:37.320816+0000","last_became_active":"2024-05-08T22:42:37.319670+0000","last_became_peered":"2024-05-08T22:42:37.319670+0000","last_unstale":"2024-05-08T22:42:37.320816+0000","last_undegraded":"2024-05-08T22:42:37.320816+0000","last_fullsized":"2024-05-08T22:42:37.320816+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:55.794126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.301799+0000","last_change":"2024-05-08T22:42:37.299783+0000","last_active":"2024-05-08T22:42:37.301799+0000","last_peered":"2024-05-08T22:42:37.301799+0000","last_clean":"2024-05-08T22:42:37.301799+0000","last_became_active":"2024-05-08T22:42:37.299512+0000","last_became_peered":"2024-05-08T22:42:37.299512+0000","last_unstale":"2024-05-08T22:42:37.301799+0000","last_undegraded":"2024-05-08T22:42:37.301799+0000","last_fullsized":"2024-05-08T22:42:37.301799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:07.848763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301862+0000","last_change":"2024-05-08T22:42:37.316847+0000","last_active":"2024-05-08T22:42:38.301862+0000","last_peered":"2024-05-08T22:42:38.301862+0000","last_clean":"2024-05-08T22:42:38.301862+0000","last_became_active":"2024-05-08T22:42:37.316596+0000","last_became_peered":"2024-05-08T22:42:37.316596+0000","last_unstale":"2024-05-08T22:42:38.301862+0000","last_undegraded":"2024-05-08T22:42:38.301862+0000","last_fullsized":"2024-05-08T22:42:38.301862+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:19.944179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301026+0000","last_change":"2024-05-08T22:42:37.321772+0000","last_active":"2024-05-08T22:42:38.301026+0000","last_peered":"2024-05-08T22:42:38.301026+0000","last_clean":"2024-05-08T22:42:38.301026+0000","last_became_active":"2024-05-08T22:42:37.321454+0000","last_became_peered":"2024-05-08T22:42:37.321454+0000","last_unstale":"2024-05-08T22:42:38.301026+0000","last_undegraded":"2024-05-08T22:42:38.301026+0000","last_fullsized":"2024-05-08T22:42:38.301026+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:23.926135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442981+0000","last_change":"2024-05-08T22:42:37.301667+0000","last_active":"2024-05-08T22:42:38.442981+0000","last_peered":"2024-05-08T22:42:38.442981+0000","last_clean":"2024-05-08T22:42:38.442981+0000","last_became_active":"2024-05-08T22:42:37.301376+0000","last_became_peered":"2024-05-08T22:42:37.301376+0000","last_unstale":"2024-05-08T22:42:38.442981+0000","last_undegraded":"2024-05-08T22:42:38.442981+0000","last_fullsized":"2024-05-08T22:42:38.442981+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:20.885608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.320768+0000","last_change":"2024-05-08T22:42:37.319861+0000","last_active":"2024-05-08T22:42:37.320768+0000","last_peered":"2024-05-08T22:42:37.320768+0000","last_clean":"2024-05-08T22:42:37.320768+0000","last_became_active":"2024-05-08T22:42:37.319598+0000","last_became_peered":"2024-05-08T22:42:37.319598+0000","last_unstale":"2024-05-08T22:42:37.320768+0000","last_undegraded":"2024-05-08T22:42:37.320768+0000","last_fullsized":"2024-05-08T22:42:37.320768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:08.421748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300464+0000","last_change":"2024-05-08T22:42:37.306840+0000","last_active":"2024-05-08T22:42:38.300464+0000","last_peered":"2024-05-08T22:42:38.300464+0000","last_clean":"2024-05-08T22:42:38.300464+0000","last_became_active":"2024-05-08T22:42:37.306586+0000","last_became_peered":"2024-05-08T22:42:37.306586+0000","last_unstale":"2024-05-08T22:42:38.300464+0000","last_undegraded":"2024-05-08T22:42:38.300464+0000","last_fullsized":"2024-05-08T22:42:38.300464+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:23.667992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440564+0000","last_change":"2024-05-08T22:42:37.302534+0000","last_active":"2024-05-08T22:42:38.440564+0000","last_peered":"2024-05-08T22:42:38.440564+0000","last_clean":"2024-05-08T22:42:38.440564+0000","last_became_active":"2024-05-08T22:42:37.302278+0000","last_became_peered":"2024-05-08T22:42:37.302278+0000","last_unstale":"2024-05-08T22:42:38.440564+0000","last_undegraded":"2024-05-08T22:42:38.440564+0000","last_fullsized":"2024-05-08T22:42:38.440564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:27.353883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441780+0000","last_change":"2024-05-08T22:42:37.306240+0000","last_active":"2024-05-08T22:42:38.441780+0000","last_peered":"2024-05-08T22:42:38.441780+0000","last_clean":"2024-05-08T22:42:38.441780+0000","last_became_active":"2024-05-08T22:42:37.305844+0000","last_became_peered":"2024-05-08T22:42:37.305844+0000","last_unstale":"2024-05-08T22:42:38.441780+0000","last_undegraded":"2024-05-08T22:42:38.441780+0000","last_fullsized":"2024-05-08T22:42:38.441780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.717118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"111'4","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301848+0000","last_change":"2024-05-08T22:42:36.923772+0000","last_active":"2024-05-08T22:42:38.301848+0000","last_peered":"2024-05-08T22:42:38.301848+0000","last_clean":"2024-05-08T22:42:38.301848+0000","last_became_active":"2024-05-08T22:42:36.923484+0000","last_became_peered":"2024-05-08T22:42:36.923484+0000","last_unstale":"2024-05-08T22:42:38.301848+0000","last_undegraded":"2024-05-08T22:42:38.301848+0000","last_fullsized":"2024-05-08T22:42:38.301848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:39.419724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440933+0000","last_change":"2024-05-08T22:42:37.303085+0000","last_active":"2024-05-08T22:42:38.440933+0000","last_peered":"2024-05-08T22:42:38.440933+0000","last_clean":"2024-05-08T22:42:38.440933+0000","last_became_active":"2024-05-08T22:42:37.302851+0000","last_became_peered":"2024-05-08T22:42:37.302851+0000","last_unstale":"2024-05-08T22:42:38.440933+0000","last_undegraded":"2024-05-08T22:42:38.440933+0000","last_fullsized":"2024-05-08T22:42:38.440933+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:24.832903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443254+0000","last_change":"2024-05-08T22:42:36.917949+0000","last_active":"2024-05-08T22:42:38.443254+0000","last_peered":"2024-05-08T22:42:38.443254+0000","last_clean":"2024-05-08T22:42:38.443254+0000","last_became_active":"2024-05-08T22:42:36.916543+0000","last_became_peered":"2024-05-08T22:42:36.916543+0000","last_unstale":"2024-05-08T22:42:38.443254+0000","last_undegraded":"2024-05-08T22:42:38.443254+0000","last_fullsized":"2024-05-08T22:42:38.443254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:27.220481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.305487+0000","last_change":"2024-05-08T22:42:37.303596+0000","last_active":"2024-05-08T22:42:37.305487+0000","last_peered":"2024-05-08T22:42:37.305487+0000","last_clean":"2024-05-08T22:42:37.305487+0000","last_became_active":"2024-05-08T22:42:37.303287+0000","last_became_peered":"2024-05-08T22:42:37.303287+0000","last_unstale":"2024-05-08T22:42:37.305487+0000","last_undegraded":"2024-05-08T22:42:37.305487+0000","last_fullsized":"2024-05-08T22:42:37.305487+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:12.709971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.323291+0000","last_change":"2024-05-08T22:42:36.920937+0000","last_active":"2024-05-08T22:42:37.323291+0000","last_peered":"2024-05-08T22:42:37.323291+0000","last_clean":"2024-05-08T22:42:37.323291+0000","last_became_active":"2024-05-08T22:42:36.920689+0000","last_became_peered":"2024-05-08T22:42:36.920689+0000","last_unstale":"2024-05-08T22:42:37.323291+0000","last_undegraded":"2024-05-08T22:42:37.323291+0000","last_fullsized":"2024-05-08T22:42:37.323291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:24.067098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301769+0000","last_change":"2024-05-08T22:42:37.321531+0000","last_active":"2024-05-08T22:42:38.301769+0000","last_peered":"2024-05-08T22:42:38.301769+0000","last_clean":"2024-05-08T22:42:38.301769+0000","last_became_active":"2024-05-08T22:42:37.321268+0000","last_became_peered":"2024-05-08T22:42:37.321268+0000","last_unstale":"2024-05-08T22:42:38.301769+0000","last_undegraded":"2024-05-08T22:42:38.301769+0000","last_fullsized":"2024-05-08T22:42:38.301769+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:23.081244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441877+0000","last_change":"2024-05-08T22:42:36.920050+0000","last_active":"2024-05-08T22:42:38.441877+0000","last_peered":"2024-05-08T22:42:38.441877+0000","last_clean":"2024-05-08T22:42:38.441877+0000","last_became_active":"2024-05-08T22:42:36.919800+0000","last_became_peered":"2024-05-08T22:42:36.919800+0000","last_unstale":"2024-05-08T22:42:38.441877+0000","last_undegraded":"2024-05-08T22:42:38.441877+0000","last_fullsized":"2024-05-08T22:42:38.441877+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:49.495530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301777+0000","last_change":"2024-05-08T22:42:37.305059+0000","last_active":"2024-05-08T22:42:38.301777+0000","last_peered":"2024-05-08T22:42:38.301777+0000","last_clean":"2024-05-08T22:42:38.301777+0000","last_became_active":"2024-05-08T22:42:37.304829+0000","last_became_peered":"2024-05-08T22:42:37.304829+0000","last_unstale":"2024-05-08T22:42:38.301777+0000","last_undegraded":"2024-05-08T22:42:38.301777+0000","last_fullsized":"2024-05-08T22:42:38.301777+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:10.536786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438345+0000","last_change":"2024-05-08T22:42:36.913789+0000","last_active":"2024-05-08T22:42:38.438345+0000","last_peered":"2024-05-08T22:42:38.438345+0000","last_clean":"2024-05-08T22:42:38.438345+0000","last_became_active":"2024-05-08T22:42:36.913524+0000","last_became_peered":"2024-05-08T22:42:36.913524+0000","last_unstale":"2024-05-08T22:42:38.438345+0000","last_undegraded":"2024-05-08T22:42:38.438345+0000","last_fullsized":"2024-05-08T22:42:38.438345+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:32.689897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442589+0000","last_change":"2024-05-08T22:42:37.306352+0000","last_active":"2024-05-08T22:42:38.442589+0000","last_peered":"2024-05-08T22:42:38.442589+0000","last_clean":"2024-05-08T22:42:38.442589+0000","last_became_active":"2024-05-08T22:42:37.306128+0000","last_became_peered":"2024-05-08T22:42:37.306128+0000","last_unstale":"2024-05-08T22:42:38.442589+0000","last_undegraded":"2024-05-08T22:42:38.442589+0000","last_fullsized":"2024-05-08T22:42:38.442589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:52.035922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301393+0000","last_change":"2024-05-08T22:42:36.919712+0000","last_active":"2024-05-08T22:42:38.301393+0000","last_peered":"2024-05-08T22:42:38.301393+0000","last_clean":"2024-05-08T22:42:38.301393+0000","last_became_active":"2024-05-08T22:42:36.919460+0000","last_became_peered":"2024-05-08T22:42:36.919460+0000","last_unstale":"2024-05-08T22:42:38.301393+0000","last_undegraded":"2024-05-08T22:42:38.301393+0000","last_fullsized":"2024-05-08T22:42:38.301393+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:03.770010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302165+0000","last_change":"2024-05-08T22:42:37.318193+0000","last_active":"2024-05-08T22:42:38.302165+0000","last_peered":"2024-05-08T22:42:38.302165+0000","last_clean":"2024-05-08T22:42:38.302165+0000","last_became_active":"2024-05-08T22:42:37.317941+0000","last_became_peered":"2024-05-08T22:42:37.317941+0000","last_unstale":"2024-05-08T22:42:38.302165+0000","last_undegraded":"2024-05-08T22:42:38.302165+0000","last_fullsized":"2024-05-08T22:42:38.302165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:20.843199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.293361+0000","last_change":"2024-05-08T22:42:36.920433+0000","last_active":"2024-05-08T22:42:37.293361+0000","last_peered":"2024-05-08T22:42:37.293361+0000","last_clean":"2024-05-08T22:42:37.293361+0000","last_became_active":"2024-05-08T22:42:36.920039+0000","last_became_peered":"2024-05-08T22:42:36.920039+0000","last_unstale":"2024-05-08T22:42:37.293361+0000","last_undegraded":"2024-05-08T22:42:37.293361+0000","last_fullsized":"2024-05-08T22:42:37.293361+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:07.749196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301749+0000","last_change":"2024-05-08T22:42:37.307746+0000","last_active":"2024-05-08T22:42:38.301749+0000","last_peered":"2024-05-08T22:42:38.301749+0000","last_clean":"2024-05-08T22:42:38.301749+0000","last_became_active":"2024-05-08T22:42:37.307450+0000","last_became_peered":"2024-05-08T22:42:37.307450+0000","last_unstale":"2024-05-08T22:42:38.301749+0000","last_undegraded":"2024-05-08T22:42:38.301749+0000","last_fullsized":"2024-05-08T22:42:38.301749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:37.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301244+0000","last_change":"2024-05-08T22:42:36.923051+0000","last_active":"2024-05-08T22:42:38.301244+0000","last_peered":"2024-05-08T22:42:38.301244+0000","last_clean":"2024-05-08T22:42:38.301244+0000","last_became_active":"2024-05-08T22:42:36.922784+0000","last_became_peered":"2024-05-08T22:42:36.922784+0000","last_unstale":"2024-05-08T22:42:38.301244+0000","last_undegraded":"2024-05-08T22:42:38.301244+0000","last_fullsized":"2024-05-08T22:42:38.301244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:46.232341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441661+0000","last_change":"2024-05-08T22:42:37.306080+0000","last_active":"2024-05-08T22:42:38.441661+0000","last_peered":"2024-05-08T22:42:38.441661+0000","last_clean":"2024-05-08T22:42:38.441661+0000","last_became_active":"2024-05-08T22:42:37.305510+0000","last_became_peered":"2024-05-08T22:42:37.305510+0000","last_unstale":"2024-05-08T22:42:38.441661+0000","last_undegraded":"2024-05-08T22:42:38.441661+0000","last_fullsized":"2024-05-08T22:42:38.441661+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:32.630371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.295220+0000","last_change":"2024-05-08T22:42:36.920413+0000","last_active":"2024-05-08T22:42:37.295220+0000","last_peered":"2024-05-08T22:42:37.295220+0000","last_clean":"2024-05-08T22:42:37.295220+0000","last_became_active":"2024-05-08T22:42:36.920139+0000","last_became_peered":"2024-05-08T22:42:36.920139+0000","last_unstale":"2024-05-08T22:42:37.295220+0000","last_undegraded":"2024-05-08T22:42:37.295220+0000","last_fullsized":"2024-05-08T22:42:37.295220+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:19.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441645+0000","last_change":"2024-05-08T22:42:37.306778+0000","last_active":"2024-05-08T22:42:38.441645+0000","last_peered":"2024-05-08T22:42:38.441645+0000","last_clean":"2024-05-08T22:42:38.441645+0000","last_became_active":"2024-05-08T22:42:37.306488+0000","last_became_peered":"2024-05-08T22:42:37.306488+0000","last_unstale":"2024-05-08T22:42:38.441645+0000","last_undegraded":"2024-05-08T22:42:38.441645+0000","last_fullsized":"2024-05-08T22:42:38.441645+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:13.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442069+0000","last_change":"2024-05-08T22:42:36.917913+0000","last_active":"2024-05-08T22:42:38.442069+0000","last_peered":"2024-05-08T22:42:38.442069+0000","last_clean":"2024-05-08T22:42:38.442069+0000","last_became_active":"2024-05-08T22:42:36.917643+0000","last_became_peered":"2024-05-08T22:42:36.917643+0000","last_unstale":"2024-05-08T22:42:38.442069+0000","last_undegraded":"2024-05-08T22:42:38.442069+0000","last_fullsized":"2024-05-08T22:42:38.442069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:11.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440424+0000","last_change":"2024-05-08T22:42:37.302730+0000","last_active":"2024-05-08T22:42:38.440424+0000","last_peered":"2024-05-08T22:42:38.440424+0000","last_clean":"2024-05-08T22:42:38.440424+0000","last_became_active":"2024-05-08T22:42:37.302490+0000","last_became_peered":"2024-05-08T22:42:37.302490+0000","last_unstale":"2024-05-08T22:42:38.440424+0000","last_undegraded":"2024-05-08T22:42:38.440424+0000","last_fullsized":"2024-05-08T22:42:38.440424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:48.137194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"111'3","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438469+0000","last_change":"2024-05-08T22:42:36.921374+0000","last_active":"2024-05-08T22:42:38.438469+0000","last_peered":"2024-05-08T22:42:38.438469+0000","last_clean":"2024-05-08T22:42:38.438469+0000","last_became_active":"2024-05-08T22:42:36.921127+0000","last_became_peered":"2024-05-08T22:42:36.921127+0000","last_unstale":"2024-05-08T22:42:38.438469+0000","last_undegraded":"2024-05-08T22:42:38.438469+0000","last_fullsized":"2024-05-08T22:42:38.438469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:19.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440997+0000","last_change":"2024-05-08T22:42:37.306910+0000","last_active":"2024-05-08T22:42:38.440997+0000","last_peered":"2024-05-08T22:42:38.440997+0000","last_clean":"2024-05-08T22:42:38.440997+0000","last_became_active":"2024-05-08T22:42:37.306611+0000","last_became_peered":"2024-05-08T22:42:37.306611+0000","last_unstale":"2024-05-08T22:42:38.440997+0000","last_undegraded":"2024-05-08T22:42:38.440997+0000","last_fullsized":"2024-05-08T22:42:38.440997+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:23.926135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"111'6","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300993+0000","last_change":"2024-05-08T22:42:36.919423+0000","last_active":"2024-05-08T22:42:38.300993+0000","last_peered":"2024-05-08T22:42:38.300993+0000","last_clean":"2024-05-08T22:42:38.300993+0000","last_became_active":"2024-05-08T22:42:36.919173+0000","last_became_peered":"2024-05-08T22:42:36.919173+0000","last_unstale":"2024-05-08T22:42:38.300993+0000","last_undegraded":"2024-05-08T22:42:38.300993+0000","last_fullsized":"2024-05-08T22:42:38.300993+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:56.453730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301773+0000","last_change":"2024-05-08T22:42:37.315703+0000","last_active":"2024-05-08T22:42:38.301773+0000","last_peered":"2024-05-08T22:42:38.301773+0000","last_clean":"2024-05-08T22:42:38.301773+0000","last_became_active":"2024-05-08T22:42:37.315477+0000","last_became_peered":"2024-05-08T22:42:37.315477+0000","last_unstale":"2024-05-08T22:42:38.301773+0000","last_undegraded":"2024-05-08T22:42:38.301773+0000","last_fullsized":"2024-05-08T22:42:38.301773+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:11.288809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441108+0000","last_change":"2024-05-08T22:42:36.925185+0000","last_active":"2024-05-08T22:42:38.441108+0000","last_peered":"2024-05-08T22:42:38.441108+0000","last_clean":"2024-05-08T22:42:38.441108+0000","last_became_active":"2024-05-08T22:42:36.924775+0000","last_became_peered":"2024-05-08T22:42:36.924775+0000","last_unstale":"2024-05-08T22:42:38.441108+0000","last_undegraded":"2024-05-08T22:42:38.441108+0000","last_fullsized":"2024-05-08T22:42:38.441108+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.748812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301463+0000","last_change":"2024-05-08T22:42:37.304551+0000","last_active":"2024-05-08T22:42:38.301463+0000","last_peered":"2024-05-08T22:42:38.301463+0000","last_clean":"2024-05-08T22:42:38.301463+0000","last_became_active":"2024-05-08T22:42:37.304254+0000","last_became_peered":"2024-05-08T22:42:37.304254+0000","last_unstale":"2024-05-08T22:42:38.301463+0000","last_undegraded":"2024-05-08T22:42:38.301463+0000","last_fullsized":"2024-05-08T22:42:38.301463+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:50.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301462+0000","last_change":"2024-05-08T22:42:36.920214+0000","last_active":"2024-05-08T22:42:38.301462+0000","last_peered":"2024-05-08T22:42:38.301462+0000","last_clean":"2024-05-08T22:42:38.301462+0000","last_became_active":"2024-05-08T22:42:36.919313+0000","last_became_peered":"2024-05-08T22:42:36.919313+0000","last_unstale":"2024-05-08T22:42:38.301462+0000","last_undegraded":"2024-05-08T22:42:38.301462+0000","last_fullsized":"2024-05-08T22:42:38.301462+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:22.914446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442962+0000","last_change":"2024-05-08T22:42:37.305012+0000","last_active":"2024-05-08T22:42:38.442962+0000","last_peered":"2024-05-08T22:42:38.442962+0000","last_clean":"2024-05-08T22:42:38.442962+0000","last_became_active":"2024-05-08T22:42:37.304775+0000","last_became_peered":"2024-05-08T22:42:37.304775+0000","last_unstale":"2024-05-08T22:42:38.442962+0000","last_undegraded":"2024-05-08T22:42:38.442962+0000","last_fullsized":"2024-05-08T22:42:38.442962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:22.670395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440931+0000","last_change":"2024-05-08T22:42:36.920533+0000","last_active":"2024-05-08T22:42:38.440931+0000","last_peered":"2024-05-08T22:42:38.440931+0000","last_clean":"2024-05-08T22:42:38.440931+0000","last_became_active":"2024-05-08T22:42:36.919788+0000","last_became_peered":"2024-05-08T22:42:36.919788+0000","last_unstale":"2024-05-08T22:42:38.440931+0000","last_undegraded":"2024-05-08T22:42:38.440931+0000","last_fullsized":"2024-05-08T22:42:38.440931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:14.080331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442839+0000","last_change":"2024-05-08T22:42:37.321587+0000","last_active":"2024-05-08T22:42:38.442839+0000","last_peered":"2024-05-08T22:42:38.442839+0000","last_clean":"2024-05-08T22:42:38.442839+0000","last_became_active":"2024-05-08T22:42:37.321339+0000","last_became_peered":"2024-05-08T22:42:37.321339+0000","last_unstale":"2024-05-08T22:42:38.442839+0000","last_undegraded":"2024-05-08T22:42:38.442839+0000","last_fullsized":"2024-05-08T22:42:38.442839+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:00.311379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"111'8","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.323853+0000","last_change":"2024-05-08T22:42:36.920571+0000","last_active":"2024-05-08T22:42:37.323853+0000","last_peered":"2024-05-08T22:42:37.323853+0000","last_clean":"2024-05-08T22:42:37.323853+0000","last_became_active":"2024-05-08T22:42:36.920077+0000","last_became_peered":"2024-05-08T22:42:36.920077+0000","last_unstale":"2024-05-08T22:42:37.323853+0000","last_undegraded":"2024-05-08T22:42:37.323853+0000","last_fullsized":"2024-05-08T22:42:37.323853+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:08.065165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440674+0000","last_change":"2024-05-08T22:42:37.306195+0000","last_active":"2024-05-08T22:42:38.440674+0000","last_peered":"2024-05-08T22:42:38.440674+0000","last_clean":"2024-05-08T22:42:38.440674+0000","last_became_active":"2024-05-08T22:42:37.305643+0000","last_became_peered":"2024-05-08T22:42:37.305643+0000","last_unstale":"2024-05-08T22:42:38.440674+0000","last_undegraded":"2024-05-08T22:42:38.440674+0000","last_fullsized":"2024-05-08T22:42:38.440674+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:30.097060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302376+0000","last_change":"2024-05-08T22:42:36.918745+0000","last_active":"2024-05-08T22:42:38.302376+0000","last_peered":"2024-05-08T22:42:38.302376+0000","last_clean":"2024-05-08T22:42:38.302376+0000","last_became_active":"2024-05-08T22:42:36.916977+0000","last_became_peered":"2024-05-08T22:42:36.916977+0000","last_unstale":"2024-05-08T22:42:38.302376+0000","last_undegraded":"2024-05-08T22:42:38.302376+0000","last_fullsized":"2024-05-08T22:42:38.302376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:28.658528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301076+0000","last_change":"2024-05-08T22:42:37.307413+0000","last_active":"2024-05-08T22:42:38.301076+0000","last_peered":"2024-05-08T22:42:38.301076+0000","last_clean":"2024-05-08T22:42:38.301076+0000","last_became_active":"2024-05-08T22:42:37.307145+0000","last_became_peered":"2024-05-08T22:42:37.307145+0000","last_unstale":"2024-05-08T22:42:38.301076+0000","last_undegraded":"2024-05-08T22:42:38.301076+0000","last_fullsized":"2024-05-08T22:42:38.301076+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:13.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438647+0000","last_change":"2024-05-08T22:42:36.914725+0000","last_active":"2024-05-08T22:42:38.438647+0000","last_peered":"2024-05-08T22:42:38.438647+0000","last_clean":"2024-05-08T22:42:38.438647+0000","last_became_active":"2024-05-08T22:42:36.914461+0000","last_became_peered":"2024-05-08T22:42:36.914461+0000","last_unstale":"2024-05-08T22:42:38.438647+0000","last_undegraded":"2024-05-08T22:42:38.438647+0000","last_fullsized":"2024-05-08T22:42:38.438647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:24.067098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440820+0000","last_change":"2024-05-08T22:42:37.304044+0000","last_active":"2024-05-08T22:42:38.440820+0000","last_peered":"2024-05-08T22:42:38.440820+0000","last_clean":"2024-05-08T22:42:38.440820+0000","last_became_active":"2024-05-08T22:42:37.303790+0000","last_became_peered":"2024-05-08T22:42:37.303790+0000","last_unstale":"2024-05-08T22:42:38.440820+0000","last_undegraded":"2024-05-08T22:42:38.440820+0000","last_fullsized":"2024-05-08T22:42:38.440820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:39.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301910+0000","last_change":"2024-05-08T22:42:36.923320+0000","last_active":"2024-05-08T22:42:38.301910+0000","last_peered":"2024-05-08T22:42:38.301910+0000","last_clean":"2024-05-08T22:42:38.301910+0000","last_became_active":"2024-05-08T22:42:36.923066+0000","last_became_peered":"2024-05-08T22:42:36.923066+0000","last_unstale":"2024-05-08T22:42:38.301910+0000","last_undegraded":"2024-05-08T22:42:38.301910+0000","last_fullsized":"2024-05-08T22:42:38.301910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:17.375368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441966+0000","last_change":"2024-05-08T22:42:37.309627+0000","last_active":"2024-05-08T22:42:38.441966+0000","last_peered":"2024-05-08T22:42:38.441966+0000","last_clean":"2024-05-08T22:42:38.441966+0000","last_became_active":"2024-05-08T22:42:37.309359+0000","last_became_peered":"2024-05-08T22:42:37.309359+0000","last_unstale":"2024-05-08T22:42:38.441966+0000","last_undegraded":"2024-05-08T22:42:38.441966+0000","last_fullsized":"2024-05-08T22:42:38.441966+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:10.400426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442947+0000","last_change":"2024-05-08T22:42:36.923085+0000","last_active":"2024-05-08T22:42:38.442947+0000","last_peered":"2024-05-08T22:42:38.442947+0000","last_clean":"2024-05-08T22:42:38.442947+0000","last_became_active":"2024-05-08T22:42:36.922799+0000","last_became_peered":"2024-05-08T22:42:36.922799+0000","last_unstale":"2024-05-08T22:42:38.442947+0000","last_undegraded":"2024-05-08T22:42:38.442947+0000","last_fullsized":"2024-05-08T22:42:38.442947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:34.124192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442457+0000","last_change":"2024-05-08T22:42:37.304399+0000","last_active":"2024-05-08T22:42:38.442457+0000","last_peered":"2024-05-08T22:42:38.442457+0000","last_clean":"2024-05-08T22:42:38.442457+0000","last_became_active":"2024-05-08T22:42:37.304154+0000","last_became_peered":"2024-05-08T22:42:37.304154+0000","last_unstale":"2024-05-08T22:42:38.442457+0000","last_undegraded":"2024-05-08T22:42:38.442457+0000","last_fullsized":"2024-05-08T22:42:38.442457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:00.543896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440255+0000","last_change":"2024-05-08T22:42:36.920391+0000","last_active":"2024-05-08T22:42:38.440255+0000","last_peered":"2024-05-08T22:42:38.440255+0000","last_clean":"2024-05-08T22:42:38.440255+0000","last_became_active":"2024-05-08T22:42:36.919986+0000","last_became_peered":"2024-05-08T22:42:36.919986+0000","last_unstale":"2024-05-08T22:42:38.440255+0000","last_undegraded":"2024-05-08T22:42:38.440255+0000","last_fullsized":"2024-05-08T22:42:38.440255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:48.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301845+0000","last_change":"2024-05-08T22:42:37.304078+0000","last_active":"2024-05-08T22:42:38.301845+0000","last_peered":"2024-05-08T22:42:38.301845+0000","last_clean":"2024-05-08T22:42:38.301845+0000","last_became_active":"2024-05-08T22:42:37.303811+0000","last_became_peered":"2024-05-08T22:42:37.303811+0000","last_unstale":"2024-05-08T22:42:38.301845+0000","last_undegraded":"2024-05-08T22:42:38.301845+0000","last_fullsized":"2024-05-08T22:42:38.301845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.854440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300680+0000","last_change":"2024-05-08T22:42:36.921185+0000","last_active":"2024-05-08T22:42:38.300680+0000","last_peered":"2024-05-08T22:42:38.300680+0000","last_clean":"2024-05-08T22:42:38.300680+0000","last_became_active":"2024-05-08T22:42:36.920918+0000","last_became_peered":"2024-05-08T22:42:36.920918+0000","last_unstale":"2024-05-08T22:42:38.300680+0000","last_undegraded":"2024-05-08T22:42:38.300680+0000","last_fullsized":"2024-05-08T22:42:38.300680+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.748812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443426+0000","last_change":"2024-05-08T22:42:37.305300+0000","last_active":"2024-05-08T22:42:38.443426+0000","last_peered":"2024-05-08T22:42:38.443426+0000","last_clean":"2024-05-08T22:42:38.443426+0000","last_became_active":"2024-05-08T22:42:37.305059+0000","last_became_peered":"2024-05-08T22:42:37.305059+0000","last_unstale":"2024-05-08T22:42:38.443426+0000","last_undegraded":"2024-05-08T22:42:38.443426+0000","last_fullsized":"2024-05-08T22:42:38.443426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:43.420046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440870+0000","last_change":"2024-05-08T22:42:36.919970+0000","last_active":"2024-05-08T22:42:38.440870+0000","last_peered":"2024-05-08T22:42:38.440870+0000","last_clean":"2024-05-08T22:42:38.440870+0000","last_became_active":"2024-05-08T22:42:36.919666+0000","last_became_peered":"2024-05-08T22:42:36.919666+0000","last_unstale":"2024-05-08T22:42:38.440870+0000","last_undegraded":"2024-05-08T22:42:38.440870+0000","last_fullsized":"2024-05-08T22:42:38.440870+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:12.546724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441602+0000","last_change":"2024-05-08T22:42:37.309393+0000","last_active":"2024-05-08T22:42:38.441602+0000","last_peered":"2024-05-08T22:42:38.441602+0000","last_clean":"2024-05-08T22:42:38.441602+0000","last_became_active":"2024-05-08T22:42:37.306964+0000","last_became_peered":"2024-05-08T22:42:37.306964+0000","last_unstale":"2024-05-08T22:42:38.441602+0000","last_undegraded":"2024-05-08T22:42:38.441602+0000","last_fullsized":"2024-05-08T22:42:38.441602+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:29.588589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440411+0000","last_change":"2024-05-08T22:42:36.925141+0000","last_active":"2024-05-08T22:42:38.440411+0000","last_peered":"2024-05-08T22:42:38.440411+0000","last_clean":"2024-05-08T22:42:38.440411+0000","last_became_active":"2024-05-08T22:42:36.924777+0000","last_became_peered":"2024-05-08T22:42:36.924777+0000","last_unstale":"2024-05-08T22:42:38.440411+0000","last_undegraded":"2024-05-08T22:42:38.440411+0000","last_fullsized":"2024-05-08T22:42:38.440411+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:23.633428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.315161+0000","last_change":"2024-05-08T22:42:37.313279+0000","last_active":"2024-05-08T22:42:37.315161+0000","last_peered":"2024-05-08T22:42:37.315161+0000","last_clean":"2024-05-08T22:42:37.315161+0000","last_became_active":"2024-05-08T22:42:37.312937+0000","last_became_peered":"2024-05-08T22:42:37.312937+0000","last_unstale":"2024-05-08T22:42:37.315161+0000","last_undegraded":"2024-05-08T22:42:37.315161+0000","last_fullsized":"2024-05-08T22:42:37.315161+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:21.612995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442277+0000","last_change":"2024-05-08T22:42:36.923145+0000","last_active":"2024-05-08T22:42:38.442277+0000","last_peered":"2024-05-08T22:42:38.442277+0000","last_clean":"2024-05-08T22:42:38.442277+0000","last_became_active":"2024-05-08T22:42:36.922855+0000","last_became_peered":"2024-05-08T22:42:36.922855+0000","last_unstale":"2024-05-08T22:42:38.442277+0000","last_undegraded":"2024-05-08T22:42:38.442277+0000","last_fullsized":"2024-05-08T22:42:38.442277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:54.782437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301530+0000","last_change":"2024-05-08T22:42:37.303260+0000","last_active":"2024-05-08T22:42:38.301530+0000","last_peered":"2024-05-08T22:42:38.301530+0000","last_clean":"2024-05-08T22:42:38.301530+0000","last_became_active":"2024-05-08T22:42:37.303030+0000","last_became_peered":"2024-05-08T22:42:37.303030+0000","last_unstale":"2024-05-08T22:42:38.301530+0000","last_undegraded":"2024-05-08T22:42:38.301530+0000","last_fullsized":"2024-05-08T22:42:38.301530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:14.217428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301563+0000","last_change":"2024-05-08T22:42:36.915921+0000","last_active":"2024-05-08T22:42:38.301563+0000","last_peered":"2024-05-08T22:42:38.301563+0000","last_clean":"2024-05-08T22:42:38.301563+0000","last_became_active":"2024-05-08T22:42:36.914498+0000","last_became_peered":"2024-05-08T22:42:36.914498+0000","last_unstale":"2024-05-08T22:42:38.301563+0000","last_undegraded":"2024-05-08T22:42:38.301563+0000","last_fullsized":"2024-05-08T22:42:38.301563+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:23.633428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301664+0000","last_change":"2024-05-08T22:42:37.316803+0000","last_active":"2024-05-08T22:42:38.301664+0000","last_peered":"2024-05-08T22:42:38.301664+0000","last_clean":"2024-05-08T22:42:38.301664+0000","last_became_active":"2024-05-08T22:42:37.315963+0000","last_became_peered":"2024-05-08T22:42:37.315963+0000","last_unstale":"2024-05-08T22:42:38.301664+0000","last_undegraded":"2024-05-08T22:42:38.301664+0000","last_fullsized":"2024-05-08T22:42:38.301664+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.696314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442076+0000","last_change":"2024-05-08T22:42:36.924014+0000","last_active":"2024-05-08T22:42:38.442076+0000","last_peered":"2024-05-08T22:42:38.442076+0000","last_clean":"2024-05-08T22:42:38.442076+0000","last_became_active":"2024-05-08T22:42:36.923727+0000","last_became_peered":"2024-05-08T22:42:36.923727+0000","last_unstale":"2024-05-08T22:42:38.442076+0000","last_undegraded":"2024-05-08T22:42:38.442076+0000","last_fullsized":"2024-05-08T22:42:38.442076+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:02.450478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442963+0000","last_change":"2024-05-08T22:42:37.303200+0000","last_active":"2024-05-08T22:42:38.442963+0000","last_peered":"2024-05-08T22:42:38.442963+0000","last_clean":"2024-05-08T22:42:38.442963+0000","last_became_active":"2024-05-08T22:42:37.302399+0000","last_became_peered":"2024-05-08T22:42:37.302399+0000","last_unstale":"2024-05-08T22:42:38.442963+0000","last_undegraded":"2024-05-08T22:42:38.442963+0000","last_fullsized":"2024-05-08T22:42:38.442963+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:08.339236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.291854+0000","last_change":"2024-05-08T22:42:36.913424+0000","last_active":"2024-05-08T22:42:37.291854+0000","last_peered":"2024-05-08T22:42:37.291854+0000","last_clean":"2024-05-08T22:42:37.291854+0000","last_became_active":"2024-05-08T22:42:36.913110+0000","last_became_peered":"2024-05-08T22:42:36.913110+0000","last_unstale":"2024-05-08T22:42:37.291854+0000","last_undegraded":"2024-05-08T22:42:37.291854+0000","last_fullsized":"2024-05-08T22:42:37.291854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:00.855199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442856+0000","last_change":"2024-05-08T22:42:37.302873+0000","last_active":"2024-05-08T22:42:38.442856+0000","last_peered":"2024-05-08T22:42:38.442856+0000","last_clean":"2024-05-08T22:42:38.442856+0000","last_became_active":"2024-05-08T22:42:37.302529+0000","last_became_peered":"2024-05-08T22:42:37.302529+0000","last_unstale":"2024-05-08T22:42:38.442856+0000","last_undegraded":"2024-05-08T22:42:38.442856+0000","last_fullsized":"2024-05-08T22:42:38.442856+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:54.233330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440564+0000","last_change":"2024-05-08T22:42:36.920235+0000","last_active":"2024-05-08T22:42:38.440564+0000","last_peered":"2024-05-08T22:42:38.440564+0000","last_clean":"2024-05-08T22:42:38.440564+0000","last_became_active":"2024-05-08T22:42:36.919639+0000","last_became_peered":"2024-05-08T22:42:36.919639+0000","last_unstale":"2024-05-08T22:42:38.440564+0000","last_undegraded":"2024-05-08T22:42:38.440564+0000","last_fullsized":"2024-05-08T22:42:38.440564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438607+0000","last_change":"2024-05-08T22:42:37.317924+0000","last_active":"2024-05-08T22:42:38.438607+0000","last_peered":"2024-05-08T22:42:38.438607+0000","last_clean":"2024-05-08T22:42:38.438607+0000","last_became_active":"2024-05-08T22:42:37.317653+0000","last_became_peered":"2024-05-08T22:42:37.317653+0000","last_unstale":"2024-05-08T22:42:38.438607+0000","last_undegraded":"2024-05-08T22:42:38.438607+0000","last_fullsized":"2024-05-08T22:42:38.438607+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:51.007832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300889+0000","last_change":"2024-05-08T22:42:36.914398+0000","last_active":"2024-05-08T22:42:38.300889+0000","last_peered":"2024-05-08T22:42:38.300889+0000","last_clean":"2024-05-08T22:42:38.300889+0000","last_became_active":"2024-05-08T22:42:36.914115+0000","last_became_peered":"2024-05-08T22:42:36.914115+0000","last_unstale":"2024-05-08T22:42:38.300889+0000","last_undegraded":"2024-05-08T22:42:38.300889+0000","last_fullsized":"2024-05-08T22:42:38.300889+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:02.450478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302266+0000","last_change":"2024-05-08T22:42:37.318186+0000","last_active":"2024-05-08T22:42:38.302266+0000","last_peered":"2024-05-08T22:42:38.302266+0000","last_clean":"2024-05-08T22:42:38.302266+0000","last_became_active":"2024-05-08T22:42:37.317901+0000","last_became_peered":"2024-05-08T22:42:37.317901+0000","last_unstale":"2024-05-08T22:42:38.302266+0000","last_undegraded":"2024-05-08T22:42:38.302266+0000","last_fullsized":"2024-05-08T22:42:38.302266+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:48.137194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440825+0000","last_change":"2024-05-08T22:42:36.914185+0000","last_active":"2024-05-08T22:42:38.440825+0000","last_peered":"2024-05-08T22:42:38.440825+0000","last_clean":"2024-05-08T22:42:38.440825+0000","last_became_active":"2024-05-08T22:42:36.913921+0000","last_became_peered":"2024-05-08T22:42:36.913921+0000","last_unstale":"2024-05-08T22:42:38.440825+0000","last_undegraded":"2024-05-08T22:42:38.440825+0000","last_fullsized":"2024-05-08T22:42:38.440825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:11.434992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300587+0000","last_change":"2024-05-08T22:42:37.316704+0000","last_active":"2024-05-08T22:42:38.300587+0000","last_peered":"2024-05-08T22:42:38.300587+0000","last_clean":"2024-05-08T22:42:38.300587+0000","last_became_active":"2024-05-08T22:42:37.316448+0000","last_became_peered":"2024-05-08T22:42:37.316448+0000","last_unstale":"2024-05-08T22:42:38.300587+0000","last_undegraded":"2024-05-08T22:42:38.300587+0000","last_fullsized":"2024-05-08T22:42:38.300587+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:24.832903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443198+0000","last_change":"2024-05-08T22:42:36.919823+0000","last_active":"2024-05-08T22:42:38.443198+0000","last_peered":"2024-05-08T22:42:38.443198+0000","last_clean":"2024-05-08T22:42:38.443198+0000","last_became_active":"2024-05-08T22:42:36.919539+0000","last_became_peered":"2024-05-08T22:42:36.919539+0000","last_unstale":"2024-05-08T22:42:38.443198+0000","last_undegraded":"2024-05-08T22:42:38.443198+0000","last_fullsized":"2024-05-08T22:42:38.443198+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:08.528138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":248,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.289960+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:42:37.289960+0000","last_peered":"2024-05-08T22:42:37.289960+0000","last_clean":"2024-05-08T22:42:37.289960+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:42:37.289960+0000","last_undegraded":"2024-05-08T22:42:37.289960+0000","last_fullsized":"2024-05-08T22:42:37.289960+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1384448,"data_stored":637078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.151,"5min":1.623,"15min":1.623},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.309,"15min":0.307},"max":{"1min":1.115,"5min":1.214,"15min":1.6899999999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.623},"min":{"1min":0.377,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.179,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42199999999999999},"max":{"1min":1.05,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.82299999999999995,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2230000000000001,"5min":1.716,"15min":1.716},"last":0.96799999999999997}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.90900000000000003},"min":{"1min":0.57699999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.306},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.94999999999999996},"min":{"1min":0.71099999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.248,"5min":1.3320000000000001,"15min":1.4139999999999999},"last":1.077}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.77400000000000002,"5min":0.70399999999999996,"15min":0.68300000000000005},"max":{"1min":1.171,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.026,"15min":1.024},"min":{"1min":0.69899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3560000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.046}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.82599999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.421,"5min":1.421,"15min":1.446},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.069,"15min":1.0609999999999999},"min":{"1min":0.79200000000000004,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4490000000000001,"5min":1.512,"15min":1.512},"last":1.006}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.76400000000000001,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.389,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.111},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0549999999999999,"15min":1.0409999999999999},"min":{"1min":0.81499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.476,"5min":1.587,"15min":1.8360000000000001},"last":1.25}]}]},{"osd":1,"up_from":15,"seq":64424509558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1384448,"data_stored":636530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46100000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.98399999999999999,"5min":1.772,"15min":1.772},"last":0.875},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.70299999999999996},"min":{"1min":0.47799999999999998,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":1.1659999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1539999999999999,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81999999999999995,"15min":0.76200000000000001},"min":{"1min":0.55400000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85699999999999998,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.228,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71599999999999997,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.42599999999999999,"15min":0.35599999999999998},"max":{"1min":1.0609999999999999,"5min":1.1579999999999999,"15min":1.583},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.95999999999999996},"min":{"1min":0.56999999999999995,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.202,"5min":1.395,"15min":1.431},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0249999999999999,"15min":0.98699999999999999},"min":{"1min":0.80100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.248,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.5},"last":1.121},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0189999999999999,"15min":1.052},"min":{"1min":0.69799999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.399,"5min":1.599,"15min":1.726},"last":0.98199999999999998}]},{"osd":6,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.60899999999999999},"max":{"1min":1.2110000000000001,"5min":1.351,"15min":1.4650000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.70799999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3520000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.034}]},{"osd":7,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.087,"15min":1.073},"min":{"1min":0.89200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.0549999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5049999999999999},"last":1.2250000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1970176,"data_stored":1220404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83699999999999997,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1120000000000001,"5min":1.232,"15min":1.282},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.879,"15min":0.86299999999999999},"min":{"1min":0.626,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2869999999999999,"5min":1.514,"15min":1.514},"last":1.0669999999999999}]},{"osd":1,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86199999999999999,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.026,"5min":1.355,"15min":1.4470000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.92800000000000005,"15min":0.88200000000000001},"min":{"1min":0.72199999999999998,"5min":0.58199999999999996,"15min":0.443},"max":{"1min":1.2130000000000001,"5min":1.2669999999999999,"15min":1.488},"last":1.0169999999999999}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91500000000000004,"15min":0.90600000000000003},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.58199999999999996},"max":{"1min":1.101,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.95399999999999996,"15min":0.95699999999999996},"min":{"1min":0.65700000000000003,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.0840000000000001,"5min":1.2669999999999999,"15min":1.337},"last":1.1080000000000001}]},{"osd":3,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.92100000000000004},"min":{"1min":0.77700000000000002,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.2190000000000001,"5min":1.3919999999999999,"15min":1.5589999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.74199999999999999,"5min":0.63600000000000001,"15min":0.58199999999999996},"max":{"1min":1.228,"5min":1.228,"15min":1.2290000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.153,"5min":1.611,"15min":1.611},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.874,"15min":0.88500000000000001},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.57399999999999995},"max":{"1min":1.165,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94099999999999995}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.53300000000000003},"max":{"1min":1.1100000000000001,"5min":1.2629999999999999,"15min":1.482},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.056,"15min":1.0509999999999999},"min":{"1min":0.93500000000000005,"5min":0.71299999999999997,"15min":0.60199999999999998},"max":{"1min":1.2090000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.147}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":0.94499999999999995},"min":{"1min":0.71799999999999997,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.27,"5min":1.379,"15min":1.3839999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.1919999999999999,"5min":1.514,"15min":1.599},"last":1.1259999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1388544,"data_stored":632044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.038,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.54200000000000004},"max":{"1min":1.0680000000000001,"5min":1.085,"15min":1.141},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81299999999999994,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.40500000000000003},"max":{"1min":1.099,"5min":1.1299999999999999,"15min":1.758},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92500000000000004,"15min":0.92600000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":1.028}]},{"osd":2,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.94999999999999996},"min":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.49099999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.4319999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.91300000000000003},"min":{"1min":0.68000000000000005,"5min":0.624,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.236,"15min":1.419},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.94299999999999995,"15min":1.0349999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.1359999999999999,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.93000000000000005,"15min":0.96299999999999997},"min":{"1min":0.70899999999999996,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.5029999999999999},"last":1.069}]},{"osd":4,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91200000000000003,"15min":0.96599999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.48599999999999999},"max":{"1min":1.3,"5min":1.3320000000000001,"15min":1.538},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94799999999999995,"15min":1.004},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.44400000000000001},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.458},"last":1.157}]},{"osd":5,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.022,"15min":1.093},"min":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3460000000000001,"5min":1.427,"15min":1.681},"last":1.278},{"interface":"front","average":{"1min":1.042,"5min":1.044,"15min":1.099},"min":{"1min":0.83399999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2909999999999999,"5min":1.393,"15min":1.6000000000000001},"last":1.228}]},{"osd":7,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.94999999999999996,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.252,"5min":1.3580000000000001,"15min":1.835},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.127,"15min":1.1899999999999999},"min":{"1min":0.85199999999999998,"5min":0.748,"15min":0.74099999999999999},"max":{"1min":1.4419999999999999,"5min":1.5600000000000001,"15min":2.2999999999999998},"last":1.341}]}]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1368064,"data_stored":629876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.90000000000000002,"15min":0.84099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":7.907,"15min":7.907},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.157,"5min":1.29,"15min":1.52},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.1259999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89100000000000001,"15min":0.84199999999999997},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.44400000000000001},"max":{"1min":1.097,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.97399999999999998}]},{"osd":2,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.89300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.119,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0109999999999999,"15min":0.90000000000000002},"min":{"1min":0.66400000000000003,"5min":0.58099999999999996,"15min":0.48799999999999999},"max":{"1min":1.212,"5min":10.121,"15min":10.121},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.874,"15min":0.876},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.1579999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.94399999999999995,"15min":0.95999999999999996},"min":{"1min":0.64000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.238,"5min":1.298,"15min":1.298},"last":1.071}]},{"osd":4,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.091,"15min":0.97499999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.0880000000000001,"5min":15.135,"15min":15.135},"last":1.171},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.014},"min":{"1min":0.73099999999999998,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.208,"5min":4.319,"15min":4.319},"last":1.3680000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.93600000000000005},"min":{"1min":0.64900000000000002,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.605},"last":1.347},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.85699999999999998,"5min":0.67100000000000004,"15min":0.54900000000000004},"max":{"1min":1.2230000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.3160000000000001}]},{"osd":7,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1080000000000001,"15min":1.097},"min":{"1min":0.88500000000000001,"5min":0.71399999999999997,"15min":0.57299999999999995},"max":{"1min":1.2709999999999999,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0489999999999999},"min":{"1min":0.80600000000000005,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.3220000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.29}]}]},{"osd":4,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1355776,"data_stored":622781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0660000000000001,"5min":1.242,"15min":1.28},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80800000000000005,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.081,"5min":1.1739999999999999,"15min":1.294},"last":0.67800000000000005}]},{"osd":1,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.624,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1990000000000001,"5min":1.2290000000000001,"15min":1.3080000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.109,"5min":1.292,"15min":1.3140000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88200000000000001,"15min":0.88100000000000001},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.1040000000000001,"5min":1.244,"15min":1.29},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.90600000000000003,"15min":0.92200000000000004},"min":{"1min":0.69999999999999996,"5min":0.57999999999999996,"15min":0.53400000000000003},"max":{"1min":1.1559999999999999,"5min":1.1779999999999999,"15min":1.637},"last":0.80800000000000005}]},{"osd":3,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93500000000000005,"15min":0.995},"min":{"1min":0.78800000000000003,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.141,"5min":1.319,"15min":2.601},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93300000000000005,"15min":1.0349999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.159,"5min":1.22,"15min":2.8769999999999998},"last":0.76400000000000001}]},{"osd":5,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91200000000000003,"15min":0.98699999999999999},"min":{"1min":0.69099999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.792},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.94099999999999995,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.125,"5min":1.3240000000000001,"15min":1.575},"last":1.1140000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97999999999999998,"15min":1.046},"min":{"1min":0.68000000000000005,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.28,"5min":1.377,"15min":1.831},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.01,"5min":1.0269999999999999,"15min":1.0760000000000001},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1719999999999999,"5min":1.3620000000000001,"15min":1.8899999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99299999999999999,"15min":1.0620000000000001},"min":{"1min":0.88100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3080000000000001,"5min":1.403,"15min":1.8500000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1080000000000001,"15min":1.151},"min":{"1min":0.97999999999999998,"5min":0.78000000000000003,"15min":0.76000000000000001},"max":{"1min":1.369,"5min":1.5169999999999999,"15min":1.944},"last":1.0069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542261,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":629910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.627},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0089999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.193},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.45600000000000002,"5min":0.39700000000000002,"15min":0.39300000000000002},"max":{"1min":0.94699999999999995,"5min":1.123,"15min":1.2190000000000001},"last":0.622}]},{"osd":1,"last update":"Wed May 8 22:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96899999999999997,"5min":1.048,"15min":1.048},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.437,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.3460000000000001},"last":0.23599999999999999}]},{"osd":3,"last update":"Wed May 8 22:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.024,"5min":1.071,"15min":1.071},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77800000000000002,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1719999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.75,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.087,"5min":1.504,"15min":1.504},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97099999999999997,"15min":1.0049999999999999},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.66500000000000004},"max":{"1min":1.2,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.32000000000000001}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.70399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":1.294,"5min":1.3779999999999999,"15min":1.484},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97999999999999998,"15min":1.004},"min":{"1min":0.79100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2769999999999999,"5min":1.343,"15min":1.456},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0269999999999999,"15min":1.0469999999999999},"min":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.69599999999999995},"max":{"1min":1.1359999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1,"5min":1.0509999999999999,"15min":1.071},"min":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1850000000000001,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0549999999999999,"15min":1.075},"min":{"1min":0.78300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.109},"min":{"1min":0.64900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.61099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1978368,"data_stored":1221898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60299999999999998,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97099999999999997,"5min":0.97199999999999998,"15min":1.617},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.42099999999999999,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.113,"15min":1.5129999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.54100000000000004,"5min":0.49099999999999999,"15min":0.44500000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.446},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.436},"max":{"1min":1.101,"5min":1.4410000000000001,"15min":1.6910000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed May 8 22:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.79300000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.999,"5min":1.609,"15min":1.714},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.70999999999999996},"min":{"1min":0.497,"5min":0.44,"15min":0.44},"max":{"1min":0.90900000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.63700000000000001,"15min":0.61699999999999999},"max":{"1min":1.0940000000000001,"5min":4.391,"15min":4.391},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.94899999999999995,"15min":0.97899999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1100000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.93799999999999994}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91900000000000004,"15min":0.91200000000000003},"min":{"1min":0.72799999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.246,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.167,"5min":1.4379999999999999,"15min":1.474},"last":1.0620000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1879999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.071,"15min":1.0509999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.66300000000000003},"max":{"1min":1.24,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":1.109}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0509999999999999,"15min":1.073},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":1.3080000000000001,"5min":1.532,"15min":1.6459999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.80200000000000005,"5min":0.79000000000000004,"15min":0.60399999999999998},"max":{"1min":1.264,"5min":1.514,"15min":1.514},"last":1.0009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:42:41.354 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:42:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.466+0000 7f4049c2e700 1 Processor -- start 2024-05-08T22:42:41.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.466+0000 7f4049c2e700 1 -- start start 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f40440c9230 0x7f40440c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f404406af90 0x7f404406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40440ce650 con 0x7f404415d130 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440c9c00 con 0x7f404406af90 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440cd340 con 0x7f40440c9230 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f404406af90 0x7f404406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40437fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f404415d130 0x7f40440cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36014/0 (socket says 172.21.15.179:36014) 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40437fe700 1 -- 172.21.15.179:0/2175111230 learned_addr learned my addr 172.21.15.179:0/2175111230 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:41.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4042ffd700 1 --2- 172.21.15.179:0/2175111230 >> v2:172.21.15.195:3300/0 conn(0x7f40440c9230 0x7f40440c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 -- 172.21.15.179:0/2175111230 >> v2:172.21.15.195:3300/0 conn(0x7f40440c9230 msgr2=0x7f40440c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40427fc700 1 -- 172.21.15.179:0/2175111230 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096614913 0 0) 0x7f40440ce650 con 0x7f404415d130 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 --2- 172.21.15.179:0/2175111230 >> v2:172.21.15.195:3300/0 conn(0x7f40440c9230 0x7f40440c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 -- 172.21.15.179:0/2175111230 >> v1:172.21.15.179:6789/0 conn(0x7f404415d130 legacy=0x7f40440cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 -- 172.21.15.179:0/2175111230 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440cd620 con 0x7f404406af90 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4043fff700 1 --2- 172.21.15.179:0/2175111230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404406b420 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4044067690 tx=0x7f403c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376f45de76ebc9fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40427fc700 1 -- 172.21.15.179:0/2175111230 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c011830 con 0x7f404406af90 2024-05-08T22:42:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40427fc700 1 -- 172.21.15.179:0/2175111230 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c011990 con 0x7f404406af90 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f40427fc700 1 -- 172.21.15.179:0/2175111230 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c011d20 con 0x7f404406af90 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- 172.21.15.179:0/2175111230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 msgr2=0x7f404406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2175111230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404406b420 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4044067690 tx=0x7f403c009b30 comp rx=0 tx=0).stop 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- 172.21.15.179:0/2175111230 shutdown_connections 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2175111230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404406b420 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2175111230 >> v2:172.21.15.195:3300/0 conn(0x7f40440c9230 0x7f40440c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- 172.21.15.179:0/2175111230 >> 172.21.15.179:0/2175111230 conn(0x7f40440cb660 msgr2=0x7f40440cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- 172.21.15.179:0/2175111230 shutdown_connections 2024-05-08T22:42:41.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.470+0000 7f4049c2e700 1 -- 172.21.15.179:0/2175111230 wait complete. 2024-05-08T22:42:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 Processor -- start 2024-05-08T22:42:41.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- start start 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40440c9230 0x7f40440c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 0x7f40441f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404415e470 con 0x7f404406af90 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40441f38b0 con 0x7f40440c9230 2024-05-08T22:42:41.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40441f3b90 con 0x7f404415d130 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37438/0 (socket says 172.21.15.179:37438) 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 -- 172.21.15.179:0/2669267163 learned_addr learned my addr 172.21.15.179:0/2669267163 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4042ffd700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40440c9230 0x7f40440c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4043fff700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 0x7f40441f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 -- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 msgr2=0x7f40441f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 0x7f40441f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 -- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40440c9230 msgr2=0x7f40440c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40440c9230 0x7f40440c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 -- 172.21.15.179:0/2669267163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0094b0 con 0x7f404406af90 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4043fff700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 0x7f40441f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f40437fe700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f403800c4e0 tx=0x7f403800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038004ca0 con 0x7f404406af90 2024-05-08T22:42:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038004e20 con 0x7f404406af90 2024-05-08T22:42:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038007310 con 0x7f404406af90 2024-05-08T22:42:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440c3e30 con 0x7f404406af90 2024-05-08T22:42:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.474+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40440c3b00 con 0x7f404406af90 2024-05-08T22:42:41.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.478+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4044067770 con 0x7f404406af90 2024-05-08T22:42:41.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.478+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4038007470 con 0x7f404406af90 2024-05-08T22:42:41.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.478+0000 7f402bfff700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 0x7f4030068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.478+0000 7f4042ffd700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 0x7f4030068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.478+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4038084610 con 0x7f404406af90 2024-05-08T22:42:41.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.482+0000 7f4042ffd700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 0x7f4030068bf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f40440c9bb0 tx=0x7f403400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.487 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.486+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4038051820 con 0x7f404406af90 2024-05-08T22:42:41.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.654+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40440ccc90 con 0x7f4030066730 2024-05-08T22:42:41.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.670+0000 7f402bfff700 1 -- 172.21.15.179:0/2669267163 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288265 (secure 0 0 0) 0x7f40440ccc90 con 0x7f4030066730 2024-05-08T22:42:41.677 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 msgr2=0x7f4030068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 0x7f4030068bf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f40440c9bb0 tx=0x7f403400b040 comp rx=0 tx=0).stop 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 msgr2=0x7f404409ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f403800c4e0 tx=0x7f403800c7f0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 shutdown_connections 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4030066730 0x7f4030068bf0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40440c9230 0x7f40440c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f404415d130 0x7f40441f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 --2- 172.21.15.179:0/2669267163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404406af90 0x7f404409ea00 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 >> 172.21.15.179:0/2669267163 conn(0x7f40440cb660 msgr2=0x7f40440b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:41.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 shutdown_connections 2024-05-08T22:42:41.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.678+0000 7f4049c2e700 1 -- 172.21.15.179:0/2669267163 wait complete. 2024-05-08T22:42:41.681 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:42:41.693 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-05-08T22:42:41.581207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265000,"kb_used_data":11916,"kb_used_omap":12,"kb_used_meta":252915,"kb_avail":754709720,"statfs":{"total":773094113280,"available":772822753280,"internally_reserved":0,"allocated":12201984,"data_stored":6230521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":258985528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.694573"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442557+0000","last_change":"2024-05-08T22:42:37.308637+0000","last_active":"2024-05-08T22:42:38.442557+0000","last_peered":"2024-05-08T22:42:38.442557+0000","last_clean":"2024-05-08T22:42:38.442557+0000","last_became_active":"2024-05-08T22:42:37.308328+0000","last_became_peered":"2024-05-08T22:42:37.308328+0000","last_unstale":"2024-05-08T22:42:38.442557+0000","last_undegraded":"2024-05-08T22:42:38.442557+0000","last_fullsized":"2024-05-08T22:42:38.442557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:51.017187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301805+0000","last_change":"2024-05-08T22:42:37.300831+0000","last_active":"2024-05-08T22:42:38.301805+0000","last_peered":"2024-05-08T22:42:38.301805+0000","last_clean":"2024-05-08T22:42:38.301805+0000","last_became_active":"2024-05-08T22:42:37.300574+0000","last_became_peered":"2024-05-08T22:42:37.300574+0000","last_unstale":"2024-05-08T22:42:38.301805+0000","last_undegraded":"2024-05-08T22:42:38.301805+0000","last_fullsized":"2024-05-08T22:42:38.301805+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:39.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442193+0000","last_change":"2024-05-08T22:42:37.322626+0000","last_active":"2024-05-08T22:42:38.442193+0000","last_peered":"2024-05-08T22:42:38.442193+0000","last_clean":"2024-05-08T22:42:38.442193+0000","last_became_active":"2024-05-08T22:42:37.322444+0000","last_became_peered":"2024-05-08T22:42:37.322444+0000","last_unstale":"2024-05-08T22:42:38.442193+0000","last_undegraded":"2024-05-08T22:42:38.442193+0000","last_fullsized":"2024-05-08T22:42:38.442193+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:45.034279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302043+0000","last_change":"2024-05-08T22:42:37.308018+0000","last_active":"2024-05-08T22:42:38.302043+0000","last_peered":"2024-05-08T22:42:38.302043+0000","last_clean":"2024-05-08T22:42:38.302043+0000","last_became_active":"2024-05-08T22:42:37.307706+0000","last_became_peered":"2024-05-08T22:42:37.307706+0000","last_unstale":"2024-05-08T22:42:38.302043+0000","last_undegraded":"2024-05-08T22:42:38.302043+0000","last_fullsized":"2024-05-08T22:42:38.302043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:01.975004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440656+0000","last_change":"2024-05-08T22:42:37.305140+0000","last_active":"2024-05-08T22:42:38.440656+0000","last_peered":"2024-05-08T22:42:38.440656+0000","last_clean":"2024-05-08T22:42:38.440656+0000","last_became_active":"2024-05-08T22:42:37.304877+0000","last_became_peered":"2024-05-08T22:42:37.304877+0000","last_unstale":"2024-05-08T22:42:38.440656+0000","last_undegraded":"2024-05-08T22:42:38.440656+0000","last_fullsized":"2024-05-08T22:42:38.440656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:20.843199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300721+0000","last_change":"2024-05-08T22:42:37.311688+0000","last_active":"2024-05-08T22:42:38.300721+0000","last_peered":"2024-05-08T22:42:38.300721+0000","last_clean":"2024-05-08T22:42:38.300721+0000","last_became_active":"2024-05-08T22:42:37.311444+0000","last_became_peered":"2024-05-08T22:42:37.311444+0000","last_unstale":"2024-05-08T22:42:38.300721+0000","last_undegraded":"2024-05-08T22:42:38.300721+0000","last_fullsized":"2024-05-08T22:42:38.300721+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:40.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300144+0000","last_change":"2024-05-08T22:42:37.312196+0000","last_active":"2024-05-08T22:42:38.300144+0000","last_peered":"2024-05-08T22:42:38.300144+0000","last_clean":"2024-05-08T22:42:38.300144+0000","last_became_active":"2024-05-08T22:42:37.311997+0000","last_became_peered":"2024-05-08T22:42:37.311997+0000","last_unstale":"2024-05-08T22:42:38.300144+0000","last_undegraded":"2024-05-08T22:42:38.300144+0000","last_fullsized":"2024-05-08T22:42:38.300144+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:03.199565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440591+0000","last_change":"2024-05-08T22:42:37.306347+0000","last_active":"2024-05-08T22:42:38.440591+0000","last_peered":"2024-05-08T22:42:38.440591+0000","last_clean":"2024-05-08T22:42:38.440591+0000","last_became_active":"2024-05-08T22:42:37.305982+0000","last_became_peered":"2024-05-08T22:42:37.305982+0000","last_unstale":"2024-05-08T22:42:38.440591+0000","last_undegraded":"2024-05-08T22:42:38.440591+0000","last_fullsized":"2024-05-08T22:42:38.440591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:19.944179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302293+0000","last_change":"2024-05-08T22:42:37.306845+0000","last_active":"2024-05-08T22:42:38.302293+0000","last_peered":"2024-05-08T22:42:38.302293+0000","last_clean":"2024-05-08T22:42:38.302293+0000","last_became_active":"2024-05-08T22:42:37.306551+0000","last_became_peered":"2024-05-08T22:42:37.306551+0000","last_unstale":"2024-05-08T22:42:38.302293+0000","last_undegraded":"2024-05-08T22:42:38.302293+0000","last_fullsized":"2024-05-08T22:42:38.302293+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:54:44.808075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440938+0000","last_change":"2024-05-08T22:42:37.305144+0000","last_active":"2024-05-08T22:42:38.440938+0000","last_peered":"2024-05-08T22:42:38.440938+0000","last_clean":"2024-05-08T22:42:38.440938+0000","last_became_active":"2024-05-08T22:42:37.304900+0000","last_became_peered":"2024-05-08T22:42:37.304900+0000","last_unstale":"2024-05-08T22:42:38.440938+0000","last_undegraded":"2024-05-08T22:42:38.440938+0000","last_fullsized":"2024-05-08T22:42:38.440938+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:50.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301675+0000","last_change":"2024-05-08T22:42:37.317444+0000","last_active":"2024-05-08T22:42:38.301675+0000","last_peered":"2024-05-08T22:42:38.301675+0000","last_clean":"2024-05-08T22:42:38.301675+0000","last_became_active":"2024-05-08T22:42:37.317192+0000","last_became_peered":"2024-05-08T22:42:37.317192+0000","last_unstale":"2024-05-08T22:42:38.301675+0000","last_undegraded":"2024-05-08T22:42:38.301675+0000","last_fullsized":"2024-05-08T22:42:38.301675+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:15.105556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.319962+0000","last_change":"2024-05-08T22:42:37.316083+0000","last_active":"2024-05-08T22:42:37.319962+0000","last_peered":"2024-05-08T22:42:37.319962+0000","last_clean":"2024-05-08T22:42:37.319962+0000","last_became_active":"2024-05-08T22:42:37.315828+0000","last_became_peered":"2024-05-08T22:42:37.315828+0000","last_unstale":"2024-05-08T22:42:37.319962+0000","last_undegraded":"2024-05-08T22:42:37.319962+0000","last_fullsized":"2024-05-08T22:42:37.319962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:38.308703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301387+0000","last_change":"2024-05-08T22:42:37.319590+0000","last_active":"2024-05-08T22:42:38.301387+0000","last_peered":"2024-05-08T22:42:38.301387+0000","last_clean":"2024-05-08T22:42:38.301387+0000","last_became_active":"2024-05-08T22:42:37.319333+0000","last_became_peered":"2024-05-08T22:42:37.319333+0000","last_unstale":"2024-05-08T22:42:38.301387+0000","last_undegraded":"2024-05-08T22:42:38.301387+0000","last_fullsized":"2024-05-08T22:42:38.301387+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:13.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438133+0000","last_change":"2024-05-08T22:42:37.303124+0000","last_active":"2024-05-08T22:42:38.438133+0000","last_peered":"2024-05-08T22:42:38.438133+0000","last_clean":"2024-05-08T22:42:38.438133+0000","last_became_active":"2024-05-08T22:42:37.302857+0000","last_became_peered":"2024-05-08T22:42:37.302857+0000","last_unstale":"2024-05-08T22:42:38.438133+0000","last_undegraded":"2024-05-08T22:42:38.438133+0000","last_fullsized":"2024-05-08T22:42:38.438133+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:37.906276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438067+0000","last_change":"2024-05-08T22:42:37.318212+0000","last_active":"2024-05-08T22:42:38.438067+0000","last_peered":"2024-05-08T22:42:38.438067+0000","last_clean":"2024-05-08T22:42:38.438067+0000","last_became_active":"2024-05-08T22:42:37.317970+0000","last_became_peered":"2024-05-08T22:42:37.317970+0000","last_unstale":"2024-05-08T22:42:38.438067+0000","last_undegraded":"2024-05-08T22:42:38.438067+0000","last_fullsized":"2024-05-08T22:42:38.438067+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:29.864562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438077+0000","last_change":"2024-05-08T22:42:37.295570+0000","last_active":"2024-05-08T22:42:38.438077+0000","last_peered":"2024-05-08T22:42:38.438077+0000","last_clean":"2024-05-08T22:42:38.438077+0000","last_became_active":"2024-05-08T22:42:37.295318+0000","last_became_peered":"2024-05-08T22:42:37.295318+0000","last_unstale":"2024-05-08T22:42:38.438077+0000","last_undegraded":"2024-05-08T22:42:38.438077+0000","last_fullsized":"2024-05-08T22:42:38.438077+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:52.333211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442785+0000","last_change":"2024-05-08T22:42:37.321577+0000","last_active":"2024-05-08T22:42:38.442785+0000","last_peered":"2024-05-08T22:42:38.442785+0000","last_clean":"2024-05-08T22:42:38.442785+0000","last_became_active":"2024-05-08T22:42:37.321319+0000","last_became_peered":"2024-05-08T22:42:37.321319+0000","last_unstale":"2024-05-08T22:42:38.442785+0000","last_undegraded":"2024-05-08T22:42:38.442785+0000","last_fullsized":"2024-05-08T22:42:38.442785+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:53.288401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302075+0000","last_change":"2024-05-08T22:42:37.310885+0000","last_active":"2024-05-08T22:42:38.302075+0000","last_peered":"2024-05-08T22:42:38.302075+0000","last_clean":"2024-05-08T22:42:38.302075+0000","last_became_active":"2024-05-08T22:42:37.310617+0000","last_became_peered":"2024-05-08T22:42:37.310617+0000","last_unstale":"2024-05-08T22:42:38.302075+0000","last_undegraded":"2024-05-08T22:42:38.302075+0000","last_fullsized":"2024-05-08T22:42:38.302075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:47.244030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442331+0000","last_change":"2024-05-08T22:42:37.308217+0000","last_active":"2024-05-08T22:42:38.442331+0000","last_peered":"2024-05-08T22:42:38.442331+0000","last_clean":"2024-05-08T22:42:38.442331+0000","last_became_active":"2024-05-08T22:42:37.307091+0000","last_became_peered":"2024-05-08T22:42:37.307091+0000","last_unstale":"2024-05-08T22:42:38.442331+0000","last_undegraded":"2024-05-08T22:42:38.442331+0000","last_fullsized":"2024-05-08T22:42:38.442331+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:59.154570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.301072+0000","last_change":"2024-05-08T22:42:37.298817+0000","last_active":"2024-05-08T22:42:37.301072+0000","last_peered":"2024-05-08T22:42:37.301072+0000","last_clean":"2024-05-08T22:42:37.301072+0000","last_became_active":"2024-05-08T22:42:37.298566+0000","last_became_peered":"2024-05-08T22:42:37.298566+0000","last_unstale":"2024-05-08T22:42:37.301072+0000","last_undegraded":"2024-05-08T22:42:37.301072+0000","last_fullsized":"2024-05-08T22:42:37.301072+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:10.363342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442780+0000","last_change":"2024-05-08T22:42:37.324175+0000","last_active":"2024-05-08T22:42:38.442780+0000","last_peered":"2024-05-08T22:42:38.442780+0000","last_clean":"2024-05-08T22:42:38.442780+0000","last_became_active":"2024-05-08T22:42:37.323899+0000","last_became_peered":"2024-05-08T22:42:37.323899+0000","last_unstale":"2024-05-08T22:42:38.442780+0000","last_undegraded":"2024-05-08T22:42:38.442780+0000","last_fullsized":"2024-05-08T22:42:38.442780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:19.934128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440203+0000","last_change":"2024-05-08T22:42:37.319201+0000","last_active":"2024-05-08T22:42:38.440203+0000","last_peered":"2024-05-08T22:42:38.440203+0000","last_clean":"2024-05-08T22:42:38.440203+0000","last_became_active":"2024-05-08T22:42:37.318913+0000","last_became_peered":"2024-05-08T22:42:37.318913+0000","last_unstale":"2024-05-08T22:42:38.440203+0000","last_undegraded":"2024-05-08T22:42:38.440203+0000","last_fullsized":"2024-05-08T22:42:38.440203+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:51.098236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442302+0000","last_change":"2024-05-08T22:42:37.319606+0000","last_active":"2024-05-08T22:42:38.442302+0000","last_peered":"2024-05-08T22:42:38.442302+0000","last_clean":"2024-05-08T22:42:38.442302+0000","last_became_active":"2024-05-08T22:42:37.319365+0000","last_became_peered":"2024-05-08T22:42:37.319365+0000","last_unstale":"2024-05-08T22:42:38.442302+0000","last_undegraded":"2024-05-08T22:42:38.442302+0000","last_fullsized":"2024-05-08T22:42:38.442302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:33.045019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440050+0000","last_change":"2024-05-08T22:42:37.305749+0000","last_active":"2024-05-08T22:42:38.440050+0000","last_peered":"2024-05-08T22:42:38.440050+0000","last_clean":"2024-05-08T22:42:38.440050+0000","last_became_active":"2024-05-08T22:42:37.305548+0000","last_became_peered":"2024-05-08T22:42:37.305548+0000","last_unstale":"2024-05-08T22:42:38.440050+0000","last_undegraded":"2024-05-08T22:42:38.440050+0000","last_fullsized":"2024-05-08T22:42:38.440050+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:36:47.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.320816+0000","last_change":"2024-05-08T22:42:37.319936+0000","last_active":"2024-05-08T22:42:37.320816+0000","last_peered":"2024-05-08T22:42:37.320816+0000","last_clean":"2024-05-08T22:42:37.320816+0000","last_became_active":"2024-05-08T22:42:37.319670+0000","last_became_peered":"2024-05-08T22:42:37.319670+0000","last_unstale":"2024-05-08T22:42:37.320816+0000","last_undegraded":"2024-05-08T22:42:37.320816+0000","last_fullsized":"2024-05-08T22:42:37.320816+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:55.794126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.301799+0000","last_change":"2024-05-08T22:42:37.299783+0000","last_active":"2024-05-08T22:42:37.301799+0000","last_peered":"2024-05-08T22:42:37.301799+0000","last_clean":"2024-05-08T22:42:37.301799+0000","last_became_active":"2024-05-08T22:42:37.299512+0000","last_became_peered":"2024-05-08T22:42:37.299512+0000","last_unstale":"2024-05-08T22:42:37.301799+0000","last_undegraded":"2024-05-08T22:42:37.301799+0000","last_fullsized":"2024-05-08T22:42:37.301799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:07.848763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301862+0000","last_change":"2024-05-08T22:42:37.316847+0000","last_active":"2024-05-08T22:42:38.301862+0000","last_peered":"2024-05-08T22:42:38.301862+0000","last_clean":"2024-05-08T22:42:38.301862+0000","last_became_active":"2024-05-08T22:42:37.316596+0000","last_became_peered":"2024-05-08T22:42:37.316596+0000","last_unstale":"2024-05-08T22:42:38.301862+0000","last_undegraded":"2024-05-08T22:42:38.301862+0000","last_fullsized":"2024-05-08T22:42:38.301862+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:19.944179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301026+0000","last_change":"2024-05-08T22:42:37.321772+0000","last_active":"2024-05-08T22:42:38.301026+0000","last_peered":"2024-05-08T22:42:38.301026+0000","last_clean":"2024-05-08T22:42:38.301026+0000","last_became_active":"2024-05-08T22:42:37.321454+0000","last_became_peered":"2024-05-08T22:42:37.321454+0000","last_unstale":"2024-05-08T22:42:38.301026+0000","last_undegraded":"2024-05-08T22:42:38.301026+0000","last_fullsized":"2024-05-08T22:42:38.301026+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:23.926135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442981+0000","last_change":"2024-05-08T22:42:37.301667+0000","last_active":"2024-05-08T22:42:38.442981+0000","last_peered":"2024-05-08T22:42:38.442981+0000","last_clean":"2024-05-08T22:42:38.442981+0000","last_became_active":"2024-05-08T22:42:37.301376+0000","last_became_peered":"2024-05-08T22:42:37.301376+0000","last_unstale":"2024-05-08T22:42:38.442981+0000","last_undegraded":"2024-05-08T22:42:38.442981+0000","last_fullsized":"2024-05-08T22:42:38.442981+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:20.885608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.320768+0000","last_change":"2024-05-08T22:42:37.319861+0000","last_active":"2024-05-08T22:42:37.320768+0000","last_peered":"2024-05-08T22:42:37.320768+0000","last_clean":"2024-05-08T22:42:37.320768+0000","last_became_active":"2024-05-08T22:42:37.319598+0000","last_became_peered":"2024-05-08T22:42:37.319598+0000","last_unstale":"2024-05-08T22:42:37.320768+0000","last_undegraded":"2024-05-08T22:42:37.320768+0000","last_fullsized":"2024-05-08T22:42:37.320768+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:08.421748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300464+0000","last_change":"2024-05-08T22:42:37.306840+0000","last_active":"2024-05-08T22:42:38.300464+0000","last_peered":"2024-05-08T22:42:38.300464+0000","last_clean":"2024-05-08T22:42:38.300464+0000","last_became_active":"2024-05-08T22:42:37.306586+0000","last_became_peered":"2024-05-08T22:42:37.306586+0000","last_unstale":"2024-05-08T22:42:38.300464+0000","last_undegraded":"2024-05-08T22:42:38.300464+0000","last_fullsized":"2024-05-08T22:42:38.300464+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:23.667992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440564+0000","last_change":"2024-05-08T22:42:37.302534+0000","last_active":"2024-05-08T22:42:38.440564+0000","last_peered":"2024-05-08T22:42:38.440564+0000","last_clean":"2024-05-08T22:42:38.440564+0000","last_became_active":"2024-05-08T22:42:37.302278+0000","last_became_peered":"2024-05-08T22:42:37.302278+0000","last_unstale":"2024-05-08T22:42:38.440564+0000","last_undegraded":"2024-05-08T22:42:38.440564+0000","last_fullsized":"2024-05-08T22:42:38.440564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:27.353883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441780+0000","last_change":"2024-05-08T22:42:37.306240+0000","last_active":"2024-05-08T22:42:38.441780+0000","last_peered":"2024-05-08T22:42:38.441780+0000","last_clean":"2024-05-08T22:42:38.441780+0000","last_became_active":"2024-05-08T22:42:37.305844+0000","last_became_peered":"2024-05-08T22:42:37.305844+0000","last_unstale":"2024-05-08T22:42:38.441780+0000","last_undegraded":"2024-05-08T22:42:38.441780+0000","last_fullsized":"2024-05-08T22:42:38.441780+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.717118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"111'4","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301848+0000","last_change":"2024-05-08T22:42:36.923772+0000","last_active":"2024-05-08T22:42:38.301848+0000","last_peered":"2024-05-08T22:42:38.301848+0000","last_clean":"2024-05-08T22:42:38.301848+0000","last_became_active":"2024-05-08T22:42:36.923484+0000","last_became_peered":"2024-05-08T22:42:36.923484+0000","last_unstale":"2024-05-08T22:42:38.301848+0000","last_undegraded":"2024-05-08T22:42:38.301848+0000","last_fullsized":"2024-05-08T22:42:38.301848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:39.419724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440933+0000","last_change":"2024-05-08T22:42:37.303085+0000","last_active":"2024-05-08T22:42:38.440933+0000","last_peered":"2024-05-08T22:42:38.440933+0000","last_clean":"2024-05-08T22:42:38.440933+0000","last_became_active":"2024-05-08T22:42:37.302851+0000","last_became_peered":"2024-05-08T22:42:37.302851+0000","last_unstale":"2024-05-08T22:42:38.440933+0000","last_undegraded":"2024-05-08T22:42:38.440933+0000","last_fullsized":"2024-05-08T22:42:38.440933+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:24.832903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443254+0000","last_change":"2024-05-08T22:42:36.917949+0000","last_active":"2024-05-08T22:42:38.443254+0000","last_peered":"2024-05-08T22:42:38.443254+0000","last_clean":"2024-05-08T22:42:38.443254+0000","last_became_active":"2024-05-08T22:42:36.916543+0000","last_became_peered":"2024-05-08T22:42:36.916543+0000","last_unstale":"2024-05-08T22:42:38.443254+0000","last_undegraded":"2024-05-08T22:42:38.443254+0000","last_fullsized":"2024-05-08T22:42:38.443254+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:27.220481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.305487+0000","last_change":"2024-05-08T22:42:37.303596+0000","last_active":"2024-05-08T22:42:37.305487+0000","last_peered":"2024-05-08T22:42:37.305487+0000","last_clean":"2024-05-08T22:42:37.305487+0000","last_became_active":"2024-05-08T22:42:37.303287+0000","last_became_peered":"2024-05-08T22:42:37.303287+0000","last_unstale":"2024-05-08T22:42:37.305487+0000","last_undegraded":"2024-05-08T22:42:37.305487+0000","last_fullsized":"2024-05-08T22:42:37.305487+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:12.709971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"111'2","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.323291+0000","last_change":"2024-05-08T22:42:36.920937+0000","last_active":"2024-05-08T22:42:37.323291+0000","last_peered":"2024-05-08T22:42:37.323291+0000","last_clean":"2024-05-08T22:42:37.323291+0000","last_became_active":"2024-05-08T22:42:36.920689+0000","last_became_peered":"2024-05-08T22:42:36.920689+0000","last_unstale":"2024-05-08T22:42:37.323291+0000","last_undegraded":"2024-05-08T22:42:37.323291+0000","last_fullsized":"2024-05-08T22:42:37.323291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:24.067098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301769+0000","last_change":"2024-05-08T22:42:37.321531+0000","last_active":"2024-05-08T22:42:38.301769+0000","last_peered":"2024-05-08T22:42:38.301769+0000","last_clean":"2024-05-08T22:42:38.301769+0000","last_became_active":"2024-05-08T22:42:37.321268+0000","last_became_peered":"2024-05-08T22:42:37.321268+0000","last_unstale":"2024-05-08T22:42:38.301769+0000","last_undegraded":"2024-05-08T22:42:38.301769+0000","last_fullsized":"2024-05-08T22:42:38.301769+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:23.081244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441877+0000","last_change":"2024-05-08T22:42:36.920050+0000","last_active":"2024-05-08T22:42:38.441877+0000","last_peered":"2024-05-08T22:42:38.441877+0000","last_clean":"2024-05-08T22:42:38.441877+0000","last_became_active":"2024-05-08T22:42:36.919800+0000","last_became_peered":"2024-05-08T22:42:36.919800+0000","last_unstale":"2024-05-08T22:42:38.441877+0000","last_undegraded":"2024-05-08T22:42:38.441877+0000","last_fullsized":"2024-05-08T22:42:38.441877+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:49.495530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301777+0000","last_change":"2024-05-08T22:42:37.305059+0000","last_active":"2024-05-08T22:42:38.301777+0000","last_peered":"2024-05-08T22:42:38.301777+0000","last_clean":"2024-05-08T22:42:38.301777+0000","last_became_active":"2024-05-08T22:42:37.304829+0000","last_became_peered":"2024-05-08T22:42:37.304829+0000","last_unstale":"2024-05-08T22:42:38.301777+0000","last_undegraded":"2024-05-08T22:42:38.301777+0000","last_fullsized":"2024-05-08T22:42:38.301777+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:10.536786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438345+0000","last_change":"2024-05-08T22:42:36.913789+0000","last_active":"2024-05-08T22:42:38.438345+0000","last_peered":"2024-05-08T22:42:38.438345+0000","last_clean":"2024-05-08T22:42:38.438345+0000","last_became_active":"2024-05-08T22:42:36.913524+0000","last_became_peered":"2024-05-08T22:42:36.913524+0000","last_unstale":"2024-05-08T22:42:38.438345+0000","last_undegraded":"2024-05-08T22:42:38.438345+0000","last_fullsized":"2024-05-08T22:42:38.438345+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:32.689897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442589+0000","last_change":"2024-05-08T22:42:37.306352+0000","last_active":"2024-05-08T22:42:38.442589+0000","last_peered":"2024-05-08T22:42:38.442589+0000","last_clean":"2024-05-08T22:42:38.442589+0000","last_became_active":"2024-05-08T22:42:37.306128+0000","last_became_peered":"2024-05-08T22:42:37.306128+0000","last_unstale":"2024-05-08T22:42:38.442589+0000","last_undegraded":"2024-05-08T22:42:38.442589+0000","last_fullsized":"2024-05-08T22:42:38.442589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:52.035922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301393+0000","last_change":"2024-05-08T22:42:36.919712+0000","last_active":"2024-05-08T22:42:38.301393+0000","last_peered":"2024-05-08T22:42:38.301393+0000","last_clean":"2024-05-08T22:42:38.301393+0000","last_became_active":"2024-05-08T22:42:36.919460+0000","last_became_peered":"2024-05-08T22:42:36.919460+0000","last_unstale":"2024-05-08T22:42:38.301393+0000","last_undegraded":"2024-05-08T22:42:38.301393+0000","last_fullsized":"2024-05-08T22:42:38.301393+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:03.770010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302165+0000","last_change":"2024-05-08T22:42:37.318193+0000","last_active":"2024-05-08T22:42:38.302165+0000","last_peered":"2024-05-08T22:42:38.302165+0000","last_clean":"2024-05-08T22:42:38.302165+0000","last_became_active":"2024-05-08T22:42:37.317941+0000","last_became_peered":"2024-05-08T22:42:37.317941+0000","last_unstale":"2024-05-08T22:42:38.302165+0000","last_undegraded":"2024-05-08T22:42:38.302165+0000","last_fullsized":"2024-05-08T22:42:38.302165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:20.843199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.293361+0000","last_change":"2024-05-08T22:42:36.920433+0000","last_active":"2024-05-08T22:42:37.293361+0000","last_peered":"2024-05-08T22:42:37.293361+0000","last_clean":"2024-05-08T22:42:37.293361+0000","last_became_active":"2024-05-08T22:42:36.920039+0000","last_became_peered":"2024-05-08T22:42:36.920039+0000","last_unstale":"2024-05-08T22:42:37.293361+0000","last_undegraded":"2024-05-08T22:42:37.293361+0000","last_fullsized":"2024-05-08T22:42:37.293361+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:07.749196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301749+0000","last_change":"2024-05-08T22:42:37.307746+0000","last_active":"2024-05-08T22:42:38.301749+0000","last_peered":"2024-05-08T22:42:38.301749+0000","last_clean":"2024-05-08T22:42:38.301749+0000","last_became_active":"2024-05-08T22:42:37.307450+0000","last_became_peered":"2024-05-08T22:42:37.307450+0000","last_unstale":"2024-05-08T22:42:38.301749+0000","last_undegraded":"2024-05-08T22:42:38.301749+0000","last_fullsized":"2024-05-08T22:42:38.301749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:37.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301244+0000","last_change":"2024-05-08T22:42:36.923051+0000","last_active":"2024-05-08T22:42:38.301244+0000","last_peered":"2024-05-08T22:42:38.301244+0000","last_clean":"2024-05-08T22:42:38.301244+0000","last_became_active":"2024-05-08T22:42:36.922784+0000","last_became_peered":"2024-05-08T22:42:36.922784+0000","last_unstale":"2024-05-08T22:42:38.301244+0000","last_undegraded":"2024-05-08T22:42:38.301244+0000","last_fullsized":"2024-05-08T22:42:38.301244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:46.232341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441661+0000","last_change":"2024-05-08T22:42:37.306080+0000","last_active":"2024-05-08T22:42:38.441661+0000","last_peered":"2024-05-08T22:42:38.441661+0000","last_clean":"2024-05-08T22:42:38.441661+0000","last_became_active":"2024-05-08T22:42:37.305510+0000","last_became_peered":"2024-05-08T22:42:37.305510+0000","last_unstale":"2024-05-08T22:42:38.441661+0000","last_undegraded":"2024-05-08T22:42:38.441661+0000","last_fullsized":"2024-05-08T22:42:38.441661+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:32.630371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.295220+0000","last_change":"2024-05-08T22:42:36.920413+0000","last_active":"2024-05-08T22:42:37.295220+0000","last_peered":"2024-05-08T22:42:37.295220+0000","last_clean":"2024-05-08T22:42:37.295220+0000","last_became_active":"2024-05-08T22:42:36.920139+0000","last_became_peered":"2024-05-08T22:42:36.920139+0000","last_unstale":"2024-05-08T22:42:37.295220+0000","last_undegraded":"2024-05-08T22:42:37.295220+0000","last_fullsized":"2024-05-08T22:42:37.295220+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:19.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441645+0000","last_change":"2024-05-08T22:42:37.306778+0000","last_active":"2024-05-08T22:42:38.441645+0000","last_peered":"2024-05-08T22:42:38.441645+0000","last_clean":"2024-05-08T22:42:38.441645+0000","last_became_active":"2024-05-08T22:42:37.306488+0000","last_became_peered":"2024-05-08T22:42:37.306488+0000","last_unstale":"2024-05-08T22:42:38.441645+0000","last_undegraded":"2024-05-08T22:42:38.441645+0000","last_fullsized":"2024-05-08T22:42:38.441645+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:13.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442069+0000","last_change":"2024-05-08T22:42:36.917913+0000","last_active":"2024-05-08T22:42:38.442069+0000","last_peered":"2024-05-08T22:42:38.442069+0000","last_clean":"2024-05-08T22:42:38.442069+0000","last_became_active":"2024-05-08T22:42:36.917643+0000","last_became_peered":"2024-05-08T22:42:36.917643+0000","last_unstale":"2024-05-08T22:42:38.442069+0000","last_undegraded":"2024-05-08T22:42:38.442069+0000","last_fullsized":"2024-05-08T22:42:38.442069+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:11.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440424+0000","last_change":"2024-05-08T22:42:37.302730+0000","last_active":"2024-05-08T22:42:38.440424+0000","last_peered":"2024-05-08T22:42:38.440424+0000","last_clean":"2024-05-08T22:42:38.440424+0000","last_became_active":"2024-05-08T22:42:37.302490+0000","last_became_peered":"2024-05-08T22:42:37.302490+0000","last_unstale":"2024-05-08T22:42:38.440424+0000","last_undegraded":"2024-05-08T22:42:38.440424+0000","last_fullsized":"2024-05-08T22:42:38.440424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:48.137194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"111'3","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438469+0000","last_change":"2024-05-08T22:42:36.921374+0000","last_active":"2024-05-08T22:42:38.438469+0000","last_peered":"2024-05-08T22:42:38.438469+0000","last_clean":"2024-05-08T22:42:38.438469+0000","last_became_active":"2024-05-08T22:42:36.921127+0000","last_became_peered":"2024-05-08T22:42:36.921127+0000","last_unstale":"2024-05-08T22:42:38.438469+0000","last_undegraded":"2024-05-08T22:42:38.438469+0000","last_fullsized":"2024-05-08T22:42:38.438469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:19.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440997+0000","last_change":"2024-05-08T22:42:37.306910+0000","last_active":"2024-05-08T22:42:38.440997+0000","last_peered":"2024-05-08T22:42:38.440997+0000","last_clean":"2024-05-08T22:42:38.440997+0000","last_became_active":"2024-05-08T22:42:37.306611+0000","last_became_peered":"2024-05-08T22:42:37.306611+0000","last_unstale":"2024-05-08T22:42:38.440997+0000","last_undegraded":"2024-05-08T22:42:38.440997+0000","last_fullsized":"2024-05-08T22:42:38.440997+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:23.926135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"111'6","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300993+0000","last_change":"2024-05-08T22:42:36.919423+0000","last_active":"2024-05-08T22:42:38.300993+0000","last_peered":"2024-05-08T22:42:38.300993+0000","last_clean":"2024-05-08T22:42:38.300993+0000","last_became_active":"2024-05-08T22:42:36.919173+0000","last_became_peered":"2024-05-08T22:42:36.919173+0000","last_unstale":"2024-05-08T22:42:38.300993+0000","last_undegraded":"2024-05-08T22:42:38.300993+0000","last_fullsized":"2024-05-08T22:42:38.300993+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:56.453730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301773+0000","last_change":"2024-05-08T22:42:37.315703+0000","last_active":"2024-05-08T22:42:38.301773+0000","last_peered":"2024-05-08T22:42:38.301773+0000","last_clean":"2024-05-08T22:42:38.301773+0000","last_became_active":"2024-05-08T22:42:37.315477+0000","last_became_peered":"2024-05-08T22:42:37.315477+0000","last_unstale":"2024-05-08T22:42:38.301773+0000","last_undegraded":"2024-05-08T22:42:38.301773+0000","last_fullsized":"2024-05-08T22:42:38.301773+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:11.288809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441108+0000","last_change":"2024-05-08T22:42:36.925185+0000","last_active":"2024-05-08T22:42:38.441108+0000","last_peered":"2024-05-08T22:42:38.441108+0000","last_clean":"2024-05-08T22:42:38.441108+0000","last_became_active":"2024-05-08T22:42:36.924775+0000","last_became_peered":"2024-05-08T22:42:36.924775+0000","last_unstale":"2024-05-08T22:42:38.441108+0000","last_undegraded":"2024-05-08T22:42:38.441108+0000","last_fullsized":"2024-05-08T22:42:38.441108+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.748812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301463+0000","last_change":"2024-05-08T22:42:37.304551+0000","last_active":"2024-05-08T22:42:38.301463+0000","last_peered":"2024-05-08T22:42:38.301463+0000","last_clean":"2024-05-08T22:42:38.301463+0000","last_became_active":"2024-05-08T22:42:37.304254+0000","last_became_peered":"2024-05-08T22:42:37.304254+0000","last_unstale":"2024-05-08T22:42:38.301463+0000","last_undegraded":"2024-05-08T22:42:38.301463+0000","last_fullsized":"2024-05-08T22:42:38.301463+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:50.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301462+0000","last_change":"2024-05-08T22:42:36.920214+0000","last_active":"2024-05-08T22:42:38.301462+0000","last_peered":"2024-05-08T22:42:38.301462+0000","last_clean":"2024-05-08T22:42:38.301462+0000","last_became_active":"2024-05-08T22:42:36.919313+0000","last_became_peered":"2024-05-08T22:42:36.919313+0000","last_unstale":"2024-05-08T22:42:38.301462+0000","last_undegraded":"2024-05-08T22:42:38.301462+0000","last_fullsized":"2024-05-08T22:42:38.301462+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:22.914446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442962+0000","last_change":"2024-05-08T22:42:37.305012+0000","last_active":"2024-05-08T22:42:38.442962+0000","last_peered":"2024-05-08T22:42:38.442962+0000","last_clean":"2024-05-08T22:42:38.442962+0000","last_became_active":"2024-05-08T22:42:37.304775+0000","last_became_peered":"2024-05-08T22:42:37.304775+0000","last_unstale":"2024-05-08T22:42:38.442962+0000","last_undegraded":"2024-05-08T22:42:38.442962+0000","last_fullsized":"2024-05-08T22:42:38.442962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:22.670395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440931+0000","last_change":"2024-05-08T22:42:36.920533+0000","last_active":"2024-05-08T22:42:38.440931+0000","last_peered":"2024-05-08T22:42:38.440931+0000","last_clean":"2024-05-08T22:42:38.440931+0000","last_became_active":"2024-05-08T22:42:36.919788+0000","last_became_peered":"2024-05-08T22:42:36.919788+0000","last_unstale":"2024-05-08T22:42:38.440931+0000","last_undegraded":"2024-05-08T22:42:38.440931+0000","last_fullsized":"2024-05-08T22:42:38.440931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:14.080331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442839+0000","last_change":"2024-05-08T22:42:37.321587+0000","last_active":"2024-05-08T22:42:38.442839+0000","last_peered":"2024-05-08T22:42:38.442839+0000","last_clean":"2024-05-08T22:42:38.442839+0000","last_became_active":"2024-05-08T22:42:37.321339+0000","last_became_peered":"2024-05-08T22:42:37.321339+0000","last_unstale":"2024-05-08T22:42:38.442839+0000","last_undegraded":"2024-05-08T22:42:38.442839+0000","last_fullsized":"2024-05-08T22:42:38.442839+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:00.311379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"111'8","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.323853+0000","last_change":"2024-05-08T22:42:36.920571+0000","last_active":"2024-05-08T22:42:37.323853+0000","last_peered":"2024-05-08T22:42:37.323853+0000","last_clean":"2024-05-08T22:42:37.323853+0000","last_became_active":"2024-05-08T22:42:36.920077+0000","last_became_peered":"2024-05-08T22:42:36.920077+0000","last_unstale":"2024-05-08T22:42:37.323853+0000","last_undegraded":"2024-05-08T22:42:37.323853+0000","last_fullsized":"2024-05-08T22:42:37.323853+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:08.065165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440674+0000","last_change":"2024-05-08T22:42:37.306195+0000","last_active":"2024-05-08T22:42:38.440674+0000","last_peered":"2024-05-08T22:42:38.440674+0000","last_clean":"2024-05-08T22:42:38.440674+0000","last_became_active":"2024-05-08T22:42:37.305643+0000","last_became_peered":"2024-05-08T22:42:37.305643+0000","last_unstale":"2024-05-08T22:42:38.440674+0000","last_undegraded":"2024-05-08T22:42:38.440674+0000","last_fullsized":"2024-05-08T22:42:38.440674+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:30.097060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302376+0000","last_change":"2024-05-08T22:42:36.918745+0000","last_active":"2024-05-08T22:42:38.302376+0000","last_peered":"2024-05-08T22:42:38.302376+0000","last_clean":"2024-05-08T22:42:38.302376+0000","last_became_active":"2024-05-08T22:42:36.916977+0000","last_became_peered":"2024-05-08T22:42:36.916977+0000","last_unstale":"2024-05-08T22:42:38.302376+0000","last_undegraded":"2024-05-08T22:42:38.302376+0000","last_fullsized":"2024-05-08T22:42:38.302376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:28.658528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301076+0000","last_change":"2024-05-08T22:42:37.307413+0000","last_active":"2024-05-08T22:42:38.301076+0000","last_peered":"2024-05-08T22:42:38.301076+0000","last_clean":"2024-05-08T22:42:38.301076+0000","last_became_active":"2024-05-08T22:42:37.307145+0000","last_became_peered":"2024-05-08T22:42:37.307145+0000","last_unstale":"2024-05-08T22:42:38.301076+0000","last_undegraded":"2024-05-08T22:42:38.301076+0000","last_fullsized":"2024-05-08T22:42:38.301076+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:13.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"111'1","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438647+0000","last_change":"2024-05-08T22:42:36.914725+0000","last_active":"2024-05-08T22:42:38.438647+0000","last_peered":"2024-05-08T22:42:38.438647+0000","last_clean":"2024-05-08T22:42:38.438647+0000","last_became_active":"2024-05-08T22:42:36.914461+0000","last_became_peered":"2024-05-08T22:42:36.914461+0000","last_unstale":"2024-05-08T22:42:38.438647+0000","last_undegraded":"2024-05-08T22:42:38.438647+0000","last_fullsized":"2024-05-08T22:42:38.438647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:24.067098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440820+0000","last_change":"2024-05-08T22:42:37.304044+0000","last_active":"2024-05-08T22:42:38.440820+0000","last_peered":"2024-05-08T22:42:38.440820+0000","last_clean":"2024-05-08T22:42:38.440820+0000","last_became_active":"2024-05-08T22:42:37.303790+0000","last_became_peered":"2024-05-08T22:42:37.303790+0000","last_unstale":"2024-05-08T22:42:38.440820+0000","last_undegraded":"2024-05-08T22:42:38.440820+0000","last_fullsized":"2024-05-08T22:42:38.440820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:39.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301910+0000","last_change":"2024-05-08T22:42:36.923320+0000","last_active":"2024-05-08T22:42:38.301910+0000","last_peered":"2024-05-08T22:42:38.301910+0000","last_clean":"2024-05-08T22:42:38.301910+0000","last_became_active":"2024-05-08T22:42:36.923066+0000","last_became_peered":"2024-05-08T22:42:36.923066+0000","last_unstale":"2024-05-08T22:42:38.301910+0000","last_undegraded":"2024-05-08T22:42:38.301910+0000","last_fullsized":"2024-05-08T22:42:38.301910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:17.375368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441966+0000","last_change":"2024-05-08T22:42:37.309627+0000","last_active":"2024-05-08T22:42:38.441966+0000","last_peered":"2024-05-08T22:42:38.441966+0000","last_clean":"2024-05-08T22:42:38.441966+0000","last_became_active":"2024-05-08T22:42:37.309359+0000","last_became_peered":"2024-05-08T22:42:37.309359+0000","last_unstale":"2024-05-08T22:42:38.441966+0000","last_undegraded":"2024-05-08T22:42:38.441966+0000","last_fullsized":"2024-05-08T22:42:38.441966+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:10.400426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442947+0000","last_change":"2024-05-08T22:42:36.923085+0000","last_active":"2024-05-08T22:42:38.442947+0000","last_peered":"2024-05-08T22:42:38.442947+0000","last_clean":"2024-05-08T22:42:38.442947+0000","last_became_active":"2024-05-08T22:42:36.922799+0000","last_became_peered":"2024-05-08T22:42:36.922799+0000","last_unstale":"2024-05-08T22:42:38.442947+0000","last_undegraded":"2024-05-08T22:42:38.442947+0000","last_fullsized":"2024-05-08T22:42:38.442947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:34.124192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442457+0000","last_change":"2024-05-08T22:42:37.304399+0000","last_active":"2024-05-08T22:42:38.442457+0000","last_peered":"2024-05-08T22:42:38.442457+0000","last_clean":"2024-05-08T22:42:38.442457+0000","last_became_active":"2024-05-08T22:42:37.304154+0000","last_became_peered":"2024-05-08T22:42:37.304154+0000","last_unstale":"2024-05-08T22:42:38.442457+0000","last_undegraded":"2024-05-08T22:42:38.442457+0000","last_fullsized":"2024-05-08T22:42:38.442457+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:00.543896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440255+0000","last_change":"2024-05-08T22:42:36.920391+0000","last_active":"2024-05-08T22:42:38.440255+0000","last_peered":"2024-05-08T22:42:38.440255+0000","last_clean":"2024-05-08T22:42:38.440255+0000","last_became_active":"2024-05-08T22:42:36.919986+0000","last_became_peered":"2024-05-08T22:42:36.919986+0000","last_unstale":"2024-05-08T22:42:38.440255+0000","last_undegraded":"2024-05-08T22:42:38.440255+0000","last_fullsized":"2024-05-08T22:42:38.440255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:48.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301845+0000","last_change":"2024-05-08T22:42:37.304078+0000","last_active":"2024-05-08T22:42:38.301845+0000","last_peered":"2024-05-08T22:42:38.301845+0000","last_clean":"2024-05-08T22:42:38.301845+0000","last_became_active":"2024-05-08T22:42:37.303811+0000","last_became_peered":"2024-05-08T22:42:37.303811+0000","last_unstale":"2024-05-08T22:42:38.301845+0000","last_undegraded":"2024-05-08T22:42:38.301845+0000","last_fullsized":"2024-05-08T22:42:38.301845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.854440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300680+0000","last_change":"2024-05-08T22:42:36.921185+0000","last_active":"2024-05-08T22:42:38.300680+0000","last_peered":"2024-05-08T22:42:38.300680+0000","last_clean":"2024-05-08T22:42:38.300680+0000","last_became_active":"2024-05-08T22:42:36.920918+0000","last_became_peered":"2024-05-08T22:42:36.920918+0000","last_unstale":"2024-05-08T22:42:38.300680+0000","last_undegraded":"2024-05-08T22:42:38.300680+0000","last_fullsized":"2024-05-08T22:42:38.300680+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.748812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443426+0000","last_change":"2024-05-08T22:42:37.305300+0000","last_active":"2024-05-08T22:42:38.443426+0000","last_peered":"2024-05-08T22:42:38.443426+0000","last_clean":"2024-05-08T22:42:38.443426+0000","last_became_active":"2024-05-08T22:42:37.305059+0000","last_became_peered":"2024-05-08T22:42:37.305059+0000","last_unstale":"2024-05-08T22:42:38.443426+0000","last_undegraded":"2024-05-08T22:42:38.443426+0000","last_fullsized":"2024-05-08T22:42:38.443426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:43.420046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440870+0000","last_change":"2024-05-08T22:42:36.919970+0000","last_active":"2024-05-08T22:42:38.440870+0000","last_peered":"2024-05-08T22:42:38.440870+0000","last_clean":"2024-05-08T22:42:38.440870+0000","last_became_active":"2024-05-08T22:42:36.919666+0000","last_became_peered":"2024-05-08T22:42:36.919666+0000","last_unstale":"2024-05-08T22:42:38.440870+0000","last_undegraded":"2024-05-08T22:42:38.440870+0000","last_fullsized":"2024-05-08T22:42:38.440870+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:12.546724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.441602+0000","last_change":"2024-05-08T22:42:37.309393+0000","last_active":"2024-05-08T22:42:38.441602+0000","last_peered":"2024-05-08T22:42:38.441602+0000","last_clean":"2024-05-08T22:42:38.441602+0000","last_became_active":"2024-05-08T22:42:37.306964+0000","last_became_peered":"2024-05-08T22:42:37.306964+0000","last_unstale":"2024-05-08T22:42:38.441602+0000","last_undegraded":"2024-05-08T22:42:38.441602+0000","last_fullsized":"2024-05-08T22:42:38.441602+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:29.588589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440411+0000","last_change":"2024-05-08T22:42:36.925141+0000","last_active":"2024-05-08T22:42:38.440411+0000","last_peered":"2024-05-08T22:42:38.440411+0000","last_clean":"2024-05-08T22:42:38.440411+0000","last_became_active":"2024-05-08T22:42:36.924777+0000","last_became_peered":"2024-05-08T22:42:36.924777+0000","last_unstale":"2024-05-08T22:42:38.440411+0000","last_undegraded":"2024-05-08T22:42:38.440411+0000","last_fullsized":"2024-05-08T22:42:38.440411+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:23.633428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.315161+0000","last_change":"2024-05-08T22:42:37.313279+0000","last_active":"2024-05-08T22:42:37.315161+0000","last_peered":"2024-05-08T22:42:37.315161+0000","last_clean":"2024-05-08T22:42:37.315161+0000","last_became_active":"2024-05-08T22:42:37.312937+0000","last_became_peered":"2024-05-08T22:42:37.312937+0000","last_unstale":"2024-05-08T22:42:37.315161+0000","last_undegraded":"2024-05-08T22:42:37.315161+0000","last_fullsized":"2024-05-08T22:42:37.315161+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:21.612995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442277+0000","last_change":"2024-05-08T22:42:36.923145+0000","last_active":"2024-05-08T22:42:38.442277+0000","last_peered":"2024-05-08T22:42:38.442277+0000","last_clean":"2024-05-08T22:42:38.442277+0000","last_became_active":"2024-05-08T22:42:36.922855+0000","last_became_peered":"2024-05-08T22:42:36.922855+0000","last_unstale":"2024-05-08T22:42:38.442277+0000","last_undegraded":"2024-05-08T22:42:38.442277+0000","last_fullsized":"2024-05-08T22:42:38.442277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:54.782437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301530+0000","last_change":"2024-05-08T22:42:37.303260+0000","last_active":"2024-05-08T22:42:38.301530+0000","last_peered":"2024-05-08T22:42:38.301530+0000","last_clean":"2024-05-08T22:42:38.301530+0000","last_became_active":"2024-05-08T22:42:37.303030+0000","last_became_peered":"2024-05-08T22:42:37.303030+0000","last_unstale":"2024-05-08T22:42:38.301530+0000","last_undegraded":"2024-05-08T22:42:38.301530+0000","last_fullsized":"2024-05-08T22:42:38.301530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:14.217428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301563+0000","last_change":"2024-05-08T22:42:36.915921+0000","last_active":"2024-05-08T22:42:38.301563+0000","last_peered":"2024-05-08T22:42:38.301563+0000","last_clean":"2024-05-08T22:42:38.301563+0000","last_became_active":"2024-05-08T22:42:36.914498+0000","last_became_peered":"2024-05-08T22:42:36.914498+0000","last_unstale":"2024-05-08T22:42:38.301563+0000","last_undegraded":"2024-05-08T22:42:38.301563+0000","last_fullsized":"2024-05-08T22:42:38.301563+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:23.633428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.301664+0000","last_change":"2024-05-08T22:42:37.316803+0000","last_active":"2024-05-08T22:42:38.301664+0000","last_peered":"2024-05-08T22:42:38.301664+0000","last_clean":"2024-05-08T22:42:38.301664+0000","last_became_active":"2024-05-08T22:42:37.315963+0000","last_became_peered":"2024-05-08T22:42:37.315963+0000","last_unstale":"2024-05-08T22:42:38.301664+0000","last_undegraded":"2024-05-08T22:42:38.301664+0000","last_fullsized":"2024-05-08T22:42:38.301664+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.696314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442076+0000","last_change":"2024-05-08T22:42:36.924014+0000","last_active":"2024-05-08T22:42:38.442076+0000","last_peered":"2024-05-08T22:42:38.442076+0000","last_clean":"2024-05-08T22:42:38.442076+0000","last_became_active":"2024-05-08T22:42:36.923727+0000","last_became_peered":"2024-05-08T22:42:36.923727+0000","last_unstale":"2024-05-08T22:42:38.442076+0000","last_undegraded":"2024-05-08T22:42:38.442076+0000","last_fullsized":"2024-05-08T22:42:38.442076+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:02.450478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442963+0000","last_change":"2024-05-08T22:42:37.303200+0000","last_active":"2024-05-08T22:42:38.442963+0000","last_peered":"2024-05-08T22:42:38.442963+0000","last_clean":"2024-05-08T22:42:38.442963+0000","last_became_active":"2024-05-08T22:42:37.302399+0000","last_became_peered":"2024-05-08T22:42:37.302399+0000","last_unstale":"2024-05-08T22:42:38.442963+0000","last_undegraded":"2024-05-08T22:42:38.442963+0000","last_fullsized":"2024-05-08T22:42:38.442963+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:08.339236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.291854+0000","last_change":"2024-05-08T22:42:36.913424+0000","last_active":"2024-05-08T22:42:37.291854+0000","last_peered":"2024-05-08T22:42:37.291854+0000","last_clean":"2024-05-08T22:42:37.291854+0000","last_became_active":"2024-05-08T22:42:36.913110+0000","last_became_peered":"2024-05-08T22:42:36.913110+0000","last_unstale":"2024-05-08T22:42:37.291854+0000","last_undegraded":"2024-05-08T22:42:37.291854+0000","last_fullsized":"2024-05-08T22:42:37.291854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:00.855199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.442856+0000","last_change":"2024-05-08T22:42:37.302873+0000","last_active":"2024-05-08T22:42:38.442856+0000","last_peered":"2024-05-08T22:42:38.442856+0000","last_clean":"2024-05-08T22:42:38.442856+0000","last_became_active":"2024-05-08T22:42:37.302529+0000","last_became_peered":"2024-05-08T22:42:37.302529+0000","last_unstale":"2024-05-08T22:42:38.442856+0000","last_undegraded":"2024-05-08T22:42:38.442856+0000","last_fullsized":"2024-05-08T22:42:38.442856+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:54.233330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440564+0000","last_change":"2024-05-08T22:42:36.920235+0000","last_active":"2024-05-08T22:42:38.440564+0000","last_peered":"2024-05-08T22:42:38.440564+0000","last_clean":"2024-05-08T22:42:38.440564+0000","last_became_active":"2024-05-08T22:42:36.919639+0000","last_became_peered":"2024-05-08T22:42:36.919639+0000","last_unstale":"2024-05-08T22:42:38.440564+0000","last_undegraded":"2024-05-08T22:42:38.440564+0000","last_fullsized":"2024-05-08T22:42:38.440564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.438607+0000","last_change":"2024-05-08T22:42:37.317924+0000","last_active":"2024-05-08T22:42:38.438607+0000","last_peered":"2024-05-08T22:42:38.438607+0000","last_clean":"2024-05-08T22:42:38.438607+0000","last_became_active":"2024-05-08T22:42:37.317653+0000","last_became_peered":"2024-05-08T22:42:37.317653+0000","last_unstale":"2024-05-08T22:42:38.438607+0000","last_undegraded":"2024-05-08T22:42:38.438607+0000","last_fullsized":"2024-05-08T22:42:38.438607+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:51.007832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300889+0000","last_change":"2024-05-08T22:42:36.914398+0000","last_active":"2024-05-08T22:42:38.300889+0000","last_peered":"2024-05-08T22:42:38.300889+0000","last_clean":"2024-05-08T22:42:38.300889+0000","last_became_active":"2024-05-08T22:42:36.914115+0000","last_became_peered":"2024-05-08T22:42:36.914115+0000","last_unstale":"2024-05-08T22:42:38.300889+0000","last_undegraded":"2024-05-08T22:42:38.300889+0000","last_fullsized":"2024-05-08T22:42:38.300889+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:02.450478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.302266+0000","last_change":"2024-05-08T22:42:37.318186+0000","last_active":"2024-05-08T22:42:38.302266+0000","last_peered":"2024-05-08T22:42:38.302266+0000","last_clean":"2024-05-08T22:42:38.302266+0000","last_became_active":"2024-05-08T22:42:37.317901+0000","last_became_peered":"2024-05-08T22:42:37.317901+0000","last_unstale":"2024-05-08T22:42:38.302266+0000","last_undegraded":"2024-05-08T22:42:38.302266+0000","last_fullsized":"2024-05-08T22:42:38.302266+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:48.137194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.440825+0000","last_change":"2024-05-08T22:42:36.914185+0000","last_active":"2024-05-08T22:42:38.440825+0000","last_peered":"2024-05-08T22:42:38.440825+0000","last_clean":"2024-05-08T22:42:38.440825+0000","last_became_active":"2024-05-08T22:42:36.913921+0000","last_became_peered":"2024-05-08T22:42:36.913921+0000","last_unstale":"2024-05-08T22:42:38.440825+0000","last_undegraded":"2024-05-08T22:42:38.440825+0000","last_fullsized":"2024-05-08T22:42:38.440825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:11.434992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.300587+0000","last_change":"2024-05-08T22:42:37.316704+0000","last_active":"2024-05-08T22:42:38.300587+0000","last_peered":"2024-05-08T22:42:38.300587+0000","last_clean":"2024-05-08T22:42:38.300587+0000","last_became_active":"2024-05-08T22:42:37.316448+0000","last_became_peered":"2024-05-08T22:42:37.316448+0000","last_unstale":"2024-05-08T22:42:38.300587+0000","last_undegraded":"2024-05-08T22:42:38.300587+0000","last_fullsized":"2024-05-08T22:42:38.300587+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:36.887541+0000","last_clean_scrub_stamp":"2024-05-08T22:42:36.887541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:24.832903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"111'2","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T22:42:38.443198+0000","last_change":"2024-05-08T22:42:36.919823+0000","last_active":"2024-05-08T22:42:38.443198+0000","last_peered":"2024-05-08T22:42:38.443198+0000","last_clean":"2024-05-08T22:42:38.443198+0000","last_became_active":"2024-05-08T22:42:36.919539+0000","last_became_peered":"2024-05-08T22:42:36.919539+0000","last_unstale":"2024-05-08T22:42:38.443198+0000","last_undegraded":"2024-05-08T22:42:38.443198+0000","last_fullsized":"2024-05-08T22:42:38.443198+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:42:35.875851+0000","last_clean_scrub_stamp":"2024-05-08T22:42:35.875851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:08.528138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":248,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-08T22:42:37.289960+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:42:37.289960+0000","last_peered":"2024-05-08T22:42:37.289960+0000","last_clean":"2024-05-08T22:42:37.289960+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:42:37.289960+0000","last_undegraded":"2024-05-08T22:42:37.289960+0000","last_fullsized":"2024-05-08T22:42:37.289960+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1384448,"data_stored":637078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.151,"5min":1.623,"15min":1.623},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.309,"15min":0.307},"max":{"1min":1.115,"5min":1.214,"15min":1.6899999999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.623},"min":{"1min":0.377,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.179,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42199999999999999},"max":{"1min":1.05,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.82299999999999995,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2230000000000001,"5min":1.716,"15min":1.716},"last":0.96799999999999997}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.90900000000000003},"min":{"1min":0.57699999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.306},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.94999999999999996},"min":{"1min":0.71099999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.248,"5min":1.3320000000000001,"15min":1.4139999999999999},"last":1.077}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.77400000000000002,"5min":0.70399999999999996,"15min":0.68300000000000005},"max":{"1min":1.171,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.026,"15min":1.024},"min":{"1min":0.69899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3560000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.046}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.82599999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.421,"5min":1.421,"15min":1.446},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.069,"15min":1.0609999999999999},"min":{"1min":0.79200000000000004,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4490000000000001,"5min":1.512,"15min":1.512},"last":1.006}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.76400000000000001,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.389,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.111},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0549999999999999,"15min":1.0409999999999999},"min":{"1min":0.81499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.476,"5min":1.587,"15min":1.8360000000000001},"last":1.25}]}]},{"osd":1,"up_from":15,"seq":64424509558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1384448,"data_stored":636530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46100000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.98399999999999999,"5min":1.772,"15min":1.772},"last":0.875},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.70299999999999996},"min":{"1min":0.47799999999999998,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":1.1659999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1539999999999999,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81999999999999995,"15min":0.76200000000000001},"min":{"1min":0.55400000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85699999999999998,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.228,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71599999999999997,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.42599999999999999,"15min":0.35599999999999998},"max":{"1min":1.0609999999999999,"5min":1.1579999999999999,"15min":1.583},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.95999999999999996},"min":{"1min":0.56999999999999995,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.202,"5min":1.395,"15min":1.431},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0249999999999999,"15min":0.98699999999999999},"min":{"1min":0.80100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.248,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.5},"last":1.121},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0189999999999999,"15min":1.052},"min":{"1min":0.69799999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.399,"5min":1.599,"15min":1.726},"last":0.98199999999999998}]},{"osd":6,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.60899999999999999},"max":{"1min":1.2110000000000001,"5min":1.351,"15min":1.4650000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.70799999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3520000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.034}]},{"osd":7,"last update":"Wed May 8 22:42:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.087,"15min":1.073},"min":{"1min":0.89200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.0549999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5049999999999999},"last":1.2250000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1970176,"data_stored":1220404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83699999999999997,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1120000000000001,"5min":1.232,"15min":1.282},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.879,"15min":0.86299999999999999},"min":{"1min":0.626,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2869999999999999,"5min":1.514,"15min":1.514},"last":1.0669999999999999}]},{"osd":1,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86199999999999999,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.026,"5min":1.355,"15min":1.4470000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.92800000000000005,"15min":0.88200000000000001},"min":{"1min":0.72199999999999998,"5min":0.58199999999999996,"15min":0.443},"max":{"1min":1.2130000000000001,"5min":1.2669999999999999,"15min":1.488},"last":1.0169999999999999}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91500000000000004,"15min":0.90600000000000003},"min":{"1min":0.78000000000000003,"5min":0.625,"15min":0.58199999999999996},"max":{"1min":1.101,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.95399999999999996,"15min":0.95699999999999996},"min":{"1min":0.65700000000000003,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.0840000000000001,"5min":1.2669999999999999,"15min":1.337},"last":1.1080000000000001}]},{"osd":3,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.92100000000000004},"min":{"1min":0.77700000000000002,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.2190000000000001,"5min":1.3919999999999999,"15min":1.5589999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.74199999999999999,"5min":0.63600000000000001,"15min":0.58199999999999996},"max":{"1min":1.228,"5min":1.228,"15min":1.2290000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.153,"5min":1.611,"15min":1.611},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.874,"15min":0.88500000000000001},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.57399999999999995},"max":{"1min":1.165,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94099999999999995}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.53300000000000003},"max":{"1min":1.1100000000000001,"5min":1.2629999999999999,"15min":1.482},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.056,"15min":1.0509999999999999},"min":{"1min":0.93500000000000005,"5min":0.71299999999999997,"15min":0.60199999999999998},"max":{"1min":1.2090000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.147}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":0.94499999999999995},"min":{"1min":0.71799999999999997,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.27,"5min":1.379,"15min":1.3839999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.1919999999999999,"5min":1.514,"15min":1.599},"last":1.1259999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1388544,"data_stored":632044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.038,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.54200000000000004},"max":{"1min":1.0680000000000001,"5min":1.085,"15min":1.141},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81299999999999994,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.40500000000000003},"max":{"1min":1.099,"5min":1.1299999999999999,"15min":1.758},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92500000000000004,"15min":0.92600000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":1.028}]},{"osd":2,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.94999999999999996},"min":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.49099999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.4319999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.91300000000000003},"min":{"1min":0.68000000000000005,"5min":0.624,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.236,"15min":1.419},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.94299999999999995,"15min":1.0349999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.1359999999999999,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.93000000000000005,"15min":0.96299999999999997},"min":{"1min":0.70899999999999996,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.5029999999999999},"last":1.069}]},{"osd":4,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91200000000000003,"15min":0.96599999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.48599999999999999},"max":{"1min":1.3,"5min":1.3320000000000001,"15min":1.538},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94799999999999995,"15min":1.004},"min":{"1min":0.65300000000000002,"5min":0.629,"15min":0.44400000000000001},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.458},"last":1.157}]},{"osd":5,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.022,"15min":1.093},"min":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3460000000000001,"5min":1.427,"15min":1.681},"last":1.278},{"interface":"front","average":{"1min":1.042,"5min":1.044,"15min":1.099},"min":{"1min":0.83399999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2909999999999999,"5min":1.393,"15min":1.6000000000000001},"last":1.228}]},{"osd":7,"last update":"Wed May 8 22:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.94999999999999996,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.252,"5min":1.3580000000000001,"15min":1.835},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.127,"15min":1.1899999999999999},"min":{"1min":0.85199999999999998,"5min":0.748,"15min":0.74099999999999999},"max":{"1min":1.4419999999999999,"5min":1.5600000000000001,"15min":2.2999999999999998},"last":1.341}]}]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1368064,"data_stored":629876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.90000000000000002,"15min":0.84099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":7.907,"15min":7.907},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.157,"5min":1.29,"15min":1.52},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.1259999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89100000000000001,"15min":0.84199999999999997},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.44400000000000001},"max":{"1min":1.097,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.97399999999999998}]},{"osd":2,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.89300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.119,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0109999999999999,"15min":0.90000000000000002},"min":{"1min":0.66400000000000003,"5min":0.58099999999999996,"15min":0.48799999999999999},"max":{"1min":1.212,"5min":10.121,"15min":10.121},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.874,"15min":0.876},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.1579999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.94399999999999995,"15min":0.95999999999999996},"min":{"1min":0.64000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.238,"5min":1.298,"15min":1.298},"last":1.071}]},{"osd":4,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.091,"15min":0.97499999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.0880000000000001,"5min":15.135,"15min":15.135},"last":1.171},{"interface":"front","average":{"1min":1.006,"5min":1.056,"15min":1.014},"min":{"1min":0.73099999999999998,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.208,"5min":4.319,"15min":4.319},"last":1.3680000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.93600000000000005},"min":{"1min":0.64900000000000002,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.605},"last":1.347},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.85699999999999998,"5min":0.67100000000000004,"15min":0.54900000000000004},"max":{"1min":1.2230000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.3160000000000001}]},{"osd":7,"last update":"Wed May 8 22:42:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1080000000000001,"15min":1.097},"min":{"1min":0.88500000000000001,"5min":0.71399999999999997,"15min":0.57299999999999995},"max":{"1min":1.2709999999999999,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0489999999999999},"min":{"1min":0.80600000000000005,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.3220000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.29}]}]},{"osd":4,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1355776,"data_stored":622781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0660000000000001,"5min":1.242,"15min":1.28},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80800000000000005,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.081,"5min":1.1739999999999999,"15min":1.294},"last":0.67800000000000005}]},{"osd":1,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.624,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1990000000000001,"5min":1.2290000000000001,"15min":1.3080000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.109,"5min":1.292,"15min":1.3140000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88200000000000001,"15min":0.88100000000000001},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.1040000000000001,"5min":1.244,"15min":1.29},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.90600000000000003,"15min":0.92200000000000004},"min":{"1min":0.69999999999999996,"5min":0.57999999999999996,"15min":0.53400000000000003},"max":{"1min":1.1559999999999999,"5min":1.1779999999999999,"15min":1.637},"last":0.80800000000000005}]},{"osd":3,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93500000000000005,"15min":0.995},"min":{"1min":0.78800000000000003,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.141,"5min":1.319,"15min":2.601},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93300000000000005,"15min":1.0349999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.159,"5min":1.22,"15min":2.8769999999999998},"last":0.76400000000000001}]},{"osd":5,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91200000000000003,"15min":0.98699999999999999},"min":{"1min":0.69099999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.792},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.94099999999999995,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.125,"5min":1.3240000000000001,"15min":1.575},"last":1.1140000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97999999999999998,"15min":1.046},"min":{"1min":0.68000000000000005,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.28,"5min":1.377,"15min":1.831},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.01,"5min":1.0269999999999999,"15min":1.0760000000000001},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1719999999999999,"5min":1.3620000000000001,"15min":1.8899999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Wed May 8 22:42:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99299999999999999,"15min":1.0620000000000001},"min":{"1min":0.88100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3080000000000001,"5min":1.403,"15min":1.8500000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1080000000000001,"15min":1.151},"min":{"1min":0.97999999999999998,"5min":0.78000000000000003,"15min":0.76000000000000001},"max":{"1min":1.369,"5min":1.5169999999999999,"15min":1.944},"last":1.0069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542261,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":629910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.627},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0089999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.193},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.45600000000000002,"5min":0.39700000000000002,"15min":0.39300000000000002},"max":{"1min":0.94699999999999995,"5min":1.123,"15min":1.2190000000000001},"last":0.622}]},{"osd":1,"last update":"Wed May 8 22:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96899999999999997,"5min":1.048,"15min":1.048},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.437,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.3460000000000001},"last":0.23599999999999999}]},{"osd":3,"last update":"Wed May 8 22:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.024,"5min":1.071,"15min":1.071},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77800000000000002,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1719999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.75,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.087,"5min":1.504,"15min":1.504},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97099999999999997,"15min":1.0049999999999999},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.66500000000000004},"max":{"1min":1.2,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.32000000000000001}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":1.0189999999999999},"min":{"1min":0.70399999999999996,"5min":0.433,"15min":0.433},"max":{"1min":1.294,"5min":1.3779999999999999,"15min":1.484},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97999999999999998,"15min":1.004},"min":{"1min":0.79100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2769999999999999,"5min":1.343,"15min":1.456},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0269999999999999,"15min":1.0469999999999999},"min":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.69599999999999995},"max":{"1min":1.1359999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1,"5min":1.0509999999999999,"15min":1.071},"min":{"1min":0.66900000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1850000000000001,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0549999999999999,"15min":1.075},"min":{"1min":0.78300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.109},"min":{"1min":0.64900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.61099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1978368,"data_stored":1221898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60299999999999998,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97099999999999997,"5min":0.97199999999999998,"15min":1.617},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.42099999999999999,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.113,"15min":1.5129999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.54100000000000004,"5min":0.49099999999999999,"15min":0.44500000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.446},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.436},"max":{"1min":1.101,"5min":1.4410000000000001,"15min":1.6910000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed May 8 22:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.79300000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.999,"5min":1.609,"15min":1.714},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.70999999999999996},"min":{"1min":0.497,"5min":0.44,"15min":0.44},"max":{"1min":0.90900000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.63700000000000001,"15min":0.61699999999999999},"max":{"1min":1.0940000000000001,"5min":4.391,"15min":4.391},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.94899999999999995,"15min":0.97899999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1100000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.93799999999999994}]},{"osd":5,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91900000000000004,"15min":0.91200000000000003},"min":{"1min":0.72799999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.246,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.167,"5min":1.4379999999999999,"15min":1.474},"last":1.0620000000000001}]},{"osd":6,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1879999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.071,"15min":1.0509999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.66300000000000003},"max":{"1min":1.24,"5min":4.4859999999999998,"15min":4.4859999999999998},"last":1.109}]},{"osd":7,"last update":"Wed May 8 22:42:20 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0509999999999999,"15min":1.073},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":1.3080000000000001,"5min":1.532,"15min":1.6459999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0349999999999999,"15min":1.024},"min":{"1min":0.80200000000000005,"5min":0.79000000000000004,"15min":0.60399999999999998},"max":{"1min":1.264,"5min":1.514,"15min":1.514},"last":1.0009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:42:41.698 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:42:41.698 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:42:41.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 Processor -- start 2024-05-08T22:42:41.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 -- start start 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb7380d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb7380ce2b0 0x7fb7380d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 0x7fb7380cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7380ce0b0 con 0x7fb7380ce2b0 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7380d2c90 con 0x7fb7380cd710 2024-05-08T22:42:41.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.822+0000 7fb73dc3e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7380d16b0 con 0x7fb73815d130 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb7380ce2b0 0x7fb7380d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb7380ce2b0 0x7fb7380d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37452/0 (socket says 172.21.15.179:37452) 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73d43d700 1 --2- 172.21.15.179:0/289933308 >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 0x7fb7380cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 -- 172.21.15.179:0/289933308 learned_addr learned my addr 172.21.15.179:0/289933308 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73cc3c700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb7380d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 -- 172.21.15.179:0/289933308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 msgr2=0x7fb7380d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb7380d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 -- 172.21.15.179:0/289933308 >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 msgr2=0x7fb7380cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 --2- 172.21.15.179:0/289933308 >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 0x7fb7380cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 -- 172.21.15.179:0/289933308 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380d1990 con 0x7fb7380ce2b0 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73d43d700 1 --2- 172.21.15.179:0/289933308 >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 0x7fb7380cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:41.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb737fff700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb7380d0db0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb7380c9ee0 tx=0x7fb724009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8068a5f6ab2f6c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb7377fe700 1 -- 172.21.15.179:0/289933308 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011830 con 0x7fb7380ce2b0 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb7377fe700 1 -- 172.21.15.179:0/289933308 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724011990 con 0x7fb7380ce2b0 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb7377fe700 1 -- 172.21.15.179:0/289933308 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011d20 con 0x7fb7380ce2b0 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 -- 172.21.15.179:0/289933308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 msgr2=0x7fb7380d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb7380d0db0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb7380c9ee0 tx=0x7fb724009bf0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 -- 172.21.15.179:0/289933308 shutdown_connections 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/289933308 >> v2:172.21.15.195:3300/0 conn(0x7fb7380cd710 0x7fb7380cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb7380d0db0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/289933308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb7380d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 -- 172.21.15.179:0/289933308 >> 172.21.15.179:0/289933308 conn(0x7fb7380b49a0 msgr2=0x7fb7380b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 -- 172.21.15.179:0/289933308 shutdown_connections 2024-05-08T22:42:41.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.826+0000 7fb73dc3e700 1 -- 172.21.15.179:0/289933308 wait complete. 2024-05-08T22:42:41.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 Processor -- start 2024-05-08T22:42:41.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- start start 2024-05-08T22:42:41.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 0x7fb7381546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb738151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73815eda0 con 0x7fb7380ce2b0 2024-05-08T22:42:41.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb738158d10 con 0x7fb7380cd710 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb738158ff0 con 0x7fb73815d130 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73cc3c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 0x7fb7381546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37464/0 (socket says 172.21.15.179:37464) 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 -- 172.21.15.179:0/2303777238 learned_addr learned my addr 172.21.15.179:0/2303777238 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73d43d700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb738151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 -- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 msgr2=0x7fb738151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb738151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 -- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 msgr2=0x7fb7381546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 0x7fb7381546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 -- 172.21.15.179:0/2303777238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730009a70 con 0x7fb7380ce2b0 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73cc3c700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 0x7fb7381546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:41.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb737fff700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fb724009860 tx=0x7fb724004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724004240 con 0x7fb7380ce2b0 2024-05-08T22:42:41.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72401f070 con 0x7fb7380ce2b0 2024-05-08T22:42:41.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb724009570 con 0x7fb7380ce2b0 2024-05-08T22:42:41.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724004600 con 0x7fb7380ce2b0 2024-05-08T22:42:41.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.830+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7381595b0 con 0x7fb7380ce2b0 2024-05-08T22:42:41.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73815ef70 con 0x7fb7380ce2b0 2024-05-08T22:42:41.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb724021030 con 0x7fb7380ce2b0 2024-05-08T22:42:41.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb7357fa700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 0x7fb71c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:41.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb7240852b0 con 0x7fb7380ce2b0 2024-05-08T22:42:41.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb73cc3c700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 0x7fb71c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:41.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.834+0000 7fb73cc3c700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 0x7fb71c068c40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009f60 tx=0x7fb72c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:41.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:41.842+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb7240525c0 con 0x7fb7380ce2b0 2024-05-08T22:42:42.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.074+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb73804e700 con 0x7fb7380ce2b0 2024-05-08T22:42:42.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb7357fa700 1 -- 172.21.15.179:0/2303777238 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb724055be0 con 0x7fb7380ce2b0 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 msgr2=0x7fb71c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 0x7fb71c068c40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009f60 tx=0x7fb72c009240 comp rx=0 tx=0).stop 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 msgr2=0x7fb738155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fb724009860 tx=0x7fb724004770 comp rx=0 tx=0).stop 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 shutdown_connections 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb71c066780 0x7fb71c068c40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb73815d130 0x7fb738151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7380ce2b0 0x7fb738155b50 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 --2- 172.21.15.179:0/2303777238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7380cd710 0x7fb7381546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 >> 172.21.15.179:0/2303777238 conn(0x7fb7380b49a0 msgr2=0x7fb7380cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:42.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 shutdown_connections 2024-05-08T22:42:42.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.082+0000 7fb73dc3e700 1 -- 172.21.15.179:0/2303777238 wait complete. 2024-05-08T22:42:42.087 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:42:42.101 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:42.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 Processor -- start 2024-05-08T22:42:42.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 -- start start 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf9015d0c0 0x7faf900ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faf900b4c90 0x7faf9006abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf900b3f60 con 0x7faf900cea50 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf900ceed0 con 0x7faf900b4c90 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf97054700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900b41b0 con 0x7faf9015d0c0 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf96853700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faf900b4c90 0x7faf9006abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf96853700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faf900b4c90 0x7faf9006abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37472/0 (socket says 172.21.15.179:37472) 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf95851700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7faf900cea50 0x7faf9006ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:57868/0 (socket says 172.21.15.179:57868) 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf96853700 1 -- 172.21.15.179:0/870554850 learned_addr learned my addr 172.21.15.179:0/870554850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:42.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.214+0000 7faf96052700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf9015d0c0 0x7faf900ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf96853700 1 -- 172.21.15.179:0/870554850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf9015d0c0 msgr2=0x7faf900ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf96853700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf9015d0c0 0x7faf900ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf96853700 1 -- 172.21.15.179:0/870554850 >> v1:172.21.15.195:6789/0 conn(0x7faf900cea50 legacy=0x7faf9006ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf96853700 1 -- 172.21.15.179:0/870554850 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900b4490 con 0x7faf900b4c90 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf96853700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900b4c90 0x7faf9006abb0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b421ec2eedad188c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf95050700 1 -- 172.21.15.179:0/870554850 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8800e070 con 0x7faf900b4c90 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf95050700 1 -- 172.21.15.179:0/870554850 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8800b910 con 0x7faf900b4c90 2024-05-08T22:42:42.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf95050700 1 -- 172.21.15.179:0/870554850 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8800bcc0 con 0x7faf900b4c90 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- 172.21.15.179:0/870554850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900b4c90 msgr2=0x7faf9006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900b4c90 0x7faf9006abb0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- 172.21.15.179:0/870554850 shutdown_connections 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900b4c90 0x7faf9006abb0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 --2- 172.21.15.179:0/870554850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf9015d0c0 0x7faf900ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- 172.21.15.179:0/870554850 >> 172.21.15.179:0/870554850 conn(0x7faf90098a70 msgr2=0x7faf900b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- 172.21.15.179:0/870554850 shutdown_connections 2024-05-08T22:42:42.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- 172.21.15.179:0/870554850 wait complete. 2024-05-08T22:42:42.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 Processor -- start 2024-05-08T22:42:42.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.218+0000 7faf97054700 1 -- start start 2024-05-08T22:42:42.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf900b4c90 0x7faf9015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf9015d0c0 0x7faf900c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900c95e0 con 0x7faf900cea50 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900ca970 con 0x7faf9015d0c0 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900cac50 con 0x7faf900b4c90 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37488/0 (socket says 172.21.15.179:37488) 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf96052700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf900b4c90 0x7faf9015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf96853700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf9015d0c0 0x7faf900c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 -- 172.21.15.179:0/2458164860 learned_addr learned my addr 172.21.15.179:0/2458164860 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:42.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 -- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf900b4c90 msgr2=0x7faf9015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf900b4c90 0x7faf9015a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 -- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf9015d0c0 msgr2=0x7faf900c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf9015d0c0 0x7faf900c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 -- 172.21.15.179:0/2458164860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009b30 con 0x7faf900cea50 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf95851700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c500 tx=0x7faf8000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80005cd0 con 0x7faf900cea50 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf80005e30 con 0x7faf900cea50 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf900caf30 con 0x7faf900cea50 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf800072f0 con 0x7faf900cea50 2024-05-08T22:42:42.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.222+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf900cb4f0 con 0x7faf900cea50 2024-05-08T22:42:42.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf80007450 con 0x7faf900cea50 2024-05-08T22:42:42.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf900cb710 con 0x7faf900cea50 2024-05-08T22:42:42.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf86ffd700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 0x7faf74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faf80083670 con 0x7faf900cea50 2024-05-08T22:42:42.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf96052700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 0x7faf74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.226+0000 7faf96052700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 0x7faf74068c20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009b50 tx=0x7faf8c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.234+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faf80050890 con 0x7faf900cea50 2024-05-08T22:42:42.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.434+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf9004e700 con 0x7faf900cea50 2024-05-08T22:42:42.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf86ffd700 1 -- 172.21.15.179:0/2458164860 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4748 (secure 0 0 0) 0x7faf80053eb0 con 0x7faf900cea50 2024-05-08T22:42:42.438 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:42.438 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:42:40.300797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57425":{"gid":57425,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/4059323811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4059323811},{"type":"v1","addr":"172.21.15.195:6837","nonce":4059323811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 msgr2=0x7faf74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 0x7faf74068c20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009b50 tx=0x7faf8c009240 comp rx=0 tx=0).stop 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 msgr2=0x7faf900c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c500 tx=0x7faf8000c810 comp rx=0 tx=0).stop 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 shutdown_connections 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7faf74066760 0x7faf74068c20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf9015d0c0 0x7faf900c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf900cea50 0x7faf900c6580 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 --2- 172.21.15.179:0/2458164860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf900b4c90 0x7faf9015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 >> 172.21.15.179:0/2458164860 conn(0x7faf90098a70 msgr2=0x7faf9006c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:42.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 shutdown_connections 2024-05-08T22:42:42.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.438+0000 7faf97054700 1 -- 172.21.15.179:0/2458164860 wait complete. 2024-05-08T22:42:42.442 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 75 2024-05-08T22:42:42.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:42:37.284354+0000', 'modified': '2024-05-08T22:42:40.300797+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57425': {'gid': 57425, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/4059323811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 4059323811}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 4059323811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:42:42.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:42:42.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:42:42.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 Processor -- start 2024-05-08T22:42:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 -- start start 2024-05-08T22:42:42.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4a00b3180 0x7ff4a00cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a015f350 con 0x7ff4a015eef0 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a00b37a0 con 0x7ff4a00b3180 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a54b5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a00b3a80 con 0x7ff4a00ad9d0 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4a00b3180 0x7ff4a00cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49f7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff4a015eef0 0x7ff4a00b2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36022/0 (socket says 172.21.15.179:36022) 2024-05-08T22:42:42.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49f7fe700 1 -- 172.21.15.179:0/254913483 learned_addr learned my addr 172.21.15.179:0/254913483 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49effd700 1 -- 172.21.15.179:0/254913483 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290624407 0 0) 0x7ff4a015f350 con 0x7ff4a015eef0 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49effd700 1 -- 172.21.15.179:0/254913483 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0035d0 con 0x7ff4a015eef0 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49ffff700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/254913483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 msgr2=0x7ff4a015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a015e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/254913483 >> v1:172.21.15.179:6789/0 conn(0x7ff4a015eef0 legacy=0x7ff4a00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/254913483 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a00b3d60 con 0x7ff4a00b3180 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a00b3180 0x7ff4a00cb9d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff498009d90 tx=0x7ff49800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b908966be1778a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49effd700 1 -- 172.21.15.179:0/254913483 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49800e070 con 0x7ff4a00b3180 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49effd700 1 -- 172.21.15.179:0/254913483 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49800bb20 con 0x7ff4a00b3180 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.978+0000 7ff49effd700 1 -- 172.21.15.179:0/254913483 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49800be90 con 0x7ff4a00b3180 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- 172.21.15.179:0/254913483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a00b3180 msgr2=0x7ff4a00cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a00b3180 0x7ff4a00cb9d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff498009d90 tx=0x7ff49800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:42.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- 172.21.15.179:0/254913483 shutdown_connections 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a00b3180 0x7ff4a00cb9d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/254913483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- 172.21.15.179:0/254913483 >> 172.21.15.179:0/254913483 conn(0x7ff4a00ce960 msgr2=0x7ff4a00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- 172.21.15.179:0/254913483 shutdown_connections 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- 172.21.15.179:0/254913483 wait complete. 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 Processor -- start 2024-05-08T22:42:42.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- start start 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a00af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 0x7ff4a0142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a015bf10 con 0x7ff4a015eef0 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a015b970 con 0x7ff4a00b3180 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a54b5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a015bc30 con 0x7ff4a00ad9d0 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37512/0 (socket says 172.21.15.179:37512) 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3561134860 learned_addr learned my addr 172.21.15.179:0/3561134860 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff49ffff700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a00af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff49f7fe700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 0x7ff4a0142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 msgr2=0x7ff4a00af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a00af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 msgr2=0x7ff4a0142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 0x7ff4a0142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3561134860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009a70 con 0x7ff4a015eef0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.982+0000 7ff49f7fe700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 0x7ff4a0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff498014040 tx=0x7ff498004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 -- 172.21.15.179:0/3561134860 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498007000 con 0x7ff4a015eef0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a00adf30 con 0x7ff4a015eef0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 -- 172.21.15.179:0/3561134860 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498012860 con 0x7ff4a015eef0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a00af160 con 0x7ff4a015eef0 2024-05-08T22:42:42.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 -- 172.21.15.179:0/3561134860 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498012bd0 con 0x7ff4a015eef0 2024-05-08T22:42:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 -- 172.21.15.179:0/3561134860 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff498012d30 con 0x7ff4a015eef0 2024-05-08T22:42:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 0x7ff480068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49cff9700 1 -- 172.21.15.179:0/3561134860 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff4980865f0 con 0x7ff4a015eef0 2024-05-08T22:42:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.986+0000 7ff49ffff700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 0x7ff480068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff49ffff700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 0x7ff480068bf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff4a015dcf0 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 Processor -- start 2024-05-08T22:42:42.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 -- start start 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1744ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x170c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x166bd80 con 0x1745420 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1669510 con 0x173bbe0 2024-05-08T22:42:42.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1664e00 con 0x1744b00 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff49ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1744ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff49f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x170c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37522/0 (socket says 172.21.15.179:37522) 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1754572475 learned_addr learned my addr 172.21.15.179:0/1754572475 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 msgr2=0x1744ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1744ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 msgr2=0x170c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x170c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1754572475 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14d7960 con 0x1745420 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.990+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff498053bc0 tx=0x7ff4980049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb7ccdfb0f94dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff495ffb700 1 -- 172.21.15.179:0/1754572475 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49805fa00 con 0x1745420 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff495ffb700 1 -- 172.21.15.179:0/1754572475 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498055800 con 0x1745420 2024-05-08T22:42:42.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff495ffb700 1 -- 172.21.15.179:0/1754572475 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49805b160 con 0x1745420 2024-05-08T22:42:42.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1754572475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 msgr2=0x170c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff498053bc0 tx=0x7ff4980049e0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1754572475 shutdown_connections 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x170c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x170c9b0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1754572475 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1744ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1754572475 >> 172.21.15.179:0/1754572475 conn(0x148c4d0 msgr2=0x1726250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1754572475 shutdown_connections 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1754572475 wait complete. 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 -- start start 2024-05-08T22:42:42.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.994+0000 7ff4a99c6740 1 Processor -- start 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x174abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x16feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1622730 con 0x1745420 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16285b0 con 0x173bbe0 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1510aa0 con 0x1744b00 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 10 client.0 _ll_register_callbacks cb 0x165e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff49ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x174abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37536/0 (socket says 172.21.15.179:37536) 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3188460741 learned_addr learned my addr 172.21.15.179:0/3188460741 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff49f7fe700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x16feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 msgr2=0x16feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x16feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 msgr2=0x174abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x174abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/3188460741 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a00af160 con 0x1745420 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ff498053bc0 tx=0x7ff498009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1db771af766a68e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49806f040 con 0x1745420 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff49805fa00 con 0x1745420 2024-05-08T22:42:43.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x14991e0 con 0x1745420 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff498056bb0 con 0x1745420 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 10 client.9024 fetch_fsmap learned FSMap version 75 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x162b220 con 0x1745420 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x14991e0 con 0x1745420 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff49805f570 con 0x1745420 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 10 client.9024 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 10 client.9024 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 20 client.9024 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x173caa0 0x173ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 4 client.9024 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x14e3460 con 0x173caa0 2024-05-08T22:42:43.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff49ffff700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x173caa0 0x173ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff49ffff700 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x173caa0 0x173ce80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec0fbbb449c357f server_cookie=98d0c1b27f78bd64 in_seq=0 out_seq=0 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:42.998+0000 7ff47effd700 10 client.9024 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff47effd700 1 -- 172.21.15.179:0/3188460741 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x14e3460 con 0x173caa0 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff47effd700 10 client.9024 handle_command_reply: tid=0 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 client.9024 shutdown 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 msgr2=0x16ff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ff498053bc0 tx=0x7ff498009750 comp rx=0 tx=0).stop 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 shutdown_connections 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x16ff0b0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x16feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x173caa0 0x173ce80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/3188460741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x174abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 >> 172.21.15.179:0/3188460741 conn(0x148c4d0 msgr2=0x1725820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff47effd700 0 client.9024 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:42:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 shutdown_connections 2024-05-08T22:42:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 1 -- 172.21.15.179:0/3188460741 wait complete. 2024-05-08T22:42:43.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.002+0000 7ff4a99c6740 20 client.9024 trim_cache size 0 max 16384 2024-05-08T22:42:43.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.022+0000 7ff4a99c6740 1 Processor -- start 2024-05-08T22:42:43.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- start start 2024-05-08T22:42:43.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x173d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1693ec0 con 0x1745420 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1622730 con 0x173bbe0 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x162f4f0 con 0x1744b00 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37552/0 (socket says 172.21.15.179:37552) 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1104222192 learned_addr learned my addr 172.21.15.179:0/1104222192 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff49ffff700 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff49f7fe700 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x173d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 msgr2=0x173d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x173d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 msgr2=0x173cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1104222192 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1611450 con 0x1745420 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7ff4980849b0 tx=0x7ff498073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0748c8fe5d47e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4967fc700 1 -- 172.21.15.179:0/1104222192 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49806f040 con 0x1745420 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4967fc700 1 -- 172.21.15.179:0/1104222192 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49805fa00 con 0x1745420 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4967fc700 1 -- 172.21.15.179:0/1104222192 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49805f570 con 0x1745420 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1104222192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 msgr2=0x1754630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7ff4980849b0 tx=0x7ff498073000 comp rx=0 tx=0).stop 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1104222192 shutdown_connections 2024-05-08T22:42:43.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x1754630 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x173d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1104222192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1104222192 >> 172.21.15.179:0/1104222192 conn(0x148c4d0 msgr2=0x173c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1104222192 shutdown_connections 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.026+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1104222192 wait complete. 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 -- start start 2024-05-08T22:42:43.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 Processor -- start 2024-05-08T22:42:43.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1739c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x166bd80 con 0x1745420 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1726ab0 con 0x173bbe0 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x170faa0 con 0x1744b00 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff49ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff49f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1739c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a99c6740 10 client.0 _ll_register_callbacks cb 0x7ff498053a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a4cb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37564/0 (socket says 172.21.15.179:37564) 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1775938933 learned_addr learned my addr 172.21.15.179:0/1775938933 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.030+0000 7ff49ffff700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:38562/0 (socket says 172.21.15.179:38562) 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 msgr2=0x1739c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1739c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 msgr2=0x173a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 -- 172.21.15.179:0/1775938933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x162b220 con 0x1745420 2024-05-08T22:42:43.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a4cb4700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7ff498009fd0 tx=0x7ff498053b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb68df9ef6c26219 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.036 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:42:43.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49806f040 con 0x1745420 2024-05-08T22:42:43.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff49806b170 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x173c530 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49805d400 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 10 client.9030 fetch_fsmap learned FSMap version 75 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x1605720 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x173c530 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff4980534b0 con 0x1745420 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 10 client.9030 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 10 client.9030 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 20 client.9030 populate_metadata read hostname 'smithi179' 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x17a64e0 0x17a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 4 client.9030 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff498055800 con 0x17a64e0 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff49ffff700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x17a64e0 0x17a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff49ffff700 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x17a64e0 0x17a8d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d9b46e2c485fd server_cookie=37ac555e4780c539 in_seq=0 out_seq=0 2024-05-08T22:42:43.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 10 client.9030 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 1 -- 172.21.15.179:0/1775938933 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x7ff498055800 con 0x17a64e0 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 10 client.9030 handle_command_reply: tid=0 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 client.9030 shutdown 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 msgr2=0x17a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7ff498009fd0 tx=0x7ff498053b30 comp rx=0 tx=0).stop 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 shutdown_connections 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1745420 0x17a5db0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1744b00 0x1739c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x17a64e0 0x17a8d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 --2- 172.21.15.179:0/1775938933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x173bbe0 0x173a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff47effd700 0 client.9030 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 >> 172.21.15.179:0/1775938933 conn(0x148c4d0 msgr2=0x170f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 shutdown_connections 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.034+0000 7ff4a99c6740 1 -- 172.21.15.179:0/1775938933 wait complete. 2024-05-08T22:42:43.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a99c6740 20 client.9030 trim_cache size 0 max 16384 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 57425, 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 5.7494280949999998, 2024-05-08T22:42:43.041 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 75, 2024-05-08T22:42:43.042 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 112, 2024-05-08T22:42:43.042 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 112, 2024-05-08T22:42:43.042 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 10.194207994999999 2024-05-08T22:42:43.042 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 msgr2=0x7ff480068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 0x7ff480068bf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff4a015dcf0 tx=0x7ff488008040 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 msgr2=0x7ff4a01f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff498014040 tx=0x7ff498004af0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 shutdown_connections 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff480066730 0x7ff480068bf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a00b3180 0x7ff4a0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a015eef0 0x7ff4a01f5be0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 --2- 172.21.15.179:0/3561134860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a00ad9d0 0x7ff4a00af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 >> 172.21.15.179:0/3561134860 conn(0x7ff4a00ce960 msgr2=0x7ff4a00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 shutdown_connections 2024-05-08T22:42:43.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.038+0000 7ff4a54b5700 1 -- 172.21.15.179:0/3561134860 wait complete. 2024-05-08T22:42:43.056 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:42:43.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 Processor -- start 2024-05-08T22:42:43.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 -- start start 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 0x7f58340cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 0x7f583406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340b1510 con 0x7f58340ce780 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583415dea0 con 0x7f58341597b0 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f583a0a3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cec20 con 0x7f58341593d0 2024-05-08T22:42:43.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58398a2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 0x7f583406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58390a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 0x7f58340cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58398a2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 0x7f583406c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:38572/0 (socket says 172.21.15.179:38572) 2024-05-08T22:42:43.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58388a0700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f58340ce780 0x7f583415d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:57880/0 (socket says 172.21.15.179:57880) 2024-05-08T22:42:43.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58398a2700 1 -- 172.21.15.179:0/1112829010 learned_addr learned my addr 172.21.15.179:0/1112829010 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58390a1700 1 -- 172.21.15.179:0/1112829010 >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 msgr2=0x7f583406c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58390a1700 1 --2- 172.21.15.179:0/1112829010 >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 0x7f583406c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58390a1700 1 -- 172.21.15.179:0/1112829010 >> v1:172.21.15.195:6789/0 conn(0x7f58340ce780 legacy=0x7f583415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.178+0000 7f58390a1700 1 -- 172.21.15.179:0/1112829010 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340cef00 con 0x7f58341593d0 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f58390a1700 1 --2- 172.21.15.179:0/1112829010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 0x7f58340cc4e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5830009850 tx=0x7f5830009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630387776d58bb34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f582bfff700 1 -- 172.21.15.179:0/1112829010 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011830 con 0x7f58341593d0 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f582bfff700 1 -- 172.21.15.179:0/1112829010 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830011990 con 0x7f58341593d0 2024-05-08T22:42:43.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f582bfff700 1 -- 172.21.15.179:0/1112829010 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011d40 con 0x7f58341593d0 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- 172.21.15.179:0/1112829010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 msgr2=0x7f58340cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- 172.21.15.179:0/1112829010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 0x7f58340cc4e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5830009850 tx=0x7f5830009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- 172.21.15.179:0/1112829010 shutdown_connections 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- 172.21.15.179:0/1112829010 >> v2:172.21.15.195:3300/0 conn(0x7f58341597b0 0x7f583406c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- 172.21.15.179:0/1112829010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341593d0 0x7f58340cc4e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- 172.21.15.179:0/1112829010 >> 172.21.15.179:0/1112829010 conn(0x7f5834098a70 msgr2=0x7f58340b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- 172.21.15.179:0/1112829010 shutdown_connections 2024-05-08T22:42:43.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- 172.21.15.179:0/1112829010 wait complete. 2024-05-08T22:42:43.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 Processor -- start 2024-05-08T22:42:43.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- start start 2024-05-08T22:42:43.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58340ce780 0x7f58340cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341597b0 0x7f58340ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583415cbe0 con 0x7f58341527d0 2024-05-08T22:42:43.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341531a0 con 0x7f58340ce780 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f583a0a3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340c94d0 con 0x7f58341597b0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.182+0000 7f58398a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58390a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58340ce780 0x7f58340cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37580/0 (socket says 172.21.15.179:37580) 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 -- 172.21.15.179:0/2968281728 learned_addr learned my addr 172.21.15.179:0/2968281728 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58388a0700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341597b0 0x7f58340ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 -- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341597b0 msgr2=0x7f58340ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341597b0 0x7f58340ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 -- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58340ce780 msgr2=0x7f58340cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58340ce780 0x7f58340cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 -- 172.21.15.179:0/2968281728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300094b0 con 0x7f58341527d0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f58398a2700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f582c009c40 tx=0x7f582c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c018070 con 0x7f58341527d0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c010040 con 0x7f58341527d0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58341f1fa0 con 0x7f58341527d0 2024-05-08T22:42:43.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00ac10 con 0x7f58341527d0 2024-05-08T22:42:43.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58341f2560 con 0x7f58341527d0 2024-05-08T22:42:43.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583404e700 con 0x7f58341527d0 2024-05-08T22:42:43.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.186+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f582c007990 con 0x7f58341527d0 2024-05-08T22:42:43.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.190+0000 7f5829ffb700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 0x7f5818068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:43.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.190+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f582c084030 con 0x7f58341527d0 2024-05-08T22:42:43.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.190+0000 7f58390a1700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 0x7f5818068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:43.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.190+0000 7f58390a1700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 0x7f5818068bf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f583406fd60 tx=0x7f5830008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:43.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.194+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f582c0512f0 con 0x7f58341527d0 2024-05-08T22:42:43.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.402+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58340b50b0 con 0x7f58341527d0 2024-05-08T22:42:43.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.406+0000 7f5829ffb700 1 -- 172.21.15.179:0/2968281728 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f582c054910 con 0x7f58341527d0 2024-05-08T22:42:43.407 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:42:43.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 msgr2=0x7f5818068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 0x7f5818068bf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f583406fd60 tx=0x7f5830008030 comp rx=0 tx=0).stop 2024-05-08T22:42:43.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 msgr2=0x7f5834152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:43.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f582c009c40 tx=0x7f582c009f50 comp rx=0 tx=0).stop 2024-05-08T22:42:43.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 shutdown_connections 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5818066730 0x7f5818068bf0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58341527d0 0x7f5834152c60 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58341597b0 0x7f58340ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 --2- 172.21.15.179:0/2968281728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58340ce780 0x7f58340cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 >> 172.21.15.179:0/2968281728 conn(0x7f5834098a70 msgr2=0x7f5834157890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 shutdown_connections 2024-05-08T22:42:43.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:43.410+0000 7f583a0a3700 1 -- 172.21.15.179:0/2968281728 wait complete. 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:42:43.426 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:43.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.427 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:43.433 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42008sec preferred_lft 42008sec 2024-05-08T22:42:43.434 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:42:43.435 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:43.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.436 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:42:43.488 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:42:43.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.489 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:42:43.498 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:42:43.499 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:42:43.499 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:42:43.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:42:43.499 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:42:43.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.534 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:42:43.585 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:42:43.585 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:42:43.586 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:43.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.586 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:42:43.603 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:42:43.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:42:43.603 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:43.610 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:43.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:42:43.610 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:43.616 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:43.675 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:43.675 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:43.691 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:43.692 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:43.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.692 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:43.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:42:43.699 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:42:43.701 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:43.718 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:43.718 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:43.731 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:43.731 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:43.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:43.732 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:43.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:42:43.786+0000 7fa5598ea640 -1 init, newargv = 0x55675a36d880 newargc=15 2024-05-08T22:42:43.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[49571]: starting ceph client 2024-05-08T22:42:43.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[49571]: starting fuse 2024-05-08T22:42:44.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:44.899 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:44.899 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:44.958 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:44.959 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:44.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:44.960 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:44.967 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:42:44.967 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:42:44.968 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:42:44.969 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:45.050 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.49571.asok 2024-05-08T22:42:45.236 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49571.asok 2024-05-08T22:42:45.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:45.237 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49571.asok status 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:42:45.339 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "49571", 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9042, 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 9042 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 629891406 2024-05-08T22:42:45.340 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 629891406 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.9042 192.168.0.1:0/629891406", 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/629891406", 2024-05-08T22:42:45.341 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 75, 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 112, 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:42:45.342 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:42:45.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:45.349 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:42:45.349 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:45.354 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:42:45.354 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:45.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:45.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:45.366 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:45.366 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:45.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:45.371 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T22:42:45.463 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records in 2024-05-08T22:42:45.463 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records out 2024-05-08T22:42:45.463 INFO:teuthology.orchestra.run.smithi179.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0718363 s, 87.6 MB/s 2024-05-08T22:42:45.464 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:45.464 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:45.471 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:45.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.590+0000 7f0508ae2700 1 Processor -- start 2024-05-08T22:42:45.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.590+0000 7f0508ae2700 1 -- start start 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f05040b45b0 0x7f050415e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05040b5180 con 0x7f05040b41d0 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05040ceb10 con 0x7f05040b4ca0 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05040cf0d0 con 0x7f05040b45b0 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05037fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f05040b41d0 0x7f05040ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44764/0 (socket says 172.21.15.179:44764) 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05037fe700 1 -- 172.21.15.179:0/1318840887 learned_addr learned my addr 172.21.15.179:0/1318840887 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0503fff700 1 --2- 172.21.15.179:0/1318840887 >> v2:172.21.15.179:3300/0 conn(0x7f05040b45b0 0x7f050415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05027fc700 1 -- 172.21.15.179:0/1318840887 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112958095 0 0) 0x7f05040b5180 con 0x7f05040b41d0 2024-05-08T22:42:45.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05027fc700 1 -- 172.21.15.179:0/1318840887 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e40035d0 con 0x7f05040b41d0 2024-05-08T22:42:45.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0503fff700 1 -- 172.21.15.179:0/1318840887 >> v1:172.21.15.195:6789/0 conn(0x7f05040b4ca0 legacy=0x7f050415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0503fff700 1 -- 172.21.15.179:0/1318840887 >> v1:172.21.15.179:6789/0 conn(0x7f05040b41d0 legacy=0x7f05040ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0503fff700 1 -- 172.21.15.179:0/1318840887 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05040cedf0 con 0x7f05040b45b0 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0503fff700 1 --2- 172.21.15.179:0/1318840887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b45b0 0x7f050415e960 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009850 tx=0x7f04fc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb27fba52f578d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05027fc700 1 -- 172.21.15.179:0/1318840887 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc011830 con 0x7f05040b45b0 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05027fc700 1 -- 172.21.15.179:0/1318840887 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc011990 con 0x7f05040b45b0 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f05027fc700 1 -- 172.21.15.179:0/1318840887 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc011d40 con 0x7f05040b45b0 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- 172.21.15.179:0/1318840887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b45b0 msgr2=0x7f050415e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1318840887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b45b0 0x7f050415e960 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009850 tx=0x7f04fc009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- 172.21.15.179:0/1318840887 shutdown_connections 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1318840887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b45b0 0x7f050415e960 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- 172.21.15.179:0/1318840887 >> 172.21.15.179:0/1318840887 conn(0x7f0504098a70 msgr2=0x7f05040b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- 172.21.15.179:0/1318840887 shutdown_connections 2024-05-08T22:42:45.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.594+0000 7f0508ae2700 1 -- 172.21.15.179:0/1318840887 wait complete. 2024-05-08T22:42:45.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 Processor -- start 2024-05-08T22:42:45.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- start start 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05040b45b0 0x7f0504158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 0x7f05041f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05040c3e80 con 0x7f05040b41d0 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05041f7250 con 0x7f05040b4ca0 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05041f6c90 con 0x7f05040b45b0 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58688/0 (socket says 172.21.15.179:58688) 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 -- 172.21.15.179:0/1791877037 learned_addr learned my addr 172.21.15.179:0/1791877037 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0503fff700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 0x7f05041f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0502ffd700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05040b45b0 0x7f0504158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 -- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05040b45b0 msgr2=0x7f0504158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05040b45b0 0x7f0504158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 -- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 msgr2=0x7f05041f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 0x7f05041f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 -- 172.21.15.179:0/1791877037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0094b0 con 0x7f05040b41d0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0503fff700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 0x7f05041f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f05037fe700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f04f400cbc0 tx=0x7f04f400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400a9e0 con 0x7f05040b41d0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400ab40 con 0x7f05040b41d0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05041f6f70 con 0x7f05040b41d0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4012400 con 0x7f05040b41d0 2024-05-08T22:42:45.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.598+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05040c4390 con 0x7f05040b41d0 2024-05-08T22:42:45.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05040c6300 con 0x7f05040b41d0 2024-05-08T22:42:45.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04f401b3b0 con 0x7f05040b41d0 2024-05-08T22:42:45.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f04fbfff700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 0x7f04e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f04f4083d50 con 0x7f05040b41d0 2024-05-08T22:42:45.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f0502ffd700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 0x7f04e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.602+0000 7f0502ffd700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 0x7f04e8068bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05040c6d50 tx=0x7f04ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.610+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f04f4087030 con 0x7f05040b41d0 2024-05-08T22:42:45.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.778+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f050415ee50 con 0x7f05040b41d0 2024-05-08T22:42:45.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.778+0000 7f04fbfff700 1 -- 172.21.15.179:0/1791877037 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16296 (secure 0 0 0) 0x7f04f4051010 con 0x7f05040b41d0 2024-05-08T22:42:45.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:45.782 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":112,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:38.288356+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 msgr2=0x7f04e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 0x7f04e8068bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05040c6d50 tx=0x7f04ec009040 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 msgr2=0x7f05041567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f04f400cbc0 tx=0x7f04f400cca0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 shutdown_connections 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f04e8066730 0x7f04e8068bf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05040b4ca0 0x7f05041f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05040b45b0 0x7f0504158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 --2- 172.21.15.179:0/1791877037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05040b41d0 0x7f05041567d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 >> 172.21.15.179:0/1791877037 conn(0x7f0504098a70 msgr2=0x7f05040cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 shutdown_connections 2024-05-08T22:42:45.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.782+0000 7f0508ae2700 1 -- 172.21.15.179:0/1791877037 wait complete. 2024-05-08T22:42:45.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:42:35.609486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:42:36.273848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:45.798 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:45.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.918+0000 7fbdb759e700 1 Processor -- start 2024-05-08T22:42:45.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.918+0000 7fbdb759e700 1 -- start start 2024-05-08T22:42:45.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.918+0000 7fbdb759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbdb815db00 0x7fbdb806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb806b110 con 0x7fbdb80cd590 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80b19e0 con 0x7fbdb815dee0 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80b1d30 con 0x7fbdb815db00 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbdb80cd590 0x7fbdb80b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44778/0 (socket says 172.21.15.179:44778) 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/745544375 learned_addr learned my addr 172.21.15.179:0/745544375 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb659c700 1 --2- 172.21.15.179:0/745544375 >> v2:172.21.15.195:3300/0 conn(0x7fbdb815db00 0x7fbdb806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349217521 0 0) 0x7fbdb806b110 con 0x7fbdb80cd590 2024-05-08T22:42:45.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda40035d0 con 0x7fbdb80cd590 2024-05-08T22:42:45.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960151710 0 0) 0x7fbdb80b19e0 con 0x7fbdb815dee0 2024-05-08T22:42:45.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb806b110 con 0x7fbdb815dee0 2024-05-08T22:42:45.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1033392571 0 0) 0x7fbda40035d0 con 0x7fbdb80cd590 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 >> v2:172.21.15.195:3300/0 conn(0x7fbdb815db00 msgr2=0x7fbdb806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 --2- 172.21.15.179:0/745544375 >> v2:172.21.15.195:3300/0 conn(0x7fbdb815db00 0x7fbdb806cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 >> v1:172.21.15.195:6789/0 conn(0x7fbdb815dee0 legacy=0x7fbdb80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8157cf0 con 0x7fbdb80cd590 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fbdb00019f0 con 0x7fbdb80cd590 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 >> v1:172.21.15.179:6789/0 conn(0x7fbdb80cd590 legacy=0x7fbdb80b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4003bb0 0x7fbda4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbda4004870 0x7fbda4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 0x7fbda400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b1d30 con 0x7fbda4005230 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400c160 con 0x7fbda4003bb0 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400c320 con 0x7fbda4004870 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb659c700 1 --2- 172.21.15.179:0/745544375 >> v2:172.21.15.195:3300/0 conn(0x7fbdb815db00 0x7fbdb806cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:45.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 0x7fbda400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb659c700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4003bb0 0x7fbda4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdadd9b700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbda4004870 0x7fbda4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbda4004870 msgr2=0x7fbda4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbda4004870 0x7fbda4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4003bb0 msgr2=0x7fbda4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4003bb0 0x7fbda4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/745544375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40035d0 con 0x7fbda4005230 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb659c700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4003bb0 0x7fbda4003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/745544375 reap_dead start 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 0x7fbda400ba30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000a7c0 tx=0x7fbda40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006c5b659223ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00072d0 con 0x7fbda4005230 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0007470 con 0x7fbda4005230 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb5d9b700 1 -- 172.21.15.179:0/745544375 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0007820 con 0x7fbda4005230 2024-05-08T22:42:45.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 msgr2=0x7fbda400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 0x7fbda400ba30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000a7c0 tx=0x7fbda40058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- 172.21.15.179:0/745544375 shutdown_connections 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 --2- 172.21.15.179:0/745544375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbda4005230 0x7fbda400ba30 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- 172.21.15.179:0/745544375 >> 172.21.15.179:0/745544375 conn(0x7fbdb80cea30 msgr2=0x7fbdb80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- 172.21.15.179:0/745544375 shutdown_connections 2024-05-08T22:42:45.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.922+0000 7fbdb759e700 1 -- 172.21.15.179:0/745544375 wait complete. 2024-05-08T22:42:45.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 Processor -- start 2024-05-08T22:42:45.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- start start 2024-05-08T22:42:45.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4005230 0x7fbdb8141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb8144ae0 0x7fbdb815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb815cd10 con 0x7fbdb8144f60 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb815cfa0 con 0x7fbda4005230 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8141030 con 0x7fbdb8144ae0 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4005230 0x7fbdb8141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58708/0 (socket says 172.21.15.179:58708) 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/2992158597 learned_addr learned my addr 172.21.15.179:0/2992158597 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdadd9b700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb8144ae0 0x7fbdb815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb8144ae0 msgr2=0x7fbdb815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb8144ae0 0x7fbdb815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4005230 msgr2=0x7fbdb8141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4005230 0x7fbdb8141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 -- 172.21.15.179:0/2992158597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8141310 con 0x7fbdb8144f60 2024-05-08T22:42:45.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb6d9d700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0002230 tx=0x7fbdb0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0022070 con 0x7fbdb8144f60 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0001ac0 con 0x7fbdb8144f60 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb81415f0 con 0x7fbdb8144f60 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00077e0 con 0x7fbdb8144f60 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.926+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb8168890 con 0x7fbdb8144f60 2024-05-08T22:42:45.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdb001a070 con 0x7fbdb8144f60 2024-05-08T22:42:45.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb81417e0 con 0x7fbdb8144f60 2024-05-08T22:42:45.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdaf7fe700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 0x7fbd98068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:45.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbdb0084ec0 con 0x7fbdb8144f60 2024-05-08T22:42:45.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdb659c700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 0x7fbd98068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:45.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.930+0000 7fbdb659c700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 0x7fbd98068c20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbda8006fd0 tx=0x7fbda8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:45.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:45.938+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbdb00520e0 con 0x7fbdb8144f60 2024-05-08T22:42:46.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.110+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdb81417e0 con 0x7fbdb8144f60 2024-05-08T22:42:46.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.110+0000 7fbdaf7fe700 1 -- 172.21.15.179:0/2992158597 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdb0055700 con 0x7fbdb8144f60 2024-05-08T22:42:46.114 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 msgr2=0x7fbd98068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 0x7fbd98068c20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbda8006fd0 tx=0x7fbda8008040 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 msgr2=0x7fbdb815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0002230 tx=0x7fbdb0001d00 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 shutdown_connections 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbd98066760 0x7fbd98068c20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdb8144f60 0x7fbdb815c7d0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb8144ae0 0x7fbdb815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 --2- 172.21.15.179:0/2992158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbda4005230 0x7fbdb8141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 >> 172.21.15.179:0/2992158597 conn(0x7fbdb80cea30 msgr2=0x7fbdb80cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 shutdown_connections 2024-05-08T22:42:46.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.114+0000 7fbdb759e700 1 -- 172.21.15.179:0/2992158597 wait complete. 2024-05-08T22:42:46.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:46.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 Processor -- start 2024-05-08T22:42:46.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- start start 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73840b1a50 0x7f73840b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f738406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738406dd50 con 0x7f738414ae30 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738406e9d0 con 0x7f73840b1a50 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840b30c0 con 0x7f73840b2c10 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73840b1a50 0x7f73840b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73840b1a50 0x7f73840b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58718/0 (socket says 172.21.15.179:58718) 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 -- 172.21.15.179:0/1423485449 learned_addr learned my addr 172.21.15.179:0/1423485449 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7383fff700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f738406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 -- 172.21.15.179:0/1423485449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 msgr2=0x7f738406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f738406abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 -- 172.21.15.179:0/1423485449 >> v1:172.21.15.195:6789/0 conn(0x7f738414ae30 legacy=0x7f7384165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 -- 172.21.15.179:0/1423485449 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738406e030 con 0x7f73840b1a50 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7382ffd700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f73840b26d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f736c009910 tx=0x7f736c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc3dccc289c43ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f73827fc700 1 -- 172.21.15.179:0/1423485449 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c011830 con 0x7f73840b1a50 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f73827fc700 1 -- 172.21.15.179:0/1423485449 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c011990 con 0x7f73840b1a50 2024-05-08T22:42:46.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f73827fc700 1 -- 172.21.15.179:0/1423485449 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c011d40 con 0x7f73840b1a50 2024-05-08T22:42:46.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- 172.21.15.179:0/1423485449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 msgr2=0x7f73840b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f73840b26d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f736c009910 tx=0x7f736c009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:46.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- 172.21.15.179:0/1423485449 shutdown_connections 2024-05-08T22:42:46.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f738406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 --2- 172.21.15.179:0/1423485449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f73840b26d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- 172.21.15.179:0/1423485449 >> 172.21.15.179:0/1423485449 conn(0x7f73840cea30 msgr2=0x7f73840cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- 172.21.15.179:0/1423485449 shutdown_connections 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.245+0000 7f7388ee8700 1 -- 172.21.15.179:0/1423485449 wait complete. 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 Processor -- start 2024-05-08T22:42:46.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 -- start start 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f7384148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738414ae30 0x7f73841497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384148b80 con 0x7f73840b1a50 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415c8f0 con 0x7f738414ae30 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7388ee8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738414a2d0 con 0x7f73840b2c10 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58722/0 (socket says 172.21.15.179:58722) 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 -- 172.21.15.179:0/90944400 learned_addr learned my addr 172.21.15.179:0/90944400 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7383fff700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738414ae30 0x7f73841497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f7382ffd700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f7384148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 -- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 msgr2=0x7f7384148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f7384148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 -- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738414ae30 msgr2=0x7f73841497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738414ae30 0x7f73841497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 -- 172.21.15.179:0/90944400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c009570 con 0x7f73840b1a50 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.249+0000 7f73837fe700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f737400c500 tx=0x7f737400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740052d0 con 0x7f73840b1a50 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374005430 con 0x7f73840b1a50 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738414a5b0 con 0x7f73840b1a50 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740055a0 con 0x7f73840b1a50 2024-05-08T22:42:46.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73841487a0 con 0x7f73840b1a50 2024-05-08T22:42:46.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7384148cc0 con 0x7f73840b1a50 2024-05-08T22:42:46.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.253+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73740038e0 con 0x7f73840b1a50 2024-05-08T22:42:46.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.257+0000 7f737bfff700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.257+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7374083640 con 0x7f73840b1a50 2024-05-08T22:42:46.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.257+0000 7f7382ffd700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.257+0000 7f7382ffd700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 0x7f7364068bf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73840b1e90 tx=0x7f736c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.265+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7384148cc0 con 0x7f73840b1a50 2024-05-08T22:42:46.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.433+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f738404e700 con 0x7f73840b1a50 2024-05-08T22:42:46.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.433+0000 7f737bfff700 1 -- 172.21.15.179:0/90944400 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7374050860 con 0x7f73840b1a50 2024-05-08T22:42:46.436 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 msgr2=0x7f7364068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 0x7f7364068bf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f73840b1e90 tx=0x7f736c008030 comp rx=0 tx=0).stop 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 msgr2=0x7f7384164ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f737400c500 tx=0x7f737400c810 comp rx=0 tx=0).stop 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 shutdown_connections 2024-05-08T22:42:46.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7364066730 0x7f7364068bf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738414ae30 0x7f73841497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73840b2c10 0x7f7384148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 --2- 172.21.15.179:0/90944400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73840b1a50 0x7f7384164ae0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 >> 172.21.15.179:0/90944400 conn(0x7f73840cea30 msgr2=0x7f73841478b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 shutdown_connections 2024-05-08T22:42:46.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.437+0000 7f7388ee8700 1 -- 172.21.15.179:0/90944400 wait complete. 2024-05-08T22:42:46.451 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:46.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.569+0000 7fa4121c1700 1 Processor -- start 2024-05-08T22:42:46.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.569+0000 7fa4121c1700 1 -- start start 2024-05-08T22:42:46.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c159320 0x7fa40c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa40c0b19e0 0x7fa40c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c06cda0 con 0x7fa40c159700 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b1f10 con 0x7fa40c0b19e0 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b21f0 con 0x7fa40c159320 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa40c0b19e0 0x7fa40c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa40c0b19e0 0x7fa40c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58734/0 (socket says 172.21.15.179:58734) 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 -- 172.21.15.179:0/1027774238 learned_addr learned my addr 172.21.15.179:0/1027774238 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:46.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4111bf700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c159320 0x7fa40c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 -- 172.21.15.179:0/1027774238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c159320 msgr2=0x7fa40c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c159320 0x7fa40c0cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 -- 172.21.15.179:0/1027774238 >> v1:172.21.15.195:6789/0 conn(0x7fa40c159700 legacy=0x7fa40c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 -- 172.21.15.179:0/1027774238 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0b24d0 con 0x7fa40c0b19e0 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4109be700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c0b19e0 0x7fa40c0cc630 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa408009910 tx=0x7fa408009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc798cd72e40836 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa3fffff700 1 -- 172.21.15.179:0/1027774238 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800ba60 con 0x7fa40c0b19e0 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa3fffff700 1 -- 172.21.15.179:0/1027774238 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800bbc0 con 0x7fa40c0b19e0 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa3fffff700 1 -- 172.21.15.179:0/1027774238 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408009e60 con 0x7fa40c0b19e0 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- 172.21.15.179:0/1027774238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c0b19e0 msgr2=0x7fa40c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c0b19e0 0x7fa40c0cc630 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa408009910 tx=0x7fa408009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- 172.21.15.179:0/1027774238 shutdown_connections 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c0b19e0 0x7fa40c0cc630 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1027774238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c159320 0x7fa40c0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- 172.21.15.179:0/1027774238 >> 172.21.15.179:0/1027774238 conn(0x7fa40c0cea30 msgr2=0x7fa40c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- 172.21.15.179:0/1027774238 shutdown_connections 2024-05-08T22:42:46.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.573+0000 7fa4121c1700 1 -- 172.21.15.179:0/1027774238 wait complete. 2024-05-08T22:42:46.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 Processor -- start 2024-05-08T22:42:46.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 -- start start 2024-05-08T22:42:46.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c0b19e0 0x7fa40c14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa40c159320 0x7fa40c13e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c13e640 con 0x7fa40c159700 2024-05-08T22:42:46.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c14dd00 con 0x7fa40c159320 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4121c1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c150630 con 0x7fa40c0b19e0 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4119c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4111bf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c0b19e0 0x7fa40c14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4109be700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa40c159320 0x7fa40c13e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4119c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58742/0 (socket says 172.21.15.179:58742) 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.577+0000 7fa4119c0700 1 -- 172.21.15.179:0/1982602637 learned_addr learned my addr 172.21.15.179:0/1982602637 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 -- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c0b19e0 msgr2=0x7fa40c14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c0b19e0 0x7fa40c14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 -- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa40c159320 msgr2=0x7fa40c13e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa40c159320 0x7fa40c13e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 -- 172.21.15.179:0/1982602637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009570 con 0x7fa40c159700 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4119c0700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa40400cbc0 tx=0x7fa40400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404003c20 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404003d80 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c14df50 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400aae0 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c1f2320 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c0cc630 con 0x7fa40c159700 2024-05-08T22:42:46.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa40400ac40 con 0x7fa40c159700 2024-05-08T22:42:46.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.581+0000 7fa3fdffb700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.585+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa404084240 con 0x7fa40c159700 2024-05-08T22:42:46.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.585+0000 7fa4111bf700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.585+0000 7fa4111bf700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 0x7fa3f0068bf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa400007a60 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.593+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa404051480 con 0x7fa40c159700 2024-05-08T22:42:46.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.761+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa40c04e700 con 0x7fa40c159700 2024-05-08T22:42:46.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.761+0000 7fa3fdffb700 1 -- 172.21.15.179:0/1982602637 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa404054aa0 con 0x7fa40c159700 2024-05-08T22:42:46.764 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:46.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 msgr2=0x7fa3f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 0x7fa3f0068bf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa400007a60 tx=0x7fa400008040 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 msgr2=0x7fa40c1500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa40400cbc0 tx=0x7fa40400ced0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 shutdown_connections 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3f0066730 0x7fa3f0068bf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa40c159320 0x7fa40c13e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa40c159700 0x7fa40c1500f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 --2- 172.21.15.179:0/1982602637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40c0b19e0 0x7fa40c14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 >> 172.21.15.179:0/1982602637 conn(0x7fa40c0cea30 msgr2=0x7fa40c0cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 shutdown_connections 2024-05-08T22:42:46.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.765+0000 7fa4121c1700 1 -- 172.21.15.179:0/1982602637 wait complete. 2024-05-08T22:42:46.784 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:42:46.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.893+0000 7f67affff700 1 Processor -- start 2024-05-08T22:42:46.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.893+0000 7f67affff700 1 -- start start 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67b006dda0 0x7f67b00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b00b25d0 0x7f67b00b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b006b540 con 0x7f67b006e180 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b006ac50 con 0x7f67b006dda0 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b006aef0 con 0x7f67b00b25d0 2024-05-08T22:42:46.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b00b25d0 0x7f67b00b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67b006dda0 0x7f67b00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67ae7fc700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f67b006e180 0x7f67b006e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:52946/0 (socket says 172.21.15.179:52946) 2024-05-08T22:42:46.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67ae7fc700 1 -- 172.21.15.179:0/2541433484 learned_addr learned my addr 172.21.15.179:0/2541433484 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:46.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 -- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b00b25d0 msgr2=0x7f67b00b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 --2- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b00b25d0 0x7f67b00b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 -- 172.21.15.179:0/2541433484 >> v1:172.21.15.195:6789/0 conn(0x7f67b006e180 legacy=0x7f67b006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67adffb700 1 -- 172.21.15.179:0/2541433484 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285710350 0 0) 0x7f67b006b540 con 0x7f67b006e180 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 -- 172.21.15.179:0/2541433484 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b006b1d0 con 0x7f67b006dda0 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67aeffd700 1 --2- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b006dda0 0x7f67b00cbc30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6798009850 tx=0x7f6798009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b403c40776694682 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67adffb700 1 -- 172.21.15.179:0/2541433484 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011830 con 0x7f67b006dda0 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67adffb700 1 -- 172.21.15.179:0/2541433484 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798011990 con 0x7f67b006dda0 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67adffb700 1 -- 172.21.15.179:0/2541433484 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011d40 con 0x7f67b006dda0 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b006dda0 msgr2=0x7f67b00cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 --2- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b006dda0 0x7f67b00cbc30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6798009850 tx=0x7f6798009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- 172.21.15.179:0/2541433484 shutdown_connections 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 --2- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b00b25d0 0x7f67b00b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 --2- 172.21.15.179:0/2541433484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b006dda0 0x7f67b00cbc30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- 172.21.15.179:0/2541433484 >> 172.21.15.179:0/2541433484 conn(0x7f67b00ce960 msgr2=0x7f67b00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:46.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- 172.21.15.179:0/2541433484 shutdown_connections 2024-05-08T22:42:46.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.897+0000 7f67affff700 1 -- 172.21.15.179:0/2541433484 wait complete. 2024-05-08T22:42:46.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 Processor -- start 2024-05-08T22:42:46.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- start start 2024-05-08T22:42:46.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b006dda0 0x7f67b01639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b00b25d0 0x7f67b01f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01478b0 con 0x7f67b006e180 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b014ada0 con 0x7f67b00b25d0 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01f69b0 con 0x7f67b006dda0 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58754/0 (socket says 172.21.15.179:58754) 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 -- 172.21.15.179:0/4158798188 learned_addr learned my addr 172.21.15.179:0/4158798188 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67af7fe700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b00b25d0 0x7f67b01f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67aeffd700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b006dda0 0x7f67b01639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 -- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b006dda0 msgr2=0x7f67b01639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b006dda0 0x7f67b01639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 -- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b00b25d0 msgr2=0x7f67b01f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b00b25d0 0x7f67b01f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 -- 172.21.15.179:0/4158798188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980094b0 con 0x7f67b006e180 2024-05-08T22:42:46.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67ae7fc700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f67a0001f80 tx=0x7f67a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000acd0 con 0x7f67b006e180 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000ae50 con 0x7f67b006e180 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b015e200 con 0x7f67b006e180 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0007310 con 0x7f67b006e180 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.901+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0147da0 con 0x7f67b006e180 2024-05-08T22:42:46.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.905+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b004e700 con 0x7f67b006e180 2024-05-08T22:42:46.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.905+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f67a0007470 con 0x7f67b006e180 2024-05-08T22:42:46.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.905+0000 7f67a77fe700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 0x7f6790068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:46.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.905+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f67a00840f0 con 0x7f67b006e180 2024-05-08T22:42:46.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.909+0000 7f67aeffd700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 0x7f6790068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:46.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.909+0000 7f67aeffd700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 0x7f6790068bf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67980090b0 tx=0x7f679801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:46.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:46.913+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f67a0051280 con 0x7f67b006e180 2024-05-08T22:42:47.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.097+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67b0162c80 con 0x7f67b006e180 2024-05-08T22:42:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.097+0000 7f67a77fe700 1 -- 172.21.15.179:0/4158798188 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67a00548a0 con 0x7f67b006e180 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 msgr2=0x7f6790068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 0x7f6790068bf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67980090b0 tx=0x7f679801c040 comp rx=0 tx=0).stop 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 msgr2=0x7f67b01f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f67a0001f80 tx=0x7f67a000a340 comp rx=0 tx=0).stop 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 shutdown_connections 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6790066730 0x7f6790068bf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67b00b25d0 0x7f67b01f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b006e180 0x7f67b01f5f30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 --2- 172.21.15.179:0/4158798188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67b006dda0 0x7f67b01639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 >> 172.21.15.179:0/4158798188 conn(0x7f67b00ce960 msgr2=0x7f67b00cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 shutdown_connections 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.101+0000 7f67affff700 1 -- 172.21.15.179:0/4158798188 wait complete. 2024-05-08T22:42:47.104 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:42:47.115 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:47.115 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:47.121 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:47.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 Processor -- start 2024-05-08T22:42:47.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 -- start start 2024-05-08T22:42:47.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffbe8164db0 0x7ffbe806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe8165d50 con 0x7ffbe815ef80 2024-05-08T22:42:47.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe806ad70 con 0x7ffbe8164db0 2024-05-08T22:42:47.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8165260 con 0x7ffbe806de70 2024-05-08T22:42:47.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffbe815ef80 0x7ffbe806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44788/0 (socket says 172.21.15.179:44788) 2024-05-08T22:42:47.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe659c700 1 -- 172.21.15.179:0/3292992163 learned_addr learned my addr 172.21.15.179:0/3292992163 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:47.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/3292992163 >> v2:172.21.15.195:3300/0 conn(0x7ffbe8164db0 0x7ffbe806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe5d9b700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe559a700 1 -- 172.21.15.179:0/3292992163 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873835575 0 0) 0x7ffbe8165d50 con 0x7ffbe815ef80 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 msgr2=0x7ffbe8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe8161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/3292992163 >> v1:172.21.15.179:6789/0 conn(0x7ffbe815ef80 legacy=0x7ffbe806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.229+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/3292992163 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe806e4b0 con 0x7ffbe8164db0 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe8164db0 0x7ffbe806b200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009d90 tx=0x7ffbe000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=441047e981829269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe559a700 1 -- 172.21.15.179:0/3292992163 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000e070 con 0x7ffbe8164db0 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe559a700 1 -- 172.21.15.179:0/3292992163 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe000bb20 con 0x7ffbe8164db0 2024-05-08T22:42:47.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe559a700 1 -- 172.21.15.179:0/3292992163 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0012430 con 0x7ffbe8164db0 2024-05-08T22:42:47.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe8164db0 msgr2=0x7ffbe806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe8164db0 0x7ffbe806b200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009d90 tx=0x7ffbe000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- 172.21.15.179:0/3292992163 shutdown_connections 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe8164db0 0x7ffbe806b200 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 --2- 172.21.15.179:0/3292992163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- 172.21.15.179:0/3292992163 >> 172.21.15.179:0/3292992163 conn(0x7ffbe80ce9d0 msgr2=0x7ffbe80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- 172.21.15.179:0/3292992163 shutdown_connections 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- 172.21.15.179:0/3292992163 wait complete. 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 Processor -- start 2024-05-08T22:42:47.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.233+0000 7ffbe759e700 1 -- start start 2024-05-08T22:42:47.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe815ef80 0x7ffbe80b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80c4c30 con 0x7ffbe8164db0 2024-05-08T22:42:47.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80c5530 con 0x7ffbe815ef80 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80c5810 con 0x7ffbe806de70 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58758/0 (socket says 172.21.15.179:58758) 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/2142492546 learned_addr learned my addr 172.21.15.179:0/2142492546 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe5d9b700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe815ef80 0x7ffbe80b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe659c700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 msgr2=0x7ffbe80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe815ef80 msgr2=0x7ffbe80b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe815ef80 0x7ffbe80b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 -- 172.21.15.179:0/2142492546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe0009a70 con 0x7ffbe8164db0 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe659c700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe6d9d700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0014040 tx=0x7ffbe0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0012430 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe0020070 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80c5af0 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0004280 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe815b270 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.237+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe80c6010 con 0x7ffbe8164db0 2024-05-08T22:42:47.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.241+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffbe00043e0 con 0x7ffbe8164db0 2024-05-08T22:42:47.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.241+0000 7ffbd6ffd700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 0x7ffbc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.241+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbe0022070 con 0x7ffbe8164db0 2024-05-08T22:42:47.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.241+0000 7ffbe659c700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 0x7ffbc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.241+0000 7ffbe659c700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 0x7ffbc8068bf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc000f60 tx=0x7ffbdc007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.249+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffbe00531a0 con 0x7ffbe8164db0 2024-05-08T22:42:47.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.417+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbe8067740 con 0x7ffbe8164db0 2024-05-08T22:42:47.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbd6ffd700 1 -- 172.21.15.179:0/2142492546 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16296 (secure 0 0 0) 0x7ffbe00567c0 con 0x7ffbe8164db0 2024-05-08T22:42:47.423 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:47.424 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":112,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:38.288356+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 msgr2=0x7ffbc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 0x7ffbc8068bf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc000f60 tx=0x7ffbdc007cc0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 msgr2=0x7ffbe80b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0014040 tx=0x7ffbe0004af0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 shutdown_connections 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffbc8066730 0x7ffbc8068bf0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe815ef80 0x7ffbe80b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbe8164db0 0x7ffbe80b3730 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 --2- 172.21.15.179:0/2142492546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbe806de70 0x7ffbe80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 >> 172.21.15.179:0/2142492546 conn(0x7ffbe80ce9d0 msgr2=0x7ffbe815f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 shutdown_connections 2024-05-08T22:42:47.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.421+0000 7ffbe759e700 1 -- 172.21.15.179:0/2142492546 wait complete. 2024-05-08T22:42:47.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:42:35.609486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:42:36.273848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:47.438 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:47.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 Processor -- start 2024-05-08T22:42:47.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 -- start start 2024-05-08T22:42:47.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f75c006e090 0x7f75c015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c015eec0 0x7f75c00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00cb370 con 0x7f75c0158850 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c015f370 con 0x7f75c006e090 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c51ca700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c006e5c0 con 0x7f75c015eec0 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f75c0158850 0x7f75c00ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44802/0 (socket says 172.21.15.179:44802) 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bffff700 1 -- 172.21.15.179:0/2895390768 learned_addr learned my addr 172.21.15.179:0/2895390768 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 --2- 172.21.15.179:0/2895390768 >> v2:172.21.15.179:3300/0 conn(0x7f75c006e090 0x7f75c015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75c49c9700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c015eec0 0x7f75c00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75beffd700 1 -- 172.21.15.179:0/2895390768 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304237243 0 0) 0x7f75c00cb370 con 0x7f75c0158850 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75beffd700 1 -- 172.21.15.179:0/2895390768 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ac0035d0 con 0x7f75c0158850 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 -- 172.21.15.179:0/2895390768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c015eec0 msgr2=0x7f75c00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:47.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c015eec0 0x7f75c00cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 -- 172.21.15.179:0/2895390768 >> v1:172.21.15.179:6789/0 conn(0x7f75c0158850 legacy=0x7f75c00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 -- 172.21.15.179:0/2895390768 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c006e8a0 con 0x7f75c006e090 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.549+0000 7f75bf7fe700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c006e090 0x7f75c015e980 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009880 tx=0x7f75a8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2b6115813a6d10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75beffd700 1 -- 172.21.15.179:0/2895390768 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8011830 con 0x7f75c006e090 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75beffd700 1 -- 172.21.15.179:0/2895390768 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8011990 con 0x7f75c006e090 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75beffd700 1 -- 172.21.15.179:0/2895390768 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8011d20 con 0x7f75c006e090 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- 172.21.15.179:0/2895390768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c006e090 msgr2=0x7f75c015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c006e090 0x7f75c015e980 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009880 tx=0x7f75a8009b90 comp rx=0 tx=0).stop 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- 172.21.15.179:0/2895390768 shutdown_connections 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c015eec0 0x7f75c00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- 172.21.15.179:0/2895390768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c006e090 0x7f75c015e980 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- 172.21.15.179:0/2895390768 >> 172.21.15.179:0/2895390768 conn(0x7f75c00cea30 msgr2=0x7f75c00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- 172.21.15.179:0/2895390768 shutdown_connections 2024-05-08T22:42:47.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- 172.21.15.179:0/2895390768 wait complete. 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 Processor -- start 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- start start 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75c006e090 0x7f75c0157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c0158850 0x7f75c015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0142820 con 0x7f75c015eec0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0141f80 con 0x7f75c006e090 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c51ca700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0142260 con 0x7f75c0158850 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c49c9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c49c9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58784/0 (socket says 172.21.15.179:58784) 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75c49c9700 1 -- 172.21.15.179:0/193965293 learned_addr learned my addr 172.21.15.179:0/193965293 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75bffff700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75c006e090 0x7f75c0157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.553+0000 7f75bf7fe700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c0158850 0x7f75c015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 -- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c0158850 msgr2=0x7f75c015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c0158850 0x7f75c015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 -- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75c006e090 msgr2=0x7f75c0157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75c006e090 0x7f75c0157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 -- 172.21.15.179:0/193965293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80094b0 con 0x7f75c015eec0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c49c9700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f75b800eb60 tx=0x7f75b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8009740 con 0x7f75c015eec0 2024-05-08T22:42:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8009e00 con 0x7f75c015eec0 2024-05-08T22:42:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c0142540 con 0x7f75c015eec0 2024-05-08T22:42:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800ca70 con 0x7f75c015eec0 2024-05-08T22:42:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c0141200 con 0x7f75c015eec0 2024-05-08T22:42:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c00b1dd0 con 0x7f75c015eec0 2024-05-08T22:42:47.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.557+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75b800cbd0 con 0x7f75c015eec0 2024-05-08T22:42:47.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.561+0000 7f75bcff9700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 0x7f75a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.561+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f75b80842b0 con 0x7f75c015eec0 2024-05-08T22:42:47.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.561+0000 7f75bffff700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 0x7f75a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.561+0000 7f75bffff700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 0x7f75a0068bf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75b0001560 tx=0x7f75b0001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.565+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75c00b1dd0 con 0x7f75c015eec0 2024-05-08T22:42:47.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.741+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75c00b21b0 con 0x7f75c015eec0 2024-05-08T22:42:47.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.741+0000 7f75bcff9700 1 -- 172.21.15.179:0/193965293 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75c00b21b0 con 0x7f75c015eec0 2024-05-08T22:42:47.744 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 msgr2=0x7f75a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 0x7f75a0068bf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75b0001560 tx=0x7f75b0001a90 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 msgr2=0x7f75c015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f75b800eb60 tx=0x7f75b800ee70 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 shutdown_connections 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f75a0066730 0x7f75a0068bf0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c015eec0 0x7f75c015db80 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75c0158850 0x7f75c015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 --2- 172.21.15.179:0/193965293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75c006e090 0x7f75c0157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 >> 172.21.15.179:0/193965293 conn(0x7f75c00cea30 msgr2=0x7f75c0142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 shutdown_connections 2024-05-08T22:42:47.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.745+0000 7f75c51ca700 1 -- 172.21.15.179:0/193965293 wait complete. 2024-05-08T22:42:47.759 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:47.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.869+0000 7f8147fff700 1 Processor -- start 2024-05-08T22:42:47.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.869+0000 7f8147fff700 1 -- start start 2024-05-08T22:42:47.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f81400e3b40 0x7f81400e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400540b0 con 0x7f8140006150 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400532f0 con 0x7f81400e0250 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81400535d0 con 0x7f81400e3b40 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8146ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f81400e3b40 0x7f81400e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f81477fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8140006150 0x7f81400395d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44812/0 (socket says 172.21.15.179:44812) 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f81477fe700 1 -- 172.21.15.179:0/180585724 learned_addr learned my addr 172.21.15.179:0/180585724 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8145ffb700 1 -- 172.21.15.179:0/180585724 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520862632 0 0) 0x7f81400540b0 con 0x7f8140006150 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8145ffb700 1 -- 172.21.15.179:0/180585724 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81300035d0 con 0x7f8140006150 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8146ffd700 1 -- 172.21.15.179:0/180585724 >> v1:172.21.15.195:6789/0 conn(0x7f81400e0250 legacy=0x7f8140038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8146ffd700 1 -- 172.21.15.179:0/180585724 >> v1:172.21.15.179:6789/0 conn(0x7f8140006150 legacy=0x7f81400395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8146ffd700 1 -- 172.21.15.179:0/180585724 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400538b0 con 0x7f81400e3b40 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8146ffd700 1 --2- 172.21.15.179:0/180585724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81400e3b40 0x7f81400e3f20 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c0df41de02d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8145ffb700 1 -- 172.21.15.179:0/180585724 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c004230 con 0x7f81400e3b40 2024-05-08T22:42:47.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8145ffb700 1 -- 172.21.15.179:0/180585724 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c004390 con 0x7f81400e3b40 2024-05-08T22:42:47.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8145ffb700 1 -- 172.21.15.179:0/180585724 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0036c0 con 0x7f81400e3b40 2024-05-08T22:42:47.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- 172.21.15.179:0/180585724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81400e3b40 msgr2=0x7f81400e3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 --2- 172.21.15.179:0/180585724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81400e3b40 0x7f81400e3f20 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- 172.21.15.179:0/180585724 shutdown_connections 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 --2- 172.21.15.179:0/180585724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81400e3b40 0x7f81400e3f20 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- 172.21.15.179:0/180585724 >> 172.21.15.179:0/180585724 conn(0x7f8140054b70 msgr2=0x7f8140054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- 172.21.15.179:0/180585724 shutdown_connections 2024-05-08T22:42:47.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.873+0000 7f8147fff700 1 -- 172.21.15.179:0/180585724 wait complete. 2024-05-08T22:42:47.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 Processor -- start 2024-05-08T22:42:47.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 -- start start 2024-05-08T22:42:47.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8140006150 0x7f81400e1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 0x7f81400dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814002c4a0 con 0x7f8140177f10 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400e1590 con 0x7f81400e3b40 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f8147fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814002b540 con 0x7f8140006150 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f81477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f81477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58800/0 (socket says 172.21.15.179:58800) 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.877+0000 7f81477fe700 1 -- 172.21.15.179:0/3030610591 learned_addr learned my addr 172.21.15.179:0/3030610591 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f8146ffd700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8140006150 0x7f81400e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81467fc700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 0x7f81400dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 -- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8140006150 msgr2=0x7f81400e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8140006150 0x7f81400e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 -- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 msgr2=0x7f81400dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:47.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 0x7f81400dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 -- 172.21.15.179:0/3030610591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0094b0 con 0x7f8140177f10 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81467fc700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 0x7f81400dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f81477fe700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f814802ce60 tx=0x7f8148067250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81480669e0 con 0x7f8140177f10 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8148066b40 con 0x7f8140177f10 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814002b820 con 0x7f8140177f10 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148066ef0 con 0x7f8140177f10 2024-05-08T22:42:47.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814002bde0 con 0x7f8140177f10 2024-05-08T22:42:47.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814002c5e0 con 0x7f8140177f10 2024-05-08T22:42:47.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8148066150 con 0x7f8140177f10 2024-05-08T22:42:47.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.881+0000 7f812f7fe700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 0x7f8134068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:47.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.885+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f81480fd8e0 con 0x7f8140177f10 2024-05-08T22:42:47.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.885+0000 7f8146ffd700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 0x7f8134068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:47.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.885+0000 7f8146ffd700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 0x7f8134068bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:47.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:47.889+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f814002c5e0 con 0x7f8140177f10 2024-05-08T22:42:48.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.057+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f814002c5e0 con 0x7f8140177f10 2024-05-08T22:42:48.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.057+0000 7f812f7fe700 1 -- 172.21.15.179:0/3030610591 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f814002c5e0 con 0x7f8140177f10 2024-05-08T22:42:48.062 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:48.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 msgr2=0x7f8134068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 0x7f8134068bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c008030 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 msgr2=0x7f814004de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f814802ce60 tx=0x7f8148067250 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 shutdown_connections 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8134066730 0x7f8134068bf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81400e3b40 0x7f81400dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8140177f10 0x7f814004de10 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 --2- 172.21.15.179:0/3030610591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8140006150 0x7f81400e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 >> 172.21.15.179:0/3030610591 conn(0x7f8140054b70 msgr2=0x7f8140052720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 shutdown_connections 2024-05-08T22:42:48.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.061+0000 7f8147fff700 1 -- 172.21.15.179:0/3030610591 wait complete. 2024-05-08T22:42:48.076 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 Processor -- start 2024-05-08T22:42:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 -- start start 2024-05-08T22:42:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45a40ce4e0 0x7f45a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 0x7f45a40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40b1810 con 0x7f45a406b440 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a415d300 con 0x7f45a40ce4e0 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a96b3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40b0a30 con 0x7f45a406c9b0 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45a40ce4e0 0x7f45a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45a40ce4e0 0x7f45a40b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58806/0 (socket says 172.21.15.179:58806) 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 -- 172.21.15.179:0/3446230174 learned_addr learned my addr 172.21.15.179:0/3446230174 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a37fe700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 0x7f45a40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 -- 172.21.15.179:0/3446230174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 msgr2=0x7f45a40cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 0x7f45a40cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 -- 172.21.15.179:0/3446230174 >> v1:172.21.15.195:6789/0 conn(0x7f45a406b440 legacy=0x7f45a406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 -- 172.21.15.179:0/3446230174 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40b1ab0 con 0x7f45a40ce4e0 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a37fe700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 0x7f45a40cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.193+0000 7f45a2ffd700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a40b4dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f459800bbb0 tx=0x7f459800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93f9ab0588e77da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a1ffb700 1 -- 172.21.15.179:0/3446230174 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800dd90 con 0x7f45a40ce4e0 2024-05-08T22:42:48.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a1ffb700 1 -- 172.21.15.179:0/3446230174 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598010470 con 0x7f45a40ce4e0 2024-05-08T22:42:48.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a1ffb700 1 -- 172.21.15.179:0/3446230174 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598010820 con 0x7f45a40ce4e0 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- 172.21.15.179:0/3446230174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 msgr2=0x7f45a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a40b4dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f459800bbb0 tx=0x7f459800bec0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- 172.21.15.179:0/3446230174 shutdown_connections 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406c9b0 0x7f45a40cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- 172.21.15.179:0/3446230174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a40b4dc0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- 172.21.15.179:0/3446230174 >> 172.21.15.179:0/3446230174 conn(0x7f45a4098a70 msgr2=0x7f45a40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- 172.21.15.179:0/3446230174 shutdown_connections 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- 172.21.15.179:0/3446230174 wait complete. 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 Processor -- start 2024-05-08T22:42:48.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- start start 2024-05-08T22:42:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406b440 0x7f45a41f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45a406c9b0 0x7f45a4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c3450 con 0x7f45a40ce4e0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4152bf0 con 0x7f45a406c9b0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a96b3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a41f6f40 con 0x7f45a406b440 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a2ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406b440 0x7f45a41f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58810/0 (socket says 172.21.15.179:58810) 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a27fc700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45a406c9b0 0x7f45a4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.197+0000 7f45a37fe700 1 -- 172.21.15.179:0/1014266785 learned_addr learned my addr 172.21.15.179:0/1014266785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 -- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406b440 msgr2=0x7f45a41f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406b440 0x7f45a41f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 -- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45a406c9b0 msgr2=0x7f45a4153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45a406c9b0 0x7f45a4153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 -- 172.21.15.179:0/1014266785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459800b810 con 0x7f45a40ce4e0 2024-05-08T22:42:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a37fe700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f459c007ef0 tx=0x7f459c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00dd80 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c00f040 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40c2db0 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c003c80 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a40c3070 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a40c3590 con 0x7f45a40ce4e0 2024-05-08T22:42:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.201+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f459c004030 con 0x7f45a40ce4e0 2024-05-08T22:42:48.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.205+0000 7f458b7fe700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 0x7f4590068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.205+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f459c084320 con 0x7f45a40ce4e0 2024-05-08T22:42:48.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.205+0000 7f45a2ffd700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 0x7f4590068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.205+0000 7f45a2ffd700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 0x7f4590068bf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f459800bf00 tx=0x7f459801b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.209+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f459c051560 con 0x7f45a40ce4e0 2024-05-08T22:42:48.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.377+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45a4073290 con 0x7f45a40ce4e0 2024-05-08T22:42:48.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f458b7fe700 1 -- 172.21.15.179:0/1014266785 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f459c054b80 con 0x7f45a40ce4e0 2024-05-08T22:42:48.382 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:48.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 msgr2=0x7f4590068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 0x7f4590068bf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f459800bf00 tx=0x7f459801b040 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 msgr2=0x7f45a4151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f459c007ef0 tx=0x7f459c00d3b0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 shutdown_connections 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4590066730 0x7f4590068bf0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45a406c9b0 0x7f45a4153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45a40ce4e0 0x7f45a4151c50 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 --2- 172.21.15.179:0/1014266785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45a406b440 0x7f45a41f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 >> 172.21.15.179:0/1014266785 conn(0x7f45a4098a70 msgr2=0x7f45a406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 shutdown_connections 2024-05-08T22:42:48.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.381+0000 7f45a96b3700 1 -- 172.21.15.179:0/1014266785 wait complete. 2024-05-08T22:42:48.403 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:42:48.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.513+0000 7ff300863700 1 Processor -- start 2024-05-08T22:42:48.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.513+0000 7ff300863700 1 -- start start 2024-05-08T22:42:48.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.513+0000 7ff300863700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2fc0b6b80 con 0x7ff2fc0afae0 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cb290 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0b00d0 con 0x7ff2fc15afe0 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58820/0 (socket says 172.21.15.179:58820) 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 -- 172.21.15.179:0/747751601 learned_addr learned my addr 172.21.15.179:0/747751601 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:48.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fb7fe700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 -- 172.21.15.179:0/747751601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 msgr2=0x7ff2fc0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 -- 172.21.15.179:0/747751601 >> v1:172.21.15.195:6789/0 conn(0x7ff2fc0afae0 legacy=0x7ff2fc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 -- 172.21.15.179:0/747751601 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc0b03b0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2fbfff700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e50 tx=0x7ff2f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ef4822e7189499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2faffd700 1 -- 172.21.15.179:0/747751601 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400e070 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2faffd700 1 -- 172.21.15.179:0/747751601 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f400b910 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff2faffd700 1 -- 172.21.15.179:0/747751601 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400bcc0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- 172.21.15.179:0/747751601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 msgr2=0x7ff2fc0cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e50 tx=0x7ff2f400b330 comp rx=0 tx=0).stop 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- 172.21.15.179:0/747751601 shutdown_connections 2024-05-08T22:42:48.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0cbb40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 --2- 172.21.15.179:0/747751601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- 172.21.15.179:0/747751601 >> 172.21.15.179:0/747751601 conn(0x7ff2fc0b2730 msgr2=0x7ff2fc0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- 172.21.15.179:0/747751601 shutdown_connections 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.517+0000 7ff300863700 1 -- 172.21.15.179:0/747751601 wait complete. 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 Processor -- start 2024-05-08T22:42:48.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- start start 2024-05-08T22:42:48.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 0x7ff2fc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc1f3860 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc1559d0 con 0x7ff2fc0afae0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc15b6e0 con 0x7ff2fc15afe0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58832/0 (socket says 172.21.15.179:58832) 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 -- 172.21.15.179:0/3901940039 learned_addr learned my addr 172.21.15.179:0/3901940039 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2fb7fe700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 0x7ff2fc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2fbfff700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 -- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 msgr2=0x7ff2fc0b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 -- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 msgr2=0x7ff2fc15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 0x7ff2fc15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 -- 172.21.15.179:0/3901940039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009b30 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2fb7fe700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 0x7ff2fc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f3fff700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c500 tx=0x7ff2ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0052d0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec005430 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0055a0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc15b9c0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.521+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc15bf80 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc04e700 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2ec0038e0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff2f8ff9700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 0x7ff2e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2ec0835b0 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff2fb7fe700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 0x7ff2e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.525+0000 7ff2fb7fe700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 0x7ff2e0068bf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc1f39c0 tx=0x7ff2e4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.533+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff2ec050870 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.713+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff2fc0b3490 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.713+0000 7ff2f8ff9700 1 -- 172.21.15.179:0/3901940039 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff2ec053e90 con 0x7ff2fc0cb6b0 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 msgr2=0x7ff2e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 0x7ff2e0068bf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc1f39c0 tx=0x7ff2e4009240 comp rx=0 tx=0).stop 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 msgr2=0x7ff2fc0af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c500 tx=0x7ff2ec00c810 comp rx=0 tx=0).stop 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 shutdown_connections 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff2e0066730 0x7ff2e0068bf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2fc15afe0 0x7ff2fc0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2fc0cb6b0 0x7ff2fc0af090 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 --2- 172.21.15.179:0/3901940039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc0afae0 0x7ff2fc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 >> 172.21.15.179:0/3901940039 conn(0x7ff2fc0b2730 msgr2=0x7ff2fc0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 shutdown_connections 2024-05-08T22:42:48.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.717+0000 7ff300863700 1 -- 172.21.15.179:0/3901940039 wait complete. 2024-05-08T22:42:48.721 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:42:48.736 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:42:48.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:48.737 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:48.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[49571]: fuse finished with error 0 and tester_r 0 2024-05-08T22:42:48.749 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:42:48.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:42:48.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:48.778 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:42:48.786 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:48.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.897+0000 7f199095d700 1 Processor -- start 2024-05-08T22:42:48.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.897+0000 7f199095d700 1 -- start start 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f198c1434d0 0x7f198c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0cd200 con 0x7f198c06ed50 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0cd4e0 con 0x7f198c06dc20 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0cd7c0 con 0x7f198c1434d0 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198b59e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f198c06ed50 0x7f198c0b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44818/0 (socket says 172.21.15.179:44818) 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198b59e700 1 -- 172.21.15.179:0/1140473830 learned_addr learned my addr 172.21.15.179:0/1140473830 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:48.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198ad9d700 1 --2- 172.21.15.179:0/1140473830 >> v2:172.21.15.195:3300/0 conn(0x7f198c1434d0 0x7f198c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198a59c700 1 -- 172.21.15.179:0/1140473830 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353782399 0 0) 0x7f198c0cd200 con 0x7f198c06ed50 2024-05-08T22:42:48.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198a59c700 1 -- 172.21.15.179:0/1140473830 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19780035d0 con 0x7f198c06ed50 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198ad9d700 1 -- 172.21.15.179:0/1140473830 >> v1:172.21.15.195:6789/0 conn(0x7f198c06dc20 legacy=0x7f198c06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198ad9d700 1 -- 172.21.15.179:0/1140473830 >> v1:172.21.15.179:6789/0 conn(0x7f198c06ed50 legacy=0x7f198c0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198ad9d700 1 -- 172.21.15.179:0/1140473830 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0cdaa0 con 0x7f198c1434d0 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198ad9d700 1 --2- 172.21.15.179:0/1140473830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c1434d0 0x7f198c06d6e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f197c009850 tx=0x7f197c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ceeac9edf68dac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198a59c700 1 -- 172.21.15.179:0/1140473830 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011830 con 0x7f198c1434d0 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198a59c700 1 -- 172.21.15.179:0/1140473830 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c011990 con 0x7f198c1434d0 2024-05-08T22:42:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f198a59c700 1 -- 172.21.15.179:0/1140473830 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011d40 con 0x7f198c1434d0 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- 172.21.15.179:0/1140473830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c1434d0 msgr2=0x7f198c06d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 --2- 172.21.15.179:0/1140473830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c1434d0 0x7f198c06d6e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f197c009850 tx=0x7f197c009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- 172.21.15.179:0/1140473830 shutdown_connections 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 --2- 172.21.15.179:0/1140473830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c1434d0 0x7f198c06d6e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- 172.21.15.179:0/1140473830 >> 172.21.15.179:0/1140473830 conn(0x7f198c0ce9d0 msgr2=0x7f198c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:48.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- 172.21.15.179:0/1140473830 shutdown_connections 2024-05-08T22:42:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.901+0000 7f199095d700 1 -- 172.21.15.179:0/1140473830 wait complete. 2024-05-08T22:42:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 Processor -- start 2024-05-08T22:42:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- start start 2024-05-08T22:42:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c06ed50 0x7f198c1e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c0cac60 0x7f198c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0c9bc0 con 0x7f198c1434d0 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0c9ea0 con 0x7f198c0cac60 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0ca180 con 0x7f198c06ed50 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58838/0 (socket says 172.21.15.179:58838) 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 -- 172.21.15.179:0/2086722008 learned_addr learned my addr 172.21.15.179:0/2086722008 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198ad9d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c06ed50 0x7f198c1e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198b59e700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c0cac60 0x7f198c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 -- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c06ed50 msgr2=0x7f198c1e27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c06ed50 0x7f198c1e27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 -- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c0cac60 msgr2=0x7f198c0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:48.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c0cac60 0x7f198c0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 -- 172.21.15.179:0/2086722008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0094b0 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f198259c700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f197400c500 tx=0x7f197400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740052d0 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974005430 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c0ca460 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740055a0 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.905+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c0c97f0 con 0x7f198c1434d0 2024-05-08T22:42:48.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.909+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c0ca980 con 0x7f198c1434d0 2024-05-08T22:42:48.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.909+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f19740038e0 con 0x7f198c1434d0 2024-05-08T22:42:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.909+0000 7f1983fff700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 0x7f196c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.909+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f19740837f0 con 0x7f198c1434d0 2024-05-08T22:42:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.909+0000 7f198ad9d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 0x7f196c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.913+0000 7f198ad9d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 0x7f196c068bf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f197c0090b0 tx=0x7f197c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:48.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:48.913+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f198c0ca980 con 0x7f198c1434d0 2024-05-08T22:42:49.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.121+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c0ca980 con 0x7f198c1434d0 2024-05-08T22:42:49.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.121+0000 7f1983fff700 1 -- 172.21.15.179:0/2086722008 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f198c0ca980 con 0x7f198c1434d0 2024-05-08T22:42:49.126 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:49.126 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:42:40.300797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57425":{"gid":57425,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/4059323811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4059323811},{"type":"v1","addr":"172.21.15.195:6837","nonce":4059323811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 msgr2=0x7f196c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 0x7f196c068bf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f197c0090b0 tx=0x7f197c01c040 comp rx=0 tx=0).stop 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 msgr2=0x7f198c13b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f197400c500 tx=0x7f197400c8c0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 shutdown_connections 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f196c066730 0x7f196c068bf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c0cac60 0x7f198c0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f198c1434d0 0x7f198c13b3c0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 --2- 172.21.15.179:0/2086722008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c06ed50 0x7f198c1e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 >> 172.21.15.179:0/2086722008 conn(0x7f198c0ce9d0 msgr2=0x7f198c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 shutdown_connections 2024-05-08T22:42:49.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.125+0000 7f199095d700 1 -- 172.21.15.179:0/2086722008 wait complete. 2024-05-08T22:42:49.129 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 75 2024-05-08T22:42:49.145 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T22:42:49.288 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:42:49.289 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:42:49.289 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:42:49.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.405+0000 7f584759e700 1 Processor -- start 2024-05-08T22:42:49.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.405+0000 7f584759e700 1 -- start start 2024-05-08T22:42:49.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584815e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58480b45b0 0x7f584815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480b5180 con 0x7f58480b41d0 2024-05-08T22:42:49.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480cf0d0 con 0x7f58480b45b0 2024-05-08T22:42:49.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480ceb10 con 0x7f58480b4ca0 2024-05-08T22:42:49.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f583ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58480b45b0 0x7f584815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f583ffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:51856/0 (socket says 172.21.15.179:51856) 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58480b45b0 0x7f584815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:50204/0 (socket says 172.21.15.179:50204) 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f583ffff700 1 -- 172.21.15.179:0/4185428231 learned_addr learned my addr 172.21.15.179:0/4185428231 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584659c700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f58480b41d0 0x7f58480ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:52976/0 (socket says 172.21.15.179:52976) 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 -- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 msgr2=0x7f584815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 --2- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584815e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 -- 172.21.15.179:0/4185428231 >> v1:172.21.15.195:6789/0 conn(0x7f58480b41d0 legacy=0x7f58480ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 -- 172.21.15.179:0/4185428231 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480cedf0 con 0x7f58480b45b0 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5846d9d700 1 --2- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b45b0 0x7f584815e790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5840009d90 tx=0x7f584000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66cd806e60211b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5845d9b700 1 -- 172.21.15.179:0/4185428231 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000e070 con 0x7f58480b45b0 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5845d9b700 1 -- 172.21.15.179:0/4185428231 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000bb20 con 0x7f58480b45b0 2024-05-08T22:42:49.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f5845d9b700 1 -- 172.21.15.179:0/4185428231 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840012430 con 0x7f58480b45b0 2024-05-08T22:42:49.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b45b0 msgr2=0x7f584815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 --2- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b45b0 0x7f584815e790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5840009d90 tx=0x7f584000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- 172.21.15.179:0/4185428231 shutdown_connections 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 --2- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b45b0 0x7f584815e790 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 --2- 172.21.15.179:0/4185428231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- 172.21.15.179:0/4185428231 >> 172.21.15.179:0/4185428231 conn(0x7f5848098a70 msgr2=0x7f58480b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- 172.21.15.179:0/4185428231 shutdown_connections 2024-05-08T22:42:49.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.409+0000 7f584759e700 1 -- 172.21.15.179:0/4185428231 wait complete. 2024-05-08T22:42:49.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.413+0000 7f584759e700 1 Processor -- start 2024-05-08T22:42:49.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.413+0000 7f584759e700 1 -- start start 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b41d0 0x7f58480b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b1910 con 0x7f58480b45b0 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584814c580 con 0x7f58480b41d0 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584814c820 con 0x7f58480b4ca0 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58850/0 (socket says 172.21.15.179:58850) 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 -- 172.21.15.179:0/4266022808 learned_addr learned my addr 172.21.15.179:0/4266022808 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:49.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584659c700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b41d0 0x7f58480b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f5846d9d700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 -- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 msgr2=0x7f584814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584814da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 -- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b41d0 msgr2=0x7f58480b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b41d0 0x7f58480b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 -- 172.21.15.179:0/4266022808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840009a70 con 0x7f58480b45b0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583ffff700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f583800e8c0 tx=0x7f583800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838013680 con 0x7f58480b45b0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380137e0 con 0x7f58480b45b0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584814cb00 con 0x7f58480b45b0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838013b70 con 0x7f58480b45b0 2024-05-08T22:42:49.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.417+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584814d0c0 con 0x7f58480b45b0 2024-05-08T22:42:49.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584814d280 con 0x7f58480b45b0 2024-05-08T22:42:49.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5838013cd0 con 0x7f58480b45b0 2024-05-08T22:42:49.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f583effd700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 0x7f5828068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f58380841a0 con 0x7f58480b45b0 2024-05-08T22:42:49.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f584659c700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 0x7f5828068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.421+0000 7f584659c700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 0x7f5828068bf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f60 tx=0x7f5830007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.425+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f58380513c0 con 0x7f58480b45b0 2024-05-08T22:42:49.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.633+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584804e700 con 0x7f58480b45b0 2024-05-08T22:42:49.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f583effd700 1 -- 172.21.15.179:0/4266022808 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f58380549e0 con 0x7f58480b45b0 2024-05-08T22:42:49.638 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:49.639 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:42:40.300797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57425":{"gid":57425,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/4059323811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4059323811},{"type":"v1","addr":"172.21.15.195:6837","nonce":4059323811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 msgr2=0x7f5828068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 0x7f5828068bf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f60 tx=0x7f5830007f30 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 msgr2=0x7f5848152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f583800e8c0 tx=0x7f583800ebd0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 shutdown_connections 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5828066730 0x7f5828068bf0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58480b4ca0 0x7f584814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480b45b0 0x7f5848152e30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 --2- 172.21.15.179:0/4266022808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58480b41d0 0x7f58480b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 >> 172.21.15.179:0/4266022808 conn(0x7f5848098a70 msgr2=0x7f584806bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 shutdown_connections 2024-05-08T22:42:49.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.637+0000 7f584759e700 1 -- 172.21.15.179:0/4266022808 wait complete. 2024-05-08T22:42:49.642 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 75 2024-05-08T22:42:49.657 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-08T22:42:49.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 Processor -- start 2024-05-08T22:42:49.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 -- start start 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3d750 0x55d0b3f3dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b4010620 0x55d0b409a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3ece2f0 con 0x55d0b3f3f780 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e8a8e0 con 0x55d0b3f3d750 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e8a580 con 0x55d0b4010620 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15abf7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.689+0000 7fe15abf7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58864/0 (socket says 172.21.15.179:58864) 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 -- 172.21.15.179:0/2376640170 learned_addr learned my addr 172.21.15.179:0/2376640170 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15b3f8700 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3d750 0x55d0b3f3dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15bbf9700 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b4010620 0x55d0b409a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 -- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b4010620 msgr2=0x55d0b409a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:49.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b4010620 0x55d0b409a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 -- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3d750 msgr2=0x55d0b3f3dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:49.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3d750 0x55d0b3f3dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 -- 172.21.15.179:0/2376640170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c00a100 con 0x55d0b3f3f780 2024-05-08T22:42:49.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15abf7700 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe144001d30 tx=0x7fe14400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff35a4f43c5cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe159bf5700 1 -- 172.21.15.179:0/2376640170 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400bd70 con 0x55d0b3f3f780 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe159bf5700 1 -- 172.21.15.179:0/2376640170 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400e470 con 0x55d0b3f3f780 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe159bf5700 1 -- 172.21.15.179:0/2376640170 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400e800 con 0x55d0b3f3f780 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 -- 172.21.15.179:0/2376640170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 msgr2=0x55d0b3f84a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe144001d30 tx=0x7fe14400b5d0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 -- 172.21.15.179:0/2376640170 shutdown_connections 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b4010620 0x55d0b409a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b3f3f780 0x55d0b3f84a50 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/2376640170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3d750 0x55d0b3f3dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 -- 172.21.15.179:0/2376640170 >> 172.21.15.179:0/2376640170 conn(0x55d0b3f68110 msgr2=0x55d0b4013940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 -- 172.21.15.179:0/2376640170 shutdown_connections 2024-05-08T22:42:49.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 -- 172.21.15.179:0/2376640170 wait complete. 2024-05-08T22:42:49.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.693+0000 7fe15d8c3140 1 Processor -- start 2024-05-08T22:42:49.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- start start 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b3f3d750 0x55d0b3f7d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3f780 0x55d0b400bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 0x55d0b3f65e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e608d0 con 0x55d0b4010620 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b40a6e20 con 0x55d0b3f3f780 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0b40a7050 con 0x55d0b3f3d750 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 0x55d0b3f65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 0x55d0b3f65e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58880/0 (socket says 172.21.15.179:58880) 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15abf7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3f780 0x55d0b400bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 -- 172.21.15.179:0/4044351441 learned_addr learned my addr 172.21.15.179:0/4044351441 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15b3f8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b3f3d750 0x55d0b3f7d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 -- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b3f3d750 msgr2=0x55d0b3f7d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d0b3f3d750 0x55d0b3f7d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 -- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3f780 msgr2=0x55d0b400bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d0b3f3f780 0x55d0b400bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 -- 172.21.15.179:0/4044351441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0b3e6bbb0 con 0x55d0b4010620 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15bbf9700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 0x55d0b3f65e80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fe154008400 tx=0x7fe154008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe158bf3700 1 -- 172.21.15.179:0/4044351441 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400f050 con 0x55d0b4010620 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe158bf3700 1 -- 172.21.15.179:0/4044351441 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154010040 con 0x55d0b4010620 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d0b3e78b00 con 0x55d0b4010620 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe158bf3700 1 -- 172.21.15.179:0/4044351441 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154015880 con 0x55d0b4010620 2024-05-08T22:42:49.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.697+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0b3e6bf40 con 0x55d0b4010620 2024-05-08T22:42:49.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe158bf3700 1 -- 172.21.15.179:0/4044351441 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1540159e0 con 0x55d0b4010620 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe158bf3700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe13c065060 0x7fe13c067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe158bf3700 1 -- 172.21.15.179:0/4044351441 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe154083140 con 0x55d0b4010620 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15b3f8700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe13c065060 0x7fe13c067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55d0b40a7f70 0x55d0b40a8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v8 -- 0x55d0b40a88c0 con 0x55d0b40a7f70 2024-05-08T22:42:49.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15abf7700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55d0b40a7f70 0x55d0b40a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15b3f8700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe13c065060 0x7fe13c067520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f80 tx=0x7fe14c007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.701+0000 7fe15abf7700 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55d0b40a7f70 0x55d0b40a8380 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15abf7700 1 -- 172.21.15.179:0/4044351441 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7fe144021040 con 0x55d0b40a7f70 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55d0b40a7f70 msgr2=0x55d0b40a8380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55d0b40a7f70 0x55d0b40a8380 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe13c065060 msgr2=0x7fe13c067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe13c065060 0x7fe13c067520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f80 tx=0x7fe14c007030 comp rx=0 tx=0).stop 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 msgr2=0x55d0b3f65e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 --2- 172.21.15.179:0/4044351441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0b4010620 0x55d0b3f65e80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fe154008400 tx=0x7fe154008710 comp rx=0 tx=0).stop 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15bbf9700 1 -- 172.21.15.179:0/4044351441 reap_dead start 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 shutdown_connections 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 >> 172.21.15.179:0/4044351441 conn(0x55d0b3f68110 msgr2=0x55d0b3f3e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 shutdown_connections 2024-05-08T22:42:49.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.705+0000 7fe15d8c3140 1 -- 172.21.15.179:0/4044351441 wait complete. 2024-05-08T22:42:49.711 INFO:teuthology.orchestra.run.smithi179.stdout:����� 2024-05-08T22:42:49.711 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:42:49.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 Processor -- start 2024-05-08T22:42:49.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 -- start start 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8ae415d5b0 0x7f8ae40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40b2120 0x7f8ae40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae406afc0 con 0x7f8ae40ce3b0 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae40ce840 con 0x7f8ae415d5b0 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aeabe2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae415d990 con 0x7f8ae40b2120 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aea3e1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8ae40ce3b0 0x7f8ae4161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44836/0 (socket says 172.21.15.179:44836) 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8aea3e1700 1 -- 172.21.15.179:0/3292795348 learned_addr learned my addr 172.21.15.179:0/3292795348 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3292795348 >> v2:172.21.15.195:3300/0 conn(0x7f8ae415d5b0 0x7f8ae40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae93df700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40b2120 0x7f8ae40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae8bde700 1 -- 172.21.15.179:0/3292795348 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741794089 0 0) 0x7f8ae406afc0 con 0x7f8ae40ce3b0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae8bde700 1 -- 172.21.15.179:0/3292795348 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad00035d0 con 0x7f8ae40ce3b0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40b2120 msgr2=0x7f8ae40cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40b2120 0x7f8ae40cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3292795348 >> v1:172.21.15.179:6789/0 conn(0x7f8ae40ce3b0 legacy=0x7f8ae4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3292795348 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae406c460 con 0x7f8ae415d5b0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40b1be0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009850 tx=0x7f8acc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e627d02ae721348 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae8bde700 1 -- 172.21.15.179:0/3292795348 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011830 con 0x7f8ae415d5b0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae8bde700 1 -- 172.21.15.179:0/3292795348 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc011990 con 0x7f8ae415d5b0 2024-05-08T22:42:49.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.825+0000 7f8ae8bde700 1 -- 172.21.15.179:0/3292795348 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011d00 con 0x7f8ae415d5b0 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 msgr2=0x7f8ae40b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40b1be0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009850 tx=0x7f8acc009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3292795348 shutdown_connections 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40b2120 0x7f8ae40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3292795348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40b1be0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3292795348 >> 172.21.15.179:0/3292795348 conn(0x7f8ae40b4940 msgr2=0x7f8ae40b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3292795348 shutdown_connections 2024-05-08T22:42:49.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.829+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3292795348 wait complete. 2024-05-08T22:42:49.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 Processor -- start 2024-05-08T22:42:49.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- start start 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40ce3b0 0x7f8ae40a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae414a980 con 0x7f8ae40b2120 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40aa230 con 0x7f8ae415d5b0 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4160790 con 0x7f8ae40ce3b0 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58882/0 (socket says 172.21.15.179:58882) 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3414348670 learned_addr learned my addr 172.21.15.179:0/3414348670 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:49.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae93df700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40ce3b0 0x7f8ae40a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aea3e1700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40ce3b0 msgr2=0x7f8ae40a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40ce3b0 0x7f8ae40a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 msgr2=0x7f8ae40a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 -- 172.21.15.179:0/3414348670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc0094b0 con 0x7f8ae40b2120 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aea3e1700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae9be0700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8ae408b480 tx=0x7f8acc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004550 con 0x7f8ae40b2120 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc01e070 con 0x7f8ae40b2120 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae4149fe0 con 0x7f8ae40b2120 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc003900 con 0x7f8ae40b2120 2024-05-08T22:42:49.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.833+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae414a5a0 con 0x7f8ae40b2120 2024-05-08T22:42:49.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae414aac0 con 0x7f8ae40b2120 2024-05-08T22:42:49.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8acc003a60 con 0x7f8ae40b2120 2024-05-08T22:42:49.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8ae27fc700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 0x7f8ac4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:49.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8acc0851e0 con 0x7f8ae40b2120 2024-05-08T22:42:49.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8ae93df700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 0x7f8ac4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:49.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.837+0000 7f8ae93df700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 0x7f8ac4068bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4160dd0 tx=0x7f8ad400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:49.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:49.841+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ae414aac0 con 0x7f8ae40b2120 2024-05-08T22:42:50.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.053+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8ae414aac0 con 0x7f8ae40b2120 2024-05-08T22:42:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.053+0000 7f8ae27fc700 1 -- 172.21.15.179:0/3414348670 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8ae414aac0 con 0x7f8ae40b2120 2024-05-08T22:42:50.071 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 msgr2=0x7f8ac4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 0x7f8ac4068bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4160dd0 tx=0x7f8ad400b040 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 msgr2=0x7f8ae40c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8ae408b480 tx=0x7f8acc004770 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 shutdown_connections 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8ac4066730 0x7f8ac4068bf0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ae415d5b0 0x7f8ae40a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ae40ce3b0 0x7f8ae40a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 --2- 172.21.15.179:0/3414348670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae40b2120 0x7f8ae40c5c20 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 >> 172.21.15.179:0/3414348670 conn(0x7f8ae40b4940 msgr2=0x7f8ae40c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 shutdown_connections 2024-05-08T22:42:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:50.057+0000 7f8aeabe2700 1 -- 172.21.15.179:0/3414348670 wait complete. 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:42:50.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:42:50.075 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:42:50.075 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:50.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.075 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.081 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:42:50.082 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:42:50.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.083 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:42:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:42:50.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.142 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:42:50.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:42:50.150 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:42:50.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.189 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:42:50.236 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:42:50.236 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:42:50.237 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:50.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.237 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:42:50.251 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:42:50.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:42:50.251 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:50.301 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:42:50.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:42:50.301 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:50.307 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:50.326 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:50.326 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:50.384 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:50.385 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:50.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.385 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:50.390 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:42:50.391 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:42:50.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:50.406 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:50.406 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:50.465 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:50.466 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:50.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:50.467 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:50.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:42:50.481+0000 7fd583a21640 -1 init, newargv = 0x55c83f1c78a0 newargc=15 2024-05-08T22:42:50.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[50034]: starting ceph client 2024-05-08T22:42:50.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[50034]: starting fuse 2024-05-08T22:42:51.573 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:42:51.592 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:42:51.592 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:42:51.602 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:42:51.603 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:42:51.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:51.604 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:42:51.609 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:42:51.609 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:42:51.609 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:42:51.610 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:42:51.611 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:51.733 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.50034.asok 2024-05-08T22:42:51.916 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50034.asok 2024-05-08T22:42:51.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:51.916 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50034.asok status 2024-05-08T22:42:52.020 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:42:52.020 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:42:52.020 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "50034", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9141, 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 9141 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:52.021 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 4112984249 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 4112984249 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.9141 192.168.0.1:0/4112984249", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/4112984249", 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:42:52.022 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 75, 2024-05-08T22:42:52.023 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 112, 2024-05-08T22:42:52.023 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:42:52.023 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:42:52.023 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:42:52.023 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:42:52.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:52.030 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:42:52.030 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:52.038 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:42:52.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:52.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:52.038 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:52.050 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:52.051 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:52.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:52.101 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T22:42:52.198 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records in 2024-05-08T22:42:52.198 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records out 2024-05-08T22:42:52.198 INFO:teuthology.orchestra.run.smithi179.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0810275 s, 77.6 MB/s 2024-05-08T22:42:52.199 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:52.199 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:52.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:52.205 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T22:42:52.326 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records in 2024-05-08T22:42:52.326 INFO:teuthology.orchestra.run.smithi179.stderr:6+0 records out 2024-05-08T22:42:52.326 INFO:teuthology.orchestra.run.smithi179.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0652357 s, 96.4 MB/s 2024-05-08T22:42:52.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:52.327 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T22:42:52.340 INFO:teuthology.orchestra.run.smithi179.stdout:./ 2024-05-08T22:42:52.340 INFO:teuthology.orchestra.run.smithi179.stdout:./file2_sixmegs 2024-05-08T22:42:52.340 INFO:teuthology.orchestra.run.smithi179.stdout:./file3_sixmegs 2024-05-08T22:42:52.340 INFO:teuthology.orchestra.run.smithi179.stdout:./file1_sixmegs 2024-05-08T22:42:52.341 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:52.341 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:52.342 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:42:52.342 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:52.342 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T22:42:52.342 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:52.411 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:42:52.599 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:52.599 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:52.600 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:42:52.600 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:52.600 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-08T22:42:52.600 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:52.713 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:42:52.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:42:52.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:42:52.901 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:52.902 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:52.902 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:42:52.902 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:52.902 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-08T22:42:52.902 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:52.975 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627778 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-08T22:42:53.163 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:42:53.275 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:42:53.464 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:53.464 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:53.472 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:53.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.589+0000 7fbba9184700 1 Processor -- start 2024-05-08T22:42:53.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.589+0000 7fbba9184700 1 -- start start 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba9184700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba415e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba9184700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbba406e8d0 0x7fbba40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba9184700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40cdb20 con 0x7fbba406c550 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba9184700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40cde00 con 0x7fbba406e8d0 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba9184700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40ce0e0 con 0x7fbba41432f0 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbba406c550 0x7fbba406e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44850/0 (socket says 172.21.15.179:44850) 2024-05-08T22:42:53.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbba406e8d0 0x7fbba40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba37fe700 1 -- 172.21.15.179:0/3833041240 learned_addr learned my addr 172.21.15.179:0/3833041240 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba3fff700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba415e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 -- 172.21.15.179:0/3833041240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 msgr2=0x7fbba415e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba415e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 -- 172.21.15.179:0/3833041240 >> v1:172.21.15.179:6789/0 conn(0x7fbba406c550 legacy=0x7fbba406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 -- 172.21.15.179:0/3833041240 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40ce3c0 con 0x7fbba406e8d0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba8983700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406e8d0 0x7fbba40ca140 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009dc0 tx=0x7fbb9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1fe15ba78ba35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba2ffd700 1 -- 172.21.15.179:0/3833041240 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00bce0 con 0x7fbba406e8d0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba2ffd700 1 -- 172.21.15.179:0/3833041240 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c00be40 con 0x7fbba406e8d0 2024-05-08T22:42:53.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.593+0000 7fbba2ffd700 1 -- 172.21.15.179:0/3833041240 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0116a0 con 0x7fbba406e8d0 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- 172.21.15.179:0/3833041240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406e8d0 msgr2=0x7fbba40ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406e8d0 0x7fbba40ca140 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009dc0 tx=0x7fbb9c00b330 comp rx=0 tx=0).stop 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- 172.21.15.179:0/3833041240 shutdown_connections 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406e8d0 0x7fbba40ca140 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- 172.21.15.179:0/3833041240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba415e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- 172.21.15.179:0/3833041240 >> 172.21.15.179:0/3833041240 conn(0x7fbba40b17c0 msgr2=0x7fbba40b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- 172.21.15.179:0/3833041240 shutdown_connections 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- 172.21.15.179:0/3833041240 wait complete. 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 Processor -- start 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- start start 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 0x7fbba40c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4140db0 con 0x7fbba406c550 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4141090 con 0x7fbba406e8d0 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba9184700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4141370 con 0x7fbba41432f0 2024-05-08T22:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58888/0 (socket says 172.21.15.179:58888) 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba3fff700 1 -- 172.21.15.179:0/4141440799 learned_addr learned my addr 172.21.15.179:0/4141440799 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba37fe700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 0x7fbba40c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.597+0000 7fbba8983700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 -- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 msgr2=0x7fbba40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba40c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 -- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 msgr2=0x7fbba40c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 0x7fbba40c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 -- 172.21.15.179:0/4141440799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c009a70 con 0x7fbba406c550 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba37fe700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 0x7fbba40c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba8983700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba3fff700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e840 tx=0x7fbb9400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb940095a0 con 0x7fbba406c550 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9400c690 con 0x7fbba406c550 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba4141650 con 0x7fbba406c550 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400ca00 con 0x7fbba406c550 2024-05-08T22:42:53.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba4158310 con 0x7fbba406c550 2024-05-08T22:42:53.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba404e700 con 0x7fbba406c550 2024-05-08T22:42:53.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.601+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb9400cb60 con 0x7fbba406c550 2024-05-08T22:42:53.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.605+0000 7fbba0ff9700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 0x7fbb90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.605+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb94084ae0 con 0x7fbba406c550 2024-05-08T22:42:53.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.605+0000 7fbba37fe700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 0x7fbb90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.605+0000 7fbba37fe700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 0x7fbb90068bf0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbba40b21f0 tx=0x7fbb98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.609+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb94051cf0 con 0x7fbba406c550 2024-05-08T22:42:53.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.781+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbba40cf230 con 0x7fbba406c550 2024-05-08T22:42:53.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba0ff9700 1 -- 172.21.15.179:0/4141440799 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16296 (secure 0 0 0) 0x7fbb94055310 con 0x7fbba406c550 2024-05-08T22:42:53.787 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:53.788 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":112,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:38.288356+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 msgr2=0x7fbb90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 0x7fbb90068bf0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbba40b21f0 tx=0x7fbb98009040 comp rx=0 tx=0).stop 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 msgr2=0x7fbba4157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e840 tx=0x7fbb9400ec00 comp rx=0 tx=0).stop 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 shutdown_connections 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbb90066730 0x7fbb90068bf0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba406e8d0 0x7fbba40c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba41432f0 0x7fbba40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 --2- 172.21.15.179:0/4141440799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbba406c550 0x7fbba4157dd0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 >> 172.21.15.179:0/4141440799 conn(0x7fbba40b17c0 msgr2=0x7fbba40caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:53.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 shutdown_connections 2024-05-08T22:42:53.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.785+0000 7fbba9184700 1 -- 172.21.15.179:0/4141440799 wait complete. 2024-05-08T22:42:53.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:42:35.609486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:42:36.273848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:53.803 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:53.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 Processor -- start 2024-05-08T22:42:53.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 -- start start 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 0x7f70780b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70780d1d20 0x7f70780cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70780ce0e0 0x7f7078160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70780ce570 con 0x7f70780ce0e0 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707815d4a0 con 0x7f707815d0c0 2024-05-08T22:42:53.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707f4f0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707806abb0 con 0x7f70780d1d20 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.921+0000 7f707ecef700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70780ce0e0 0x7f7078160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70780ce0e0 0x7f7078160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58900/0 (socket says 172.21.15.179:58900) 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 -- 172.21.15.179:0/3084119279 learned_addr learned my addr 172.21.15.179:0/3084119279 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707e4ee700 1 --2- 172.21.15.179:0/3084119279 >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 0x7f70780b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707dced700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70780d1d20 0x7f70780cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 -- 172.21.15.179:0/3084119279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70780d1d20 msgr2=0x7f70780cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70780d1d20 0x7f70780cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 -- 172.21.15.179:0/3084119279 >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 msgr2=0x7f70780b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 --2- 172.21.15.179:0/3084119279 >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 0x7f70780b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 -- 172.21.15.179:0/3084119279 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780d2500 con 0x7f70780ce0e0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707e4ee700 1 --2- 172.21.15.179:0/3084119279 >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 0x7f70780b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707ecef700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f7078160ba0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f707808b1c0 tx=0x7f707000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433dcf711106153c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707d4ec700 1 -- 172.21.15.179:0/3084119279 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000e070 con 0x7f70780ce0e0 2024-05-08T22:42:53.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707d4ec700 1 -- 172.21.15.179:0/3084119279 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707000b9c0 con 0x7f70780ce0e0 2024-05-08T22:42:53.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707d4ec700 1 -- 172.21.15.179:0/3084119279 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000bd70 con 0x7f70780ce0e0 2024-05-08T22:42:53.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 -- 172.21.15.179:0/3084119279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 msgr2=0x7f7078160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f7078160ba0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f707808b1c0 tx=0x7f707000b330 comp rx=0 tx=0).stop 2024-05-08T22:42:53.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 -- 172.21.15.179:0/3084119279 shutdown_connections 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f7078160ba0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 --2- 172.21.15.179:0/3084119279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70780d1d20 0x7f70780cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 --2- 172.21.15.179:0/3084119279 >> v2:172.21.15.195:3300/0 conn(0x7f707815d0c0 0x7f70780b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 -- 172.21.15.179:0/3084119279 >> 172.21.15.179:0/3084119279 conn(0x7f70780d2ed0 msgr2=0x7f70780d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 -- 172.21.15.179:0/3084119279 shutdown_connections 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.925+0000 7f707f4f0700 1 -- 172.21.15.179:0/3084119279 wait complete. 2024-05-08T22:42:53.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 Processor -- start 2024-05-08T22:42:53.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- start start 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70780d1d20 0x7f707815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707815d0c0 0x7f70780b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707815ede0 con 0x7f70780ce0e0 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707815e160 con 0x7f70780d1d20 2024-05-08T22:42:53.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707815e440 con 0x7f707815d0c0 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707dced700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70780d1d20 0x7f707815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58916/0 (socket says 172.21.15.179:58916) 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 -- 172.21.15.179:0/2991596126 learned_addr learned my addr 172.21.15.179:0/2991596126 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707ecef700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707815d0c0 0x7f70780b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 -- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707815d0c0 msgr2=0x7f70780b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707815d0c0 0x7f70780b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 -- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70780d1d20 msgr2=0x7f707815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70780d1d20 0x7f707815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 -- 172.21.15.179:0/2991596126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009b30 con 0x7f70780ce0e0 2024-05-08T22:42:53.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707e4ee700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f706800c770 tx=0x7f706800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068011800 con 0x7f70780ce0e0 2024-05-08T22:42:53.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707815e720 con 0x7f70780ce0e0 2024-05-08T22:42:53.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068011960 con 0x7f70780ce0e0 2024-05-08T22:42:53.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068011d10 con 0x7f70780ce0e0 2024-05-08T22:42:53.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.929+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078160670 con 0x7f70780ce0e0 2024-05-08T22:42:53.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.933+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780cb4c0 con 0x7f70780ce0e0 2024-05-08T22:42:53.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.933+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7068018360 con 0x7f70780ce0e0 2024-05-08T22:42:53.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.933+0000 7f706effd700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 0x7f705c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:53.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.933+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7068084180 con 0x7f70780ce0e0 2024-05-08T22:42:53.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.933+0000 7f707dced700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 0x7f705c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:53.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.937+0000 7f707dced700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 0x7f705c068bf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f70780cd1a0 tx=0x7f7074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:53.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:53.941+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f70680513c0 con 0x7f70780ce0e0 2024-05-08T22:42:54.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.109+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f707815ef20 con 0x7f70780ce0e0 2024-05-08T22:42:54.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f706effd700 1 -- 172.21.15.179:0/2991596126 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f707815ef20 con 0x7f70780ce0e0 2024-05-08T22:42:54.115 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:54.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 msgr2=0x7f705c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 0x7f705c068bf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f70780cd1a0 tx=0x7f7074008040 comp rx=0 tx=0).stop 2024-05-08T22:42:54.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 msgr2=0x7f70781563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f706800c770 tx=0x7f706800ca80 comp rx=0 tx=0).stop 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 shutdown_connections 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f705c066730 0x7f705c068bf0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70780d1d20 0x7f707815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707815d0c0 0x7f70780b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 --2- 172.21.15.179:0/2991596126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780ce0e0 0x7f70781563f0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 >> 172.21.15.179:0/2991596126 conn(0x7f70780d2ed0 msgr2=0x7f707806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 shutdown_connections 2024-05-08T22:42:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.113+0000 7f707f4f0700 1 -- 172.21.15.179:0/2991596126 wait complete. 2024-05-08T22:42:54.129 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:54.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 Processor -- start 2024-05-08T22:42:54.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 -- start start 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ee4158850 0x7f7ee40ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 0x7f7ee40cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee40cb370 con 0x7f7ee406e090 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee406e5c0 con 0x7f7ee4158850 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee9f60700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee406e8a0 con 0x7f7ee415f090 2024-05-08T22:42:54.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ee4158850 0x7f7ee40ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee3fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7ee406e090 0x7f7ee415e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44864/0 (socket says 172.21.15.179:44864) 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee3fff700 1 -- 172.21.15.179:0/1836290443 learned_addr learned my addr 172.21.15.179:0/1836290443 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee975f700 1 --2- 172.21.15.179:0/1836290443 >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 0x7f7ee40cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee37fe700 1 -- 172.21.15.179:0/1836290443 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135496429 0 0) 0x7f7ee40cb370 con 0x7f7ee406e090 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee37fe700 1 -- 172.21.15.179:0/1836290443 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed00035d0 con 0x7f7ee406e090 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 -- 172.21.15.179:0/1836290443 >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 msgr2=0x7f7ee40cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 --2- 172.21.15.179:0/1836290443 >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 0x7f7ee40cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 -- 172.21.15.179:0/1836290443 >> v1:172.21.15.179:6789/0 conn(0x7f7ee406e090 legacy=0x7f7ee415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 -- 172.21.15.179:0/1836290443 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee40cb610 con 0x7f7ee4158850 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee975f700 1 --2- 172.21.15.179:0/1836290443 >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 0x7f7ee40cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:42:54.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.249+0000 7f7ee8f5e700 1 --2- 172.21.15.179:0/1836290443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee4158850 0x7f7ee40ce230 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009850 tx=0x7f7ed4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f93e0fface4272c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee37fe700 1 -- 172.21.15.179:0/1836290443 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4011830 con 0x7f7ee4158850 2024-05-08T22:42:54.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee37fe700 1 -- 172.21.15.179:0/1836290443 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4011990 con 0x7f7ee4158850 2024-05-08T22:42:54.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee37fe700 1 -- 172.21.15.179:0/1836290443 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4011d00 con 0x7f7ee4158850 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1836290443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee4158850 msgr2=0x7f7ee40ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1836290443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee4158850 0x7f7ee40ce230 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009850 tx=0x7f7ed4009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1836290443 shutdown_connections 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1836290443 >> v2:172.21.15.195:3300/0 conn(0x7f7ee415f090 0x7f7ee40cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1836290443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee4158850 0x7f7ee40ce230 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1836290443 >> 172.21.15.179:0/1836290443 conn(0x7f7ee40cea30 msgr2=0x7f7ee40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1836290443 shutdown_connections 2024-05-08T22:42:54.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1836290443 wait complete. 2024-05-08T22:42:54.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 Processor -- start 2024-05-08T22:42:54.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.253+0000 7f7ee9f60700 1 -- start start 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ee406e090 0x7f7ee40b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ee4158850 0x7f7ee40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40ac560 con 0x7f7ee415f090 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40b2c10 con 0x7f7ee406e090 2024-05-08T22:42:54.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40ac800 con 0x7f7ee4158850 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58940/0 (socket says 172.21.15.179:58940) 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 -- 172.21.15.179:0/1154139408 learned_addr learned my addr 172.21.15.179:0/1154139408 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee8f5e700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ee406e090 0x7f7ee40b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee3fff700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ee4158850 0x7f7ee40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 -- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ee4158850 msgr2=0x7f7ee40b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ee4158850 0x7f7ee40b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 -- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ee406e090 msgr2=0x7f7ee40b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ee406e090 0x7f7ee40b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 -- 172.21.15.179:0/1154139408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40094b0 con 0x7f7ee415f090 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee975f700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00e7e0 tx=0x7f7edc00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00c070 con 0x7f7ee415f090 2024-05-08T22:42:54.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc013610 con 0x7f7ee415f090 2024-05-08T22:42:54.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee40acae0 con 0x7f7ee415f090 2024-05-08T22:42:54.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc013980 con 0x7f7ee415f090 2024-05-08T22:42:54.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.257+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee40ad0a0 con 0x7f7ee415f090 2024-05-08T22:42:54.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee40ad2c0 con 0x7f7ee415f090 2024-05-08T22:42:54.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7edc013ae0 con 0x7f7ee415f090 2024-05-08T22:42:54.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee17fa700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 0x7f7ec4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7edc084070 con 0x7f7ee415f090 2024-05-08T22:42:54.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee8f5e700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 0x7f7ec4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.261+0000 7f7ee8f5e700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 0x7f7ec4068bf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40090b0 tx=0x7f7ed4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.265+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7edc0b5cf0 con 0x7f7ee415f090 2024-05-08T22:42:54.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.429+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ee404e700 con 0x7f7ee415f090 2024-05-08T22:42:54.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.429+0000 7f7ee17fa700 1 -- 172.21.15.179:0/1154139408 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7edc051210 con 0x7f7ee415f090 2024-05-08T22:42:54.433 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 msgr2=0x7f7ec4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 0x7f7ec4068bf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40090b0 tx=0x7f7ed4008030 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 msgr2=0x7f7ee4141950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00e7e0 tx=0x7f7edc00eaf0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 shutdown_connections 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7ec4066730 0x7f7ec4068bf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ee415f090 0x7f7ee4141950 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ee4158850 0x7f7ee40b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 --2- 172.21.15.179:0/1154139408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ee406e090 0x7f7ee40b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 >> 172.21.15.179:0/1154139408 conn(0x7f7ee40cea30 msgr2=0x7f7ee4142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 shutdown_connections 2024-05-08T22:42:54.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.433+0000 7f7ee9f60700 1 -- 172.21.15.179:0/1154139408 wait complete. 2024-05-08T22:42:54.448 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:54.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.553+0000 7f3d81565700 1 Processor -- start 2024-05-08T22:42:54.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.553+0000 7f3d81565700 1 -- start start 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 0x7f3d7c0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d7c15f090 0x7f3d7c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cb370 con 0x7f3d7c06e090 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c06e5c0 con 0x7f3d7c15f090 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06e8a0 con 0x7f3d7c158850 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d7c15f090 0x7f3d7c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3d7c06e090 0x7f3d7c15e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44876/0 (socket says 172.21.15.179:44876) 2024-05-08T22:42:54.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7b7fe700 1 -- 172.21.15.179:0/2927988843 learned_addr learned my addr 172.21.15.179:0/2927988843 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7bfff700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 0x7f3d7c0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 -- 172.21.15.179:0/2927988843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 msgr2=0x7f3d7c0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 0x7f3d7c0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 -- 172.21.15.179:0/2927988843 >> v1:172.21.15.179:6789/0 conn(0x7f3d7c06e090 legacy=0x7f3d7c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 -- 172.21.15.179:0/2927988843 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0cb610 con 0x7f3d7c15f090 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7bfff700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 0x7f3d7c0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d80d64700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0cbb30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009850 tx=0x7f3d74009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b4d6be4e29f9f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7affd700 1 -- 172.21.15.179:0/2927988843 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74011830 con 0x7f3d7c15f090 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7affd700 1 -- 172.21.15.179:0/2927988843 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74011990 con 0x7f3d7c15f090 2024-05-08T22:42:54.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d7affd700 1 -- 172.21.15.179:0/2927988843 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74011d20 con 0x7f3d7c15f090 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- 172.21.15.179:0/2927988843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c15f090 msgr2=0x7f3d7c0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0cbb30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009850 tx=0x7f3d74009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- 172.21.15.179:0/2927988843 shutdown_connections 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0cbb30 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 --2- 172.21.15.179:0/2927988843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c158850 0x7f3d7c0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- 172.21.15.179:0/2927988843 >> 172.21.15.179:0/2927988843 conn(0x7f3d7c0cea30 msgr2=0x7f3d7c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- 172.21.15.179:0/2927988843 shutdown_connections 2024-05-08T22:42:54.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.557+0000 7f3d81565700 1 -- 172.21.15.179:0/2927988843 wait complete. 2024-05-08T22:42:54.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 Processor -- start 2024-05-08T22:42:54.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- start start 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c06e090 0x7f3d7c140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0aa770 con 0x7f3d7c158850 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0a8420 con 0x7f3d7c15f090 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0a86c0 con 0x7f3d7c06e090 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.566 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58958/0 (socket says 172.21.15.179:58958) 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 -- 172.21.15.179:0/408353560 learned_addr learned my addr 172.21.15.179:0/408353560 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7bfff700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c06e090 0x7f3d7c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d80d64700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 -- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c06e090 msgr2=0x7f3d7c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c06e090 0x7f3d7c140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 -- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d7c15f090 msgr2=0x7f3d7c0a7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0a7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 -- 172.21.15.179:0/408353560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740094b0 con 0x7f3d7c158850 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d7b7fe700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400cbc0 tx=0x7f3d6400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400a9e0 con 0x7f3d7c158850 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0a89a0 con 0x7f3d7c158850 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400ab40 con 0x7f3d7c158850 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400ce20 con 0x7f3d7c158850 2024-05-08T22:42:54.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.561+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c0a8f60 con 0x7f3d7c158850 2024-05-08T22:42:54.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.565+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0aa8e0 con 0x7f3d7c158850 2024-05-08T22:42:54.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.565+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d6401b3b0 con 0x7f3d7c158850 2024-05-08T22:42:54.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.565+0000 7f3d78ff9700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 0x7f3d5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.569+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3d64083b70 con 0x7f3d7c158850 2024-05-08T22:42:54.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.569+0000 7f3d7bfff700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 0x7f3d5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.569+0000 7f3d7bfff700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 0x7f3d5c068bf0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.573+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3d7c0aa8e0 con 0x7f3d7c158850 2024-05-08T22:42:54.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.749+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d7c0aa8b0 con 0x7f3d7c158850 2024-05-08T22:42:54.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.749+0000 7f3d78ff9700 1 -- 172.21.15.179:0/408353560 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d64017020 con 0x7f3d7c158850 2024-05-08T22:42:54.753 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:54.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 msgr2=0x7f3d5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 0x7f3d5c068bf0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).stop 2024-05-08T22:42:54.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 msgr2=0x7f3d7c141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400cbc0 tx=0x7f3d6400cca0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 shutdown_connections 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d5c066730 0x7f3d5c068bf0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d7c15f090 0x7f3d7c0a7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d7c158850 0x7f3d7c141ee0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 --2- 172.21.15.179:0/408353560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d7c06e090 0x7f3d7c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 >> 172.21.15.179:0/408353560 conn(0x7f3d7c0cea30 msgr2=0x7f3d7c143870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 shutdown_connections 2024-05-08T22:42:54.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.753+0000 7f3d81565700 1 -- 172.21.15.179:0/408353560 wait complete. 2024-05-08T22:42:54.769 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:42:54.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.877+0000 7f4114c1b700 1 Processor -- start 2024-05-08T22:42:54.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.877+0000 7f4114c1b700 1 -- start start 2024-05-08T22:42:54.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f411015cf60 0x7f41100ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f411015d340 0x7f41100b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100cd6d0 con 0x7f41100b4da0 2024-05-08T22:42:54.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41100b5210 con 0x7f411015d340 2024-05-08T22:42:54.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4110161b30 con 0x7f411015cf60 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f411015d340 0x7f41100b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f41100b4da0 0x7f4110160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:44886/0 (socket says 172.21.15.179:44886) 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410ffff700 1 -- 172.21.15.179:0/4201585767 learned_addr learned my addr 172.21.15.179:0/4201585767 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410f7fe700 1 --2- 172.21.15.179:0/4201585767 >> v2:172.21.15.195:3300/0 conn(0x7f411015cf60 0x7f41100ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 -- 172.21.15.179:0/4201585767 >> v2:172.21.15.195:3300/0 conn(0x7f411015cf60 msgr2=0x7f41100ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 --2- 172.21.15.179:0/4201585767 >> v2:172.21.15.195:3300/0 conn(0x7f411015cf60 0x7f41100ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410e7fc700 1 -- 172.21.15.179:0/4201585767 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10688265 0 0) 0x7f41100cd6d0 con 0x7f41100b4da0 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 -- 172.21.15.179:0/4201585767 >> v1:172.21.15.179:6789/0 conn(0x7f41100b4da0 legacy=0x7f4110160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 -- 172.21.15.179:0/4201585767 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100cd970 con 0x7f411015d340 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410effd700 1 --2- 172.21.15.179:0/4201585767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f411015d340 0x7f41100b4860 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f41100d3e10 tx=0x7f40f8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342a6cf94a07e56e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410e7fc700 1 -- 172.21.15.179:0/4201585767 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011830 con 0x7f411015d340 2024-05-08T22:42:54.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410e7fc700 1 -- 172.21.15.179:0/4201585767 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8011990 con 0x7f411015d340 2024-05-08T22:42:54.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f410e7fc700 1 -- 172.21.15.179:0/4201585767 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011d20 con 0x7f411015d340 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- 172.21.15.179:0/4201585767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f411015d340 msgr2=0x7f41100b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 --2- 172.21.15.179:0/4201585767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f411015d340 0x7f41100b4860 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f41100d3e10 tx=0x7f40f8009b30 comp rx=0 tx=0).stop 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- 172.21.15.179:0/4201585767 shutdown_connections 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 --2- 172.21.15.179:0/4201585767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f411015d340 0x7f41100b4860 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 --2- 172.21.15.179:0/4201585767 >> v2:172.21.15.195:3300/0 conn(0x7f411015cf60 0x7f41100ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- 172.21.15.179:0/4201585767 >> 172.21.15.179:0/4201585767 conn(0x7f41100cf860 msgr2=0x7f41100cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- 172.21.15.179:0/4201585767 shutdown_connections 2024-05-08T22:42:54.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.881+0000 7f4114c1b700 1 -- 172.21.15.179:0/4201585767 wait complete. 2024-05-08T22:42:54.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.885+0000 7f4114c1b700 1 Processor -- start 2024-05-08T22:42:54.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.885+0000 7f4114c1b700 1 -- start start 2024-05-08T22:42:54.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411015cf60 0x7f41100cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411015d340 0x7f411015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41100c85c0 con 0x7f41100b4da0 2024-05-08T22:42:54.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41100cc090 con 0x7f411015d340 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41100c79b0 con 0x7f411015cf60 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58974/0 (socket says 172.21.15.179:58974) 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 -- 172.21.15.179:0/2012803866 learned_addr learned my addr 172.21.15.179:0/2012803866 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410ffff700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411015d340 0x7f411015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410effd700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411015cf60 0x7f41100cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 -- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411015cf60 msgr2=0x7f41100cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411015cf60 0x7f41100cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 -- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411015d340 msgr2=0x7f411015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411015d340 0x7f411015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 -- 172.21.15.179:0/2012803866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80094b0 con 0x7f41100b4da0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f410f7fe700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f410000ebe0 tx=0x7f4100009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100009740 con 0x7f41100b4da0 2024-05-08T22:42:54.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410000c8b0 con 0x7f41100b4da0 2024-05-08T22:42:54.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41100c7c90 con 0x7f41100b4da0 2024-05-08T22:42:54.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000cc40 con 0x7f41100b4da0 2024-05-08T22:42:54.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41100c8250 con 0x7f41100b4da0 2024-05-08T22:42:54.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.889+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411004e700 con 0x7f41100b4da0 2024-05-08T22:42:54.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.893+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4100013470 con 0x7f41100b4da0 2024-05-08T22:42:54.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.893+0000 7f4107fff700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 0x7f40f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:54.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.893+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f41000844a0 con 0x7f41100b4da0 2024-05-08T22:42:54.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.893+0000 7f410effd700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 0x7f40f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:54.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.893+0000 7f410effd700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 0x7f40f0068bf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:54.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:54.901+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4100051640 con 0x7f41100b4da0 2024-05-08T22:42:55.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.081+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f411006c0e0 con 0x7f41100b4da0 2024-05-08T22:42:55.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.081+0000 7f4107fff700 1 -- 172.21.15.179:0/2012803866 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4100054c60 con 0x7f41100b4da0 2024-05-08T22:42:55.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 msgr2=0x7f40f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 0x7f40f0068bf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f8008030 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 msgr2=0x7f41100c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f410000ebe0 tx=0x7f4100009e00 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 shutdown_connections 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f40f0066730 0x7f40f0068bf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411015d340 0x7f411015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411015cf60 0x7f41100cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 --2- 172.21.15.179:0/2012803866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100b4da0 0x7f41100c7320 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 >> 172.21.15.179:0/2012803866 conn(0x7f41100cf860 msgr2=0x7f411006aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 shutdown_connections 2024-05-08T22:42:55.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.085+0000 7f4114c1b700 1 -- 172.21.15.179:0/2012803866 wait complete. 2024-05-08T22:42:55.090 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:42:55.101 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:42:55.101 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:42:55.108 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:42:55.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.213+0000 7f41a57a6700 1 Processor -- start 2024-05-08T22:42:55.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.213+0000 7f41a57a6700 1 -- start start 2024-05-08T22:42:55.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f41a00d18a0 0x7f41a00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41a0160f30 0x7f41a01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00d2640 con 0x7f41a015d580 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a0161900 con 0x7f41a0160f30 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a006b3a0 con 0x7f41a00d18a0 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41a0160f30 0x7f41a01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41a0160f30 0x7f41a01613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58982/0 (socket says 172.21.15.179:58982) 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 -- 172.21.15.179:0/2627144235 learned_addr learned my addr 172.21.15.179:0/2627144235 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:55.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f419f7fe700 1 --2- 172.21.15.179:0/2627144235 >> v2:172.21.15.195:3300/0 conn(0x7f41a00d18a0 0x7f41a00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 -- 172.21.15.179:0/2627144235 >> v2:172.21.15.195:3300/0 conn(0x7f41a00d18a0 msgr2=0x7f41a00d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 --2- 172.21.15.179:0/2627144235 >> v2:172.21.15.195:3300/0 conn(0x7f41a00d18a0 0x7f41a00d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 -- 172.21.15.179:0/2627144235 >> v1:172.21.15.195:6789/0 conn(0x7f41a015d580 legacy=0x7f41a00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 -- 172.21.15.179:0/2627144235 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00d2360 con 0x7f41a0160f30 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a4fa5700 1 --2- 172.21.15.179:0/2627144235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a0160f30 0x7f41a01613c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f41a00b58e0 tx=0x7f419800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7396a5d58244dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f419effd700 1 -- 172.21.15.179:0/2627144235 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800e070 con 0x7f41a0160f30 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f419effd700 1 -- 172.21.15.179:0/2627144235 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419800b910 con 0x7f41a0160f30 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f419effd700 1 -- 172.21.15.179:0/2627144235 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800bcc0 con 0x7f41a0160f30 2024-05-08T22:42:55.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- 172.21.15.179:0/2627144235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a0160f30 msgr2=0x7f41a01613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 --2- 172.21.15.179:0/2627144235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a0160f30 0x7f41a01613c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f41a00b58e0 tx=0x7f419800b330 comp rx=0 tx=0).stop 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- 172.21.15.179:0/2627144235 shutdown_connections 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 --2- 172.21.15.179:0/2627144235 >> v2:172.21.15.195:3300/0 conn(0x7f41a00d18a0 0x7f41a00d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 --2- 172.21.15.179:0/2627144235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a0160f30 0x7f41a01613c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- 172.21.15.179:0/2627144235 >> 172.21.15.179:0/2627144235 conn(0x7f41a00d2d90 msgr2=0x7f41a00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- 172.21.15.179:0/2627144235 shutdown_connections 2024-05-08T22:42:55.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.217+0000 7f41a57a6700 1 -- 172.21.15.179:0/2627144235 wait complete. 2024-05-08T22:42:55.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 Processor -- start 2024-05-08T22:42:55.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 -- start start 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 0x7f41a00b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41a0160f30 0x7f41a00ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00b11b0 con 0x7f41a00d18a0 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00a9e60 con 0x7f41a015d580 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a57a6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00aa140 con 0x7f41a0160f30 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f419ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f419ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58990/0 (socket says 172.21.15.179:58990) 2024-05-08T22:42:55.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f419ffff700 1 -- 172.21.15.179:0/4022630138 learned_addr learned my addr 172.21.15.179:0/4022630138 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f419f7fe700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 0x7f41a00b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.221+0000 7f41a4fa5700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41a0160f30 0x7f41a00ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 -- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41a0160f30 msgr2=0x7f41a00ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41a0160f30 0x7f41a00ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 -- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 msgr2=0x7f41a00b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 0x7f41a00b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 -- 172.21.15.179:0/4022630138 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198009b30 con 0x7f41a00d18a0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419f7fe700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 0x7f41a00b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419ffff700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f419400c500 tx=0x7f419400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940052d0 con 0x7f41a00d18a0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194005430 con 0x7f41a00d18a0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a00aa420 con 0x7f41a00d18a0 2024-05-08T22:42:55.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940055a0 con 0x7f41a00d18a0 2024-05-08T22:42:55.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a00aa9e0 con 0x7f41a00d18a0 2024-05-08T22:42:55.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a004e700 con 0x7f41a00d18a0 2024-05-08T22:42:55.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.225+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41940036a0 con 0x7f41a00d18a0 2024-05-08T22:42:55.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.229+0000 7f419cff9700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 0x7f418c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.229+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f41940833d0 con 0x7f41a00d18a0 2024-05-08T22:42:55.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.229+0000 7f419f7fe700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 0x7f418c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.229+0000 7f419f7fe700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 0x7f418c068bf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f41a00cac00 tx=0x7f4190009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.233+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4194050690 con 0x7f41a00d18a0 2024-05-08T22:42:55.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.401+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41a00d0c70 con 0x7f41a00d18a0 2024-05-08T22:42:55.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.405+0000 7f419cff9700 1 -- 172.21.15.179:0/4022630138 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16296 (secure 0 0 0) 0x7f4194053cb0 con 0x7f41a00d18a0 2024-05-08T22:42:55.408 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:42:55.408 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":112,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:42:38.288356+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 msgr2=0x7f418c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 0x7f418c068bf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f41a00cac00 tx=0x7f4190009240 comp rx=0 tx=0).stop 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 msgr2=0x7f41a00aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f419400c500 tx=0x7f419400c810 comp rx=0 tx=0).stop 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 shutdown_connections 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f418c066730 0x7f418c068bf0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41a015d580 0x7f41a00b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41a0160f30 0x7f41a00ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 --2- 172.21.15.179:0/4022630138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41a00d18a0 0x7f41a00aaf00 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 >> 172.21.15.179:0/4022630138 conn(0x7f41a00d2d90 msgr2=0x7f41a00b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 shutdown_connections 2024-05-08T22:42:55.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.409+0000 7f41a57a6700 1 -- 172.21.15.179:0/4022630138 wait complete. 2024-05-08T22:42:55.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:42:35.609486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:42:36.273848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:42:55.424 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:42:55.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.537+0000 7ff9dffff700 1 Processor -- start 2024-05-08T22:42:55.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.537+0000 7ff9dffff700 1 -- start start 2024-05-08T22:42:55.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff9e00b57d0 0x7ff9e015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e00cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00b4a90 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00b4d10 con 0x7ff9e00b57d0 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e015c1e0 con 0x7ff9e015b930 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9deffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff9e00b2ac0 0x7ff9e00b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60948/0 (socket says 172.21.15.179:60948) 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9deffd700 1 -- 172.21.15.179:0/856726926 learned_addr learned my addr 172.21.15.179:0/856726926 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/856726926 >> v2:172.21.15.179:3300/0 conn(0x7ff9e00b57d0 0x7ff9e015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9df7fe700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e00cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 -- 172.21.15.179:0/856726926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 msgr2=0x7ff9e00cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e00cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 -- 172.21.15.179:0/856726926 >> v1:172.21.15.179:6789/0 conn(0x7ff9e00b2ac0 legacy=0x7ff9e00b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 -- 172.21.15.179:0/856726926 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00b4fd0 con 0x7ff9e00b57d0 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9de7fc700 1 -- 172.21.15.179:0/856726926 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855384811 0 0) 0x7ff9e00b4a90 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b57d0 0x7ff9e015b3f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009850 tx=0x7ff9c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b859f27d7baf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9de7fc700 1 -- 172.21.15.179:0/856726926 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011830 con 0x7ff9e00b57d0 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9de7fc700 1 -- 172.21.15.179:0/856726926 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8011990 con 0x7ff9e00b57d0 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9de7fc700 1 -- 172.21.15.179:0/856726926 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011d40 con 0x7ff9e00b57d0 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- 172.21.15.179:0/856726926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b57d0 msgr2=0x7ff9e015b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b57d0 0x7ff9e015b3f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009850 tx=0x7ff9c8009b60 comp rx=0 tx=0).stop 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- 172.21.15.179:0/856726926 shutdown_connections 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e00cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 --2- 172.21.15.179:0/856726926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b57d0 0x7ff9e015b3f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- 172.21.15.179:0/856726926 >> 172.21.15.179:0/856726926 conn(0x7ff9e00cc5d0 msgr2=0x7ff9e00cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- 172.21.15.179:0/856726926 shutdown_connections 2024-05-08T22:42:55.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.541+0000 7ff9dffff700 1 -- 172.21.15.179:0/856726926 wait complete. 2024-05-08T22:42:55.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 Processor -- start 2024-05-08T22:42:55.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 -- start start 2024-05-08T22:42:55.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e00b57d0 0x7ff9e00c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e013f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c8280 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c7670 con 0x7ff9e00b57d0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9dffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c7950 con 0x7ff9e015b930 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9deffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9deffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50460/0 (socket says 172.21.15.179:50460) 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.545+0000 7ff9deffd700 1 -- 172.21.15.179:0/3477442947 learned_addr learned my addr 172.21.15.179:0/3477442947 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e00b57d0 0x7ff9e00c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9df7fe700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e013f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 -- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 msgr2=0x7ff9e013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e013f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 -- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e00b57d0 msgr2=0x7ff9e00c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e00b57d0 0x7ff9e00c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 -- 172.21.15.179:0/3477442947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80094b0 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9deffd700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c4e0 tx=0x7ff9d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0004ca0 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d0004e20 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00c7c30 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007310 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e006cba0 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e004e700 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.549+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9d0007470 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.553+0000 7ff9d7fff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 0x7ff9c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.553+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9d0083f50 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.553+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 0x7ff9c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.553+0000 7ff9d67fc700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 0x7ff9c0068c20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009850 tx=0x7ff9c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.557+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9d0051180 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.729+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9e00b3220 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.729+0000 7ff9d7fff700 1 -- 172.21.15.179:0/3477442947 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9d0007720 con 0x7ff9e00b2ac0 2024-05-08T22:42:55.733 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 msgr2=0x7ff9c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 0x7ff9c0068c20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009850 tx=0x7ff9c8008030 comp rx=0 tx=0).stop 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 msgr2=0x7ff9e01f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c4e0 tx=0x7ff9d000c8a0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 shutdown_connections 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9c0066760 0x7ff9c0068c20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9e00b57d0 0x7ff9e00c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9e015b930 0x7ff9e013f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 --2- 172.21.15.179:0/3477442947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9e00b2ac0 0x7ff9e01f3660 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 >> 172.21.15.179:0/3477442947 conn(0x7ff9e00cc5d0 msgr2=0x7ff9e015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 shutdown_connections 2024-05-08T22:42:55.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.733+0000 7ff9dffff700 1 -- 172.21.15.179:0/3477442947 wait complete. 2024-05-08T22:42:55.749 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:42:55.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.865+0000 7f49cffff700 1 Processor -- start 2024-05-08T22:42:55.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.865+0000 7f49cffff700 1 -- start start 2024-05-08T22:42:55.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f49d00b4f70 0x7f49d00d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d00d1290 0x7f49d00b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d00b6420 con 0x7f49d015cfd0 2024-05-08T22:42:55.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00d2360 con 0x7f49d00b4f70 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00d1ac0 con 0x7f49d00d1290 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f49d00b4f70 0x7f49d00d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f49d00b4f70 0x7f49d00d0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50474/0 (socket says 172.21.15.179:50474) 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 -- 172.21.15.179:0/1886328627 learned_addr learned my addr 172.21.15.179:0/1886328627 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ceffd700 1 --1- 172.21.15.179:0/1886328627 >> v1:172.21.15.195:6789/0 conn(0x7f49d015cfd0 0x7f49d00b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:40064/0 (socket says 172.21.15.179:40064) 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cf7fe700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d00d1290 0x7f49d00b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 -- 172.21.15.179:0/1886328627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d00d1290 msgr2=0x7f49d00b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d00d1290 0x7f49d00b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 -- 172.21.15.179:0/1886328627 >> v1:172.21.15.195:6789/0 conn(0x7f49d015cfd0 legacy=0x7f49d00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cdffb700 1 -- 172.21.15.179:0/1886328627 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151091946 0 0) 0x7f49d00b6420 con 0x7f49d015cfd0 2024-05-08T22:42:55.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 -- 172.21.15.179:0/1886328627 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00d1da0 con 0x7f49d00b4f70 2024-05-08T22:42:55.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49ce7fc700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00b4f70 0x7f49d00d0d50 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009910 tx=0x7f49c4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd76256ab94f6176 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cdffb700 1 -- 172.21.15.179:0/1886328627 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4011830 con 0x7f49d00b4f70 2024-05-08T22:42:55.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cdffb700 1 -- 172.21.15.179:0/1886328627 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4011990 con 0x7f49d00b4f70 2024-05-08T22:42:55.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cdffb700 1 -- 172.21.15.179:0/1886328627 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4011d40 con 0x7f49d00b4f70 2024-05-08T22:42:55.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- 172.21.15.179:0/1886328627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00b4f70 msgr2=0x7f49d00d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00b4f70 0x7f49d00d0d50 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009910 tx=0x7f49c4009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:55.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- 172.21.15.179:0/1886328627 shutdown_connections 2024-05-08T22:42:55.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d00d1290 0x7f49d00b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 --2- 172.21.15.179:0/1886328627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00b4f70 0x7f49d00d0d50 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- 172.21.15.179:0/1886328627 >> 172.21.15.179:0/1886328627 conn(0x7f49d00cf860 msgr2=0x7f49d00cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:55.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- 172.21.15.179:0/1886328627 shutdown_connections 2024-05-08T22:42:55.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.869+0000 7f49cffff700 1 -- 172.21.15.179:0/1886328627 wait complete. 2024-05-08T22:42:55.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 Processor -- start 2024-05-08T22:42:55.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 -- start start 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d00b4f70 0x7f49d00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d015cfd0 0x7f49d00acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d014ab70 con 0x7f49d00d1290 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00b1760 con 0x7f49d00b4f70 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.873+0000 7f49cffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00cd9f0 con 0x7f49d015cfd0 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50486/0 (socket says 172.21.15.179:50486) 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 -- 172.21.15.179:0/3132832918 learned_addr learned my addr 172.21.15.179:0/3132832918 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:55.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ceffd700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d00b4f70 0x7f49d00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49cf7fe700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d015cfd0 0x7f49d00acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 -- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d015cfd0 msgr2=0x7f49d00acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d015cfd0 0x7f49d00acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 -- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d00b4f70 msgr2=0x7f49d00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d00b4f70 0x7f49d00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:55.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 -- 172.21.15.179:0/3132832918 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4009570 con 0x7f49d00d1290 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49ce7fc700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f49c4011db0 tx=0x7f49c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c401a900 con 0x7f49d00d1290 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c401f070 con 0x7f49d00d1290 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d014a240 con 0x7f49d00d1290 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c401acc0 con 0x7f49d00d1290 2024-05-08T22:42:55.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d014a800 con 0x7f49d00d1290 2024-05-08T22:42:55.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d014ad20 con 0x7f49d00d1290 2024-05-08T22:42:55.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.877+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49c401b430 con 0x7f49d00d1290 2024-05-08T22:42:55.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.881+0000 7f49b77fe700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 0x7f49bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:55.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.881+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f49c4021070 con 0x7f49d00d1290 2024-05-08T22:42:55.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.881+0000 7f49ceffd700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 0x7f49bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:55.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.881+0000 7f49ceffd700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 0x7f49bc068bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000f60 tx=0x7f49c0007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:55.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:55.885+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f49d014ad20 con 0x7f49d00d1290 2024-05-08T22:42:56.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49d014ad20 con 0x7f49d00d1290 2024-05-08T22:42:56.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49b77fe700 1 -- 172.21.15.179:0/3132832918 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49c4015020 con 0x7f49d00d1290 2024-05-08T22:42:56.051 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:42:56.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 msgr2=0x7f49bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 0x7f49bc068bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000f60 tx=0x7f49c0007e90 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 msgr2=0x7f49d00aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f49c4011db0 tx=0x7f49c4004770 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 shutdown_connections 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49bc066730 0x7f49bc068bf0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49d015cfd0 0x7f49d00acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49d00d1290 0x7f49d00aca90 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 --2- 172.21.15.179:0/3132832918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d00b4f70 0x7f49d00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.049+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 >> 172.21.15.179:0/3132832918 conn(0x7f49d00cf860 msgr2=0x7f49d00b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.053+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 shutdown_connections 2024-05-08T22:42:56.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.053+0000 7f49cffff700 1 -- 172.21.15.179:0/3132832918 wait complete. 2024-05-08T22:42:56.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:42:56.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 Processor -- start 2024-05-08T22:42:56.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 -- start start 2024-05-08T22:42:56.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0300b8d20 0x7fb03006abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0300d1940 con 0x7fb03006b0f0 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0300b7140 con 0x7fb03015d4b0 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb035aae700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0300d1ba0 con 0x7fb0300b8d20 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb03006b0f0 0x7fb03006b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60962/0 (socket says 172.21.15.179:60962) 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02ffff700 1 -- 172.21.15.179:0/1784891559 learned_addr learned my addr 172.21.15.179:0/1784891559 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02effd700 1 --2- 172.21.15.179:0/1784891559 >> v2:172.21.15.195:3300/0 conn(0x7fb0300b8d20 0x7fb03006abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848492184 0 0) 0x7fb0300d1940 con 0x7fb03006b0f0 2024-05-08T22:42:56.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0180035d0 con 0x7fb03006b0f0 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622661026 0 0) 0x7fb0180035d0 con 0x7fb03006b0f0 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 >> v2:172.21.15.195:3300/0 conn(0x7fb0300b8d20 msgr2=0x7fb03006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 --2- 172.21.15.179:0/1784891559 >> v2:172.21.15.195:3300/0 conn(0x7fb0300b8d20 0x7fb03006abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 >> v1:172.21.15.195:6789/0 conn(0x7fb03015d4b0 legacy=0x7fb0300d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300d1e80 con 0x7fb03006b0f0 2024-05-08T22:42:56.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fb0280019f0 con 0x7fb03006b0f0 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 >> v1:172.21.15.179:6789/0 conn(0x7fb03006b0f0 legacy=0x7fb03006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb018003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb018004870 0x7fb018004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb018005230 0x7fb01800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300d1ba0 con 0x7fb018005230 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01800c0d0 con 0x7fb018003a20 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01800c290 con 0x7fb018004870 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb018003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02ffff700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb018005230 0x7fb01800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02effd700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb018004870 0x7fb018004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 -- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb018004870 msgr2=0x7fb018004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb018004870 0x7fb018004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 -- 172.21.15.179:0/1784891559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb018005230 msgr2=0x7fb01800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:56.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb018005230 0x7fb01800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02f7fe700 1 -- 172.21.15.179:0/1784891559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180035d0 con 0x7fb018003a20 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02ffff700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb018005230 0x7fb01800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.185+0000 7fb02ffff700 1 -- 172.21.15.179:0/1784891559 reap_dead start 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb018003e40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb0200017e0 tx=0x7fb0180058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d124c594497fdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200052d0 con 0x7fb018003a20 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020005430 con 0x7fb018003a20 2024-05-08T22:42:56.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb02e7fc700 1 -- 172.21.15.179:0/1784891559 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200055a0 con 0x7fb018003a20 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 -- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 msgr2=0x7fb018003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb018003e40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb0200017e0 tx=0x7fb0180058a0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 -- 172.21.15.179:0/1784891559 shutdown_connections 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 --2- 172.21.15.179:0/1784891559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb018003e40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 -- 172.21.15.179:0/1784891559 >> 172.21.15.179:0/1784891559 conn(0x7fb0300b4af0 msgr2=0x7fb0300b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 -- 172.21.15.179:0/1784891559 shutdown_connections 2024-05-08T22:42:56.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.189+0000 7fb035aae700 1 -- 172.21.15.179:0/1784891559 wait complete. 2024-05-08T22:42:56.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 Processor -- start 2024-05-08T22:42:56.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- start start 2024-05-08T22:42:56.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb0301503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0300ca310 0x7fb0300cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300cb6a0 con 0x7fb0301f2e30 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015ee50 con 0x7fb018003a20 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015f130 con 0x7fb0300ca310 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50506/0 (socket says 172.21.15.179:50506) 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 -- 172.21.15.179:0/1095192026 learned_addr learned my addr 172.21.15.179:0/1095192026 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02effd700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0300ca310 0x7fb0300cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb0301503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 -- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0300ca310 msgr2=0x7fb0300cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0300ca310 0x7fb0300cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 -- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 msgr2=0x7fb0301503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb0301503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 -- 172.21.15.179:0/1095192026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03015f410 con 0x7fb0301f2e30 2024-05-08T22:42:56.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb02ffff700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fb03006def0 tx=0x7fb028007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280079d0 con 0x7fb0301f2e30 2024-05-08T22:42:56.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb028007b30 con 0x7fb0301f2e30 2024-05-08T22:42:56.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb030150900 con 0x7fb0301f2e30 2024-05-08T22:42:56.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280135f0 con 0x7fb0301f2e30 2024-05-08T22:42:56.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.193+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0300cbb60 con 0x7fb0301f2e30 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03015f600 con 0x7fb0301f2e30 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb028013750 con 0x7fb0301f2e30 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb017fff700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 0x7fb01c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb0280848b0 con 0x7fb0301f2e30 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 0x7fb01c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.197+0000 7fb02f7fe700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 0x7fb01c068bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb020015040 tx=0x7fb020009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.201+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb028051a50 con 0x7fb0301f2e30 2024-05-08T22:42:56.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.369+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb030073290 con 0x7fb0301f2e30 2024-05-08T22:42:56.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb017fff700 1 -- 172.21.15.179:0/1095192026 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb028055070 con 0x7fb0301f2e30 2024-05-08T22:42:56.374 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 msgr2=0x7fb01c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 0x7fb01c068bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb020015040 tx=0x7fb020009030 comp rx=0 tx=0).stop 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 msgr2=0x7fb0300c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fb03006def0 tx=0x7fb028007340 comp rx=0 tx=0).stop 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 shutdown_connections 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb01c066730 0x7fb01c068bf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0301f2e30 0x7fb0300c80d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb0300ca310 0x7fb0300cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 --2- 172.21.15.179:0/1095192026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018003a20 0x7fb0301503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 >> 172.21.15.179:0/1095192026 conn(0x7fb0300b4af0 msgr2=0x7fb0300b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:56.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 shutdown_connections 2024-05-08T22:42:56.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.373+0000 7fb035aae700 1 -- 172.21.15.179:0/1095192026 wait complete. 2024-05-08T22:42:56.389 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:42:56.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 Processor -- start 2024-05-08T22:42:56.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 -- start start 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff9cc15a210 0x7ff9cc0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15a5f0 con 0x7ff9cc0d0650 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc0d0ae0 con 0x7ff9cc15a210 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.501+0000 7ff9d19db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0acc20 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:59364/0 (socket says 172.21.15.179:59364) 2024-05-08T22:42:56.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 -- 172.21.15.179:0/234235966 learned_addr learned my addr 172.21.15.179:0/234235966 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:56.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9caffd700 1 --2- 172.21.15.179:0/234235966 >> v2:172.21.15.195:3300/0 conn(0x7ff9cc15a210 0x7ff9cc0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 -- 172.21.15.179:0/234235966 >> v2:172.21.15.195:3300/0 conn(0x7ff9cc15a210 msgr2=0x7ff9cc0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 --2- 172.21.15.179:0/234235966 >> v2:172.21.15.195:3300/0 conn(0x7ff9cc15a210 0x7ff9cc0cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 -- 172.21.15.179:0/234235966 >> v1:172.21.15.195:6789/0 conn(0x7ff9cc0d0650 legacy=0x7ff9cc0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 -- 172.21.15.179:0/234235966 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0acf00 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9c9ffb700 1 -- 172.21.15.179:0/234235966 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190265274 0 0) 0x7ff9c4001110 con 0x7ff9cc0d0650 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9ca7fc700 1 --2- 172.21.15.179:0/234235966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009910 tx=0x7ff9b4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2416fc94321d4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9c9ffb700 1 -- 172.21.15.179:0/234235966 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4011830 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9c9ffb700 1 -- 172.21.15.179:0/234235966 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4011990 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9c9ffb700 1 -- 172.21.15.179:0/234235966 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4011d40 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 -- 172.21.15.179:0/234235966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 msgr2=0x7ff9cc0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 --2- 172.21.15.179:0/234235966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009910 tx=0x7ff9b4009c20 comp rx=0 tx=0).stop 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 -- 172.21.15.179:0/234235966 shutdown_connections 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 --2- 172.21.15.179:0/234235966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0d0110 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 --2- 172.21.15.179:0/234235966 >> v2:172.21.15.195:3300/0 conn(0x7ff9cc15a210 0x7ff9cc0cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 -- 172.21.15.179:0/234235966 >> 172.21.15.179:0/234235966 conn(0x7ff9cc099070 msgr2=0x7ff9cc0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:56.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 -- 172.21.15.179:0/234235966 shutdown_connections 2024-05-08T22:42:56.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.505+0000 7ff9d19db700 1 -- 172.21.15.179:0/234235966 wait complete. 2024-05-08T22:42:56.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 Processor -- start 2024-05-08T22:42:56.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- start start 2024-05-08T22:42:56.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0d0650 0x7ff9cc14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15be00 con 0x7ff9cc0cfcd0 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc14ef20 con 0x7ff9cc15a210 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc14f200 con 0x7ff9cc0d0650 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9ca7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0d0650 0x7ff9cc14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:41734/0 (socket says 172.21.15.179:41734) 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50512/0 (socket says 172.21.15.179:50512) 2024-05-08T22:42:56.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 -- 172.21.15.179:0/54681203 learned_addr learned my addr 172.21.15.179:0/54681203 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:42:56.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 -- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0d0650 msgr2=0x7ff9cc14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0d0650 0x7ff9cc14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 -- 172.21.15.179:0/54681203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 msgr2=0x7ff9cc0c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 -- 172.21.15.179:0/54681203 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4009570 con 0x7ff9cc15a210 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9caffd700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9cb7fe700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c750 tx=0x7ff9c400ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40054e0 con 0x7ff9cc15a210 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40036a0 con 0x7ff9cc15a210 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc14f4e0 con 0x7ff9cc15a210 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4003a10 con 0x7ff9cc15a210 2024-05-08T22:42:56.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.509+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc14eb30 con 0x7ff9cc15a210 2024-05-08T22:42:56.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc15bfb0 con 0x7ff9cc15a210 2024-05-08T22:42:56.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9c4003b70 con 0x7ff9cc15a210 2024-05-08T22:42:56.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9c3fff700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 0x7ff9b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:42:56.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9c40836d0 con 0x7ff9cc15a210 2024-05-08T22:42:56.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9caffd700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 0x7ff9b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:42:56.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.513+0000 7ff9caffd700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 0x7ff9b0068bf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009fd0 tx=0x7ff9bc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:42:56.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.521+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9c40509b0 con 0x7ff9cc15a210 2024-05-08T22:42:56.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.705+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff9cc15bfb0 con 0x7ff9cc15a210 2024-05-08T22:42:56.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.705+0000 7ff9c3fff700 1 -- 172.21.15.179:0/54681203 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff9cc15bfb0 con 0x7ff9cc15a210 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 msgr2=0x7ff9b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 0x7ff9b0068bf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009fd0 tx=0x7ff9bc00b040 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 msgr2=0x7ff9cc0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c750 tx=0x7ff9c400ca60 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 shutdown_connections 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9b0066730 0x7ff9b0068bf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc15a210 0x7ff9cc0ce030 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc0d0650 0x7ff9cc14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 --2- 172.21.15.179:0/54681203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0cfcd0 0x7ff9cc0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 >> 172.21.15.179:0/54681203 conn(0x7ff9cc099070 msgr2=0x7ff9cc14fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:42:56.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 shutdown_connections 2024-05-08T22:42:56.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:42:56.709+0000 7ff9d19db700 1 -- 172.21.15.179:0/54681203 wait complete. 2024-05-08T22:42:56.714 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:42:56.725 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:42:56.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:42:56.725 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:42:56.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[50034]: fuse finished with error 0 and tester_r 0 2024-05-08T22:42:56.738 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:43:02.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:43:02.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:43:02.840 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:43:02.848 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:43:02.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 Processor -- start 2024-05-08T22:43:02.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 -- start start 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98100ce380 0x7f98100cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981006abb0 con 0x7f98100cc700 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981006ae90 con 0x7f98100cb510 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100cb980 con 0x7f98100ce380 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98100ce380 0x7f98100cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98100ce380 0x7f98100cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50524/0 (socket says 172.21.15.179:50524) 2024-05-08T22:43:02.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 -- 172.21.15.179:0/872721710 learned_addr learned my addr 172.21.15.179:0/872721710 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:02.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980e7fc700 1 -- 172.21.15.179:0/872721710 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705032744 0 0) 0x7f981006abb0 con 0x7f98100cc700 2024-05-08T22:43:02.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 -- 172.21.15.179:0/872721710 >> v1:172.21.15.195:6789/0 conn(0x7f98100cb510 legacy=0x7f981006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 -- 172.21.15.179:0/872721710 >> v1:172.21.15.179:6789/0 conn(0x7f98100cc700 legacy=0x7f981015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 -- 172.21.15.179:0/872721710 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100ce800 con 0x7f98100ce380 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980effd700 1 --2- 172.21.15.179:0/872721710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100ce380 0x7f98100cafd0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9808009e50 tx=0x7f980800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636e180b1422426a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980e7fc700 1 -- 172.21.15.179:0/872721710 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800e070 con 0x7f98100ce380 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980e7fc700 1 -- 172.21.15.179:0/872721710 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980800b9c0 con 0x7f98100ce380 2024-05-08T22:43:02.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f980e7fc700 1 -- 172.21.15.179:0/872721710 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800bd70 con 0x7f98100ce380 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 -- 172.21.15.179:0/872721710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100ce380 msgr2=0x7f98100cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.969+0000 7f9814a5d700 1 --2- 172.21.15.179:0/872721710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100ce380 0x7f98100cafd0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9808009e50 tx=0x7f980800b330 comp rx=0 tx=0).stop 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 -- 172.21.15.179:0/872721710 shutdown_connections 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 --2- 172.21.15.179:0/872721710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100ce380 0x7f98100cafd0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 -- 172.21.15.179:0/872721710 >> 172.21.15.179:0/872721710 conn(0x7f98100ce9d0 msgr2=0x7f98100cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 -- 172.21.15.179:0/872721710 shutdown_connections 2024-05-08T22:43:02.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 -- 172.21.15.179:0/872721710 wait complete. 2024-05-08T22:43:02.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 Processor -- start 2024-05-08T22:43:02.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.973+0000 7f9814a5d700 1 -- start start 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98100cc700 0x7f98101575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98100ce380 0x7f9810157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100a7570 con 0x7f98100cb510 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100a7850 con 0x7f98100ce380 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98100a7b30 con 0x7f98100cc700 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:02.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50538/0 (socket says 172.21.15.179:50538) 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 -- 172.21.15.179:0/2824149595 learned_addr learned my addr 172.21.15.179:0/2824149595 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980ffff700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98100ce380 0x7f9810157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980effd700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98100cc700 0x7f98101575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 -- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98100cc700 msgr2=0x7f98101575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98100cc700 0x7f98101575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 -- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98100ce380 msgr2=0x7f9810157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98100ce380 0x7f9810157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 -- 172.21.15.179:0/2824149595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808009b30 con 0x7f98100cb510 2024-05-08T22:43:02.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f980f7fe700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c520 tx=0x7f97f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:02.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f80054e0 con 0x7f98100cb510 2024-05-08T22:43:02.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f80036a0 con 0x7f98100cb510 2024-05-08T22:43:02.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98100a7e10 con 0x7f98100cb510 2024-05-08T22:43:02.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8003a10 con 0x7f98100cb510 2024-05-08T22:43:02.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101498b0 con 0x7f98100cb510 2024-05-08T22:43:02.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98100ade30 con 0x7f98100cb510 2024-05-08T22:43:02.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.977+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97f8003b70 con 0x7f98100cb510 2024-05-08T22:43:02.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.981+0000 7f9807fff700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 0x7f97f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:02.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.981+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97f80836b0 con 0x7f98100cb510 2024-05-08T22:43:02.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.981+0000 7f980effd700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 0x7f97f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:02.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.981+0000 7f980effd700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 0x7f97f0068bf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9808000f80 tx=0x7f9808009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:02.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:02.989+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97f8050a20 con 0x7f98100cb510 2024-05-08T22:43:03.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.197+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9810067740 con 0x7f98100cb510 2024-05-08T22:43:03.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.197+0000 7f9807fff700 1 -- 172.21.15.179:0/2824149595 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f97f8054040 con 0x7f98100cb510 2024-05-08T22:43:03.200 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:03.200 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:42:40.300797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57425":{"gid":57425,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/4059323811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4059323811},{"type":"v1","addr":"172.21.15.195:6837","nonce":4059323811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 msgr2=0x7f97f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 0x7f97f0068bf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9808000f80 tx=0x7f9808009000 comp rx=0 tx=0).stop 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 msgr2=0x7f9810149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c520 tx=0x7f97f800c8e0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 shutdown_connections 2024-05-08T22:43:03.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f97f0066730 0x7f97f0068bf0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98100ce380 0x7f9810157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98100cc700 0x7f98101575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 --2- 172.21.15.179:0/2824149595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98100cb510 0x7f9810149370 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 >> 172.21.15.179:0/2824149595 conn(0x7f98100ce9d0 msgr2=0x7f981015e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 shutdown_connections 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.201+0000 7f9814a5d700 1 -- 172.21.15.179:0/2824149595 wait complete. 2024-05-08T22:43:03.204 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 75 2024-05-08T22:43:03.215 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T22:43:03.359 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2024-05-08T22:43:03.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T22:43:03.359 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:43:03.359 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:43:03.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:43:03.366 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi179.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-08T22:43:03.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:43:03.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:43:03.367 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:43:03.416 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T22:43:03.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:43:03.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T22:43:03.419 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-08T22:43:03.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.545+0000 7f9b16524700 1 Processor -- start 2024-05-08T22:43:03.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.545+0000 7f9b16524700 1 -- start start 2024-05-08T22:43:03.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b101598a0 0x7f9b1006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1006bc80 con 0x7f9b1006b440 2024-05-08T22:43:03.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100b4f90 con 0x7f9b100ced10 2024-05-08T22:43:03.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b10159c80 con 0x7f9b101598a0 2024-05-08T22:43:03.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15522700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b101598a0 0x7f9b1006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15d23700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9b1006b440 0x7f9b100b4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60978/0 (socket says 172.21.15.179:60978) 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15d23700 1 -- 172.21.15.179:0/2760037070 learned_addr learned my addr 172.21.15.179:0/2760037070 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15522700 1 -- 172.21.15.179:0/2760037070 >> v1:172.21.15.195:6789/0 conn(0x7f9b100ced10 legacy=0x7f9b100b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15522700 1 -- 172.21.15.179:0/2760037070 >> v1:172.21.15.179:6789/0 conn(0x7f9b1006b440 legacy=0x7f9b100b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b07fff700 1 -- 172.21.15.179:0/2760037070 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319270908 0 0) 0x7f9b1006bc80 con 0x7f9b1006b440 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15522700 1 -- 172.21.15.179:0/2760037070 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1006b9a0 con 0x7f9b101598a0 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b15522700 1 --2- 172.21.15.179:0/2760037070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1006c3a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009850 tx=0x7f9b0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e6c4094308ef0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b07fff700 1 -- 172.21.15.179:0/2760037070 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011830 con 0x7f9b101598a0 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b07fff700 1 -- 172.21.15.179:0/2760037070 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c011990 con 0x7f9b101598a0 2024-05-08T22:43:03.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b07fff700 1 -- 172.21.15.179:0/2760037070 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011d20 con 0x7f9b101598a0 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- 172.21.15.179:0/2760037070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 msgr2=0x7f9b1006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 --2- 172.21.15.179:0/2760037070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1006c3a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009850 tx=0x7f9b0c009b60 comp rx=0 tx=0).stop 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- 172.21.15.179:0/2760037070 shutdown_connections 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 --2- 172.21.15.179:0/2760037070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1006c3a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- 172.21.15.179:0/2760037070 >> 172.21.15.179:0/2760037070 conn(0x7f9b10098a70 msgr2=0x7f9b100b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- 172.21.15.179:0/2760037070 shutdown_connections 2024-05-08T22:43:03.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.549+0000 7f9b16524700 1 -- 172.21.15.179:0/2760037070 wait complete. 2024-05-08T22:43:03.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 Processor -- start 2024-05-08T22:43:03.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- start start 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1006b440 0x7f9b1013efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 0x7f9b1014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10150b00 con 0x7f9b101598a0 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10143280 con 0x7f9b100ced10 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1013f5a0 con 0x7f9b1006b440 2024-05-08T22:43:03.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f588ec2d700 1 Processor -- start 2024-05-08T22:43:03.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f588ec2d700 1 -- start start 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50556/0 (socket says 172.21.15.179:50556) 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 -- 172.21.15.179:0/1031482810 learned_addr learned my addr 172.21.15.179:0/1031482810 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15522700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1006b440 0x7f9b1013efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 0x7f9b1014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 -- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1006b440 msgr2=0x7f9b1013efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1006b440 0x7f9b1013efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 -- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 msgr2=0x7f9b1014e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 0x7f9b1014e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 -- 172.21.15.179:0/1031482810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0094b0 con 0x7f9b101598a0 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 0x7f9b1014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:03.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b15d23700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800cc70 tx=0x7f9b0800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b05ffb700 1 -- 172.21.15.179:0/1031482810 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800ab50 con 0x7f9b101598a0 2024-05-08T22:43:03.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b05ffb700 1 -- 172.21.15.179:0/1031482810 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0800acd0 con 0x7f9b101598a0 2024-05-08T22:43:03.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1013f880 con 0x7f9b101598a0 2024-05-08T22:43:03.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b05ffb700 1 -- 172.21.15.179:0/1031482810 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08007310 con 0x7f9b101598a0 2024-05-08T22:43:03.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1013cb90 con 0x7f9b101598a0 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.553+0000 7f588ec2d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 0x7f58880cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58880cc170 0x7f588806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880b4d10 con 0x7f588815b240 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588815b620 con 0x7f58880cc170 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880c9b60 con 0x7f58880b09d0 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588dc2b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f588815b240 0x7f58880b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60980/0 (socket says 172.21.15.179:60980) 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588dc2b700 1 -- 172.21.15.179:0/3043979434 learned_addr learned my addr 172.21.15.179:0/3043979434 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:03.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 --2- 172.21.15.179:0/3043979434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 0x7f58880cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588e42c700 1 --2- 172.21.15.179:0/3043979434 >> v2:172.21.15.195:3300/0 conn(0x7f58880cc170 0x7f588806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588cc29700 1 -- 172.21.15.179:0/3043979434 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322760330 0 0) 0x7f58880b4d10 con 0x7f588815b240 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588cc29700 1 -- 172.21.15.179:0/3043979434 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58740035d0 con 0x7f588815b240 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 -- 172.21.15.179:0/3043979434 >> v2:172.21.15.195:3300/0 conn(0x7f58880cc170 msgr2=0x7f588806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 --2- 172.21.15.179:0/3043979434 >> v2:172.21.15.195:3300/0 conn(0x7f58880cc170 0x7f588806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 -- 172.21.15.179:0/3043979434 >> v1:172.21.15.179:6789/0 conn(0x7f588815b240 legacy=0x7f58880b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 -- 172.21.15.179:0/3043979434 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880cc5e0 con 0x7f58880b09d0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588d42a700 1 --2- 172.21.15.179:0/3043979434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 0x7f58880cbc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5878009850 tx=0x7f5878009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd125a628b91571a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588cc29700 1 -- 172.21.15.179:0/3043979434 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878011830 con 0x7f58880b09d0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588cc29700 1 -- 172.21.15.179:0/3043979434 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5878011990 con 0x7f58880b09d0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588cc29700 1 -- 172.21.15.179:0/3043979434 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878011d00 con 0x7f58880b09d0 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- 172.21.15.179:0/3043979434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 msgr2=0x7f58880cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 --2- 172.21.15.179:0/3043979434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 0x7f58880cbc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5878009850 tx=0x7f5878009b60 comp rx=0 tx=0).stop 2024-05-08T22:43:03.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 Processor -- start 2024-05-08T22:43:03.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- 172.21.15.179:0/3043979434 shutdown_connections 2024-05-08T22:43:03.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 --2- 172.21.15.179:0/3043979434 >> v2:172.21.15.195:3300/0 conn(0x7f58880cc170 0x7f588806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 --2- 172.21.15.179:0/3043979434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880b09d0 0x7f58880cbc30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- 172.21.15.179:0/3043979434 >> 172.21.15.179:0/3043979434 conn(0x7f5888098a70 msgr2=0x7f58880b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 -- start start 2024-05-08T22:43:03.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- 172.21.15.179:0/3043979434 shutdown_connections 2024-05-08T22:43:03.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f588ec2d700 1 -- 172.21.15.179:0/3043979434 wait complete. 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18b0500 con 0x19896f0 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18adc90 con 0x1944670 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18a9580 con 0x198a010 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50568/0 (socket says 172.21.15.179:50568) 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.557+0000 7f9b14d21700 1 -- 172.21.15.179:0/2020288179 learned_addr learned my addr 172.21.15.179:0/2020288179 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b15522700 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 -- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 msgr2=0x1952800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 -- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 msgr2=0x19522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 -- 172.21.15.179:0/2020288179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x179bee0 con 0x19896f0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001bf0 tx=0x7f9b000012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc254a96b82990b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9afaffd700 1 -- 172.21.15.179:0/2020288179 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00010bb0 con 0x19896f0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9afaffd700 1 -- 172.21.15.179:0/2020288179 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00010d10 con 0x19896f0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9afaffd700 1 -- 172.21.15.179:0/2020288179 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00005310 con 0x19896f0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b05ffb700 1 -- 172.21.15.179:0/1031482810 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b08007470 con 0x7f9b101598a0 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2020288179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 msgr2=0x1989ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001bf0 tx=0x7f9b000012f0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2020288179 shutdown_connections 2024-05-08T22:43:03.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b05ffb700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 0x7f9af4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2020288179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1989ad0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2020288179 >> 172.21.15.179:0/2020288179 conn(0x16d0780 msgr2=0x196b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2020288179 shutdown_connections 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2020288179 wait complete. 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b05ffb700 1 -- 172.21.15.179:0/1031482810 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9b08083c60 con 0x7f9b101598a0 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 0x7f9af4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 Processor -- start 2024-05-08T22:43:03.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 -- start start 2024-05-08T22:43:03.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 -- start start 2024-05-08T22:43:03.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 0x7f9af4068bf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009010 tx=0x7f9b0000a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f9b1aa35740 1 Processor -- start 2024-05-08T22:43:03.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58880b09d0 0x7f588814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880cc170 0x7f58880c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815b240 0x7f58880c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880c2b50 con 0x7f58880b09d0 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888159df0 con 0x7f588815b240 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588ec2d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588815a0d0 con 0x7f58880cc170 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58880b09d0 0x7f588814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58880b09d0 0x7f588814e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50582/0 (socket says 172.21.15.179:50582) 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 -- 172.21.15.179:0/1545818697 learned_addr learned my addr 172.21.15.179:0/1545818697 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588d42a700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880cc170 0x7f58880c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588e42c700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815b240 0x7f58880c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 -- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880cc170 msgr2=0x7f58880c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880cc170 0x7f58880c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 -- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815b240 msgr2=0x7f58880c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815b240 0x7f58880c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 -- 172.21.15.179:0/1545818697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780094b0 con 0x7f58880b09d0 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588d42a700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58880cc170 0x7f58880c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:03.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f588dc2b700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58880b09d0 0x7f588814e870 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f588400c4d0 tx=0x7f588400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840011d0 con 0x7f58880b09d0 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884005d90 con 0x7f58880b09d0 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.561+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840072d0 con 0x7f58880b09d0 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f588ec2d700 1 -- 172.21.15.179:0/1545818697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588815a3b0 con 0x7f58880b09d0 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1988d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f588ec2d700 1 -- 172.21.15.179:0/1545818697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588814edb0 con 0x7f58880b09d0 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x186c920 con 0x19896f0 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x186c5c0 con 0x1944670 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x183ee10 con 0x198a010 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 10 client.0 _ll_register_callbacks cb 0x18a2bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50594/0 (socket says 172.21.15.179:50594) 2024-05-08T22:43:03.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 -- 172.21.15.179:0/4003035671 learned_addr learned my addr 172.21.15.179:0/4003035671 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b15522700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b15d23700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1988d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 -- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 msgr2=0x1988d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1988d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 -- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 msgr2=0x19921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 -- 172.21.15.179:0/4003035671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b100b4f90 con 0x19896f0 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b14d21700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9b000107f0 tx=0x7f9b00001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1348a171c07044 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:03.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f588ec2d700 1 -- 172.21.15.179:0/1545818697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f588815aa60 con 0x7f58880b09d0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0001a870 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9b0001a9d0 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16dd490 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b00005590 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 10 client.9243 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x186f600 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16dd490 con 0x19896f0 2024-05-08T22:43:03.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9b0001ab90 con 0x19896f0 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 10 client.9243 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 10 client.9243 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 20 client.9243 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x19414c0 0x1991270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 4 client.9243 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1727710 con 0x19414c0 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5884007430 con 0x7f58880b09d0 2024-05-08T22:43:03.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b15522700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x19414c0 0x1991270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f587e7fc700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f586c066730 0x7f586c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9aeffff700 10 client.9243 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f9b15522700 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x19414c0 0x1991270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4bd4e985df42c4a server_cookie=ba3eef8e0ac9b547 in_seq=0 out_seq=0 2024-05-08T22:43:03.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f58840841c0 con 0x7f58880b09d0 2024-05-08T22:43:03.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f588d42a700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f586c066730 0x7f586c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+514 (secure 0 0 0) 0x7f5884084630 con 0x7f58880b09d0 2024-05-08T22:43:03.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.565+0000 7f588ec2d700 1 -- 172.21.15.179:0/1545818697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f588804e700 con 0x7f58880b09d0 2024-05-08T22:43:03.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f588d42a700 1 --2- 172.21.15.179:0/1545818697 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f586c066730 0x7f586c068bf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f58780090b0 tx=0x7f5878008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9aeffff700 1 -- 172.21.15.179:0/4003035671 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1727710 con 0x19414c0 2024-05-08T22:43:03.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9aeffff700 10 client.9243 handle_command_reply: tid=0 2024-05-08T22:43:03.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 client.9243 shutdown 2024-05-08T22:43:03.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 msgr2=0x1953be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9b000107f0 tx=0x7f9b00001120 comp rx=0 tx=0).stop 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 shutdown_connections 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1988d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x1953be0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x19414c0 0x1991270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/4003035671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x19921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9aeffff700 0 client.9243 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 >> 172.21.15.179:0/4003035671 conn(0x16d0780 msgr2=0x1986c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 shutdown_connections 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 1 -- 172.21.15.179:0/4003035671 wait complete. 2024-05-08T22:43:03.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.569+0000 7f9b1aa35740 20 client.9243 trim_cache size 0 max 16384 2024-05-08T22:43:03.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 Processor -- start 2024-05-08T22:43:03.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 -- start start 2024-05-08T22:43:03.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x198c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a82a0 con 0x19896f0 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a6d90 con 0x1944670 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18475a0 con 0x198a010 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b14d21700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50608/0 (socket says 172.21.15.179:50608) 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b14d21700 1 -- 172.21.15.179:0/1805173700 learned_addr learned my addr 172.21.15.179:0/1805173700 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b15522700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x198c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.585+0000 7f9b15d23700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 -- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 msgr2=0x1952210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 -- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 msgr2=0x198c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x198c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 -- 172.21.15.179:0/1805173700 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1855830 con 0x19896f0 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b15522700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x198c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b14d21700 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001160 tx=0x7f9b0002b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4ecd8981f8bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9afb7fe700 1 -- 172.21.15.179:0/1805173700 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00003d10 con 0x19896f0 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9afb7fe700 1 -- 172.21.15.179:0/1805173700 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b000053f0 con 0x19896f0 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9afb7fe700 1 -- 172.21.15.179:0/1805173700 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00028640 con 0x19896f0 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- 172.21.15.179:0/1805173700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 msgr2=0x198cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001160 tx=0x7f9b0002b070 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- 172.21.15.179:0/1805173700 shutdown_connections 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x198a010 0x1952210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19896f0 0x198cb30 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/1805173700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1944670 0x198c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- 172.21.15.179:0/1805173700 >> 172.21.15.179:0/1805173700 conn(0x16d0780 msgr2=0x19990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- 172.21.15.179:0/1805173700 shutdown_connections 2024-05-08T22:43:03.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- 172.21.15.179:0/1805173700 wait complete. 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- start start 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 Processor -- start 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19896f0 0x19ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x198a010 0x198e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1863010 con 0x1944670 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x190f940 con 0x198a010 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.589+0000 7f9b1aa35740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x19edee0 con 0x19896f0 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 10 client.0 _ll_register_callbacks cb 0x18a8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50610/0 (socket says 172.21.15.179:50610) 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 -- 172.21.15.179:0/2017956639 learned_addr learned my addr 172.21.15.179:0/2017956639 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15d23700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x198a010 0x198e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19896f0 0x19ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 -- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19896f0 msgr2=0x19ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19896f0 0x19ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 -- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x198a010 msgr2=0x198e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x198a010 0x198e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 -- 172.21.15.179:0/2017956639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x186f600 con 0x1944670 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b15522700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91d57b5e6613ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c02a6b0 con 0x1944670 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9b0c01f040 con 0x1944670 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b00010d10 con 0x1944670 2024-05-08T22:43:03.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b0c02a810 con 0x1944670 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 10 client.9249 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x171bc10 con 0x1944670 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00010d10 con 0x1944670 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9b0c033440 con 0x1944670 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 10 client.9249 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 10 client.9249 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 20 client.9249 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x198ee70 0x19a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 4 client.9249 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x174e5f0 con 0x198ee70 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x198ee70 0x19a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9aeffff700 10 client.9249 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.593+0000 7f9b14d21700 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x198ee70 0x19a9580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8f464b3c650f6d server_cookie=76679fe7c0c9b2c3 in_seq=0 out_seq=0 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9aeffff700 1 -- 172.21.15.179:0/2017956639 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x174e5f0 con 0x198ee70 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9aeffff700 10 client.9249 handle_command_reply: tid=0 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 client.9249 shutdown 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 msgr2=0x19929a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 shutdown_connections 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x198ee70 0x19a9580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x198a010 0x198e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9aeffff700 0 client.9249 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19896f0 0x19ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 --2- 172.21.15.179:0/2017956639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1944670 0x19929a0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 >> 172.21.15.179:0/2017956639 conn(0x16d0780 msgr2=0x1991270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 shutdown_connections 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 1 -- 172.21.15.179:0/2017956639 wait complete. 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b1aa35740 20 client.9249 trim_cache size 0 max 16384 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "3e87e015-9812-41cf-b356-6a38917db5fc", 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:43:03.602 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 msgr2=0x7f9af4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 0x7f9af4068bf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009010 tx=0x7f9b0000a000 comp rx=0 tx=0).stop 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 msgr2=0x7f9b1013c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800cc70 tx=0x7f9b0800cf80 comp rx=0 tx=0).stop 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 shutdown_connections 2024-05-08T22:43:03.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9af4066730 0x7f9af4068bf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b100ced10 0x7f9b1014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b101598a0 0x7f9b1013c650 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 --2- 172.21.15.179:0/1031482810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1006b440 0x7f9b1013efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 >> 172.21.15.179:0/1031482810 conn(0x7f9b10098a70 msgr2=0x7f9b100ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 shutdown_connections 2024-05-08T22:43:03.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.597+0000 7f9b16524700 1 -- 172.21.15.179:0/1031482810 wait complete. 2024-05-08T22:43:03.615 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T22:43:03.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 Processor -- start 2024-05-08T22:43:03.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 -- start start 2024-05-08T22:43:03.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcae406c2d0 0x7fcae40cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae4165b30 con 0x7fcae40b30a0 2024-05-08T22:43:03.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae4165fc0 con 0x7fcae41648b0 2024-05-08T22:43:03.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae3fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae40b3640 con 0x7fcae406c2d0 2024-05-08T22:43:03.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae37fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcae40b30a0 0x7fcae40cc420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60988/0 (socket says 172.21.15.179:60988) 2024-05-08T22:43:03.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae37fe700 1 -- 172.21.15.179:0/1714010018 learned_addr learned my addr 172.21.15.179:0/1714010018 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae27fc700 1 --2- 172.21.15.179:0/1714010018 >> v2:172.21.15.195:3300/0 conn(0x7fcae406c2d0 0x7fcae40cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029063829 0 0) 0x7fcae4165b30 con 0x7fcae40b30a0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad00035d0 con 0x7fcae40b30a0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579618337 0 0) 0x7fcae4165fc0 con 0x7fcae41648b0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae4165b30 con 0x7fcae41648b0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294518288 0 0) 0x7fcad00035d0 con 0x7fcae40b30a0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 >> v2:172.21.15.195:3300/0 conn(0x7fcae406c2d0 msgr2=0x7fcae40cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 --2- 172.21.15.179:0/1714010018 >> v2:172.21.15.195:3300/0 conn(0x7fcae406c2d0 0x7fcae40cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 >> v1:172.21.15.195:6789/0 conn(0x7fcae41648b0 legacy=0x7fcae406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4165200 con 0x7fcae40b30a0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fcadc0019f0 con 0x7fcae40b30a0 2024-05-08T22:43:03.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 >> v1:172.21.15.179:6789/0 conn(0x7fcae40b30a0 legacy=0x7fcae40cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 0x7fcad0004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad0004870 0x7fcad0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0005280 0x7fcad000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae40b3640 con 0x7fcad0005280 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad000c1b0 con 0x7fcad0003c00 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad000c370 con 0x7fcad0004870 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae37fe700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0005280 0x7fcad000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae27fc700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad0004870 0x7fcad0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.729+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 0x7fcad0004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 -- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad0004870 msgr2=0x7fcad0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad0004870 0x7fcad0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 -- 172.21.15.179:0/1714010018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0005280 msgr2=0x7fcad000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0005280 0x7fcad000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 -- 172.21.15.179:0/1714010018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00035d0 con 0x7fcad0003c00 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae27fc700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad0004870 0x7fcad0004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae37fe700 1 -- 172.21.15.179:0/1714010018 reap_dead start 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 0x7fcad0004000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcacc001960 tx=0x7fcad00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465d25938128a5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc001450 con 0x7fcad0003c00 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc006aa0 con 0x7fcad0003c00 2024-05-08T22:43:03.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae1ffb700 1 -- 172.21.15.179:0/1714010018 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc006e50 con 0x7fcad0003c00 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 -- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 msgr2=0x7fcad0004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 0x7fcad0004000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcacc001960 tx=0x7fcad00058a0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 -- 172.21.15.179:0/1714010018 shutdown_connections 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 --2- 172.21.15.179:0/1714010018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad0003c00 0x7fcad0004000 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 -- 172.21.15.179:0/1714010018 >> 172.21.15.179:0/1714010018 conn(0x7fcae4070d80 msgr2=0x7fcae4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 -- 172.21.15.179:0/1714010018 shutdown_connections 2024-05-08T22:43:03.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.733+0000 7fcae3fff700 1 -- 172.21.15.179:0/1714010018 wait complete. 2024-05-08T22:43:03.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 Processor -- start 2024-05-08T22:43:03.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 -- start start 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcae415c650 0x7fcae41fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcae4145000 0x7fcae4145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae41459c0 con 0x7fcad0003c00 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4157c00 con 0x7fcae415c650 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4157ee0 con 0x7fcae4145000 2024-05-08T22:43:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50630/0 (socket says 172.21.15.179:50630) 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 -- 172.21.15.179:0/827197943 learned_addr learned my addr 172.21.15.179:0/827197943 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae27fc700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcae415c650 0x7fcae41fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae37fe700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcae4145000 0x7fcae4145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 -- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcae4145000 msgr2=0x7fcae4145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcae4145000 0x7fcae4145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 -- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcae415c650 msgr2=0x7fcae41fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcae415c650 0x7fcae41fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.737+0000 7fcae2ffd700 1 -- 172.21.15.179:0/827197943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae41581c0 con 0x7fcad0003c00 2024-05-08T22:43:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c340 tx=0x7fcacc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 -- 172.21.15.179:0/827197943 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc001450 con 0x7fcad0003c00 2024-05-08T22:43:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 -- 172.21.15.179:0/827197943 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0054d0 con 0x7fcad0003c00 2024-05-08T22:43:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae4161fa0 con 0x7fcad0003c00 2024-05-08T22:43:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 -- 172.21.15.179:0/827197943 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007270 con 0x7fcad0003c00 2024-05-08T22:43:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae41578a0 con 0x7fcad0003c00 2024-05-08T22:43:03.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 -- 172.21.15.179:0/827197943 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcacc0073d0 con 0x7fcad0003c00 2024-05-08T22:43:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 0x7fcac4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.741+0000 7fcadbfff700 1 -- 172.21.15.179:0/827197943 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcacc084eb0 con 0x7fcad0003c00 2024-05-08T22:43:03.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae27fc700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 0x7fcac4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae27fc700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 0x7fcac4068bf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcae415c150 tx=0x7fcad4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 Processor -- start 2024-05-08T22:43:03.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- start start 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x20e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21145d0 0x20e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x207a010 con 0x20e7840 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2007f40 con 0x21116f0 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x205ae30 con 0x21145d0 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50636/0 (socket says 172.21.15.179:50636) 2024-05-08T22:43:03.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3427024995 learned_addr learned my addr 172.21.15.179:0/3427024995 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae27fc700 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x20e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae37fe700 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21145d0 0x20e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21145d0 msgr2=0x20e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21145d0 0x20e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 msgr2=0x20e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x20e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3427024995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ec6a40 con 0x20e7840 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcacc015040 tx=0x7fcacc0559f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc0cacfd5a9a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcad97fa700 1 -- 172.21.15.179:0/3427024995 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc056530 con 0x20e7840 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcad97fa700 1 -- 172.21.15.179:0/3427024995 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc055e30 con 0x20e7840 2024-05-08T22:43:03.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcad97fa700 1 -- 172.21.15.179:0/3427024995 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc059c60 con 0x20e7840 2024-05-08T22:43:03.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3427024995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 msgr2=0x20ecd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcacc015040 tx=0x7fcacc0559f0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3427024995 shutdown_connections 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x20e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21145d0 0x20e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3427024995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x20ecd90 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3427024995 >> 172.21.15.179:0/3427024995 conn(0x1e7b5b0 msgr2=0x2113ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3427024995 shutdown_connections 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.745+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3427024995 wait complete. 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 -- start start 2024-05-08T22:43:03.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 Processor -- start 2024-05-08T22:43:03.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20e7840 0x2131e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2116010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2056150 con 0x21145d0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2053eb0 con 0x20e7840 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20118e0 con 0x21116f0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcaecc9d740 10 client.0 _ll_register_callbacks cb 0x1ff0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20e7840 0x2131e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50652/0 (socket says 172.21.15.179:50652) 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 -- 172.21.15.179:0/2160790523 learned_addr learned my addr 172.21.15.179:0/2160790523 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae27fc700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 -- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 msgr2=0x2116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2116010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 -- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20e7840 msgr2=0x2131e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20e7840 0x2131e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 -- 172.21.15.179:0/2160790523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae41578a0 con 0x21145d0 2024-05-08T22:43:03.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcae37fe700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcae406d070 tx=0x7fcadc007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f265c4ca2782a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.749+0000 7fcac27fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc001990 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcadc0159a0 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e882c0 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc015b60 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 10 client.9261 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x201a290 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e882c0 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcadc012030 con 0x21145d0 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 10 client.9261 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 10 client.9261 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 20 client.9261 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2117c00 0x2118010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 4 client.9261 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:03.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ed2540 con 0x2117c00 2024-05-08T22:43:03.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2117c00 0x2118010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2117c00 0x2118010 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b9298ac1c80d5b6 server_cookie=57613bbf03af6270 in_seq=0 out_seq=0 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 10 client.9261 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 1 -- 172.21.15.179:0/2160790523 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1ed2540 con 0x2117c00 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 10 client.9261 handle_command_reply: tid=0 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 client.9261 shutdown 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 msgr2=0x20ea980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcae406d070 tx=0x7fcadc007a60 comp rx=0 tx=0).stop 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 shutdown_connections 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x20ea980 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2117c00 0x2118010 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/2160790523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x20e7840 0x2131e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcac27fc700 0 client.9261 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 >> 172.21.15.179:0/2160790523 conn(0x1e7b5b0 msgr2=0x2132900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 shutdown_connections 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.753+0000 7fcaecc9d740 1 -- 172.21.15.179:0/2160790523 wait complete. 2024-05-08T22:43:03.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.757+0000 7fcaecc9d740 20 client.9261 trim_cache size 0 max 16384 2024-05-08T22:43:03.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 Processor -- start 2024-05-08T22:43:03.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- start start 2024-05-08T22:43:03.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 0x20eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1effb80 con 0x20e7840 2024-05-08T22:43:03.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ed1a0 con 0x21145d0 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20536c0 con 0x21116f0 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50664/0 (socket says 172.21.15.179:50664) 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3379275407 learned_addr learned my addr 172.21.15.179:0/3379275407 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae27fc700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae37fe700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 0x20eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 msgr2=0x2142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 msgr2=0x20eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 0x20eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 -- 172.21.15.179:0/3379275407 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20004c0 con 0x20e7840 2024-05-08T22:43:03.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae27fc700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae37fe700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 0x20eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcacc007ab0 tx=0x7fcacc027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22ffdf97111768 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcad9ffb700 1 -- 172.21.15.179:0/3379275407 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc059c60 con 0x20e7840 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcad9ffb700 1 -- 172.21.15.179:0/3379275407 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0015b0 con 0x20e7840 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcad9ffb700 1 -- 172.21.15.179:0/3379275407 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007680 con 0x20e7840 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3379275407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 msgr2=0x21426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcacc007ab0 tx=0x7fcacc027000 comp rx=0 tx=0).stop 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3379275407 shutdown_connections 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21145d0 0x20eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x21116f0 0x2142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/3379275407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x20e7840 0x21426f0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3379275407 >> 172.21.15.179:0/3379275407 conn(0x1e7b5b0 msgr2=0x2115d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3379275407 shutdown_connections 2024-05-08T22:43:03.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- 172.21.15.179:0/3379275407 wait complete. 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 -- start start 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.777+0000 7fcaecc9d740 1 Processor -- start 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20e7840 0x214acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x2149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2152220 con 0x21145d0 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x214b230 con 0x21116f0 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x21943a0 con 0x20e7840 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 10 client.0 _ll_register_callbacks cb 0x200dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae2ffd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20e7840 0x214acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50680/0 (socket says 172.21.15.179:50680) 2024-05-08T22:43:03.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 -- 172.21.15.179:0/913047941 learned_addr learned my addr 172.21.15.179:0/913047941 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae27fc700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x2149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 -- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20e7840 msgr2=0x214acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20e7840 0x214acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 -- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 msgr2=0x2149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x2149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 -- 172.21.15.179:0/913047941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ec6a40 con 0x21145d0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae37fe700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006010 tx=0x7fcadc0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b72cf5faf7942e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc020b70 con 0x21145d0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcadc020cd0 con 0x21145d0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcacc055e30 con 0x21145d0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc015e00 con 0x21145d0 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 10 client.9267 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:03.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x1ff4790 con 0x21145d0 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc055e30 con 0x21145d0 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcadc007d20 con 0x21145d0 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 10 client.9267 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 10 client.9267 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 20 client.9267 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2147f90 0x21483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 4 client.9267 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1ef72c0 con 0x2147f90 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2147f90 0x21483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcae2ffd700 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2147f90 0x21483a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be846b04550d0d6a server_cookie=7e8ea2689258776a in_seq=0 out_seq=0 2024-05-08T22:43:03.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.781+0000 7fcac27fc700 10 client.9267 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcac27fc700 1 -- 172.21.15.179:0/913047941 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x1ef72c0 con 0x2147f90 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcac27fc700 10 client.9267 handle_command_reply: tid=0 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 client.9267 shutdown 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 msgr2=0x2147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006010 tx=0x7fcadc0020c0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 shutdown_connections 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2147f90 0x21483a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x21116f0 0x2149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcac27fc700 0 client.9267 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:03.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x21145d0 0x2147a50 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 --2- 172.21.15.179:0/913047941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x20e7840 0x214acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 >> 172.21.15.179:0/913047941 conn(0x1e7b5b0 msgr2=0x2115480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 shutdown_connections 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 1 -- 172.21.15.179:0/913047941 wait complete. 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcaecc9d740 20 client.9267 trim_cache size 0 max 16384 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stdout: "3e87e015-9812-41cf-b356-6a38917db5fc": { 2024-05-08T22:43:03.791 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/", 2024-05-08T22:43:03.792 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "3e87e015-9812-41cf-b356-6a38917db5fc", 2024-05-08T22:43:03.792 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:43:03.792 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:43:03.792 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:43:03.792 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 msgr2=0x7fcac4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 0x7fcac4068bf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcae415c150 tx=0x7fcad4009240 comp rx=0 tx=0).stop 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 msgr2=0x7fcae4157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c340 tx=0x7fcacc0012d0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 shutdown_connections 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcac4066730 0x7fcac4068bf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcae415c650 0x7fcae41fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcae4145000 0x7fcae4145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 --2- 172.21.15.179:0/827197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcad0003c00 0x7fcae4157360 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:03.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 >> 172.21.15.179:0/827197943 conn(0x7fcae4070d80 msgr2=0x7fcae40cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:03.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 shutdown_connections 2024-05-08T22:43:03.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:03.785+0000 7fcae3fff700 1 -- 172.21.15.179:0/827197943 wait complete. 2024-05-08T22:43:03.804 INFO:tasks.cephfs.filesystem:scrub status for tag:3e87e015-9812-41cf-b356-6a38917db5fc - {'path': '/', 'tag': '3e87e015-9812-41cf-b356-6a38917db5fc', 'options': 'recursive,repair'} 2024-05-08T22:43:04.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:04.709+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 7 ==== log(4 entries from seq 1 at 2024-05-08T22:43:03.596833+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7f5884055160 con 0x7f58880b09d0 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: cluster: 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: id: 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: health: HEALTH_OK 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: services: 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: mgr: x(active, since 10m), standbys: y 2024-05-08T22:43:04.715 INFO:teuthology.orchestra.run.smithi179.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: data: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: volumes: 1/1 healthy 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: pools: 3 pools, 97 pgs 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: objects: 30 objects, 19 MiB 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: usage: 299 MiB used, 720 GiB / 720 GiB avail 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: pgs: 97 active+clean 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: io: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:03.596833+0000 mds.d [INF] scrub queued for path: / 2024-05-08T22:43:04.716 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:03.596841+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:43:04.717 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:03.596851+0000 mds.d [INF] scrub summary: active paths [/] 2024-05-08T22:43:04.717 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:03.599413+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:43:06.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:06.717+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 8 ==== log(4 entries from seq 153 at 2024-05-08T22:43:06.198852+0000) v1 ==== 1116+0+0 (secure 0 0 0) 0x7f5884058ee0 con 0x7f58880b09d0 2024-05-08T22:43:07.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:07.733+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 9 ==== log(2 entries from seq 1085 at 2024-05-08T22:43:06.730413+0000) v1 ==== 560+0+0 (secure 0 0 0) 0x7f588405d4a0 con 0x7f58880b09d0 2024-05-08T22:43:09.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:09.741+0000 7f587e7fc700 1 -- 172.21.15.179:0/1545818697 <== mon.0 v2:172.21.15.179:3300/0 10 ==== log(1 entries from seq 5 at 2024-05-08T22:43:07.850259+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f58840826c0 con 0x7f58880b09d0 2024-05-08T22:43:09.745 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:07.850259+0000 mds.d [INF] scrub summary: idle 2024-05-08T22:43:22.809 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:43:22.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:43:33.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T22:43:33.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 Processor -- start 2024-05-08T22:43:33.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 -- start start 2024-05-08T22:43:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 0x7f05440b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05440b60b0 con 0x7f054406abb0 2024-05-08T22:43:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054415e750 con 0x7f054415d360 2024-05-08T22:43:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f054b238700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b8fc0 con 0x7f05440b51e0 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f05497d5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f0548fd4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 0x7f05440b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f0543fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f054406abb0 0x7f05440b4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:40458/0 (socket says 172.21.15.179:40458) 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f05497d5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:43168/0 (socket says 172.21.15.179:43168) 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f0548fd4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 0x7f05440b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53570/0 (socket says 172.21.15.179:53570) 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.928+0000 7f0543fff700 1 -- 172.21.15.179:0/2464318660 learned_addr learned my addr 172.21.15.179:0/2464318660 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05497d5700 1 -- 172.21.15.179:0/2464318660 >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 msgr2=0x7f05440b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05497d5700 1 --2- 172.21.15.179:0/2464318660 >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 0x7f05440b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05497d5700 1 -- 172.21.15.179:0/2464318660 >> v1:172.21.15.195:6789/0 conn(0x7f054406abb0 legacy=0x7f05440b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05497d5700 1 -- 172.21.15.179:0/2464318660 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054406b130 con 0x7f05440b51e0 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05497d5700 1 --2- 172.21.15.179:0/2464318660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d90 tx=0x7f053c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88e33501d609637 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05437fe700 1 -- 172.21.15.179:0/2464318660 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00e070 con 0x7f05440b51e0 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05437fe700 1 -- 172.21.15.179:0/2464318660 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00bb20 con 0x7f05440b51e0 2024-05-08T22:43:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f05437fe700 1 -- 172.21.15.179:0/2464318660 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c012430 con 0x7f05440b51e0 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 -- 172.21.15.179:0/2464318660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 msgr2=0x7f05440d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 --2- 172.21.15.179:0/2464318660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d90 tx=0x7f053c00b330 comp rx=0 tx=0).stop 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 -- 172.21.15.179:0/2464318660 shutdown_connections 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 --2- 172.21.15.179:0/2464318660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f05440d15e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 --2- 172.21.15.179:0/2464318660 >> v2:172.21.15.195:3300/0 conn(0x7f054415d360 0x7f05440b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 -- 172.21.15.179:0/2464318660 >> 172.21.15.179:0/2464318660 conn(0x7f05440d3050 msgr2=0x7f05440d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 -- 172.21.15.179:0/2464318660 shutdown_connections 2024-05-08T22:43:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.932+0000 7f054b238700 1 -- 172.21.15.179:0/2464318660 wait complete. 2024-05-08T22:43:33.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 Processor -- start 2024-05-08T22:43:33.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- start start 2024-05-08T22:43:33.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f054415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f054415d360 0x7f054415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054414dd60 con 0x7f054406abb0 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440c8a00 con 0x7f054415d360 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05441f3bc0 con 0x7f05440b51e0 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47906/0 (socket says 172.21.15.179:47906) 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 -- 172.21.15.179:0/566476873 learned_addr learned my addr 172.21.15.179:0/566476873 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f05497d5700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f054415d360 0x7f054415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0543fff700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f054415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 -- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 msgr2=0x7f054415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f054415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 -- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f054415d360 msgr2=0x7f054415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f054415d360 0x7f054415fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 -- 172.21.15.179:0/566476873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c009a70 con 0x7f054406abb0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0543fff700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f054415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f0548fd4700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f053400e840 tx=0x7f053400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f05417fa700 1 -- 172.21.15.179:0/566476873 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340095a0 con 0x7f054406abb0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05441f3ea0 con 0x7f054406abb0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05441f4460 con 0x7f054406abb0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f05417fa700 1 -- 172.21.15.179:0/566476873 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534005a40 con 0x7f054406abb0 2024-05-08T22:43:33.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.936+0000 7f05417fa700 1 -- 172.21.15.179:0/566476873 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534005db0 con 0x7f054406abb0 2024-05-08T22:43:33.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f05417fa700 1 -- 172.21.15.179:0/566476873 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0534013680 con 0x7f054406abb0 2024-05-08T22:43:33.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f05417fa700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 0x7f0524068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f05417fa700 1 -- 172.21.15.179:0/566476873 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0534084110 con 0x7f054406abb0 2024-05-08T22:43:33.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f0543fff700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 0x7f0524068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 Processor -- start 2024-05-08T22:43:33.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 -- start start 2024-05-08T22:43:33.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f0543fff700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 0x7f0524068bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f052c007fd0 tx=0x7f052c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2ac1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 0x2ac1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a08010 con 0x2a75840 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2995f40 con 0x2aa29b0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x29e8e30 con 0x2aa25d0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47922/0 (socket says 172.21.15.179:47922) 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.940+0000 7f0548fd4700 1 -- 172.21.15.179:0/3757254027 learned_addr learned my addr 172.21.15.179:0/3757254027 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f05497d5700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 0x2ac1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0543fff700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2ac1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 -- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 msgr2=0x2ac1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2ac1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 -- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 msgr2=0x2ac1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 0x2ac1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 -- 172.21.15.179:0/3757254027 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2854a40 con 0x2a75840 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f05497d5700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 0x2ac1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f05340824f0 tx=0x7f0534054a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88ea3bfb70a1489 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f053a7fc700 1 -- 172.21.15.179:0/3757254027 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053405d440 con 0x2a75840 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f053a7fc700 1 -- 172.21.15.179:0/3757254027 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534068e90 con 0x2a75840 2024-05-08T22:43:33.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f053a7fc700 1 -- 172.21.15.179:0/3757254027 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340533f0 con 0x2a75840 2024-05-08T22:43:33.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- 172.21.15.179:0/3757254027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 msgr2=0x2a75c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f05340824f0 tx=0x7f0534054a80 comp rx=0 tx=0).stop 2024-05-08T22:43:33.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- 172.21.15.179:0/3757254027 shutdown_connections 2024-05-08T22:43:33.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa29b0 0x2ac1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2ac1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3757254027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a75c20 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- 172.21.15.179:0/3757254027 >> 172.21.15.179:0/3757254027 conn(0x28095b0 msgr2=0x2aa1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- 172.21.15.179:0/3757254027 shutdown_connections 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- 172.21.15.179:0/3757254027 wait complete. 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 -- start start 2024-05-08T22:43:33.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.944+0000 7f054e4e7740 1 Processor -- start 2024-05-08T22:43:33.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 0x2aa6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa29b0 0x2ac7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x29e4150 con 0x2a75840 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x29e1eb0 con 0x2aa25d0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x299f8e0 con 0x2aa29b0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 10 client.0 _ll_register_callbacks cb 0x297e670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47938/0 (socket says 172.21.15.179:47938) 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 -- 172.21.15.179:0/753087772 learned_addr learned my addr 172.21.15.179:0/753087772 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05497d5700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa29b0 0x2ac7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0543fff700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 0x2aa6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 -- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa29b0 msgr2=0x2ac7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa29b0 0x2ac7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 -- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 msgr2=0x2aa6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 0x2aa6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 -- 172.21.15.179:0/753087772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05441f4460 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0543fff700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 0x2aa6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0548fd4700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f05340825a0 tx=0x7f053400c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6161d16937dd31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534051390 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f053405d440 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28162c0 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0534054540 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 10 client.9276 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x29a8290 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28162c0 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f053405cfb0 con 0x2a75840 2024-05-08T22:43:33.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 10 client.9276 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 10 client.9276 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 20 client.9276 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2a742a0 0x2aa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 4 client.9276 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2860540 con 0x2a742a0 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.948+0000 7f0543fff700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2a742a0 0x2aa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f05237fe700 10 client.9276 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:33.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f0543fff700 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2a742a0 0x2aa7040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5aed0d7996a000e server_cookie=2680fbc2221db6df in_seq=0 out_seq=0 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f05237fe700 1 -- 172.21.15.179:0/753087772 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x2860540 con 0x2a742a0 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f05237fe700 10 client.9276 handle_command_reply: tid=0 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 client.9276 shutdown 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 msgr2=0x2a9f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f05340825a0 tx=0x7f053400c250 comp rx=0 tx=0).stop 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 shutdown_connections 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2a742a0 0x2aa7040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa25d0 0x2aa6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa29b0 0x2ac7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 --2- 172.21.15.179:0/753087772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2a9f4f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f05237fe700 0 client.9276 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:33.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 >> 172.21.15.179:0/753087772 conn(0x28095b0 msgr2=0x2aa1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 shutdown_connections 2024-05-08T22:43:33.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 1 -- 172.21.15.179:0/753087772 wait complete. 2024-05-08T22:43:33.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.952+0000 7f054e4e7740 20 client.9276 trim_cache size 0 max 16384 2024-05-08T22:43:33.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 Processor -- start 2024-05-08T22:43:33.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 -- start start 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2a78610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2b18be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2b19120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x288db80 con 0x2a75840 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a09610 con 0x28095b0 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x29ea230 con 0x2aa25d0 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f0543fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2b18be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2a78610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f05497d5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2b19120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f0548fd4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2a78610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53614/0 (socket says 172.21.15.179:53614) 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.972+0000 7f0548fd4700 1 -- 172.21.15.179:0/101246344 learned_addr learned my addr 172.21.15.179:0/101246344 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 -- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 msgr2=0x2b19120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2b19120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 -- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 msgr2=0x2a78610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2a78610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 -- 172.21.15.179:0/101246344 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2a6fa10 con 0x2a75840 2024-05-08T22:43:33.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2b18be0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f05440c9970 tx=0x7f052c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce2c50524608c70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f053affd700 1 -- 172.21.15.179:0/101246344 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0243c0 con 0x2a75840 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f053affd700 1 -- 172.21.15.179:0/101246344 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c024520 con 0x2a75840 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f053affd700 1 -- 172.21.15.179:0/101246344 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c024690 con 0x2a75840 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- 172.21.15.179:0/101246344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 msgr2=0x2b18be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2b18be0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f05440c9970 tx=0x7f052c012000 comp rx=0 tx=0).stop 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- 172.21.15.179:0/101246344 shutdown_connections 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa25d0 0x2b19120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a75840 0x2b18be0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- 172.21.15.179:0/101246344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2a78610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- 172.21.15.179:0/101246344 >> 172.21.15.179:0/101246344 conn(0x2aa6160 msgr2=0x2a9e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- 172.21.15.179:0/101246344 shutdown_connections 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- 172.21.15.179:0/101246344 wait complete. 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- start start 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 Processor -- start 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2ad5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a75840 0x2b17850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x29b2c00 con 0x2aa25d0 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x296ba50 con 0x28095b0 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2980620 con 0x2a75840 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f054e4e7740 10 client.0 _ll_register_callbacks cb 0x2a7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f05497d5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f05497d5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47942/0 (socket says 172.21.15.179:47942) 2024-05-08T22:43:33.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f05497d5700 1 -- 172.21.15.179:0/3438470418 learned_addr learned my addr 172.21.15.179:0/3438470418 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:33.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2ad5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.976+0000 7f0543fff700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a75840 0x2b17850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 -- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a75840 msgr2=0x2b17850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a75840 0x2b17850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 -- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 msgr2=0x2ad5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2ad5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 -- 172.21.15.179:0/3438470418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c00f040 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05497d5700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f053c013000 tx=0x7f053c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b908f90c940c2e1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c021040 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f053c012e30 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0534068e90 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c004040 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 10 client.9282 fetch_fsmap learned FSMap version 75 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x298e4c0 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534068e90 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 75) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f053c0041a0 con 0x2aa25d0 2024-05-08T22:43:33.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 10 client.9282 fetch_fsmap finished waiting for FSMap version 75 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 10 client.9282 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 20 client.9282 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2aa29b0 0x2b188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 4 client.9282 mds_command: new command op to 57425 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 --> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x28852c0 con 0x2aa29b0 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2aa29b0 0x2b188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f0548fd4700 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2aa29b0 0x2b188f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e436b465f78d67 server_cookie=81f9697fb3170e38 in_seq=0 out_seq=0 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 10 client.9282 ms_handle_connect on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 1 -- 172.21.15.179:0/3438470418 <== mds.0 v2:172.21.15.195:6836/4059323811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x28852c0 con 0x2aa29b0 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 10 client.9282 handle_command_reply: tid=0 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 client.9282 shutdown 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 msgr2=0x2b17d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f053c013000 tx=0x7f053c004b60 comp rx=0 tx=0).stop 2024-05-08T22:43:33.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 shutdown_connections 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa25d0 0x2b17d90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a75840 0x2b17850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:6836/4059323811,v1:172.21.15.195:6837/4059323811] conn(0x2aa29b0 0x2b188f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 --2- 172.21.15.179:0/3438470418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28095b0 0x2ad5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f05237fe700 0 client.9282 ms_handle_reset on v2:172.21.15.195:6836/4059323811 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 >> 172.21.15.179:0/3438470418 conn(0x2aa6160 msgr2=0x2ad0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 shutdown_connections 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 1 -- 172.21.15.179:0/3438470418 wait complete. 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.980+0000 7f054e4e7740 20 client.9282 trim_cache size 0 max 16384 2024-05-08T22:43:33.989 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:33.990 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:43:33.990 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:43:33.990 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 msgr2=0x7f0524068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 0x7f0524068bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f052c007fd0 tx=0x7f052c009040 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 msgr2=0x7f05440c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f053400e840 tx=0x7f053400eb50 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 shutdown_connections 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0524066730 0x7f0524068bf0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f054415d360 0x7f054415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05440b51e0 0x7f054415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 --2- 172.21.15.179:0/566476873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f054406abb0 0x7f05440c84c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 >> 172.21.15.179:0/566476873 conn(0x7f05440d3050 msgr2=0x7f05440b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 shutdown_connections 2024-05-08T22:43:33.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:33.984+0000 7f054b238700 1 -- 172.21.15.179:0/566476873 wait complete. 2024-05-08T22:43:34.005 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:43:34.107 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:43:34.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 Processor -- start 2024-05-08T22:43:34.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 -- start start 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b040c2170 0x7f6b040cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b0406cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b04161af0 con 0x7f6b040cda40 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b040cded0 con 0x7f6b040c2170 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b0b54a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040ca670 con 0x7f6b0406c2d0 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b092e6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b040c2170 0x7f6b040cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b08ae5700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6b040cda40 0x7f6b0406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43724/0 (socket says 172.21.15.179:43724) 2024-05-08T22:43:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b08ae5700 1 -- 172.21.15.179:0/2073538469 learned_addr learned my addr 172.21.15.179:0/2073538469 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:34.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.224+0000 7f6b09ae7700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b0406cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b092e6700 1 -- 172.21.15.179:0/2073538469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 msgr2=0x7f6b0406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b092e6700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b0406cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b092e6700 1 -- 172.21.15.179:0/2073538469 >> v1:172.21.15.179:6789/0 conn(0x7f6b040cda40 legacy=0x7f6b0406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b03fff700 1 -- 172.21.15.179:0/2073538469 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365242214 0 0) 0x7f6afc001490 con 0x7f6b040cda40 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b092e6700 1 -- 172.21.15.179:0/2073538469 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040ca950 con 0x7f6b040c2170 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b092e6700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040c2170 0x7f6b040cd500 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009850 tx=0x7f6af8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8fc793c8b1c3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b03fff700 1 -- 172.21.15.179:0/2073538469 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8011830 con 0x7f6b040c2170 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b03fff700 1 -- 172.21.15.179:0/2073538469 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8011990 con 0x7f6b040c2170 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b03fff700 1 -- 172.21.15.179:0/2073538469 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8011d40 con 0x7f6b040c2170 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- 172.21.15.179:0/2073538469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040c2170 msgr2=0x7f6b040cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040c2170 0x7f6b040cd500 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009850 tx=0x7f6af8009b60 comp rx=0 tx=0).stop 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- 172.21.15.179:0/2073538469 shutdown_connections 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b0406cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/2073538469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040c2170 0x7f6b040cd500 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- 172.21.15.179:0/2073538469 >> 172.21.15.179:0/2073538469 conn(0x7f6b04070e50 msgr2=0x7f6b04071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- 172.21.15.179:0/2073538469 shutdown_connections 2024-05-08T22:43:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- 172.21.15.179:0/2073538469 wait complete. 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 Processor -- start 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.228+0000 7f6b0b54a700 1 -- start start 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b041531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 0x7f6b040c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041fa8d0 con 0x7f6b040cda40 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040c7230 con 0x7f6b040c2170 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040c7510 con 0x7f6b0406c2d0 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47958/0 (socket says 172.21.15.179:47958) 2024-05-08T22:43:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 -- 172.21.15.179:0/3095233205 learned_addr learned my addr 172.21.15.179:0/3095233205 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b092e6700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b041531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b08ae5700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 0x7f6b040c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 -- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 msgr2=0x7f6b041531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b041531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 -- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 msgr2=0x7f6b040c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 0x7f6b040c1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 -- 172.21.15.179:0/3095233205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80094b0 con 0x7f6b040cda40 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b08ae5700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 0x7f6b040c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b09ae7700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e7f0 tx=0x7f6af400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40095c0 con 0x7f6b040cda40 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af4013680 con 0x7f6b040cda40 2024-05-08T22:43:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b040c77f0 con 0x7f6b040cda40 2024-05-08T22:43:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4013a10 con 0x7f6b040cda40 2024-05-08T22:43:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04153730 con 0x7f6b040cda40 2024-05-08T22:43:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04067770 con 0x7f6b040cda40 2024-05-08T22:43:34.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.232+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6af4013b70 con 0x7f6b040cda40 2024-05-08T22:43:34.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.236+0000 7f6b01ffb700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 0x7f6af0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.236+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6af4084180 con 0x7f6b040cda40 2024-05-08T22:43:34.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.236+0000 7f6b092e6700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 0x7f6af0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.236+0000 7f6b092e6700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 0x7f6af0068bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6af80090b0 tx=0x7f6af801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.244+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6af4051370 con 0x7f6b040cda40 2024-05-08T22:43:34.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.452+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b04073290 con 0x7f6b040cda40 2024-05-08T22:43:34.457 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:43:34.452+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:43:34.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.464+0000 7f6b01ffb700 1 -- 172.21.15.179:0/3095233205 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6af4054990 con 0x7f6b040cda40 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 msgr2=0x7f6af0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 0x7f6af0068bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6af80090b0 tx=0x7f6af801c040 comp rx=0 tx=0).stop 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 msgr2=0x7f6b040bd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e7f0 tx=0x7f6af400eb00 comp rx=0 tx=0).stop 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 shutdown_connections 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6af0066730 0x7f6af0068bf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b040c2170 0x7f6b040c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b040cda40 0x7f6b040bd860 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 --2- 172.21.15.179:0/3095233205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b0406c2d0 0x7f6b041531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 >> 172.21.15.179:0/3095233205 conn(0x7f6b04070e50 msgr2=0x7f6b040b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 shutdown_connections 2024-05-08T22:43:34.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.468+0000 7f6b0b54a700 1 -- 172.21.15.179:0/3095233205 wait complete. 2024-05-08T22:43:34.474 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:43:34.475 INFO:tasks.ceph.mds.d.smithi195.stderr: -6015> 2024-05-08T22:42:38.202+0000 7f3d336ae700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:43:34.487 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-08T22:43:34.509 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:43:34.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 -- start start 2024-05-08T22:43:34.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 Processor -- start 2024-05-08T22:43:34.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 0x55b595c62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595bbd000 0x55b595c63080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e780 con 0x55b595bbd400 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3ea00 con 0x55b595bbd000 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e640 con 0x55b595bbd800 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef99b30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef99b30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47960/0 (socket says 172.21.15.179:47960) 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 0x55b595c62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef99b30700 1 -- 172.21.15.179:0/3475322883 learned_addr learned my addr 172.21.15.179:0/3475322883 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9ab32700 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595bbd000 0x55b595c63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 -- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595bbd000 msgr2=0x55b595c63080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595bbd000 0x55b595c63080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 -- 172.21.15.179:0/3475322883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 msgr2=0x55b595c62b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 -- 172.21.15.179:0/3475322883 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b595bde9c0 con 0x55b595bbd800 2024-05-08T22:43:34.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef99b30700 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:34.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef9a331700 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 0x55b595c62580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55b595ccb320 tx=0x55b595ea8120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35eb423978e54bba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef97b2c700 1 -- 172.21.15.179:0/3475322883 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595bed600 con 0x55b595bbd800 2024-05-08T22:43:34.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef97b2c700 1 -- 172.21.15.179:0/3475322883 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b595c08480 con 0x55b595bbd800 2024-05-08T22:43:34.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.520+0000 7fef97b2c700 1 -- 172.21.15.179:0/3475322883 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595bed1e0 con 0x55b595bbd800 2024-05-08T22:43:34.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b595bdeb60 con 0x55b595bbd800 2024-05-08T22:43:34.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b595bde340 con 0x55b595bbd800 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 4 waiting for MDS map... 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef97b2c700 1 -- 172.21.15.179:0/3475322883 <== mon.2 v2:172.21.15.195:3301/0 4 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x55b595bf6c40 con 0x55b595bbd800 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef97b2c700 1 -- 172.21.15.179:0/3475322883 <== mon.2 v2:172.21.15.195:3301/0 5 ==== fsmap(e 76) v1 ==== 2573+0+0 (secure 0 0 0) 0x55b595eb2000 con 0x55b595bbd800 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 4 Got MDS map 76 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 10 main: JournalTool::main 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 4 main: JournalTool: connecting to RADOS... 2024-05-08T22:43:34.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 Processor -- start 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- start start 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebac00 0x55b595c63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebb000 0x55b595c64100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595bf3e00 con 0x55b595eba800 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e780 con 0x55b595ebac00 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e500 con 0x55b595ebb000 2024-05-08T22:43:34.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.524+0000 7fef9ab32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47964/0 (socket says 172.21.15.179:47964) 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/970124683 learned_addr learned my addr 172.21.15.179:0/970124683 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9a331700 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebac00 0x55b595c63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef99b30700 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebb000 0x55b595c64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebb000 msgr2=0x55b595c64100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebb000 0x55b595c64100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebac00 msgr2=0x55b595c63b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebac00 0x55b595c63b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/970124683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b595bde000 con 0x55b595eba800 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x55b5969ae690 tx=0x55b595ed4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e9ecb309bf6377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef95b28700 1 -- 172.21.15.179:0/970124683 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595bed600 con 0x55b595eba800 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef95b28700 1 -- 172.21.15.179:0/970124683 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b5969b4d80 con 0x55b595eba800 2024-05-08T22:43:34.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef95b28700 1 -- 172.21.15.179:0/970124683 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595bed1e0 con 0x55b595eba800 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- 172.21.15.179:0/970124683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 msgr2=0x55b595c63600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x55b5969ae690 tx=0x55b595ed4c00 comp rx=0 tx=0).stop 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- 172.21.15.179:0/970124683 shutdown_connections 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebac00 0x55b595c63b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebb000 0x55b595c64100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- 172.21.15.179:0/970124683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c63600 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- 172.21.15.179:0/970124683 >> 172.21.15.179:0/970124683 conn(0x55b595bbcc00 msgr2=0x55b595c79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- 172.21.15.179:0/970124683 shutdown_connections 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- 172.21.15.179:0/970124683 wait complete. 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 Processor -- start 2024-05-08T22:43:34.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- start start 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebac00 0x55b595c64c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebb000 0x55b595c64680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e3c0 con 0x55b595eba800 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e280 con 0x55b595ebb000 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9bb77500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b595c3e140 con 0x55b595ebac00 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47972/0 (socket says 172.21.15.179:47972) 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/2483261679 learned_addr learned my addr 172.21.15.179:0/2483261679 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9a331700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebac00 0x55b595c64c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef99b30700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebb000 0x55b595c64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.528+0000 7fef9ab32700 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebac00 msgr2=0x55b595c64c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9ab32700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebac00 0x55b595c64c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9ab32700 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebb000 msgr2=0x55b595c64680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9ab32700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595ebb000 0x55b595c64680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9ab32700 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b595bde9c0 con 0x55b595eba800 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9a331700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595ebac00 0x55b595c64c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9ab32700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x55b5969af350 tx=0x55b595ea80c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef94325700 1 -- 172.21.15.179:0/2483261679 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595becdc0 con 0x55b595eba800 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef94325700 1 -- 172.21.15.179:0/2483261679 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b5969b5980 con 0x55b595eba800 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b595bdeb60 con 0x55b595eba800 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef94325700 1 -- 172.21.15.179:0/2483261679 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b595bed600 con 0x55b595eba800 2024-05-08T22:43:34.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b595bdeea0 con 0x55b595eba800 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef9bb77500 4 main: JournalTool: resolving pool 15 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef94325700 1 -- 172.21.15.179:0/2483261679 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55b595c1c840 con 0x55b595eba800 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.532+0000 7fef94325700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55b595f36400 0x55b595c65700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef94325700 1 -- 172.21.15.179:0/2483261679 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x55b595bf6a80 con 0x55b595eba800 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 4 main: JournalTool: creating IoCtx.. 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 4 main: Executing for rank 0 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 4 Filtering by inode '1099511627777' 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9a331700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55b595f36400 0x55b595c65700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55b595ebb400 0x55b595c64100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x55b595ebb800 con 0x55b595ebb400 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef99b30700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55b595ebb400 0x55b595c64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9a331700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55b595f36400 0x55b595c65700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55b595ccb5f0 tx=0x55b595c920c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef99b30700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55b595ebb400 0x55b595c64100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef99b30700 1 -- 172.21.15.179:0/2483261679 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55b5969a2b40 con 0x55b595ebb400 2024-05-08T22:43:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 4 Pointer 400.00000000 is readable 2024-05-08T22:43:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-08T22:43:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x55b595ebbc00 con 0x55b595ebb400 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef99b30700 1 -- 172.21.15.179:0/2483261679 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55b5969a2b40 con 0x55b595ebb400 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 10 Header 0x400000 0x405ee2 0x40625e 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 10 Starting journal scan from offset 0x405ee2 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x55b595f36800 con 0x55b595ebb400 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.536+0000 7fef99b30700 1 -- 172.21.15.179:0/2483261679 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55b5969a2b40 con 0x55b595ebb400 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 10 Parsing data, 0x37c bytes available 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 10 Attempting decode at 0x405ee2 2024-05-08T22:43:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 10 Consumed 0x37c bytes 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 10 Valid entry at 0x405ee2 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55b595f36c00 0x55b595c63b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x55b595f37000 con 0x55b595f36c00 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9ab32700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55b595f36c00 0x55b595c63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.540+0000 7fef9ab32700 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55b595f36c00 0x55b595c63b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9ab32700 1 -- 172.21.15.179:0/2483261679 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b5969a2d80 con 0x55b595f36c00 2024-05-08T22:43:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 Reached end of journal objects 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 Scanned objects, 0 missing, 1 valid 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 Events scanned, 0 gaps 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 Found 1 valid events 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 Selected 0 events events for processing 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stdout:Events by type: 2024-05-08T22:43:34.555 INFO:teuthology.orchestra.run.smithi179.stdout:Errors: 0 2024-05-08T22:43:34.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 4 0 events 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55b595ebb400 msgr2=0x55b595c64100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55b595ebb400 0x55b595c64100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55b595f36c00 msgr2=0x55b595c63b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55b595f36c00 0x55b595c63b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55b595f36400 msgr2=0x55b595c65700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55b595f36400 0x55b595c65700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55b595ccb5f0 tx=0x55b595c920c0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 msgr2=0x55b595c65180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef99b30700 1 -- 172.21.15.179:0/2483261679 reap_dead start 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x55b5969af350 tx=0x55b595ea80c0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 shutdown_connections 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/2483261679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595eba800 0x55b595c65180 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 >> 172.21.15.179:0/2483261679 conn(0x55b595bbcc00 msgr2=0x55b595c79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 shutdown_connections 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/2483261679 wait complete. 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 msgr2=0x55b595c62580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 0x55b595c62580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55b595ccb320 tx=0x55b595ea8120 comp rx=0 tx=0).stop 2024-05-08T22:43:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 shutdown_connections 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b595bbd000 0x55b595c63080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b595bbd400 0x55b595c62b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 --2- 172.21.15.179:0/3475322883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b595bbd800 0x55b595c62580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 >> 172.21.15.179:0/3475322883 conn(0x55b595bbdc00 msgr2=0x55b595c79000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 shutdown_connections 2024-05-08T22:43:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.544+0000 7fef9bb77500 1 -- 172.21.15.179:0/3475322883 wait complete. 2024-05-08T22:43:34.561 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.073888, result: Events by type: Errors: 0 2024-05-08T22:43:34.561 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-08T22:43:34.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 -- start start 2024-05-08T22:43:34.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 Processor -- start 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e6e51800 0x5648e6ef6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e6e51400 0x5648e6ef6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6e87e00 con 0x5648e6e51000 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2a00 con 0x5648e6e51400 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2640 con 0x5648e6e51800 2024-05-08T22:43:34.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3a300700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3a300700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47988/0 (socket says 172.21.15.179:47988) 2024-05-08T22:43:34.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c392fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e6e51400 0x5648e6ef6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c3a300700 1 -- 172.21.15.179:0/2072080580 learned_addr learned my addr 172.21.15.179:0/2072080580 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.644+0000 7f3c39aff700 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e6e51800 0x5648e6ef6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 -- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e6e51800 msgr2=0x5648e6ef6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e6e51800 0x5648e6ef6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 -- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e6e51400 msgr2=0x5648e6ef6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e6e51400 0x5648e6ef6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 -- 172.21.15.179:0/2072080580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e6e72b60 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3a300700 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x5648e6f5f410 tx=0x5648e7cd0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5e0b95e74adff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c372fa700 1 -- 172.21.15.179:0/2072080580 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e81600 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c372fa700 1 -- 172.21.15.179:0/2072080580 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e7c28000 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c372fa700 1 -- 172.21.15.179:0/2072080580 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e811e0 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648e6e72340 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5648e6e72000 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 4 waiting for MDS map... 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c372fa700 1 -- 172.21.15.179:0/2072080580 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x5648e6e8ac40 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c372fa700 1 -- 172.21.15.179:0/2072080580 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 76) v1 ==== 2573+0+0 (secure 0 0 0) 0x5648e7c3a000 con 0x5648e6e51000 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 4 Got MDS map 76 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 10 main: JournalTool::main 2024-05-08T22:43:34.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 4 main: JournalTool: connecting to RADOS... 2024-05-08T22:43:34.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 1 Processor -- start 2024-05-08T22:43:34.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.648+0000 7f3c3b345500 1 -- start start 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144400 0x5648e6ef7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2500 con 0x5648e7144800 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed23c0 con 0x5648e7144400 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2a00 con 0x5648e7144c00 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47990/0 (socket says 172.21.15.179:47990) 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 -- 172.21.15.179:0/3621724335 learned_addr learned my addr 172.21.15.179:0/3621724335 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c392fe700 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144400 0x5648e6ef7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c392fe700 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 -- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 msgr2=0x5648e6ef8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 -- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144400 msgr2=0x5648e6ef7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144400 0x5648e6ef7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 -- 172.21.15.179:0/3621724335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e6e72ea0 con 0x5648e7144800 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c39aff700 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x5648e7152810 tx=0x5648e7158c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97eb6f416431db4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c352f6700 1 -- 172.21.15.179:0/3621724335 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e81600 con 0x5648e7144800 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c352f6700 1 -- 172.21.15.179:0/3621724335 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e7c29500 con 0x5648e7144800 2024-05-08T22:43:34.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c352f6700 1 -- 172.21.15.179:0/3621724335 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e811e0 con 0x5648e7144800 2024-05-08T22:43:34.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- 172.21.15.179:0/3621724335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 msgr2=0x5648e6ef7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x5648e7152810 tx=0x5648e7158c00 comp rx=0 tx=0).stop 2024-05-08T22:43:34.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- 172.21.15.179:0/3621724335 shutdown_connections 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144800 0x5648e6ef7b80 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 --2- 172.21.15.179:0/3621724335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144400 0x5648e6ef7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- 172.21.15.179:0/3621724335 >> 172.21.15.179:0/3621724335 conn(0x5648e6e50c00 msgr2=0x5648e6f0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- 172.21.15.179:0/3621724335 shutdown_connections 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.652+0000 7f3c3b345500 1 -- 172.21.15.179:0/3621724335 wait complete. 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 Processor -- start 2024-05-08T22:43:34.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- start start 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 0x5648e6ef9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144800 0x5648e6ef8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2280 con 0x5648e7144400 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2140 con 0x5648e7144800 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e6ed2000 con 0x5648e7144c00 2024-05-08T22:43:34.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 0x5648e6ef9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 0x5648e6ef9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47998/0 (socket says 172.21.15.179:47998) 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 learned_addr learned my addr 172.21.15.179:0/1644835220 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c39aff700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144800 0x5648e6ef8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 msgr2=0x5648e6ef8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e7144c00 0x5648e6ef8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144800 msgr2=0x5648e6ef8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e7144800 0x5648e6ef8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e6e72b60 con 0x5648e7144400 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 0x5648e6ef9180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x5648e71534d0 tx=0x5648e6f26060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c33af3700 1 -- 172.21.15.179:0/1644835220 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e81600 con 0x5648e7144400 2024-05-08T22:43:34.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c33af3700 1 -- 172.21.15.179:0/1644835220 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e6e9c180 con 0x5648e7144400 2024-05-08T22:43:34.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648e6e72340 con 0x5648e7144400 2024-05-08T22:43:34.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c33af3700 1 -- 172.21.15.179:0/1644835220 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e6e811e0 con 0x5648e7144400 2024-05-08T22:43:34.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648e6e72000 con 0x5648e7144400 2024-05-08T22:43:34.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.656+0000 7f3c3b345500 4 main: JournalTool: resolving pool 15 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c33af3700 1 -- 172.21.15.179:0/1644835220 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x5648e6eb0840 con 0x5648e7144400 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c33af3700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x5648e72ac000 0x5648e6ef9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c33af3700 1 -- 172.21.15.179:0/1644835220 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x5648e6e8aa80 con 0x5648e7144400 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c3b345500 4 main: JournalTool: creating IoCtx.. 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c3b345500 4 main: Executing for rank 0 2024-05-08T22:43:34.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c3b345500 4 Filtering by inode '1099511627778' 2024-05-08T22:43:34.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5648e7145000 0x5648e6ef8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x5648e7145400 con 0x5648e7145000 2024-05-08T22:43:34.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c39aff700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x5648e72ac000 0x5648e6ef9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5648e7145000 0x5648e6ef8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c39aff700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x5648e72ac000 0x5648e6ef9700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5648e7152810 tx=0x5648e7cd01e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.660+0000 7f3c392fe700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5648e7145000 0x5648e6ef8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5648e7cc6b40 con 0x5648e7145000 2024-05-08T22:43:34.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Pointer 400.00000000 is readable 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x5648e7145800 con 0x5648e7145000 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5648e7cc6b40 con 0x5648e7145000 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Header 0x400000 0x405ee2 0x40625e 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Starting journal scan from offset 0x405ee2 2024-05-08T22:43:34.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x5648e72ac400 con 0x5648e7145000 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c392fe700 1 -- 172.21.15.179:0/1644835220 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5648e7cc6b40 con 0x5648e7145000 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Parsing data, 0x37c bytes available 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Attempting decode at 0x405ee2 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Consumed 0x37c bytes 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 10 Valid entry at 0x405ee2 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5648e72ac800 0x5648e6ef7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) v8 -- 0x5648e72acc00 con 0x5648e72ac800 2024-05-08T22:43:34.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3a300700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5648e72ac800 0x5648e6ef7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3a300700 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5648e72ac800 0x5648e6ef7b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3a300700 1 -- 172.21.15.179:0/1644835220 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5648e7cc6d80 con 0x5648e72ac800 2024-05-08T22:43:34.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Reached end of journal objects 2024-05-08T22:43:34.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Scanned objects, 0 missing, 1 valid 2024-05-08T22:43:34.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Events scanned, 0 gaps 2024-05-08T22:43:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:Events by type: 2024-05-08T22:43:34.671 INFO:teuthology.orchestra.run.smithi179.stdout:Errors: 0 2024-05-08T22:43:34.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Found 1 valid events 2024-05-08T22:43:34.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 Selected 0 events events for processing 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 4 0 events 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5648e7145000 msgr2=0x5648e6ef8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x5648e7145000 0x5648e6ef8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5648e72ac800 msgr2=0x5648e6ef7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5648e72ac800 0x5648e6ef7b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x5648e72ac000 msgr2=0x5648e6ef9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x5648e72ac000 0x5648e6ef9700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5648e7152810 tx=0x5648e7cd01e0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 msgr2=0x5648e6ef9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 --2- 172.21.15.179:0/1644835220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e7144400 0x5648e6ef9180 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x5648e71534d0 tx=0x5648e6f26060 comp rx=0 tx=0).stop 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c39aff700 1 -- 172.21.15.179:0/1644835220 reap_dead start 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 shutdown_connections 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 >> 172.21.15.179:0/1644835220 conn(0x5648e6e50c00 msgr2=0x5648e6f0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c39aff700 1 -- 172.21.15.179:0/1644835220 reap_dead start 2024-05-08T22:43:34.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.664+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 shutdown_connections 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/1644835220 wait complete. 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 msgr2=0x5648e6ef7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x5648e6f5f410 tx=0x5648e7cd0240 comp rx=0 tx=0).stop 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 shutdown_connections 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648e6e51400 0x5648e6ef6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648e6e51000 0x5648e6ef7080 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 --2- 172.21.15.179:0/2072080580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5648e6e51800 0x5648e6ef6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 >> 172.21.15.179:0/2072080580 conn(0x5648e6e51c00 msgr2=0x5648e6f0d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 shutdown_connections 2024-05-08T22:43:34.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.668+0000 7f3c3b345500 1 -- 172.21.15.179:0/2072080580 wait complete. 2024-05-08T22:43:34.675 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.113699, result: Events by type: Errors: 0 2024-05-08T22:43:34.675 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-08T22:43:34.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 Processor -- start 2024-05-08T22:43:34.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 -- start start 2024-05-08T22:43:34.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.720 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83bc8b0 0x555fe845c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe8460700 0x555fe845cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fe82d88e0 con 0x555fe83891e0 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fe82d8580 con 0x555fe8460700 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb9669140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fe82ae8d0 con 0x555fe83bc8b0 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48028/0 (socket says 172.21.15.179:48028) 2024-05-08T22:43:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 -- 172.21.15.179:0/2482893460 learned_addr learned my addr 172.21.15.179:0/2482893460 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb799f700 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe8460700 0x555fe845cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb699d700 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83bc8b0 0x555fe845c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 -- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83bc8b0 msgr2=0x555fe845c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83bc8b0 0x555fe845c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 -- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe8460700 msgr2=0x555fe845cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe8460700 0x555fe845cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.712+0000 7faeb719e700 1 -- 172.21.15.179:0/2482893460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea000a100 con 0x555fe83891e0 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb719e700 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7faeb0015040 tx=0x7faeb000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17483a56b0f8826 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb599b700 1 -- 172.21.15.179:0/2482893460 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0008690 con 0x555fe83891e0 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb599b700 1 -- 172.21.15.179:0/2482893460 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb0008b60 con 0x555fe83891e0 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb599b700 1 -- 172.21.15.179:0/2482893460 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb001c410 con 0x555fe83891e0 2024-05-08T22:43:34.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- 172.21.15.179:0/2482893460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 msgr2=0x555fe83bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7faeb0015040 tx=0x7faeb000d000 comp rx=0 tx=0).stop 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- 172.21.15.179:0/2482893460 shutdown_connections 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe8460700 0x555fe845cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83bc8b0 0x555fe845c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 --2- 172.21.15.179:0/2482893460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe83891e0 0x555fe83bac10 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- 172.21.15.179:0/2482893460 >> 172.21.15.179:0/2482893460 conn(0x555fe838f740 msgr2=0x555fe845ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- 172.21.15.179:0/2482893460 shutdown_connections 2024-05-08T22:43:34.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- 172.21.15.179:0/2482893460 wait complete. 2024-05-08T22:43:34.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 Processor -- start 2024-05-08T22:43:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.716+0000 7faeb9669140 1 -- start start 2024-05-08T22:43:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83891e0 0x555fe8457a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe83bc8b0 0x555fe83ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 0x555fe84428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fe8457f80 con 0x555fe8460700 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fe843f970 con 0x555fe83bc8b0 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fe843fc10 con 0x555fe83891e0 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 0x555fe84428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 0x555fe84428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48044/0 (socket says 172.21.15.179:48044) 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 -- 172.21.15.179:0/565191130 learned_addr learned my addr 172.21.15.179:0/565191130 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb699d700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe83bc8b0 0x555fe83ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb719e700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83891e0 0x555fe8457a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 -- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83891e0 msgr2=0x555fe8457a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555fe83891e0 0x555fe8457a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 -- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe83bc8b0 msgr2=0x555fe83ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555fe83bc8b0 0x555fe83ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 -- 172.21.15.179:0/565191130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fe82b9bb0 con 0x555fe8460700 2024-05-08T22:43:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb799f700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 0x555fe84428c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7faea8008760 tx=0x7faea8008a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb4999700 1 -- 172.21.15.179:0/565191130 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800f040 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb4999700 1 -- 172.21.15.179:0/565191130 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea80148a0 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb4999700 1 -- 172.21.15.179:0/565191130 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8014c30 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb0005de0 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.720+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fe82e0ca0 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb4999700 1 -- 172.21.15.179:0/565191130 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faea8015470 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb4999700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae98065090 0x7fae98067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb4999700 1 -- 172.21.15.179:0/565191130 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7faea8083950 con 0x555fe8460700 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb719e700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae98065090 0x7fae98067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb9669140 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555fe8442e00 0x555fe845cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x555fe843f430 con 0x555fe8442e00 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb699d700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555fe8442e00 0x555fe845cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb699d700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555fe8442e00 0x555fe845cab0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.724+0000 7faeb719e700 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae98065090 0x7fae98067550 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faeb0022000 tx=0x7faeb0008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.728+0000 7faeb699d700 1 -- 172.21.15.179:0/565191130 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v115'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7faea000ab80 con 0x555fe8442e00 2024-05-08T22:43:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555fe8442e00 msgr2=0x555fe845cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555fe8442e00 0x555fe845cab0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae98065090 msgr2=0x7fae98067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fae98065090 0x7fae98067550 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faeb0022000 tx=0x7faeb0008000 comp rx=0 tx=0).stop 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 msgr2=0x555fe84428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 --2- 172.21.15.179:0/565191130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fe8460700 0x555fe84428c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7faea8008760 tx=0x7faea8008a70 comp rx=0 tx=0).stop 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb799f700 1 -- 172.21.15.179:0/565191130 reap_dead start 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 shutdown_connections 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 >> 172.21.15.179:0/565191130 conn(0x555fe838f740 msgr2=0x555fe845deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 shutdown_connections 2024-05-08T22:43:34.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.732+0000 7faeb9669140 1 -- 172.21.15.179:0/565191130 wait complete. 2024-05-08T22:43:34.739 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:43:34.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 Processor -- start 2024-05-08T22:43:34.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 -- start start 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd8e0159320 0x7fd8e00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e006cda0 con 0x7fd8e0159700 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00ccd40 con 0x7fd8e00b19e0 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5d72700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00b1f10 con 0x7fd8e0159320 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5571700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd8e0159700 0x7fd8e006b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:43752/0 (socket says 172.21.15.179:43752) 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e5571700 1 -- 172.21.15.179:0/3372511515 learned_addr learned my addr 172.21.15.179:0/3372511515 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> v2:172.21.15.195:3300/0 conn(0x7fd8e0159320 0x7fd8e00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611204798 0 0) 0x7fd8e006cda0 con 0x7fd8e0159700 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8cc0035d0 con 0x7fd8e0159700 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731914275 0 0) 0x7fd8cc0035d0 con 0x7fd8e0159700 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 >> v2:172.21.15.195:3300/0 conn(0x7fd8e0159320 msgr2=0x7fd8e00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.904+0000 7fd8df7fe700 1 --2- 172.21.15.179:0/3372511515 >> v2:172.21.15.195:3300/0 conn(0x7fd8e0159320 0x7fd8e00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 >> v1:172.21.15.195:6789/0 conn(0x7fd8e00b19e0 legacy=0x7fd8e00cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00b21f0 con 0x7fd8e0159700 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fd8d80019f0 con 0x7fd8e0159700 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 >> v1:172.21.15.179:6789/0 conn(0x7fd8e0159700 legacy=0x7fd8e006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 0x7fd8cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc004420 0x7fd8cc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8cc0050f0 0x7fd8cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b1f10 con 0x7fd8cc003a20 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc00c050 con 0x7fd8cc004420 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc00c210 con 0x7fd8cc0050f0 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> v2:172.21.15.195:3300/0 conn(0x7fd8e0159320 0x7fd8e00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 0x7fd8cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5571700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8cc0050f0 0x7fd8cc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8dffff700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc004420 0x7fd8cc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 -- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8cc0050f0 msgr2=0x7fd8cc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8cc0050f0 0x7fd8cc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 -- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc004420 msgr2=0x7fd8cc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc004420 0x7fd8cc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5571700 1 -- 172.21.15.179:0/3372511515 reap_dead start 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 -- 172.21.15.179:0/3372511515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc0035d0 con 0x7fd8cc003a20 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 0x7fd8cc003e00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009d20 tx=0x7fd8cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3645386b1c0bd94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8009e70 con 0x7fd8cc003a20 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800e9a0 con 0x7fd8cc003a20 2024-05-08T22:43:34.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8df7fe700 1 -- 172.21.15.179:0/3372511515 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800ed10 con 0x7fd8cc003a20 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 -- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 msgr2=0x7fd8cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 0x7fd8cc003e00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009d20 tx=0x7fd8cc0058a0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 -- 172.21.15.179:0/3372511515 shutdown_connections 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/3372511515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8cc003a20 0x7fd8cc003e00 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 -- 172.21.15.179:0/3372511515 >> 172.21.15.179:0/3372511515 conn(0x7fd8e00cea30 msgr2=0x7fd8e00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 -- 172.21.15.179:0/3372511515 shutdown_connections 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.908+0000 7fd8e5d72700 1 -- 172.21.15.179:0/3372511515 wait complete. 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 Processor -- start 2024-05-08T22:43:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- start start 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc003a20 0x7fd8e013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8e01f0550 0x7fd8e01472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e014c210 con 0x7fd8e014a830 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0145260 con 0x7fd8cc003a20 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e014bc60 con 0x7fd8e01f0550 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48056/0 (socket says 172.21.15.179:48056) 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 -- 172.21.15.179:0/1784066654 learned_addr learned my addr 172.21.15.179:0/1784066654 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5571700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8e01f0550 0x7fd8e01472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc003a20 0x7fd8e013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 -- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8e01f0550 msgr2=0x7fd8e01472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8e01f0550 0x7fd8e01472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 -- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc003a20 msgr2=0x7fd8e013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc003a20 0x7fd8e013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 -- 172.21.15.179:0/1784066654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e014bf30 con 0x7fd8e014a830 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dffff700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000f770 tx=0x7fd8d000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00052e0 con 0x7fd8e014a830 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d0005460 con 0x7fd8e014a830 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e0144c10 con 0x7fd8e014a830 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00038e0 con 0x7fd8e014a830 2024-05-08T22:43:34.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.912+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e01477f0 con 0x7fd8e014a830 2024-05-08T22:43:34.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e0144a70 con 0x7fd8e014a830 2024-05-08T22:43:34.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8d0003a40 con 0x7fd8e014a830 2024-05-08T22:43:34.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8dd7fa700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 0x7fd8c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:34.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd8d0083ed0 con 0x7fd8e014a830 2024-05-08T22:43:34.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 0x7fd8c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:34.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.916+0000 7fd8e4d70700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 0x7fd8c0068c20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80090b0 tx=0x7fd8c801e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:34.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:34.928+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8e0144a70 con 0x7fd8e014a830 2024-05-08T22:43:35.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.128+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd8e0144a70 con 0x7fd8e014a830 2024-05-08T22:43:35.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.472+0000 7fd8dd7fa700 1 -- 172.21.15.179:0/1784066654 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v77) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd8e0144a70 con 0x7fd8e014a830 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 msgr2=0x7fd8c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 0x7fd8c0068c20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80090b0 tx=0x7fd8c801e040 comp rx=0 tx=0).stop 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 msgr2=0x7fd8e014b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000f770 tx=0x7fd8d000fa80 comp rx=0 tx=0).stop 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 shutdown_connections 2024-05-08T22:43:35.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd8c0066760 0x7fd8c0068c20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8e01f0550 0x7fd8e01472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e014a830 0x7fd8e014b720 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 --2- 172.21.15.179:0/1784066654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8cc003a20 0x7fd8e013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 >> 172.21.15.179:0/1784066654 conn(0x7fd8e00cea30 msgr2=0x7fd8e00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 shutdown_connections 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.476+0000 7fd8e5d72700 1 -- 172.21.15.179:0/1784066654 wait complete. 2024-05-08T22:43:35.480 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:43:35.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:43:35.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 Processor -- start 2024-05-08T22:43:35.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 -- start start 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 0x7f955006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f955006ed50 0x7f95500b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500cd200 con 0x7f955006dc20 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500cd4e0 con 0x7f955006ed50 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9556817700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500cd7c0 con 0x7f95501434d0 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9555014700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f955006dc20 0x7f955006e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45958/0 (socket says 172.21.15.179:45958) 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.608+0000 7f9555014700 1 -- 172.21.15.179:0/683031815 learned_addr learned my addr 172.21.15.179:0/683031815 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 --2- 172.21.15.179:0/683031815 >> v2:172.21.15.195:3300/0 conn(0x7f955006ed50 0x7f95500b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9555815700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 0x7f955006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9554813700 1 -- 172.21.15.179:0/683031815 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559479486 0 0) 0x7f95500cd200 con 0x7f955006dc20 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9554813700 1 -- 172.21.15.179:0/683031815 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953c0035d0 con 0x7f955006dc20 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 -- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 msgr2=0x7f955006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 0x7f955006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 -- 172.21.15.179:0/683031815 >> v1:172.21.15.179:6789/0 conn(0x7f955006dc20 legacy=0x7f955006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 -- 172.21.15.179:0/683031815 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500cdaa0 con 0x7f955006ed50 2024-05-08T22:43:35.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9555815700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 0x7f955006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556016700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500b19e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fc9629e9dda8e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9554813700 1 -- 172.21.15.179:0/683031815 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800bd20 con 0x7f955006ed50 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9554813700 1 -- 172.21.15.179:0/683031815 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954800be80 con 0x7f955006ed50 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9554813700 1 -- 172.21.15.179:0/683031815 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548011660 con 0x7f955006ed50 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 -- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 msgr2=0x7f95500b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500b19e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954800b330 comp rx=0 tx=0).stop 2024-05-08T22:43:35.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 -- 172.21.15.179:0/683031815 shutdown_connections 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500b19e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 --2- 172.21.15.179:0/683031815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501434d0 0x7f955006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 -- 172.21.15.179:0/683031815 >> 172.21.15.179:0/683031815 conn(0x7f95500ce9d0 msgr2=0x7f95500cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 -- 172.21.15.179:0/683031815 shutdown_connections 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.612+0000 7f9556817700 1 -- 172.21.15.179:0/683031815 wait complete. 2024-05-08T22:43:35.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 Processor -- start 2024-05-08T22:43:35.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- start start 2024-05-08T22:43:35.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 0x7f95500c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500ca8d0 con 0x7f95501434d0 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c41a0 con 0x7f955006ed50 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c4480 con 0x7f955013d6f0 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555014700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555014700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39684/0 (socket says 172.21.15.179:39684) 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555014700 1 -- 172.21.15.179:0/3955848339 learned_addr learned my addr 172.21.15.179:0/3955848339 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:35.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556016700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 0x7f95500c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 -- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 msgr2=0x7f95500c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 0x7f95500c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 -- 172.21.15.179:0/3955848339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 msgr2=0x7f955006a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 -- 172.21.15.179:0/3955848339 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0094b0 con 0x7f955006ed50 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556016700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 0x7f95500c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555014700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9555815700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500c3950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f954c00e9b0 tx=0x7f954c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00c640 con 0x7f955006ed50 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00c7a0 con 0x7f955006ed50 2024-05-08T22:43:35.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9548009a70 con 0x7f955006ed50 2024-05-08T22:43:35.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00cb50 con 0x7f955006ed50 2024-05-08T22:43:35.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.616+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500c4a40 con 0x7f955006ed50 2024-05-08T22:43:35.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550067740 con 0x7f955006ed50 2024-05-08T22:43:35.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f954c00ccb0 con 0x7f955006ed50 2024-05-08T22:43:35.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f95467fc700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 0x7f9534068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:35.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f954c0847a0 con 0x7f955006ed50 2024-05-08T22:43:35.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f9555014700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 0x7f9534068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:35.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.620+0000 7f9555014700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 0x7f9534068bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9550072d00 tx=0x7f9540009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:35.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.628+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f954c0518a0 con 0x7f955006ed50 2024-05-08T22:43:35.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.840+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f955004e700 con 0x7f955006ed50 2024-05-08T22:43:35.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.844+0000 7f95467fc700 1 -- 172.21.15.179:0/3955848339 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f954c054ec0 con 0x7f955006ed50 2024-05-08T22:43:35.847 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:35.847 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:43:35.475654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57725":{"gid":57725,"name":"d","rank":0,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6837/602724347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":602724347},{"type":"v1","addr":"172.21.15.195:6837","nonce":602724347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:43:35.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 msgr2=0x7f9534068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 0x7f9534068bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9550072d00 tx=0x7f9540009040 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 msgr2=0x7f95500c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500c3950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f954c00e9b0 tx=0x7f954c00ecc0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 shutdown_connections 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9534066730 0x7f9534068bf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955013d6f0 0x7f95500c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501434d0 0x7f955006a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 --2- 172.21.15.179:0/3955848339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955006ed50 0x7f95500c3950 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 >> 172.21.15.179:0/3955848339 conn(0x7f95500ce9d0 msgr2=0x7f9550142570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 shutdown_connections 2024-05-08T22:43:35.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:35.848+0000 7f9556817700 1 -- 172.21.15.179:0/3955848339 wait complete. 2024-05-08T22:43:35.853 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 78 2024-05-08T22:43:36.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:42:37.284354+0000', 'modified': '2024-05-08T22:43:35.475654+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57725': {'gid': 57725, 'name': 'd', 'rank': 0, 'incarnation': 78, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/602724347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 602724347}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 602724347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:43:36.270 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_57725:up:replay 2024-05-08T22:43:37.270 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:43:37.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 Processor -- start 2024-05-08T22:43:37.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- start start 2024-05-08T22:43:37.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3700c9230 0x7fa3700c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa37006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3700ce650 con 0x7fa37015d130 2024-05-08T22:43:37.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3700c9c00 con 0x7fa3700c9230 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700cd340 con 0x7fa37006af90 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa37536e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa37015d130 0x7fa3700cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45966/0 (socket says 172.21.15.179:45966) 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa37536e700 1 -- 172.21.15.179:0/1581914397 learned_addr learned my addr 172.21.15.179:0/1581914397 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa375b6f700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa37006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 --2- 172.21.15.179:0/1581914397 >> v2:172.21.15.195:3300/0 conn(0x7fa3700c9230 0x7fa3700c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906170641 0 0) 0x7fa3700ce650 con 0x7fa37015d130 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa35c0035d0 con 0x7fa37015d130 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 -- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 msgr2=0x7fa37006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa37006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 -- 172.21.15.179:0/1581914397 >> v1:172.21.15.179:6789/0 conn(0x7fa37015d130 legacy=0x7fa3700cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847661883 0 0) 0x7fa35c0035d0 con 0x7fa37015d130 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 -- 172.21.15.179:0/1581914397 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700cd620 con 0x7fa3700c9230 2024-05-08T22:43:37.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa374b6d700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c96c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009850 tx=0x7fa36c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1d01fa84698430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360001670 con 0x7fa3700c9230 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c004030 con 0x7fa3700c9230 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa367fff700 1 -- 172.21.15.179:0/1581914397 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0043a0 con 0x7fa3700c9230 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 msgr2=0x7fa3700c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c96c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009850 tx=0x7fa36c009b60 comp rx=0 tx=0).stop 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- 172.21.15.179:0/1581914397 shutdown_connections 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa37006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 --2- 172.21.15.179:0/1581914397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c96c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- 172.21.15.179:0/1581914397 >> 172.21.15.179:0/1581914397 conn(0x7fa3700cb660 msgr2=0x7fa3700cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- 172.21.15.179:0/1581914397 shutdown_connections 2024-05-08T22:43:37.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.388+0000 7fa3775d2700 1 -- 172.21.15.179:0/1581914397 wait complete. 2024-05-08T22:43:37.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 Processor -- start 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- start start 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa3701594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c7010 con 0x7fa37015d130 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c5020 con 0x7fa3700c9230 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c6670 con 0x7fa37006af90 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39696/0 (socket says 172.21.15.179:39696) 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 -- 172.21.15.179:0/3977397623 learned_addr learned my addr 172.21.15.179:0/3977397623 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa374b6d700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa37536e700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa3701594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 -- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 msgr2=0x7fa3701594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa3701594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 -- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 msgr2=0x7fa3700c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 -- 172.21.15.179:0/3977397623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0094b0 con 0x7fa37015d130 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa37536e700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa3701594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa375b6f700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa36800efc0 tx=0x7fa36800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368009ea0 con 0x7fa37015d130 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36800f040 con 0x7fa37015d130 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3700c6950 con 0x7fa37015d130 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368014480 con 0x7fa37015d130 2024-05-08T22:43:37.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.392+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa370159a00 con 0x7fa37015d130 2024-05-08T22:43:37.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa370159bf0 con 0x7fa37015d130 2024-05-08T22:43:37.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3680145e0 con 0x7fa37015d130 2024-05-08T22:43:37.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa365ffb700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 0x7fa354068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:37.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fa368084c10 con 0x7fa37015d130 2024-05-08T22:43:37.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa37536e700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 0x7fa354068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:37.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.396+0000 7fa37536e700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 0x7fa354068bf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa360001eb0 tx=0x7fa360001c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:37.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.404+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa368051d20 con 0x7fa37015d130 2024-05-08T22:43:37.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.612+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa370067770 con 0x7fa37015d130 2024-05-08T22:43:37.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.612+0000 7fa365ffb700 1 -- 172.21.15.179:0/3977397623 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fa368018070 con 0x7fa37015d130 2024-05-08T22:43:37.617 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:37.617 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":80,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:43:37.486360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57725":{"gid":57725,"name":"d","rank":0,"incarnation":78,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.195:6837/602724347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":602724347},{"type":"v1","addr":"172.21.15.195:6837","nonce":602724347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:43:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 msgr2=0x7fa354068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 0x7fa354068bf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa360001eb0 tx=0x7fa360001c60 comp rx=0 tx=0).stop 2024-05-08T22:43:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 msgr2=0x7fa3701583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa36800efc0 tx=0x7fa36800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 shutdown_connections 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa354066730 0x7fa354068bf0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3700c9230 0x7fa3700c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37015d130 0x7fa3701583c0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 --2- 172.21.15.179:0/3977397623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa37006af90 0x7fa3701594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 >> 172.21.15.179:0/3977397623 conn(0x7fa3700cb660 msgr2=0x7fa37006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 shutdown_connections 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:37.616+0000 7fa3775d2700 1 -- 172.21.15.179:0/3977397623 wait complete. 2024-05-08T22:43:37.621 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 80 2024-05-08T22:43:38.035 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:42:37.284354+0000', 'modified': '2024-05-08T22:43:37.486360+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57725': {'gid': 57725, 'name': 'd', 'rank': 0, 'incarnation': 78, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.195:6837/602724347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 602724347}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 602724347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:43:38.035 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_57725:up:rejoin 2024-05-08T22:43:39.038 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:43:39.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.156+0000 7fbf822c5700 1 Processor -- start 2024-05-08T22:43:39.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.156+0000 7fbf822c5700 1 -- start start 2024-05-08T22:43:39.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf822c5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0cd540 0x7fbf7c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf822c5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf822c5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0ca310 con 0x7fbf7c159e20 2024-05-08T22:43:39.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf822c5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c15a200 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf822c5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b4e20 con 0x7fbf7c0cd540 2024-05-08T22:43:39.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39698/0 (socket says 172.21.15.179:39698) 2024-05-08T22:43:39.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 -- 172.21.15.179:0/3164127119 learned_addr learned my addr 172.21.15.179:0/3164127119 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf80ac2700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0cd540 0x7fbf7c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 -- 172.21.15.179:0/3164127119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0cd540 msgr2=0x7fbf7c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0cd540 0x7fbf7c0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 -- 172.21.15.179:0/3164127119 >> v1:172.21.15.195:6789/0 conn(0x7fbf7c159e20 legacy=0x7fbf7c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf73fff700 1 -- 172.21.15.179:0/3164127119 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293826225 0 0) 0x7fbf7c0ca310 con 0x7fbf7c159e20 2024-05-08T22:43:39.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 -- 172.21.15.179:0/3164127119 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0c99e0 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf81ac4700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d150 tx=0x7fbf7400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e60decf9a06a201 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf73fff700 1 -- 172.21.15.179:0/3164127119 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74010070 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf73fff700 1 -- 172.21.15.179:0/3164127119 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400dd10 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.160+0000 7fbf73fff700 1 -- 172.21.15.179:0/3164127119 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400b310 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- 172.21.15.179:0/3164127119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0b50b0 msgr2=0x7fbf7c0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d150 tx=0x7fbf7400d460 comp rx=0 tx=0).stop 2024-05-08T22:43:39.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- 172.21.15.179:0/3164127119 shutdown_connections 2024-05-08T22:43:39.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0cd540 0x7fbf7c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 --2- 172.21.15.179:0/3164127119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0b50b0 0x7fbf7c0cace0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- 172.21.15.179:0/3164127119 >> 172.21.15.179:0/3164127119 conn(0x7fbf7c098a70 msgr2=0x7fbf7c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- 172.21.15.179:0/3164127119 shutdown_connections 2024-05-08T22:43:39.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- 172.21.15.179:0/3164127119 wait complete. 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 Processor -- start 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 -- start start 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.164+0000 7fbf822c5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0b50b0 0x7fbf7c1f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7c159e20 0x7fbf7c14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0af980 con 0x7fbf7c0cd540 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ae8e0 con 0x7fbf7c159e20 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0aebc0 con 0x7fbf7c0b50b0 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39704/0 (socket says 172.21.15.179:39704) 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 -- 172.21.15.179:0/371932713 learned_addr learned my addr 172.21.15.179:0/371932713 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf812c3700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0b50b0 0x7fbf7c1f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf81ac4700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7c159e20 0x7fbf7c14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 -- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0b50b0 msgr2=0x7fbf7c1f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0b50b0 0x7fbf7c1f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 -- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7c159e20 msgr2=0x7fbf7c14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7c159e20 0x7fbf7c14a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 -- 172.21.15.179:0/371932713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7400bd40 con 0x7fbf7c0cd540 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf80ac2700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00e9d0 tx=0x7fbf6c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c009720 con 0x7fbf7c0cd540 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c00ee70 con 0x7fbf7c0cd540 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c0aeea0 con 0x7fbf7c0cd540 2024-05-08T22:43:39.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00ca50 con 0x7fbf7c0cd540 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c0af460 con 0x7fbf7c0cd540 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c0af620 con 0x7fbf7c0cd540 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.168+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf6c00cbb0 con 0x7fbf7c0cd540 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.172+0000 7fbf71ffb700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 0x7fbf60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.172+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fbf6c084300 con 0x7fbf7c0cd540 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.172+0000 7fbf812c3700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 0x7fbf60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.172+0000 7fbf812c3700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 0x7fbf60068bf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000f60 tx=0x7fbf78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.176+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbf6c0514a0 con 0x7fbf7c0cd540 2024-05-08T22:43:39.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.384+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf7c04e700 con 0x7fbf7c0cd540 2024-05-08T22:43:39.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.384+0000 7fbf71ffb700 1 -- 172.21.15.179:0/371932713 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fbf6c00ce60 con 0x7fbf7c0cd540 2024-05-08T22:43:39.389 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:39.390 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:43:38.489070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57725":{"gid":57725,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6837/602724347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":602724347},{"type":"v1","addr":"172.21.15.195:6837","nonce":602724347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 msgr2=0x7fbf60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 0x7fbf60068bf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000f60 tx=0x7fbf78009040 comp rx=0 tx=0).stop 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 msgr2=0x7fbf7c1f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00e9d0 tx=0x7fbf6c00ece0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 shutdown_connections 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbf60066730 0x7fbf60068bf0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf7c159e20 0x7fbf7c14a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c0cd540 0x7fbf7c1f1fe0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 --2- 172.21.15.179:0/371932713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf7c0b50b0 0x7fbf7c1f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 >> 172.21.15.179:0/371932713 conn(0x7fbf7c098a70 msgr2=0x7fbf7c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 shutdown_connections 2024-05-08T22:43:39.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.388+0000 7fbf822c5700 1 -- 172.21.15.179:0/371932713 wait complete. 2024-05-08T22:43:39.393 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 81 2024-05-08T22:43:39.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:42:37.284354+0000', 'modified': '2024-05-08T22:43:38.489070+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57725': {'gid': 57725, 'name': 'd', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6837/602724347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 602724347}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 602724347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:43:39.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:43:39.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:43:39.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 Processor -- start 2024-05-08T22:43:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 -- start start 2024-05-08T22:43:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f440ce6b0 0x7f2f440cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2f440ccb50 0x7f2f440ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2f44160a50 0x7f2f44160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440b2f40 con 0x7f2f44160a50 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440cefd0 con 0x7f2f440ccb50 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f43fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440cded0 con 0x7f2f440ce6b0 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f437fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2f44160a50 0x7f2f44160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f437fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2f44160a50 0x7f2f44160ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39710/0 (socket says 172.21.15.179:39710) 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f437fe700 1 -- 172.21.15.179:0/3213074485 learned_addr learned my addr 172.21.15.179:0/3213074485 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f427fc700 1 --2- 172.21.15.179:0/3213074485 >> v2:172.21.15.195:3300/0 conn(0x7f2f440ccb50 0x7f2f440ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.936+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f440ce6b0 0x7f2f440cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 -- 172.21.15.179:0/3213074485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f440ce6b0 msgr2=0x7f2f440cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f440ce6b0 0x7f2f440cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 -- 172.21.15.179:0/3213074485 >> v2:172.21.15.195:3300/0 conn(0x7f2f440ccb50 msgr2=0x7f2f440ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 --2- 172.21.15.179:0/3213074485 >> v2:172.21.15.195:3300/0 conn(0x7f2f440ccb50 0x7f2f440ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 -- 172.21.15.179:0/3213074485 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440cd910 con 0x7f2f44160a50 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f44160a50 0x7f2f44160ee0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b26c0 tx=0x7f2f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90e9ae0f77db9a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f41ffb700 1 -- 172.21.15.179:0/3213074485 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e070 con 0x7f2f44160a50 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f41ffb700 1 -- 172.21.15.179:0/3213074485 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00b910 con 0x7f2f44160a50 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f41ffb700 1 -- 172.21.15.179:0/3213074485 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00bcc0 con 0x7f2f44160a50 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- 172.21.15.179:0/3213074485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f44160a50 msgr2=0x7f2f44160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f44160a50 0x7f2f44160ee0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b26c0 tx=0x7f2f3c00b330 comp rx=0 tx=0).stop 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- 172.21.15.179:0/3213074485 shutdown_connections 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f44160a50 0x7f2f44160ee0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- 172.21.15.179:0/3213074485 >> v2:172.21.15.195:3300/0 conn(0x7f2f440ccb50 0x7f2f440ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- 172.21.15.179:0/3213074485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f440ce6b0 0x7f2f440cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- 172.21.15.179:0/3213074485 >> 172.21.15.179:0/3213074485 conn(0x7f2f440b1ed0 msgr2=0x7f2f440cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- 172.21.15.179:0/3213074485 shutdown_connections 2024-05-08T22:43:39.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- 172.21.15.179:0/3213074485 wait complete. 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 Processor -- start 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- start start 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f440ce6b0 0x7f2f4406de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 0x7f2f4406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44142b30 con 0x7f2f440ccb50 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4415c5c0 con 0x7f2f440ce6b0 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f43fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4415c8a0 con 0x7f2f44160a50 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39714/0 (socket says 172.21.15.179:39714) 2024-05-08T22:43:39.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f42ffd700 1 -- 172.21.15.179:0/2021713105 learned_addr learned my addr 172.21.15.179:0/2021713105 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f437fe700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 0x7f2f4406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.940+0000 7f2f427fc700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f440ce6b0 0x7f2f4406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 -- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 msgr2=0x7f2f4406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 0x7f2f4406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 -- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f440ce6b0 msgr2=0x7f2f4406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f440ce6b0 0x7f2f4406de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 -- 172.21.15.179:0/2021713105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009b30 con 0x7f2f440ccb50 2024-05-08T22:43:39.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f437fe700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 0x7f2f4406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e870 tx=0x7f2f3800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f2b7fe700 1 -- 172.21.15.179:0/2021713105 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380095c0 con 0x7f2f440ccb50 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f2b7fe700 1 -- 172.21.15.179:0/2021713105 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38013680 con 0x7f2f440ccb50 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f44157ba0 con 0x7f2f440ccb50 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f2b7fe700 1 -- 172.21.15.179:0/2021713105 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38013a10 con 0x7f2f440ccb50 2024-05-08T22:43:39.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.944+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44158160 con 0x7f2f440ccb50 2024-05-08T22:43:39.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f2b7fe700 1 -- 172.21.15.179:0/2021713105 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f38013b70 con 0x7f2f440ccb50 2024-05-08T22:43:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f2b7fe700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 0x7f2f30068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f2b7fe700 1 -- 172.21.15.179:0/2021713105 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f2f38084290 con 0x7f2f440ccb50 2024-05-08T22:43:39.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f427fc700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 0x7f2f30068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f427fc700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 0x7f2f30068c20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2f44161810 tx=0x7f2f34009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f4c5ba740 1 Processor -- start 2024-05-08T22:43:39.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.948+0000 7f2f4c5ba740 1 -- start start 2024-05-08T22:43:39.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2a68470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa1420 0x2a689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c7d80 con 0x2aa0b00 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c5510 con 0x2a97be0 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x29c0e00 con 0x2aa1420 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39716/0 (socket says 172.21.15.179:39716) 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 -- 172.21.15.179:0/3552632593 learned_addr learned my addr 172.21.15.179:0/3552632593 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f427fc700 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2a68470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f437fe700 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa1420 0x2a689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 -- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa1420 msgr2=0x2a689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa1420 0x2a689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 -- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 msgr2=0x2a68470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2a68470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 -- 172.21.15.179:0/3552632593 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2833960 con 0x2aa0b00 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2f38051550 tx=0x7f2f38055120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c96ccf68e7a13ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f28ff9700 1 -- 172.21.15.179:0/3552632593 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380533f0 con 0x2aa0b00 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f28ff9700 1 -- 172.21.15.179:0/3552632593 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38068e90 con 0x2aa0b00 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f28ff9700 1 -- 172.21.15.179:0/3552632593 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380552b0 con 0x2aa0b00 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/3552632593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 msgr2=0x2aa0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2f38051550 tx=0x7f2f38055120 comp rx=0 tx=0).stop 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/3552632593 shutdown_connections 2024-05-08T22:43:39.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2a68470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa1420 0x2a689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/3552632593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa0b00 0x2aa0ee0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/3552632593 >> 172.21.15.179:0/3552632593 conn(0x27e84d0 msgr2=0x2a82250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/3552632593 shutdown_connections 2024-05-08T22:43:39.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/3552632593 wait complete. 2024-05-08T22:43:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 -- start start 2024-05-08T22:43:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.952+0000 7f2f4c5ba740 1 Processor -- start 2024-05-08T22:43:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2aa9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2a9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x297e730 con 0x2aa1420 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x29845b0 con 0x2aa0b00 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x286caa0 con 0x2a97be0 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 10 client.0 _ll_register_callbacks cb 0x29ba610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39718/0 (socket says 172.21.15.179:39718) 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 -- 172.21.15.179:0/1589699503 learned_addr learned my addr 172.21.15.179:0/1589699503 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2aa9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2a9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 -- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 msgr2=0x2aa9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2aa9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 -- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 msgr2=0x2a9fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2a9fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 -- 172.21.15.179:0/1589699503 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44158160 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b2880 tx=0x7f2f3c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d5995e2946a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c003e40 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f2f3c004580 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27f51e0 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3c004740 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 10 client.9366 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x2987220 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27f51e0 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2f3c019050 con 0x2aa1420 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 10 client.9366 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 10 client.9366 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 20 client.9366 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:39.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2a99450 0x2ab4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 4 client.9366 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x283f460 con 0x2a99450 2024-05-08T22:43:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.956+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2a99450 0x2ab4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f21ffb700 10 client.9366 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:39.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2a99450 0x2ab4460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c94ad9af7d0b1a32 server_cookie=39be64821e7893c1 in_seq=0 out_seq=0 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1589699503 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x283f460 con 0x2a99450 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f21ffb700 10 client.9366 handle_command_reply: tid=0 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 client.9366 shutdown 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 msgr2=0x2aa6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b2880 tx=0x7f2f3c00e000 comp rx=0 tx=0).stop 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 shutdown_connections 2024-05-08T22:43:39.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2a99450 0x2ab4460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2a9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f21ffb700 0 client.9366 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2aa1420 0x2aa6ce0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1589699503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2aa9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 >> 172.21.15.179:0/1589699503 conn(0x27e84d0 msgr2=0x2a81990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 shutdown_connections 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1589699503 wait complete. 2024-05-08T22:43:39.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.960+0000 7f2f4c5ba740 20 client.9366 trim_cache size 0 max 16384 2024-05-08T22:43:39.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 Processor -- start 2024-05-08T22:43:39.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 -- start start 2024-05-08T22:43:39.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2ab8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 0x2a54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x298b4f0 con 0x2a99450 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x29efec0 con 0x2a97be0 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x29bf940 con 0x2aa0b00 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.980+0000 7f2f427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39726/0 (socket says 172.21.15.179:39726) 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1291810132 learned_addr learned my addr 172.21.15.179:0/1291810132 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2ab8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 0x2a54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 msgr2=0x2a54700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 0x2a54700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 msgr2=0x2ab8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2ab8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1291810132 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x296d450 con 0x2a99450 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 0x2a54700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004580 tx=0x7f2f34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8181f6227086dc36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f297fa700 1 -- 172.21.15.179:0/1291810132 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3401c040 con 0x2a99450 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f297fa700 1 -- 172.21.15.179:0/1291810132 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f340045b0 con 0x2a99450 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f297fa700 1 -- 172.21.15.179:0/1291810132 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003ad0 con 0x2a99450 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1291810132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 msgr2=0x2ab89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004580 tx=0x7f2f34014000 comp rx=0 tx=0).stop 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1291810132 shutdown_connections 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2aa0b00 0x2a54700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2ab89c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1291810132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2a97be0 0x2ab8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1291810132 >> 172.21.15.179:0/1291810132 conn(0x27e84d0 msgr2=0x2ab0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1291810132 shutdown_connections 2024-05-08T22:43:39.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1291810132 wait complete. 2024-05-08T22:43:39.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- start start 2024-05-08T22:43:39.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 Processor -- start 2024-05-08T22:43:39.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2a58880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2abe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a59f00 con 0x2a99450 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a830a0 con 0x2aa0b00 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ab0c00 con 0x2a97be0 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f4c5ba740 10 client.0 _ll_register_callbacks cb 0x7f2f38054ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39736/0 (socket says 172.21.15.179:39736) 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1059692474 learned_addr learned my addr 172.21.15.179:0/1059692474 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2a58880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f437fe700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2abe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 msgr2=0x2a58880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2a58880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 msgr2=0x2abe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:39.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2abe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.984+0000 7f2f427fc700 1 -- 172.21.15.179:0/1059692474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34004200 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f427fc700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2f34003e50 tx=0x7f2f340047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14609996d2850134 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34031070 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f2f340045b0 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28641e0 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f34014650 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 10 client.9372 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x2961720 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28641e0 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2f340147b0 con 0x2a99450 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 10 client.9372 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 10 client.9372 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 20 client.9372 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:39.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2aa1420 0x2a576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 4 client.9372 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2abea80 con 0x2aa1420 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2aa1420 0x2a576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f42ffd700 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2aa1420 0x2a576b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d84c1ba1796e06 server_cookie=338db5066a995f8f in_seq=0 out_seq=0 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 10 client.9372 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 1 -- 172.21.15.179:0/1059692474 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x2abea80 con 0x2aa1420 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 10 client.9372 handle_command_reply: tid=0 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 client.9372 shutdown 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 msgr2=0x2abdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2f34003e50 tx=0x7f2f340047e0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 shutdown_connections 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x2aa1420 0x2a576b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2aa0b00 0x2abe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2a99450 0x2abdf80 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f21ffb700 0 client.9372 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 --2- 172.21.15.179:0/1059692474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2a97be0 0x2a58880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 >> 172.21.15.179:0/1059692474 conn(0x27e84d0 msgr2=0x2ab8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 shutdown_connections 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.988+0000 7f2f4c5ba740 1 -- 172.21.15.179:0/1059692474 wait complete. 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f4c5ba740 20 client.9372 trim_cache size 0 max 16384 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:43:39.998 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 57725, 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.5133495449999996, 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 81, 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 115, 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 115, 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 5.4812593270000001 2024-05-08T22:43:39.999 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 msgr2=0x7f2f30068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 0x7f2f30068c20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2f44161810 tx=0x7f2f34009240 comp rx=0 tx=0).stop 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 msgr2=0x7f2f44142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e870 tx=0x7f2f3800eb80 comp rx=0 tx=0).stop 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 shutdown_connections 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2f30066760 0x7f2f30068c20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f440ce6b0 0x7f2f4406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f44160a50 0x7f2f4406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 --2- 172.21.15.179:0/2021713105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f440ccb50 0x7f2f44142050 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 >> 172.21.15.179:0/2021713105 conn(0x7f2f440b1ed0 msgr2=0x7f2f4415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 shutdown_connections 2024-05-08T22:43:40.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:39.992+0000 7f2f43fff700 1 -- 172.21.15.179:0/2021713105 wait complete. 2024-05-08T22:43:40.011 DEBUG:teuthology.orchestra.run.smithi179:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T22:43:40.014 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-08T22:43:40.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.124+0000 7fd9e9b8e700 1 Processor -- start 2024-05-08T22:43:40.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.124+0000 7fd9e9b8e700 1 -- start start 2024-05-08T22:43:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd9e4159a30 0x7fd9e40cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e406bc20 0x7fd9e40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e40b5360 con 0x7fd9e40b0310 2024-05-08T22:43:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e406ce20 con 0x7fd9e4159a30 2024-05-08T22:43:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40b4070 con 0x7fd9e406bc20 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e8b8c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd9e40b0310 0x7fd9e40b06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45982/0 (socket says 172.21.15.179:45982) 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e8b8c700 1 -- 172.21.15.179:0/1116561061 learned_addr learned my addr 172.21.15.179:0/1116561061 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1116561061 >> v2:172.21.15.179:3300/0 conn(0x7fd9e4159a30 0x7fd9e40cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e938d700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e406bc20 0x7fd9e40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e37fe700 1 -- 172.21.15.179:0/1116561061 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402734491 0 0) 0x7fd9d4001110 con 0x7fd9e40b0310 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e37fe700 1 -- 172.21.15.179:0/1116561061 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d00035d0 con 0x7fd9e40b0310 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 -- 172.21.15.179:0/1116561061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e406bc20 msgr2=0x7fd9e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e406bc20 0x7fd9e40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 -- 172.21.15.179:0/1116561061 >> v1:172.21.15.179:6789/0 conn(0x7fd9e40b0310 legacy=0x7fd9e40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 -- 172.21.15.179:0/1116561061 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40b1df0 con 0x7fd9e4159a30 2024-05-08T22:43:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e4159a30 0x7fd9e40cd1c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00a9f0 tx=0x7fd9cc00ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6009493683ad95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e37fe700 1 -- 172.21.15.179:0/1116561061 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00cb70 con 0x7fd9e4159a30 2024-05-08T22:43:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e37fe700 1 -- 172.21.15.179:0/1116561061 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00ae90 con 0x7fd9e4159a30 2024-05-08T22:43:40.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e37fe700 1 -- 172.21.15.179:0/1116561061 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc018400 con 0x7fd9e4159a30 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1116561061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e4159a30 msgr2=0x7fd9e40cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e4159a30 0x7fd9e40cd1c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00a9f0 tx=0x7fd9cc00ad00 comp rx=0 tx=0).stop 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1116561061 shutdown_connections 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e406bc20 0x7fd9e40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1116561061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e4159a30 0x7fd9e40cd1c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1116561061 >> 172.21.15.179:0/1116561061 conn(0x7fd9e4098a70 msgr2=0x7fd9e40b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1116561061 shutdown_connections 2024-05-08T22:43:40.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.128+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1116561061 wait complete. 2024-05-08T22:43:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 Processor -- start 2024-05-08T22:43:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- start start 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e40b0310 0x7fd9e4147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 0x7fd9e40ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415c530 con 0x7fd9e406bc20 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415c810 con 0x7fd9e40b0310 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415caf0 con 0x7fd9e4159a30 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39748/0 (socket says 172.21.15.179:39748) 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 -- 172.21.15.179:0/1055977469 learned_addr learned my addr 172.21.15.179:0/1055977469 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e938d700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 0x7fd9e40ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e40b0310 0x7fd9e4147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 -- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 msgr2=0x7fd9e40ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 0x7fd9e40ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 -- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e40b0310 msgr2=0x7fd9e4147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e40b0310 0x7fd9e4147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 -- 172.21.15.179:0/1055977469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc00a650 con 0x7fd9e406bc20 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e938d700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 0x7fd9e40ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001270 tx=0x7fd9d400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e17fa700 1 -- 172.21.15.179:0/1055977469 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4005d90 con 0x7fd9e406bc20 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e17fa700 1 -- 172.21.15.179:0/1055977469 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d40070d0 con 0x7fd9e406bc20 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e415cdd0 con 0x7fd9e406bc20 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e17fa700 1 -- 172.21.15.179:0/1055977469 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4007480 con 0x7fd9e406bc20 2024-05-08T22:43:40.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.132+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e415a940 con 0x7fd9e406bc20 2024-05-08T22:43:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9e17fa700 1 -- 172.21.15.179:0/1055977469 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9d40075e0 con 0x7fd9e406bc20 2024-05-08T22:43:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9e17fa700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 0x7fd9c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9e17fa700 1 -- 172.21.15.179:0/1055977469 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd9d4083640 con 0x7fd9e406bc20 2024-05-08T22:43:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 0x7fd9c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 0x7fd9c4068c20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9ee09f740 1 Processor -- start 2024-05-08T22:43:40.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.136+0000 7fd9ee09f740 1 -- start start 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xed0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xe85720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xdf3500 con 0xed13d0 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xdf0c90 con 0xec6dd0 2024-05-08T22:43:40.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xdec580 con 0xed0ab0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39760/0 (socket says 172.21.15.179:39760) 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 -- 172.21.15.179:0/3277552249 learned_addr learned my addr 172.21.15.179:0/3277552249 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xed0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xe85720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 -- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 msgr2=0xed0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xed0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 -- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 msgr2=0xe85720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xe85720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 -- 172.21.15.179:0/3277552249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcdeee0 con 0xed13d0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c4d0 tx=0x7fd9cc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8edc15ffeb1c6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9da7fc700 1 -- 172.21.15.179:0/3277552249 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004240 con 0xed13d0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9da7fc700 1 -- 172.21.15.179:0/3277552249 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc01f040 con 0xed13d0 2024-05-08T22:43:40.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9da7fc700 1 -- 172.21.15.179:0/3277552249 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004600 con 0xed13d0 2024-05-08T22:43:40.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3277552249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 msgr2=0xe851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c4d0 tx=0x7fd9cc011000 comp rx=0 tx=0).stop 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3277552249 shutdown_connections 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xe85720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xe851e0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3277552249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xed0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3277552249 >> 172.21.15.179:0/3277552249 conn(0xc13780 msgr2=0xeae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3277552249 shutdown_connections 2024-05-08T22:43:40.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3277552249 wait complete. 2024-05-08T22:43:40.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7f591d06e700 1 Processor -- start 2024-05-08T22:43:40.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.140+0000 7f591d06e700 1 -- start start 2024-05-08T22:43:40.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- start start 2024-05-08T22:43:40.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 Processor -- start 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59180ca150 0x7f591815e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59180cddc0 con 0x7f5918157630 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5918157a10 con 0x7f59180c49c0 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591815e5a0 con 0x7f59180ca150 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f5917fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5918157630 0x7f59180ce250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45984/0 (socket says 172.21.15.179:45984) 2024-05-08T22:43:40.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f5917fff700 1 -- 172.21.15.179:0/1774964832 learned_addr learned my addr 172.21.15.179:0/1774964832 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:43:40.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f59177fe700 1 --2- 172.21.15.179:0/1774964832 >> v2:172.21.15.179:3300/0 conn(0x7f59180ca150 0x7f591815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f59177fe700 1 -- 172.21.15.179:0/1774964832 >> v1:172.21.15.195:6789/0 conn(0x7f59180c49c0 legacy=0x7f59180cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f59177fe700 1 -- 172.21.15.179:0/1774964832 >> v1:172.21.15.179:6789/0 conn(0x7f5918157630 legacy=0x7f59180ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f59177fe700 1 -- 172.21.15.179:0/1774964832 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180ca5d0 con 0x7f59180ca150 2024-05-08T22:43:40.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f59177fe700 1 --2- 172.21.15.179:0/1774964832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180ca150 0x7f591815e060 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f590c009850 tx=0x7f590c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c060fb273c90e418 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xecfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xea67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xdaf920 con 0xed13d0 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xdaf5c0 con 0xec6dd0 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xd81e10 con 0xed0ab0 2024-05-08T22:43:40.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 10 client.0 _ll_register_callbacks cb 0xde5bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39786/0 (socket says 172.21.15.179:39786) 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 -- 172.21.15.179:0/3828149318 learned_addr learned my addr 172.21.15.179:0/3828149318 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f5916ffd700 1 -- 172.21.15.179:0/1774964832 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c011830 con 0x7f59180ca150 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f5916ffd700 1 -- 172.21.15.179:0/1774964832 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c011990 con 0x7f59180ca150 2024-05-08T22:43:40.157 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xecfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f5916ffd700 1 -- 172.21.15.179:0/1774964832 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c011d20 con 0x7f59180ca150 2024-05-08T22:43:40.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xea67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 -- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 msgr2=0xea67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xea67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 -- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 msgr2=0xecfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xecfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 -- 172.21.15.179:0/3828149318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e40b1df0 con 0xed13d0 2024-05-08T22:43:40.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc013cf0 tx=0x7fd9dc009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f7d13c17b67f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- 172.21.15.179:0/1774964832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180ca150 msgr2=0x7f591815e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 --2- 172.21.15.179:0/1774964832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180ca150 0x7f591815e060 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f590c009850 tx=0x7f590c009b60 comp rx=0 tx=0).stop 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- 172.21.15.179:0/1774964832 shutdown_connections 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 --2- 172.21.15.179:0/1774964832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180ca150 0x7f591815e060 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- 172.21.15.179:0/1774964832 >> 172.21.15.179:0/1774964832 conn(0x7f59180cfb00 msgr2=0x7f59180cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00e040 con 0xed13d0 2024-05-08T22:43:40.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- 172.21.15.179:0/1774964832 shutdown_connections 2024-05-08T22:43:40.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd9dc007930 con 0xed13d0 2024-05-08T22:43:40.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc20490 con 0xed13d0 2024-05-08T22:43:40.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7f591d06e700 1 -- 172.21.15.179:0/1774964832 wait complete. 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9dc007af0 con 0xed13d0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 10 client.9393 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0xdb2600 con 0xed13d0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc20490 con 0xed13d0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9dc007c50 con 0xed13d0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 10 client.9393 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 10 client.9393 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 20 client.9393 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xea81d0 0xee4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 4 client.9393 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc6a710 con 0xea81d0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.144+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xea81d0 0xee4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9c37fe700 10 client.9393 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xea81d0 0xee4800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522f8bd1164bb850 server_cookie=ba45189db394c3ee in_seq=0 out_seq=0 2024-05-08T22:43:40.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 Processor -- start 2024-05-08T22:43:40.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- start start 2024-05-08T22:43:40.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180c49c0 0x7f591806b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180ca150 0x7f5918144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5918157630 0x7f59181446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918144be0 con 0x7f59180c49c0 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180c7d70 con 0x7f59180ca150 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59180c7160 con 0x7f5918157630 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3828149318 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xc6a710 con 0xea81d0 2024-05-08T22:43:40.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9c37fe700 10 client.9393 handle_command_reply: tid=0 2024-05-08T22:43:40.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180c49c0 0x7f591806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 client.9393 shutdown 2024-05-08T22:43:40.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180c49c0 0x7f591806b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39800/0 (socket says 172.21.15.179:39800) 2024-05-08T22:43:40.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 -- 172.21.15.179:0/980443281 learned_addr learned my addr 172.21.15.179:0/980443281 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f59177fe700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180ca150 0x7f5918144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591c86d700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5918157630 0x7f59181446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 msgr2=0xea6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc013cf0 tx=0x7fd9dc009f20 comp rx=0 tx=0).stop 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 shutdown_connections 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xea81d0 0xee4800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xea6cf0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed0ab0 0xea67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3828149318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xecfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 >> 172.21.15.179:0/3828149318 conn(0xc13780 msgr2=0xeaba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 -- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5918157630 msgr2=0x7f59181446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5918157630 0x7f59181446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 -- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180ca150 msgr2=0x7f5918144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180ca150 0x7f5918144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9c37fe700 0 client.9393 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 shutdown_connections 2024-05-08T22:43:40.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 -- 172.21.15.179:0/980443281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0094b0 con 0x7f59180c49c0 2024-05-08T22:43:40.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3828149318 wait complete. 2024-05-08T22:43:40.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5917fff700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59180c49c0 0x7f591806b690 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5908001eb0 tx=0x7f590800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7fd9ee09f740 20 client.9393 trim_cache size 0 max 16384 2024-05-08T22:43:40.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908005d90 con 0x7f59180c49c0 2024-05-08T22:43:40.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080070d0 con 0x7f59180c49c0 2024-05-08T22:43:40.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- 172.21.15.179:0/980443281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59180c7440 con 0x7f59180c49c0 2024-05-08T22:43:40.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908007440 con 0x7f59180c49c0 2024-05-08T22:43:40.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- 172.21.15.179:0/980443281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59180c7a00 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.148+0000 7f591d06e700 1 -- 172.21.15.179:0/980443281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f591804e700 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59080075a0 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f5914ff9700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5904066730 0x7f5904068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5908083530 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+447 (secure 0 0 0) 0x7f590808a070 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f59177fe700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5904066730 0x7f5904068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f591d06e700 1 -- 172.21.15.179:0/980443281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f59180c65d0 con 0x7f59180c49c0 2024-05-08T22:43:40.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 7 ==== log(2 entries from seq 1108 at 2024-05-08T22:43:38.489163+0000) v1 ==== 389+0+0 (secure 0 0 0) 0x7f5908054440 con 0x7f59180c49c0 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: cluster: 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: id: 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: health: HEALTH_OK 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: services: 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: mgr: x(active, since 10m), standbys: y 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: data: 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: volumes: 1/1 healthy 2024-05-08T22:43:40.178 INFO:teuthology.orchestra.run.smithi179.stdout: pools: 3 pools, 97 pgs 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout: objects: 30 objects, 19 MiB 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout: usage: 299 MiB used, 720 GiB / 720 GiB avail 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout: pgs: 97 active+clean 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:38.489163+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:38.489217+0000 mon.a [INF] Cluster is now healthy 2024-05-08T22:43:40.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.152+0000 7f59177fe700 1 --2- 172.21.15.179:0/980443281 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5904066730 0x7f5904068bf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f590c009fd0 tx=0x7f590c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.181 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-08T22:43:40.181 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-08T22:43:40.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.164+0000 7fd9ee09f740 1 Processor -- start 2024-05-08T22:43:40.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.164+0000 7fd9ee09f740 1 -- start start 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xee6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 0xecf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xd8a5a0 con 0xed0ab0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xec6930 con 0xec6dd0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xdac3c0 con 0xed13d0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39814/0 (socket says 172.21.15.179:39814) 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 -- 172.21.15.179:0/2801295315 learned_addr learned my addr 172.21.15.179:0/2801295315 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xee6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e938d700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 0xecf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.187 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 -- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 msgr2=0xecf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 0xecf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 -- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 msgr2=0xee6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xee6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 -- 172.21.15.179:0/2801295315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd98830 con 0xed0ab0 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e938d700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 0xecf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00ad40 tx=0x7fd9cc00a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40ca92566398928 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9daffd700 1 -- 172.21.15.179:0/2801295315 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004380 con 0xed0ab0 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9daffd700 1 -- 172.21.15.179:0/2801295315 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc004500 con 0xed0ab0 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9daffd700 1 -- 172.21.15.179:0/2801295315 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc003900 con 0xed0ab0 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- 172.21.15.179:0/2801295315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 msgr2=0xecf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00ad40 tx=0x7fd9cc00a350 comp rx=0 tx=0).stop 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- 172.21.15.179:0/2801295315 shutdown_connections 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xed13d0 0xecf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed0ab0 0xecf1d0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/2801295315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xec6dd0 0xee6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- 172.21.15.179:0/2801295315 >> 172.21.15.179:0/2801295315 conn(0xc13780 msgr2=0xee0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- 172.21.15.179:0/2801295315 shutdown_connections 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.168+0000 7fd9ee09f740 1 -- 172.21.15.179:0/2801295315 wait complete. 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- start start 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 Processor -- start 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xec6dd0 0xeef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xed0ab0 0xeee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0xef0de0 con 0xed13d0 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0xed01c0 con 0xed0ab0 2024-05-08T22:43:40.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0xed9e70 con 0xec6dd0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 10 client.0 _ll_register_callbacks cb 0xdeb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39824/0 (socket says 172.21.15.179:39824) 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 -- 172.21.15.179:0/3216842188 learned_addr learned my addr 172.21.15.179:0/3216842188 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e3fff700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xed0ab0 0xeee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xec6dd0 0xeef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 -- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xec6dd0 msgr2=0xeef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xec6dd0 0xeef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 -- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xed0ab0 msgr2=0xeee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xed0ab0 0xeee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 -- 172.21.15.179:0/3216842188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xcdeee0 con 0xed13d0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:40.190 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9e938d700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007130 tx=0x7fd9dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f85aa6bc2da5245 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00e070 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9cc01f040 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd9dc020ac0 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9dc020530 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 10 client.9402 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0xc5ec10 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.172+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc01f040 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9dc017430 con 0xed13d0 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 10 client.9402 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 10 client.9402 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 20 client.9402 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xeaea70 0xee5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.191 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 4 client.9402 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0xc915f0 con 0xeaea70 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xeaea70 0xee5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9c37fe700 10 client.9402 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9e8b8c700 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xeaea70 0xee5d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde702eebb4a4822 server_cookie=79abd61a9a3d79db in_seq=0 out_seq=0 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9c37fe700 1 -- 172.21.15.179:0/3216842188 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0xc915f0 con 0xeaea70 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9c37fe700 10 client.9402 handle_command_reply: tid=0 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 client.9402 shutdown 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 msgr2=0xee5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007130 tx=0x7fd9dc004af0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 shutdown_connections 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0xeaea70 0xee5d90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0xed0ab0 0xeee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9c37fe700 0 client.9402 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0xed13d0 0xee5850 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 --2- 172.21.15.179:0/3216842188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0xec6dd0 0xeef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 >> 172.21.15.179:0/3216842188 conn(0xc13780 msgr2=0xecf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 shutdown_connections 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 1 -- 172.21.15.179:0/3216842188 wait complete. 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.176+0000 7fd9ee09f740 20 client.9402 trim_cache size 0 max 16384 2024-05-08T22:43:40.193 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:40.194 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0, 2024-05-08T22:43:40.194 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_tag": "417bd87f-2941-4bcb-ad36-caa5fe852c5a", 2024-05-08T22:43:40.194 INFO:teuthology.orchestra.run.smithi179.stdout: "mode": "asynchronous" 2024-05-08T22:43:40.194 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:40.194 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-08T22:43:40.194 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 msgr2=0x7fd9c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 0x7fd9c4068c20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc012000 comp rx=0 tx=0).stop 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 msgr2=0x7fd9e40ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001270 tx=0x7fd9d400c9e0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 shutdown_connections 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9c4066760 0x7fd9c4068c20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e40b0310 0x7fd9e4147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e4159a30 0x7fd9e40ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 --2- 172.21.15.179:0/1055977469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9e406bc20 0x7fd9e40ae010 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 >> 172.21.15.179:0/1055977469 conn(0x7fd9e4098a70 msgr2=0x7fd9e40cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 shutdown_connections 2024-05-08T22:43:40.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.180+0000 7fd9e9b8e700 1 -- 172.21.15.179:0/1055977469 wait complete. 2024-05-08T22:43:40.199 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T22:43:40.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.316+0000 7f5c5afab700 1 Processor -- start 2024-05-08T22:43:40.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.316+0000 7f5c5afab700 1 -- start start 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5afab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5afab700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5c540b1720 0x7f5c540b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5afab700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c540caa40 con 0x7f5c540ae740 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5afab700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c54071940 con 0x7f5c540b1720 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5afab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540b4eb0 con 0x7f5c5415a030 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:45196/0 (socket says 172.21.15.179:45196) 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 -- 172.21.15.179:0/3425766733 learned_addr learned my addr 172.21.15.179:0/3425766733 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c597a8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5c540b1720 0x7f5c540b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c5a7aa700 1 --1- 172.21.15.179:0/3425766733 >> v1:172.21.15.195:6789/0 conn(0x7f5c540ae740 0x7f5c540c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:35596/0 (socket says 172.21.15.179:35596) 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c58fa7700 1 -- 172.21.15.179:0/3425766733 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655854844 0 0) 0x7f5c540caa40 con 0x7f5c540ae740 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 -- 172.21.15.179:0/3425766733 >> v2:172.21.15.195:3300/0 conn(0x7f5c540b1720 msgr2=0x7f5c540b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/3425766733 >> v2:172.21.15.195:3300/0 conn(0x7f5c540b1720 0x7f5c540b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 -- 172.21.15.179:0/3425766733 >> v1:172.21.15.195:6789/0 conn(0x7f5c540ae740 legacy=0x7f5c540c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 -- 172.21.15.179:0/3425766733 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540b5170 con 0x7f5c5415a030 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/3425766733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bbb0 tx=0x7f5c5000bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe842cd035277cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c58fa7700 1 -- 172.21.15.179:0/3425766733 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000dd30 con 0x7f5c5415a030 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c58fa7700 1 -- 172.21.15.179:0/3425766733 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5000de90 con 0x7f5c5415a030 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.320+0000 7f5c58fa7700 1 -- 172.21.15.179:0/3425766733 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50010690 con 0x7f5c5415a030 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 -- 172.21.15.179:0/3425766733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 msgr2=0x7f5c540b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 --2- 172.21.15.179:0/3425766733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bbb0 tx=0x7f5c5000bec0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 -- 172.21.15.179:0/3425766733 shutdown_connections 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 --2- 172.21.15.179:0/3425766733 >> v2:172.21.15.195:3300/0 conn(0x7f5c540b1720 0x7f5c540b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 --2- 172.21.15.179:0/3425766733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540b4170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 -- 172.21.15.179:0/3425766733 >> 172.21.15.179:0/3425766733 conn(0x7f5c54098a70 msgr2=0x7f5c540b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 -- 172.21.15.179:0/3425766733 shutdown_connections 2024-05-08T22:43:40.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.324+0000 7f5c5afab700 1 -- 172.21.15.179:0/3425766733 wait complete. 2024-05-08T22:43:40.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 Processor -- start 2024-05-08T22:43:40.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- start start 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c540ae740 0x7f5c54147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54099530 con 0x7f5c540b1720 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5406ac90 con 0x7f5c540ae740 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5406af70 con 0x7f5c5415a030 2024-05-08T22:43:40.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39840/0 (socket says 172.21.15.179:39840) 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 -- 172.21.15.179:0/2117907446 learned_addr learned my addr 172.21.15.179:0/2117907446 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c540ae740 0x7f5c54147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 -- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 msgr2=0x7f5c540cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 -- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c540ae740 msgr2=0x7f5c54147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c540ae740 0x7f5c54147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 -- 172.21.15.179:0/2117907446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5000b810 con 0x7f5c540b1720 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c597a8700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9a0 tx=0x7f5c4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c4a7fc700 1 -- 172.21.15.179:0/2117907446 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44009720 con 0x7f5c540b1720 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c4a7fc700 1 -- 172.21.15.179:0/2117907446 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4400c690 con 0x7f5c540b1720 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5406b250 con 0x7f5c540b1720 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c4a7fc700 1 -- 172.21.15.179:0/2117907446 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400ca40 con 0x7f5c540b1720 2024-05-08T22:43:40.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.328+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541483c0 con 0x7f5c540b1720 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c4a7fc700 1 -- 172.21.15.179:0/2117907446 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c4400cba0 con 0x7f5c540b1720 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c4a7fc700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 0x7f5c38068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c4a7fc700 1 -- 172.21.15.179:0/2117907446 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5c440842c0 con 0x7f5c540b1720 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 0x7f5c38068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 Processor -- start 2024-05-08T22:43:40.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 -- start start 2024-05-08T22:43:40.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 0x7f5c38068c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bfd0 tx=0x7f5c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x124e010 con 0x12bb840 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x11dbf40 con 0x12e85d0 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x122ee30 con 0x12e56f0 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c59fa9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.332+0000 7f5c59fa9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39854/0 (socket says 172.21.15.179:39854) 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 -- 172.21.15.179:0/1939803467 learned_addr learned my addr 172.21.15.179:0/1939803467 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c597a8700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 -- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 msgr2=0x12bbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12bbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 -- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 msgr2=0x12bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12bc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 -- 172.21.15.179:0/1939803467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x109aa40 con 0x12bb840 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c597a8700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5c540b4ff0 tx=0x7f5c50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d96107fb7e2bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c377fe700 1 -- 172.21.15.179:0/1939803467 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50004340 con 0x12bb840 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c377fe700 1 -- 172.21.15.179:0/1939803467 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5002a040 con 0x12bb840 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c377fe700 1 -- 172.21.15.179:0/1939803467 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50010840 con 0x12bb840 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/1939803467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 msgr2=0x12c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5c540b4ff0 tx=0x7f5c50012000 comp rx=0 tx=0).stop 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/1939803467 shutdown_connections 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/1939803467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12bb840 0x12c0d90 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/1939803467 >> 172.21.15.179:0/1939803467 conn(0x104f5b0 msgr2=0x12e7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/1939803467 shutdown_connections 2024-05-08T22:43:40.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/1939803467 wait complete. 2024-05-08T22:43:40.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- start start 2024-05-08T22:43:40.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 Processor -- start 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x1310590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x122a150 con 0x12e85d0 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1227eb0 con 0x12bb840 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x11e58e0 con 0x12e56f0 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5f4bc740 10 client.0 _ll_register_callbacks cb 0x11c4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39856/0 (socket says 172.21.15.179:39856) 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 -- 172.21.15.179:0/2981437995 learned_addr learned my addr 172.21.15.179:0/2981437995 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x1310590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c597a8700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 -- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 msgr2=0x12ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 -- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 msgr2=0x1310590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x1310590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c5a7aa700 1 -- 172.21.15.179:0/2981437995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c540b5170 con 0x12e85d0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.336+0000 7f5c597a8700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c007b40 tx=0x7f5c4c011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130a6e3257680748 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0020b0 con 0x12e85d0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5c4c011d50 con 0x12e85d0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x105c2c0 con 0x12e85d0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4c015470 con 0x12e85d0 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 10 client.9411 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:40.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x11ee290 con 0x12e85d0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x105c2c0 con 0x12e85d0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5c4c0155d0 con 0x12e85d0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 10 client.9411 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 10 client.9411 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 20 client.9411 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x12be980 0x12bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 4 client.9411 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10a6540 con 0x12be980 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x12be980 0x12bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 10 client.9411 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x12be980 0x12bed90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e1ecab456b9a27 server_cookie=918a498cd59eca97 in_seq=0 out_seq=0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 1 -- 172.21.15.179:0/2981437995 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x10a6540 con 0x12be980 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 10 client.9411 handle_command_reply: tid=0 2024-05-08T22:43:40.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 client.9411 shutdown 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 msgr2=0x12ebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c007b40 tx=0x7f5c4c011530 comp rx=0 tx=0).stop 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 shutdown_connections 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x12be980 0x12bed90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e85d0 0x12ebf80 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e56f0 0x12ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c34ff9700 0 client.9411 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/2981437995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x1310590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 >> 172.21.15.179:0/2981437995 conn(0x104f5b0 msgr2=0x12e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.340+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 shutdown_connections 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.344+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/2981437995 wait complete. 2024-05-08T22:43:40.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.344+0000 7f5c5f4bc740 20 client.9411 trim_cache size 0 max 16384 2024-05-08T22:43:40.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 Processor -- start 2024-05-08T22:43:40.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 -- start start 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12bb840 0x12badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12e3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d3b80 con 0x12e56f0 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x11e82d0 con 0x12e85d0 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12eb350 con 0x12bb840 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39860/0 (socket says 172.21.15.179:39860) 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c597a8700 1 -- 172.21.15.179:0/511131946 learned_addr learned my addr 172.21.15.179:0/511131946 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.360+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12bb840 0x12badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12bb840 msgr2=0x12badc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12bb840 0x12badc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 msgr2=0x12e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12e3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/511131946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11d44c0 con 0x12e56f0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5c440550b0 tx=0x7f5c4405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3540b4a3815dcb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c37fff700 1 -- 172.21.15.179:0/511131946 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44057070 con 0x12e56f0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c37fff700 1 -- 172.21.15.179:0/511131946 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44058f50 con 0x12e56f0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c37fff700 1 -- 172.21.15.179:0/511131946 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4405d510 con 0x12e56f0 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/511131946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 msgr2=0x12bb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5c440550b0 tx=0x7f5c4405b040 comp rx=0 tx=0).stop 2024-05-08T22:43:40.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/511131946 shutdown_connections 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12e85d0 0x12e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x12bb300 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/511131946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12bb840 0x12badc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/511131946 >> 172.21.15.179:0/511131946 conn(0x104f5b0 msgr2=0x12e95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/511131946 shutdown_connections 2024-05-08T22:43:40.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/511131946 wait complete. 2024-05-08T22:43:40.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- start start 2024-05-08T22:43:40.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 Processor -- start 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x131ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e85d0 0x12bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x11f8c00 con 0x12e56f0 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x12ec120 con 0x12bb840 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x12e8d50 con 0x12e85d0 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5f4bc740 10 client.0 _ll_register_callbacks cb 0x11e1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39862/0 (socket says 172.21.15.179:39862) 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/3022421092 learned_addr learned my addr 172.21.15.179:0/3022421092 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c5a7aa700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e85d0 0x12bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x131ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e85d0 msgr2=0x12bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e85d0 0x12bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 msgr2=0x131ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x131ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.364+0000 7f5c597a8700 1 -- 172.21.15.179:0/3022421092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c44059e70 con 0x12e56f0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c597a8700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006b40 tx=0x7f5c4401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ee1813cf2586da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44057070 con 0x12e56f0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5c4405d510 con 0x12e56f0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c5002a040 con 0x12e56f0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c44082300 con 0x12e56f0 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 10 client.9417 fetch_fsmap learned FSMap version 81 2024-05-08T22:43:40.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x11c8790 con 0x12e56f0 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44079080 con 0x12e56f0 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5c44055380 con 0x12e56f0 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 10 client.9417 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 10 client.9417 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 20 client.9417 populate_metadata read hostname 'smithi179' 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x1370820 0x12bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 4 client.9417 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x10cb2c0 con 0x1370820 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x1370820 0x12bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c59fa9700 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x1370820 0x12bf2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ccc111a7fd65dd server_cookie=fbe980dfce75b549 in_seq=0 out_seq=0 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 10 client.9417 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 1 -- 172.21.15.179:0/3022421092 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x10cb2c0 con 0x1370820 2024-05-08T22:43:40.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 10 client.9417 handle_command_reply: tid=0 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 client.9417 shutdown 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 msgr2=0x131b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006b40 tx=0x7f5c4401e070 comp rx=0 tx=0).stop 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 shutdown_connections 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x1370820 0x12bf2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x12e85d0 0x12bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x12e56f0 0x131b2c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 --2- 172.21.15.179:0/3022421092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x12bb840 0x131ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 >> 172.21.15.179:0/3022421092 conn(0x104f5b0 msgr2=0x1317400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c34ff9700 0 client.9417 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 shutdown_connections 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 1 -- 172.21.15.179:0/3022421092 wait complete. 2024-05-08T22:43:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5f4bc740 20 client.9417 trim_cache size 0 max 16384 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": { 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "417bd87f-2941-4bcb-ad36-caa5fe852c5a": { 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "path": "/", 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "tag": "417bd87f-2941-4bcb-ad36-caa5fe852c5a", 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: "options": "recursive,repair" 2024-05-08T22:43:40.378 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:43:40.379 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:43:40.379 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 msgr2=0x7f5c38068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 0x7f5c38068c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bfd0 tx=0x7f5c50008040 comp rx=0 tx=0).stop 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 msgr2=0x7f5c5406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.368+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9a0 tx=0x7f5c4400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 shutdown_connections 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5c38066760 0x7f5c38068c20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c5415a030 0x7f5c540cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c540b1720 0x7f5c5406a320 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 --2- 172.21.15.179:0/2117907446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c540ae740 0x7f5c54147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:43:40.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 >> 172.21.15.179:0/2117907446 conn(0x7f5c54098a70 msgr2=0x7f5c54158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:43:40.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 shutdown_connections 2024-05-08T22:43:40.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:40.372+0000 7f5c5afab700 1 -- 172.21.15.179:0/2117907446 wait complete. 2024-05-08T22:43:40.389 INFO:tasks.cephfs.filesystem:scrub status for tag:417bd87f-2941-4bcb-ad36-caa5fe852c5a - {'path': '/', 'tag': '417bd87f-2941-4bcb-ad36-caa5fe852c5a', 'options': 'recursive,repair'} 2024-05-08T22:43:41.481 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:43:41.476+0000 7f20dd71d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:43:41.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:41.504+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-08T22:43:40.179386+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f59080581c0 con 0x7f59180c49c0 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.179386+0000 mds.d [INF] scrub queued for path: / 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.179393+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.179403+0000 mds.d [INF] scrub summary: active paths [/] 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.180562+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.180582+0000 mds.d [ERR] inode table repaired for inode: 0x10000000001 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.181069+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-08T22:43:41.510 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.181085+0000 mds.d [ERR] inode table repaired for inode: 0x10000000002 2024-05-08T22:43:41.511 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:40.181728+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-05-08T22:43:45.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:43:45.668+0000 7f5914ff9700 1 -- 172.21.15.179:0/980443281 <== mon.0 v2:172.21.15.179:3300/0 9 ==== log(1 entries from seq 9 at 2024-05-08T22:43:44.513876+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f59080819a0 con 0x7f59180c49c0 2024-05-08T22:43:45.672 INFO:teuthology.orchestra.run.smithi179.stdout:2024-05-08T22:43:44.513876+0000 mds.d [INF] scrub summary: idle 2024-05-08T22:43:52.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:43:52.883 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:44:10.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T22:44:10.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 Processor -- start 2024-05-08T22:44:10.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- start start 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88a41448b0 0x7f88a40ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40ce650 0x7f88a40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a40cbfd0 con 0x7f88a415c0b0 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a40cd970 con 0x7f88a40ce650 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a40cb3c0 con 0x7f88a41448b0 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40ce650 0x7f88a40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88a40ce650 0x7f88a40cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54504/0 (socket says 172.21.15.179:54504) 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 -- 172.21.15.179:0/3319893336 learned_addr learned my addr 172.21.15.179:0/3319893336 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88aa12d700 1 --2- 172.21.15.179:0/3319893336 >> v2:172.21.15.195:3300/0 conn(0x7f88a41448b0 0x7f88a40ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 -- 172.21.15.179:0/3319893336 >> v2:172.21.15.195:3300/0 conn(0x7f88a41448b0 msgr2=0x7f88a40ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 --2- 172.21.15.179:0/3319893336 >> v2:172.21.15.195:3300/0 conn(0x7f88a41448b0 0x7f88a40ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 -- 172.21.15.179:0/3319893336 >> v1:172.21.15.195:6789/0 conn(0x7f88a415c0b0 legacy=0x7f88a406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 -- 172.21.15.179:0/3319893336 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40cb6a0 con 0x7f88a40ce650 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a992c700 1 --2- 172.21.15.179:0/3319893336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a40cc460 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c720 tx=0x7f8898009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac57f31752d40944 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a912b700 1 -- 172.21.15.179:0/3319893336 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889800b8a0 con 0x7f88a40ce650 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a912b700 1 -- 172.21.15.179:0/3319893336 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889800ba00 con 0x7f88a40ce650 2024-05-08T22:44:10.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88a912b700 1 -- 172.21.15.179:0/3319893336 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889800bd90 con 0x7f88a40ce650 2024-05-08T22:44:10.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- 172.21.15.179:0/3319893336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 msgr2=0x7f88a40cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3319893336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a40cc460 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c720 tx=0x7f8898009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- 172.21.15.179:0/3319893336 shutdown_connections 2024-05-08T22:44:10.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3319893336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a40cc460 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3319893336 >> v2:172.21.15.195:3300/0 conn(0x7f88a41448b0 0x7f88a40ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- 172.21.15.179:0/3319893336 >> 172.21.15.179:0/3319893336 conn(0x7f88a4070d80 msgr2=0x7f88a4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- 172.21.15.179:0/3319893336 shutdown_connections 2024-05-08T22:44:10.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.516+0000 7f88ab12f700 1 -- 172.21.15.179:0/3319893336 wait complete. 2024-05-08T22:44:10.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 Processor -- start 2024-05-08T22:44:10.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- start start 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 0x7f88a406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 0x7f88a40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a415b9e0 con 0x7f88a40ce650 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a415a940 con 0x7f88a415c0b0 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a415ac20 con 0x7f88a41448b0 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54518/0 (socket says 172.21.15.179:54518) 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 -- 172.21.15.179:0/3772229248 learned_addr learned my addr 172.21.15.179:0/3772229248 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa92e700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 0x7f88a40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88a992c700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 0x7f88a406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.526 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 -- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 msgr2=0x7f88a406d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 0x7f88a406d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 -- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 msgr2=0x7f88a40ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 0x7f88a40ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 -- 172.21.15.179:0/3772229248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898009570 con 0x7f88a40ce650 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa92e700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 0x7f88a40ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88a992c700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 0x7f88a406d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88aa12d700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f88a000e760 tx=0x7f88a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f8896ffd700 1 -- 172.21.15.179:0/3772229248 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0013610 con 0x7f88a40ce650 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a415af00 con 0x7f88a40ce650 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a415b4c0 con 0x7f88a40ce650 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f8896ffd700 1 -- 172.21.15.179:0/3772229248 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0013770 con 0x7f88a40ce650 2024-05-08T22:44:10.527 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.520+0000 7f8896ffd700 1 -- 172.21.15.179:0/3772229248 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0013ae0 con 0x7f88a40ce650 2024-05-08T22:44:10.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.524+0000 7f8896ffd700 1 -- 172.21.15.179:0/3772229248 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88a0013c40 con 0x7f88a40ce650 2024-05-08T22:44:10.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.524+0000 7f8896ffd700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 0x7f8888068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.524+0000 7f8896ffd700 1 -- 172.21.15.179:0/3772229248 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f88a00840d0 con 0x7f88a40ce650 2024-05-08T22:44:10.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.524+0000 7f88a992c700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 0x7f8888068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.530 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88a992c700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 0x7f8888068bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c720 tx=0x7f8898008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 Processor -- start 2024-05-08T22:44:10.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 -- start start 2024-05-08T22:44:10.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2835010 con 0x28cf5d0 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x27c2f40 con 0x28cc6f0 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2815e30 con 0x28a2840 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54522/0 (socket says 172.21.15.179:54522) 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 -- 172.21.15.179:0/1460505108 learned_addr learned my addr 172.21.15.179:0/1460505108 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa12d700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88a992c700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 -- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 msgr2=0x28a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 -- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 msgr2=0x28a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 -- 172.21.15.179:0/1460505108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2681a40 con 0x28cf5d0 2024-05-08T22:44:10.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88a992c700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:10.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f889c002080 tx=0x7f889c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b506402087d87f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f8887fff700 1 -- 172.21.15.179:0/1460505108 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c00b070 con 0x28cf5d0 2024-05-08T22:44:10.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f8887fff700 1 -- 172.21.15.179:0/1460505108 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c003e50 con 0x28cf5d0 2024-05-08T22:44:10.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.528+0000 7f8887fff700 1 -- 172.21.15.179:0/1460505108 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c00dbd0 con 0x28cf5d0 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- 172.21.15.179:0/1460505108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 msgr2=0x28a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f889c002080 tx=0x7f889c00d430 comp rx=0 tx=0).stop 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- 172.21.15.179:0/1460505108 shutdown_connections 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cf5d0 0x28a3810 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- 172.21.15.179:0/1460505108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- 172.21.15.179:0/1460505108 >> 172.21.15.179:0/1460505108 conn(0x26365b0 msgr2=0x28ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- 172.21.15.179:0/1460505108 shutdown_connections 2024-05-08T22:44:10.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- 172.21.15.179:0/1460505108 wait complete. 2024-05-08T22:44:10.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- start start 2024-05-08T22:44:10.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 Processor -- start 2024-05-08T22:44:10.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cf5d0 0x28d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2811150 con 0x28cc6f0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x280eeb0 con 0x28cf5d0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x27cc8e0 con 0x28a2840 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88af640740 10 client.0 _ll_register_callbacks cb 0x27ab670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54532/0 (socket says 172.21.15.179:54532) 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 -- 172.21.15.179:0/877450117 learned_addr learned my addr 172.21.15.179:0/877450117 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88aa12d700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88aa92e700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cf5d0 0x28d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 -- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 msgr2=0x28f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 -- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cf5d0 msgr2=0x28d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cf5d0 0x28d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.532+0000 7f88a992c700 1 -- 172.21.15.179:0/877450117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a40cbfd0 con 0x28cc6f0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88a992c700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f88a406cd30 tx=0x7f88980047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806826e96a06c99b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898004600 con 0x28cc6f0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f8898032470 con 0x28cc6f0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26432c0 con 0x28cc6f0 2024-05-08T22:44:10.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8898032630 con 0x28cc6f0 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 10 client.9429 fetch_fsmap learned FSMap version 81 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x27d5290 con 0x28cc6f0 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898039070 con 0x28cc6f0 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8898032790 con 0x28cc6f0 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 10 client.9429 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 10 client.9429 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 20 client.9429 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28a5980 0x28a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 4 client.9429 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x268d540 con 0x28a5980 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88aa12d700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28a5980 0x28a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88aa12d700 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28a5980 0x28a5d90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e864fe96dd1cd1 server_cookie=1def91393e90c77b in_seq=0 out_seq=0 2024-05-08T22:44:10.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 10 client.9429 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 1 -- 172.21.15.179:0/877450117 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x268d540 con 0x28a5980 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88857fa700 10 client.9429 handle_command_reply: tid=0 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.536+0000 7f88af640740 1 client.9429 shutdown 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 msgr2=0x28d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f88a406cd30 tx=0x7f88980047e0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 shutdown_connections 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28a5980 0x28a5d90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cf5d0 0x28d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88857fa700 0 client.9429 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x28d0fe0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 --2- 172.21.15.179:0/877450117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 >> 172.21.15.179:0/877450117 conn(0x26365b0 msgr2=0x28cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 shutdown_connections 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 1 -- 172.21.15.179:0/877450117 wait complete. 2024-05-08T22:44:10.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.540+0000 7f88af640740 20 client.9429 trim_cache size 0 max 16384 2024-05-08T22:44:10.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 Processor -- start 2024-05-08T22:44:10.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 -- start start 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x26bab80 con 0x28a5980 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b3bd0 con 0x28cc6f0 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x280e6c0 con 0x28a2840 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88a992c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88a992c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54544/0 (socket says 172.21.15.179:54544) 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88a992c700 1 -- 172.21.15.179:0/3313535955 learned_addr learned my addr 172.21.15.179:0/3313535955 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88aa92e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.560+0000 7f88aa12d700 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 -- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 msgr2=0x28fdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28fdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 -- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 msgr2=0x28cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 -- 172.21.15.179:0/3313535955 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x289ca10 con 0x28a5980 2024-05-08T22:44:10.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88a992c700 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f88980039a0 tx=0x7f889802b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62c769cc9901cf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f8894dae700 1 -- 172.21.15.179:0/3313535955 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898032d70 con 0x28a5980 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f8894dae700 1 -- 172.21.15.179:0/3313535955 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898039040 con 0x28a5980 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f8894dae700 1 -- 172.21.15.179:0/3313535955 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898032470 con 0x28a5980 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- 172.21.15.179:0/3313535955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 msgr2=0x28ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f88980039a0 tx=0x7f889802b070 comp rx=0 tx=0).stop 2024-05-08T22:44:10.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- 172.21.15.179:0/3313535955 shutdown_connections 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28cc6f0 0x28cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28a5980 0x28ca710 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- 172.21.15.179:0/3313535955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a2840 0x28fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- 172.21.15.179:0/3313535955 >> 172.21.15.179:0/3313535955 conn(0x26365b0 msgr2=0x28f3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- 172.21.15.179:0/3313535955 shutdown_connections 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- 172.21.15.179:0/3313535955 wait complete. 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- start start 2024-05-08T22:44:10.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 Processor -- start 2024-05-08T22:44:10.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a5980 0x2951750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x290ddd0 con 0x28cc6f0 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2952cd0 con 0x28a2840 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x28f6ed0 con 0x28a5980 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88af640740 10 client.0 _ll_register_callbacks cb 0x27c8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88aa12d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88aa92e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88aa12d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:35364/0 (socket says 172.21.15.179:35364) 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88aa92e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54556/0 (socket says 172.21.15.179:54556) 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.564+0000 7f88aa12d700 1 -- 172.21.15.179:0/1750925570 learned_addr learned my addr 172.21.15.179:0/1750925570 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88a992c700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a5980 0x2951750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 -- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a5980 msgr2=0x2951750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a5980 0x2951750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 -- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 msgr2=0x2951210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 -- 172.21.15.179:0/1750925570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x27d5290 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa12d700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa92e700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f889c002050 tx=0x7f889c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caba979c158dabd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c024070 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f889c003c50 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f889c003e50 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c0053b0 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 10 client.9435 fetch_fsmap learned FSMap version 81 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x27bb4c0 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c003e50 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 81) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f889c005510 con 0x28cc6f0 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 10 client.9435 fetch_fsmap finished waiting for FSMap version 81 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 10 client.9435 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-05-08T22:44:10.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 20 client.9435 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28cf5d0 0x294f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 4 client.9435 mds_command: new command op to 57725 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 --> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x26b22c0 con 0x28cf5d0 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa12d700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28cf5d0 0x294f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 10 client.9435 ms_handle_connect on v2:172.21.15.195:6836/602724347 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88aa12d700 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28cf5d0 0x294f880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bbd577e8881b63b server_cookie=38851b39f3694a0a in_seq=0 out_seq=0 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 1 -- 172.21.15.179:0/1750925570 <== mds.0 v2:172.21.15.195:6836/602724347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x26b22c0 con 0x28cf5d0 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88857fa700 10 client.9435 handle_command_reply: tid=0 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.568+0000 7f88af640740 1 client.9435 shutdown 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 msgr2=0x2951c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f889c002050 tx=0x7f889c001250 comp rx=0 tx=0).stop 2024-05-08T22:44:10.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 shutdown_connections 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:6836/602724347,v1:172.21.15.195:6837/602724347] conn(0x28cf5d0 0x294f880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x28cc6f0 0x2951c90 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x28a5980 0x2951750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88857fa700 0 client.9435 ms_handle_reset on v2:172.21.15.195:6836/602724347 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 --2- 172.21.15.179:0/1750925570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x28a2840 0x2951210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 >> 172.21.15.179:0/1750925570 conn(0x26365b0 msgr2=0x28f3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 shutdown_connections 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 1 -- 172.21.15.179:0/1750925570 wait complete. 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88af640740 20 client.9435 trim_cache size 0 max 16384 2024-05-08T22:44:10.579 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:10.580 INFO:teuthology.orchestra.run.smithi179.stdout: "status": "no active scrubs running", 2024-05-08T22:44:10.580 INFO:teuthology.orchestra.run.smithi179.stdout: "scrubs": {} 2024-05-08T22:44:10.580 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 msgr2=0x7f8888068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 0x7f8888068bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c720 tx=0x7f8898008030 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 msgr2=0x7f88a406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f88a000e760 tx=0x7f88a000eb20 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 shutdown_connections 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8888066730 0x7f8888068bf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a415c0b0 0x7f88a40ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88a41448b0 0x7f88a406d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 --2- 172.21.15.179:0/3772229248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a40ce650 0x7f88a406d310 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 >> 172.21.15.179:0/3772229248 conn(0x7f88a4070d80 msgr2=0x7f88a415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 shutdown_connections 2024-05-08T22:44:10.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.572+0000 7f88ab12f700 1 -- 172.21.15.179:0/3772229248 wait complete. 2024-05-08T22:44:10.595 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T22:44:10.696 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:44:10.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.812+0000 7f9b20a1d700 1 Processor -- start 2024-05-08T22:44:10.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.812+0000 7f9b20a1d700 1 -- start start 2024-05-08T22:44:10.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0ad9d0 0x7f9b1c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c15f350 con 0x7f9b1c0b2fb0 2024-05-08T22:44:10.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0cc9e0 con 0x7f9b1c15eef0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b37a0 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54558/0 (socket says 172.21.15.179:54558) 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9b1c0b2fb0 0x7f9b1c15b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37160/0 (socket says 172.21.15.179:37160) 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 -- 172.21.15.179:0/455035290 learned_addr learned my addr 172.21.15.179:0/455035290 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1b7fe700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0ad9d0 0x7f9b1c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 -- 172.21.15.179:0/455035290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0ad9d0 msgr2=0x7f9b1c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0ad9d0 0x7f9b1c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 -- 172.21.15.179:0/455035290 >> v1:172.21.15.179:6789/0 conn(0x7f9b1c0b2fb0 legacy=0x7f9b1c15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1a7fc700 1 -- 172.21.15.179:0/455035290 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719167431 0 0) 0x7f9b1c15f350 con 0x7f9b1c0b2fb0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 -- 172.21.15.179:0/455035290 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0b3a80 con 0x7f9b1c15eef0 2024-05-08T22:44:10.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1affd700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009e50 tx=0x7f9b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8539c313118770b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1a7fc700 1 -- 172.21.15.179:0/455035290 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c00e070 con 0x7f9b1c15eef0 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1a7fc700 1 -- 172.21.15.179:0/455035290 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c00b910 con 0x7f9b1c15eef0 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b1a7fc700 1 -- 172.21.15.179:0/455035290 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c00bcc0 con 0x7f9b1c15eef0 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- 172.21.15.179:0/455035290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c15eef0 msgr2=0x7f9b1c0b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009e50 tx=0x7f9b0c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- 172.21.15.179:0/455035290 shutdown_connections 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c0b2a70 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/455035290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0ad9d0 0x7f9b1c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- 172.21.15.179:0/455035290 >> 172.21.15.179:0/455035290 conn(0x7f9b1c0ce960 msgr2=0x7f9b1c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- 172.21.15.179:0/455035290 shutdown_connections 2024-05-08T22:44:10.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.816+0000 7f9b20a1d700 1 -- 172.21.15.179:0/455035290 wait complete. 2024-05-08T22:44:10.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 Processor -- start 2024-05-08T22:44:10.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 -- start start 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0b2fb0 0x7f9b1c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c06da10 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c06e110 con 0x7f9b1c15eef0 2024-05-08T22:44:10.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c156170 con 0x7f9b1c0b2fb0 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54560/0 (socket says 172.21.15.179:54560) 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 -- 172.21.15.179:0/3296500747 learned_addr learned my addr 172.21.15.179:0/3296500747 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1affd700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0b2fb0 0x7f9b1c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1bfff700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 -- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0b2fb0 msgr2=0x7f9b1c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0b2fb0 0x7f9b1c156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 -- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 msgr2=0x7f9b1c06dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c06dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 -- 172.21.15.179:0/3296500747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c009b30 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1bfff700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c06dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:10.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b1b7fe700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400ebe0 tx=0x7f9b04009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04009740 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0400c8b0 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c06e340 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.820+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400cc40 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c155e00 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b04013470 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c067360 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b13fff700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:10.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f9b040844b0 con 0x7f9b1c0ad9d0 2024-05-08T22:44:10.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b1affd700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:10.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.824+0000 7f9b1affd700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 0x7f9afc068bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c1f60f0 tx=0x7f9b0c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:10.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:10.832+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9b04051640 con 0x7f9b1c0ad9d0 2024-05-08T22:44:11.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.044+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9b1c04e700 con 0x7f9b1c0ad9d0 2024-05-08T22:44:11.051 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:44:11.048+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:44:11.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b13fff700 1 -- 172.21.15.179:0/3296500747 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9b04054c60 con 0x7f9b1c0ad9d0 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 msgr2=0x7f9afc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 0x7f9afc068bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c1f60f0 tx=0x7f9b0c009000 comp rx=0 tx=0).stop 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 msgr2=0x7f9b1c1405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400ebe0 tx=0x7f9b04009e00 comp rx=0 tx=0).stop 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 shutdown_connections 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b1c15eef0 0x7f9b1c06dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b1c0b2fb0 0x7f9b1c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 --2- 172.21.15.179:0/3296500747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b1c0ad9d0 0x7f9b1c1405d0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 >> 172.21.15.179:0/3296500747 conn(0x7f9b1c0ce960 msgr2=0x7f9b1c0cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 shutdown_connections 2024-05-08T22:44:11.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.060+0000 7f9b20a1d700 1 -- 172.21.15.179:0/3296500747 wait complete. 2024-05-08T22:44:11.068 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:44:11.069 INFO:tasks.ceph.mds.d.smithi195.stderr: -1012> 2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-08T22:44:11.069 INFO:tasks.ceph.mds.d.smithi195.stderr: -1009> 2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-08T22:44:11.069 INFO:tasks.ceph.mds.d.smithi195.stderr: -983> 2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-08T22:44:11.069 INFO:tasks.ceph.mds.d.smithi195.stderr: -980> 2024-05-08T22:43:40.176+0000 7f20d9f16700 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-08T22:44:11.069 INFO:tasks.ceph.mds.d.smithi195.stderr: -827> 2024-05-08T22:43:41.476+0000 7f20dd71d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:44:11.081 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-08T22:44:11.098 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:44:11.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- start start 2024-05-08T22:44:11.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 Processor -- start 2024-05-08T22:44:11.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 0x555f53a9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f539f9000 0x555f53a9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a2fe00 con 0x555f539f9800 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7aa00 con 0x555f539f9000 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a640 con 0x555f539f9400 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c70fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 0x555f53a9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54566/0 (socket says 172.21.15.179:54566) 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 -- 172.21.15.179:0/1682097793 learned_addr learned my addr 172.21.15.179:0/1682097793 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f539f9000 0x555f53a9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 -- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 msgr2=0x555f53a9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 0x555f53a9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 -- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f539f9000 msgr2=0x555f53a9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f539f9000 0x555f53a9f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 -- 172.21.15.179:0/1682097793 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f53a1a9c0 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c70fc700 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 0x555f53a9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x555f53b58d50 tx=0x555f5487c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f991e1e3cb8e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c50f8700 1 -- 172.21.15.179:0/1682097793 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a29600 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c50f8700 1 -- 172.21.15.179:0/1682097793 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f53b5ea80 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c50f8700 1 -- 172.21.15.179:0/1682097793 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a291e0 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f53a1ab60 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555f53a1a340 con 0x555f539f9800 2024-05-08T22:44:11.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 4 waiting for MDS map... 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c50f8700 1 -- 172.21.15.179:0/1682097793 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x555f53a32c40 con 0x555f539f9800 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c50f8700 1 -- 172.21.15.179:0/1682097793 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 82) v1 ==== 2573+0+0 (secure 0 0 0) 0x555f5487e900 con 0x555f539f9800 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 4 Got MDS map 82 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 10 main: main 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 4 main: connecting to RADOS... 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.116+0000 7fe1c9143500 1 Processor -- start 2024-05-08T22:44:11.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 -- start start 2024-05-08T22:44:11.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a500 con 0x555f53cf8c00 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a3c0 con 0x555f53cf9000 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7aa00 con 0x555f53cf8800 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54570/0 (socket says 172.21.15.179:54570) 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 -- 172.21.15.179:0/568749875 learned_addr learned my addr 172.21.15.179:0/568749875 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c70fc700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 -- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 msgr2=0x555f53a9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53a9f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 -- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 msgr2=0x555f53aa0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 -- 172.21.15.179:0/568749875 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f53a1a000 con 0x555f53cf8c00 2024-05-08T22:44:11.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53a9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:11.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x555f53d58090 tx=0x555f5487c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc8de9a7bfccab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c30f4700 1 -- 172.21.15.179:0/568749875 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a28f20 con 0x555f53cf8c00 2024-05-08T22:44:11.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c30f4700 1 -- 172.21.15.179:0/568749875 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f53b5f500 con 0x555f53cf8c00 2024-05-08T22:44:11.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.120+0000 7fe1c30f4700 1 -- 172.21.15.179:0/568749875 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a29600 con 0x555f53cf8c00 2024-05-08T22:44:11.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- 172.21.15.179:0/568749875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 msgr2=0x555f53a9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x555f53d58090 tx=0x555f5487c060 comp rx=0 tx=0).stop 2024-05-08T22:44:11.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- 172.21.15.179:0/568749875 shutdown_connections 2024-05-08T22:44:11.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53a9fb80 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- 172.21.15.179:0/568749875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53a9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- 172.21.15.179:0/568749875 >> 172.21.15.179:0/568749875 conn(0x555f539f8c00 msgr2=0x555f53ae9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- 172.21.15.179:0/568749875 shutdown_connections 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- 172.21.15.179:0/568749875 wait complete. 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 Processor -- start 2024-05-08T22:44:11.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- start start 2024-05-08T22:44:11.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53aa1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53aa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a280 con 0x555f53cf8c00 2024-05-08T22:44:11.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a140 con 0x555f53cf9000 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c9143500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f53a7a000 con 0x555f53cf8800 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53aa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c78fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53aa0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54582/0 (socket says 172.21.15.179:54582) 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c78fd700 1 -- 172.21.15.179:0/30339522 learned_addr learned my addr 172.21.15.179:0/30339522 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53aa1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.124+0000 7fe1c70fc700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 -- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 msgr2=0x555f53aa1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53aa1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 -- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 msgr2=0x555f53aa0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f53cf9000 0x555f53aa0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 -- 172.21.15.179:0/30339522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f53a1a9c0 con 0x555f53cf8c00 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f53cf8800 0x555f53aa1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c78fd700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53aa0c00 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x555f53d58300 tx=0x555f53d0cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 -- 172.21.15.179:0/30339522 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a291e0 con 0x555f53cf8c00 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 -- 172.21.15.179:0/30339522 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f53b5f680 con 0x555f53cf8c00 2024-05-08T22:44:11.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f53a1ab60 con 0x555f53cf8c00 2024-05-08T22:44:11.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 -- 172.21.15.179:0/30339522 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555f53a28dc0 con 0x555f53cf8c00 2024-05-08T22:44:11.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f53a1a340 con 0x555f53cf8c00 2024-05-08T22:44:11.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 4 main: resolving pool 15 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 -- 172.21.15.179:0/30339522 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x555f53a58840 con 0x555f53cf8c00 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x555f53d6a800 0x555f53aa1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c18f1700 1 -- 172.21.15.179:0/30339522 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x555f53a32a80 con 0x555f53cf8c00 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 4 main: creating IoCtx.. 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555f53cf9400 0x555f53aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) v8 -- 0x555f53cf9800 con 0x555f53cf9400 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x555f53d6a800 0x555f53aa1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.128+0000 7fe1c70fc700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555f53cf9400 0x555f53aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c70fc700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555f53cf9400 0x555f53aa0100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c80fe700 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x555f53d6a800 0x555f53aa1700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x555f53b582a0 tx=0x555f53d0dc20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c70fc700 1 -- 172.21.15.179:0/30339522 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x555f53b5cb40 con 0x555f53cf9400 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "0": { 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "data": { 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "version": 5, 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "inotable": { 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "projected_free": [ 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:11.138 INFO:teuthology.orchestra.run.smithi179.stdout: "start": 1099511627779, 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: "len": 1099511627773 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: ], 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: "free": [ 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: "start": 1099511627779, 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: "len": 1099511627773 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: "result": 0 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:11.139 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555f53cf9400 msgr2=0x555f53aa0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x555f53cf9400 0x555f53aa0100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x555f53d6a800 msgr2=0x555f53aa1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x555f53d6a800 0x555f53aa1700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x555f53b582a0 tx=0x555f53d0dc20 comp rx=0 tx=0).stop 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 msgr2=0x555f53aa0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 --2- 172.21.15.179:0/30339522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f53cf8c00 0x555f53aa0c00 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x555f53d58300 tx=0x555f53d0cc00 comp rx=0 tx=0).stop 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c70fc700 1 -- 172.21.15.179:0/30339522 reap_dead start 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 shutdown_connections 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.132+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 >> 172.21.15.179:0/30339522 conn(0x555f539f8c00 msgr2=0x555f53ae9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 shutdown_connections 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/30339522 wait complete. 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 msgr2=0x555f53a9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x555f53b58d50 tx=0x555f5487c240 comp rx=0 tx=0).stop 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 shutdown_connections 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f539f9000 0x555f53a9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555f539f9400 0x555f53a9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 --2- 172.21.15.179:0/1682097793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f539f9800 0x555f53a9e580 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 >> 172.21.15.179:0/1682097793 conn(0x555f539f9c00 msgr2=0x555f53ae9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 shutdown_connections 2024-05-08T22:44:11.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.136+0000 7fe1c9143500 1 -- 172.21.15.179:0/1682097793 wait complete. 2024-05-08T22:44:11.143 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.062698, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-08T22:44:11.144 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:44:11.144 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:44:11.208 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:44:11.229 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:44:11.230 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:11.230 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:11.237 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:44:11.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:44:11.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:11.238 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:44:11.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:44:11.245 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T22:44:11.245 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:44:11.245 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:11.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:44:11.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:44:11.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:11.253 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:44:11.259 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:44:11.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:44:11.260 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:11.260 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:11.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:44:11.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:44:11.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:11.294 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:44:11.300 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:44:11.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:44:11.301 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:44:11.302 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:11.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:44:11.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:44:11.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:11.312 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:44:11.318 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:44:11.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:44:11.320 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:11.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 Processor -- start 2024-05-08T22:44:11.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- start start 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f519c15df60 0x7f519c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c06eb20 con 0x7f519c0cab60 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c15e880 con 0x7f519c06ac90 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c06df10 con 0x7f519c15df60 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f519c15df60 0x7f519c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f519c0cab60 0x7f519c0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37176/0 (socket says 172.21.15.179:37176) 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519b7fe700 1 -- 172.21.15.179:0/583138870 learned_addr learned my addr 172.21.15.179:0/583138870 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:11.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f519c15df60 0x7f519c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54598/0 (socket says 172.21.15.179:54598) 2024-05-08T22:44:11.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 -- 172.21.15.179:0/583138870 >> v1:172.21.15.195:6789/0 conn(0x7f519c06ac90 legacy=0x7f519c0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 -- 172.21.15.179:0/583138870 >> v1:172.21.15.179:6789/0 conn(0x7f519c0cab60 legacy=0x7f519c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 -- 172.21.15.179:0/583138870 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c06e1f0 con 0x7f519c15df60 2024-05-08T22:44:11.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f5199ffb700 1 -- 172.21.15.179:0/583138870 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768058701 0 0) 0x7f519c06eb20 con 0x7f519c0cab60 2024-05-08T22:44:11.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f519affd700 1 --2- 172.21.15.179:0/583138870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c15df60 0x7f519c15e340 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5190009850 tx=0x7f5190009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac7bdfa33f7abb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f5199ffb700 1 -- 172.21.15.179:0/583138870 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011830 con 0x7f519c15df60 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f5199ffb700 1 -- 172.21.15.179:0/583138870 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190011990 con 0x7f519c15df60 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f5199ffb700 1 -- 172.21.15.179:0/583138870 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011d20 con 0x7f519c15df60 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- 172.21.15.179:0/583138870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c15df60 msgr2=0x7f519c15e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 --2- 172.21.15.179:0/583138870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c15df60 0x7f519c15e340 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5190009850 tx=0x7f5190009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:11.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- 172.21.15.179:0/583138870 shutdown_connections 2024-05-08T22:44:11.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 --2- 172.21.15.179:0/583138870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c15df60 0x7f519c15e340 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- 172.21.15.179:0/583138870 >> 172.21.15.179:0/583138870 conn(0x7f519c0cb660 msgr2=0x7f519c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- 172.21.15.179:0/583138870 shutdown_connections 2024-05-08T22:44:11.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.439+0000 7f51a188a700 1 -- 172.21.15.179:0/583138870 wait complete. 2024-05-08T22:44:11.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 Processor -- start 2024-05-08T22:44:11.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 -- start start 2024-05-08T22:44:11.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f519c0cab60 0x7f519c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 0x7f519c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c6220 con 0x7f519c06ac90 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c6500 con 0x7f519c15df60 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.443+0000 7f51a188a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c67e0 con 0x7f519c0cab60 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54600/0 (socket says 172.21.15.179:54600) 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 -- 172.21.15.179:0/418210368 learned_addr learned my addr 172.21.15.179:0/418210368 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519a7fc700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f519c0cab60 0x7f519c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519b7fe700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 0x7f519c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 -- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f519c0cab60 msgr2=0x7f519c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f519c0cab60 0x7f519c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 -- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 msgr2=0x7f519c1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 0x7f519c1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 -- 172.21.15.179:0/418210368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094b0 con 0x7f519c06ac90 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519b7fe700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 0x7f519c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f519affd700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5190009fd0 tx=0x7f5190004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000bb60 con 0x7f519c06ac90 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519001e070 con 0x7f519c06ac90 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0c6ac0 con 0x7f519c06ac90 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519001a420 con 0x7f519c06ac90 2024-05-08T22:44:11.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0c54c0 con 0x7f519c06ac90 2024-05-08T22:44:11.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c067770 con 0x7f519c06ac90 2024-05-08T22:44:11.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.447+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f519001a580 con 0x7f519c06ac90 2024-05-08T22:44:11.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.451+0000 7f5183fff700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 0x7f5188068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.451+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f51900856b0 con 0x7f519c06ac90 2024-05-08T22:44:11.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.451+0000 7f519a7fc700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 0x7f5188068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.451+0000 7f519a7fc700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 0x7f5188068bf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f519c15cf90 tx=0x7f518c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.455+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5190052880 con 0x7f519c06ac90 2024-05-08T22:44:11.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.667+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f519c06ef90 con 0x7f519c06ac90 2024-05-08T22:44:11.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.667+0000 7f5183fff700 1 -- 172.21.15.179:0/418210368 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3979 (secure 0 0 0) 0x7f519c06ef90 con 0x7f519c06ac90 2024-05-08T22:44:11.672 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:11.672 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:44:11.051975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 msgr2=0x7f5188068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 0x7f5188068bf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f519c15cf90 tx=0x7f518c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 msgr2=0x7f519c0c4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5190009fd0 tx=0x7f5190004770 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 shutdown_connections 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5188066730 0x7f5188068bf0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f519c15df60 0x7f519c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f519c0cab60 0x7f519c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 --2- 172.21.15.179:0/418210368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c06ac90 0x7f519c0c4f80 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 >> 172.21.15.179:0/418210368 conn(0x7f519c0cb660 msgr2=0x7f519c0ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 shutdown_connections 2024-05-08T22:44:11.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.671+0000 7f51a188a700 1 -- 172.21.15.179:0/418210368 wait complete. 2024-05-08T22:44:11.677 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 82 2024-05-08T22:44:11.688 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:11.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 Processor -- start 2024-05-08T22:44:11.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 -- start start 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f88ac165a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0b6bf0 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0cc750 con 0x7f88ac15b070 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b2dde700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0cca30 con 0x7f88ac0cff50 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b15db700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f88ac0cc2c0 0x7f88ac0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37186/0 (socket says 172.21.15.179:37186) 2024-05-08T22:44:11.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b15db700 1 -- 172.21.15.179:0/3502607043 learned_addr learned my addr 172.21.15.179:0/3502607043 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b25dd700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f88ac165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.791+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151574604 0 0) 0x7f88ac0b6bf0 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88980035d0 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626228966 0 0) 0x7f88ac0cc750 con 0x7f88ac15b070 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88ac0b6bf0 con 0x7f88ac15b070 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540625384 0 0) 0x7f88980035d0 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 msgr2=0x7f88ac165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f88ac165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 >> v1:172.21.15.195:6789/0 conn(0x7f88ac15b070 legacy=0x7f88ac0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0ccd10 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f88a8001670 con 0x7f88ac0cc2c0 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 >> v1:172.21.15.179:6789/0 conn(0x7f88ac0cc2c0 legacy=0x7f88ac0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f8898003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004190 0x7f8898004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8898004db0 0x7f8898005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0cca30 con 0x7f8898004db0 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889800bb70 con 0x7f8898004190 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800bd30 con 0x7f88ac0cff50 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8898004db0 0x7f8898005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b1ddc700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f8898003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b15db700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004190 0x7f8898004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 -- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 msgr2=0x7f8898003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac0cff50 0x7f8898003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 -- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004190 msgr2=0x7f8898004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004190 0x7f8898004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 -- 172.21.15.179:0/3502607043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980035d0 con 0x7f8898004db0 2024-05-08T22:44:11.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8898004db0 0x7f8898005230 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0d0630 tx=0x7f88980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b403e9a409d92f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400ebd0 con 0x7f8898004db0 2024-05-08T22:44:11.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400f040 con 0x7f8898004db0 2024-05-08T22:44:11.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b0dda700 1 -- 172.21.15.179:0/3502607043 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4014480 con 0x7f8898004db0 2024-05-08T22:44:11.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 -- 172.21.15.179:0/3502607043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8898004db0 msgr2=0x7f8898005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 --2- 172.21.15.179:0/3502607043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8898004db0 0x7f8898005230 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0d0630 tx=0x7f88980058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b25dd700 1 -- 172.21.15.179:0/3502607043 reap_dead start 2024-05-08T22:44:11.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 -- 172.21.15.179:0/3502607043 shutdown_connections 2024-05-08T22:44:11.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 -- 172.21.15.179:0/3502607043 >> 172.21.15.179:0/3502607043 conn(0x7f88ac0cd6e0 msgr2=0x7f88ac0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 -- 172.21.15.179:0/3502607043 shutdown_connections 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 -- 172.21.15.179:0/3502607043 wait complete. 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.795+0000 7f88b2dde700 1 Processor -- start 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- start start 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004db0 0x7f88ac147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac14dd90 0x7f88ac0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac14ac10 con 0x7f88ac0b2000 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac14a090 con 0x7f8898004db0 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac14a370 con 0x7f88ac14dd90 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54606/0 (socket says 172.21.15.179:54606) 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 -- 172.21.15.179:0/2108381494 learned_addr learned my addr 172.21.15.179:0/2108381494 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:11.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b1ddc700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004db0 0x7f88ac147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b15db700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac14dd90 0x7f88ac0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 -- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac14dd90 msgr2=0x7f88ac0b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac14dd90 0x7f88ac0b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 -- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004db0 msgr2=0x7f88ac147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004db0 0x7f88ac147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 -- 172.21.15.179:0/2108381494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac14a650 con 0x7f88ac0b2000 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b25dd700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f889800bb40 tx=0x7f88a4004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4007000 con 0x7f88ac0b2000 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac14a930 con 0x7f88ac0b2000 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4014c90 con 0x7f88ac0b2000 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac149d10 con 0x7f88ac0b2000 2024-05-08T22:44:11.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.799+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4003a40 con 0x7f88ac0b2000 2024-05-08T22:44:11.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac04e700 con 0x7f88ac0b2000 2024-05-08T22:44:11.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88a4023070 con 0x7f88ac0b2000 2024-05-08T22:44:11.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88a27fc700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 0x7f8890068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:11.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f88a40852d0 con 0x7f88ac0b2000 2024-05-08T22:44:11.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88b1ddc700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 0x7f8890068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:11.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.803+0000 7f88b1ddc700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 0x7f8890068bf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0afbc0 tx=0x7f889c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:11.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.811+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f88a4055fd0 con 0x7f88ac0b2000 2024-05-08T22:44:11.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.979+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88ac0cfc50 con 0x7f88ac0b2000 2024-05-08T22:44:11.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.983+0000 7f88a27fc700 1 -- 172.21.15.179:0/2108381494 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16655 (secure 0 0 0) 0x7f88a4055b60 con 0x7f88ac0b2000 2024-05-08T22:44:11.989 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:11.989 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":116,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:11.051025+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6836/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6837/4059323811":"2024-05-09T22:43:34.457765+0000","172.21.15.195:6836/4059323811":"2024-05-09T22:43:34.457765+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 msgr2=0x7f8890068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 0x7f8890068bf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0afbc0 tx=0x7f889c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 msgr2=0x7f88ac0b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f889800bb40 tx=0x7f88a4004b60 comp rx=0 tx=0).stop 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 shutdown_connections 2024-05-08T22:44:11.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8890066730 0x7f8890068bf0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88ac0b2000 0x7f88ac0b53e0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88ac14dd90 0x7f88ac0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 --2- 172.21.15.179:0/2108381494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898004db0 0x7f88ac147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 >> 172.21.15.179:0/2108381494 conn(0x7f88ac0cd6e0 msgr2=0x7f88ac15d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 shutdown_connections 2024-05-08T22:44:11.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:11.987+0000 7f88b2dde700 1 -- 172.21.15.179:0/2108381494 wait complete. 2024-05-08T22:44:12.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:42:35.609486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:42:36.273848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:12.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:12.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 Processor -- start 2024-05-08T22:44:12.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 -- start start 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f74b80b19e0 0x7f74b80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 0x7f74b806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b806cda0 con 0x7f74b8159320 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80cb4c0 con 0x7f74b80b19e0 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74bcbdf700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80b1f10 con 0x7f74b8159700 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f74b80b19e0 0x7f74b80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f74b8159320 0x7f74b80cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37196/0 (socket says 172.21.15.179:37196) 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b77fe700 1 -- 172.21.15.179:0/1553926216 learned_addr learned my addr 172.21.15.179:0/1553926216 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b7fff700 1 --2- 172.21.15.179:0/1553926216 >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 0x7f74b806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 -- 172.21.15.179:0/1553926216 >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 msgr2=0x7f74b806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:12.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 --2- 172.21.15.179:0/1553926216 >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 0x7f74b806b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 -- 172.21.15.179:0/1553926216 >> v1:172.21.15.179:6789/0 conn(0x7f74b8159320 legacy=0x7f74b80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b67fc700 1 -- 172.21.15.179:0/1553926216 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262786628 0 0) 0x7f74b806cda0 con 0x7f74b8159320 2024-05-08T22:44:12.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 -- 172.21.15.179:0/1553926216 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80b21f0 con 0x7f74b80b19e0 2024-05-08T22:44:12.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b7fff700 1 --2- 172.21.15.179:0/1553926216 >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 0x7f74b806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:12.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.123+0000 7f74b6ffd700 1 --2- 172.21.15.179:0/1553926216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b80b19e0 0x7f74b80cc630 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f74b80b2b20 tx=0x7f74a8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2775e668be6f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74b67fc700 1 -- 172.21.15.179:0/1553926216 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8011830 con 0x7f74b80b19e0 2024-05-08T22:44:12.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74b67fc700 1 -- 172.21.15.179:0/1553926216 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8011990 con 0x7f74b80b19e0 2024-05-08T22:44:12.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74b67fc700 1 -- 172.21.15.179:0/1553926216 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8011d40 con 0x7f74b80b19e0 2024-05-08T22:44:12.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1553926216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b80b19e0 msgr2=0x7f74b80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1553926216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b80b19e0 0x7f74b80cc630 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f74b80b2b20 tx=0x7f74a8009b30 comp rx=0 tx=0).stop 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1553926216 shutdown_connections 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1553926216 >> v2:172.21.15.195:3300/0 conn(0x7f74b8159700 0x7f74b806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1553926216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b80b19e0 0x7f74b80cc630 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1553926216 >> 172.21.15.179:0/1553926216 conn(0x7f74b80cea30 msgr2=0x7f74b80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1553926216 shutdown_connections 2024-05-08T22:44:12.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.127+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1553926216 wait complete. 2024-05-08T22:44:12.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 Processor -- start 2024-05-08T22:44:12.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- start start 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74b80b19e0 0x7f74b815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74b8159320 0x7f74b8147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8142470 con 0x7f74b8159700 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b81486b0 con 0x7f74b8159320 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b81480e0 con 0x7f74b80b19e0 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54626/0 (socket says 172.21.15.179:54626) 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 -- 172.21.15.179:0/1185272630 learned_addr learned my addr 172.21.15.179:0/1185272630 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b6ffd700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74b8159320 0x7f74b8147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b77fe700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74b80b19e0 0x7f74b815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 -- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74b80b19e0 msgr2=0x7f74b815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74b80b19e0 0x7f74b815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 -- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74b8159320 msgr2=0x7f74b8147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74b8159320 0x7f74b8147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 -- 172.21.15.179:0/1185272630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80094b0 con 0x7f74b8159700 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74b7fff700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009fd0 tx=0x7f74b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000dde0 con 0x7f74b8159700 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000f040 con 0x7f74b8159700 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b815c6c0 con 0x7f74b8159700 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003c80 con 0x7f74b8159700 2024-05-08T22:44:12.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.131+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b8141db0 con 0x7f74b8159700 2024-05-08T22:44:12.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b804e700 con 0x7f74b8159700 2024-05-08T22:44:12.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74b0004030 con 0x7f74b8159700 2024-05-08T22:44:12.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74affff700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 0x7f7498068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f74b0084310 con 0x7f74b8159700 2024-05-08T22:44:12.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74b77fe700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 0x7f7498068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.135+0000 7f74b77fe700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 0x7f7498068c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000f60 tx=0x7f74a0007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.143+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f74b00514e0 con 0x7f74b8159700 2024-05-08T22:44:12.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.315+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74b815d4f0 con 0x7f74b8159700 2024-05-08T22:44:12.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.315+0000 7f74affff700 1 -- 172.21.15.179:0/1185272630 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74b0054b00 con 0x7f74b8159700 2024-05-08T22:44:12.321 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:12.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 msgr2=0x7f7498068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 0x7f7498068c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000f60 tx=0x7f74a0007e00 comp rx=0 tx=0).stop 2024-05-08T22:44:12.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 msgr2=0x7f74b8149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009fd0 tx=0x7f74b000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 shutdown_connections 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7498066760 0x7f7498068c20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74b8159320 0x7f74b8147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b8159700 0x7f74b8149ad0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 --2- 172.21.15.179:0/1185272630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74b80b19e0 0x7f74b815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 >> 172.21.15.179:0/1185272630 conn(0x7f74b80cea30 msgr2=0x7f74b806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 shutdown_connections 2024-05-08T22:44:12.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.319+0000 7f74bcbdf700 1 -- 172.21.15.179:0/1185272630 wait complete. 2024-05-08T22:44:12.335 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:12.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 Processor -- start 2024-05-08T22:44:12.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 -- start start 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdec0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0cb640 con 0x7efdec160c70 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0cb310 con 0x7efdec15d050 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdf0c0e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec161b50 con 0x7efdec1610d0 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdeaffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efdec160c70 0x7efdec0cbad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37198/0 (socket says 172.21.15.179:37198) 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdeaffd700 1 -- 172.21.15.179:0/455105905 learned_addr learned my addr 172.21.15.179:0/455105905 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:12.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdebfff700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdec0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.447+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858620490 0 0) 0x7efdec0cb640 con 0x7efdec160c70 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd40035d0 con 0x7efdec160c70 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046338602 0 0) 0x7efdec0cb310 con 0x7efdec15d050 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdec0cb640 con 0x7efdec15d050 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126501060 0 0) 0x7efdd40035d0 con 0x7efdec160c70 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 msgr2=0x7efdec0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdec0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 >> v1:172.21.15.195:6789/0 conn(0x7efdec15d050 legacy=0x7efdec0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0b65a0 con 0x7efdec160c70 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7efddc0019f0 con 0x7efdec160c70 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdebfff700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdec0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:12.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 >> v1:172.21.15.179:6789/0 conn(0x7efdec160c70 legacy=0x7efdec0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdd4003be0 0x7efdd4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdd4004870 0x7efdd4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdd4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec161b50 con 0x7efdd4003be0 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd400ba60 con 0x7efdd4004870 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd400bc20 con 0x7efdec1610d0 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdd4003be0 0x7efdd4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdebfff700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdd4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeaffd700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdd4004870 0x7efdd4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 -- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 msgr2=0x7efdd4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec1610d0 0x7efdd4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 -- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdd4004870 msgr2=0x7efdd4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdd4004870 0x7efdd4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 -- 172.21.15.179:0/455105905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40094b0 con 0x7efdd4003be0 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdd4003be0 0x7efdd4003fe0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efde0007fd0 tx=0x7efdd40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1055566063deabcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0001450 con 0x7efdd4003be0 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde00052f0 con 0x7efdd4003be0 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdea7fc700 1 -- 172.21.15.179:0/455105905 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0001d60 con 0x7efdd4003be0 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 -- 172.21.15.179:0/455105905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdd4003be0 msgr2=0x7efdd4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/455105905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdd4003be0 0x7efdd4003fe0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efde0007fd0 tx=0x7efdd40058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdebfff700 1 -- 172.21.15.179:0/455105905 reap_dead start 2024-05-08T22:44:12.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 -- 172.21.15.179:0/455105905 shutdown_connections 2024-05-08T22:44:12.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 -- 172.21.15.179:0/455105905 >> 172.21.15.179:0/455105905 conn(0x7efdec0b2710 msgr2=0x7efdec0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 -- 172.21.15.179:0/455105905 shutdown_connections 2024-05-08T22:44:12.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.451+0000 7efdf0c0e700 1 -- 172.21.15.179:0/455105905 wait complete. 2024-05-08T22:44:12.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 Processor -- start 2024-05-08T22:44:12.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 -- start start 2024-05-08T22:44:12.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 0x7efdec06c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec15bf50 0x7efdec06e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06e9d0 con 0x7efdec06e550 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0c4750 con 0x7efdec15bf50 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06ca20 con 0x7efdec151fa0 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54640/0 (socket says 172.21.15.179:54640) 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 -- 172.21.15.179:0/2533900327 learned_addr learned my addr 172.21.15.179:0/2533900327 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 0x7efdec06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdeaffd700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec15bf50 0x7efdec06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 -- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 msgr2=0x7efdec06c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 0x7efdec06c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 -- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec15bf50 msgr2=0x7efdec06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec15bf50 0x7efdec06e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 -- 172.21.15.179:0/2533900327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec06cd00 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 0x7efdec06c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdebfff700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b3320 tx=0x7efde4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400f990 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde400fb10 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec06cfe0 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.455+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400fec0 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec0c79f0 con 0x7efdec06e550 2024-05-08T22:44:12.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec04e700 con 0x7efdec06e550 2024-05-08T22:44:12.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efde4014440 con 0x7efdec06e550 2024-05-08T22:44:12.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdd3fff700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 0x7efdd8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7efde4084850 con 0x7efdec06e550 2024-05-08T22:44:12.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 0x7efdd8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.459+0000 7efdeb7fe700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 0x7efdd8068c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b0630 tx=0x7efde000a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.467+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efde4051980 con 0x7efdec06e550 2024-05-08T22:44:12.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.627+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdec06ddf0 con 0x7efdec06e550 2024-05-08T22:44:12.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdd3fff700 1 -- 172.21.15.179:0/2533900327 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7efde4054fa0 con 0x7efdec06e550 2024-05-08T22:44:12.635 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:12.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 msgr2=0x7efdd8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 0x7efdd8068c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b0630 tx=0x7efde000a000 comp rx=0 tx=0).stop 2024-05-08T22:44:12.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 msgr2=0x7efdec1f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b3320 tx=0x7efde4011f60 comp rx=0 tx=0).stop 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 shutdown_connections 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efdd8066760 0x7efdd8068c20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdec15bf50 0x7efdec06e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec06e550 0x7efdec1f64a0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 --2- 172.21.15.179:0/2533900327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdec151fa0 0x7efdec06c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 >> 172.21.15.179:0/2533900327 conn(0x7efdec0b2710 msgr2=0x7efdec0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 shutdown_connections 2024-05-08T22:44:12.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.631+0000 7efdf0c0e700 1 -- 172.21.15.179:0/2533900327 wait complete. 2024-05-08T22:44:12.649 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:12.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 Processor -- start 2024-05-08T22:44:12.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 -- start start 2024-05-08T22:44:12.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 0x7f4fb80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4fb80b2c10 0x7f4fb806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb806dd50 con 0x7f4fb814ae30 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb806e9d0 con 0x7f4fb80b2c10 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbd155700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80b30c0 con 0x7f4fb80b1a50 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4fb814ae30 0x7f4fb8165b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37202/0 (socket says 172.21.15.179:37202) 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb7fff700 1 -- 172.21.15.179:0/4115090498 learned_addr learned my addr 172.21.15.179:0/4115090498 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb77fe700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 0x7f4fb80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 --2- 172.21.15.179:0/4115090498 >> v2:172.21.15.195:3300/0 conn(0x7f4fb80b2c10 0x7f4fb806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb6ffd700 1 -- 172.21.15.179:0/4115090498 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148151098 0 0) 0x7f4fb806dd50 con 0x7f4fb814ae30 2024-05-08T22:44:12.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb6ffd700 1 -- 172.21.15.179:0/4115090498 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa00035d0 con 0x7f4fb814ae30 2024-05-08T22:44:12.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 -- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 msgr2=0x7f4fb80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:12.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 0x7f4fb80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 -- 172.21.15.179:0/4115090498 >> v1:172.21.15.179:6789/0 conn(0x7f4fb814ae30 legacy=0x7f4fb8165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 -- 172.21.15.179:0/4115090498 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb806e030 con 0x7f4fb80b2c10 2024-05-08T22:44:12.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb77fe700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 0x7f4fb80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fbc954700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b2c10 0x7f4fb806abb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009d90 tx=0x7f4fb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d639a50f6fd696 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb6ffd700 1 -- 172.21.15.179:0/4115090498 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000e070 con 0x7f4fb80b2c10 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb6ffd700 1 -- 172.21.15.179:0/4115090498 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000bb20 con 0x7f4fb80b2c10 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.767+0000 7f4fb6ffd700 1 -- 172.21.15.179:0/4115090498 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000be90 con 0x7f4fb80b2c10 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b2c10 msgr2=0x7f4fb806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b2c10 0x7f4fb806abb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009d90 tx=0x7f4fb000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- 172.21.15.179:0/4115090498 shutdown_connections 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b2c10 0x7f4fb806abb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 --2- 172.21.15.179:0/4115090498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b1a50 0x7f4fb80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- 172.21.15.179:0/4115090498 >> 172.21.15.179:0/4115090498 conn(0x7f4fb80cea30 msgr2=0x7f4fb80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- 172.21.15.179:0/4115090498 shutdown_connections 2024-05-08T22:44:12.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- 172.21.15.179:0/4115090498 wait complete. 2024-05-08T22:44:12.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 Processor -- start 2024-05-08T22:44:12.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.771+0000 7f4fbd155700 1 -- start start 2024-05-08T22:44:12.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 0x7f4fb8160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b2c10 0x7f4fb81f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8141e40 con 0x7f4fb814ae30 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb81f7580 con 0x7f4fb80b1a50 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb81f8060 con 0x7f4fb80b2c10 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54656/0 (socket says 172.21.15.179:54656) 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 -- 172.21.15.179:0/721970627 learned_addr learned my addr 172.21.15.179:0/721970627 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb7fff700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 0x7f4fb8160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb77fe700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b2c10 0x7f4fb81f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 -- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b2c10 msgr2=0x7f4fb81f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b2c10 0x7f4fb81f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 -- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 msgr2=0x7f4fb8160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 0x7f4fb8160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 -- 172.21.15.179:0/721970627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80094b0 con 0x7f4fb814ae30 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb7fff700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 0x7f4fb8160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:12.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbc954700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f4fb806f0c0 tx=0x7f4fb0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00098f0 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0003e30 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb0009a70 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0012650 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb81fb620 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.775+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb8141f80 con 0x7f4fb814ae30 2024-05-08T22:44:12.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.779+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4fb00129c0 con 0x7f4fb814ae30 2024-05-08T22:44:12.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.779+0000 7f4fb4ff9700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 0x7f4fa4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:12.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.779+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f4fb00865b0 con 0x7f4fb814ae30 2024-05-08T22:44:12.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.779+0000 7f4fb7fff700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 0x7f4fa4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:12.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.779+0000 7f4fb7fff700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 0x7f4fa4068bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005930 tx=0x7f4fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:12.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.787+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4fb0053830 con 0x7f4fb814ae30 2024-05-08T22:44:12.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.955+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fb8141f80 con 0x7f4fb814ae30 2024-05-08T22:44:12.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.955+0000 7f4fb4ff9700 1 -- 172.21.15.179:0/721970627 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fb8141f80 con 0x7f4fb814ae30 2024-05-08T22:44:12.960 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:12.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 msgr2=0x7f4fa4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 0x7f4fa4068bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005930 tx=0x7f4fac009040 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 msgr2=0x7f4fb81fb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f4fb806f0c0 tx=0x7f4fb0004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 shutdown_connections 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4fa4066730 0x7f4fa4068bf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb814ae30 0x7f4fb81fb0e0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fb80b2c10 0x7f4fb81f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 --2- 172.21.15.179:0/721970627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fb80b1a50 0x7f4fb8160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 >> 172.21.15.179:0/721970627 conn(0x7f4fb80cea30 msgr2=0x7f4fb81484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 shutdown_connections 2024-05-08T22:44:12.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:12.959+0000 7f4fbd155700 1 -- 172.21.15.179:0/721970627 wait complete. 2024-05-08T22:44:12.975 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:13.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.079+0000 7f79319c5700 1 Processor -- start 2024-05-08T22:44:13.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.079+0000 7f79319c5700 1 -- start start 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f792c0b3720 0x7f792c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792c159b60 con 0x7f792c1539c0 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792c159e40 con 0x7f792c0b19e0 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792c15a120 con 0x7f792c0b3720 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79309c3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f792c1539c0 0x7f792c06b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37204/0 (socket says 172.21.15.179:37204) 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79309c3700 1 -- 172.21.15.179:0/3265209719 learned_addr learned my addr 172.21.15.179:0/3265209719 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:13.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79311c4700 1 --2- 172.21.15.179:0/3265209719 >> v2:172.21.15.195:3300/0 conn(0x7f792c0b3720 0x7f792c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471965550 0 0) 0x7f792c159b60 con 0x7f792c1539c0 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c0034f0 con 0x7f792c1539c0 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79311c4700 1 -- 172.21.15.179:0/3265209719 >> v1:172.21.15.195:6789/0 conn(0x7f792c0b19e0 legacy=0x7f792c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805116231 0 0) 0x7f790c0034f0 con 0x7f792c1539c0 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79311c4700 1 -- 172.21.15.179:0/3265209719 >> v1:172.21.15.179:6789/0 conn(0x7f792c1539c0 legacy=0x7f792c06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79311c4700 1 -- 172.21.15.179:0/3265209719 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c15a400 con 0x7f792c0b3720 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79311c4700 1 --2- 172.21.15.179:0/3265209719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c0b3720 0x7f792c15d120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7924009d90 tx=0x7f792400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abc972f43e10d3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400e070 con 0x7f792c0b3720 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924003a40 con 0x7f792c0b3720 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f792b7fe700 1 -- 172.21.15.179:0/3265209719 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c001670 con 0x7f792c0b3720 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- 172.21.15.179:0/3265209719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c0b3720 msgr2=0x7f792c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 --2- 172.21.15.179:0/3265209719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c0b3720 0x7f792c15d120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7924009d90 tx=0x7f792400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- 172.21.15.179:0/3265209719 shutdown_connections 2024-05-08T22:44:13.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 --2- 172.21.15.179:0/3265209719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c0b3720 0x7f792c15d120 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- 172.21.15.179:0/3265209719 >> 172.21.15.179:0/3265209719 conn(0x7f792c0ce9d0 msgr2=0x7f792c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- 172.21.15.179:0/3265209719 shutdown_connections 2024-05-08T22:44:13.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.083+0000 7f79319c5700 1 -- 172.21.15.179:0/3265209719 wait complete. 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 Processor -- start 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- start start 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c0b3720 0x7f792c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c1539c0 0x7f792c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0a7ee0 con 0x7f792c0b19e0 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0a7920 con 0x7f792c1539c0 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0a7c00 con 0x7f792c0b3720 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79311c4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c1539c0 0x7f792c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54660/0 (socket says 172.21.15.179:54660) 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 -- 172.21.15.179:0/3205190097 learned_addr learned my addr 172.21.15.179:0/3205190097 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f792bfff700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c0b3720 0x7f792c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 -- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c0b3720 msgr2=0x7f792c158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c0b3720 0x7f792c158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 -- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c1539c0 msgr2=0x7f792c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c1539c0 0x7f792c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 -- 172.21.15.179:0/3205190097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924009a70 con 0x7f792c0b19e0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79309c3700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c6d0 tx=0x7f791c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c004ca0 con 0x7f792c0b19e0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c004e00 con 0x7f792c0b19e0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792c15abf0 con 0x7f792c0b19e0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0072d0 con 0x7f792c0b19e0 2024-05-08T22:44:13.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.087+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c0a74d0 con 0x7f792c0b19e0 2024-05-08T22:44:13.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792c0a76f0 con 0x7f792c0b19e0 2024-05-08T22:44:13.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f791c007430 con 0x7f792c0b19e0 2024-05-08T22:44:13.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f79297fa700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 0x7f7910068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f791c084080 con 0x7f792c0b19e0 2024-05-08T22:44:13.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f792bfff700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 0x7f7910068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.091+0000 7f792bfff700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 0x7f7910068bf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f792c0ae560 tx=0x7f7914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.099+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f791c0511c0 con 0x7f792c0b19e0 2024-05-08T22:44:13.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.307+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f792c15b110 con 0x7f792c0b19e0 2024-05-08T22:44:13.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.307+0000 7f79297fa700 1 -- 172.21.15.179:0/3205190097 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f792c15b110 con 0x7f792c0b19e0 2024-05-08T22:44:13.314 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:13.314 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":9133,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1771492548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1771492548},{"type":"v1","addr":"172.21.15.195:6837","nonce":1771492548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:44:11.051975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:44:13.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 msgr2=0x7f7910068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 0x7f7910068bf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f792c0ae560 tx=0x7f7914009040 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 msgr2=0x7f792c06e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c6d0 tx=0x7f791c00c9e0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 shutdown_connections 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7910066730 0x7f7910068bf0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f792c1539c0 0x7f792c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f792c0b3720 0x7f792c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 --2- 172.21.15.179:0/3205190097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f792c0b19e0 0x7f792c06e550 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 >> 172.21.15.179:0/3205190097 conn(0x7f792c0ce9d0 msgr2=0x7f792c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 shutdown_connections 2024-05-08T22:44:13.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.311+0000 7f79319c5700 1 -- 172.21.15.179:0/3205190097 wait complete. 2024-05-08T22:44:13.319 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 83 2024-05-08T22:44:13.331 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:13.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.435+0000 7f83eb1b5700 1 Processor -- start 2024-05-08T22:44:13.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.435+0000 7f83eb1b5700 1 -- start start 2024-05-08T22:44:13.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 0x7f83e40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e40ca310 con 0x7f83e40cd540 2024-05-08T22:44:13.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e415a200 con 0x7f83e4159e20 2024-05-08T22:44:13.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40b4e20 con 0x7f83e40b50b0 2024-05-08T22:44:13.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e99b2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f83e40cd540 0x7f83e40ca7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37218/0 (socket says 172.21.15.179:37218) 2024-05-08T22:44:13.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e99b2700 1 -- 172.21.15.179:0/1857512365 learned_addr learned my addr 172.21.15.179:0/1857512365 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:13.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83ea9b4700 1 --2- 172.21.15.179:0/1857512365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 0x7f83e40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142393660 0 0) 0x7f83e40ca310 con 0x7f83e40cd540 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d00035d0 con 0x7f83e40cd540 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427430728 0 0) 0x7f83e415a200 con 0x7f83e4159e20 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e40ca310 con 0x7f83e4159e20 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83ea9b4700 1 -- 172.21.15.179:0/1857512365 >> v1:172.21.15.195:6789/0 conn(0x7f83e4159e20 legacy=0x7f83e40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83ea9b4700 1 -- 172.21.15.179:0/1857512365 >> v1:172.21.15.179:6789/0 conn(0x7f83e40cd540 legacy=0x7f83e40ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83ea9b4700 1 -- 172.21.15.179:0/1857512365 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40c99e0 con 0x7f83e40b50b0 2024-05-08T22:44:13.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83ea9b4700 1 --2- 172.21.15.179:0/1857512365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 0x7f83e40caeb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009dc0 tx=0x7f83dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab5a9948e4a4f10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00e070 con 0x7f83e40b50b0 2024-05-08T22:44:13.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc00bae0 con 0x7f83e40b50b0 2024-05-08T22:44:13.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83e91b1700 1 -- 172.21.15.179:0/1857512365 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00be90 con 0x7f83e40b50b0 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- 172.21.15.179:0/1857512365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 msgr2=0x7f83e40caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/1857512365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 0x7f83e40caeb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009dc0 tx=0x7f83dc00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- 172.21.15.179:0/1857512365 shutdown_connections 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/1857512365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40b50b0 0x7f83e40caeb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.439+0000 7f83eb1b5700 1 -- 172.21.15.179:0/1857512365 >> 172.21.15.179:0/1857512365 conn(0x7f83e4098a70 msgr2=0x7f83e40b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- 172.21.15.179:0/1857512365 shutdown_connections 2024-05-08T22:44:13.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- 172.21.15.179:0/1857512365 wait complete. 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 Processor -- start 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- start start 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e40b50b0 0x7f83e414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40cd540 0x7f83e406b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e415d300 con 0x7f83e4159e20 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e406b740 con 0x7f83e40b50b0 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83eb1b5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e406ba20 con 0x7f83e40cd540 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54668/0 (socket says 172.21.15.179:54668) 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 -- 172.21.15.179:0/3657171800 learned_addr learned my addr 172.21.15.179:0/3657171800 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea1b3700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e40b50b0 0x7f83e414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83e99b2700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40cd540 0x7f83e406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 -- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40cd540 msgr2=0x7f83e406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40cd540 0x7f83e406b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 -- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e40b50b0 msgr2=0x7f83e414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e40b50b0 0x7f83e414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 -- 172.21.15.179:0/3657171800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc009a70 con 0x7f83e4159e20 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83ea9b4700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009d90 tx=0x7f83dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.443+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0044c0 con 0x7f83e4159e20 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc004640 con 0x7f83e4159e20 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e406bd00 con 0x7f83e4159e20 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc012430 con 0x7f83e4159e20 2024-05-08T22:44:13.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e406c2c0 con 0x7f83e4159e20 2024-05-08T22:44:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e404e700 con 0x7f83e4159e20 2024-05-08T22:44:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83dc012590 con 0x7f83e4159e20 2024-05-08T22:44:13.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83daffd700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 0x7f83c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f83dc085c10 con 0x7f83e4159e20 2024-05-08T22:44:13.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.447+0000 7f83ea1b3700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 0x7f83c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.451+0000 7f83ea1b3700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 0x7f83c8068bf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f60 tx=0x7f83e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.455+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f83dc052d60 con 0x7f83e4159e20 2024-05-08T22:44:13.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.627+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e4157a50 con 0x7f83e4159e20 2024-05-08T22:44:13.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.631+0000 7f83daffd700 1 -- 172.21.15.179:0/3657171800 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16655 (secure 0 0 0) 0x7f83dc056380 con 0x7f83e4159e20 2024-05-08T22:44:13.636 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:13.636 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":116,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:11.051025+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6836/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6837/4059323811":"2024-05-09T22:43:34.457765+0000","172.21.15.195:6836/4059323811":"2024-05-09T22:43:34.457765+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.631+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 msgr2=0x7f83c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.631+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 0x7f83c8068bf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f60 tx=0x7f83e0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.631+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 msgr2=0x7f83e415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.631+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009d90 tx=0x7f83dc004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 shutdown_connections 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83c8066730 0x7f83c8068bf0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83e4159e20 0x7f83e415cdc0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83e40cd540 0x7f83e406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 --2- 172.21.15.179:0/3657171800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e40b50b0 0x7f83e414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 >> 172.21.15.179:0/3657171800 conn(0x7f83e4098a70 msgr2=0x7f83e40ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 shutdown_connections 2024-05-08T22:44:13.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.635+0000 7f83eb1b5700 1 -- 172.21.15.179:0/3657171800 wait complete. 2024-05-08T22:44:13.652 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:44:13.652 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:13.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.755+0000 7f393f506700 1 Processor -- start 2024-05-08T22:44:13.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.755+0000 7f393f506700 1 -- start start 2024-05-08T22:44:13.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 0x7f39380cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393806abb0 con 0x7f39380cc700 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393806ae90 con 0x7f39380cb510 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39380cb9a0 con 0x7f39380ce380 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393e504700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f39380cc700 0x7f393815b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37220/0 (socket says 172.21.15.179:37220) 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393e504700 1 -- 172.21.15.179:0/1270089164 learned_addr learned my addr 172.21.15.179:0/1270089164 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:13.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393dd03700 1 --2- 172.21.15.179:0/1270089164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 0x7f39380cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393d502700 1 -- 172.21.15.179:0/1270089164 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941111141 0 0) 0x7f393806abb0 con 0x7f39380cc700 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393dd03700 1 -- 172.21.15.179:0/1270089164 >> v1:172.21.15.195:6789/0 conn(0x7f39380cb510 legacy=0x7f393806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393dd03700 1 -- 172.21.15.179:0/1270089164 >> v1:172.21.15.179:6789/0 conn(0x7f39380cc700 legacy=0x7f393815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393dd03700 1 -- 172.21.15.179:0/1270089164 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39380ce800 con 0x7f39380ce380 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393dd03700 1 --2- 172.21.15.179:0/1270089164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 0x7f39380cafd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3934009dc0 tx=0x7f393400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c7d4f29bbf0c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393d502700 1 -- 172.21.15.179:0/1270089164 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393400e070 con 0x7f39380ce380 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393d502700 1 -- 172.21.15.179:0/1270089164 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393400bbd0 con 0x7f39380ce380 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393d502700 1 -- 172.21.15.179:0/1270089164 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934012430 con 0x7f39380ce380 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- 172.21.15.179:0/1270089164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 msgr2=0x7f39380cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 --2- 172.21.15.179:0/1270089164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 0x7f39380cafd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3934009dc0 tx=0x7f393400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- 172.21.15.179:0/1270089164 shutdown_connections 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 --2- 172.21.15.179:0/1270089164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380ce380 0x7f39380cafd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- 172.21.15.179:0/1270089164 >> 172.21.15.179:0/1270089164 conn(0x7f39380ce9d0 msgr2=0x7f39380cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- 172.21.15.179:0/1270089164 shutdown_connections 2024-05-08T22:44:13.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- 172.21.15.179:0/1270089164 wait complete. 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 Processor -- start 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.759+0000 7f393f506700 1 -- start start 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39380cb510 0x7f393814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380cc700 0x7f39380b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393814bd90 con 0x7f39380ce380 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393814e320 con 0x7f39380cb510 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39380ac660 con 0x7f39380cc700 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54676/0 (socket says 172.21.15.179:54676) 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 -- 172.21.15.179:0/3059277663 learned_addr learned my addr 172.21.15.179:0/3059277663 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393dd03700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380cc700 0x7f39380b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393e504700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39380cb510 0x7f393814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 -- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380cc700 msgr2=0x7f39380b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380cc700 0x7f39380b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 -- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39380cb510 msgr2=0x7f393814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39380cb510 0x7f393814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 -- 172.21.15.179:0/3059277663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934009a70 con 0x7f39380ce380 2024-05-08T22:44:13.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393ed05700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f393000c480 tx=0x7f393000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300052d0 con 0x7f39380ce380 2024-05-08T22:44:13.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393000ce90 con 0x7f39380ce380 2024-05-08T22:44:13.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39380ac940 con 0x7f39380ce380 2024-05-08T22:44:13.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300036a0 con 0x7f39380ce380 2024-05-08T22:44:13.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39380acf00 con 0x7f39380ce380 2024-05-08T22:44:13.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3938067740 con 0x7f39380ce380 2024-05-08T22:44:13.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.763+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3930003800 con 0x7f39380ce380 2024-05-08T22:44:13.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.767+0000 7f392affd700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 0x7f391c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:13.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.767+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f39300836b0 con 0x7f39380ce380 2024-05-08T22:44:13.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.767+0000 7f393e504700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 0x7f391c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:13.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.767+0000 7f393e504700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 0x7f391c068c20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f392c007b30 tx=0x7f392c007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:13.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.771+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f39300507f0 con 0x7f39380ce380 2024-05-08T22:44:13.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f393804e700 con 0x7f39380ce380 2024-05-08T22:44:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f392affd700 1 -- 172.21.15.179:0/3059277663 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f3930053e10 con 0x7f39380ce380 2024-05-08T22:44:13.984 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:13.984 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":9133,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1771492548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1771492548},{"type":"v1","addr":"172.21.15.195:6837","nonce":1771492548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:44:11.051975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:44:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 msgr2=0x7f391c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 0x7f391c068c20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f392c007b30 tx=0x7f392c007a80 comp rx=0 tx=0).stop 2024-05-08T22:44:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 msgr2=0x7f393814e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.979+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f393000c480 tx=0x7f393000c790 comp rx=0 tx=0).stop 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 shutdown_connections 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f391c066760 0x7f391c068c20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39380ce380 0x7f393814e720 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39380cc700 0x7f39380b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 --2- 172.21.15.179:0/3059277663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39380cb510 0x7f393814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 >> 172.21.15.179:0/3059277663 conn(0x7f39380ce9d0 msgr2=0x7f393806e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 shutdown_connections 2024-05-08T22:44:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:13.983+0000 7f393f506700 1 -- 172.21.15.179:0/3059277663 wait complete. 2024-05-08T22:44:13.988 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 83 2024-05-08T22:44:13.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:14.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 Processor -- start 2024-05-08T22:44:14.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 -- start start 2024-05-08T22:44:14.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe9f4161790 0x7fe9f4164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f4165d50 con 0x7fe9f415ef80 2024-05-08T22:44:14.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f406ad70 con 0x7fe9f406de70 2024-05-08T22:44:14.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.107+0000 7fe9fadd2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f406e4b0 con 0x7fe9f4161790 2024-05-08T22:44:14.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fa5d1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe9f4161790 0x7fe9f4164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f9dd0700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe9f415ef80 0x7fe9f406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37234/0 (socket says 172.21.15.179:37234) 2024-05-08T22:44:14.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f9dd0700 1 -- 172.21.15.179:0/3729904857 learned_addr learned my addr 172.21.15.179:0/3729904857 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:14.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fa5d1700 1 -- 172.21.15.179:0/3729904857 >> v1:172.21.15.195:6789/0 conn(0x7fe9f406de70 legacy=0x7fe9f406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fa5d1700 1 -- 172.21.15.179:0/3729904857 >> v1:172.21.15.179:6789/0 conn(0x7fe9f415ef80 legacy=0x7fe9f406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f8dce700 1 -- 172.21.15.179:0/3729904857 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947373069 0 0) 0x7fe9f406ad70 con 0x7fe9f406de70 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fa5d1700 1 -- 172.21.15.179:0/3729904857 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f41654b0 con 0x7fe9f4161790 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fa5d1700 1 --2- 172.21.15.179:0/3729904857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f4161790 0x7fe9f4164db0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009850 tx=0x7fe9ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85c636711953b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f8dce700 1 -- 172.21.15.179:0/3729904857 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004230 con 0x7fe9f4161790 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f8dce700 1 -- 172.21.15.179:0/3729904857 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec004390 con 0x7fe9f4161790 2024-05-08T22:44:14.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9f8dce700 1 -- 172.21.15.179:0/3729904857 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0036c0 con 0x7fe9f4161790 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 -- 172.21.15.179:0/3729904857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f4161790 msgr2=0x7fe9f4164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/3729904857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f4161790 0x7fe9f4164db0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009850 tx=0x7fe9ec009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 -- 172.21.15.179:0/3729904857 shutdown_connections 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/3729904857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f4161790 0x7fe9f4164db0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 -- 172.21.15.179:0/3729904857 >> 172.21.15.179:0/3729904857 conn(0x7fe9f40ce9d0 msgr2=0x7fe9f40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 -- 172.21.15.179:0/3729904857 shutdown_connections 2024-05-08T22:44:14.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.111+0000 7fe9fadd2700 1 -- 172.21.15.179:0/3729904857 wait complete. 2024-05-08T22:44:14.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 Processor -- start 2024-05-08T22:44:14.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 -- start start 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9f415ef80 0x7fe9f41f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9f4161790 0x7fe9f40c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f415b530 con 0x7fe9f406de70 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c5160 con 0x7fe9f4161790 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9fadd2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40cc630 con 0x7fe9f415ef80 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.115+0000 7fe9f9dd0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54700/0 (socket says 172.21.15.179:54700) 2024-05-08T22:44:14.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 -- 172.21.15.179:0/2449461043 learned_addr learned my addr 172.21.15.179:0/2449461043 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9fa5d1700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9f4161790 0x7fe9f40c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f95cf700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9f415ef80 0x7fe9f41f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 -- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9f415ef80 msgr2=0x7fe9f41f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9f415ef80 0x7fe9f41f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 -- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9f4161790 msgr2=0x7fe9f40c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9f4161790 0x7fe9f40c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 -- 172.21.15.179:0/2449461043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec0094b0 con 0x7fe9f406de70 2024-05-08T22:44:14.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f9dd0700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c780 tx=0x7fe9e400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4004ca0 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4004e00 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f40cc910 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4007310 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f40cced0 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f4067740 con 0x7fe9f406de70 2024-05-08T22:44:14.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9e4007470 con 0x7fe9f406de70 2024-05-08T22:44:14.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 0x7fe9e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fe9e40840a0 con 0x7fe9f406de70 2024-05-08T22:44:14.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.119+0000 7fe9f95cf700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 0x7fe9e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.123+0000 7fe9f95cf700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 0x7fe9e0068bf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe9f415daa0 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.127+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe9e40511e0 con 0x7fe9f406de70 2024-05-08T22:44:14.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.335+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9f41648c0 con 0x7fe9f406de70 2024-05-08T22:44:14.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.335+0000 7fe9f27fc700 1 -- 172.21.15.179:0/2449461043 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fe9e4007720 con 0x7fe9f406de70 2024-05-08T22:44:14.341 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:14.342 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":9133,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1771492548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1771492548},{"type":"v1","addr":"172.21.15.195:6837","nonce":1771492548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:42:37.284354+0000","modified":"2024-05-08T22:44:11.051975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 msgr2=0x7fe9e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 0x7fe9e0068bf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe9f415daa0 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 msgr2=0x7fe9f415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c780 tx=0x7fe9e400ca90 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 shutdown_connections 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe9e0066730 0x7fe9e0068bf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9f4161790 0x7fe9f40c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9f415ef80 0x7fe9f41f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 --2- 172.21.15.179:0/2449461043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9f406de70 0x7fe9f415aff0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 >> 172.21.15.179:0/2449461043 conn(0x7fe9f40ce9d0 msgr2=0x7fe9f406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 shutdown_connections 2024-05-08T22:44:14.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.339+0000 7fe9fadd2700 1 -- 172.21.15.179:0/2449461043 wait complete. 2024-05-08T22:44:14.346 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 83 2024-05-08T22:44:14.357 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:14.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 Processor -- start 2024-05-08T22:44:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 -- start start 2024-05-08T22:44:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9bc0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc06abb0 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc06ae90 con 0x7ff9bc15b1b0 2024-05-08T22:44:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2d07700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc15b9c0 con 0x7ff9bc0cc700 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2506700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff9bc0cb6e0 0x7ff9bc06b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37246/0 (socket says 172.21.15.179:37246) 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c2506700 1 -- 172.21.15.179:0/2135687061 learned_addr learned my addr 172.21.15.179:0/2135687061 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c1d05700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9bc0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145094843 0 0) 0x7ff9bc06abb0 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a40035d0 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459070365 0 0) 0x7ff9bc06ae90 con 0x7ff9bc15b1b0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9bc06abb0 con 0x7ff9bc15b1b0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778655860 0 0) 0x7ff9a40035d0 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 msgr2=0x7ff9bc0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:14.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9bc0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 >> v1:172.21.15.195:6789/0 conn(0x7ff9bc15b1b0 legacy=0x7ff9bc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc15bca0 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff9b4001670 con 0x7ff9bc0cb6e0 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 >> v1:172.21.15.179:6789/0 conn(0x7ff9bc0cb6e0 legacy=0x7ff9bc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9a4003be0 0x7ff9a4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9a4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc15b9c0 con 0x7ff9a4004870 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400bae0 con 0x7ff9a4003be0 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400bca0 con 0x7ff9bc0cc700 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c1504700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9a4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.463+0000 7ff9c1d05700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9a4003be0 0x7ff9a4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2506700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 -- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 msgr2=0x7ff9a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc0cc700 0x7ff9a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 -- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9a4003be0 msgr2=0x7ff9a4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9a4003be0 0x7ff9a4003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 -- 172.21.15.179:0/2135687061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40035d0 con 0x7ff9a4004870 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c1504700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9a4004cf0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8002230 tx=0x7ff9a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a645228b70b7a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8001f30 con 0x7ff9a4004870 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b800ce60 con 0x7ff9a4004870 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c0d03700 1 -- 172.21.15.179:0/2135687061 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8003c60 con 0x7ff9a4004870 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- 172.21.15.179:0/2135687061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 msgr2=0x7ff9a4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/2135687061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9a4004cf0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8002230 tx=0x7ff9a40058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2506700 1 -- 172.21.15.179:0/2135687061 reap_dead start 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- 172.21.15.179:0/2135687061 shutdown_connections 2024-05-08T22:44:14.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- 172.21.15.179:0/2135687061 >> 172.21.15.179:0/2135687061 conn(0x7ff9bc0ce9d0 msgr2=0x7ff9bc0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:14.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- 172.21.15.179:0/2135687061 shutdown_connections 2024-05-08T22:44:14.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- 172.21.15.179:0/2135687061 wait complete. 2024-05-08T22:44:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 Processor -- start 2024-05-08T22:44:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.467+0000 7ff9c2d07700 1 -- start start 2024-05-08T22:44:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9bc1f3a40 0x7ff9bc150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc1506d0 0x7ff9bc1f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc150b80 con 0x7ff9a4004870 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc1e80f0 con 0x7ff9bc1f3a40 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc1f35b0 con 0x7ff9bc1506d0 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54708/0 (socket says 172.21.15.179:54708) 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 -- 172.21.15.179:0/598321932 learned_addr learned my addr 172.21.15.179:0/598321932 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1504700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9bc1f3a40 0x7ff9bc150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2506700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc1506d0 0x7ff9bc1f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 -- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc1506d0 msgr2=0x7ff9bc1f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc1506d0 0x7ff9bc1f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 -- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9bc1f3a40 msgr2=0x7ff9bc150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9bc1f3a40 0x7ff9bc150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 -- 172.21.15.179:0/598321932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc1f29a0 con 0x7ff9a4004870 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c1d05700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac011970 tx=0x7ff9ac011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00f750 con 0x7ff9a4004870 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac00f8b0 con 0x7ff9a4004870 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc1f2c80 con 0x7ff9a4004870 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00fc40 con 0x7ff9a4004870 2024-05-08T22:44:14.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.471+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc1f3240 con 0x7ff9a4004870 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc067740 con 0x7ff9a4004870 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9ac014440 con 0x7ff9a4004870 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9b27fc700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 0x7ff9a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff9ac084820 con 0x7ff9a4004870 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9c1504700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 0x7ff9a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.475+0000 7ff9c1504700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 0x7ff9a8068c20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005040 tx=0x7ff9b800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.479+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9ac051960 con 0x7ff9a4004870 2024-05-08T22:44:14.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.651+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9bc04e700 con 0x7ff9a4004870 2024-05-08T22:44:14.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9b27fc700 1 -- 172.21.15.179:0/598321932 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16655 (secure 0 0 0) 0x7ff9ac054f80 con 0x7ff9a4004870 2024-05-08T22:44:14.659 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:14.659 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":116,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:11.051025+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:42:35.609486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T22:42:36.273848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6836/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6837/4059323811":"2024-05-09T22:43:34.457765+0000","172.21.15.195:6836/4059323811":"2024-05-09T22:43:34.457765+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 msgr2=0x7ff9a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 0x7ff9a8068c20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005040 tx=0x7ff9b800a000 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 msgr2=0x7ff9bc14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac011970 tx=0x7ff9ac011c80 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 shutdown_connections 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff9a8066760 0x7ff9a8068c20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9bc1f3a40 0x7ff9bc150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9bc1506d0 0x7ff9bc1f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 --2- 172.21.15.179:0/598321932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a4004870 0x7ff9bc14e0a0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 >> 172.21.15.179:0/598321932 conn(0x7ff9bc0ce9d0 msgr2=0x7ff9bc0af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 shutdown_connections 2024-05-08T22:44:14.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.655+0000 7ff9c2d07700 1 -- 172.21.15.179:0/598321932 wait complete. 2024-05-08T22:44:14.674 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:44:14.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 Processor -- start 2024-05-08T22:44:14.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 -- start start 2024-05-08T22:44:14.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 0x7f08bc0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0ae0a0 con 0x7f08bc0ccdd0 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0b1cb0 con 0x7f08bc150bc0 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c2dd2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b2930 con 0x7f08bc0ad740 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c25d1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f08bc0ccdd0 0x7f08bc0cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:37250/0 (socket says 172.21.15.179:37250) 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c25d1700 1 -- 172.21.15.179:0/3002008380 learned_addr learned my addr 172.21.15.179:0/3002008380 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:14.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.783+0000 7f08c15cf700 1 --2- 172.21.15.179:0/3002008380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 0x7f08bc0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c0dce700 1 -- 172.21.15.179:0/3002008380 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771077853 0 0) 0x7f08bc0ae0a0 con 0x7f08bc0ccdd0 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c15cf700 1 -- 172.21.15.179:0/3002008380 >> v1:172.21.15.195:6789/0 conn(0x7f08bc150bc0 legacy=0x7f08bc15ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c15cf700 1 -- 172.21.15.179:0/3002008380 >> v1:172.21.15.179:6789/0 conn(0x7f08bc0ccdd0 legacy=0x7f08bc0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c15cf700 1 -- 172.21.15.179:0/3002008380 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0b1f90 con 0x7f08bc0ad740 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c15cf700 1 --2- 172.21.15.179:0/3002008380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 0x7f08bc0adb60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009dc0 tx=0x7f08b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d6eb90c8abb675 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c0dce700 1 -- 172.21.15.179:0/3002008380 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800e070 con 0x7f08bc0ad740 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c0dce700 1 -- 172.21.15.179:0/3002008380 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b800bbd0 con 0x7f08bc0ad740 2024-05-08T22:44:14.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c0dce700 1 -- 172.21.15.179:0/3002008380 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8012430 con 0x7f08bc0ad740 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3002008380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 msgr2=0x7f08bc0adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3002008380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 0x7f08bc0adb60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009dc0 tx=0x7f08b800b330 comp rx=0 tx=0).stop 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3002008380 shutdown_connections 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3002008380 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ad740 0x7f08bc0adb60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3002008380 >> 172.21.15.179:0/3002008380 conn(0x7f08bc0ce960 msgr2=0x7f08bc0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:14.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3002008380 shutdown_connections 2024-05-08T22:44:14.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3002008380 wait complete. 2024-05-08T22:44:14.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 Processor -- start 2024-05-08T22:44:14.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.787+0000 7f08c2dd2700 1 -- start start 2024-05-08T22:44:14.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0ad740 0x7f08bc151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ccdd0 0x7f08bc153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15cbf0 con 0x7f08bc150bc0 2024-05-08T22:44:14.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15c910 con 0x7f08bc0ad740 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0aeaa0 con 0x7f08bc0ccdd0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54722/0 (socket says 172.21.15.179:54722) 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 -- 172.21.15.179:0/3631135220 learned_addr learned my addr 172.21.15.179:0/3631135220 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c15cf700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ccdd0 0x7f08bc153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c1dd0700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0ad740 0x7f08bc151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 -- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ccdd0 msgr2=0x7f08bc153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ccdd0 0x7f08bc153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 -- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0ad740 msgr2=0x7f08bc151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0ad740 0x7f08bc151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 -- 172.21.15.179:0/3631135220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009a70 con 0x7f08bc150bc0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c25d1700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c550 tx=0x7f08b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4004ca0 con 0x7f08bc150bc0 2024-05-08T22:44:14.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc0aed80 con 0x7f08bc150bc0 2024-05-08T22:44:14.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc151750 con 0x7f08bc150bc0 2024-05-08T22:44:14.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4004e00 con 0x7f08bc150bc0 2024-05-08T22:44:14.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40072d0 con 0x7f08bc150bc0 2024-05-08T22:44:14.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.791+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a4005310 con 0x7f08bc150bc0 2024-05-08T22:44:14.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.795+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08b4007430 con 0x7f08bc150bc0 2024-05-08T22:44:14.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.795+0000 7f08ae7fc700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 0x7f08a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:14.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.795+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f08b40842b0 con 0x7f08bc150bc0 2024-05-08T22:44:14.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.795+0000 7f08c1dd0700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 0x7f08a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:14.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.795+0000 7f08c1dd0700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 0x7f08a0068bf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000f60 tx=0x7f08b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:14.799+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f08b40513e0 con 0x7f08bc150bc0 2024-05-08T22:44:15.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.011+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f08a4005890 con 0x7f08bc150bc0 2024-05-08T22:44:15.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.751+0000 7f08ae7fc700 1 -- 172.21.15.179:0/3631135220 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08b4054a00 con 0x7f08bc150bc0 2024-05-08T22:44:15.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 msgr2=0x7f08a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 0x7f08a0068bf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000f60 tx=0x7f08b0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 msgr2=0x7f08bc0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c550 tx=0x7f08b400c860 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 shutdown_connections 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f08a0066730 0x7f08a0068bf0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08bc150bc0 0x7f08bc0af580 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0ccdd0 0x7f08bc153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 --2- 172.21.15.179:0/3631135220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0ad740 0x7f08bc151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 >> 172.21.15.179:0/3631135220 conn(0x7f08bc0ce960 msgr2=0x7f08bc0adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 shutdown_connections 2024-05-08T22:44:15.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.755+0000 7f08c2dd2700 1 -- 172.21.15.179:0/3631135220 wait complete. 2024-05-08T22:44:15.762 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:44:15.777 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:44:15.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.899+0000 7f01a120a700 1 Processor -- start 2024-05-08T22:44:15.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.899+0000 7f01a120a700 1 -- start start 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f019c0d1970 0x7f019c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0b8c20 con 0x7f019c06c240 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c06b760 con 0x7f019c0cdca0 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019c0b8f00 con 0x7f019c0d1970 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f019c06c240 0x7f019c0d1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39436/0 (socket says 172.21.15.179:39436) 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019b7fe700 1 -- 172.21.15.179:0/3158842692 learned_addr learned my addr 172.21.15.179:0/3158842692 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:15.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a0a09700 1 --2- 172.21.15.179:0/3158842692 >> v2:172.21.15.195:3300/0 conn(0x7f019c0d1970 0x7f019c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019affd700 1 -- 172.21.15.179:0/3158842692 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213126430 0 0) 0x7f019c0b8c20 con 0x7f019c06c240 2024-05-08T22:44:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019affd700 1 -- 172.21.15.179:0/3158842692 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01880035d0 con 0x7f019c06c240 2024-05-08T22:44:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a0a09700 1 -- 172.21.15.179:0/3158842692 >> v1:172.21.15.195:6789/0 conn(0x7f019c0cdca0 legacy=0x7f019c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a0a09700 1 -- 172.21.15.179:0/3158842692 >> v1:172.21.15.179:6789/0 conn(0x7f019c06c240 legacy=0x7f019c0d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a0a09700 1 -- 172.21.15.179:0/3158842692 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c06ba40 con 0x7f019c0d1970 2024-05-08T22:44:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a0a09700 1 --2- 172.21.15.179:0/3158842692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c0d1970 0x7f019c0d1e20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0194009d90 tx=0x7f019400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bf3a897c3b4e243 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019affd700 1 -- 172.21.15.179:0/3158842692 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019400e070 con 0x7f019c0d1970 2024-05-08T22:44:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019affd700 1 -- 172.21.15.179:0/3158842692 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019400bb20 con 0x7f019c0d1970 2024-05-08T22:44:15.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f019affd700 1 -- 172.21.15.179:0/3158842692 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019400be90 con 0x7f019c0d1970 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- 172.21.15.179:0/3158842692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c0d1970 msgr2=0x7f019c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 --2- 172.21.15.179:0/3158842692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c0d1970 0x7f019c0d1e20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0194009d90 tx=0x7f019400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- 172.21.15.179:0/3158842692 shutdown_connections 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 --2- 172.21.15.179:0/3158842692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c0d1970 0x7f019c0d1e20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- 172.21.15.179:0/3158842692 >> 172.21.15.179:0/3158842692 conn(0x7f019c0d2ed0 msgr2=0x7f019c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- 172.21.15.179:0/3158842692 shutdown_connections 2024-05-08T22:44:15.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.903+0000 7f01a120a700 1 -- 172.21.15.179:0/3158842692 wait complete. 2024-05-08T22:44:15.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.907+0000 7f01a120a700 1 Processor -- start 2024-05-08T22:44:15.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.907+0000 7f01a120a700 1 -- start start 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c06c240 0x7f019c1f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019c0cdca0 0x7f019c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c1f5910 con 0x7f019c0d1970 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0b48d0 con 0x7f019c06c240 2024-05-08T22:44:15.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0ab250 con 0x7f019c0cdca0 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33620/0 (socket says 172.21.15.179:33620) 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 -- 172.21.15.179:0/3164556815 learned_addr learned my addr 172.21.15.179:0/3164556815 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f019b7fe700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019c0cdca0 0x7f019c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f019bfff700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c06c240 0x7f019c1f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 -- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019c0cdca0 msgr2=0x7f019c0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019c0cdca0 0x7f019c0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 -- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c06c240 msgr2=0x7f019c1f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c06c240 0x7f019c1f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 -- 172.21.15.179:0/3164556815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194009a70 con 0x7f019c0d1970 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a0a09700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f019400b370 tx=0x7f0194004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194004840 con 0x7f019c0d1970 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194020070 con 0x7f019c0d1970 2024-05-08T22:44:15.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940276c0 con 0x7f019c0d1970 2024-05-08T22:44:15.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0ab530 con 0x7f019c0d1970 2024-05-08T22:44:15.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c0abaf0 con 0x7f019c0d1970 2024-05-08T22:44:15.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.911+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c04e700 con 0x7f019c0d1970 2024-05-08T22:44:15.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.915+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0194012430 con 0x7f019c0d1970 2024-05-08T22:44:15.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.915+0000 7f0198ff9700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 0x7f017c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:15.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.915+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0194022070 con 0x7f019c0d1970 2024-05-08T22:44:15.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.915+0000 7f019bfff700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 0x7f017c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:15.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.915+0000 7f019bfff700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 0x7f017c068bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f018c0018a0 tx=0x7f018c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:15.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:15.923+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0194053130 con 0x7f019c0d1970 2024-05-08T22:44:16.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.135+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f019c0676e0 con 0x7f019c0d1970 2024-05-08T22:44:16.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.751+0000 7f0198ff9700 1 -- 172.21.15.179:0/3164556815 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0194056750 con 0x7f019c0d1970 2024-05-08T22:44:16.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 msgr2=0x7f017c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 0x7f017c068bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f018c0018a0 tx=0x7f018c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 msgr2=0x7f019c15b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f019400b370 tx=0x7f0194004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 shutdown_connections 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f017c066730 0x7f017c068bf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f019c0d1970 0x7f019c15b4f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019c0cdca0 0x7f019c0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 --2- 172.21.15.179:0/3164556815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f019c06c240 0x7f019c1f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 >> 172.21.15.179:0/3164556815 conn(0x7f019c0d2ed0 msgr2=0x7f019c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 shutdown_connections 2024-05-08T22:44:16.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.759+0000 7f01a120a700 1 -- 172.21.15.179:0/3164556815 wait complete. 2024-05-08T22:44:16.779 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:44:16.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 Processor -- start 2024-05-08T22:44:16.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 -- start start 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98d4143650 0x7f98d40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d40b4310 0x7f98d41445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d40cf1b0 con 0x7f98d40ced50 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d40b48f0 con 0x7f98d4143650 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.899+0000 7f98d3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d40b4bd0 con 0x7f98d40b4310 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d27fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98d40ced50 0x7f98d40b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39446/0 (socket says 172.21.15.179:39446) 2024-05-08T22:44:16.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d27fc700 1 -- 172.21.15.179:0/272966837 learned_addr learned my addr 172.21.15.179:0/272966837 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/272966837 >> v2:172.21.15.179:3300/0 conn(0x7f98d4143650 0x7f98d40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d37fe700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d40b4310 0x7f98d41445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d1ffb700 1 -- 172.21.15.179:0/272966837 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670720105 0 0) 0x7f98d40cf1b0 con 0x7f98d40ced50 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d1ffb700 1 -- 172.21.15.179:0/272966837 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c00035d0 con 0x7f98d40ced50 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 -- 172.21.15.179:0/272966837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d40b4310 msgr2=0x7f98d41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d40b4310 0x7f98d41445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 -- 172.21.15.179:0/272966837 >> v1:172.21.15.179:6789/0 conn(0x7f98d40ced50 legacy=0x7f98d40b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 -- 172.21.15.179:0/272966837 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40b4eb0 con 0x7f98d4143650 2024-05-08T22:44:16.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d4143650 0x7f98d40ce810 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009850 tx=0x7f98c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3e17035bbad4ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:16.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d1ffb700 1 -- 172.21.15.179:0/272966837 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4011830 con 0x7f98d4143650 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d1ffb700 1 -- 172.21.15.179:0/272966837 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4011990 con 0x7f98d4143650 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d1ffb700 1 -- 172.21.15.179:0/272966837 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4011d40 con 0x7f98d4143650 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 -- 172.21.15.179:0/272966837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d4143650 msgr2=0x7f98d40ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d4143650 0x7f98d40ce810 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009850 tx=0x7f98c4009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 -- 172.21.15.179:0/272966837 shutdown_connections 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d40b4310 0x7f98d41445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 --2- 172.21.15.179:0/272966837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d4143650 0x7f98d40ce810 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 -- 172.21.15.179:0/272966837 >> 172.21.15.179:0/272966837 conn(0x7f98d406f690 msgr2=0x7f98d406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 -- 172.21.15.179:0/272966837 shutdown_connections 2024-05-08T22:44:16.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.903+0000 7f98d3fff700 1 -- 172.21.15.179:0/272966837 wait complete. 2024-05-08T22:44:16.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 Processor -- start 2024-05-08T22:44:16.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 -- start start 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98d40ced50 0x7f98d40b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d4143650 0x7f98d40af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d41f2ae0 con 0x7f98d40b4310 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d40cba00 con 0x7f98d40ced50 2024-05-08T22:44:16.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d3fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d413fb30 con 0x7f98d4143650 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33632/0 (socket says 172.21.15.179:33632) 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 -- 172.21.15.179:0/699984157 learned_addr learned my addr 172.21.15.179:0/699984157 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d27fc700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98d40ced50 0x7f98d40b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d37fe700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d4143650 0x7f98d40af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 -- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d4143650 msgr2=0x7f98d40af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d4143650 0x7f98d40af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 -- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98d40ced50 msgr2=0x7f98d40b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98d40ced50 0x7f98d40b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.907+0000 7f98d2ffd700 1 -- 172.21.15.179:0/699984157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c40094b0 con 0x7f98d40b4310 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98d2ffd700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f98c4014040 tx=0x7f98c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4004510 con 0x7f98d40b4310 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c401e070 con 0x7f98d40b4310 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d41f2d20 con 0x7f98d40b4310 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4003900 con 0x7f98d40b4310 2024-05-08T22:44:16.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d41f32e0 con 0x7f98d40b4310 2024-05-08T22:44:16.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d404e770 con 0x7f98d40b4310 2024-05-08T22:44:16.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.911+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98c4003a60 con 0x7f98d40b4310 2024-05-08T22:44:16.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.915+0000 7f98cb7fe700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 0x7f98b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:16.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.915+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f98c4020070 con 0x7f98d40b4310 2024-05-08T22:44:16.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.915+0000 7f98d27fc700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 0x7f98b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:16.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.915+0000 7f98d27fc700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 0x7f98b4068bf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98d4160340 tx=0x7f98bc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:16.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:16.923+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98c4052830 con 0x7f98d40b4310 2024-05-08T22:44:17.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.091+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98d4070280 con 0x7f98d40b4310 2024-05-08T22:44:17.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.771+0000 7f98cb7fe700 1 -- 172.21.15.179:0/699984157 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7f98c4006ce0 con 0x7f98d40b4310 2024-05-08T22:44:17.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 msgr2=0x7f98b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 0x7f98b4068bf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98d4160340 tx=0x7f98bc007de0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 msgr2=0x7f98d413f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f98c4014040 tx=0x7f98c4004770 comp rx=0 tx=0).stop 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 shutdown_connections 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f98b4066730 0x7f98b4068bf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98d40ced50 0x7f98d40b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98d4143650 0x7f98d40af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 --2- 172.21.15.179:0/699984157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98d40b4310 0x7f98d413f5f0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 >> 172.21.15.179:0/699984157 conn(0x7f98d406f690 msgr2=0x7f98d40b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 shutdown_connections 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.775+0000 7f98d3fff700 1 -- 172.21.15.179:0/699984157 wait complete. 2024-05-08T22:44:17.781 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:44:17.794 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:44:17.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.915+0000 7f7d8255c700 1 Processor -- start 2024-05-08T22:44:17.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.915+0000 7f7d8255c700 1 -- start start 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d7c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0b7210 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c06b4b0 con 0x7f7d7c0ce380 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c06c130 con 0x7f7d7c0b8ad0 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7d7c0cdfa0 0x7f7d7c06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39456/0 (socket says 172.21.15.179:39456) 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7bfff700 1 -- 172.21.15.179:0/3199174463 learned_addr learned my addr 172.21.15.179:0/3199174463 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d7c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349049898 0 0) 0x7f7d7c0b7210 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d680035d0 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751758229 0 0) 0x7f7d7c06b4b0 con 0x7f7d7c0ce380 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d7c0b7210 con 0x7f7d7c0ce380 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748565174 0 0) 0x7f7d680035d0 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 msgr2=0x7f7d7c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d7c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 >> v1:172.21.15.195:6789/0 conn(0x7f7d7c0ce380 legacy=0x7f7d7c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c06b790 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f7d64001670 con 0x7f7d7c0cdfa0 2024-05-08T22:44:17.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 >> v1:172.21.15.179:6789/0 conn(0x7f7d7c0cdfa0 legacy=0x7f7d7c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d68003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d68004140 0x7f7d68004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004db0 0x7f7d68005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c06c130 con 0x7f7d68004140 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6800bb10 con 0x7f7d68004db0 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6800bcd0 con 0x7f7d7c0b8ad0 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d68004140 0x7f7d68004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d68003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d80af9700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004db0 0x7f7d68005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 msgr2=0x7f7d68003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d68003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004db0 msgr2=0x7f7d68005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004db0 0x7f7d68005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/3199174463 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680035d0 con 0x7f7d68004140 2024-05-08T22:44:17.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c0b8ad0 0x7f7d68003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d68004140 0x7f7d68004870 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009b40 tx=0x7f7d680058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa556abef25c0c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00e6c0 con 0x7f7d68004140 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00e820 con 0x7f7d68004140 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d7affd700 1 -- 172.21.15.179:0/3199174463 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00eb90 con 0x7f7d68004140 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- 172.21.15.179:0/3199174463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d68004140 msgr2=0x7f7d68004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 --2- 172.21.15.179:0/3199174463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d68004140 0x7f7d68004870 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009b40 tx=0x7f7d680058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d80af9700 1 -- 172.21.15.179:0/3199174463 reap_dead start 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- 172.21.15.179:0/3199174463 shutdown_connections 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- 172.21.15.179:0/3199174463 >> 172.21.15.179:0/3199174463 conn(0x7f7d7c0b4b50 msgr2=0x7f7d7c0b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- 172.21.15.179:0/3199174463 shutdown_connections 2024-05-08T22:44:17.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.919+0000 7f7d8255c700 1 -- 172.21.15.179:0/3199174463 wait complete. 2024-05-08T22:44:17.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 Processor -- start 2024-05-08T22:44:17.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 -- start start 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 0x7f7d7c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 0x7f7d7c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c8120 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c6420 con 0x7f7d68004140 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d8255c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c7780 con 0x7f7d7c072780 2024-05-08T22:44:17.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d7b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d7b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33660/0 (socket says 172.21.15.179:33660) 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/681147850 learned_addr learned my addr 172.21.15.179:0/681147850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d80af9700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 0x7f7d7c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.923+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 0x7f7d7c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 msgr2=0x7f7d7c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 0x7f7d7c155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 msgr2=0x7f7d7c154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 0x7f7d7c154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 -- 172.21.15.179:0/681147850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c06c6d0 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d80af9700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 0x7f7d7c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 0x7f7d7c154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d7b7fe700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800bae0 tx=0x7f7d6c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00e6c0 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c013e00 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0c7a60 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c004270 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c1580a0 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c073290 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.927+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d6c0043d0 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.931+0000 7f7d78ff9700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 0x7f7d5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:17.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.931+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f7d6c0856f0 con 0x7f7d7c0c69e0 2024-05-08T22:44:17.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.931+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 0x7f7d5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:17.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.931+0000 7f7d7bfff700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 0x7f7d5c068bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7d64001050 tx=0x7f7d64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:17.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:17.935+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7d6c052ba0 con 0x7f7d7c0c69e0 2024-05-08T22:44:18.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.103+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7d7c0d2da0 con 0x7f7d7c0c69e0 2024-05-08T22:44:18.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.779+0000 7f7d78ff9700 1 -- 172.21.15.179:0/681147850 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7d6c0561c0 con 0x7f7d7c0c69e0 2024-05-08T22:44:18.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 msgr2=0x7f7d5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 0x7f7d5c068bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7d64001050 tx=0x7f7d64009040 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 msgr2=0x7f7d7c0c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800bae0 tx=0x7f7d6c0047e0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 shutdown_connections 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7d5c066730 0x7f7d5c068bf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d7c072780 0x7f7d7c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0c69e0 0x7f7d7c0c6e60 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 --2- 172.21.15.179:0/681147850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d68004140 0x7f7d7c154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 >> 172.21.15.179:0/681147850 conn(0x7f7d7c0b4b50 msgr2=0x7f7d7c0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 shutdown_connections 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.787+0000 7f7d8255c700 1 -- 172.21.15.179:0/681147850 wait complete. 2024-05-08T22:44:18.795 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:44:18.806 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:44:18.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 Processor -- start 2024-05-08T22:44:18.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 -- start start 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0324143580 0x7f032415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032415e050 con 0x7f032415ae10 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032415b2c0 con 0x7f032406c2d0 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03296c0700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032406f0a0 con 0x7f0324143580 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03237fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f032415ae10 0x7f032406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39458/0 (socket says 172.21.15.179:39458) 2024-05-08T22:44:18.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.915+0000 7f03237fe700 1 -- 172.21.15.179:0/1080172715 learned_addr learned my addr 172.21.15.179:0/1080172715 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:18.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0323fff700 1 --2- 172.21.15.179:0/1080172715 >> v2:172.21.15.195:3300/0 conn(0x7f0324143580 0x7f032415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362885460 0 0) 0x7f032415e050 con 0x7f032415ae10 2024-05-08T22:44:18.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03040035d0 con 0x7f032415ae10 2024-05-08T22:44:18.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658355533 0 0) 0x7f032415b2c0 con 0x7f032406c2d0 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f032415e050 con 0x7f032406c2d0 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129245395 0 0) 0x7f03040035d0 con 0x7f032415ae10 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 >> v2:172.21.15.195:3300/0 conn(0x7f0324143580 msgr2=0x7f032415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 --2- 172.21.15.179:0/1080172715 >> v2:172.21.15.195:3300/0 conn(0x7f0324143580 0x7f032415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 >> v1:172.21.15.195:6789/0 conn(0x7f032406c2d0 legacy=0x7f03240cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032406edc0 con 0x7f032415ae10 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f030c001670 con 0x7f032415ae10 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0323fff700 1 --2- 172.21.15.179:0/1080172715 >> v2:172.21.15.195:3300/0 conn(0x7f0324143580 0x7f032415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 >> v1:172.21.15.179:6789/0 conn(0x7f032415ae10 legacy=0x7f032406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304003bb0 0x7f0304003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304004870 0x7f0304004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 0x7f030400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032406f0a0 con 0x7f0304005230 2024-05-08T22:44:18.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030400c160 con 0x7f0304003bb0 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030400c320 con 0x7f0304004870 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 0x7f030400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0323fff700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304003bb0 0x7f0304003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03237fe700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304004870 0x7f0304004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 -- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304004870 msgr2=0x7f0304004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304004870 0x7f0304004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 -- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304003bb0 msgr2=0x7f0304003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304003bb0 0x7f0304003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 -- 172.21.15.179:0/1080172715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040035d0 con 0x7f0304005230 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 -- 172.21.15.179:0/1080172715 reap_dead start 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0328ebf700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 0x7f030400ba30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f031c002230 tx=0x7f03040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d573f4380d194a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c001f30 con 0x7f0304005230 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c00ce80 con 0x7f0304005230 2024-05-08T22:44:18.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f0322ffd700 1 -- 172.21.15.179:0/1080172715 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c003c60 con 0x7f0304005230 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 -- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 msgr2=0x7f030400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 0x7f030400ba30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f031c002230 tx=0x7f03040058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 -- 172.21.15.179:0/1080172715 shutdown_connections 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 --2- 172.21.15.179:0/1080172715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0304005230 0x7f030400ba30 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 -- 172.21.15.179:0/1080172715 >> 172.21.15.179:0/1080172715 conn(0x7f0324070d80 msgr2=0x7f0324071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 -- 172.21.15.179:0/1080172715 shutdown_connections 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.919+0000 7f03296c0700 1 -- 172.21.15.179:0/1080172715 wait complete. 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 Processor -- start 2024-05-08T22:44:18.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 -- start start 2024-05-08T22:44:18.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005230 0x7f03240a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03240af910 0x7f0324159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240adc40 con 0x7f032413e640 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0324159970 con 0x7f0304005230 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03296c0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032413ec00 con 0x7f03240af910 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f0328ebf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f03237fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03240af910 0x7f0324159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f0328ebf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33672/0 (socket says 172.21.15.179:33672) 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f0323fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005230 0x7f03240a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.923+0000 7f0328ebf700 1 -- 172.21.15.179:0/2312931883 learned_addr learned my addr 172.21.15.179:0/2312931883 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 -- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03240af910 msgr2=0x7f0324159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03240af910 0x7f0324159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 -- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005230 msgr2=0x7f03240a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:18.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005230 0x7f03240a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:18.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 -- 172.21.15.179:0/2312931883 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032413eee0 con 0x7f032413e640 2024-05-08T22:44:18.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0328ebf700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c8b0 tx=0x7f031c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c005050 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c0051b0 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032413f1c0 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c005560 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032413f780 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0324067400 con 0x7f032413e640 2024-05-08T22:44:18.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f031c0056c0 con 0x7f032413e640 2024-05-08T22:44:18.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 0x7f0308068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:18.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f031c085420 con 0x7f032413e640 2024-05-08T22:44:18.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.927+0000 7f0323fff700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 0x7f0308068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:18.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.931+0000 7f0323fff700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 0x7f0308068bf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0314009480 tx=0x7f0314009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:18.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:18.939+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f031c052a20 con 0x7f032413e640 2024-05-08T22:44:19.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.111+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0324067400 con 0x7f032413e640 2024-05-08T22:44:19.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.119+0000 7f0320ff9700 1 -- 172.21.15.179:0/2312931883 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f031c056040 con 0x7f032413e640 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 msgr2=0x7f0308068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 0x7f0308068bf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0314009480 tx=0x7f0314009240 comp rx=0 tx=0).stop 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 msgr2=0x7f03240ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c8b0 tx=0x7f031c001650 comp rx=0 tx=0).stop 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 shutdown_connections 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0308066730 0x7f0308068bf0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f032413e640 0x7f03240ae730 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03240af910 0x7f0324159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 --2- 172.21.15.179:0/2312931883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005230 0x7f03240a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 >> 172.21.15.179:0/2312931883 conn(0x7f0324070d80 msgr2=0x7f032406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:19.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 shutdown_connections 2024-05-08T22:44:19.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.123+0000 7f03296c0700 1 -- 172.21.15.179:0/2312931883 wait complete. 2024-05-08T22:44:19.142 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-08T22:44:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.251+0000 7f82cd894700 1 Processor -- start 2024-05-08T22:44:19.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.251+0000 7f82cd894700 1 -- start start 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f82c81434d0 0x7f82c806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c806ed50 0x7f82c80b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80cd200 con 0x7f82c806dc20 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c80cd4e0 con 0x7f82c81434d0 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80cd7c0 con 0x7f82c806ed50 2024-05-08T22:44:19.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f82c806dc20 0x7f82c806e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39472/0 (socket says 172.21.15.179:39472) 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c7fff700 1 -- 172.21.15.179:0/1545440727 learned_addr learned my addr 172.21.15.179:0/1545440727 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 --2- 172.21.15.179:0/1545440727 >> v2:172.21.15.195:3300/0 conn(0x7f82c81434d0 0x7f82c806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd093700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c806ed50 0x7f82c80b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c77fe700 1 -- 172.21.15.179:0/1545440727 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467507989 0 0) 0x7f82c80cd200 con 0x7f82c806dc20 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c77fe700 1 -- 172.21.15.179:0/1545440727 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b00035d0 con 0x7f82c806dc20 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 -- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c806ed50 msgr2=0x7f82c80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c806ed50 0x7f82c80b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 -- 172.21.15.179:0/1545440727 >> v1:172.21.15.179:6789/0 conn(0x7f82c806dc20 legacy=0x7f82c806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 -- 172.21.15.179:0/1545440727 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c80cdaa0 con 0x7f82c81434d0 2024-05-08T22:44:19.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cc892700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c81434d0 0x7f82c806d6e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009d90 tx=0x7f82c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295119bc719d93ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c77fe700 1 -- 172.21.15.179:0/1545440727 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000e070 con 0x7f82c81434d0 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c77fe700 1 -- 172.21.15.179:0/1545440727 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c000bb20 con 0x7f82c81434d0 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82c77fe700 1 -- 172.21.15.179:0/1545440727 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000be90 con 0x7f82c81434d0 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c81434d0 msgr2=0x7f82c806d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c81434d0 0x7f82c806d6e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009d90 tx=0x7f82c000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- 172.21.15.179:0/1545440727 shutdown_connections 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c806ed50 0x7f82c80b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 --2- 172.21.15.179:0/1545440727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c81434d0 0x7f82c806d6e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- 172.21.15.179:0/1545440727 >> 172.21.15.179:0/1545440727 conn(0x7f82c80ce9d0 msgr2=0x7f82c80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- 172.21.15.179:0/1545440727 shutdown_connections 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.255+0000 7f82cd894700 1 -- 172.21.15.179:0/1545440727 wait complete. 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 Processor -- start 2024-05-08T22:44:19.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- start start 2024-05-08T22:44:19.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 0x7f82c81351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 0x7f82c812fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c81356e0 con 0x7f82c815b6c0 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8130190 con 0x7f82c806ed50 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c81358d0 con 0x7f82c81434d0 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33682/0 (socket says 172.21.15.179:33682) 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 -- 172.21.15.179:0/314612468 learned_addr learned my addr 172.21.15.179:0/314612468 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82c7fff700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 0x7f82c812fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cc892700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 0x7f82c81351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 -- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 msgr2=0x7f82c812fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 0x7f82c812fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 -- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 msgr2=0x7f82c81351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 0x7f82c81351a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 -- 172.21.15.179:0/314612468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0009a70 con 0x7f82c815b6c0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82c7fff700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 0x7f82c812fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cc892700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 0x7f82c81351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd093700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f82b800eb60 tx=0x7f82b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8009740 con 0x7f82c815b6c0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c8135bb0 con 0x7f82c815b6c0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b800c850 con 0x7f82c815b6c0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800cbe0 con 0x7f82c815b6c0 2024-05-08T22:44:19.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c8136170 con 0x7f82c815b6c0 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.259+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c81f0c50 con 0x7f82c815b6c0 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.263+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82b800cd40 con 0x7f82c815b6c0 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.263+0000 7f82c57fa700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 0x7f82b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.263+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f82b8084180 con 0x7f82c815b6c0 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.263+0000 7f82cc892700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 0x7f82b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.263+0000 7f82cc892700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 0x7f82b4068bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f82c0014040 tx=0x7f82c0011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.267+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f82c81f0c50 con 0x7f82c815b6c0 2024-05-08T22:44:19.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.507+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f82c81f0c50 con 0x7f82c815b6c0 2024-05-08T22:44:19.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.779+0000 7f82c57fa700 1 -- 172.21.15.179:0/314612468 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v497) v1 ==== 146+0+0 (secure 0 0 0) 0x7f82c81f0c50 con 0x7f82c815b6c0 2024-05-08T22:44:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 msgr2=0x7f82b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 0x7f82b4068bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f82c0014040 tx=0x7f82c0011000 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 msgr2=0x7f82c8133ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f82b800eb60 tx=0x7f82b800ef20 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 shutdown_connections 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f82b4066730 0x7f82b4068bf0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c815b6c0 0x7f82c8133ad0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82c81434d0 0x7f82c812fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 --2- 172.21.15.179:0/314612468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c806ed50 0x7f82c81351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.783+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 >> 172.21.15.179:0/314612468 conn(0x7f82c80ce9d0 msgr2=0x7f82c81426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.787+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 shutdown_connections 2024-05-08T22:44:19.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.787+0000 7f82cd894700 1 -- 172.21.15.179:0/314612468 wait complete. 2024-05-08T22:44:19.806 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T22:44:19.806 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T22:44:19.807 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:19.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 Processor -- start 2024-05-08T22:44:19.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 -- start start 2024-05-08T22:44:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdbc158b80 0x7fcdbc15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc0c8d70 0x7fcdbc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcdbc06c230 0x7fcdbc0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc158f60 con 0x7fcdbc158b80 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc06b860 con 0x7fcdbc06c230 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdc0f5a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc06ac50 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdba59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdbc158b80 0x7fcdbc15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdba59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdbc158b80 0x7fcdbc15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33688/0 (socket says 172.21.15.179:33688) 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdba59c700 1 -- 172.21.15.179:0/3378166820 learned_addr learned my addr 172.21.15.179:0/3378166820 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdb1d9b700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc0c8d70 0x7fcdbc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.923+0000 7fcdbad9d700 1 --2- 172.21.15.179:0/3378166820 >> v2:172.21.15.195:3300/0 conn(0x7fcdbc06c230 0x7fcdbc0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 -- 172.21.15.179:0/3378166820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc0c8d70 msgr2=0x7fcdbc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc0c8d70 0x7fcdbc06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 -- 172.21.15.179:0/3378166820 >> v2:172.21.15.195:3300/0 conn(0x7fcdbc06c230 msgr2=0x7fcdbc0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 --2- 172.21.15.179:0/3378166820 >> v2:172.21.15.195:3300/0 conn(0x7fcdbc06c230 0x7fcdbc0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 -- 172.21.15.179:0/3378166820 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc06af30 con 0x7fcdbc158b80 2024-05-08T22:44:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdba59c700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc158b80 0x7fcdbc15b4d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc06cd20 tx=0x7fcdac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cb05ece4bbbd68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdb9d9b700 1 -- 172.21.15.179:0/3378166820 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac011830 con 0x7fcdbc158b80 2024-05-08T22:44:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdb9d9b700 1 -- 172.21.15.179:0/3378166820 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac011990 con 0x7fcdbc158b80 2024-05-08T22:44:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdb9d9b700 1 -- 172.21.15.179:0/3378166820 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac011d20 con 0x7fcdbc158b80 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/3378166820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc158b80 msgr2=0x7fcdbc15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc158b80 0x7fcdbc15b4d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc06cd20 tx=0x7fcdac009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/3378166820 shutdown_connections 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/3378166820 >> v2:172.21.15.195:3300/0 conn(0x7fcdbc06c230 0x7fcdbc0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc0c8d70 0x7fcdbc06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/3378166820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc158b80 0x7fcdbc15b4d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/3378166820 >> 172.21.15.179:0/3378166820 conn(0x7fcdbc0cb660 msgr2=0x7fcdbc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/3378166820 shutdown_connections 2024-05-08T22:44:19.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.927+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/3378166820 wait complete. 2024-05-08T22:44:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 Processor -- start 2024-05-08T22:44:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 -- start start 2024-05-08T22:44:19.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdbc06c230 0x7fcdbc0c1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc158b80 0x7fcdbc1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc159200 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc154700 con 0x7fcdbc06c230 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdc0f5a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1549e0 con 0x7fcdbc158b80 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdb1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdb1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33704/0 (socket says 172.21.15.179:33704) 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdb1d9b700 1 -- 172.21.15.179:0/4029100898 learned_addr learned my addr 172.21.15.179:0/4029100898 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdbad9d700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc158b80 0x7fcdbc1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.931+0000 7fcdba59c700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdbc06c230 0x7fcdbc0c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 -- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc158b80 msgr2=0x7fcdbc1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc158b80 0x7fcdbc1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 -- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdbc06c230 msgr2=0x7fcdbc0c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdbc06c230 0x7fcdbc0c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 -- 172.21.15.179:0/4029100898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009570 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb1d9b700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ebe0 tx=0x7fcda400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4009740 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda400c850 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc154cc0 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400cbe0 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc0c1840 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc067770 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.935+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcda400cd40 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.939+0000 7fcdb37fe700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 0x7fcd9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:19.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.939+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcda40840a0 con 0x7fcdbc0c8d70 2024-05-08T22:44:19.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.939+0000 7fcdba59c700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 0x7fcd9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:19.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.939+0000 7fcdba59c700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 0x7fcd9c068c20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:19.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:19.943+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcda4051440 con 0x7fcdbc0c8d70 2024-05-08T22:44:20.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.115+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdbc073290 con 0x7fcdbc0c8d70 2024-05-08T22:44:20.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdb37fe700 1 -- 172.21.15.179:0/4029100898 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+12913 (secure 0 0 0) 0x7fcda4017090 con 0x7fcdbc0c8d70 2024-05-08T22:44:20.127 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:20.127 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":118,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:18.770407+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6836/602724347":"2024-05-09T22:44:11.050968+0000","172.21.15.195:6837/4059323811":"2024-05-09T22:43:34.457765+0000","172.21.15.195:6836/4059323811":"2024-05-09T22:43:34.457765+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:20.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 msgr2=0x7fcd9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 0x7fcd9c068c20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac008030 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 msgr2=0x7fcdbc15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ebe0 tx=0x7fcda400eef0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 shutdown_connections 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcd9c066760 0x7fcd9c068c20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcdbc158b80 0x7fcdbc1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0c8d70 0x7fcdbc15b870 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 --2- 172.21.15.179:0/4029100898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcdbc06c230 0x7fcdbc0c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 >> 172.21.15.179:0/4029100898 conn(0x7fcdbc0cb660 msgr2=0x7fcdbc15dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 shutdown_connections 2024-05-08T22:44:20.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.123+0000 7fcdc0f5a700 1 -- 172.21.15.179:0/4029100898 wait complete. 2024-05-08T22:44:20.143 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:20.144 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:20.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 Processor -- start 2024-05-08T22:44:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 -- start start 2024-05-08T22:44:20.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68d406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d406b9a0 con 0x7f68d40acfc0 2024-05-08T22:44:20.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d406bc80 con 0x7f68d406d2a0 2024-05-08T22:44:20.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68dc0a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d406def0 con 0x7f68d4161ed0 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68da63d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f68d40acfc0 0x7f68d416b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39488/0 (socket says 172.21.15.179:39488) 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68da63d700 1 -- 172.21.15.179:0/806431826 learned_addr learned my addr 172.21.15.179:0/806431826 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68d406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086683151 0 0) 0x7f68d406b9a0 con 0x7f68d40acfc0 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c00035d0 con 0x7f68d40acfc0 2024-05-08T22:44:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955918076 0 0) 0x7f68d406bc80 con 0x7f68d406d2a0 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d406b9a0 con 0x7f68d406d2a0 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106306662 0 0) 0x7f68c00035d0 con 0x7f68d40acfc0 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 msgr2=0x7f68d406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68d406cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 >> v1:172.21.15.195:6789/0 conn(0x7f68d406d2a0 legacy=0x7f68d41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d406bf60 con 0x7f68d40acfc0 2024-05-08T22:44:20.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f68cc0019f0 con 0x7f68d40acfc0 2024-05-08T22:44:20.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 >> v1:172.21.15.179:6789/0 conn(0x7f68d40acfc0 legacy=0x7f68d416b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c0003be0 0x7f68c0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68c0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c0004db0 0x7f68c0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c000bb00 con 0x7f68c0003be0 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c000bc70 con 0x7f68c0004db0 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c000be30 con 0x7f68d4161ed0 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c0003be0 0x7f68c0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.247+0000 7f68da63d700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c0004db0 0x7f68c0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d963b700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68c0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 -- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 msgr2=0x7f68c0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:20.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d4161ed0 0x7f68c0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 -- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c0004db0 msgr2=0x7f68c0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:20.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c0004db0 0x7f68c0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 -- 172.21.15.179:0/806431826 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00035d0 con 0x7f68c0003be0 2024-05-08T22:44:20.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c0003be0 0x7f68c0003fe0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009b70 tx=0x7f68c00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eecc8841b0e252d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400e6c0 con 0x7f68c0003be0 2024-05-08T22:44:20.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400e820 con 0x7f68c0003be0 2024-05-08T22:44:20.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68d8e3a700 1 -- 172.21.15.179:0/806431826 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400eb90 con 0x7f68c0003be0 2024-05-08T22:44:20.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 -- 172.21.15.179:0/806431826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c0003be0 msgr2=0x7f68c0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/806431826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c0003be0 0x7f68c0003fe0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009b70 tx=0x7f68c00058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68da63d700 1 -- 172.21.15.179:0/806431826 reap_dead start 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 -- 172.21.15.179:0/806431826 shutdown_connections 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 -- 172.21.15.179:0/806431826 >> 172.21.15.179:0/806431826 conn(0x7f68d40ceca0 msgr2=0x7f68d40cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 -- 172.21.15.179:0/806431826 shutdown_connections 2024-05-08T22:44:20.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.251+0000 7f68dc0a0700 1 -- 172.21.15.179:0/806431826 wait complete. 2024-05-08T22:44:20.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 Processor -- start 2024-05-08T22:44:20.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- start start 2024-05-08T22:44:20.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 0x7f68d41554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d40b2f30 0x7f68d41559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4155f20 con 0x7f68d409be20 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4099520 con 0x7f68d40b2b50 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4099790 con 0x7f68d40b2f30 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68d9e3c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 0x7f68d41554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33722/0 (socket says 172.21.15.179:33722) 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 -- 172.21.15.179:0/433465992 learned_addr learned my addr 172.21.15.179:0/433465992 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68d963b700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d40b2f30 0x7f68d41559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 -- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d40b2f30 msgr2=0x7f68d41559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d40b2f30 0x7f68d41559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 -- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 msgr2=0x7f68d41554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 0x7f68d41554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 -- 172.21.15.179:0/433465992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4099a70 con 0x7f68d409be20 2024-05-08T22:44:20.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 0x7f68d41554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:20.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68da63d700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f68d406e7a0 tx=0x7f68cc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00a920 con 0x7f68d409be20 2024-05-08T22:44:20.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc00aa80 con 0x7f68d409be20 2024-05-08T22:44:20.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d4099d50 con 0x7f68d409be20 2024-05-08T22:44:20.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00ae30 con 0x7f68d409be20 2024-05-08T22:44:20.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.255+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d41f71b0 con 0x7f68d409be20 2024-05-08T22:44:20.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d409a270 con 0x7f68d409be20 2024-05-08T22:44:20.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f68cc01d460 con 0x7f68d409be20 2024-05-08T22:44:20.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68ca7fc700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 0x7f68b8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f68cc084d90 con 0x7f68d409be20 2024-05-08T22:44:20.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 0x7f68b8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.259+0000 7f68d9e3c700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 0x7f68b8068c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68d4062010 tx=0x7f68c401f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.267+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f68cc052280 con 0x7f68d409be20 2024-05-08T22:44:20.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68d409a270 con 0x7f68d409be20 2024-05-08T22:44:20.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68ca7fc700 1 -- 172.21.15.179:0/433465992 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68d409a270 con 0x7f68d409be20 2024-05-08T22:44:20.439 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 msgr2=0x7f68b8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 0x7f68b8068c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68d4062010 tx=0x7f68c401f040 comp rx=0 tx=0).stop 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 msgr2=0x7f68d409c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f68d406e7a0 tx=0x7f68cc001a20 comp rx=0 tx=0).stop 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 shutdown_connections 2024-05-08T22:44:20.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f68b8066760 0x7f68b8068c20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68d409be20 0x7f68d409c2a0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68d40b2f30 0x7f68d41559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 --2- 172.21.15.179:0/433465992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68d40b2b50 0x7f68d41554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 >> 172.21.15.179:0/433465992 conn(0x7f68d40ceca0 msgr2=0x7f68d40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 shutdown_connections 2024-05-08T22:44:20.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.435+0000 7f68dc0a0700 1 -- 172.21.15.179:0/433465992 wait complete. 2024-05-08T22:44:20.455 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-08T22:44:20.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 Processor -- start 2024-05-08T22:44:20.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- start start 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe99c15d970 0x7fe99c15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c06ce80 con 0x7fe99c06eaf0 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c143a20 con 0x7fe99c143640 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c15e320 con 0x7fe99c15d970 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe9927fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe99c06eaf0 0x7fe99c0b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39502/0 (socket says 172.21.15.179:39502) 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe9927fc700 1 -- 172.21.15.179:0/1160641431 learned_addr learned my addr 172.21.15.179:0/1160641431 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99b7fe700 1 --2- 172.21.15.179:0/1160641431 >> v2:172.21.15.179:3300/0 conn(0x7fe99c15d970 0x7fe99c15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102014085 0 0) 0x7fe99c06ce80 con 0x7fe99c06eaf0 2024-05-08T22:44:20.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9880035d0 con 0x7fe99c06eaf0 2024-05-08T22:44:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422189011 0 0) 0x7fe99c143a20 con 0x7fe99c143640 2024-05-08T22:44:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe99c06ce80 con 0x7fe99c143640 2024-05-08T22:44:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99b7fe700 1 -- 172.21.15.179:0/1160641431 >> v1:172.21.15.195:6789/0 conn(0x7fe99c143640 legacy=0x7fe99c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99b7fe700 1 -- 172.21.15.179:0/1160641431 >> v1:172.21.15.179:6789/0 conn(0x7fe99c06eaf0 legacy=0x7fe99c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850718717 0 0) 0x7fe9880035d0 con 0x7fe99c06eaf0 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99b7fe700 1 -- 172.21.15.179:0/1160641431 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c0cf220 con 0x7fe99c15d970 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99b7fe700 1 --2- 172.21.15.179:0/1160641431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c15dde0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe994009d90 tx=0x7fe99400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e0e28146ecd7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400e070 con 0x7fe99c15d970 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994003a40 con 0x7fe99c15d970 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99a7fc700 1 -- 172.21.15.179:0/1160641431 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984001670 con 0x7fe99c15d970 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- 172.21.15.179:0/1160641431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 msgr2=0x7fe99c15dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 --2- 172.21.15.179:0/1160641431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c15dde0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe994009d90 tx=0x7fe99400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- 172.21.15.179:0/1160641431 shutdown_connections 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 --2- 172.21.15.179:0/1160641431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c15dde0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- 172.21.15.179:0/1160641431 >> 172.21.15.179:0/1160641431 conn(0x7fe99c0cfae0 msgr2=0x7fe99c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- 172.21.15.179:0/1160641431 shutdown_connections 2024-05-08T22:44:20.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.563+0000 7fe99bfff700 1 -- 172.21.15.179:0/1160641431 wait complete. 2024-05-08T22:44:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 Processor -- start 2024-05-08T22:44:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 -- start start 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99c06eaf0 0x7fe99c1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99c143640 0x7fe99c0a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c141760 con 0x7fe99c15d970 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c140be0 con 0x7fe99c143640 2024-05-08T22:44:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c140ec0 con 0x7fe99c06eaf0 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33732/0 (socket says 172.21.15.179:33732) 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99b7fe700 1 -- 172.21.15.179:0/3962504443 learned_addr learned my addr 172.21.15.179:0/3962504443 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe9927fc700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99c143640 0x7fe99c0a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.567+0000 7fe99affd700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99c06eaf0 0x7fe99c1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 -- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99c06eaf0 msgr2=0x7fe99c1f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99c06eaf0 0x7fe99c1f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 -- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99c143640 msgr2=0x7fe99c0a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99c143640 0x7fe99c0a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:20.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 -- 172.21.15.179:0/3962504443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994009a70 con 0x7fe99c15d970 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99b7fe700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe994015040 tx=0x7fe994004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940044c0 con 0x7fe99c15d970 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994004620 con 0x7fe99c15d970 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c1411a0 con 0x7fe99c15d970 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994012430 con 0x7fe99c15d970 2024-05-08T22:44:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c1f4750 con 0x7fe99c15d970 2024-05-08T22:44:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c1f48e0 con 0x7fe99c15d970 2024-05-08T22:44:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe994012590 con 0x7fe99c15d970 2024-05-08T22:44:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 0x7fe97c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe994085410 con 0x7fe99c15d970 2024-05-08T22:44:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.571+0000 7fe99affd700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 0x7fe97c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:20.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.575+0000 7fe99affd700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 0x7fe97c068bf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0011e0 tx=0x7fe98c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:20.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.575+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe9940529f0 con 0x7fe99c15d970 2024-05-08T22:44:20.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:20.811+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fe99c04e700 con 0x7fe99c15d970 2024-05-08T22:44:21.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.791+0000 7fe993fff700 1 -- 172.21.15.179:0/3962504443 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v499) v1 ==== 147+0+0 (secure 0 0 0) 0x7fe994056010 con 0x7fe99c15d970 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 msgr2=0x7fe97c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 0x7fe97c068bf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0011e0 tx=0x7fe98c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 msgr2=0x7fe99c152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe994015040 tx=0x7fe994004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 shutdown_connections 2024-05-08T22:44:21.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe97c066730 0x7fe97c068bf0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe99c143640 0x7fe99c0a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99c15d970 0x7fe99c152d40 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 --2- 172.21.15.179:0/3962504443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe99c06eaf0 0x7fe99c1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 >> 172.21.15.179:0/3962504443 conn(0x7fe99c0cfae0 msgr2=0x7fe99c06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 shutdown_connections 2024-05-08T22:44:21.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.795+0000 7fe99bfff700 1 -- 172.21.15.179:0/3962504443 wait complete. 2024-05-08T22:44:21.818 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T22:44:21.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 Processor -- start 2024-05-08T22:44:21.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 -- start start 2024-05-08T22:44:21.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd29c15d0c0 0x7fd29c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c15d4a0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c06abb0 con 0x7fd29c15d0c0 2024-05-08T22:44:21.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a3568700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c0d2500 con 0x7fd29c0ce2b0 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd29c15d0c0 0x7fd29c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2d67700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a1d65700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd29c0d1d20 0x7fd29c0cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:37152/0 (socket says 172.21.15.179:37152) 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd29c15d0c0 0x7fd29c0b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:47798/0 (socket says 172.21.15.179:47798) 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a1d65700 1 -- 172.21.15.179:0/1484609000 learned_addr learned my addr 172.21.15.179:0/1484609000 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 -- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 msgr2=0x7fd29c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 --2- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 -- 172.21.15.179:0/1484609000 >> v1:172.21.15.195:6789/0 conn(0x7fd29c0d1d20 legacy=0x7fd29c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a1564700 1 -- 172.21.15.179:0/1484609000 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255871132 0 0) 0x7fd29c15d4a0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 -- 172.21.15.179:0/1484609000 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0d27e0 con 0x7fd29c15d0c0 2024-05-08T22:44:21.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.943+0000 7fd2a2566700 1 --2- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c15d0c0 0x7fd29c0b8a80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd298009850 tx=0x7fd298009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d969452736d4a36b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:21.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a1564700 1 -- 172.21.15.179:0/1484609000 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298011830 con 0x7fd29c15d0c0 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a1564700 1 -- 172.21.15.179:0/1484609000 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd298011990 con 0x7fd29c15d0c0 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a1564700 1 -- 172.21.15.179:0/1484609000 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298011d40 con 0x7fd29c15d0c0 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c15d0c0 msgr2=0x7fd29c0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 --2- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c15d0c0 0x7fd29c0b8a80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd298009850 tx=0x7fd298009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- 172.21.15.179:0/1484609000 shutdown_connections 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 --2- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 --2- 172.21.15.179:0/1484609000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c15d0c0 0x7fd29c0b8a80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- 172.21.15.179:0/1484609000 >> 172.21.15.179:0/1484609000 conn(0x7fd29c0d2ed0 msgr2=0x7fd29c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- 172.21.15.179:0/1484609000 shutdown_connections 2024-05-08T22:44:21.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- 172.21.15.179:0/1484609000 wait complete. 2024-05-08T22:44:21.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 Processor -- start 2024-05-08T22:44:21.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.947+0000 7fd2a3568700 1 -- start start 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 0x7fd29c1f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c151920 con 0x7fd29c0d1d20 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1f34e0 con 0x7fd29c1f2af0 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c159b30 con 0x7fd29c0ce2b0 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a2566700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33744/0 (socket says 172.21.15.179:33744) 2024-05-08T22:44:21.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 -- 172.21.15.179:0/2721796887 learned_addr learned my addr 172.21.15.179:0/2721796887 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:21.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a2d67700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 0x7fd29c1f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 -- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 msgr2=0x7fd29c1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c1513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 -- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 msgr2=0x7fd29c1f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 0x7fd29c1f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 -- 172.21.15.179:0/2721796887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980094b0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a2d67700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 0x7fd29c1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:21.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a2566700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a1d65700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c4e0 tx=0x7fd29000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290004ca0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290004e20 con 0x7fd29c0d1d20 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c159e10 con 0x7fd29c0d1d20 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290007310 con 0x7fd29c0d1d20 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c15a3d0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.951+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c0c9f80 con 0x7fd29c0d1d20 2024-05-08T22:44:21.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.955+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd290007470 con 0x7fd29c0d1d20 2024-05-08T22:44:21.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.955+0000 7fd28effd700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 0x7fd280068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:21.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.955+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fd290083af0 con 0x7fd29c0d1d20 2024-05-08T22:44:21.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.955+0000 7fd2a2566700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 0x7fd280068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:21.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.955+0000 7fd2a2566700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 0x7fd280068bf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0d3df0 tx=0x7fd298008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:21.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:21.963+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd290051050 con 0x7fd29c0d1d20 2024-05-08T22:44:22.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.139+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd29c15c920 con 0x7fd29c0d1d20 2024-05-08T22:44:22.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.147+0000 7fd28effd700 1 -- 172.21.15.179:0/2721796887 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd290007720 con 0x7fd29c0d1d20 2024-05-08T22:44:22.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 msgr2=0x7fd280068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 0x7fd280068bf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0d3df0 tx=0x7fd298008030 comp rx=0 tx=0).stop 2024-05-08T22:44:22.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 msgr2=0x7fd29c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c4e0 tx=0x7fd29000c7f0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 shutdown_connections 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd280066730 0x7fd280068bf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c1f2af0 0x7fd29c1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd29c0d1d20 0x7fd29c15ba90 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 --2- 172.21.15.179:0/2721796887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd29c0ce2b0 0x7fd29c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 >> 172.21.15.179:0/2721796887 conn(0x7fd29c0d2ed0 msgr2=0x7fd29c1533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 shutdown_connections 2024-05-08T22:44:22.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.151+0000 7fd2a3568700 1 -- 172.21.15.179:0/2721796887 wait complete. 2024-05-08T22:44:22.169 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:44:22.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:44:22.189 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:44:22.210 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:44:22.210 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:22.210 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:22.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:44:22.217 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:44:22.217 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:22.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:44:22.261 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:22.261 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:22.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:44:22.268 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:44:22.268 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:22.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:44:22.275 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T22:44:22.275 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:22.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 Processor -- start 2024-05-08T22:44:22.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 -- start start 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f33a00d18a0 0x7f33a00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a0160f30 0x7f33a01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a00d2640 con 0x7f33a015d580 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0161900 con 0x7f33a00d18a0 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a006b3a0 con 0x7f33a0160f30 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f33a00d18a0 0x7f33a00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f33a00d18a0 0x7f33a00d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33748/0 (socket says 172.21.15.179:33748) 2024-05-08T22:44:22.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 -- 172.21.15.179:0/3301573655 learned_addr learned my addr 172.21.15.179:0/3301573655 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339e59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f33a015d580 0x7f33a00b6210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39516/0 (socket says 172.21.15.179:39516) 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339ed9d700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a0160f30 0x7f33a01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 -- 172.21.15.179:0/3301573655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a0160f30 msgr2=0x7f33a01613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a0160f30 0x7f33a01613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 -- 172.21.15.179:0/3301573655 >> v1:172.21.15.179:6789/0 conn(0x7f33a015d580 legacy=0x7f33a00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 -- 172.21.15.179:0/3301573655 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00d2360 con 0x7f33a00d18a0 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339d59a700 1 -- 172.21.15.179:0/3301573655 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461418612 0 0) 0x7f33a00d2640 con 0x7f33a015d580 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339dd9b700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a00d1d30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f3388009910 tx=0x7f3388009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9961d59767383c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.387+0000 7f339d59a700 1 -- 172.21.15.179:0/3301573655 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800ba60 con 0x7f33a00d18a0 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339d59a700 1 -- 172.21.15.179:0/3301573655 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800bbc0 con 0x7f33a00d18a0 2024-05-08T22:44:22.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339d59a700 1 -- 172.21.15.179:0/3301573655 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388009e60 con 0x7f33a00d18a0 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 -- 172.21.15.179:0/3301573655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 msgr2=0x7f33a00d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a00d1d30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f3388009910 tx=0x7f3388009c20 comp rx=0 tx=0).stop 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 -- 172.21.15.179:0/3301573655 shutdown_connections 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a0160f30 0x7f33a01613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 --2- 172.21.15.179:0/3301573655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a00d1d30 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 -- 172.21.15.179:0/3301573655 >> 172.21.15.179:0/3301573655 conn(0x7f33a00d2d90 msgr2=0x7f33a00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:22.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 -- 172.21.15.179:0/3301573655 shutdown_connections 2024-05-08T22:44:22.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.391+0000 7f339f59e700 1 -- 172.21.15.179:0/3301573655 wait complete. 2024-05-08T22:44:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 Processor -- start 2024-05-08T22:44:22.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 -- start start 2024-05-08T22:44:22.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a015d580 0x7f33a00b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a0160f30 0x7f33a015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00b2d30 con 0x7f33a00d18a0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015f540 con 0x7f33a0160f30 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0159be0 con 0x7f33a015d580 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33756/0 (socket says 172.21.15.179:33756) 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 -- 172.21.15.179:0/2153558115 learned_addr learned my addr 172.21.15.179:0/2153558115 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339dd9b700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a015d580 0x7f33a00b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339ed9d700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a0160f30 0x7f33a015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 -- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a015d580 msgr2=0x7f33a00b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a015d580 0x7f33a00b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 -- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a0160f30 msgr2=0x7f33a015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a0160f30 0x7f33a015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 -- 172.21.15.179:0/2153558115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388009570 con 0x7f33a00d18a0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f339e59c700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3390001f80 tx=0x7f339000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.395+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000acd0 con 0x7f33a00d18a0 2024-05-08T22:44:22.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339000ae50 con 0x7f33a00d18a0 2024-05-08T22:44:22.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a0159ec0 con 0x7f33a00d18a0 2024-05-08T22:44:22.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390007310 con 0x7f33a00d18a0 2024-05-08T22:44:22.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0160260 con 0x7f33a00d18a0 2024-05-08T22:44:22.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0160450 con 0x7f33a00d18a0 2024-05-08T22:44:22.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.399+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3390007470 con 0x7f33a00d18a0 2024-05-08T22:44:22.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.403+0000 7f3396ffd700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 0x7f3380068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.403+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f3390083dc0 con 0x7f33a00d18a0 2024-05-08T22:44:22.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.403+0000 7f339dd9b700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 0x7f3380068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.403+0000 7f339dd9b700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 0x7f3380068bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3388000c00 tx=0x7f3388008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.407+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3390051280 con 0x7f33a00d18a0 2024-05-08T22:44:22.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.619+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a006b810 con 0x7f33a00d18a0 2024-05-08T22:44:22.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.619+0000 7f3396ffd700 1 -- 172.21.15.179:0/2153558115 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f33a006b810 con 0x7f33a00d18a0 2024-05-08T22:44:22.626 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:22.626 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1440009211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1440009211},{"type":"v1","addr":"172.21.15.179:6833","nonce":1440009211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4005938206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":4005938206},{"type":"v1","addr":"172.21.15.179:6835","nonce":4005938206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/468342587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":468342587},{"type":"v1","addr":"172.21.15.195:6835","nonce":468342587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":9133,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1771492548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1771492548},{"type":"v1","addr":"172.21.15.195:6837","nonce":1771492548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2024-05-08T22:44:22.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 msgr2=0x7f3380068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 0x7f3380068bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3388000c00 tx=0x7f3388008030 comp rx=0 tx=0).stop 2024-05-08T22:44:22.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 msgr2=0x7f33a014c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3390001f80 tx=0x7f339000a340 comp rx=0 tx=0).stop 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 shutdown_connections 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3380066730 0x7f3380068bf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a0160f30 0x7f33a015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a015d580 0x7f33a00b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 --2- 172.21.15.179:0/2153558115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a00d18a0 0x7f33a014c140 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 >> 172.21.15.179:0/2153558115 conn(0x7f33a00d2d90 msgr2=0x7f33a00cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 shutdown_connections 2024-05-08T22:44:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.623+0000 7f339f59e700 1 -- 172.21.15.179:0/2153558115 wait complete. 2024-05-08T22:44:22.630 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 85 2024-05-08T22:44:22.641 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T22:44:22.641 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T22:44:22.641 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:44:22.641 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:22.642 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T22:44:22.642 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T22:44:22.642 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:44:22.642 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:22.642 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T22:44:22.642 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T22:44:22.643 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:44:22.643 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:22.643 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T22:44:22.643 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T22:44:22.643 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:44:22.643 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:22.656 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:44:22.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T22:44:22.657 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:44:22.657 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T22:44:22.658 INFO:tasks.ceph.mds.c:Started 2024-05-08T22:44:22.659 DEBUG:teuthology.parallel:result is None 2024-05-08T22:44:22.660 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:44:22.660 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T22:44:22.660 INFO:tasks.ceph.mds.a:Started 2024-05-08T22:44:22.661 DEBUG:teuthology.parallel:result is None 2024-05-08T22:44:22.661 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:44:22.661 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T22:44:22.703 INFO:tasks.ceph.mds.b:Started 2024-05-08T22:44:22.704 DEBUG:teuthology.parallel:result is None 2024-05-08T22:44:22.704 INFO:tasks.ceph.mds.d:Started 2024-05-08T22:44:22.704 DEBUG:teuthology.parallel:result is None 2024-05-08T22:44:22.705 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T22:44:22.734 INFO:tasks.ceph.mds.c.smithi179.stdout:starting mds.c at 2024-05-08T22:44:22.734 INFO:tasks.ceph.mds.a.smithi179.stdout:starting mds.a at 2024-05-08T22:44:22.785 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:44:22.786 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-05-08T22:44:22.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 Processor -- start 2024-05-08T22:44:22.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 -- start start 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc15d1e0 0x7ff5bc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc15d5c0 con 0x7ff5bc0d29b0 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0d1640 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0d1910 con 0x7ff5bc15d1e0 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33806/0 (socket says 172.21.15.179:33806) 2024-05-08T22:44:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 -- 172.21.15.179:0/828033134 learned_addr learned my addr 172.21.15.179:0/828033134 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:22.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5ba59c700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc15d1e0 0x7ff5bc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 -- 172.21.15.179:0/828033134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc15d1e0 msgr2=0x7ff5bc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:22.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc15d1e0 0x7ff5bc0d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 -- 172.21.15.179:0/828033134 >> v1:172.21.15.179:6789/0 conn(0x7ff5bc0d29b0 legacy=0x7ff5bc0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 -- 172.21.15.179:0/828033134 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0d1bf0 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5b959a700 1 -- 172.21.15.179:0/828033134 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190386276 0 0) 0x7ff5bc15d5c0 con 0x7ff5bc0d29b0 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5bad9d700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009e50 tx=0x7ff5b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6d80e12a9e29b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5b959a700 1 -- 172.21.15.179:0/828033134 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400e070 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5b959a700 1 -- 172.21.15.179:0/828033134 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400b910 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.811+0000 7ff5b959a700 1 -- 172.21.15.179:0/828033134 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400bcc0 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/828033134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 msgr2=0x7ff5bc0b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009e50 tx=0x7ff5b400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/828033134 shutdown_connections 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0b6610 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/828033134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc15d1e0 0x7ff5bc0d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/828033134 >> 172.21.15.179:0/828033134 conn(0x7ff5bc0d3400 msgr2=0x7ff5bc0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/828033134 shutdown_connections 2024-05-08T22:44:22.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/828033134 wait complete. 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 Processor -- start 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- start start 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0d29b0 0x7ff5bc0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc15d1e0 0x7ff5bc15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0cb310 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0c7f70 con 0x7ff5bc15d1e0 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15ab40 con 0x7ff5bc0d29b0 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33812/0 (socket says 172.21.15.179:33812) 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 -- 172.21.15.179:0/485774785 learned_addr learned my addr 172.21.15.179:0/485774785 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:22.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5bad9d700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc15d1e0 0x7ff5bc15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5b9d9b700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0d29b0 0x7ff5bc0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 -- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0d29b0 msgr2=0x7ff5bc0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0d29b0 0x7ff5bc0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 -- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc15d1e0 msgr2=0x7ff5bc15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc15d1e0 0x7ff5bc15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.815+0000 7ff5ba59c700 1 -- 172.21.15.179:0/485774785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b4009b30 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5ba59c700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e870 tx=0x7ff5ac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac013680 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc0cdf00 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc0cdc20 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac005a40 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac005db0 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a0005310 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.819+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5ac0137e0 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.823+0000 7ff5aaffd700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 0x7ff59c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:22.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.823+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff5ac083f80 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.823+0000 7ff5b9d9b700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 0x7ff59c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:22.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.823+0000 7ff5b9d9b700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 0x7ff59c068bf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc1f5df0 tx=0x7ff5b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:22.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:22.827+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5ac0b5cf0 con 0x7ff5bc0b8a40 2024-05-08T22:44:22.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:44:22.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:44:23.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.015+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff5a00055c0 con 0x7ff5bc0b8a40 2024-05-08T22:44:23.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.811+0000 7ff5aaffd700 1 -- 172.21.15.179:0/485774785 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff5ac0513d0 con 0x7ff5bc0b8a40 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 msgr2=0x7ff59c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 0x7ff59c068bf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc1f5df0 tx=0x7ff5b0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 msgr2=0x7ff5bc0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e870 tx=0x7ff5ac00eb80 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 shutdown_connections 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff59c066730 0x7ff59c068bf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5bc15d1e0 0x7ff5bc15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5bc0d29b0 0x7ff5bc0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 --2- 172.21.15.179:0/485774785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5bc0b8a40 0x7ff5bc0cd2b0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 >> 172.21.15.179:0/485774785 conn(0x7ff5bc0d3400 msgr2=0x7ff5bc15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:23.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 shutdown_connections 2024-05-08T22:44:23.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.815+0000 7ff5bb59e700 1 -- 172.21.15.179:0/485774785 wait complete. 2024-05-08T22:44:23.823 INFO:teuthology.orchestra.run.smithi179.stderr: removed all blocklist entries 2024-05-08T22:44:23.839 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T22:44:23.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 Processor -- start 2024-05-08T22:44:23.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 -- start start 2024-05-08T22:44:23.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 0x7f4694161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 0x7f469406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4694165d50 con 0x7f4694164db0 2024-05-08T22:44:23.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469406ad70 con 0x7f469406de70 2024-05-08T22:44:23.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469b05b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4694165230 con 0x7f469415ef80 2024-05-08T22:44:23.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 0x7f4694161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a85a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 0x7f469406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699858700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f4694164db0 0x7f469406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:37166/0 (socket says 172.21.15.179:37166) 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699858700 1 -- 172.21.15.179:0/1100696425 learned_addr learned my addr 172.21.15.179:0/1100696425 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699057700 1 -- 172.21.15.179:0/1100696425 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866435871 0 0) 0x7f4694165d50 con 0x7f4694164db0 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699057700 1 -- 172.21.15.179:0/1100696425 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46800035d0 con 0x7f4694164db0 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 -- 172.21.15.179:0/1100696425 >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 msgr2=0x7f469406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 --2- 172.21.15.179:0/1100696425 >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 0x7f469406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 -- 172.21.15.179:0/1100696425 >> v1:172.21.15.195:6789/0 conn(0x7f4694164db0 legacy=0x7f469406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 -- 172.21.15.179:0/1100696425 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469406e4b0 con 0x7f469415ef80 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a85a700 1 --2- 172.21.15.179:0/1100696425 >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 0x7f469406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f469a059700 1 --2- 172.21.15.179:0/1100696425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 0x7f4694161790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4690009850 tx=0x7f4690009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c91ed958ed92e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699057700 1 -- 172.21.15.179:0/1100696425 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690011830 con 0x7f469415ef80 2024-05-08T22:44:23.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699057700 1 -- 172.21.15.179:0/1100696425 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4690011990 con 0x7f469415ef80 2024-05-08T22:44:23.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.955+0000 7f4699057700 1 -- 172.21.15.179:0/1100696425 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690011d40 con 0x7f469415ef80 2024-05-08T22:44:23.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- 172.21.15.179:0/1100696425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 msgr2=0x7f4694161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- 172.21.15.179:0/1100696425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 0x7f4694161790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4690009850 tx=0x7f4690009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:23.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- 172.21.15.179:0/1100696425 shutdown_connections 2024-05-08T22:44:23.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- 172.21.15.179:0/1100696425 >> v2:172.21.15.195:3300/0 conn(0x7f469406de70 0x7f469406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- 172.21.15.179:0/1100696425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f469415ef80 0x7f4694161790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- 172.21.15.179:0/1100696425 >> 172.21.15.179:0/1100696425 conn(0x7f46940ce9d0 msgr2=0x7f46940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:23.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- 172.21.15.179:0/1100696425 shutdown_connections 2024-05-08T22:44:23.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- 172.21.15.179:0/1100696425 wait complete. 2024-05-08T22:44:23.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 Processor -- start 2024-05-08T22:44:23.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 -- start start 2024-05-08T22:44:23.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 0x7f46941585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.959+0000 7f469b05b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4694164db0 0x7f4694157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694158b00 con 0x7f469406de70 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694158d40 con 0x7f469415ef80 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4694159020 con 0x7f4694164db0 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33822/0 (socket says 172.21.15.179:33822) 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 -- 172.21.15.179:0/156808438 learned_addr learned my addr 172.21.15.179:0/156808438 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f4699858700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 0x7f46941585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a85a700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4694164db0 0x7f4694157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 -- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4694164db0 msgr2=0x7f4694157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4694164db0 0x7f4694157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 -- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 msgr2=0x7f46941585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:23.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 0x7f46941585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 -- 172.21.15.179:0/156808438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c009a70 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f4699858700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 0x7f46941585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469a059700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f46900090b0 tx=0x7f4690004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690004480 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469001f070 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46900094b0 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690003900 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46941595e0 con 0x7f469406de70 2024-05-08T22:44:23.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694067740 con 0x7f469406de70 2024-05-08T22:44:23.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.963+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4690021030 con 0x7f469406de70 2024-05-08T22:44:23.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.967+0000 7f468affd700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 0x7f4678068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:23.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.967+0000 7f4699858700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 0x7f4678068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:23.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.967+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4690085270 con 0x7f469406de70 2024-05-08T22:44:23.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.967+0000 7f4699858700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 0x7f4678068bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4694159d80 tx=0x7f4684009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:23.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:23.975+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4690052900 con 0x7f469406de70 2024-05-08T22:44:24.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.207+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f46940cf930 con 0x7f469406de70 2024-05-08T22:44:24.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.207+0000 7f468affd700 1 -- 172.21.15.179:0/156808438 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4690006ce0 con 0x7f469406de70 2024-05-08T22:44:24.214 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:24.214 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "auth_dump": [ 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.a", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmA4pTJxAA34PkOeP9OJsp8yODeDdhnA==", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.b", 2024-05-08T22:44:24.215 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJPWnLhAABTcqnLoOb1Jc/eFVRo0viA==", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.c", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmKffeKhAACuQQnpp4IDoIz6Fci87U0g==", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.216 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mds.d", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmQcZQMhAAJD/JL7OepLXyk/1Lne4lmg==", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.0", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB5/TtmMuRBKBAAxN/O8xIeWM+Ex0sFBYRpAg==", 2024-05-08T22:44:24.217 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.1", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB8/Ttm8XaWMBAAJJfMUF4AGbbWcgcjmCvmTg==", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.218 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.2", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB//TtmxphKMxAAH9iwEc1+Wg2i+uNSoHBTww==", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.3", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCC/Ttm7Jq5OhAARhiETNgH8DqRsOEuen/Xtg==", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.219 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.4", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCG/Ttm8KgzJBAAXGwvzVcm5Jt2iiQTUuIftA==", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.5", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCJ/TtmbkYnKhAA5a4ByfiHnJuAwedPm/pxcg==", 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.220 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.6", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCM/TtmBTCGLRAAKYTyVrZOWPMQ+Vxg+aAd4Q==", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "osd.7", 2024-05-08T22:44:24.221 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQCP/TtmyblXNBAActH8IAeloG4tSw510Q7UGQ==", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow profile osd", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile osd", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.0", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmEgnNNRAAbtdpOQnx406jSoBUtzGQbA==", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow r", 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.222 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.1", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmaAJlABAAHoAu9kx2kPpBvNnwRLSi/Q==", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.2", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmJQPaOBAANBrg5g6ZGXsGWoV2VbpvrA==", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.223 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.3", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB3/TtmRbgFBhAAOBec0U719prdDa0FdOYTHw==", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow r", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow rw", 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow rwx" 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.224 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "client.admin", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/Ttmww9wFhAAGOTMnhlZZqAIWXycSkY/Yw==", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "mgr": "allow *", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow *", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.x", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmhgmrIxAAHpA2/nwic/FGK5o1GrTK6g==", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:44:24.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "entity": "mgr.y", 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "key": "AQB2/TtmsRGDIBAAHZUu0QSvZvzt+vHZC3mwgw==", 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": { 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": "allow *", 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "mon": "allow profile mgr", 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd": "allow *" 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-08T22:44:24.226 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:24.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 msgr2=0x7f4678068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 0x7f4678068bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4694159d80 tx=0x7f4684009040 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 msgr2=0x7f469415ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f46900090b0 tx=0x7f4690004770 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 shutdown_connections 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4678066730 0x7f4678068bf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469415ef80 0x7f46941585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4694164db0 0x7f4694157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 --2- 172.21.15.179:0/156808438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469406de70 0x7f469415ab10 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 >> 172.21.15.179:0/156808438 conn(0x7f46940ce9d0 msgr2=0x7f469406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.211+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 shutdown_connections 2024-05-08T22:44:24.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.215+0000 7f469b05b700 1 -- 172.21.15.179:0/156808438 wait complete. 2024-05-08T22:44:24.234 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:24.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 Processor -- start 2024-05-08T22:44:24.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 -- start start 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0340b19e0 0x7ff0340b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0340b3550 0x7ff03415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340b3980 con 0x7ff0341539c0 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff034159b60 con 0x7ff0340b3550 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff033fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff034159e40 con 0x7ff0340b19e0 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0340b3550 0x7ff03415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0340b3550 0x7ff03415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33834/0 (socket says 172.21.15.179:33834) 2024-05-08T22:44:24.346 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 -- 172.21.15.179:0/2179578362 learned_addr learned my addr 172.21.15.179:0/2179578362 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:24.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0327fc700 1 --2- 172.21.15.179:0/2179578362 >> v2:172.21.15.195:3300/0 conn(0x7ff0340b19e0 0x7ff0340b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 -- 172.21.15.179:0/2179578362 >> v2:172.21.15.195:3300/0 conn(0x7ff0340b19e0 msgr2=0x7ff0340b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:24.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 --2- 172.21.15.179:0/2179578362 >> v2:172.21.15.195:3300/0 conn(0x7ff0340b19e0 0x7ff0340b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 -- 172.21.15.179:0/2179578362 >> v1:172.21.15.195:6789/0 conn(0x7ff0341539c0 legacy=0x7ff03406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.347 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.339+0000 7ff0337fe700 1 -- 172.21.15.179:0/2179578362 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03415a120 con 0x7ff0340b3550 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff0337fe700 1 --2- 172.21.15.179:0/2179578362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0340b3550 0x7ff03415d120 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff0340708c0 tx=0x7ff02c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faed25a9e2fcea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff031ffb700 1 -- 172.21.15.179:0/2179578362 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c011830 con 0x7ff0340b3550 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff031ffb700 1 -- 172.21.15.179:0/2179578362 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c011990 con 0x7ff0340b3550 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff031ffb700 1 -- 172.21.15.179:0/2179578362 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c011d20 con 0x7ff0340b3550 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- 172.21.15.179:0/2179578362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0340b3550 msgr2=0x7ff03415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 --2- 172.21.15.179:0/2179578362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0340b3550 0x7ff03415d120 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff0340708c0 tx=0x7ff02c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- 172.21.15.179:0/2179578362 shutdown_connections 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 --2- 172.21.15.179:0/2179578362 >> v2:172.21.15.195:3300/0 conn(0x7ff0340b19e0 0x7ff0340b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 --2- 172.21.15.179:0/2179578362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0340b3550 0x7ff03415d120 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- 172.21.15.179:0/2179578362 >> 172.21.15.179:0/2179578362 conn(0x7ff0340ce9d0 msgr2=0x7ff0340cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- 172.21.15.179:0/2179578362 shutdown_connections 2024-05-08T22:44:24.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- 172.21.15.179:0/2179578362 wait complete. 2024-05-08T22:44:24.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 Processor -- start 2024-05-08T22:44:24.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.343+0000 7ff033fff700 1 -- start start 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0340b19e0 0x7ff034150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 0x7ff0340afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341517e0 con 0x7ff0341539c0 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034151ac0 con 0x7ff0340b3550 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff034151da0 con 0x7ff0340b19e0 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33842/0 (socket says 172.21.15.179:33842) 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0337fe700 1 -- 172.21.15.179:0/4154738412 learned_addr learned my addr 172.21.15.179:0/4154738412 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:24.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 0x7ff0340afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff032ffd700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0340b19e0 0x7ff034150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 -- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0340b19e0 msgr2=0x7ff034150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0340b19e0 0x7ff034150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 -- 172.21.15.179:0/4154738412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 msgr2=0x7ff034152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 -- 172.21.15.179:0/4154738412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c009570 con 0x7ff0340b3550 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0337fe700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff0327fc700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 0x7ff0340afbf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb90 tx=0x7ff02400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400ca80 con 0x7ff0340b3550 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02400f040 con 0x7ff0340b3550 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400ce00 con 0x7ff0340b3550 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034152080 con 0x7ff0340b3550 2024-05-08T22:44:24.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034151490 con 0x7ff0340b3550 2024-05-08T22:44:24.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.347+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034067740 con 0x7ff0340b3550 2024-05-08T22:44:24.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.351+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff024014480 con 0x7ff0340b3550 2024-05-08T22:44:24.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.351+0000 7ff02b7fe700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 0x7ff014068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.351+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff024084f40 con 0x7ff0340b3550 2024-05-08T22:44:24.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.351+0000 7ff032ffd700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 0x7ff014068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.351+0000 7ff032ffd700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 0x7ff014068bf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff03415c730 tx=0x7ff01c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.359+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff0240524a0 con 0x7ff0340b3550 2024-05-08T22:44:24.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.531+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff03404e700 con 0x7ff0340b3550 2024-05-08T22:44:24.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.535+0000 7ff02b7fe700 1 -- 172.21.15.179:0/4154738412 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+12652 (secure 0 0 0) 0x7ff024055ac0 con 0x7ff0340b3550 2024-05-08T22:44:24.540 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:24.540 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":123,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:23.808783+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:24.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.535+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 msgr2=0x7ff014068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.535+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 0x7ff014068bf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff03415c730 tx=0x7ff01c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:24.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.535+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 msgr2=0x7ff0340afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.535+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 0x7ff0340afbf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb90 tx=0x7ff02400eea0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 shutdown_connections 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff014066730 0x7ff014068bf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0340b3550 0x7ff0340afbf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0341539c0 0x7ff034152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 --2- 172.21.15.179:0/4154738412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0340b19e0 0x7ff034150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 >> 172.21.15.179:0/4154738412 conn(0x7ff0340ce9d0 msgr2=0x7ff0340cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 shutdown_connections 2024-05-08T22:44:24.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.539+0000 7ff033fff700 1 -- 172.21.15.179:0/4154738412 wait complete. 2024-05-08T22:44:24.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:24.556 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:24.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 Processor -- start 2024-05-08T22:44:24.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 -- start start 2024-05-08T22:44:24.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3a4c15b190 0x7f3a4c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a4c0cbce0 0x7f3a4c0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0b6f20 con 0x7f3a4c0b4040 2024-05-08T22:44:24.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0cc6b0 con 0x7f3a4c15b190 2024-05-08T22:44:24.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a5102b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0b4820 con 0x7f3a4c0cbce0 2024-05-08T22:44:24.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3a4c15b190 0x7f3a4c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3a4c15b190 0x7f3a4c0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33846/0 (socket says 172.21.15.179:33846) 2024-05-08T22:44:24.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/3301641267 learned_addr learned my addr 172.21.15.179:0/3301641267 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:24.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4b59e700 1 --2- 172.21.15.179:0/3301641267 >> v2:172.21.15.195:3300/0 conn(0x7f3a4c0cbce0 0x7f3a4c0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/3301641267 >> v2:172.21.15.195:3300/0 conn(0x7f3a4c0cbce0 msgr2=0x7f3a4c0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:24.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 --2- 172.21.15.179:0/3301641267 >> v2:172.21.15.195:3300/0 conn(0x7f3a4c0cbce0 0x7f3a4c0cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/3301641267 >> v1:172.21.15.195:6789/0 conn(0x7f3a4c0b4040 legacy=0x7f3a4c0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/3301641267 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0b4b00 con 0x7f3a4c15b190 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.671+0000 7f3a4ad9d700 1 --2- 172.21.15.179:0/3301641267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c0b69e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0cb4e0 tx=0x7f3a34009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c5b21089475816 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a49d9b700 1 -- 172.21.15.179:0/3301641267 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34011830 con 0x7f3a4c15b190 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a49d9b700 1 -- 172.21.15.179:0/3301641267 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34011990 con 0x7f3a4c15b190 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a49d9b700 1 -- 172.21.15.179:0/3301641267 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34011d20 con 0x7f3a4c15b190 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- 172.21.15.179:0/3301641267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 msgr2=0x7f3a4c0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 --2- 172.21.15.179:0/3301641267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c0b69e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0cb4e0 tx=0x7f3a34009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- 172.21.15.179:0/3301641267 shutdown_connections 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 --2- 172.21.15.179:0/3301641267 >> v2:172.21.15.195:3300/0 conn(0x7f3a4c0cbce0 0x7f3a4c0cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 --2- 172.21.15.179:0/3301641267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c0b69e0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- 172.21.15.179:0/3301641267 >> 172.21.15.179:0/3301641267 conn(0x7f3a4c0d0f10 msgr2=0x7f3a4c0d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- 172.21.15.179:0/3301641267 shutdown_connections 2024-05-08T22:44:24.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- 172.21.15.179:0/3301641267 wait complete. 2024-05-08T22:44:24.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 Processor -- start 2024-05-08T22:44:24.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.675+0000 7f3a5102b700 1 -- start start 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c0cbce0 0x7f3a4c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0a7900 con 0x7f3a4c15b190 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c15cde0 con 0x7f3a4c0b4040 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c7840 con 0x7f3a4c0cbce0 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33848/0 (socket says 172.21.15.179:33848) 2024-05-08T22:44:24.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:47928/0 (socket says 172.21.15.179:47928) 2024-05-08T22:44:24.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4b59e700 1 -- 172.21.15.179:0/26401559 learned_addr learned my addr 172.21.15.179:0/26401559 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:24.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4a59c700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c0cbce0 0x7f3a4c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c0cbce0 msgr2=0x7f3a4c15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c0cbce0 0x7f3a4c15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/26401559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 msgr2=0x7f3a4c1e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 -- 172.21.15.179:0/26401559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34009570 con 0x7f3a4c0b4040 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4b59e700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a4ad9d700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3a34016000 tx=0x7f3a34004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34004510 con 0x7f3a4c0b4040 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3401e070 con 0x7f3a4c0b4040 2024-05-08T22:44:24.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c15c1d0 con 0x7f3a4c0b4040 2024-05-08T22:44:24.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34003900 con 0x7f3a4c0b4040 2024-05-08T22:44:24.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.679+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c15c790 con 0x7f3a4c0b4040 2024-05-08T22:44:24.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c0cd470 con 0x7f3a4c0b4040 2024-05-08T22:44:24.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a34003a60 con 0x7f3a4c0b4040 2024-05-08T22:44:24.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a437fe700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 0x7f3a2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3a34020070 con 0x7f3a4c0b4040 2024-05-08T22:44:24.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a4a59c700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 0x7f3a2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.683+0000 7f3a4a59c700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 0x7f3a2c068bf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c159ce0 tx=0x7f3a3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.691+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3a340525c0 con 0x7f3a4c0b4040 2024-05-08T22:44:24.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.859+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a4c0cd710 con 0x7f3a4c0b4040 2024-05-08T22:44:24.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.859+0000 7f3a437fe700 1 -- 172.21.15.179:0/26401559 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a34055be0 con 0x7f3a4c0b4040 2024-05-08T22:44:24.866 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:24.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 msgr2=0x7f3a2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 0x7f3a2c068bf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c159ce0 tx=0x7f3a3c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 msgr2=0x7f3a4c0c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3a34016000 tx=0x7f3a34004770 comp rx=0 tx=0).stop 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 shutdown_connections 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3a2c066730 0x7f3a2c068bf0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c15b190 0x7f3a4c1e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a4c0cbce0 0x7f3a4c15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 --2- 172.21.15.179:0/26401559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c0b4040 0x7f3a4c0c7300 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 >> 172.21.15.179:0/26401559 conn(0x7f3a4c0d0f10 msgr2=0x7f3a4c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 shutdown_connections 2024-05-08T22:44:24.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.863+0000 7f3a5102b700 1 -- 172.21.15.179:0/26401559 wait complete. 2024-05-08T22:44:24.881 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:44:24.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 Processor -- start 2024-05-08T22:44:24.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 -- start start 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3db815b150 0x7f3db80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db80ae740 0x7f3db80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80acbc0 con 0x7f3db80af2c0 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80af750 con 0x7f3db815b150 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbe90c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80ae060 con 0x7f3db80ae740 2024-05-08T22:44:24.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbd90a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3db815b150 0x7f3db80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbd90a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3db815b150 0x7f3db80b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33850/0 (socket says 172.21.15.179:33850) 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbd90a700 1 -- 172.21.15.179:0/3967696543 learned_addr learned my addr 172.21.15.179:0/3967696543 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.987+0000 7f3dbd109700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db80ae740 0x7f3db80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbd90a700 1 -- 172.21.15.179:0/3967696543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db80ae740 msgr2=0x7f3db80aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbd90a700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db80ae740 0x7f3db80aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbd90a700 1 -- 172.21.15.179:0/3967696543 >> v1:172.21.15.195:6789/0 conn(0x7f3db80af2c0 legacy=0x7f3db80af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbd90a700 1 -- 172.21.15.179:0/3967696543 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80ac290 con 0x7f3db815b150 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbd90a700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db815b150 0x7f3db80b4a30 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009910 tx=0x7f3db4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bd59da21b9e463 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbc908700 1 -- 172.21.15.179:0/3967696543 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4011830 con 0x7f3db815b150 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbc908700 1 -- 172.21.15.179:0/3967696543 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db4011990 con 0x7f3db815b150 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbc908700 1 -- 172.21.15.179:0/3967696543 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4011d40 con 0x7f3db815b150 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- 172.21.15.179:0/3967696543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db815b150 msgr2=0x7f3db80b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db815b150 0x7f3db80b4a30 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009910 tx=0x7f3db4009c20 comp rx=0 tx=0).stop 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- 172.21.15.179:0/3967696543 shutdown_connections 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db80ae740 0x7f3db80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/3967696543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db815b150 0x7f3db80b4a30 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:24.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- 172.21.15.179:0/3967696543 >> 172.21.15.179:0/3967696543 conn(0x7f3db8098a70 msgr2=0x7f3db80b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:24.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- 172.21.15.179:0/3967696543 shutdown_connections 2024-05-08T22:44:24.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- 172.21.15.179:0/3967696543 wait complete. 2024-05-08T22:44:24.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 Processor -- start 2024-05-08T22:44:24.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.991+0000 7f3dbe90c700 1 -- start start 2024-05-08T22:44:25.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3db80af2c0 0x7f3db8157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db815b150 0x7f3db8155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db814f410 con 0x7f3db80ae740 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db814e6c0 con 0x7f3db80af2c0 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81588c0 con 0x7f3db815b150 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33854/0 (socket says 172.21.15.179:33854) 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 -- 172.21.15.179:0/4251683695 learned_addr learned my addr 172.21.15.179:0/4251683695 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd109700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3db80af2c0 0x7f3db8157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe10b700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db815b150 0x7f3db8155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 -- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db815b150 msgr2=0x7f3db8155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db815b150 0x7f3db8155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 -- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3db80af2c0 msgr2=0x7f3db8157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3db80af2c0 0x7f3db8157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 -- 172.21.15.179:0/4251683695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4009570 con 0x7f3db80ae740 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbd90a700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f3db4000c00 tx=0x7f3db4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4004510 con 0x7f3db80ae740 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db40036c0 con 0x7f3db80ae740 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4003a70 con 0x7f3db80ae740 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db8158ba0 con 0x7f3db80ae740 2024-05-08T22:44:25.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.995+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db8158e80 con 0x7f3db80ae740 2024-05-08T22:44:25.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db814f8c0 con 0x7f3db80ae740 2024-05-08T22:44:25.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3db4003bd0 con 0x7f3db80ae740 2024-05-08T22:44:25.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dae7fc700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 0x7f3d9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3db4021030 con 0x7f3db80ae740 2024-05-08T22:44:25.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dbd109700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 0x7f3d9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:24.999+0000 7f3dbd109700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 0x7f3d9c068c20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3db80717c0 tx=0x7f3da8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.007+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3db814f8c0 con 0x7f3db80ae740 2024-05-08T22:44:25.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.179+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3db814f8c0 con 0x7f3db80ae740 2024-05-08T22:44:25.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.179+0000 7f3dae7fc700 1 -- 172.21.15.179:0/4251683695 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3db814f8c0 con 0x7f3db80ae740 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout:[ 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout: { 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout: "poolnum": 1, 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout: "poolname": ".mgr" 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:25.186 INFO:teuthology.orchestra.run.smithi179.stdout:] 2024-05-08T22:44:25.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 msgr2=0x7f3d9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 0x7f3d9c068c20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3db80717c0 tx=0x7f3da8009240 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 msgr2=0x7f3db814e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f3db4000c00 tx=0x7f3db4004770 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 shutdown_connections 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3d9c066760 0x7f3d9c068c20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3db80af2c0 0x7f3db8157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3db815b150 0x7f3db8155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 --2- 172.21.15.179:0/4251683695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80ae740 0x7f3db814e180 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 >> 172.21.15.179:0/4251683695 conn(0x7f3db8098a70 msgr2=0x7f3db80cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 shutdown_connections 2024-05-08T22:44:25.189 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.183+0000 7f3dbe90c700 1 -- 172.21.15.179:0/4251683695 wait complete. 2024-05-08T22:44:25.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:44:25.201 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:44:25.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 Processor -- start 2024-05-08T22:44:25.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 -- start start 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5e240b0310 0x7f5e240b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24159a30 0x7f5e240cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5e2406bc20 0x7f5e240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2406ce20 con 0x7f5e2406bc20 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e240b5360 con 0x7f5e240b0310 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e23fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240b4070 con 0x7f5e24159a30 2024-05-08T22:44:25.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5e2406bc20 0x7f5e240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5e2406bc20 0x7f5e240cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51848/0 (socket says 172.21.15.179:51848) 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 -- 172.21.15.179:0/2709733592 learned_addr learned my addr 172.21.15.179:0/2709733592 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e227fc700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24159a30 0x7f5e240cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e22ffd700 1 --2- 172.21.15.179:0/2709733592 >> v2:172.21.15.195:3300/0 conn(0x7f5e240b0310 0x7f5e240b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 -- 172.21.15.179:0/2709733592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24159a30 msgr2=0x7f5e240cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24159a30 0x7f5e240cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 -- 172.21.15.179:0/2709733592 >> v2:172.21.15.195:3300/0 conn(0x7f5e240b0310 msgr2=0x7f5e240b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 --2- 172.21.15.179:0/2709733592 >> v2:172.21.15.195:3300/0 conn(0x7f5e240b0310 0x7f5e240b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.315+0000 7f5e237fe700 1 -- 172.21.15.179:0/2709733592 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240b1df0 con 0x7f5e2406bc20 2024-05-08T22:44:25.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e237fe700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2406bc20 0x7f5e240cc210 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5e24070b50 tx=0x7f5e1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fc3592b654cd5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e21ffb700 1 -- 172.21.15.179:0/2709733592 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00e070 con 0x7f5e2406bc20 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e21ffb700 1 -- 172.21.15.179:0/2709733592 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00b910 con 0x7f5e2406bc20 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e21ffb700 1 -- 172.21.15.179:0/2709733592 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00bcc0 con 0x7f5e2406bc20 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- 172.21.15.179:0/2709733592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2406bc20 msgr2=0x7f5e240cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2406bc20 0x7f5e240cc210 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5e24070b50 tx=0x7f5e1c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- 172.21.15.179:0/2709733592 shutdown_connections 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2406bc20 0x7f5e240cc210 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 --2- 172.21.15.179:0/2709733592 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24159a30 0x7f5e240cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 --2- 172.21.15.179:0/2709733592 >> v2:172.21.15.195:3300/0 conn(0x7f5e240b0310 0x7f5e240b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- 172.21.15.179:0/2709733592 >> 172.21.15.179:0/2709733592 conn(0x7f5e24098a70 msgr2=0x7f5e240b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- 172.21.15.179:0/2709733592 shutdown_connections 2024-05-08T22:44:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- 172.21.15.179:0/2709733592 wait complete. 2024-05-08T22:44:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 Processor -- start 2024-05-08T22:44:25.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.319+0000 7f5e23fff700 1 -- start start 2024-05-08T22:44:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e2406bc20 0x7f5e240c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e24159a30 0x7f5e240c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240c4fa0 con 0x7f5e240b0310 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240ae1e0 con 0x7f5e24159a30 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240adc20 con 0x7f5e2406bc20 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51856/0 (socket says 172.21.15.179:51856) 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 -- 172.21.15.179:0/1440223205 learned_addr learned my addr 172.21.15.179:0/1440223205 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e22ffd700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e2406bc20 0x7f5e240c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e237fe700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e24159a30 0x7f5e240c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 -- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e2406bc20 msgr2=0x7f5e240c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e2406bc20 0x7f5e240c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 -- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e24159a30 msgr2=0x7f5e240c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e24159a30 0x7f5e240c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 -- 172.21.15.179:0/1440223205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c009b30 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e227fc700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800ebe0 tx=0x7f5e1800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18009740 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1800c850 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e240adf00 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800cbe0 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e240cdc10 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2404e770 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.323+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e1800cd40 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.327+0000 7f5e0b7fe700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 0x7f5e10068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.327+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5e18083fe0 con 0x7f5e240b0310 2024-05-08T22:44:25.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.327+0000 7f5e22ffd700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 0x7f5e10068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.327+0000 7f5e22ffd700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 0x7f5e10068c20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009ad0 tx=0x7f5e14009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.331+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5e180514c0 con 0x7f5e240b0310 2024-05-08T22:44:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.503+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e240c91a0 con 0x7f5e240b0310 2024-05-08T22:44:25.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.823+0000 7f5e0b7fe700 1 -- 172.21.15.179:0/1440223205 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e18054ae0 con 0x7f5e240b0310 2024-05-08T22:44:25.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 msgr2=0x7f5e10068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 0x7f5e10068c20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009ad0 tx=0x7f5e14009240 comp rx=0 tx=0).stop 2024-05-08T22:44:25.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 msgr2=0x7f5e240cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800ebe0 tx=0x7f5e1800eef0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 shutdown_connections 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5e10066760 0x7f5e10068c20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e24159a30 0x7f5e240c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e240b0310 0x7f5e240cd6d0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 --2- 172.21.15.179:0/1440223205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e2406bc20 0x7f5e240c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 >> 172.21.15.179:0/1440223205 conn(0x7f5e24098a70 msgr2=0x7f5e2406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:25.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 shutdown_connections 2024-05-08T22:44:25.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.835+0000 7f5e23fff700 1 -- 172.21.15.179:0/1440223205 wait complete. 2024-05-08T22:44:25.843 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' created 2024-05-08T22:44:25.855 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:44:25.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 Processor -- start 2024-05-08T22:44:25.978 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- start start 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 0x7fd7440b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd7440ce2b0 0x7fd74406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74415d4a0 con 0x7fd7440d1d20 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74406abb0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7440d2500 con 0x7fd74415d0c0 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd7440ce2b0 0x7fd74406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd742ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd7440d1d20 0x7fd7440cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54134/0 (socket says 172.21.15.179:54134) 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd742ffd700 1 -- 172.21.15.179:0/800718890 learned_addr learned my addr 172.21.15.179:0/800718890 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7437fe700 1 --2- 172.21.15.179:0/800718890 >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 0x7fd7440b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7427fc700 1 -- 172.21.15.179:0/800718890 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562861413 0 0) 0x7fd74415d4a0 con 0x7fd7440d1d20 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7427fc700 1 -- 172.21.15.179:0/800718890 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72c0035d0 con 0x7fd7440d1d20 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 -- 172.21.15.179:0/800718890 >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 msgr2=0x7fd7440b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 --2- 172.21.15.179:0/800718890 >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 0x7fd7440b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 -- 172.21.15.179:0/800718890 >> v1:172.21.15.179:6789/0 conn(0x7fd7440d1d20 legacy=0x7fd7440cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 -- 172.21.15.179:0/800718890 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440d27e0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7437fe700 1 --2- 172.21.15.179:0/800718890 >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 0x7fd7440b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd743fff700 1 --2- 172.21.15.179:0/800718890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd74406b370 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009d90 tx=0x7fd73c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a8c323eaa6ebf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7427fc700 1 -- 172.21.15.179:0/800718890 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00e070 con 0x7fd7440ce2b0 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7427fc700 1 -- 172.21.15.179:0/800718890 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c00bae0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd7427fc700 1 -- 172.21.15.179:0/800718890 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00be90 con 0x7fd7440ce2b0 2024-05-08T22:44:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- 172.21.15.179:0/800718890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 msgr2=0x7fd74406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 --2- 172.21.15.179:0/800718890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd74406b370 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009d90 tx=0x7fd73c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- 172.21.15.179:0/800718890 shutdown_connections 2024-05-08T22:44:25.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 --2- 172.21.15.179:0/800718890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd74406b370 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 --2- 172.21.15.179:0/800718890 >> v2:172.21.15.195:3300/0 conn(0x7fd74415d0c0 0x7fd7440b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- 172.21.15.179:0/800718890 >> 172.21.15.179:0/800718890 conn(0x7fd7440d2ed0 msgr2=0x7fd7440d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- 172.21.15.179:0/800718890 shutdown_connections 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.975+0000 7fd748cc2700 1 -- 172.21.15.179:0/800718890 wait complete. 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 Processor -- start 2024-05-08T22:44:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 -- start start 2024-05-08T22:44:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7440d1d20 0x7fd7440c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd74415d0c0 0x7fd7440c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440c76a0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440c7980 con 0x7fd7440d1d20 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd748cc2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440c7c60 con 0x7fd74415d0c0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd7437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd7437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51876/0 (socket says 172.21.15.179:51876) 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.979+0000 7fd7437fe700 1 -- 172.21.15.179:0/92836107 learned_addr learned my addr 172.21.15.179:0/92836107 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd742ffd700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7440d1d20 0x7fd7440c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd743fff700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd74415d0c0 0x7fd7440c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 -- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd74415d0c0 msgr2=0x7fd7440c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd74415d0c0 0x7fd7440c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 -- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7440d1d20 msgr2=0x7fd7440c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7440d1d20 0x7fd7440c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 -- 172.21.15.179:0/92836107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380094b0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd7437fe700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fd73800ebe0 tx=0x7fd73800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738009770 con 0x7fd7440ce2b0 2024-05-08T22:44:25.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73800c850 con 0x7fd7440ce2b0 2024-05-08T22:44:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c009a70 con 0x7fd7440ce2b0 2024-05-08T22:44:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73800cbe0 con 0x7fd7440ce2b0 2024-05-08T22:44:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7440c8220 con 0x7fd7440ce2b0 2024-05-08T22:44:25.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74404e700 con 0x7fd7440ce2b0 2024-05-08T22:44:25.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.983+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd73800cd40 con 0x7fd7440ce2b0 2024-05-08T22:44:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.987+0000 7fd72bfff700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 0x7fd730068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.987+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fd738084190 con 0x7fd7440ce2b0 2024-05-08T22:44:25.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.987+0000 7fd742ffd700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 0x7fd730068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:25.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.987+0000 7fd742ffd700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 0x7fd730068bf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd7440c9a00 tx=0x7fd734007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:25.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:25.991+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd738051470 con 0x7fd7440ce2b0 2024-05-08T22:44:26.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.167+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7440676e0 con 0x7fd7440ce2b0 2024-05-08T22:44:26.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.835+0000 7fd72bfff700 1 -- 172.21.15.179:0/92836107 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd738054a90 con 0x7fd7440ce2b0 2024-05-08T22:44:26.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.843+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 msgr2=0x7fd730068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.843+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 0x7fd730068bf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd7440c9a00 tx=0x7fd734007de0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.843+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 msgr2=0x7fd7440c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.843+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fd73800ebe0 tx=0x7fd73800eef0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 shutdown_connections 2024-05-08T22:44:26.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066730 0x7fd730068bf0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7440d1d20 0x7fd7440c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd74415d0c0 0x7fd7440c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 --2- 172.21.15.179:0/92836107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7440ce2b0 0x7fd7440c9fa0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.847+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 >> 172.21.15.179:0/92836107 conn(0x7fd7440d2ed0 msgr2=0x7fd7440b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:26.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.855+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 shutdown_connections 2024-05-08T22:44:26.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.855+0000 7fd748cc2700 1 -- 172.21.15.179:0/92836107 wait complete. 2024-05-08T22:44:26.859 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' created 2024-05-08T22:44:26.872 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:44:26.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 Processor -- start 2024-05-08T22:44:26.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 -- start start 2024-05-08T22:44:26.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b8415d0c0 0x7f4b840b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840d1d20 0x7f4b840cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8415d4a0 con 0x7f4b840ce0e0 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b840ce570 con 0x7f4b8415d0c0 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.983+0000 7f4b8a631700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8406abb0 con 0x7f4b840d1d20 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4b8415d0c0 0x7f4b840b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b89e30700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f4b840ce0e0 0x7f4b84160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:41280/0 (socket says 172.21.15.179:41280) 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b89e30700 1 -- 172.21.15.179:0/1448214897 learned_addr learned my addr 172.21.15.179:0/1448214897 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:26.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b88e2e700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840d1d20 0x7f4b840cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 -- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840d1d20 msgr2=0x7f4b840cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840d1d20 0x7f4b840cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 -- 172.21.15.179:0/1448214897 >> v1:172.21.15.195:6789/0 conn(0x7f4b840ce0e0 legacy=0x7f4b84160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 -- 172.21.15.179:0/1448214897 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840d2500 con 0x7f4b8415d0c0 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8962f700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b8415d0c0 0x7f4b840b8a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009d90 tx=0x7f4b7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559b00bd90308daf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b83fff700 1 -- 172.21.15.179:0/1448214897 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400e070 con 0x7f4b8415d0c0 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b83fff700 1 -- 172.21.15.179:0/1448214897 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74003a40 con 0x7f4b8415d0c0 2024-05-08T22:44:26.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b83fff700 1 -- 172.21.15.179:0/1448214897 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74003df0 con 0x7f4b8415d0c0 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 -- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b8415d0c0 msgr2=0x7f4b840b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b8415d0c0 0x7f4b840b8a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009d90 tx=0x7f4b7400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 -- 172.21.15.179:0/1448214897 shutdown_connections 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840d1d20 0x7f4b840cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 --2- 172.21.15.179:0/1448214897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b8415d0c0 0x7f4b840b8a80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 -- 172.21.15.179:0/1448214897 >> 172.21.15.179:0/1448214897 conn(0x7f4b840d2ed0 msgr2=0x7f4b840d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 -- 172.21.15.179:0/1448214897 shutdown_connections 2024-05-08T22:44:26.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.987+0000 7f4b8a631700 1 -- 172.21.15.179:0/1448214897 wait complete. 2024-05-08T22:44:26.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 Processor -- start 2024-05-08T22:44:26.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 -- start start 2024-05-08T22:44:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840ce0e0 0x7f4b840c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840d1d20 0x7f4b84160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84160b20 con 0x7f4b8415d0c0 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415fea0 con 0x7f4b840d1d20 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84160180 con 0x7f4b840ce0e0 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8962f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840ce0e0 0x7f4b840c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b88e2e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840d1d20 0x7f4b84160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51886/0 (socket says 172.21.15.179:51886) 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 -- 172.21.15.179:0/2838328981 learned_addr learned my addr 172.21.15.179:0/2838328981 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:26.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 -- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840ce0e0 msgr2=0x7f4b840c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840ce0e0 0x7f4b840c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 -- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840d1d20 msgr2=0x7f4b84160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840d1d20 0x7f4b84160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 -- 172.21.15.179:0/2838328981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009a70 con 0x7f4b8415d0c0 2024-05-08T22:44:26.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b89e30700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c780 tx=0x7f4b7c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c004ca0 con 0x7f4b8415d0c0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b84160460 con 0x7f4b8415d0c0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c004e20 con 0x7f4b8415d0c0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.991+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c007310 con 0x7f4b8415d0c0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b84161480 con 0x7f4b8415d0c0 2024-05-08T22:44:26.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b840cef20 con 0x7f4b8415d0c0 2024-05-08T22:44:26.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b7c007470 con 0x7f4b8415d0c0 2024-05-08T22:44:26.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b81ffb700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 0x7f4b70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:26.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f4b7c083fe0 con 0x7f4b8415d0c0 2024-05-08T22:44:26.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.995+0000 7f4b8962f700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 0x7f4b70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:26.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:26.999+0000 7f4b8962f700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 0x7f4b70068bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400b370 tx=0x7f4b74011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.007+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b7c051150 con 0x7f4b8415d0c0 2024-05-08T22:44:27.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.211+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4b84160c60 con 0x7f4b8415d0c0 2024-05-08T22:44:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.219+0000 7f4b81ffb700 1 -- 172.21.15.179:0/2838328981 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4b84160c60 con 0x7f4b8415d0c0 2024-05-08T22:44:27.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 msgr2=0x7f4b70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 0x7f4b70068bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400b370 tx=0x7f4b74011000 comp rx=0 tx=0).stop 2024-05-08T22:44:27.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 msgr2=0x7f4b8406b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c780 tx=0x7f4b7c00ca90 comp rx=0 tx=0).stop 2024-05-08T22:44:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 shutdown_connections 2024-05-08T22:44:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4b70066730 0x7f4b70068bf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b840d1d20 0x7f4b84160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8415d0c0 0x7f4b8406b1c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 --2- 172.21.15.179:0/2838328981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b840ce0e0 0x7f4b840c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.227+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 >> 172.21.15.179:0/2838328981 conn(0x7f4b840d2ed0 msgr2=0x7f4b840ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:27.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.231+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 shutdown_connections 2024-05-08T22:44:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.231+0000 7f4b8a631700 1 -- 172.21.15.179:0/2838328981 wait complete. 2024-05-08T22:44:27.238 INFO:teuthology.orchestra.run.smithi179.stderr:new fs with metadata pool 17 and data pool 18 2024-05-08T22:44:27.255 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:27.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 Processor -- start 2024-05-08T22:44:27.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 -- start start 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7841599a0 0x7fc78406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc784159d80 0x7fc7840b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78406e060 con 0x7fc78406ec70 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78406de70 con 0x7fc7841599a0 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78b4d9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78406e340 con 0x7fc784159d80 2024-05-08T22:44:27.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78a4d7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7841599a0 0x7fc78406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78a4d7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7841599a0 0x7fc78406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51894/0 (socket says 172.21.15.179:51894) 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc78a4d7700 1 -- 172.21.15.179:0/3125630043 learned_addr learned my addr 172.21.15.179:0/3125630043 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.355+0000 7fc789cd6700 1 --2- 172.21.15.179:0/3125630043 >> v2:172.21.15.195:3300/0 conn(0x7fc784159d80 0x7fc7840b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78a4d7700 1 -- 172.21.15.179:0/3125630043 >> v2:172.21.15.195:3300/0 conn(0x7fc784159d80 msgr2=0x7fc7840b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78a4d7700 1 --2- 172.21.15.179:0/3125630043 >> v2:172.21.15.195:3300/0 conn(0x7fc784159d80 0x7fc7840b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78a4d7700 1 -- 172.21.15.179:0/3125630043 >> v1:172.21.15.195:6789/0 conn(0x7fc78406ec70 legacy=0x7fc78415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78a4d7700 1 -- 172.21.15.179:0/3125630043 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78406e620 con 0x7fc7841599a0 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78a4d7700 1 --2- 172.21.15.179:0/3125630043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7841599a0 0x7fc78406d930 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b3d20 tx=0x7fc77c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19feb930dc90478 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc7894d5700 1 -- 172.21.15.179:0/3125630043 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00e070 con 0x7fc7841599a0 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc7894d5700 1 -- 172.21.15.179:0/3125630043 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00b910 con 0x7fc7841599a0 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc7894d5700 1 -- 172.21.15.179:0/3125630043 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00bcc0 con 0x7fc7841599a0 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- 172.21.15.179:0/3125630043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7841599a0 msgr2=0x7fc78406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/3125630043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7841599a0 0x7fc78406d930 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b3d20 tx=0x7fc77c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- 172.21.15.179:0/3125630043 shutdown_connections 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/3125630043 >> v2:172.21.15.195:3300/0 conn(0x7fc784159d80 0x7fc7840b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/3125630043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7841599a0 0x7fc78406d930 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- 172.21.15.179:0/3125630043 >> 172.21.15.179:0/3125630043 conn(0x7fc7840ce9d0 msgr2=0x7fc7840cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- 172.21.15.179:0/3125630043 shutdown_connections 2024-05-08T22:44:27.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- 172.21.15.179:0/3125630043 wait complete. 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 Processor -- start 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.359+0000 7fc78b4d9700 1 -- start start 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7841599a0 0x7fc784152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc784159d80 0x7fc78413e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784140dc0 con 0x7fc78406ec70 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784140180 con 0x7fc784159d80 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc784140420 con 0x7fc7841599a0 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51908/0 (socket says 172.21.15.179:51908) 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 -- 172.21.15.179:0/2464102935 learned_addr learned my addr 172.21.15.179:0/2464102935 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc789cd6700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7841599a0 0x7fc784152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78acd8700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc784159d80 0x7fc78413e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 -- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7841599a0 msgr2=0x7fc784152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7841599a0 0x7fc784152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 -- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc784159d80 msgr2=0x7fc78413e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc784159d80 0x7fc78413e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 -- 172.21.15.179:0/2464102935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c009b30 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78a4d7700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000f80 tx=0x7fc77c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004900 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0092e0 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc784140700 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c01c660 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc784152c60 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.363+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784067740 con 0x7fc78406ec70 2024-05-08T22:44:27.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.367+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc77c01c7c0 con 0x7fc78406ec70 2024-05-08T22:44:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.367+0000 7fc77affd700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 0x7fc768068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.367+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc77c085ba0 con 0x7fc78406ec70 2024-05-08T22:44:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.367+0000 7fc789cd6700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 0x7fc768068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.367+0000 7fc789cd6700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 0x7fc768068bf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc78406abb0 tx=0x7fc774009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.375+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc77c052dc0 con 0x7fc78406ec70 2024-05-08T22:44:27.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.547+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc78404e700 con 0x7fc78406ec70 2024-05-08T22:44:27.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.551+0000 7fc77affd700 1 -- 172.21.15.179:0/2464102935 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16377 (secure 0 0 0) 0x7fc77c0563e0 con 0x7fc78406ec70 2024-05-08T22:44:27.552 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:27.553 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":126,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:27.215545+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:27.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.551+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 msgr2=0x7fc768068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.551+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 0x7fc768068bf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc78406abb0 tx=0x7fc774009240 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.551+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 msgr2=0x7fc7841562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.551+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000f80 tx=0x7fc77c004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 shutdown_connections 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc768066730 0x7fc768068bf0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc784159d80 0x7fc78413e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7841599a0 0x7fc784152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 --2- 172.21.15.179:0/2464102935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc78406ec70 0x7fc7841562f0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 >> 172.21.15.179:0/2464102935 conn(0x7fc7840ce9d0 msgr2=0x7fc784142340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:27.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 shutdown_connections 2024-05-08T22:44:27.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.555+0000 7fc78b4d9700 1 -- 172.21.15.179:0/2464102935 wait complete. 2024-05-08T22:44:27.567 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:27.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 Processor -- start 2024-05-08T22:44:27.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 -- start start 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6d8161790 0x7fd6d8164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d8165d50 con 0x7fd6d815ef80 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d806ad70 con 0x7fd6d806de70 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6dcf16700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d806e4b0 con 0x7fd6d8161790 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6d8161790 0x7fd6d8164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6d8161790 0x7fd6d8164db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51924/0 (socket says 172.21.15.179:51924) 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2652511953 learned_addr learned my addr 172.21.15.179:0/2652511953 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2652511953 >> v1:172.21.15.195:6789/0 conn(0x7fd6d806de70 legacy=0x7fd6d806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2652511953 >> v1:172.21.15.179:6789/0 conn(0x7fd6d815ef80 legacy=0x7fd6d806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d67fc700 1 -- 172.21.15.179:0/2652511953 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450685120 0 0) 0x7fd6d8165d50 con 0x7fd6d815ef80 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.679+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2652511953 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d81654b0 con 0x7fd6d8161790 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6d7fff700 1 --2- 172.21.15.179:0/2652511953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d8164db0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009e50 tx=0x7fd6d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca71b762b77947 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6d67fc700 1 -- 172.21.15.179:0/2652511953 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000e070 con 0x7fd6d8161790 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6d67fc700 1 -- 172.21.15.179:0/2652511953 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000b9c0 con 0x7fd6d8161790 2024-05-08T22:44:27.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6d67fc700 1 -- 172.21.15.179:0/2652511953 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000bd70 con 0x7fd6d8161790 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2652511953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 msgr2=0x7fd6d8164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2652511953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d8164db0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009e50 tx=0x7fd6d000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2652511953 shutdown_connections 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2652511953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d8164db0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2652511953 >> 172.21.15.179:0/2652511953 conn(0x7fd6d80ce9d0 msgr2=0x7fd6d80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2652511953 shutdown_connections 2024-05-08T22:44:27.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2652511953 wait complete. 2024-05-08T22:44:27.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 Processor -- start 2024-05-08T22:44:27.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.683+0000 7fd6dcf16700 1 -- start start 2024-05-08T22:44:27.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 0x7fd6d814d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d815ef80 0x7fd6d81f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d814da60 con 0x7fd6d8161790 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d814adb0 con 0x7fd6d815ef80 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d814b090 con 0x7fd6d806de70 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51932/0 (socket says 172.21.15.179:51932) 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2779417922 learned_addr learned my addr 172.21.15.179:0/2779417922 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d6ffd700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d815ef80 0x7fd6d81f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d77fe700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 0x7fd6d814d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 msgr2=0x7fd6d814d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 0x7fd6d814d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d815ef80 msgr2=0x7fd6d81f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d815ef80 0x7fd6d81f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 -- 172.21.15.179:0/2779417922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009b30 con 0x7fd6d8161790 2024-05-08T22:44:27.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d77fe700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 0x7fd6d814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6d7fff700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8072020 tx=0x7fd6d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00092e0 con 0x7fd6d8161790 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d001b430 con 0x7fd6d8161790 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d814b370 con 0x7fd6d8161790 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d001b7e0 con 0x7fd6d8161790 2024-05-08T22:44:27.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d814b930 con 0x7fd6d8161790 2024-05-08T22:44:27.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.687+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d8067740 con 0x7fd6d8161790 2024-05-08T22:44:27.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.691+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6d001b940 con 0x7fd6d8161790 2024-05-08T22:44:27.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.691+0000 7fd6cffff700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 0x7fd6b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:27.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.691+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd6d0085d70 con 0x7fd6d8161790 2024-05-08T22:44:27.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.691+0000 7fd6d77fe700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 0x7fd6b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:27.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.691+0000 7fd6d77fe700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 0x7fd6b8068bf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000f60 tx=0x7fd6c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:27.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.695+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd6d0052f10 con 0x7fd6d8161790 2024-05-08T22:44:27.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.867+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d804e700 con 0x7fd6d8161790 2024-05-08T22:44:27.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.871+0000 7fd6cffff700 1 -- 172.21.15.179:0/2779417922 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16377 (secure 0 0 0) 0x7fd6d0056530 con 0x7fd6d8161790 2024-05-08T22:44:27.872 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:27.873 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":126,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:27.215545+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 msgr2=0x7fd6b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 0x7fd6b8068bf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000f60 tx=0x7fd6c0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 msgr2=0x7fd6d80cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8072020 tx=0x7fd6d0004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 shutdown_connections 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd6b8066730 0x7fd6b8068bf0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d815ef80 0x7fd6d81f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6d8161790 0x7fd6d80cbbb0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 --2- 172.21.15.179:0/2779417922 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d806de70 0x7fd6d814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:27.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 >> 172.21.15.179:0/2779417922 conn(0x7fd6d80ce9d0 msgr2=0x7fd6d806d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:27.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 shutdown_connections 2024-05-08T22:44:27.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:27.875+0000 7fd6dcf16700 1 -- 172.21.15.179:0/2779417922 wait complete. 2024-05-08T22:44:27.893 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:44:28.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 Processor -- start 2024-05-08T22:44:28.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 -- start start 2024-05-08T22:44:28.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22b40ce430 0x7f22b40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b40cc430 con 0x7f22b40c9d70 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b40ca190 con 0x7f22b40c9900 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.011+0000 7f22b919e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b40b3f60 con 0x7f22b40ce430 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b899d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f22b40c9d70 0x7f22b40b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54150/0 (socket says 172.21.15.179:54150) 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b899d700 1 -- 172.21.15.179:0/2826700860 learned_addr learned my addr 172.21.15.179:0/2826700860 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b3fff700 1 --2- 172.21.15.179:0/2826700860 >> v2:172.21.15.195:3300/0 conn(0x7f22b40ce430 0x7f22b40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b2ffd700 1 -- 172.21.15.179:0/2826700860 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392951140 0 0) 0x7f22b40cc430 con 0x7f22b40c9d70 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b2ffd700 1 -- 172.21.15.179:0/2826700860 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f229c0035d0 con 0x7f22b40c9d70 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b3fff700 1 -- 172.21.15.179:0/2826700860 >> v1:172.21.15.195:6789/0 conn(0x7f22b40c9900 legacy=0x7f22b40ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b3fff700 1 -- 172.21.15.179:0/2826700860 >> v1:172.21.15.179:6789/0 conn(0x7f22b40c9d70 legacy=0x7f22b40b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b3fff700 1 -- 172.21.15.179:0/2826700860 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40cc660 con 0x7f22b40ce430 2024-05-08T22:44:28.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b3fff700 1 --2- 172.21.15.179:0/2826700860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40ce430 0x7f22b40caf60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009850 tx=0x7f22a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e173d7157299a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b2ffd700 1 -- 172.21.15.179:0/2826700860 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8011830 con 0x7f22b40ce430 2024-05-08T22:44:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b2ffd700 1 -- 172.21.15.179:0/2826700860 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8011990 con 0x7f22b40ce430 2024-05-08T22:44:28.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b2ffd700 1 -- 172.21.15.179:0/2826700860 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8011d40 con 0x7f22b40ce430 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 -- 172.21.15.179:0/2826700860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40ce430 msgr2=0x7f22b40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 --2- 172.21.15.179:0/2826700860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40ce430 0x7f22b40caf60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009850 tx=0x7f22a8009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 -- 172.21.15.179:0/2826700860 shutdown_connections 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 --2- 172.21.15.179:0/2826700860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40ce430 0x7f22b40caf60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 -- 172.21.15.179:0/2826700860 >> 172.21.15.179:0/2826700860 conn(0x7f22b4098a70 msgr2=0x7f22b40b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 -- 172.21.15.179:0/2826700860 shutdown_connections 2024-05-08T22:44:28.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 -- 172.21.15.179:0/2826700860 wait complete. 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.015+0000 7f22b919e700 1 Processor -- start 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- start start 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b40c9900 0x7f22b415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40c9d70 0x7f22b40af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c6960 con 0x7f22b40ce430 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40afdb0 con 0x7f22b40c9d70 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4156ff0 con 0x7f22b40c9900 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51942/0 (socket says 172.21.15.179:51942) 2024-05-08T22:44:28.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b3fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b40c9900 0x7f22b415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 -- 172.21.15.179:0/1578824997 learned_addr learned my addr 172.21.15.179:0/1578824997 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:28.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b37fe700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40c9d70 0x7f22b40af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 -- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b40c9900 msgr2=0x7f22b415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b40c9900 0x7f22b415b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 -- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40c9d70 msgr2=0x7f22b40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40c9d70 0x7f22b40af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 -- 172.21.15.179:0/1578824997 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80094b0 con 0x7f22b40ce430 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b899d700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00cb70 tx=0x7f22ac00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac007480 con 0x7f22b40ce430 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac0075e0 con 0x7f22b40ce430 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b41572c0 con 0x7f22b40ce430 2024-05-08T22:44:28.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac007950 con 0x7f22b40ce430 2024-05-08T22:44:28.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.019+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b4157880 con 0x7f22b40ce430 2024-05-08T22:44:28.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.023+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b40c6b30 con 0x7f22b40ce430 2024-05-08T22:44:28.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.023+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22ac01b3b0 con 0x7f22b40ce430 2024-05-08T22:44:28.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.023+0000 7f22b0ff9700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 0x7f22a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:28.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.023+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f22ac083aa0 con 0x7f22b40ce430 2024-05-08T22:44:28.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.023+0000 7f22b3fff700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 0x7f22a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:28.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.027+0000 7f22b3fff700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 0x7f22a0068bf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f22a80090b0 tx=0x7f22a801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:28.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.031+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f22b40c6b30 con 0x7f22b40ce430 2024-05-08T22:44:28.160 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:44:28.155+0000 7fb124ad3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:44:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:28.231+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f22b404e700 con 0x7f22b40ce430 2024-05-08T22:44:29.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.235+0000 7f22b0ff9700 1 -- 172.21.15.179:0/1578824997 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7f22ac050d70 con 0x7f22b40ce430 2024-05-08T22:44:29.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 msgr2=0x7f22a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 0x7f22a0068bf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f22a80090b0 tx=0x7f22a801c040 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 msgr2=0x7f22b4156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00cb70 tx=0x7f22ac00cba0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 shutdown_connections 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f22a0066730 0x7f22a0068bf0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22b40c9d70 0x7f22b40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b40ce430 0x7f22b4156ab0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 --2- 172.21.15.179:0/1578824997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22b40c9900 0x7f22b415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 >> 172.21.15.179:0/1578824997 conn(0x7f22b4098a70 msgr2=0x7f22b4159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 shutdown_connections 2024-05-08T22:44:29.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.239+0000 7f22b919e700 1 -- 172.21.15.179:0/1578824997 wait complete. 2024-05-08T22:44:29.258 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T22:44:29.258 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T22:44:29.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 Processor -- start 2024-05-08T22:44:29.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 -- start start 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17b80aeb30 0x7f17b806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17b806b0f0 0x7f17b80ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b80ae8e0 con 0x7f17b81607d0 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b806e310 con 0x7f17b80aeb30 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17c029f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b806e5f0 con 0x7f17b806b0f0 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17b80aeb30 0x7f17b806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17b80aeb30 0x7f17b806abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51948/0 (socket says 172.21.15.179:51948) 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 -- 172.21.15.179:0/4245783741 learned_addr learned my addr 172.21.15.179:0/4245783741 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17be83c700 1 --2- 172.21.15.179:0/4245783741 >> v2:172.21.15.195:3300/0 conn(0x7f17b806b0f0 0x7f17b80ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 -- 172.21.15.179:0/4245783741 >> v2:172.21.15.195:3300/0 conn(0x7f17b806b0f0 msgr2=0x7f17b80ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 --2- 172.21.15.179:0/4245783741 >> v2:172.21.15.195:3300/0 conn(0x7f17b806b0f0 0x7f17b80ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 -- 172.21.15.179:0/4245783741 >> v1:172.21.15.195:6789/0 conn(0x7f17b81607d0 legacy=0x7f17b806f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 -- 172.21.15.179:0/4245783741 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b806e8d0 con 0x7f17b80aeb30 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.383+0000 7f17bd83a700 1 --2- 172.21.15.179:0/4245783741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b80aeb30 0x7f17b806abb0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f17b806d630 tx=0x7f17a8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18f354fdbb0c092 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17bd039700 1 -- 172.21.15.179:0/4245783741 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8004230 con 0x7f17b80aeb30 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17bd039700 1 -- 172.21.15.179:0/4245783741 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a8004390 con 0x7f17b80aeb30 2024-05-08T22:44:29.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17bd039700 1 -- 172.21.15.179:0/4245783741 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a80036c0 con 0x7f17b80aeb30 2024-05-08T22:44:29.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 -- 172.21.15.179:0/4245783741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b80aeb30 msgr2=0x7f17b806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 --2- 172.21.15.179:0/4245783741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b80aeb30 0x7f17b806abb0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f17b806d630 tx=0x7f17a8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 -- 172.21.15.179:0/4245783741 shutdown_connections 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 --2- 172.21.15.179:0/4245783741 >> v2:172.21.15.195:3300/0 conn(0x7f17b806b0f0 0x7f17b80ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 --2- 172.21.15.179:0/4245783741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b80aeb30 0x7f17b806abb0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 -- 172.21.15.179:0/4245783741 >> 172.21.15.179:0/4245783741 conn(0x7f17b8098a70 msgr2=0x7f17b80b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 -- 172.21.15.179:0/4245783741 shutdown_connections 2024-05-08T22:44:29.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 -- 172.21.15.179:0/4245783741 wait complete. 2024-05-08T22:44:29.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.387+0000 7f17c029f700 1 Processor -- start 2024-05-08T22:44:29.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- start start 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b806b0f0 0x7f17b815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b80aeb30 0x7f17b8154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8159f20 con 0x7f17b81607d0 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80ae050 con 0x7f17b806b0f0 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8155210 con 0x7f17b80aeb30 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51956/0 (socket says 172.21.15.179:51956) 2024-05-08T22:44:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 -- 172.21.15.179:0/2790657690 learned_addr learned my addr 172.21.15.179:0/2790657690 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be03b700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b806b0f0 0x7f17b815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17bd83a700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b80aeb30 0x7f17b8154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 -- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b80aeb30 msgr2=0x7f17b8154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b80aeb30 0x7f17b8154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 -- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b806b0f0 msgr2=0x7f17b815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b806b0f0 0x7f17b815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 -- 172.21.15.179:0/2790657690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8009570 con 0x7f17b81607d0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17be83c700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007ea0 tx=0x7f17b000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000edf0 con 0x7f17b81607d0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0014420 con 0x7f17b81607d0 2024-05-08T22:44:29.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00147d0 con 0x7f17b81607d0 2024-05-08T22:44:29.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b80ad7b0 con 0x7f17b81607d0 2024-05-08T22:44:29.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.391+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b80add70 con 0x7f17b81607d0 2024-05-08T22:44:29.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b8073290 con 0x7f17b81607d0 2024-05-08T22:44:29.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17b0014930 con 0x7f17b81607d0 2024-05-08T22:44:29.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17aeffd700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 0x7f179c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:29.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f17b0084920 con 0x7f17b81607d0 2024-05-08T22:44:29.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17be03b700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 0x7f179c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.395+0000 7f17be03b700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 0x7f179c068c20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009980 tx=0x7f17b40090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:29.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.403+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17b0051be0 con 0x7f17b81607d0 2024-05-08T22:44:29.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:29.611+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17b80cedf0 con 0x7f17b81607d0 2024-05-08T22:44:30.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.243+0000 7f17aeffd700 1 -- 172.21.15.179:0/2790657690 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17b0055200 con 0x7f17b81607d0 2024-05-08T22:44:30.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 msgr2=0x7f179c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 0x7f179c068c20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009980 tx=0x7f17b40090e0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 msgr2=0x7f17b80acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007ea0 tx=0x7f17b000e3c0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 shutdown_connections 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f179c066760 0x7f179c068c20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b81607d0 0x7f17b80acf50 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17b80aeb30 0x7f17b8154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 --2- 172.21.15.179:0/2790657690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b806b0f0 0x7f17b815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 >> 172.21.15.179:0/2790657690 conn(0x7f17b8098a70 msgr2=0x7f17b806c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 shutdown_connections 2024-05-08T22:44:30.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.247+0000 7f17c029f700 1 -- 172.21.15.179:0/2790657690 wait complete. 2024-05-08T22:44:30.266 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T22:44:30.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:30.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 Processor -- start 2024-05-08T22:44:30.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 -- start start 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb2c06bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c06afc0 con 0x7ffb2c0cde70 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c15d990 con 0x7ffb2c15d5b0 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0d26f0 con 0x7ffb2c0ce2d0 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb29d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffb2c0cde70 0x7ffb2c06b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54152/0 (socket says 172.21.15.179:54152) 2024-05-08T22:44:30.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb29d9b700 1 -- 172.21.15.179:0/881512901 learned_addr learned my addr 172.21.15.179:0/881512901 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:30.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861462592 0 0) 0x7ffb2c06afc0 con 0x7ffb2c0cde70 2024-05-08T22:44:30.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb2c06bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb180035d0 con 0x7ffb2c0cde70 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986520502 0 0) 0x7ffb180035d0 con 0x7ffb2c0cde70 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 msgr2=0x7ffb2c06bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb2c06bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.383+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 >> v1:172.21.15.195:6789/0 conn(0x7ffb2c15d5b0 legacy=0x7ffb2c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0d2130 con 0x7ffb2c0cde70 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ffb1c0019f0 con 0x7ffb2c0cde70 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 >> v1:172.21.15.179:6789/0 conn(0x7ffb2c0cde70 legacy=0x7ffb2c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18003a50 0x7ffb18003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb18004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb18004db0 0x7ffb18005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0d26f0 con 0x7ffb18004db0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1800ba30 con 0x7ffb18003a50 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1800bb70 con 0x7ffb2c0ce2d0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb18004db0 0x7ffb18005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb29d9b700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb18004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2a59c700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18003a50 0x7ffb18003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 msgr2=0x7ffb18004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0ce2d0 0x7ffb18004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18003a50 msgr2=0x7ffb18003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18003a50 0x7ffb18003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/881512901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180035d0 con 0x7ffb18004db0 2024-05-08T22:44:30.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb18004db0 0x7ffb18005230 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009b70 tx=0x7ffb180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d0fb00c8f471b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400e6c0 con 0x7ffb18004db0 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2400e820 con 0x7ffb18004db0 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2959a700 1 -- 172.21.15.179:0/881512901 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400eb90 con 0x7ffb18004db0 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 -- 172.21.15.179:0/881512901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb18004db0 msgr2=0x7ffb18005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/881512901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb18004db0 0x7ffb18005230 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009b70 tx=0x7ffb180058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/881512901 reap_dead start 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 -- 172.21.15.179:0/881512901 shutdown_connections 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 -- 172.21.15.179:0/881512901 >> 172.21.15.179:0/881512901 conn(0x7ffb2c0b4940 msgr2=0x7ffb2c0b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 -- 172.21.15.179:0/881512901 shutdown_connections 2024-05-08T22:44:30.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.387+0000 7ffb2b59e700 1 -- 172.21.15.179:0/881512901 wait complete. 2024-05-08T22:44:30.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 Processor -- start 2024-05-08T22:44:30.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 -- start start 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18004db0 0x7ffb2c1e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 0x7ffb2c0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cca10 con 0x7ffb2c0cc590 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c144ac0 con 0x7ffb18004db0 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c144d60 con 0x7ffb2c0c5c40 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51974/0 (socket says 172.21.15.179:51974) 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/3867954821 learned_addr learned my addr 172.21.15.179:0/3867954821 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb2a59c700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18004db0 0x7ffb2c1e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.391+0000 7ffb29d9b700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 0x7ffb2c0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 msgr2=0x7ffb2c0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 0x7ffb2c0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18004db0 msgr2=0x7ffb2c1e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18004db0 0x7ffb2c1e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 -- 172.21.15.179:0/3867954821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c145040 con 0x7ffb2c0cc590 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb29d9b700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 0x7ffb2c0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2ad9d700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7ffb180047e0 tx=0x7ffb240047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400e6c0 con 0x7ffb2c0cc590 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb240044a0 con 0x7ffb2c0cc590 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c145320 con 0x7ffb2c0cc590 2024-05-08T22:44:30.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24004610 con 0x7ffb2c0cc590 2024-05-08T22:44:30.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c0c60c0 con 0x7ffb2c0cc590 2024-05-08T22:44:30.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c145840 con 0x7ffb2c0cc590 2024-05-08T22:44:30.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb24013a10 con 0x7ffb2c0cc590 2024-05-08T22:44:30.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 0x7ffb0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffb24000e40 con 0x7ffb2c0cc590 2024-05-08T22:44:30.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.395+0000 7ffb2a59c700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 0x7ffb0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.399+0000 7ffb2a59c700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 0x7ffb0c068c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffb14000c00 tx=0x7ffb14001330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.403+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb2c145840 con 0x7ffb2c0cc590 2024-05-08T22:44:30.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.607+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb2c145840 con 0x7ffb2c0cc590 2024-05-08T22:44:30.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.607+0000 7ffb22ffd700 1 -- 172.21.15.179:0/3867954821 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4751 (secure 0 0 0) 0x7ffb24017090 con 0x7ffb2c0cc590 2024-05-08T22:44:30.609 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:30.609 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":90,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:30.233148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57983":{"gid":57983,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/2184096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2184096099},{"type":"v1","addr":"172.21.15.195:6835","nonce":2184096099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 msgr2=0x7ffb0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 0x7ffb0c068c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffb14000c00 tx=0x7ffb14001330 comp rx=0 tx=0).stop 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 msgr2=0x7ffb2c0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7ffb180047e0 tx=0x7ffb240047e0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 shutdown_connections 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ffb0c066760 0x7ffb0c068c20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb2c0cc590 0x7ffb2c0caeb0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2c0c5c40 0x7ffb2c0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 --2- 172.21.15.179:0/3867954821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb18004db0 0x7ffb2c1e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 >> 172.21.15.179:0/3867954821 conn(0x7ffb2c0b4940 msgr2=0x7ffb2c0b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 shutdown_connections 2024-05-08T22:44:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.611+0000 7ffb2b59e700 1 -- 172.21.15.179:0/3867954821 wait complete. 2024-05-08T22:44:30.614 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 90 2024-05-08T22:44:30.624 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:30.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.735+0000 7f36eec14700 1 Processor -- start 2024-05-08T22:44:30.738 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.735+0000 7f36eec14700 1 -- start start 2024-05-08T22:44:30.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f36e80ca790 0x7f36e80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e815d9c0 con 0x7f36e8146260 2024-05-08T22:44:30.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e815dca0 con 0x7f36e80ca3b0 2024-05-08T22:44:30.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e815df80 con 0x7f36e80ca790 2024-05-08T22:44:30.739 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ee413700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f36e8146260 0x7f36e81466d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54156/0 (socket says 172.21.15.179:54156) 2024-05-08T22:44:30.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ee413700 1 -- 172.21.15.179:0/1704924405 learned_addr learned my addr 172.21.15.179:0/1704924405 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:30.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ed411700 1 --2- 172.21.15.179:0/1704924405 >> v2:172.21.15.195:3300/0 conn(0x7f36e80ca790 0x7f36e80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248709964 0 0) 0x7f36e815d9c0 con 0x7f36e8146260 2024-05-08T22:44:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d40035d0 con 0x7f36e8146260 2024-05-08T22:44:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ed411700 1 -- 172.21.15.179:0/1704924405 >> v1:172.21.15.195:6789/0 conn(0x7f36e80ca3b0 legacy=0x7f36e80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180863316 0 0) 0x7f36e815dca0 con 0x7f36e80ca3b0 2024-05-08T22:44:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ed411700 1 -- 172.21.15.179:0/1704924405 >> v1:172.21.15.179:6789/0 conn(0x7f36e8146260 legacy=0x7f36e81466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ed411700 1 -- 172.21.15.179:0/1704924405 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e815e260 con 0x7f36e80ca790 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ed411700 1 --2- 172.21.15.179:0/1704924405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca790 0x7f36e80ce120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009880 tx=0x7f36d8009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8ec83b3bbdcaad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8011830 con 0x7f36e80ca790 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8011990 con 0x7f36e80ca790 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36ecc10700 1 -- 172.21.15.179:0/1704924405 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8011d00 con 0x7f36e80ca790 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- 172.21.15.179:0/1704924405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca790 msgr2=0x7f36e80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 --2- 172.21.15.179:0/1704924405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca790 0x7f36e80ce120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009880 tx=0x7f36d8009b90 comp rx=0 tx=0).stop 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- 172.21.15.179:0/1704924405 shutdown_connections 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 --2- 172.21.15.179:0/1704924405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca790 0x7f36e80ce120 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- 172.21.15.179:0/1704924405 >> 172.21.15.179:0/1704924405 conn(0x7f36e80b17c0 msgr2=0x7f36e80b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- 172.21.15.179:0/1704924405 shutdown_connections 2024-05-08T22:44:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.739+0000 7f36eec14700 1 -- 172.21.15.179:0/1704924405 wait complete. 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 Processor -- start 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 -- start start 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 0x7f36e81f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36e8146260 0x7f36e80b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81f6730 con 0x7f36e80ca790 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80b56c0 con 0x7f36e80ca3b0 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e580 con 0x7f36e8146260 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51986/0 (socket says 172.21.15.179:51986) 2024-05-08T22:44:30.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 -- 172.21.15.179:0/3844952087 learned_addr learned my addr 172.21.15.179:0/3844952087 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36edc12700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 0x7f36e81f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ee413700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36e8146260 0x7f36e80b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 -- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36e8146260 msgr2=0x7f36e80b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36e8146260 0x7f36e80b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 -- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 msgr2=0x7f36e81f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 0x7f36e81f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 -- 172.21.15.179:0/3844952087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80094b0 con 0x7f36e80ca790 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36edc12700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 0x7f36e81f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36ed411700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f36d80090b0 tx=0x7f36d8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004480 con 0x7f36e80ca790 2024-05-08T22:44:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d801f070 con 0x7f36e80ca790 2024-05-08T22:44:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.743+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80b53e0 con 0x7f36e80ca790 2024-05-08T22:44:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.747+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8003900 con 0x7f36e80ca790 2024-05-08T22:44:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.747+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81f6350 con 0x7f36e80ca790 2024-05-08T22:44:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.747+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e804e700 con 0x7f36e80ca790 2024-05-08T22:44:30.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.747+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36d8021030 con 0x7f36e80ca790 2024-05-08T22:44:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.747+0000 7f36de7fc700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 0x7f36cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.751+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f36d8085610 con 0x7f36e80ca790 2024-05-08T22:44:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.751+0000 7f36edc12700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 0x7f36cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:30.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.751+0000 7f36edc12700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 0x7f36cc068bf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f36e4007fd0 tx=0x7f36e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:30.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.755+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f36d8052950 con 0x7f36e80ca790 2024-05-08T22:44:30.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.931+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36e80cdf00 con 0x7f36e80ca790 2024-05-08T22:44:30.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.931+0000 7f36de7fc700 1 -- 172.21.15.179:0/3844952087 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16364 (secure 0 0 0) 0x7f36d8055f70 con 0x7f36e80ca790 2024-05-08T22:44:30.934 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:30.934 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":127,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:28.218971+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 msgr2=0x7f36cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 0x7f36cc068bf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f36e4007fd0 tx=0x7f36e4009040 comp rx=0 tx=0).stop 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 msgr2=0x7f36e806e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f36d80090b0 tx=0x7f36d8004770 comp rx=0 tx=0).stop 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 shutdown_connections 2024-05-08T22:44:30.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f36cc066730 0x7f36cc068bf0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36e8146260 0x7f36e80b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36e80ca790 0x7f36e806e040 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 --2- 172.21.15.179:0/3844952087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36e80ca3b0 0x7f36e81f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 >> 172.21.15.179:0/3844952087 conn(0x7f36e80b17c0 msgr2=0x7f36e80cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 shutdown_connections 2024-05-08T22:44:30.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:30.935+0000 7f36eec14700 1 -- 172.21.15.179:0/3844952087 wait complete. 2024-05-08T22:44:30.949 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:44:30.949 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:44:31.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 Processor -- start 2024-05-08T22:44:31.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 -- start start 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 0x7f796004fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796003a500 0x7f79600068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 0x7f79600e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7960053310 con 0x7f796003a500 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600500f0 con 0x7f796004f7d0 2024-05-08T22:44:31.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7967fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79600071e0 con 0x7f79600e6c80 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796003a500 0x7f79600068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f796003a500 0x7f79600068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51990/0 (socket says 172.21.15.179:51990) 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 -- 172.21.15.179:0/1720275903 learned_addr learned my addr 172.21.15.179:0/1720275903 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7966ffd700 1 --2- 172.21.15.179:0/1720275903 >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 0x7f796004fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 -- 172.21.15.179:0/1720275903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 msgr2=0x7f79600e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79677fe700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 0x7f79600e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 0x7f79600e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 -- 172.21.15.179:0/1720275903 >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 msgr2=0x7f796004fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 --2- 172.21.15.179:0/1720275903 >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 0x7f796004fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 -- 172.21.15.179:0/1720275903 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600074c0 con 0x7f796003a500 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7966ffd700 1 --2- 172.21.15.179:0/1720275903 >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 0x7f796004fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79677fe700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 0x7f79600e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f79667fc700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796003a500 0x7f79600068b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f796000c190 tx=0x7f7958009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d173d26ad03331 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7965ffb700 1 -- 172.21.15.179:0/1720275903 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800b8a0 con 0x7f796003a500 2024-05-08T22:44:31.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7965ffb700 1 -- 172.21.15.179:0/1720275903 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800ba00 con 0x7f796003a500 2024-05-08T22:44:31.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.063+0000 7f7965ffb700 1 -- 172.21.15.179:0/1720275903 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800bd90 con 0x7f796003a500 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- 172.21.15.179:0/1720275903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796003a500 msgr2=0x7f79600068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796003a500 0x7f79600068b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f796000c190 tx=0x7f7958009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- 172.21.15.179:0/1720275903 shutdown_connections 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e6c80 0x7f79600e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- 172.21.15.179:0/1720275903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796003a500 0x7f79600068b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- 172.21.15.179:0/1720275903 >> v2:172.21.15.195:3300/0 conn(0x7f796004f7d0 0x7f796004fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- 172.21.15.179:0/1720275903 >> 172.21.15.179:0/1720275903 conn(0x7f7960054b00 msgr2=0x7f7960054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- 172.21.15.179:0/1720275903 shutdown_connections 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- 172.21.15.179:0/1720275903 wait complete. 2024-05-08T22:44:31.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 Processor -- start 2024-05-08T22:44:31.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- start start 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79600e6c80 0x7f79600e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e62f0 0x7f79600e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600e16e0 con 0x7f796004f7d0 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600e1960 con 0x7f79600e6c80 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7967fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79600e1c40 con 0x7f79600e62f0 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.067+0000 7f7966ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f79677fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e62f0 0x7f79600e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51998/0 (socket says 172.21.15.179:51998) 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f79667fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79600e6c80 0x7f79600e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 -- 172.21.15.179:0/2566374831 learned_addr learned my addr 172.21.15.179:0/2566374831 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 -- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e62f0 msgr2=0x7f79600e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e62f0 0x7f79600e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 -- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79600e6c80 msgr2=0x7f79600e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79600e6c80 0x7f79600e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 -- 172.21.15.179:0/2566374831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009570 con 0x7f796004f7d0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7966ffd700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c7f0 tx=0x7f795c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c011800 con 0x7f796004f7d0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600e1f20 con 0x7f796004f7d0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c011960 con 0x7f796004f7d0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c011d10 con 0x7f796004f7d0 2024-05-08T22:44:31.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79600e24e0 con 0x7f796004f7d0 2024-05-08T22:44:31.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.071+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79600053d0 con 0x7f796004f7d0 2024-05-08T22:44:31.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.075+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f795c024480 con 0x7f796004f7d0 2024-05-08T22:44:31.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.075+0000 7f794ffff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 0x7f7954068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.075+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f795c0846c0 con 0x7f796004f7d0 2024-05-08T22:44:31.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.075+0000 7f79667fc700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 0x7f7954068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.075+0000 7f79667fc700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 0x7f7954068bf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f796003a940 tx=0x7f7958008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.079+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f795c0517c0 con 0x7f796004f7d0 2024-05-08T22:44:31.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.251+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79600554b0 con 0x7f7954066730 2024-05-08T22:44:31.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.267+0000 7f794ffff700 1 -- 172.21.15.179:0/2566374831 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288297 (secure 0 0 0) 0x7f79600554b0 con 0x7f7954066730 2024-05-08T22:44:31.273 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:31.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 msgr2=0x7f7954068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 0x7f7954068bf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f796003a940 tx=0x7f7958008030 comp rx=0 tx=0).stop 2024-05-08T22:44:31.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 msgr2=0x7f79600cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.277 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c7f0 tx=0x7f795c00cb00 comp rx=0 tx=0).stop 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 shutdown_connections 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7954066730 0x7f7954068bf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79600e6c80 0x7f79600e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79600e62f0 0x7f79600e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 --2- 172.21.15.179:0/2566374831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796004f7d0 0x7f79600cdec0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 >> 172.21.15.179:0/2566374831 conn(0x7f7960054b00 msgr2=0x7f796004d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 shutdown_connections 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.275+0000 7f7967fff700 1 -- 172.21.15.179:0/2566374831 wait complete. 2024-05-08T22:44:31.278 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:44:31.291 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-05-08T22:44:29.621489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270564,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":257011,"kb_avail":754704156,"statfs":{"total":773094113280,"available":772817055744,"internally_reserved":0,"allocated":13750272,"data_stored":6973941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":263179832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001173"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.244627+0000","last_change":"2024-05-08T22:44:27.243185+0000","last_active":"2024-05-08T22:44:27.244627+0000","last_peered":"2024-05-08T22:44:27.244627+0000","last_clean":"2024-05-08T22:44:27.244627+0000","last_became_active":"2024-05-08T22:44:27.242934+0000","last_became_peered":"2024-05-08T22:44:27.242934+0000","last_unstale":"2024-05-08T22:44:27.244627+0000","last_undegraded":"2024-05-08T22:44:27.244627+0000","last_fullsized":"2024-05-08T22:44:27.244627+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:18.331315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434606+0000","last_change":"2024-05-08T22:44:27.254798+0000","last_active":"2024-05-08T22:44:28.434606+0000","last_peered":"2024-05-08T22:44:28.434606+0000","last_clean":"2024-05-08T22:44:28.434606+0000","last_became_active":"2024-05-08T22:44:27.254552+0000","last_became_peered":"2024-05-08T22:44:27.254552+0000","last_unstale":"2024-05-08T22:44:28.434606+0000","last_undegraded":"2024-05-08T22:44:28.434606+0000","last_fullsized":"2024-05-08T22:44:28.434606+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:12.761662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434955+0000","last_change":"2024-05-08T22:44:27.251932+0000","last_active":"2024-05-08T22:44:28.434955+0000","last_peered":"2024-05-08T22:44:28.434955+0000","last_clean":"2024-05-08T22:44:28.434955+0000","last_became_active":"2024-05-08T22:44:27.251723+0000","last_became_peered":"2024-05-08T22:44:27.251723+0000","last_unstale":"2024-05-08T22:44:28.434955+0000","last_undegraded":"2024-05-08T22:44:28.434955+0000","last_fullsized":"2024-05-08T22:44:28.434955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:07.371835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225855+0000","last_change":"2024-05-08T22:44:27.231377+0000","last_active":"2024-05-08T22:44:28.225855+0000","last_peered":"2024-05-08T22:44:28.225855+0000","last_clean":"2024-05-08T22:44:28.225855+0000","last_became_active":"2024-05-08T22:44:27.231118+0000","last_became_peered":"2024-05-08T22:44:27.231118+0000","last_unstale":"2024-05-08T22:44:28.225855+0000","last_undegraded":"2024-05-08T22:44:28.225855+0000","last_fullsized":"2024-05-08T22:44:28.225855+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:39.074254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.244653+0000","last_change":"2024-05-08T22:44:27.243346+0000","last_active":"2024-05-08T22:44:27.244653+0000","last_peered":"2024-05-08T22:44:27.244653+0000","last_clean":"2024-05-08T22:44:27.244653+0000","last_became_active":"2024-05-08T22:44:27.242455+0000","last_became_peered":"2024-05-08T22:44:27.242455+0000","last_unstale":"2024-05-08T22:44:27.244653+0000","last_undegraded":"2024-05-08T22:44:27.244653+0000","last_fullsized":"2024-05-08T22:44:27.244653+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:53.437189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435768+0000","last_change":"2024-05-08T22:44:27.235148+0000","last_active":"2024-05-08T22:44:28.435768+0000","last_peered":"2024-05-08T22:44:28.435768+0000","last_clean":"2024-05-08T22:44:28.435768+0000","last_became_active":"2024-05-08T22:44:27.234421+0000","last_became_peered":"2024-05-08T22:44:27.234421+0000","last_unstale":"2024-05-08T22:44:28.435768+0000","last_undegraded":"2024-05-08T22:44:28.435768+0000","last_fullsized":"2024-05-08T22:44:28.435768+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:19.244578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433989+0000","last_change":"2024-05-08T22:44:27.250795+0000","last_active":"2024-05-08T22:44:28.433989+0000","last_peered":"2024-05-08T22:44:28.433989+0000","last_clean":"2024-05-08T22:44:28.433989+0000","last_became_active":"2024-05-08T22:44:27.250533+0000","last_became_peered":"2024-05-08T22:44:27.250533+0000","last_unstale":"2024-05-08T22:44:28.433989+0000","last_undegraded":"2024-05-08T22:44:28.433989+0000","last_fullsized":"2024-05-08T22:44:28.433989+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:14.232235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433091+0000","last_change":"2024-05-08T22:44:27.251831+0000","last_active":"2024-05-08T22:44:28.433091+0000","last_peered":"2024-05-08T22:44:28.433091+0000","last_clean":"2024-05-08T22:44:28.433091+0000","last_became_active":"2024-05-08T22:44:27.251632+0000","last_became_peered":"2024-05-08T22:44:27.251632+0000","last_unstale":"2024-05-08T22:44:28.433091+0000","last_undegraded":"2024-05-08T22:44:28.433091+0000","last_fullsized":"2024-05-08T22:44:28.433091+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:26.921335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226887+0000","last_change":"2024-05-08T22:44:27.252994+0000","last_active":"2024-05-08T22:44:28.226887+0000","last_peered":"2024-05-08T22:44:28.226887+0000","last_clean":"2024-05-08T22:44:28.226887+0000","last_became_active":"2024-05-08T22:44:27.252241+0000","last_became_peered":"2024-05-08T22:44:27.252241+0000","last_unstale":"2024-05-08T22:44:28.226887+0000","last_undegraded":"2024-05-08T22:44:28.226887+0000","last_fullsized":"2024-05-08T22:44:28.226887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:43.444745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435910+0000","last_change":"2024-05-08T22:44:27.238660+0000","last_active":"2024-05-08T22:44:28.435910+0000","last_peered":"2024-05-08T22:44:28.435910+0000","last_clean":"2024-05-08T22:44:28.435910+0000","last_became_active":"2024-05-08T22:44:27.234854+0000","last_became_peered":"2024-05-08T22:44:27.234854+0000","last_unstale":"2024-05-08T22:44:28.435910+0000","last_undegraded":"2024-05-08T22:44:28.435910+0000","last_fullsized":"2024-05-08T22:44:28.435910+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:26.651963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.245658+0000","last_change":"2024-05-08T22:44:27.244982+0000","last_active":"2024-05-08T22:44:27.245658+0000","last_peered":"2024-05-08T22:44:27.245658+0000","last_clean":"2024-05-08T22:44:27.245658+0000","last_became_active":"2024-05-08T22:44:27.244719+0000","last_became_peered":"2024-05-08T22:44:27.244719+0000","last_unstale":"2024-05-08T22:44:27.245658+0000","last_undegraded":"2024-05-08T22:44:27.245658+0000","last_fullsized":"2024-05-08T22:44:27.245658+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434125+0000","last_change":"2024-05-08T22:44:27.236479+0000","last_active":"2024-05-08T22:44:28.434125+0000","last_peered":"2024-05-08T22:44:28.434125+0000","last_clean":"2024-05-08T22:44:28.434125+0000","last_became_active":"2024-05-08T22:44:27.235046+0000","last_became_peered":"2024-05-08T22:44:27.235046+0000","last_unstale":"2024-05-08T22:44:28.434125+0000","last_undegraded":"2024-05-08T22:44:28.434125+0000","last_fullsized":"2024-05-08T22:44:28.434125+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:49.488792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434394+0000","last_change":"2024-05-08T22:44:27.247229+0000","last_active":"2024-05-08T22:44:28.434394+0000","last_peered":"2024-05-08T22:44:28.434394+0000","last_clean":"2024-05-08T22:44:28.434394+0000","last_became_active":"2024-05-08T22:44:27.246948+0000","last_became_peered":"2024-05-08T22:44:27.246948+0000","last_unstale":"2024-05-08T22:44:28.434394+0000","last_undegraded":"2024-05-08T22:44:28.434394+0000","last_fullsized":"2024-05-08T22:44:28.434394+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:37.623210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434017+0000","last_change":"2024-05-08T22:44:27.250037+0000","last_active":"2024-05-08T22:44:28.434017+0000","last_peered":"2024-05-08T22:44:28.434017+0000","last_clean":"2024-05-08T22:44:28.434017+0000","last_became_active":"2024-05-08T22:44:27.249810+0000","last_became_peered":"2024-05-08T22:44:27.249810+0000","last_unstale":"2024-05-08T22:44:28.434017+0000","last_undegraded":"2024-05-08T22:44:28.434017+0000","last_fullsized":"2024-05-08T22:44:28.434017+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:50.087190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.241128+0000","last_change":"2024-05-08T22:44:27.240295+0000","last_active":"2024-05-08T22:44:27.241128+0000","last_peered":"2024-05-08T22:44:27.241128+0000","last_clean":"2024-05-08T22:44:27.241128+0000","last_became_active":"2024-05-08T22:44:27.239992+0000","last_became_peered":"2024-05-08T22:44:27.239992+0000","last_unstale":"2024-05-08T22:44:27.241128+0000","last_undegraded":"2024-05-08T22:44:27.241128+0000","last_fullsized":"2024-05-08T22:44:27.241128+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:17.911331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433080+0000","last_change":"2024-05-08T22:44:27.233255+0000","last_active":"2024-05-08T22:44:28.433080+0000","last_peered":"2024-05-08T22:44:28.433080+0000","last_clean":"2024-05-08T22:44:28.433080+0000","last_became_active":"2024-05-08T22:44:27.232129+0000","last_became_peered":"2024-05-08T22:44:27.232129+0000","last_unstale":"2024-05-08T22:44:28.433080+0000","last_undegraded":"2024-05-08T22:44:28.433080+0000","last_fullsized":"2024-05-08T22:44:28.433080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:43.472298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435181+0000","last_change":"2024-05-08T22:44:27.250127+0000","last_active":"2024-05-08T22:44:28.435181+0000","last_peered":"2024-05-08T22:44:28.435181+0000","last_clean":"2024-05-08T22:44:28.435181+0000","last_became_active":"2024-05-08T22:44:27.249936+0000","last_became_peered":"2024-05-08T22:44:27.249936+0000","last_unstale":"2024-05-08T22:44:28.435181+0000","last_undegraded":"2024-05-08T22:44:28.435181+0000","last_fullsized":"2024-05-08T22:44:28.435181+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:08.412148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434768+0000","last_change":"2024-05-08T22:44:27.254771+0000","last_active":"2024-05-08T22:44:28.434768+0000","last_peered":"2024-05-08T22:44:28.434768+0000","last_clean":"2024-05-08T22:44:28.434768+0000","last_became_active":"2024-05-08T22:44:27.254531+0000","last_became_peered":"2024-05-08T22:44:27.254531+0000","last_unstale":"2024-05-08T22:44:28.434768+0000","last_undegraded":"2024-05-08T22:44:28.434768+0000","last_fullsized":"2024-05-08T22:44:28.434768+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:07.783492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225162+0000","last_change":"2024-05-08T22:44:27.247540+0000","last_active":"2024-05-08T22:44:28.225162+0000","last_peered":"2024-05-08T22:44:28.225162+0000","last_clean":"2024-05-08T22:44:28.225162+0000","last_became_active":"2024-05-08T22:44:27.246565+0000","last_became_peered":"2024-05-08T22:44:27.246565+0000","last_unstale":"2024-05-08T22:44:28.225162+0000","last_undegraded":"2024-05-08T22:44:28.225162+0000","last_fullsized":"2024-05-08T22:44:28.225162+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.204876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435870+0000","last_change":"2024-05-08T22:44:27.244141+0000","last_active":"2024-05-08T22:44:28.435870+0000","last_peered":"2024-05-08T22:44:28.435870+0000","last_clean":"2024-05-08T22:44:28.435870+0000","last_became_active":"2024-05-08T22:44:27.243898+0000","last_became_peered":"2024-05-08T22:44:27.243898+0000","last_unstale":"2024-05-08T22:44:28.435870+0000","last_undegraded":"2024-05-08T22:44:28.435870+0000","last_fullsized":"2024-05-08T22:44:28.435870+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:11.225537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226140+0000","last_change":"2024-05-08T22:44:27.256514+0000","last_active":"2024-05-08T22:44:28.226140+0000","last_peered":"2024-05-08T22:44:28.226140+0000","last_clean":"2024-05-08T22:44:28.226140+0000","last_became_active":"2024-05-08T22:44:27.253033+0000","last_became_peered":"2024-05-08T22:44:27.253033+0000","last_unstale":"2024-05-08T22:44:28.226140+0000","last_undegraded":"2024-05-08T22:44:28.226140+0000","last_fullsized":"2024-05-08T22:44:28.226140+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:57.586085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.237785+0000","last_change":"2024-05-08T22:44:27.236167+0000","last_active":"2024-05-08T22:44:27.237785+0000","last_peered":"2024-05-08T22:44:27.237785+0000","last_clean":"2024-05-08T22:44:27.237785+0000","last_became_active":"2024-05-08T22:44:27.235784+0000","last_became_peered":"2024-05-08T22:44:27.235784+0000","last_unstale":"2024-05-08T22:44:27.237785+0000","last_undegraded":"2024-05-08T22:44:27.237785+0000","last_fullsized":"2024-05-08T22:44:27.237785+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:30.446672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225658+0000","last_change":"2024-05-08T22:44:27.239063+0000","last_active":"2024-05-08T22:44:28.225658+0000","last_peered":"2024-05-08T22:44:28.225658+0000","last_clean":"2024-05-08T22:44:28.225658+0000","last_became_active":"2024-05-08T22:44:27.238775+0000","last_became_peered":"2024-05-08T22:44:27.238775+0000","last_unstale":"2024-05-08T22:44:28.225658+0000","last_undegraded":"2024-05-08T22:44:28.225658+0000","last_fullsized":"2024-05-08T22:44:28.225658+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:20.426660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.430832+0000","last_change":"2024-05-08T22:44:27.250597+0000","last_active":"2024-05-08T22:44:28.430832+0000","last_peered":"2024-05-08T22:44:28.430832+0000","last_clean":"2024-05-08T22:44:28.430832+0000","last_became_active":"2024-05-08T22:44:27.250153+0000","last_became_peered":"2024-05-08T22:44:27.250153+0000","last_unstale":"2024-05-08T22:44:28.430832+0000","last_undegraded":"2024-05-08T22:44:28.430832+0000","last_fullsized":"2024-05-08T22:44:28.430832+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:28.921494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432011+0000","last_change":"2024-05-08T22:44:27.241822+0000","last_active":"2024-05-08T22:44:28.432011+0000","last_peered":"2024-05-08T22:44:28.432011+0000","last_clean":"2024-05-08T22:44:28.432011+0000","last_became_active":"2024-05-08T22:44:27.241541+0000","last_became_peered":"2024-05-08T22:44:27.241541+0000","last_unstale":"2024-05-08T22:44:28.432011+0000","last_undegraded":"2024-05-08T22:44:28.432011+0000","last_fullsized":"2024-05-08T22:44:28.432011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:47.989051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436083+0000","last_change":"2024-05-08T22:44:27.235866+0000","last_active":"2024-05-08T22:44:28.436083+0000","last_peered":"2024-05-08T22:44:28.436083+0000","last_clean":"2024-05-08T22:44:28.436083+0000","last_became_active":"2024-05-08T22:44:27.235496+0000","last_became_peered":"2024-05-08T22:44:27.235496+0000","last_unstale":"2024-05-08T22:44:28.436083+0000","last_undegraded":"2024-05-08T22:44:28.436083+0000","last_fullsized":"2024-05-08T22:44:28.436083+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:03.613627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225316+0000","last_change":"2024-05-08T22:44:27.253430+0000","last_active":"2024-05-08T22:44:28.225316+0000","last_peered":"2024-05-08T22:44:28.225316+0000","last_clean":"2024-05-08T22:44:28.225316+0000","last_became_active":"2024-05-08T22:44:27.253221+0000","last_became_peered":"2024-05-08T22:44:27.253221+0000","last_unstale":"2024-05-08T22:44:28.225316+0000","last_undegraded":"2024-05-08T22:44:28.225316+0000","last_fullsized":"2024-05-08T22:44:28.225316+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:27.616461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434347+0000","last_change":"2024-05-08T22:44:27.242683+0000","last_active":"2024-05-08T22:44:28.434347+0000","last_peered":"2024-05-08T22:44:28.434347+0000","last_clean":"2024-05-08T22:44:28.434347+0000","last_became_active":"2024-05-08T22:44:27.242256+0000","last_became_peered":"2024-05-08T22:44:27.242256+0000","last_unstale":"2024-05-08T22:44:28.434347+0000","last_undegraded":"2024-05-08T22:44:28.434347+0000","last_fullsized":"2024-05-08T22:44:28.434347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:36.298015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436450+0000","last_change":"2024-05-08T22:44:27.237727+0000","last_active":"2024-05-08T22:44:28.436450+0000","last_peered":"2024-05-08T22:44:28.436450+0000","last_clean":"2024-05-08T22:44:28.436450+0000","last_became_active":"2024-05-08T22:44:27.237397+0000","last_became_peered":"2024-05-08T22:44:27.237397+0000","last_unstale":"2024-05-08T22:44:28.436450+0000","last_undegraded":"2024-05-08T22:44:28.436450+0000","last_fullsized":"2024-05-08T22:44:28.436450+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:19.170375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436019+0000","last_change":"2024-05-08T22:44:27.237197+0000","last_active":"2024-05-08T22:44:28.436019+0000","last_peered":"2024-05-08T22:44:28.436019+0000","last_clean":"2024-05-08T22:44:28.436019+0000","last_became_active":"2024-05-08T22:44:27.236865+0000","last_became_peered":"2024-05-08T22:44:27.236865+0000","last_unstale":"2024-05-08T22:44:28.436019+0000","last_undegraded":"2024-05-08T22:44:28.436019+0000","last_fullsized":"2024-05-08T22:44:28.436019+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:23.582534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431210+0000","last_change":"2024-05-08T22:44:27.241198+0000","last_active":"2024-05-08T22:44:28.431210+0000","last_peered":"2024-05-08T22:44:28.431210+0000","last_clean":"2024-05-08T22:44:28.431210+0000","last_became_active":"2024-05-08T22:44:27.237468+0000","last_became_peered":"2024-05-08T22:44:27.237468+0000","last_unstale":"2024-05-08T22:44:28.431210+0000","last_undegraded":"2024-05-08T22:44:28.431210+0000","last_fullsized":"2024-05-08T22:44:28.431210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:30.571208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.430876+0000","last_change":"2024-05-08T22:44:27.250322+0000","last_active":"2024-05-08T22:44:28.430876+0000","last_peered":"2024-05-08T22:44:28.430876+0000","last_clean":"2024-05-08T22:44:28.430876+0000","last_became_active":"2024-05-08T22:44:27.247868+0000","last_became_peered":"2024-05-08T22:44:27.247868+0000","last_unstale":"2024-05-08T22:44:28.430876+0000","last_undegraded":"2024-05-08T22:44:28.430876+0000","last_fullsized":"2024-05-08T22:44:28.430876+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435403+0000","last_change":"2024-05-08T22:44:27.237697+0000","last_active":"2024-05-08T22:44:28.435403+0000","last_peered":"2024-05-08T22:44:28.435403+0000","last_clean":"2024-05-08T22:44:28.435403+0000","last_became_active":"2024-05-08T22:44:27.234111+0000","last_became_peered":"2024-05-08T22:44:27.234111+0000","last_unstale":"2024-05-08T22:44:28.435403+0000","last_undegraded":"2024-05-08T22:44:28.435403+0000","last_fullsized":"2024-05-08T22:44:28.435403+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:43.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"126'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224893+0000","last_change":"2024-05-08T22:44:26.849125+0000","last_active":"2024-05-08T22:44:28.224893+0000","last_peered":"2024-05-08T22:44:28.224893+0000","last_clean":"2024-05-08T22:44:28.224893+0000","last_became_active":"2024-05-08T22:44:26.848838+0000","last_became_peered":"2024-05-08T22:44:26.848838+0000","last_unstale":"2024-05-08T22:44:28.224893+0000","last_undegraded":"2024-05-08T22:44:28.224893+0000","last_fullsized":"2024-05-08T22:44:28.224893+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:15.506911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435568+0000","last_change":"2024-05-08T22:44:27.237005+0000","last_active":"2024-05-08T22:44:28.435568+0000","last_peered":"2024-05-08T22:44:28.435568+0000","last_clean":"2024-05-08T22:44:28.435568+0000","last_became_active":"2024-05-08T22:44:27.236763+0000","last_became_peered":"2024-05-08T22:44:27.236763+0000","last_unstale":"2024-05-08T22:44:28.435568+0000","last_undegraded":"2024-05-08T22:44:28.435568+0000","last_fullsized":"2024-05-08T22:44:28.435568+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:21.155072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436062+0000","last_change":"2024-05-08T22:44:26.856476+0000","last_active":"2024-05-08T22:44:28.436062+0000","last_peered":"2024-05-08T22:44:28.436062+0000","last_clean":"2024-05-08T22:44:28.436062+0000","last_became_active":"2024-05-08T22:44:26.856204+0000","last_became_peered":"2024-05-08T22:44:26.856204+0000","last_unstale":"2024-05-08T22:44:28.436062+0000","last_undegraded":"2024-05-08T22:44:28.436062+0000","last_fullsized":"2024-05-08T22:44:28.436062+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:59.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431809+0000","last_change":"2024-05-08T22:44:27.246729+0000","last_active":"2024-05-08T22:44:28.431809+0000","last_peered":"2024-05-08T22:44:28.431809+0000","last_clean":"2024-05-08T22:44:28.431809+0000","last_became_active":"2024-05-08T22:44:27.246361+0000","last_became_peered":"2024-05-08T22:44:27.246361+0000","last_unstale":"2024-05-08T22:44:28.431809+0000","last_undegraded":"2024-05-08T22:44:28.431809+0000","last_fullsized":"2024-05-08T22:44:28.431809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:45.337960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226532+0000","last_change":"2024-05-08T22:44:26.854243+0000","last_active":"2024-05-08T22:44:28.226532+0000","last_peered":"2024-05-08T22:44:28.226532+0000","last_clean":"2024-05-08T22:44:28.226532+0000","last_became_active":"2024-05-08T22:44:26.853945+0000","last_became_peered":"2024-05-08T22:44:26.853945+0000","last_unstale":"2024-05-08T22:44:28.226532+0000","last_undegraded":"2024-05-08T22:44:28.226532+0000","last_fullsized":"2024-05-08T22:44:28.226532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:19.060876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226963+0000","last_change":"2024-05-08T22:44:27.252893+0000","last_active":"2024-05-08T22:44:28.226963+0000","last_peered":"2024-05-08T22:44:28.226963+0000","last_clean":"2024-05-08T22:44:28.226963+0000","last_became_active":"2024-05-08T22:44:27.252565+0000","last_became_peered":"2024-05-08T22:44:27.252565+0000","last_unstale":"2024-05-08T22:44:28.226963+0000","last_undegraded":"2024-05-08T22:44:28.226963+0000","last_fullsized":"2024-05-08T22:44:28.226963+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:39.962232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435574+0000","last_change":"2024-05-08T22:44:26.857400+0000","last_active":"2024-05-08T22:44:28.435574+0000","last_peered":"2024-05-08T22:44:28.435574+0000","last_clean":"2024-05-08T22:44:28.435574+0000","last_became_active":"2024-05-08T22:44:26.857165+0000","last_became_peered":"2024-05-08T22:44:26.857165+0000","last_unstale":"2024-05-08T22:44:28.435574+0000","last_undegraded":"2024-05-08T22:44:28.435574+0000","last_fullsized":"2024-05-08T22:44:28.435574+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:15.076936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433767+0000","last_change":"2024-05-08T22:44:27.234690+0000","last_active":"2024-05-08T22:44:28.433767+0000","last_peered":"2024-05-08T22:44:28.433767+0000","last_clean":"2024-05-08T22:44:28.433767+0000","last_became_active":"2024-05-08T22:44:27.233902+0000","last_became_peered":"2024-05-08T22:44:27.233902+0000","last_unstale":"2024-05-08T22:44:28.433767+0000","last_undegraded":"2024-05-08T22:44:28.433767+0000","last_fullsized":"2024-05-08T22:44:28.433767+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:05.063358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.242397+0000","last_change":"2024-05-08T22:44:26.853540+0000","last_active":"2024-05-08T22:44:27.242397+0000","last_peered":"2024-05-08T22:44:27.242397+0000","last_clean":"2024-05-08T22:44:27.242397+0000","last_became_active":"2024-05-08T22:44:26.853234+0000","last_became_peered":"2024-05-08T22:44:26.853234+0000","last_unstale":"2024-05-08T22:44:27.242397+0000","last_undegraded":"2024-05-08T22:44:27.242397+0000","last_fullsized":"2024-05-08T22:44:27.242397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:44.136301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.241110+0000","last_change":"2024-05-08T22:44:27.240211+0000","last_active":"2024-05-08T22:44:27.241110+0000","last_peered":"2024-05-08T22:44:27.241110+0000","last_clean":"2024-05-08T22:44:27.241110+0000","last_became_active":"2024-05-08T22:44:27.239893+0000","last_became_peered":"2024-05-08T22:44:27.239893+0000","last_unstale":"2024-05-08T22:44:27.241110+0000","last_undegraded":"2024-05-08T22:44:27.241110+0000","last_fullsized":"2024-05-08T22:44:27.241110+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:14.568501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434260+0000","last_change":"2024-05-08T22:44:26.858947+0000","last_active":"2024-05-08T22:44:28.434260+0000","last_peered":"2024-05-08T22:44:28.434260+0000","last_clean":"2024-05-08T22:44:28.434260+0000","last_became_active":"2024-05-08T22:44:26.857501+0000","last_became_peered":"2024-05-08T22:44:26.857501+0000","last_unstale":"2024-05-08T22:44:28.434260+0000","last_undegraded":"2024-05-08T22:44:28.434260+0000","last_fullsized":"2024-05-08T22:44:28.434260+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:32.349718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434293+0000","last_change":"2024-05-08T22:44:27.246721+0000","last_active":"2024-05-08T22:44:28.434293+0000","last_peered":"2024-05-08T22:44:28.434293+0000","last_clean":"2024-05-08T22:44:28.434293+0000","last_became_active":"2024-05-08T22:44:27.245475+0000","last_became_peered":"2024-05-08T22:44:27.245475+0000","last_unstale":"2024-05-08T22:44:28.434293+0000","last_undegraded":"2024-05-08T22:44:28.434293+0000","last_fullsized":"2024-05-08T22:44:28.434293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:05.420141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431226+0000","last_change":"2024-05-08T22:44:26.850463+0000","last_active":"2024-05-08T22:44:28.431226+0000","last_peered":"2024-05-08T22:44:28.431226+0000","last_clean":"2024-05-08T22:44:28.431226+0000","last_became_active":"2024-05-08T22:44:26.849473+0000","last_became_peered":"2024-05-08T22:44:26.849473+0000","last_unstale":"2024-05-08T22:44:28.431226+0000","last_undegraded":"2024-05-08T22:44:28.431226+0000","last_fullsized":"2024-05-08T22:44:28.431226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:44.256283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434564+0000","last_change":"2024-05-08T22:44:27.238899+0000","last_active":"2024-05-08T22:44:28.434564+0000","last_peered":"2024-05-08T22:44:28.434564+0000","last_clean":"2024-05-08T22:44:28.434564+0000","last_became_active":"2024-05-08T22:44:27.234886+0000","last_became_peered":"2024-05-08T22:44:27.234886+0000","last_unstale":"2024-05-08T22:44:28.434564+0000","last_undegraded":"2024-05-08T22:44:28.434564+0000","last_fullsized":"2024-05-08T22:44:28.434564+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.204876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433734+0000","last_change":"2024-05-08T22:44:26.857187+0000","last_active":"2024-05-08T22:44:28.433734+0000","last_peered":"2024-05-08T22:44:28.433734+0000","last_clean":"2024-05-08T22:44:28.433734+0000","last_became_active":"2024-05-08T22:44:26.856905+0000","last_became_peered":"2024-05-08T22:44:26.856905+0000","last_unstale":"2024-05-08T22:44:28.433734+0000","last_undegraded":"2024-05-08T22:44:28.433734+0000","last_fullsized":"2024-05-08T22:44:28.433734+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:50.425391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224791+0000","last_change":"2024-05-08T22:44:27.253706+0000","last_active":"2024-05-08T22:44:28.224791+0000","last_peered":"2024-05-08T22:44:28.224791+0000","last_clean":"2024-05-08T22:44:28.224791+0000","last_became_active":"2024-05-08T22:44:27.253473+0000","last_became_peered":"2024-05-08T22:44:27.253473+0000","last_unstale":"2024-05-08T22:44:28.224791+0000","last_undegraded":"2024-05-08T22:44:28.224791+0000","last_fullsized":"2024-05-08T22:44:28.224791+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:26.651963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225041+0000","last_change":"2024-05-08T22:44:26.854460+0000","last_active":"2024-05-08T22:44:28.225041+0000","last_peered":"2024-05-08T22:44:28.225041+0000","last_clean":"2024-05-08T22:44:28.225041+0000","last_became_active":"2024-05-08T22:44:26.854157+0000","last_became_peered":"2024-05-08T22:44:26.854157+0000","last_unstale":"2024-05-08T22:44:28.225041+0000","last_undegraded":"2024-05-08T22:44:28.225041+0000","last_fullsized":"2024-05-08T22:44:28.225041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:43.817777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431494+0000","last_change":"2024-05-08T22:44:27.242915+0000","last_active":"2024-05-08T22:44:28.431494+0000","last_peered":"2024-05-08T22:44:28.431494+0000","last_clean":"2024-05-08T22:44:28.431494+0000","last_became_active":"2024-05-08T22:44:27.242650+0000","last_became_peered":"2024-05-08T22:44:27.242650+0000","last_unstale":"2024-05-08T22:44:28.431494+0000","last_undegraded":"2024-05-08T22:44:28.431494+0000","last_fullsized":"2024-05-08T22:44:28.431494+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:03.843907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431816+0000","last_change":"2024-05-08T22:44:26.859156+0000","last_active":"2024-05-08T22:44:28.431816+0000","last_peered":"2024-05-08T22:44:28.431816+0000","last_clean":"2024-05-08T22:44:28.431816+0000","last_became_active":"2024-05-08T22:44:26.858902+0000","last_became_peered":"2024-05-08T22:44:26.858902+0000","last_unstale":"2024-05-08T22:44:28.431816+0000","last_undegraded":"2024-05-08T22:44:28.431816+0000","last_fullsized":"2024-05-08T22:44:28.431816+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:55.996129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434915+0000","last_change":"2024-05-08T22:44:27.250024+0000","last_active":"2024-05-08T22:44:28.434915+0000","last_peered":"2024-05-08T22:44:28.434915+0000","last_clean":"2024-05-08T22:44:28.434915+0000","last_became_active":"2024-05-08T22:44:27.249766+0000","last_became_peered":"2024-05-08T22:44:27.249766+0000","last_unstale":"2024-05-08T22:44:28.434915+0000","last_undegraded":"2024-05-08T22:44:28.434915+0000","last_fullsized":"2024-05-08T22:44:28.434915+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:39.290129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"126'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431720+0000","last_change":"2024-05-08T22:44:26.853398+0000","last_active":"2024-05-08T22:44:28.431720+0000","last_peered":"2024-05-08T22:44:28.431720+0000","last_clean":"2024-05-08T22:44:28.431720+0000","last_became_active":"2024-05-08T22:44:26.851432+0000","last_became_peered":"2024-05-08T22:44:26.851432+0000","last_unstale":"2024-05-08T22:44:28.431720+0000","last_undegraded":"2024-05-08T22:44:28.431720+0000","last_fullsized":"2024-05-08T22:44:28.431720+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:05.497573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432013+0000","last_change":"2024-05-08T22:44:27.241947+0000","last_active":"2024-05-08T22:44:28.432013+0000","last_peered":"2024-05-08T22:44:28.432013+0000","last_clean":"2024-05-08T22:44:28.432013+0000","last_became_active":"2024-05-08T22:44:27.241681+0000","last_became_peered":"2024-05-08T22:44:27.241681+0000","last_unstale":"2024-05-08T22:44:28.432013+0000","last_undegraded":"2024-05-08T22:44:28.432013+0000","last_fullsized":"2024-05-08T22:44:28.432013+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:35.223658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"126'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434346+0000","last_change":"2024-05-08T22:44:26.856488+0000","last_active":"2024-05-08T22:44:28.434346+0000","last_peered":"2024-05-08T22:44:28.434346+0000","last_clean":"2024-05-08T22:44:28.434346+0000","last_became_active":"2024-05-08T22:44:26.856178+0000","last_became_peered":"2024-05-08T22:44:26.856178+0000","last_unstale":"2024-05-08T22:44:28.434346+0000","last_undegraded":"2024-05-08T22:44:28.434346+0000","last_fullsized":"2024-05-08T22:44:28.434346+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:36.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224639+0000","last_change":"2024-05-08T22:44:27.242772+0000","last_active":"2024-05-08T22:44:28.224639+0000","last_peered":"2024-05-08T22:44:28.224639+0000","last_clean":"2024-05-08T22:44:28.224639+0000","last_became_active":"2024-05-08T22:44:27.241723+0000","last_became_peered":"2024-05-08T22:44:27.241723+0000","last_unstale":"2024-05-08T22:44:28.224639+0000","last_undegraded":"2024-05-08T22:44:28.224639+0000","last_fullsized":"2024-05-08T22:44:28.224639+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:05.063358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435415+0000","last_change":"2024-05-08T22:44:26.855885+0000","last_active":"2024-05-08T22:44:28.435415+0000","last_peered":"2024-05-08T22:44:28.435415+0000","last_clean":"2024-05-08T22:44:28.435415+0000","last_became_active":"2024-05-08T22:44:26.855627+0000","last_became_peered":"2024-05-08T22:44:26.855627+0000","last_unstale":"2024-05-08T22:44:28.435415+0000","last_undegraded":"2024-05-08T22:44:28.435415+0000","last_fullsized":"2024-05-08T22:44:28.435415+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:50.156208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434058+0000","last_change":"2024-05-08T22:44:27.259680+0000","last_active":"2024-05-08T22:44:28.434058+0000","last_peered":"2024-05-08T22:44:28.434058+0000","last_clean":"2024-05-08T22:44:28.434058+0000","last_became_active":"2024-05-08T22:44:27.259427+0000","last_became_peered":"2024-05-08T22:44:27.259427+0000","last_unstale":"2024-05-08T22:44:28.434058+0000","last_undegraded":"2024-05-08T22:44:28.434058+0000","last_fullsized":"2024-05-08T22:44:28.434058+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:27.934776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434876+0000","last_change":"2024-05-08T22:44:26.858383+0000","last_active":"2024-05-08T22:44:28.434876+0000","last_peered":"2024-05-08T22:44:28.434876+0000","last_clean":"2024-05-08T22:44:28.434876+0000","last_became_active":"2024-05-08T22:44:26.858034+0000","last_became_peered":"2024-05-08T22:44:26.858034+0000","last_unstale":"2024-05-08T22:44:28.434876+0000","last_undegraded":"2024-05-08T22:44:28.434876+0000","last_fullsized":"2024-05-08T22:44:28.434876+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:25.641268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436144+0000","last_change":"2024-05-08T22:44:27.236192+0000","last_active":"2024-05-08T22:44:28.436144+0000","last_peered":"2024-05-08T22:44:28.436144+0000","last_clean":"2024-05-08T22:44:28.436144+0000","last_became_active":"2024-05-08T22:44:27.235945+0000","last_became_peered":"2024-05-08T22:44:27.235945+0000","last_unstale":"2024-05-08T22:44:28.436144+0000","last_undegraded":"2024-05-08T22:44:28.436144+0000","last_fullsized":"2024-05-08T22:44:28.436144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:49:30.921993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433565+0000","last_change":"2024-05-08T22:44:26.852752+0000","last_active":"2024-05-08T22:44:28.433565+0000","last_peered":"2024-05-08T22:44:28.433565+0000","last_clean":"2024-05-08T22:44:28.433565+0000","last_became_active":"2024-05-08T22:44:26.852486+0000","last_became_peered":"2024-05-08T22:44:26.852486+0000","last_unstale":"2024-05-08T22:44:28.433565+0000","last_undegraded":"2024-05-08T22:44:28.433565+0000","last_fullsized":"2024-05-08T22:44:28.433565+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:47.764271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431480+0000","last_change":"2024-05-08T22:44:27.246835+0000","last_active":"2024-05-08T22:44:28.431480+0000","last_peered":"2024-05-08T22:44:28.431480+0000","last_clean":"2024-05-08T22:44:28.431480+0000","last_became_active":"2024-05-08T22:44:27.246514+0000","last_became_peered":"2024-05-08T22:44:27.246514+0000","last_unstale":"2024-05-08T22:44:28.431480+0000","last_undegraded":"2024-05-08T22:44:28.431480+0000","last_fullsized":"2024-05-08T22:44:28.431480+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:46.361700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435318+0000","last_change":"2024-05-08T22:44:26.855821+0000","last_active":"2024-05-08T22:44:28.435318+0000","last_peered":"2024-05-08T22:44:28.435318+0000","last_clean":"2024-05-08T22:44:28.435318+0000","last_became_active":"2024-05-08T22:44:26.855545+0000","last_became_peered":"2024-05-08T22:44:26.855545+0000","last_unstale":"2024-05-08T22:44:28.435318+0000","last_undegraded":"2024-05-08T22:44:28.435318+0000","last_fullsized":"2024-05-08T22:44:28.435318+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:59.944611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.237012+0000","last_change":"2024-05-08T22:44:27.234998+0000","last_active":"2024-05-08T22:44:27.237012+0000","last_peered":"2024-05-08T22:44:27.237012+0000","last_clean":"2024-05-08T22:44:27.237012+0000","last_became_active":"2024-05-08T22:44:27.234701+0000","last_became_peered":"2024-05-08T22:44:27.234701+0000","last_unstale":"2024-05-08T22:44:27.237012+0000","last_undegraded":"2024-05-08T22:44:27.237012+0000","last_fullsized":"2024-05-08T22:44:27.237012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:02.054358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436174+0000","last_change":"2024-05-08T22:44:26.858280+0000","last_active":"2024-05-08T22:44:28.436174+0000","last_peered":"2024-05-08T22:44:28.436174+0000","last_clean":"2024-05-08T22:44:28.436174+0000","last_became_active":"2024-05-08T22:44:26.858043+0000","last_became_peered":"2024-05-08T22:44:26.858043+0000","last_unstale":"2024-05-08T22:44:28.436174+0000","last_undegraded":"2024-05-08T22:44:28.436174+0000","last_fullsized":"2024-05-08T22:44:28.436174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:22.774203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436120+0000","last_change":"2024-05-08T22:44:27.240949+0000","last_active":"2024-05-08T22:44:28.436120+0000","last_peered":"2024-05-08T22:44:28.436120+0000","last_clean":"2024-05-08T22:44:28.436120+0000","last_became_active":"2024-05-08T22:44:27.240707+0000","last_became_peered":"2024-05-08T22:44:27.240707+0000","last_unstale":"2024-05-08T22:44:28.436120+0000","last_undegraded":"2024-05-08T22:44:28.436120+0000","last_fullsized":"2024-05-08T22:44:28.436120+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:02.625962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226558+0000","last_change":"2024-05-08T22:44:26.852443+0000","last_active":"2024-05-08T22:44:28.226558+0000","last_peered":"2024-05-08T22:44:28.226558+0000","last_clean":"2024-05-08T22:44:28.226558+0000","last_became_active":"2024-05-08T22:44:26.852168+0000","last_became_peered":"2024-05-08T22:44:26.852168+0000","last_unstale":"2024-05-08T22:44:28.226558+0000","last_undegraded":"2024-05-08T22:44:28.226558+0000","last_fullsized":"2024-05-08T22:44:28.226558+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:10.425902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432140+0000","last_change":"2024-05-08T22:44:27.257729+0000","last_active":"2024-05-08T22:44:28.432140+0000","last_peered":"2024-05-08T22:44:28.432140+0000","last_clean":"2024-05-08T22:44:28.432140+0000","last_became_active":"2024-05-08T22:44:27.257453+0000","last_became_peered":"2024-05-08T22:44:27.257453+0000","last_unstale":"2024-05-08T22:44:28.432140+0000","last_undegraded":"2024-05-08T22:44:28.432140+0000","last_fullsized":"2024-05-08T22:44:28.432140+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:06.344892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225987+0000","last_change":"2024-05-08T22:44:26.856157+0000","last_active":"2024-05-08T22:44:28.225987+0000","last_peered":"2024-05-08T22:44:28.225987+0000","last_clean":"2024-05-08T22:44:28.225987+0000","last_became_active":"2024-05-08T22:44:26.855878+0000","last_became_peered":"2024-05-08T22:44:26.855878+0000","last_unstale":"2024-05-08T22:44:28.225987+0000","last_undegraded":"2024-05-08T22:44:28.225987+0000","last_fullsized":"2024-05-08T22:44:28.225987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:00.304253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433333+0000","last_change":"2024-05-08T22:44:27.240922+0000","last_active":"2024-05-08T22:44:28.433333+0000","last_peered":"2024-05-08T22:44:28.433333+0000","last_clean":"2024-05-08T22:44:28.433333+0000","last_became_active":"2024-05-08T22:44:27.240665+0000","last_became_peered":"2024-05-08T22:44:27.240665+0000","last_unstale":"2024-05-08T22:44:28.433333+0000","last_undegraded":"2024-05-08T22:44:28.433333+0000","last_fullsized":"2024-05-08T22:44:28.433333+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:14.418456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434332+0000","last_change":"2024-05-08T22:44:26.854920+0000","last_active":"2024-05-08T22:44:28.434332+0000","last_peered":"2024-05-08T22:44:28.434332+0000","last_clean":"2024-05-08T22:44:28.434332+0000","last_became_active":"2024-05-08T22:44:26.854563+0000","last_became_peered":"2024-05-08T22:44:26.854563+0000","last_unstale":"2024-05-08T22:44:28.434332+0000","last_undegraded":"2024-05-08T22:44:28.434332+0000","last_fullsized":"2024-05-08T22:44:28.434332+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:37.361168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433388+0000","last_change":"2024-05-08T22:44:27.247027+0000","last_active":"2024-05-08T22:44:28.433388+0000","last_peered":"2024-05-08T22:44:28.433388+0000","last_clean":"2024-05-08T22:44:28.433388+0000","last_became_active":"2024-05-08T22:44:27.246820+0000","last_became_peered":"2024-05-08T22:44:27.246820+0000","last_unstale":"2024-05-08T22:44:28.433388+0000","last_undegraded":"2024-05-08T22:44:28.433388+0000","last_fullsized":"2024-05-08T22:44:28.433388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:27.711668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225757+0000","last_change":"2024-05-08T22:44:26.843380+0000","last_active":"2024-05-08T22:44:28.225757+0000","last_peered":"2024-05-08T22:44:28.225757+0000","last_clean":"2024-05-08T22:44:28.225757+0000","last_became_active":"2024-05-08T22:44:26.843097+0000","last_became_peered":"2024-05-08T22:44:26.843097+0000","last_unstale":"2024-05-08T22:44:28.225757+0000","last_undegraded":"2024-05-08T22:44:28.225757+0000","last_fullsized":"2024-05-08T22:44:28.225757+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:45.702545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434683+0000","last_change":"2024-05-08T22:44:27.251231+0000","last_active":"2024-05-08T22:44:28.434683+0000","last_peered":"2024-05-08T22:44:28.434683+0000","last_clean":"2024-05-08T22:44:28.434683+0000","last_became_active":"2024-05-08T22:44:27.250986+0000","last_became_peered":"2024-05-08T22:44:27.250986+0000","last_unstale":"2024-05-08T22:44:28.434683+0000","last_undegraded":"2024-05-08T22:44:28.434683+0000","last_fullsized":"2024-05-08T22:44:28.434683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:25.785836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433738+0000","last_change":"2024-05-08T22:44:26.858863+0000","last_active":"2024-05-08T22:44:28.433738+0000","last_peered":"2024-05-08T22:44:28.433738+0000","last_clean":"2024-05-08T22:44:28.433738+0000","last_became_active":"2024-05-08T22:44:26.858622+0000","last_became_peered":"2024-05-08T22:44:26.858622+0000","last_unstale":"2024-05-08T22:44:28.433738+0000","last_undegraded":"2024-05-08T22:44:28.433738+0000","last_fullsized":"2024-05-08T22:44:28.433738+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:38.194181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431201+0000","last_change":"2024-05-08T22:44:27.242046+0000","last_active":"2024-05-08T22:44:28.431201+0000","last_peered":"2024-05-08T22:44:28.431201+0000","last_clean":"2024-05-08T22:44:28.431201+0000","last_became_active":"2024-05-08T22:44:27.241782+0000","last_became_peered":"2024-05-08T22:44:27.241782+0000","last_unstale":"2024-05-08T22:44:28.431201+0000","last_undegraded":"2024-05-08T22:44:28.431201+0000","last_fullsized":"2024-05-08T22:44:28.431201+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:17.116798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.223742+0000","last_change":"2024-05-08T22:44:26.845048+0000","last_active":"2024-05-08T22:44:28.223742+0000","last_peered":"2024-05-08T22:44:28.223742+0000","last_clean":"2024-05-08T22:44:28.223742+0000","last_became_active":"2024-05-08T22:44:26.844708+0000","last_became_peered":"2024-05-08T22:44:26.844708+0000","last_unstale":"2024-05-08T22:44:28.223742+0000","last_undegraded":"2024-05-08T22:44:28.223742+0000","last_fullsized":"2024-05-08T22:44:28.223742+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:57.320506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435038+0000","last_change":"2024-05-08T22:44:27.242497+0000","last_active":"2024-05-08T22:44:28.435038+0000","last_peered":"2024-05-08T22:44:28.435038+0000","last_clean":"2024-05-08T22:44:28.435038+0000","last_became_active":"2024-05-08T22:44:27.241951+0000","last_became_peered":"2024-05-08T22:44:27.241951+0000","last_unstale":"2024-05-08T22:44:28.435038+0000","last_undegraded":"2024-05-08T22:44:28.435038+0000","last_fullsized":"2024-05-08T22:44:28.435038+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:08.018968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434871+0000","last_change":"2024-05-08T22:44:26.849947+0000","last_active":"2024-05-08T22:44:28.434871+0000","last_peered":"2024-05-08T22:44:28.434871+0000","last_clean":"2024-05-08T22:44:28.434871+0000","last_became_active":"2024-05-08T22:44:26.849682+0000","last_became_peered":"2024-05-08T22:44:26.849682+0000","last_unstale":"2024-05-08T22:44:28.434871+0000","last_undegraded":"2024-05-08T22:44:28.434871+0000","last_fullsized":"2024-05-08T22:44:28.434871+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:27.421388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435022+0000","last_change":"2024-05-08T22:44:27.238406+0000","last_active":"2024-05-08T22:44:28.435022+0000","last_peered":"2024-05-08T22:44:28.435022+0000","last_clean":"2024-05-08T22:44:28.435022+0000","last_became_active":"2024-05-08T22:44:27.234475+0000","last_became_peered":"2024-05-08T22:44:27.234475+0000","last_unstale":"2024-05-08T22:44:28.435022+0000","last_undegraded":"2024-05-08T22:44:28.435022+0000","last_fullsized":"2024-05-08T22:44:28.435022+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:27.616461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224759+0000","last_change":"2024-05-08T22:44:26.855583+0000","last_active":"2024-05-08T22:44:28.224759+0000","last_peered":"2024-05-08T22:44:28.224759+0000","last_clean":"2024-05-08T22:44:28.224759+0000","last_became_active":"2024-05-08T22:44:26.855293+0000","last_became_peered":"2024-05-08T22:44:26.855293+0000","last_unstale":"2024-05-08T22:44:28.224759+0000","last_undegraded":"2024-05-08T22:44:28.224759+0000","last_fullsized":"2024-05-08T22:44:28.224759+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:48.927233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226392+0000","last_change":"2024-05-08T22:44:27.247151+0000","last_active":"2024-05-08T22:44:28.226392+0000","last_peered":"2024-05-08T22:44:28.226392+0000","last_clean":"2024-05-08T22:44:28.226392+0000","last_became_active":"2024-05-08T22:44:27.246896+0000","last_became_peered":"2024-05-08T22:44:27.246896+0000","last_unstale":"2024-05-08T22:44:28.226392+0000","last_undegraded":"2024-05-08T22:44:28.226392+0000","last_fullsized":"2024-05-08T22:44:28.226392+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225265+0000","last_change":"2024-05-08T22:44:26.854419+0000","last_active":"2024-05-08T22:44:28.225265+0000","last_peered":"2024-05-08T22:44:28.225265+0000","last_clean":"2024-05-08T22:44:28.225265+0000","last_became_active":"2024-05-08T22:44:26.854113+0000","last_became_peered":"2024-05-08T22:44:26.854113+0000","last_unstale":"2024-05-08T22:44:28.225265+0000","last_undegraded":"2024-05-08T22:44:28.225265+0000","last_fullsized":"2024-05-08T22:44:28.225265+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:07.352180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436208+0000","last_change":"2024-05-08T22:44:27.253303+0000","last_active":"2024-05-08T22:44:28.436208+0000","last_peered":"2024-05-08T22:44:28.436208+0000","last_clean":"2024-05-08T22:44:28.436208+0000","last_became_active":"2024-05-08T22:44:27.253070+0000","last_became_peered":"2024-05-08T22:44:27.253070+0000","last_unstale":"2024-05-08T22:44:28.436208+0000","last_undegraded":"2024-05-08T22:44:28.436208+0000","last_fullsized":"2024-05-08T22:44:28.436208+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:56.095882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434547+0000","last_change":"2024-05-08T22:44:26.859610+0000","last_active":"2024-05-08T22:44:28.434547+0000","last_peered":"2024-05-08T22:44:28.434547+0000","last_clean":"2024-05-08T22:44:28.434547+0000","last_became_active":"2024-05-08T22:44:26.859368+0000","last_became_peered":"2024-05-08T22:44:26.859368+0000","last_unstale":"2024-05-08T22:44:28.434547+0000","last_undegraded":"2024-05-08T22:44:28.434547+0000","last_fullsized":"2024-05-08T22:44:28.434547+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:46:43.956592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433545+0000","last_change":"2024-05-08T22:44:27.236447+0000","last_active":"2024-05-08T22:44:28.433545+0000","last_peered":"2024-05-08T22:44:28.433545+0000","last_clean":"2024-05-08T22:44:28.433545+0000","last_became_active":"2024-05-08T22:44:27.235584+0000","last_became_peered":"2024-05-08T22:44:27.235584+0000","last_unstale":"2024-05-08T22:44:28.433545+0000","last_undegraded":"2024-05-08T22:44:28.433545+0000","last_fullsized":"2024-05-08T22:44:28.433545+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:31.224173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.228122+0000","last_change":"2024-05-08T22:44:26.844366+0000","last_active":"2024-05-08T22:44:27.228122+0000","last_peered":"2024-05-08T22:44:27.228122+0000","last_clean":"2024-05-08T22:44:27.228122+0000","last_became_active":"2024-05-08T22:44:26.844087+0000","last_became_peered":"2024-05-08T22:44:26.844087+0000","last_unstale":"2024-05-08T22:44:27.228122+0000","last_undegraded":"2024-05-08T22:44:27.228122+0000","last_fullsized":"2024-05-08T22:44:27.228122+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:38.879657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433530+0000","last_change":"2024-05-08T22:44:27.246062+0000","last_active":"2024-05-08T22:44:28.433530+0000","last_peered":"2024-05-08T22:44:28.433530+0000","last_clean":"2024-05-08T22:44:28.433530+0000","last_became_active":"2024-05-08T22:44:27.245658+0000","last_became_peered":"2024-05-08T22:44:27.245658+0000","last_unstale":"2024-05-08T22:44:28.433530+0000","last_undegraded":"2024-05-08T22:44:28.433530+0000","last_fullsized":"2024-05-08T22:44:28.433530+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:38.181480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.230710+0000","last_change":"2024-05-08T22:44:26.852457+0000","last_active":"2024-05-08T22:44:27.230710+0000","last_peered":"2024-05-08T22:44:27.230710+0000","last_clean":"2024-05-08T22:44:27.230710+0000","last_became_active":"2024-05-08T22:44:26.852098+0000","last_became_peered":"2024-05-08T22:44:26.852098+0000","last_unstale":"2024-05-08T22:44:27.230710+0000","last_undegraded":"2024-05-08T22:44:27.230710+0000","last_fullsized":"2024-05-08T22:44:27.230710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:30.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225614+0000","last_change":"2024-05-08T22:44:27.248206+0000","last_active":"2024-05-08T22:44:28.225614+0000","last_peered":"2024-05-08T22:44:28.225614+0000","last_clean":"2024-05-08T22:44:28.225614+0000","last_became_active":"2024-05-08T22:44:27.247905+0000","last_became_peered":"2024-05-08T22:44:27.247905+0000","last_unstale":"2024-05-08T22:44:28.225614+0000","last_undegraded":"2024-05-08T22:44:28.225614+0000","last_fullsized":"2024-05-08T22:44:28.225614+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:05.983635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434104+0000","last_change":"2024-05-08T22:44:26.853919+0000","last_active":"2024-05-08T22:44:28.434104+0000","last_peered":"2024-05-08T22:44:28.434104+0000","last_clean":"2024-05-08T22:44:28.434104+0000","last_became_active":"2024-05-08T22:44:26.853658+0000","last_became_peered":"2024-05-08T22:44:26.853658+0000","last_unstale":"2024-05-08T22:44:28.434104+0000","last_undegraded":"2024-05-08T22:44:28.434104+0000","last_fullsized":"2024-05-08T22:44:28.434104+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:55.816420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434838+0000","last_change":"2024-05-08T22:44:27.259673+0000","last_active":"2024-05-08T22:44:28.434838+0000","last_peered":"2024-05-08T22:44:28.434838+0000","last_clean":"2024-05-08T22:44:28.434838+0000","last_became_active":"2024-05-08T22:44:27.259418+0000","last_became_peered":"2024-05-08T22:44:27.259418+0000","last_unstale":"2024-05-08T22:44:28.434838+0000","last_undegraded":"2024-05-08T22:44:28.434838+0000","last_fullsized":"2024-05-08T22:44:28.434838+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:48.327322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.223970+0000","last_change":"2024-05-08T22:44:26.845695+0000","last_active":"2024-05-08T22:44:28.223970+0000","last_peered":"2024-05-08T22:44:28.223970+0000","last_clean":"2024-05-08T22:44:28.223970+0000","last_became_active":"2024-05-08T22:44:26.845330+0000","last_became_peered":"2024-05-08T22:44:26.845330+0000","last_unstale":"2024-05-08T22:44:28.223970+0000","last_undegraded":"2024-05-08T22:44:28.223970+0000","last_fullsized":"2024-05-08T22:44:28.223970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:44.136301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226231+0000","last_change":"2024-05-08T22:44:27.256439+0000","last_active":"2024-05-08T22:44:28.226231+0000","last_peered":"2024-05-08T22:44:28.226231+0000","last_clean":"2024-05-08T22:44:28.226231+0000","last_became_active":"2024-05-08T22:44:27.252590+0000","last_became_peered":"2024-05-08T22:44:27.252590+0000","last_unstale":"2024-05-08T22:44:28.226231+0000","last_undegraded":"2024-05-08T22:44:28.226231+0000","last_fullsized":"2024-05-08T22:44:28.226231+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:53.154095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"126'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433522+0000","last_change":"2024-05-08T22:44:26.851306+0000","last_active":"2024-05-08T22:44:28.433522+0000","last_peered":"2024-05-08T22:44:28.433522+0000","last_clean":"2024-05-08T22:44:28.433522+0000","last_became_active":"2024-05-08T22:44:26.851071+0000","last_became_peered":"2024-05-08T22:44:26.851071+0000","last_unstale":"2024-05-08T22:44:28.433522+0000","last_undegraded":"2024-05-08T22:44:28.433522+0000","last_fullsized":"2024-05-08T22:44:28.433522+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.354185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":263,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.224179+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:44:27.224179+0000","last_peered":"2024-05-08T22:44:27.224179+0000","last_clean":"2024-05-08T22:44:27.224179+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:44:27.224179+0000","last_undegraded":"2024-05-08T22:44:27.224179+0000","last_fullsized":"2024-05-08T22:44:27.224179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":730378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.63},"min":{"1min":0.377,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90600000000000003,"5min":1.623,"15min":1.623},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.375,"15min":0.307},"max":{"1min":1.028,"5min":1.115,"15min":1.6899999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.496,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.1639999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.75},"min":{"1min":0.58999999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.997,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.42199999999999999},"max":{"1min":1.052,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.79300000000000004},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0780000000000001,"5min":1.716,"15min":1.716},"last":1.012}]},{"osd":4,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.89900000000000002,"15min":0.90500000000000003},"min":{"1min":0.623,"5min":0.48799999999999999,"15min":0.47599999999999998},"max":{"1min":1.222,"5min":1.2929999999999999,"15min":1.306},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95499999999999996,"15min":0.95099999999999996},"min":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1399999999999999,"5min":1.345,"15min":1.4139999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.99099999999999999,"15min":0.97999999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.111,"5min":1.429,"15min":1.429},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.81599999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.472,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.085}]},{"osd":6,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99199999999999999,"15min":0.98299999999999998},"min":{"1min":0.71999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.331,"5min":1.421,"15min":1.446},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.78100000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.214,"5min":1.512,"15min":1.512},"last":1.272}]},{"osd":7,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.091,"15min":1.095},"min":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.409,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3149999999999999,"5min":1.476,"15min":1.8360000000000001},"last":1.244}]}]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1576960,"data_stored":728802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.772,"15min":1.772},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.95699999999999996,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.91600000000000004,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.042,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.88}]},{"osd":3,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.79600000000000004},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.131,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.583},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.91700000000000004,"15min":0.94899999999999995},"min":{"1min":0.64400000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.228,"5min":1.395,"15min":1.431},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0129999999999999,"15min":0.98599999999999999},"min":{"1min":0.747,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.224,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2210000000000001,"5min":1.389,"15min":1.5},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.98399999999999999,"15min":1.042},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3640000000000001,"5min":1.399,"15min":1.726},"last":1.2170000000000001}]},{"osd":6,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.98499999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.181,"5min":1.351,"15min":1.4650000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.82199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":1.0720000000000001}]},{"osd":7,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0780000000000001,"15min":1.071},"min":{"1min":0.81200000000000006,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3580000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.274},{"interface":"front","average":{"1min":1.034,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.4119999999999999,"5min":1.446,"15min":1.5049999999999999},"last":1.123}]}]},{"osd":7,"up_from":14,"seq":60129542282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2183168,"data_stored":1319924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.53700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.88300000000000001,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.514},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.88900000000000001},"min":{"1min":0.61799999999999999,"5min":0.47899999999999998,"15min":0.46999999999999997},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92600000000000005,"15min":0.89100000000000001},"min":{"1min":0.629,"5min":0.58199999999999996,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.415,"15min":1.488},"last":0.63}]},{"osd":2,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92700000000000005,"15min":0.90800000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58199999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94499999999999995,"15min":0.95499999999999996},"min":{"1min":0.79900000000000004,"5min":0.61499999999999999,"15min":0.53800000000000003},"max":{"1min":1.1459999999999999,"5min":1.256,"15min":1.337},"last":1.1679999999999999}]},{"osd":3,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.53900000000000003},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.042},{"interface":"front","average":{"1min":0.878,"5min":0.877,"15min":0.86099999999999999},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.571,"5min":1.571,"15min":1.611},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89300000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.94399999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.53300000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.304},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0860000000000001,"15min":1.0580000000000001},"min":{"1min":0.82199999999999995,"5min":0.80600000000000005,"15min":0.60199999999999998},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.373}]},{"osd":6,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.96399999999999997,"15min":0.95199999999999996},"min":{"1min":0.69399999999999995,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.81799999999999995,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.599},"last":1.4319999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1564672,"data_stored":722742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.64400000000000002,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.2430000000000001,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.758},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.1140000000000001,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":1.0049999999999999}]},{"osd":2,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.93799999999999994},"min":{"1min":0.498,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.1479999999999999,"5min":1.214,"15min":1.4319999999999999},"last":0.745},{"interface":"front","average":{"1min":0.87,"5min":0.89500000000000002,"15min":0.90800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.98599999999999999}]},{"osd":3,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94599999999999995,"15min":1.024},"min":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.204,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.5029999999999999},"last":1.095}]},{"osd":4,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92800000000000005,"15min":0.96399999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.538},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94999999999999996,"15min":0.995},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.44400000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.03}]},{"osd":5,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0249999999999999,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.64200000000000002,"15min":0.64000000000000001},"max":{"1min":1.278,"5min":1.427,"15min":1.681},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.044,"15min":1.0900000000000001},"min":{"1min":0.79700000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.6000000000000001},"last":1.161}]},{"osd":7,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95599999999999996,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.835},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.1799999999999999},"min":{"1min":0.81499999999999995,"5min":0.748,"15min":0.74099999999999999},"max":{"1min":1.405,"5min":1.5600000000000001,"15min":2.2999999999999998},"last":1.1319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542282,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1568768,"data_stored":722320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90200000000000002,"15min":0.84199999999999997},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.149,"5min":7.907,"15min":7.907},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":0.82699999999999996},"min":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.1499999999999999,"5min":1.29,"15min":1.52},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90900000000000003,"15min":0.84899999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.1459999999999999,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":1.119}]},{"osd":2,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92300000000000004,"15min":0.89900000000000002},"min":{"1min":0.749,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2549999999999999,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.996,"15min":0.90400000000000003},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.48799999999999999},"max":{"1min":1.1379999999999999,"5min":10.121,"15min":10.121},"last":0.98799999999999999}]},{"osd":3,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88800000000000001,"15min":0.88100000000000001},"min":{"1min":0.72099999999999997,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.93700000000000006,"15min":0.96099999999999997},"min":{"1min":0.745,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.298},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.093,"15min":0.97899999999999998},"min":{"1min":0.67800000000000005,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.198,"5min":15.135,"15min":15.135},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.0609999999999999,"15min":1.0189999999999999},"min":{"1min":0.68799999999999994,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.3680000000000001,"5min":4.319,"15min":4.319},"last":0.70299999999999996}]},{"osd":6,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95499999999999996,"15min":0.94199999999999995},"min":{"1min":0.70099999999999996,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.347,"5min":1.3580000000000001,"15min":1.605},"last":1.026},{"interface":"front","average":{"1min":1.069,"5min":1.056,"15min":1.0529999999999999},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.54900000000000004},"max":{"1min":1.3160000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.111,"15min":1.097},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.57299999999999995},"max":{"1min":1.222,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.876},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0780000000000001,"15min":1.0569999999999999},"min":{"1min":0.92300000000000004,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.3220000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.073}]}]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1548288,"data_stored":715065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85499999999999998,"15min":0.83999999999999997},"min":{"1min":0.53900000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.294},"last":0.76400000000000001}]},{"osd":1,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.53300000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.872,"15min":0.88600000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53100000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.3140000000000001},"last":1.097}]},{"osd":2,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.65700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.91700000000000004,"15min":0.92100000000000004},"min":{"1min":0.67200000000000004,"5min":0.623,"15min":0.53400000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.637},"last":0.95199999999999996}]},{"osd":3,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":2.601},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92900000000000005,"15min":1.02},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.139,"5min":1.22,"15min":2.8769999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90600000000000003,"15min":0.97599999999999998},"min":{"1min":0.64300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.792},"last":1.008},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.98899999999999999},"min":{"1min":0.77000000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.169,"5min":1.2949999999999999,"15min":1.575},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96299999999999997,"15min":1.0349999999999999},"min":{"1min":0.73199999999999998,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.276,"5min":1.331,"15min":1.831},"last":1.165},{"interface":"front","average":{"1min":1.105,"5min":1.04,"15min":1.075},"min":{"1min":0.78900000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.349}]},{"osd":7,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.76500000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.319,"5min":1.403,"15min":1.8500000000000001},"last":1.323},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4139999999999999,"5min":1.46,"15min":1.944},"last":1.266}]}]},{"osd":2,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1560576,"data_stored":722160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":0.93899999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.2190000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Wed May 8 22:43:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.96699999999999997,"5min":0.97399999999999998,"15min":1.048},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.96699999999999997,"5min":1.085,"15min":1.3460000000000001},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed May 8 22:43:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.56100000000000005,"5min":0.50700000000000001,"15min":0.312},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.88600000000000001,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.89900000000000002,"15min":0.91700000000000004},"min":{"1min":0.72499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.113,"5min":1.361,"15min":1.504},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93899999999999995,"15min":0.98999999999999999},"min":{"1min":0.78200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.73799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.25,"5min":1.3129999999999999,"15min":1.484},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.75900000000000001,"5min":0.68000000000000005,"15min":0.67200000000000004},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.456},"last":1.1180000000000001}]},{"osd":6,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0089999999999999,"15min":1.0389999999999999},"min":{"1min":0.88100000000000001,"5min":0.71399999999999997,"15min":0.69599999999999995},"max":{"1min":1.391,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0069999999999999,"15min":1.0580000000000001},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.244,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.0389999999999999}]},{"osd":7,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0489999999999999,"15min":1.069},"min":{"1min":0.81699999999999995,"5min":0.69999999999999996,"15min":0.39300000000000002},"max":{"1min":1.3169999999999999,"5min":1.498,"15min":1.5409999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0629999999999999,"15min":1.0980000000000001},"min":{"1min":0.84699999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.357,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542282,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1312550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:43:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.617},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.33600000000000002},"max":{"1min":0.85899999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.446},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.755},"min":{"1min":0.52200000000000002,"5min":0.45100000000000001,"15min":0.436},"max":{"1min":1.044,"5min":1.101,"15min":1.6910000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 22:43:33 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95399999999999996,"5min":1.1020000000000001,"15min":1.714},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":0.438}]},{"osd":4,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.876,"15min":0.90100000000000002},"min":{"1min":0.66400000000000003,"5min":0.63700000000000001,"15min":0.61699999999999999},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":4.391},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.93200000000000005,"15min":0.96699999999999997},"min":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":2.0590000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91200000000000003,"15min":0.90900000000000003},"min":{"1min":0.746,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1599999999999999,"5min":1.246,"15min":1.419},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95999999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.72899999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1120000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0109999999999999,"15min":1.04},"min":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":0.93500000000000005}]},{"osd":7,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0649999999999999},"min":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.6459999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0169999999999999,"15min":1.02},"min":{"1min":0.85899999999999999,"5min":0.72499999999999998,"15min":0.60399999999999998},"max":{"1min":1.3200000000000001,"5min":1.514,"15min":1.514},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:44:31.333 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:44:31.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 Processor -- start 2024-05-08T22:44:31.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 -- start start 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 0x7f69840c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f698406af90 0x7f698406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840c9bd0 con 0x7f698415d130 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840ce650 con 0x7f698406af90 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f698b6ed700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cd340 con 0x7f69840c9230 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f698406af90 0x7f698406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f698406af90 0x7f698406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52004/0 (socket says 172.21.15.179:52004) 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 -- 172.21.15.179:0/3206860136 learned_addr learned my addr 172.21.15.179:0/3206860136 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6988c88700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 0x7f69840c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 -- 172.21.15.179:0/3206860136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 msgr2=0x7f69840c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 0x7f69840c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 -- 172.21.15.179:0/3206860136 >> v1:172.21.15.195:6789/0 conn(0x7f698415d130 legacy=0x7f69840cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 -- 172.21.15.179:0/3206860136 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840cd620 con 0x7f698406af90 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6988c88700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 0x7f69840c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f6989c8a700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698406af90 0x7f698406b420 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f697c009910 tx=0x7f697c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55170c5463fa2bce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f697bfff700 1 -- 172.21.15.179:0/3206860136 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c011830 con 0x7f698406af90 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f697bfff700 1 -- 172.21.15.179:0/3206860136 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c011990 con 0x7f698406af90 2024-05-08T22:44:31.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.451+0000 7f697bfff700 1 -- 172.21.15.179:0/3206860136 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c011d40 con 0x7f698406af90 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 -- 172.21.15.179:0/3206860136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698406af90 msgr2=0x7f698406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698406af90 0x7f698406b420 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f697c009910 tx=0x7f697c009c20 comp rx=0 tx=0).stop 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 -- 172.21.15.179:0/3206860136 shutdown_connections 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69840c9230 0x7f69840c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 --2- 172.21.15.179:0/3206860136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698406af90 0x7f698406b420 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 -- 172.21.15.179:0/3206860136 >> 172.21.15.179:0/3206860136 conn(0x7f69840cb660 msgr2=0x7f69840cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 -- 172.21.15.179:0/3206860136 shutdown_connections 2024-05-08T22:44:31.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.455+0000 7f698b6ed700 1 -- 172.21.15.179:0/3206860136 wait complete. 2024-05-08T22:44:31.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 Processor -- start 2024-05-08T22:44:31.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 -- start start 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f698406af90 0x7f6984159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69840c9230 0x7f6984158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6984158b90 con 0x7f698415d130 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b2a30 con 0x7f69840c9230 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b2d10 con 0x7f698406af90 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52016/0 (socket says 172.21.15.179:52016) 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 -- 172.21.15.179:0/1638650598 learned_addr learned my addr 172.21.15.179:0/1638650598 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6988c88700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69840c9230 0x7f6984158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989489700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f698406af90 0x7f6984159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 -- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f698406af90 msgr2=0x7f6984159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f698406af90 0x7f6984159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 -- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69840c9230 msgr2=0x7f6984158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69840c9230 0x7f6984158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 -- 172.21.15.179:0/1638650598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009a70 con 0x7f698415d130 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6989c8a700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6984067e10 tx=0x7f697c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c004380 con 0x7f698415d130 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c01f070 con 0x7f698415d130 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c009570 con 0x7f698415d130 2024-05-08T22:44:31.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.459+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0036c0 con 0x7f698415d130 2024-05-08T22:44:31.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840b32d0 con 0x7f698415d130 2024-05-08T22:44:31.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6984067770 con 0x7f698415d130 2024-05-08T22:44:31.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f697c021030 con 0x7f698415d130 2024-05-08T22:44:31.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f6979ffb700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 0x7f6968068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f697c085480 con 0x7f698415d130 2024-05-08T22:44:31.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.463+0000 7f6989489700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 0x7f6968068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.467+0000 7f6989489700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 0x7f6968068bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6974000f60 tx=0x7f6974009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.471+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f697c052740 con 0x7f698415d130 2024-05-08T22:44:31.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.643+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f698415d510 con 0x7f6968066730 2024-05-08T22:44:31.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.659+0000 7f6979ffb700 1 -- 172.21.15.179:0/1638650598 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288297 (secure 0 0 0) 0x7f6974001540 con 0x7f6968066730 2024-05-08T22:44:31.666 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 msgr2=0x7f6968068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 0x7f6968068bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6974000f60 tx=0x7f6974009040 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 msgr2=0x7f698415fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6984067e10 tx=0x7f697c004770 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 shutdown_connections 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6968066730 0x7f6968068bf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69840c9230 0x7f6984158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f698415d130 0x7f698415fe80 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 --2- 172.21.15.179:0/1638650598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f698406af90 0x7f6984159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 >> 172.21.15.179:0/1638650598 conn(0x7f69840cb660 msgr2=0x7f69840ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 shutdown_connections 2024-05-08T22:44:31.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.667+0000 7f698b6ed700 1 -- 172.21.15.179:0/1638650598 wait complete. 2024-05-08T22:44:31.670 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:44:31.684 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-05-08T22:44:31.622348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270564,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":257011,"kb_avail":754704156,"statfs":{"total":773094113280,"available":772817055744,"internally_reserved":0,"allocated":13750272,"data_stored":6973941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":263179832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.796671"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.244627+0000","last_change":"2024-05-08T22:44:27.243185+0000","last_active":"2024-05-08T22:44:27.244627+0000","last_peered":"2024-05-08T22:44:27.244627+0000","last_clean":"2024-05-08T22:44:27.244627+0000","last_became_active":"2024-05-08T22:44:27.242934+0000","last_became_peered":"2024-05-08T22:44:27.242934+0000","last_unstale":"2024-05-08T22:44:27.244627+0000","last_undegraded":"2024-05-08T22:44:27.244627+0000","last_fullsized":"2024-05-08T22:44:27.244627+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:18.331315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434606+0000","last_change":"2024-05-08T22:44:27.254798+0000","last_active":"2024-05-08T22:44:28.434606+0000","last_peered":"2024-05-08T22:44:28.434606+0000","last_clean":"2024-05-08T22:44:28.434606+0000","last_became_active":"2024-05-08T22:44:27.254552+0000","last_became_peered":"2024-05-08T22:44:27.254552+0000","last_unstale":"2024-05-08T22:44:28.434606+0000","last_undegraded":"2024-05-08T22:44:28.434606+0000","last_fullsized":"2024-05-08T22:44:28.434606+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:12.761662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434955+0000","last_change":"2024-05-08T22:44:27.251932+0000","last_active":"2024-05-08T22:44:28.434955+0000","last_peered":"2024-05-08T22:44:28.434955+0000","last_clean":"2024-05-08T22:44:28.434955+0000","last_became_active":"2024-05-08T22:44:27.251723+0000","last_became_peered":"2024-05-08T22:44:27.251723+0000","last_unstale":"2024-05-08T22:44:28.434955+0000","last_undegraded":"2024-05-08T22:44:28.434955+0000","last_fullsized":"2024-05-08T22:44:28.434955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:07.371835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225855+0000","last_change":"2024-05-08T22:44:27.231377+0000","last_active":"2024-05-08T22:44:28.225855+0000","last_peered":"2024-05-08T22:44:28.225855+0000","last_clean":"2024-05-08T22:44:28.225855+0000","last_became_active":"2024-05-08T22:44:27.231118+0000","last_became_peered":"2024-05-08T22:44:27.231118+0000","last_unstale":"2024-05-08T22:44:28.225855+0000","last_undegraded":"2024-05-08T22:44:28.225855+0000","last_fullsized":"2024-05-08T22:44:28.225855+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:39.074254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.244653+0000","last_change":"2024-05-08T22:44:27.243346+0000","last_active":"2024-05-08T22:44:27.244653+0000","last_peered":"2024-05-08T22:44:27.244653+0000","last_clean":"2024-05-08T22:44:27.244653+0000","last_became_active":"2024-05-08T22:44:27.242455+0000","last_became_peered":"2024-05-08T22:44:27.242455+0000","last_unstale":"2024-05-08T22:44:27.244653+0000","last_undegraded":"2024-05-08T22:44:27.244653+0000","last_fullsized":"2024-05-08T22:44:27.244653+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:53.437189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435768+0000","last_change":"2024-05-08T22:44:27.235148+0000","last_active":"2024-05-08T22:44:28.435768+0000","last_peered":"2024-05-08T22:44:28.435768+0000","last_clean":"2024-05-08T22:44:28.435768+0000","last_became_active":"2024-05-08T22:44:27.234421+0000","last_became_peered":"2024-05-08T22:44:27.234421+0000","last_unstale":"2024-05-08T22:44:28.435768+0000","last_undegraded":"2024-05-08T22:44:28.435768+0000","last_fullsized":"2024-05-08T22:44:28.435768+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:19.244578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433989+0000","last_change":"2024-05-08T22:44:27.250795+0000","last_active":"2024-05-08T22:44:28.433989+0000","last_peered":"2024-05-08T22:44:28.433989+0000","last_clean":"2024-05-08T22:44:28.433989+0000","last_became_active":"2024-05-08T22:44:27.250533+0000","last_became_peered":"2024-05-08T22:44:27.250533+0000","last_unstale":"2024-05-08T22:44:28.433989+0000","last_undegraded":"2024-05-08T22:44:28.433989+0000","last_fullsized":"2024-05-08T22:44:28.433989+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:14.232235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433091+0000","last_change":"2024-05-08T22:44:27.251831+0000","last_active":"2024-05-08T22:44:28.433091+0000","last_peered":"2024-05-08T22:44:28.433091+0000","last_clean":"2024-05-08T22:44:28.433091+0000","last_became_active":"2024-05-08T22:44:27.251632+0000","last_became_peered":"2024-05-08T22:44:27.251632+0000","last_unstale":"2024-05-08T22:44:28.433091+0000","last_undegraded":"2024-05-08T22:44:28.433091+0000","last_fullsized":"2024-05-08T22:44:28.433091+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:26.921335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226887+0000","last_change":"2024-05-08T22:44:27.252994+0000","last_active":"2024-05-08T22:44:28.226887+0000","last_peered":"2024-05-08T22:44:28.226887+0000","last_clean":"2024-05-08T22:44:28.226887+0000","last_became_active":"2024-05-08T22:44:27.252241+0000","last_became_peered":"2024-05-08T22:44:27.252241+0000","last_unstale":"2024-05-08T22:44:28.226887+0000","last_undegraded":"2024-05-08T22:44:28.226887+0000","last_fullsized":"2024-05-08T22:44:28.226887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:43.444745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435910+0000","last_change":"2024-05-08T22:44:27.238660+0000","last_active":"2024-05-08T22:44:28.435910+0000","last_peered":"2024-05-08T22:44:28.435910+0000","last_clean":"2024-05-08T22:44:28.435910+0000","last_became_active":"2024-05-08T22:44:27.234854+0000","last_became_peered":"2024-05-08T22:44:27.234854+0000","last_unstale":"2024-05-08T22:44:28.435910+0000","last_undegraded":"2024-05-08T22:44:28.435910+0000","last_fullsized":"2024-05-08T22:44:28.435910+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:26.651963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.245658+0000","last_change":"2024-05-08T22:44:27.244982+0000","last_active":"2024-05-08T22:44:27.245658+0000","last_peered":"2024-05-08T22:44:27.245658+0000","last_clean":"2024-05-08T22:44:27.245658+0000","last_became_active":"2024-05-08T22:44:27.244719+0000","last_became_peered":"2024-05-08T22:44:27.244719+0000","last_unstale":"2024-05-08T22:44:27.245658+0000","last_undegraded":"2024-05-08T22:44:27.245658+0000","last_fullsized":"2024-05-08T22:44:27.245658+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434125+0000","last_change":"2024-05-08T22:44:27.236479+0000","last_active":"2024-05-08T22:44:28.434125+0000","last_peered":"2024-05-08T22:44:28.434125+0000","last_clean":"2024-05-08T22:44:28.434125+0000","last_became_active":"2024-05-08T22:44:27.235046+0000","last_became_peered":"2024-05-08T22:44:27.235046+0000","last_unstale":"2024-05-08T22:44:28.434125+0000","last_undegraded":"2024-05-08T22:44:28.434125+0000","last_fullsized":"2024-05-08T22:44:28.434125+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:40:49.488792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434394+0000","last_change":"2024-05-08T22:44:27.247229+0000","last_active":"2024-05-08T22:44:28.434394+0000","last_peered":"2024-05-08T22:44:28.434394+0000","last_clean":"2024-05-08T22:44:28.434394+0000","last_became_active":"2024-05-08T22:44:27.246948+0000","last_became_peered":"2024-05-08T22:44:27.246948+0000","last_unstale":"2024-05-08T22:44:28.434394+0000","last_undegraded":"2024-05-08T22:44:28.434394+0000","last_fullsized":"2024-05-08T22:44:28.434394+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:37.623210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434017+0000","last_change":"2024-05-08T22:44:27.250037+0000","last_active":"2024-05-08T22:44:28.434017+0000","last_peered":"2024-05-08T22:44:28.434017+0000","last_clean":"2024-05-08T22:44:28.434017+0000","last_became_active":"2024-05-08T22:44:27.249810+0000","last_became_peered":"2024-05-08T22:44:27.249810+0000","last_unstale":"2024-05-08T22:44:28.434017+0000","last_undegraded":"2024-05-08T22:44:28.434017+0000","last_fullsized":"2024-05-08T22:44:28.434017+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:50.087190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.241128+0000","last_change":"2024-05-08T22:44:27.240295+0000","last_active":"2024-05-08T22:44:27.241128+0000","last_peered":"2024-05-08T22:44:27.241128+0000","last_clean":"2024-05-08T22:44:27.241128+0000","last_became_active":"2024-05-08T22:44:27.239992+0000","last_became_peered":"2024-05-08T22:44:27.239992+0000","last_unstale":"2024-05-08T22:44:27.241128+0000","last_undegraded":"2024-05-08T22:44:27.241128+0000","last_fullsized":"2024-05-08T22:44:27.241128+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:17.911331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433080+0000","last_change":"2024-05-08T22:44:27.233255+0000","last_active":"2024-05-08T22:44:28.433080+0000","last_peered":"2024-05-08T22:44:28.433080+0000","last_clean":"2024-05-08T22:44:28.433080+0000","last_became_active":"2024-05-08T22:44:27.232129+0000","last_became_peered":"2024-05-08T22:44:27.232129+0000","last_unstale":"2024-05-08T22:44:28.433080+0000","last_undegraded":"2024-05-08T22:44:28.433080+0000","last_fullsized":"2024-05-08T22:44:28.433080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:43.472298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435181+0000","last_change":"2024-05-08T22:44:27.250127+0000","last_active":"2024-05-08T22:44:28.435181+0000","last_peered":"2024-05-08T22:44:28.435181+0000","last_clean":"2024-05-08T22:44:28.435181+0000","last_became_active":"2024-05-08T22:44:27.249936+0000","last_became_peered":"2024-05-08T22:44:27.249936+0000","last_unstale":"2024-05-08T22:44:28.435181+0000","last_undegraded":"2024-05-08T22:44:28.435181+0000","last_fullsized":"2024-05-08T22:44:28.435181+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:08.412148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434768+0000","last_change":"2024-05-08T22:44:27.254771+0000","last_active":"2024-05-08T22:44:28.434768+0000","last_peered":"2024-05-08T22:44:28.434768+0000","last_clean":"2024-05-08T22:44:28.434768+0000","last_became_active":"2024-05-08T22:44:27.254531+0000","last_became_peered":"2024-05-08T22:44:27.254531+0000","last_unstale":"2024-05-08T22:44:28.434768+0000","last_undegraded":"2024-05-08T22:44:28.434768+0000","last_fullsized":"2024-05-08T22:44:28.434768+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:07.783492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225162+0000","last_change":"2024-05-08T22:44:27.247540+0000","last_active":"2024-05-08T22:44:28.225162+0000","last_peered":"2024-05-08T22:44:28.225162+0000","last_clean":"2024-05-08T22:44:28.225162+0000","last_became_active":"2024-05-08T22:44:27.246565+0000","last_became_peered":"2024-05-08T22:44:27.246565+0000","last_unstale":"2024-05-08T22:44:28.225162+0000","last_undegraded":"2024-05-08T22:44:28.225162+0000","last_fullsized":"2024-05-08T22:44:28.225162+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.204876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435870+0000","last_change":"2024-05-08T22:44:27.244141+0000","last_active":"2024-05-08T22:44:28.435870+0000","last_peered":"2024-05-08T22:44:28.435870+0000","last_clean":"2024-05-08T22:44:28.435870+0000","last_became_active":"2024-05-08T22:44:27.243898+0000","last_became_peered":"2024-05-08T22:44:27.243898+0000","last_unstale":"2024-05-08T22:44:28.435870+0000","last_undegraded":"2024-05-08T22:44:28.435870+0000","last_fullsized":"2024-05-08T22:44:28.435870+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:18:11.225537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226140+0000","last_change":"2024-05-08T22:44:27.256514+0000","last_active":"2024-05-08T22:44:28.226140+0000","last_peered":"2024-05-08T22:44:28.226140+0000","last_clean":"2024-05-08T22:44:28.226140+0000","last_became_active":"2024-05-08T22:44:27.253033+0000","last_became_peered":"2024-05-08T22:44:27.253033+0000","last_unstale":"2024-05-08T22:44:28.226140+0000","last_undegraded":"2024-05-08T22:44:28.226140+0000","last_fullsized":"2024-05-08T22:44:28.226140+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:57.586085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.237785+0000","last_change":"2024-05-08T22:44:27.236167+0000","last_active":"2024-05-08T22:44:27.237785+0000","last_peered":"2024-05-08T22:44:27.237785+0000","last_clean":"2024-05-08T22:44:27.237785+0000","last_became_active":"2024-05-08T22:44:27.235784+0000","last_became_peered":"2024-05-08T22:44:27.235784+0000","last_unstale":"2024-05-08T22:44:27.237785+0000","last_undegraded":"2024-05-08T22:44:27.237785+0000","last_fullsized":"2024-05-08T22:44:27.237785+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:30.446672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225658+0000","last_change":"2024-05-08T22:44:27.239063+0000","last_active":"2024-05-08T22:44:28.225658+0000","last_peered":"2024-05-08T22:44:28.225658+0000","last_clean":"2024-05-08T22:44:28.225658+0000","last_became_active":"2024-05-08T22:44:27.238775+0000","last_became_peered":"2024-05-08T22:44:27.238775+0000","last_unstale":"2024-05-08T22:44:28.225658+0000","last_undegraded":"2024-05-08T22:44:28.225658+0000","last_fullsized":"2024-05-08T22:44:28.225658+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:20.426660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.430832+0000","last_change":"2024-05-08T22:44:27.250597+0000","last_active":"2024-05-08T22:44:28.430832+0000","last_peered":"2024-05-08T22:44:28.430832+0000","last_clean":"2024-05-08T22:44:28.430832+0000","last_became_active":"2024-05-08T22:44:27.250153+0000","last_became_peered":"2024-05-08T22:44:27.250153+0000","last_unstale":"2024-05-08T22:44:28.430832+0000","last_undegraded":"2024-05-08T22:44:28.430832+0000","last_fullsized":"2024-05-08T22:44:28.430832+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:28.921494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432011+0000","last_change":"2024-05-08T22:44:27.241822+0000","last_active":"2024-05-08T22:44:28.432011+0000","last_peered":"2024-05-08T22:44:28.432011+0000","last_clean":"2024-05-08T22:44:28.432011+0000","last_became_active":"2024-05-08T22:44:27.241541+0000","last_became_peered":"2024-05-08T22:44:27.241541+0000","last_unstale":"2024-05-08T22:44:28.432011+0000","last_undegraded":"2024-05-08T22:44:28.432011+0000","last_fullsized":"2024-05-08T22:44:28.432011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:47.989051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436083+0000","last_change":"2024-05-08T22:44:27.235866+0000","last_active":"2024-05-08T22:44:28.436083+0000","last_peered":"2024-05-08T22:44:28.436083+0000","last_clean":"2024-05-08T22:44:28.436083+0000","last_became_active":"2024-05-08T22:44:27.235496+0000","last_became_peered":"2024-05-08T22:44:27.235496+0000","last_unstale":"2024-05-08T22:44:28.436083+0000","last_undegraded":"2024-05-08T22:44:28.436083+0000","last_fullsized":"2024-05-08T22:44:28.436083+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:03.613627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225316+0000","last_change":"2024-05-08T22:44:27.253430+0000","last_active":"2024-05-08T22:44:28.225316+0000","last_peered":"2024-05-08T22:44:28.225316+0000","last_clean":"2024-05-08T22:44:28.225316+0000","last_became_active":"2024-05-08T22:44:27.253221+0000","last_became_peered":"2024-05-08T22:44:27.253221+0000","last_unstale":"2024-05-08T22:44:28.225316+0000","last_undegraded":"2024-05-08T22:44:28.225316+0000","last_fullsized":"2024-05-08T22:44:28.225316+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:27.616461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434347+0000","last_change":"2024-05-08T22:44:27.242683+0000","last_active":"2024-05-08T22:44:28.434347+0000","last_peered":"2024-05-08T22:44:28.434347+0000","last_clean":"2024-05-08T22:44:28.434347+0000","last_became_active":"2024-05-08T22:44:27.242256+0000","last_became_peered":"2024-05-08T22:44:27.242256+0000","last_unstale":"2024-05-08T22:44:28.434347+0000","last_undegraded":"2024-05-08T22:44:28.434347+0000","last_fullsized":"2024-05-08T22:44:28.434347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:36.298015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436450+0000","last_change":"2024-05-08T22:44:27.237727+0000","last_active":"2024-05-08T22:44:28.436450+0000","last_peered":"2024-05-08T22:44:28.436450+0000","last_clean":"2024-05-08T22:44:28.436450+0000","last_became_active":"2024-05-08T22:44:27.237397+0000","last_became_peered":"2024-05-08T22:44:27.237397+0000","last_unstale":"2024-05-08T22:44:28.436450+0000","last_undegraded":"2024-05-08T22:44:28.436450+0000","last_fullsized":"2024-05-08T22:44:28.436450+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:19.170375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436019+0000","last_change":"2024-05-08T22:44:27.237197+0000","last_active":"2024-05-08T22:44:28.436019+0000","last_peered":"2024-05-08T22:44:28.436019+0000","last_clean":"2024-05-08T22:44:28.436019+0000","last_became_active":"2024-05-08T22:44:27.236865+0000","last_became_peered":"2024-05-08T22:44:27.236865+0000","last_unstale":"2024-05-08T22:44:28.436019+0000","last_undegraded":"2024-05-08T22:44:28.436019+0000","last_fullsized":"2024-05-08T22:44:28.436019+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:23.582534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431210+0000","last_change":"2024-05-08T22:44:27.241198+0000","last_active":"2024-05-08T22:44:28.431210+0000","last_peered":"2024-05-08T22:44:28.431210+0000","last_clean":"2024-05-08T22:44:28.431210+0000","last_became_active":"2024-05-08T22:44:27.237468+0000","last_became_peered":"2024-05-08T22:44:27.237468+0000","last_unstale":"2024-05-08T22:44:28.431210+0000","last_undegraded":"2024-05-08T22:44:28.431210+0000","last_fullsized":"2024-05-08T22:44:28.431210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:30.571208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.430876+0000","last_change":"2024-05-08T22:44:27.250322+0000","last_active":"2024-05-08T22:44:28.430876+0000","last_peered":"2024-05-08T22:44:28.430876+0000","last_clean":"2024-05-08T22:44:28.430876+0000","last_became_active":"2024-05-08T22:44:27.247868+0000","last_became_peered":"2024-05-08T22:44:27.247868+0000","last_unstale":"2024-05-08T22:44:28.430876+0000","last_undegraded":"2024-05-08T22:44:28.430876+0000","last_fullsized":"2024-05-08T22:44:28.430876+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435403+0000","last_change":"2024-05-08T22:44:27.237697+0000","last_active":"2024-05-08T22:44:28.435403+0000","last_peered":"2024-05-08T22:44:28.435403+0000","last_clean":"2024-05-08T22:44:28.435403+0000","last_became_active":"2024-05-08T22:44:27.234111+0000","last_became_peered":"2024-05-08T22:44:27.234111+0000","last_unstale":"2024-05-08T22:44:28.435403+0000","last_undegraded":"2024-05-08T22:44:28.435403+0000","last_fullsized":"2024-05-08T22:44:28.435403+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:43.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"126'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224893+0000","last_change":"2024-05-08T22:44:26.849125+0000","last_active":"2024-05-08T22:44:28.224893+0000","last_peered":"2024-05-08T22:44:28.224893+0000","last_clean":"2024-05-08T22:44:28.224893+0000","last_became_active":"2024-05-08T22:44:26.848838+0000","last_became_peered":"2024-05-08T22:44:26.848838+0000","last_unstale":"2024-05-08T22:44:28.224893+0000","last_undegraded":"2024-05-08T22:44:28.224893+0000","last_fullsized":"2024-05-08T22:44:28.224893+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:15.506911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435568+0000","last_change":"2024-05-08T22:44:27.237005+0000","last_active":"2024-05-08T22:44:28.435568+0000","last_peered":"2024-05-08T22:44:28.435568+0000","last_clean":"2024-05-08T22:44:28.435568+0000","last_became_active":"2024-05-08T22:44:27.236763+0000","last_became_peered":"2024-05-08T22:44:27.236763+0000","last_unstale":"2024-05-08T22:44:28.435568+0000","last_undegraded":"2024-05-08T22:44:28.435568+0000","last_fullsized":"2024-05-08T22:44:28.435568+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:53:21.155072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436062+0000","last_change":"2024-05-08T22:44:26.856476+0000","last_active":"2024-05-08T22:44:28.436062+0000","last_peered":"2024-05-08T22:44:28.436062+0000","last_clean":"2024-05-08T22:44:28.436062+0000","last_became_active":"2024-05-08T22:44:26.856204+0000","last_became_peered":"2024-05-08T22:44:26.856204+0000","last_unstale":"2024-05-08T22:44:28.436062+0000","last_undegraded":"2024-05-08T22:44:28.436062+0000","last_fullsized":"2024-05-08T22:44:28.436062+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:59.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431809+0000","last_change":"2024-05-08T22:44:27.246729+0000","last_active":"2024-05-08T22:44:28.431809+0000","last_peered":"2024-05-08T22:44:28.431809+0000","last_clean":"2024-05-08T22:44:28.431809+0000","last_became_active":"2024-05-08T22:44:27.246361+0000","last_became_peered":"2024-05-08T22:44:27.246361+0000","last_unstale":"2024-05-08T22:44:28.431809+0000","last_undegraded":"2024-05-08T22:44:28.431809+0000","last_fullsized":"2024-05-08T22:44:28.431809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:45.337960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226532+0000","last_change":"2024-05-08T22:44:26.854243+0000","last_active":"2024-05-08T22:44:28.226532+0000","last_peered":"2024-05-08T22:44:28.226532+0000","last_clean":"2024-05-08T22:44:28.226532+0000","last_became_active":"2024-05-08T22:44:26.853945+0000","last_became_peered":"2024-05-08T22:44:26.853945+0000","last_unstale":"2024-05-08T22:44:28.226532+0000","last_undegraded":"2024-05-08T22:44:28.226532+0000","last_fullsized":"2024-05-08T22:44:28.226532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:19.060876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226963+0000","last_change":"2024-05-08T22:44:27.252893+0000","last_active":"2024-05-08T22:44:28.226963+0000","last_peered":"2024-05-08T22:44:28.226963+0000","last_clean":"2024-05-08T22:44:28.226963+0000","last_became_active":"2024-05-08T22:44:27.252565+0000","last_became_peered":"2024-05-08T22:44:27.252565+0000","last_unstale":"2024-05-08T22:44:28.226963+0000","last_undegraded":"2024-05-08T22:44:28.226963+0000","last_fullsized":"2024-05-08T22:44:28.226963+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:39.962232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435574+0000","last_change":"2024-05-08T22:44:26.857400+0000","last_active":"2024-05-08T22:44:28.435574+0000","last_peered":"2024-05-08T22:44:28.435574+0000","last_clean":"2024-05-08T22:44:28.435574+0000","last_became_active":"2024-05-08T22:44:26.857165+0000","last_became_peered":"2024-05-08T22:44:26.857165+0000","last_unstale":"2024-05-08T22:44:28.435574+0000","last_undegraded":"2024-05-08T22:44:28.435574+0000","last_fullsized":"2024-05-08T22:44:28.435574+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:27:15.076936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433767+0000","last_change":"2024-05-08T22:44:27.234690+0000","last_active":"2024-05-08T22:44:28.433767+0000","last_peered":"2024-05-08T22:44:28.433767+0000","last_clean":"2024-05-08T22:44:28.433767+0000","last_became_active":"2024-05-08T22:44:27.233902+0000","last_became_peered":"2024-05-08T22:44:27.233902+0000","last_unstale":"2024-05-08T22:44:28.433767+0000","last_undegraded":"2024-05-08T22:44:28.433767+0000","last_fullsized":"2024-05-08T22:44:28.433767+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:05.063358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.242397+0000","last_change":"2024-05-08T22:44:26.853540+0000","last_active":"2024-05-08T22:44:27.242397+0000","last_peered":"2024-05-08T22:44:27.242397+0000","last_clean":"2024-05-08T22:44:27.242397+0000","last_became_active":"2024-05-08T22:44:26.853234+0000","last_became_peered":"2024-05-08T22:44:26.853234+0000","last_unstale":"2024-05-08T22:44:27.242397+0000","last_undegraded":"2024-05-08T22:44:27.242397+0000","last_fullsized":"2024-05-08T22:44:27.242397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:44.136301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.241110+0000","last_change":"2024-05-08T22:44:27.240211+0000","last_active":"2024-05-08T22:44:27.241110+0000","last_peered":"2024-05-08T22:44:27.241110+0000","last_clean":"2024-05-08T22:44:27.241110+0000","last_became_active":"2024-05-08T22:44:27.239893+0000","last_became_peered":"2024-05-08T22:44:27.239893+0000","last_unstale":"2024-05-08T22:44:27.241110+0000","last_undegraded":"2024-05-08T22:44:27.241110+0000","last_fullsized":"2024-05-08T22:44:27.241110+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:14.568501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434260+0000","last_change":"2024-05-08T22:44:26.858947+0000","last_active":"2024-05-08T22:44:28.434260+0000","last_peered":"2024-05-08T22:44:28.434260+0000","last_clean":"2024-05-08T22:44:28.434260+0000","last_became_active":"2024-05-08T22:44:26.857501+0000","last_became_peered":"2024-05-08T22:44:26.857501+0000","last_unstale":"2024-05-08T22:44:28.434260+0000","last_undegraded":"2024-05-08T22:44:28.434260+0000","last_fullsized":"2024-05-08T22:44:28.434260+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:32.349718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434293+0000","last_change":"2024-05-08T22:44:27.246721+0000","last_active":"2024-05-08T22:44:28.434293+0000","last_peered":"2024-05-08T22:44:28.434293+0000","last_clean":"2024-05-08T22:44:28.434293+0000","last_became_active":"2024-05-08T22:44:27.245475+0000","last_became_peered":"2024-05-08T22:44:27.245475+0000","last_unstale":"2024-05-08T22:44:28.434293+0000","last_undegraded":"2024-05-08T22:44:28.434293+0000","last_fullsized":"2024-05-08T22:44:28.434293+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:05.420141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431226+0000","last_change":"2024-05-08T22:44:26.850463+0000","last_active":"2024-05-08T22:44:28.431226+0000","last_peered":"2024-05-08T22:44:28.431226+0000","last_clean":"2024-05-08T22:44:28.431226+0000","last_became_active":"2024-05-08T22:44:26.849473+0000","last_became_peered":"2024-05-08T22:44:26.849473+0000","last_unstale":"2024-05-08T22:44:28.431226+0000","last_undegraded":"2024-05-08T22:44:28.431226+0000","last_fullsized":"2024-05-08T22:44:28.431226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:44.256283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434564+0000","last_change":"2024-05-08T22:44:27.238899+0000","last_active":"2024-05-08T22:44:28.434564+0000","last_peered":"2024-05-08T22:44:28.434564+0000","last_clean":"2024-05-08T22:44:28.434564+0000","last_became_active":"2024-05-08T22:44:27.234886+0000","last_became_peered":"2024-05-08T22:44:27.234886+0000","last_unstale":"2024-05-08T22:44:28.434564+0000","last_undegraded":"2024-05-08T22:44:28.434564+0000","last_fullsized":"2024-05-08T22:44:28.434564+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:39.204876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433734+0000","last_change":"2024-05-08T22:44:26.857187+0000","last_active":"2024-05-08T22:44:28.433734+0000","last_peered":"2024-05-08T22:44:28.433734+0000","last_clean":"2024-05-08T22:44:28.433734+0000","last_became_active":"2024-05-08T22:44:26.856905+0000","last_became_peered":"2024-05-08T22:44:26.856905+0000","last_unstale":"2024-05-08T22:44:28.433734+0000","last_undegraded":"2024-05-08T22:44:28.433734+0000","last_fullsized":"2024-05-08T22:44:28.433734+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:50.425391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224791+0000","last_change":"2024-05-08T22:44:27.253706+0000","last_active":"2024-05-08T22:44:28.224791+0000","last_peered":"2024-05-08T22:44:28.224791+0000","last_clean":"2024-05-08T22:44:28.224791+0000","last_became_active":"2024-05-08T22:44:27.253473+0000","last_became_peered":"2024-05-08T22:44:27.253473+0000","last_unstale":"2024-05-08T22:44:28.224791+0000","last_undegraded":"2024-05-08T22:44:28.224791+0000","last_fullsized":"2024-05-08T22:44:28.224791+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:26.651963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225041+0000","last_change":"2024-05-08T22:44:26.854460+0000","last_active":"2024-05-08T22:44:28.225041+0000","last_peered":"2024-05-08T22:44:28.225041+0000","last_clean":"2024-05-08T22:44:28.225041+0000","last_became_active":"2024-05-08T22:44:26.854157+0000","last_became_peered":"2024-05-08T22:44:26.854157+0000","last_unstale":"2024-05-08T22:44:28.225041+0000","last_undegraded":"2024-05-08T22:44:28.225041+0000","last_fullsized":"2024-05-08T22:44:28.225041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:43.817777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431494+0000","last_change":"2024-05-08T22:44:27.242915+0000","last_active":"2024-05-08T22:44:28.431494+0000","last_peered":"2024-05-08T22:44:28.431494+0000","last_clean":"2024-05-08T22:44:28.431494+0000","last_became_active":"2024-05-08T22:44:27.242650+0000","last_became_peered":"2024-05-08T22:44:27.242650+0000","last_unstale":"2024-05-08T22:44:28.431494+0000","last_undegraded":"2024-05-08T22:44:28.431494+0000","last_fullsized":"2024-05-08T22:44:28.431494+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:03.843907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431816+0000","last_change":"2024-05-08T22:44:26.859156+0000","last_active":"2024-05-08T22:44:28.431816+0000","last_peered":"2024-05-08T22:44:28.431816+0000","last_clean":"2024-05-08T22:44:28.431816+0000","last_became_active":"2024-05-08T22:44:26.858902+0000","last_became_peered":"2024-05-08T22:44:26.858902+0000","last_unstale":"2024-05-08T22:44:28.431816+0000","last_undegraded":"2024-05-08T22:44:28.431816+0000","last_fullsized":"2024-05-08T22:44:28.431816+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:55.996129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434915+0000","last_change":"2024-05-08T22:44:27.250024+0000","last_active":"2024-05-08T22:44:28.434915+0000","last_peered":"2024-05-08T22:44:28.434915+0000","last_clean":"2024-05-08T22:44:28.434915+0000","last_became_active":"2024-05-08T22:44:27.249766+0000","last_became_peered":"2024-05-08T22:44:27.249766+0000","last_unstale":"2024-05-08T22:44:28.434915+0000","last_undegraded":"2024-05-08T22:44:28.434915+0000","last_fullsized":"2024-05-08T22:44:28.434915+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:39.290129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"126'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431720+0000","last_change":"2024-05-08T22:44:26.853398+0000","last_active":"2024-05-08T22:44:28.431720+0000","last_peered":"2024-05-08T22:44:28.431720+0000","last_clean":"2024-05-08T22:44:28.431720+0000","last_became_active":"2024-05-08T22:44:26.851432+0000","last_became_peered":"2024-05-08T22:44:26.851432+0000","last_unstale":"2024-05-08T22:44:28.431720+0000","last_undegraded":"2024-05-08T22:44:28.431720+0000","last_fullsized":"2024-05-08T22:44:28.431720+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:05.497573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432013+0000","last_change":"2024-05-08T22:44:27.241947+0000","last_active":"2024-05-08T22:44:28.432013+0000","last_peered":"2024-05-08T22:44:28.432013+0000","last_clean":"2024-05-08T22:44:28.432013+0000","last_became_active":"2024-05-08T22:44:27.241681+0000","last_became_peered":"2024-05-08T22:44:27.241681+0000","last_unstale":"2024-05-08T22:44:28.432013+0000","last_undegraded":"2024-05-08T22:44:28.432013+0000","last_fullsized":"2024-05-08T22:44:28.432013+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:35.223658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"126'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434346+0000","last_change":"2024-05-08T22:44:26.856488+0000","last_active":"2024-05-08T22:44:28.434346+0000","last_peered":"2024-05-08T22:44:28.434346+0000","last_clean":"2024-05-08T22:44:28.434346+0000","last_became_active":"2024-05-08T22:44:26.856178+0000","last_became_peered":"2024-05-08T22:44:26.856178+0000","last_unstale":"2024-05-08T22:44:28.434346+0000","last_undegraded":"2024-05-08T22:44:28.434346+0000","last_fullsized":"2024-05-08T22:44:28.434346+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:36.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224639+0000","last_change":"2024-05-08T22:44:27.242772+0000","last_active":"2024-05-08T22:44:28.224639+0000","last_peered":"2024-05-08T22:44:28.224639+0000","last_clean":"2024-05-08T22:44:28.224639+0000","last_became_active":"2024-05-08T22:44:27.241723+0000","last_became_peered":"2024-05-08T22:44:27.241723+0000","last_unstale":"2024-05-08T22:44:28.224639+0000","last_undegraded":"2024-05-08T22:44:28.224639+0000","last_fullsized":"2024-05-08T22:44:28.224639+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:05.063358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435415+0000","last_change":"2024-05-08T22:44:26.855885+0000","last_active":"2024-05-08T22:44:28.435415+0000","last_peered":"2024-05-08T22:44:28.435415+0000","last_clean":"2024-05-08T22:44:28.435415+0000","last_became_active":"2024-05-08T22:44:26.855627+0000","last_became_peered":"2024-05-08T22:44:26.855627+0000","last_unstale":"2024-05-08T22:44:28.435415+0000","last_undegraded":"2024-05-08T22:44:28.435415+0000","last_fullsized":"2024-05-08T22:44:28.435415+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:50.156208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434058+0000","last_change":"2024-05-08T22:44:27.259680+0000","last_active":"2024-05-08T22:44:28.434058+0000","last_peered":"2024-05-08T22:44:28.434058+0000","last_clean":"2024-05-08T22:44:28.434058+0000","last_became_active":"2024-05-08T22:44:27.259427+0000","last_became_peered":"2024-05-08T22:44:27.259427+0000","last_unstale":"2024-05-08T22:44:28.434058+0000","last_undegraded":"2024-05-08T22:44:28.434058+0000","last_fullsized":"2024-05-08T22:44:28.434058+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:27.934776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434876+0000","last_change":"2024-05-08T22:44:26.858383+0000","last_active":"2024-05-08T22:44:28.434876+0000","last_peered":"2024-05-08T22:44:28.434876+0000","last_clean":"2024-05-08T22:44:28.434876+0000","last_became_active":"2024-05-08T22:44:26.858034+0000","last_became_peered":"2024-05-08T22:44:26.858034+0000","last_unstale":"2024-05-08T22:44:28.434876+0000","last_undegraded":"2024-05-08T22:44:28.434876+0000","last_fullsized":"2024-05-08T22:44:28.434876+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:25.641268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436144+0000","last_change":"2024-05-08T22:44:27.236192+0000","last_active":"2024-05-08T22:44:28.436144+0000","last_peered":"2024-05-08T22:44:28.436144+0000","last_clean":"2024-05-08T22:44:28.436144+0000","last_became_active":"2024-05-08T22:44:27.235945+0000","last_became_peered":"2024-05-08T22:44:27.235945+0000","last_unstale":"2024-05-08T22:44:28.436144+0000","last_undegraded":"2024-05-08T22:44:28.436144+0000","last_fullsized":"2024-05-08T22:44:28.436144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:49:30.921993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433565+0000","last_change":"2024-05-08T22:44:26.852752+0000","last_active":"2024-05-08T22:44:28.433565+0000","last_peered":"2024-05-08T22:44:28.433565+0000","last_clean":"2024-05-08T22:44:28.433565+0000","last_became_active":"2024-05-08T22:44:26.852486+0000","last_became_peered":"2024-05-08T22:44:26.852486+0000","last_unstale":"2024-05-08T22:44:28.433565+0000","last_undegraded":"2024-05-08T22:44:28.433565+0000","last_fullsized":"2024-05-08T22:44:28.433565+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:47.764271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431480+0000","last_change":"2024-05-08T22:44:27.246835+0000","last_active":"2024-05-08T22:44:28.431480+0000","last_peered":"2024-05-08T22:44:28.431480+0000","last_clean":"2024-05-08T22:44:28.431480+0000","last_became_active":"2024-05-08T22:44:27.246514+0000","last_became_peered":"2024-05-08T22:44:27.246514+0000","last_unstale":"2024-05-08T22:44:28.431480+0000","last_undegraded":"2024-05-08T22:44:28.431480+0000","last_fullsized":"2024-05-08T22:44:28.431480+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:46.361700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435318+0000","last_change":"2024-05-08T22:44:26.855821+0000","last_active":"2024-05-08T22:44:28.435318+0000","last_peered":"2024-05-08T22:44:28.435318+0000","last_clean":"2024-05-08T22:44:28.435318+0000","last_became_active":"2024-05-08T22:44:26.855545+0000","last_became_peered":"2024-05-08T22:44:26.855545+0000","last_unstale":"2024-05-08T22:44:28.435318+0000","last_undegraded":"2024-05-08T22:44:28.435318+0000","last_fullsized":"2024-05-08T22:44:28.435318+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:59.944611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.237012+0000","last_change":"2024-05-08T22:44:27.234998+0000","last_active":"2024-05-08T22:44:27.237012+0000","last_peered":"2024-05-08T22:44:27.237012+0000","last_clean":"2024-05-08T22:44:27.237012+0000","last_became_active":"2024-05-08T22:44:27.234701+0000","last_became_peered":"2024-05-08T22:44:27.234701+0000","last_unstale":"2024-05-08T22:44:27.237012+0000","last_undegraded":"2024-05-08T22:44:27.237012+0000","last_fullsized":"2024-05-08T22:44:27.237012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:02.054358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436174+0000","last_change":"2024-05-08T22:44:26.858280+0000","last_active":"2024-05-08T22:44:28.436174+0000","last_peered":"2024-05-08T22:44:28.436174+0000","last_clean":"2024-05-08T22:44:28.436174+0000","last_became_active":"2024-05-08T22:44:26.858043+0000","last_became_peered":"2024-05-08T22:44:26.858043+0000","last_unstale":"2024-05-08T22:44:28.436174+0000","last_undegraded":"2024-05-08T22:44:28.436174+0000","last_fullsized":"2024-05-08T22:44:28.436174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:22.774203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436120+0000","last_change":"2024-05-08T22:44:27.240949+0000","last_active":"2024-05-08T22:44:28.436120+0000","last_peered":"2024-05-08T22:44:28.436120+0000","last_clean":"2024-05-08T22:44:28.436120+0000","last_became_active":"2024-05-08T22:44:27.240707+0000","last_became_peered":"2024-05-08T22:44:27.240707+0000","last_unstale":"2024-05-08T22:44:28.436120+0000","last_undegraded":"2024-05-08T22:44:28.436120+0000","last_fullsized":"2024-05-08T22:44:28.436120+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:22:02.625962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226558+0000","last_change":"2024-05-08T22:44:26.852443+0000","last_active":"2024-05-08T22:44:28.226558+0000","last_peered":"2024-05-08T22:44:28.226558+0000","last_clean":"2024-05-08T22:44:28.226558+0000","last_became_active":"2024-05-08T22:44:26.852168+0000","last_became_peered":"2024-05-08T22:44:26.852168+0000","last_unstale":"2024-05-08T22:44:28.226558+0000","last_undegraded":"2024-05-08T22:44:28.226558+0000","last_fullsized":"2024-05-08T22:44:28.226558+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:10.425902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.432140+0000","last_change":"2024-05-08T22:44:27.257729+0000","last_active":"2024-05-08T22:44:28.432140+0000","last_peered":"2024-05-08T22:44:28.432140+0000","last_clean":"2024-05-08T22:44:28.432140+0000","last_became_active":"2024-05-08T22:44:27.257453+0000","last_became_peered":"2024-05-08T22:44:27.257453+0000","last_unstale":"2024-05-08T22:44:28.432140+0000","last_undegraded":"2024-05-08T22:44:28.432140+0000","last_fullsized":"2024-05-08T22:44:28.432140+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:06.344892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225987+0000","last_change":"2024-05-08T22:44:26.856157+0000","last_active":"2024-05-08T22:44:28.225987+0000","last_peered":"2024-05-08T22:44:28.225987+0000","last_clean":"2024-05-08T22:44:28.225987+0000","last_became_active":"2024-05-08T22:44:26.855878+0000","last_became_peered":"2024-05-08T22:44:26.855878+0000","last_unstale":"2024-05-08T22:44:28.225987+0000","last_undegraded":"2024-05-08T22:44:28.225987+0000","last_fullsized":"2024-05-08T22:44:28.225987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:00.304253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433333+0000","last_change":"2024-05-08T22:44:27.240922+0000","last_active":"2024-05-08T22:44:28.433333+0000","last_peered":"2024-05-08T22:44:28.433333+0000","last_clean":"2024-05-08T22:44:28.433333+0000","last_became_active":"2024-05-08T22:44:27.240665+0000","last_became_peered":"2024-05-08T22:44:27.240665+0000","last_unstale":"2024-05-08T22:44:28.433333+0000","last_undegraded":"2024-05-08T22:44:28.433333+0000","last_fullsized":"2024-05-08T22:44:28.433333+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:15:14.418456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434332+0000","last_change":"2024-05-08T22:44:26.854920+0000","last_active":"2024-05-08T22:44:28.434332+0000","last_peered":"2024-05-08T22:44:28.434332+0000","last_clean":"2024-05-08T22:44:28.434332+0000","last_became_active":"2024-05-08T22:44:26.854563+0000","last_became_peered":"2024-05-08T22:44:26.854563+0000","last_unstale":"2024-05-08T22:44:28.434332+0000","last_undegraded":"2024-05-08T22:44:28.434332+0000","last_fullsized":"2024-05-08T22:44:28.434332+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:37.361168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433388+0000","last_change":"2024-05-08T22:44:27.247027+0000","last_active":"2024-05-08T22:44:28.433388+0000","last_peered":"2024-05-08T22:44:28.433388+0000","last_clean":"2024-05-08T22:44:28.433388+0000","last_became_active":"2024-05-08T22:44:27.246820+0000","last_became_peered":"2024-05-08T22:44:27.246820+0000","last_unstale":"2024-05-08T22:44:28.433388+0000","last_undegraded":"2024-05-08T22:44:28.433388+0000","last_fullsized":"2024-05-08T22:44:28.433388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:27.711668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225757+0000","last_change":"2024-05-08T22:44:26.843380+0000","last_active":"2024-05-08T22:44:28.225757+0000","last_peered":"2024-05-08T22:44:28.225757+0000","last_clean":"2024-05-08T22:44:28.225757+0000","last_became_active":"2024-05-08T22:44:26.843097+0000","last_became_peered":"2024-05-08T22:44:26.843097+0000","last_unstale":"2024-05-08T22:44:28.225757+0000","last_undegraded":"2024-05-08T22:44:28.225757+0000","last_fullsized":"2024-05-08T22:44:28.225757+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:45.702545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434683+0000","last_change":"2024-05-08T22:44:27.251231+0000","last_active":"2024-05-08T22:44:28.434683+0000","last_peered":"2024-05-08T22:44:28.434683+0000","last_clean":"2024-05-08T22:44:28.434683+0000","last_became_active":"2024-05-08T22:44:27.250986+0000","last_became_peered":"2024-05-08T22:44:27.250986+0000","last_unstale":"2024-05-08T22:44:28.434683+0000","last_undegraded":"2024-05-08T22:44:28.434683+0000","last_fullsized":"2024-05-08T22:44:28.434683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:41:25.785836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433738+0000","last_change":"2024-05-08T22:44:26.858863+0000","last_active":"2024-05-08T22:44:28.433738+0000","last_peered":"2024-05-08T22:44:28.433738+0000","last_clean":"2024-05-08T22:44:28.433738+0000","last_became_active":"2024-05-08T22:44:26.858622+0000","last_became_peered":"2024-05-08T22:44:26.858622+0000","last_unstale":"2024-05-08T22:44:28.433738+0000","last_undegraded":"2024-05-08T22:44:28.433738+0000","last_fullsized":"2024-05-08T22:44:28.433738+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:38.194181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.431201+0000","last_change":"2024-05-08T22:44:27.242046+0000","last_active":"2024-05-08T22:44:28.431201+0000","last_peered":"2024-05-08T22:44:28.431201+0000","last_clean":"2024-05-08T22:44:28.431201+0000","last_became_active":"2024-05-08T22:44:27.241782+0000","last_became_peered":"2024-05-08T22:44:27.241782+0000","last_unstale":"2024-05-08T22:44:28.431201+0000","last_undegraded":"2024-05-08T22:44:28.431201+0000","last_fullsized":"2024-05-08T22:44:28.431201+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:17.116798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.223742+0000","last_change":"2024-05-08T22:44:26.845048+0000","last_active":"2024-05-08T22:44:28.223742+0000","last_peered":"2024-05-08T22:44:28.223742+0000","last_clean":"2024-05-08T22:44:28.223742+0000","last_became_active":"2024-05-08T22:44:26.844708+0000","last_became_peered":"2024-05-08T22:44:26.844708+0000","last_unstale":"2024-05-08T22:44:28.223742+0000","last_undegraded":"2024-05-08T22:44:28.223742+0000","last_fullsized":"2024-05-08T22:44:28.223742+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:57.320506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435038+0000","last_change":"2024-05-08T22:44:27.242497+0000","last_active":"2024-05-08T22:44:28.435038+0000","last_peered":"2024-05-08T22:44:28.435038+0000","last_clean":"2024-05-08T22:44:28.435038+0000","last_became_active":"2024-05-08T22:44:27.241951+0000","last_became_peered":"2024-05-08T22:44:27.241951+0000","last_unstale":"2024-05-08T22:44:28.435038+0000","last_undegraded":"2024-05-08T22:44:28.435038+0000","last_fullsized":"2024-05-08T22:44:28.435038+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:08.018968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434871+0000","last_change":"2024-05-08T22:44:26.849947+0000","last_active":"2024-05-08T22:44:28.434871+0000","last_peered":"2024-05-08T22:44:28.434871+0000","last_clean":"2024-05-08T22:44:28.434871+0000","last_became_active":"2024-05-08T22:44:26.849682+0000","last_became_peered":"2024-05-08T22:44:26.849682+0000","last_unstale":"2024-05-08T22:44:28.434871+0000","last_undegraded":"2024-05-08T22:44:28.434871+0000","last_fullsized":"2024-05-08T22:44:28.434871+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:27.421388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.435022+0000","last_change":"2024-05-08T22:44:27.238406+0000","last_active":"2024-05-08T22:44:28.435022+0000","last_peered":"2024-05-08T22:44:28.435022+0000","last_clean":"2024-05-08T22:44:28.435022+0000","last_became_active":"2024-05-08T22:44:27.234475+0000","last_became_peered":"2024-05-08T22:44:27.234475+0000","last_unstale":"2024-05-08T22:44:28.435022+0000","last_undegraded":"2024-05-08T22:44:28.435022+0000","last_fullsized":"2024-05-08T22:44:28.435022+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:27.616461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.224759+0000","last_change":"2024-05-08T22:44:26.855583+0000","last_active":"2024-05-08T22:44:28.224759+0000","last_peered":"2024-05-08T22:44:28.224759+0000","last_clean":"2024-05-08T22:44:28.224759+0000","last_became_active":"2024-05-08T22:44:26.855293+0000","last_became_peered":"2024-05-08T22:44:26.855293+0000","last_unstale":"2024-05-08T22:44:28.224759+0000","last_undegraded":"2024-05-08T22:44:28.224759+0000","last_fullsized":"2024-05-08T22:44:28.224759+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:48.927233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226392+0000","last_change":"2024-05-08T22:44:27.247151+0000","last_active":"2024-05-08T22:44:28.226392+0000","last_peered":"2024-05-08T22:44:28.226392+0000","last_clean":"2024-05-08T22:44:28.226392+0000","last_became_active":"2024-05-08T22:44:27.246896+0000","last_became_peered":"2024-05-08T22:44:27.246896+0000","last_unstale":"2024-05-08T22:44:28.226392+0000","last_undegraded":"2024-05-08T22:44:28.226392+0000","last_fullsized":"2024-05-08T22:44:28.226392+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:13.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225265+0000","last_change":"2024-05-08T22:44:26.854419+0000","last_active":"2024-05-08T22:44:28.225265+0000","last_peered":"2024-05-08T22:44:28.225265+0000","last_clean":"2024-05-08T22:44:28.225265+0000","last_became_active":"2024-05-08T22:44:26.854113+0000","last_became_peered":"2024-05-08T22:44:26.854113+0000","last_unstale":"2024-05-08T22:44:28.225265+0000","last_undegraded":"2024-05-08T22:44:28.225265+0000","last_fullsized":"2024-05-08T22:44:28.225265+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:30:07.352180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.436208+0000","last_change":"2024-05-08T22:44:27.253303+0000","last_active":"2024-05-08T22:44:28.436208+0000","last_peered":"2024-05-08T22:44:28.436208+0000","last_clean":"2024-05-08T22:44:28.436208+0000","last_became_active":"2024-05-08T22:44:27.253070+0000","last_became_peered":"2024-05-08T22:44:27.253070+0000","last_unstale":"2024-05-08T22:44:28.436208+0000","last_undegraded":"2024-05-08T22:44:28.436208+0000","last_fullsized":"2024-05-08T22:44:28.436208+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:56.095882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434547+0000","last_change":"2024-05-08T22:44:26.859610+0000","last_active":"2024-05-08T22:44:28.434547+0000","last_peered":"2024-05-08T22:44:28.434547+0000","last_clean":"2024-05-08T22:44:28.434547+0000","last_became_active":"2024-05-08T22:44:26.859368+0000","last_became_peered":"2024-05-08T22:44:26.859368+0000","last_unstale":"2024-05-08T22:44:28.434547+0000","last_undegraded":"2024-05-08T22:44:28.434547+0000","last_fullsized":"2024-05-08T22:44:28.434547+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:46:43.956592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433545+0000","last_change":"2024-05-08T22:44:27.236447+0000","last_active":"2024-05-08T22:44:28.433545+0000","last_peered":"2024-05-08T22:44:28.433545+0000","last_clean":"2024-05-08T22:44:28.433545+0000","last_became_active":"2024-05-08T22:44:27.235584+0000","last_became_peered":"2024-05-08T22:44:27.235584+0000","last_unstale":"2024-05-08T22:44:28.433545+0000","last_undegraded":"2024-05-08T22:44:28.433545+0000","last_fullsized":"2024-05-08T22:44:28.433545+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:31.224173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.228122+0000","last_change":"2024-05-08T22:44:26.844366+0000","last_active":"2024-05-08T22:44:27.228122+0000","last_peered":"2024-05-08T22:44:27.228122+0000","last_clean":"2024-05-08T22:44:27.228122+0000","last_became_active":"2024-05-08T22:44:26.844087+0000","last_became_peered":"2024-05-08T22:44:26.844087+0000","last_unstale":"2024-05-08T22:44:27.228122+0000","last_undegraded":"2024-05-08T22:44:27.228122+0000","last_fullsized":"2024-05-08T22:44:27.228122+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:38.879657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433530+0000","last_change":"2024-05-08T22:44:27.246062+0000","last_active":"2024-05-08T22:44:28.433530+0000","last_peered":"2024-05-08T22:44:28.433530+0000","last_clean":"2024-05-08T22:44:28.433530+0000","last_became_active":"2024-05-08T22:44:27.245658+0000","last_became_peered":"2024-05-08T22:44:27.245658+0000","last_unstale":"2024-05-08T22:44:28.433530+0000","last_undegraded":"2024-05-08T22:44:28.433530+0000","last_fullsized":"2024-05-08T22:44:28.433530+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:38.181480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.230710+0000","last_change":"2024-05-08T22:44:26.852457+0000","last_active":"2024-05-08T22:44:27.230710+0000","last_peered":"2024-05-08T22:44:27.230710+0000","last_clean":"2024-05-08T22:44:27.230710+0000","last_became_active":"2024-05-08T22:44:26.852098+0000","last_became_peered":"2024-05-08T22:44:26.852098+0000","last_unstale":"2024-05-08T22:44:27.230710+0000","last_undegraded":"2024-05-08T22:44:27.230710+0000","last_fullsized":"2024-05-08T22:44:27.230710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:30.743783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.225614+0000","last_change":"2024-05-08T22:44:27.248206+0000","last_active":"2024-05-08T22:44:28.225614+0000","last_peered":"2024-05-08T22:44:28.225614+0000","last_clean":"2024-05-08T22:44:28.225614+0000","last_became_active":"2024-05-08T22:44:27.247905+0000","last_became_peered":"2024-05-08T22:44:27.247905+0000","last_unstale":"2024-05-08T22:44:28.225614+0000","last_undegraded":"2024-05-08T22:44:28.225614+0000","last_fullsized":"2024-05-08T22:44:28.225614+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:05.983635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434104+0000","last_change":"2024-05-08T22:44:26.853919+0000","last_active":"2024-05-08T22:44:28.434104+0000","last_peered":"2024-05-08T22:44:28.434104+0000","last_clean":"2024-05-08T22:44:28.434104+0000","last_became_active":"2024-05-08T22:44:26.853658+0000","last_became_peered":"2024-05-08T22:44:26.853658+0000","last_unstale":"2024-05-08T22:44:28.434104+0000","last_undegraded":"2024-05-08T22:44:28.434104+0000","last_fullsized":"2024-05-08T22:44:28.434104+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:55.816420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.434838+0000","last_change":"2024-05-08T22:44:27.259673+0000","last_active":"2024-05-08T22:44:28.434838+0000","last_peered":"2024-05-08T22:44:28.434838+0000","last_clean":"2024-05-08T22:44:28.434838+0000","last_became_active":"2024-05-08T22:44:27.259418+0000","last_became_peered":"2024-05-08T22:44:27.259418+0000","last_unstale":"2024-05-08T22:44:28.434838+0000","last_undegraded":"2024-05-08T22:44:28.434838+0000","last_fullsized":"2024-05-08T22:44:28.434838+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:48.327322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.223970+0000","last_change":"2024-05-08T22:44:26.845695+0000","last_active":"2024-05-08T22:44:28.223970+0000","last_peered":"2024-05-08T22:44:28.223970+0000","last_clean":"2024-05-08T22:44:28.223970+0000","last_became_active":"2024-05-08T22:44:26.845330+0000","last_became_peered":"2024-05-08T22:44:26.845330+0000","last_unstale":"2024-05-08T22:44:28.223970+0000","last_undegraded":"2024-05-08T22:44:28.223970+0000","last_fullsized":"2024-05-08T22:44:28.223970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:44.136301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.226231+0000","last_change":"2024-05-08T22:44:27.256439+0000","last_active":"2024-05-08T22:44:28.226231+0000","last_peered":"2024-05-08T22:44:28.226231+0000","last_clean":"2024-05-08T22:44:28.226231+0000","last_became_active":"2024-05-08T22:44:27.252590+0000","last_became_peered":"2024-05-08T22:44:27.252590+0000","last_unstale":"2024-05-08T22:44:28.226231+0000","last_undegraded":"2024-05-08T22:44:28.226231+0000","last_fullsized":"2024-05-08T22:44:28.226231+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:26.826128+0000","last_clean_scrub_stamp":"2024-05-08T22:44:26.826128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:53.154095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"126'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T22:44:28.433522+0000","last_change":"2024-05-08T22:44:26.851306+0000","last_active":"2024-05-08T22:44:28.433522+0000","last_peered":"2024-05-08T22:44:28.433522+0000","last_clean":"2024-05-08T22:44:28.433522+0000","last_became_active":"2024-05-08T22:44:26.851071+0000","last_became_peered":"2024-05-08T22:44:26.851071+0000","last_unstale":"2024-05-08T22:44:28.433522+0000","last_undegraded":"2024-05-08T22:44:28.433522+0000","last_fullsized":"2024-05-08T22:44:28.433522+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:44:25.815433+0000","last_clean_scrub_stamp":"2024-05-08T22:44:25.815433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:02.354185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":263,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T22:44:27.224179+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:44:27.224179+0000","last_peered":"2024-05-08T22:44:27.224179+0000","last_clean":"2024-05-08T22:44:27.224179+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:44:27.224179+0000","last_undegraded":"2024-05-08T22:44:27.224179+0000","last_fullsized":"2024-05-08T22:44:27.224179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":730378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.63},"min":{"1min":0.377,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90600000000000003,"5min":1.623,"15min":1.623},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.46500000000000002,"5min":0.375,"15min":0.307},"max":{"1min":1.028,"5min":1.115,"15min":1.6899999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.496,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.1639999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.75},"min":{"1min":0.58999999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.997,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.42199999999999999},"max":{"1min":1.052,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.79300000000000004},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0780000000000001,"5min":1.716,"15min":1.716},"last":1.012}]},{"osd":4,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.89900000000000002,"15min":0.90500000000000003},"min":{"1min":0.623,"5min":0.48799999999999999,"15min":0.47599999999999998},"max":{"1min":1.222,"5min":1.2929999999999999,"15min":1.306},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95499999999999996,"15min":0.95099999999999996},"min":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1399999999999999,"5min":1.345,"15min":1.4139999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.99099999999999999,"15min":0.97999999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.111,"5min":1.429,"15min":1.429},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0389999999999999,"15min":1.0269999999999999},"min":{"1min":0.81599999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.472,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.085}]},{"osd":6,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99199999999999999,"15min":0.98299999999999998},"min":{"1min":0.71999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.331,"5min":1.421,"15min":1.446},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.78100000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.214,"5min":1.512,"15min":1.512},"last":1.272}]},{"osd":7,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.091,"15min":1.095},"min":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.409,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3149999999999999,"5min":1.476,"15min":1.8360000000000001},"last":1.244}]}]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1576960,"data_stored":728802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.772,"15min":1.772},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.95699999999999996,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.91600000000000004,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.81299999999999994,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.042,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.88}]},{"osd":3,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.79600000000000004},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.131,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.583},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.91700000000000004,"15min":0.94899999999999995},"min":{"1min":0.64400000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.228,"5min":1.395,"15min":1.431},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0129999999999999,"15min":0.98599999999999999},"min":{"1min":0.747,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.224,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2210000000000001,"5min":1.389,"15min":1.5},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.98399999999999999,"15min":1.042},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3640000000000001,"5min":1.399,"15min":1.726},"last":1.2170000000000001}]},{"osd":6,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.98499999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.181,"5min":1.351,"15min":1.4650000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.82199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":1.0720000000000001}]},{"osd":7,"last update":"Wed May 8 22:43:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0780000000000001,"15min":1.071},"min":{"1min":0.81200000000000006,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3580000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.274},{"interface":"front","average":{"1min":1.034,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.4119999999999999,"5min":1.446,"15min":1.5049999999999999},"last":1.123}]}]},{"osd":7,"up_from":14,"seq":60129542282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2183168,"data_stored":1319924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.53700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.88300000000000001,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.514},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.88900000000000001},"min":{"1min":0.61799999999999999,"5min":0.47899999999999998,"15min":0.46999999999999997},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92600000000000005,"15min":0.89100000000000001},"min":{"1min":0.629,"5min":0.58199999999999996,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.415,"15min":1.488},"last":0.63}]},{"osd":2,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92700000000000005,"15min":0.90800000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58199999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94499999999999995,"15min":0.95499999999999996},"min":{"1min":0.79900000000000004,"5min":0.61499999999999999,"15min":0.53800000000000003},"max":{"1min":1.1459999999999999,"5min":1.256,"15min":1.337},"last":1.1679999999999999}]},{"osd":3,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.53900000000000003},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.042},{"interface":"front","average":{"1min":0.878,"5min":0.877,"15min":0.86099999999999999},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.571,"5min":1.571,"15min":1.611},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.90000000000000002,"15min":0.89300000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.94399999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.53300000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.304},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0860000000000001,"15min":1.0580000000000001},"min":{"1min":0.82199999999999995,"5min":0.80600000000000005,"15min":0.60199999999999998},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.373}]},{"osd":6,"last update":"Wed May 8 22:44:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.96399999999999997,"15min":0.95199999999999996},"min":{"1min":0.69399999999999995,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.81799999999999995,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.599},"last":1.4319999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1564672,"data_stored":722742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.64400000000000002,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.2430000000000001,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.758},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.1140000000000001,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":1.0049999999999999}]},{"osd":2,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.93799999999999994},"min":{"1min":0.498,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.1479999999999999,"5min":1.214,"15min":1.4319999999999999},"last":0.745},{"interface":"front","average":{"1min":0.87,"5min":0.89500000000000002,"15min":0.90800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.98599999999999999}]},{"osd":3,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94599999999999995,"15min":1.024},"min":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.204,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.41799999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.5029999999999999},"last":1.095}]},{"osd":4,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92800000000000005,"15min":0.96399999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.538},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94999999999999996,"15min":0.995},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.44400000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.03}]},{"osd":5,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0249999999999999,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.64200000000000002,"15min":0.64000000000000001},"max":{"1min":1.278,"5min":1.427,"15min":1.681},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.044,"15min":1.0900000000000001},"min":{"1min":0.79700000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.6000000000000001},"last":1.161}]},{"osd":7,"last update":"Wed May 8 22:43:27 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95599999999999996,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.835},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.1799999999999999},"min":{"1min":0.81499999999999995,"5min":0.748,"15min":0.74099999999999999},"max":{"1min":1.405,"5min":1.5600000000000001,"15min":2.2999999999999998},"last":1.1319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542282,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1568768,"data_stored":722320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90200000000000002,"15min":0.84199999999999997},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.149,"5min":7.907,"15min":7.907},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":0.82699999999999996},"min":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.1499999999999999,"5min":1.29,"15min":1.52},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.90900000000000003,"15min":0.84899999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.1459999999999999,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":1.119}]},{"osd":2,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92300000000000004,"15min":0.89900000000000002},"min":{"1min":0.749,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2549999999999999,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.996,"15min":0.90400000000000003},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.48799999999999999},"max":{"1min":1.1379999999999999,"5min":10.121,"15min":10.121},"last":0.98799999999999999}]},{"osd":3,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88800000000000001,"15min":0.88100000000000001},"min":{"1min":0.72099999999999997,"5min":0.54000000000000004,"15min":0.501},"max":{"1min":1.1279999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.93700000000000006,"15min":0.96099999999999997},"min":{"1min":0.745,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.298},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.093,"15min":0.97899999999999998},"min":{"1min":0.67800000000000005,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.198,"5min":15.135,"15min":15.135},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.0609999999999999,"15min":1.0189999999999999},"min":{"1min":0.68799999999999994,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.3680000000000001,"5min":4.319,"15min":4.319},"last":0.70299999999999996}]},{"osd":6,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95499999999999996,"15min":0.94199999999999995},"min":{"1min":0.70099999999999996,"5min":0.59899999999999998,"15min":0.47599999999999998},"max":{"1min":1.347,"5min":1.3580000000000001,"15min":1.605},"last":1.026},{"interface":"front","average":{"1min":1.069,"5min":1.056,"15min":1.0529999999999999},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.54900000000000004},"max":{"1min":1.3160000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed May 8 22:43:29 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.111,"15min":1.097},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.57299999999999995},"max":{"1min":1.222,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.876},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.0780000000000001,"15min":1.0569999999999999},"min":{"1min":0.92300000000000004,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.3220000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.073}]}]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1548288,"data_stored":715065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85499999999999998,"15min":0.83999999999999997},"min":{"1min":0.53900000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.294},"last":0.76400000000000001}]},{"osd":1,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.53300000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.872,"15min":0.88600000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53100000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.3140000000000001},"last":1.097}]},{"osd":2,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.65700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.91700000000000004,"15min":0.92100000000000004},"min":{"1min":0.67200000000000004,"5min":0.623,"15min":0.53400000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.637},"last":0.95199999999999996}]},{"osd":3,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":2.601},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92900000000000005,"15min":1.02},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.139,"5min":1.22,"15min":2.8769999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90600000000000003,"15min":0.97599999999999998},"min":{"1min":0.64300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.792},"last":1.008},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.98899999999999999},"min":{"1min":0.77000000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.169,"5min":1.2949999999999999,"15min":1.575},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96299999999999997,"15min":1.0349999999999999},"min":{"1min":0.73199999999999998,"5min":0.64600000000000002,"15min":0.58599999999999997},"max":{"1min":1.276,"5min":1.331,"15min":1.831},"last":1.165},{"interface":"front","average":{"1min":1.105,"5min":1.04,"15min":1.075},"min":{"1min":0.78900000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.349}]},{"osd":7,"last update":"Wed May 8 22:43:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.76500000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.319,"5min":1.403,"15min":1.8500000000000001},"last":1.323},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4139999999999999,"5min":1.46,"15min":1.944},"last":1.266}]}]},{"osd":2,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1560576,"data_stored":722160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":0.93899999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.2190000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Wed May 8 22:43:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.96699999999999997,"5min":0.97399999999999998,"15min":1.048},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.96699999999999997,"5min":1.085,"15min":1.3460000000000001},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed May 8 22:43:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.56100000000000005,"5min":0.50700000000000001,"15min":0.312},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.88600000000000001,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.89900000000000002,"15min":0.91700000000000004},"min":{"1min":0.72499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.113,"5min":1.361,"15min":1.504},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93899999999999995,"15min":0.98999999999999999},"min":{"1min":0.78200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95599999999999996,"15min":1.004},"min":{"1min":0.73799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.25,"5min":1.3129999999999999,"15min":1.484},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.75900000000000001,"5min":0.68000000000000005,"15min":0.67200000000000004},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.456},"last":1.1180000000000001}]},{"osd":6,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0089999999999999,"15min":1.0389999999999999},"min":{"1min":0.88100000000000001,"5min":0.71399999999999997,"15min":0.69599999999999995},"max":{"1min":1.391,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0069999999999999,"15min":1.0580000000000001},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.244,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.0389999999999999}]},{"osd":7,"last update":"Wed May 8 22:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0489999999999999,"15min":1.069},"min":{"1min":0.81699999999999995,"5min":0.69999999999999996,"15min":0.39300000000000002},"max":{"1min":1.3169999999999999,"5min":1.498,"15min":1.5409999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0629999999999999,"15min":1.0980000000000001},"min":{"1min":0.84699999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.357,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542282,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1312550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:43:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.64500000000000002},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.617},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.33600000000000002},"max":{"1min":0.85899999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.446},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.755},"min":{"1min":0.52200000000000002,"5min":0.45100000000000001,"15min":0.436},"max":{"1min":1.044,"5min":1.101,"15min":1.6910000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 22:43:33 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95399999999999996,"5min":1.1020000000000001,"15min":1.714},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":0.438}]},{"osd":4,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.876,"15min":0.90100000000000002},"min":{"1min":0.66400000000000003,"5min":0.63700000000000001,"15min":0.61699999999999999},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":4.391},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.93200000000000005,"15min":0.96699999999999997},"min":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":2.0590000000000002},"last":0.81799999999999995}]},{"osd":5,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91200000000000003,"15min":0.90900000000000003},"min":{"1min":0.746,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1599999999999999,"5min":1.246,"15min":1.419},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95999999999999996,"15min":0.97999999999999998},"min":{"1min":0.77900000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.72899999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1120000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0109999999999999,"15min":1.04},"min":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":0.93500000000000005}]},{"osd":7,"last update":"Wed May 8 22:44:24 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0649999999999999},"min":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.6459999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0169999999999999,"15min":1.02},"min":{"1min":0.85899999999999999,"5min":0.72499999999999998,"15min":0.60399999999999998},"max":{"1min":1.3200000000000001,"5min":1.514,"15min":1.514},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:44:31.689 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:44:31.689 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T22:44:31.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 Processor -- start 2024-05-08T22:44:31.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 -- start start 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48780b8a80 0x7f48780cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f48780ce0e0 0x7f487806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780b8ee0 con 0x7f487815d0c0 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780ce570 con 0x7f48780ce0e0 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487f59b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487815d4a0 con 0x7f48780b8a80 2024-05-08T22:44:31.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487ed9a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f48780ce0e0 0x7f487806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487dd98700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48780b8a80 0x7f48780cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487ed9a700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f48780ce0e0 0x7f487806b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53832/0 (socket says 172.21.15.179:53832) 2024-05-08T22:44:31.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487e599700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f487815d0c0 0x7f48780d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:41354/0 (socket says 172.21.15.179:41354) 2024-05-08T22:44:31.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.815+0000 7f487ed9a700 1 -- 172.21.15.179:0/1490510560 learned_addr learned my addr 172.21.15.179:0/1490510560 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487ed9a700 1 -- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48780b8a80 msgr2=0x7f48780cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487ed9a700 1 --2- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48780b8a80 0x7f48780cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487ed9a700 1 -- 172.21.15.179:0/1490510560 >> v1:172.21.15.195:6789/0 conn(0x7f487815d0c0 legacy=0x7f48780d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487ed9a700 1 -- 172.21.15.179:0/1490510560 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487806abb0 con 0x7f48780ce0e0 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487ed9a700 1 --2- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780ce0e0 0x7f487806b370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e80 tx=0x7f487000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231ec214e4353ebf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487d597700 1 -- 172.21.15.179:0/1490510560 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000e070 con 0x7f48780ce0e0 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487d597700 1 -- 172.21.15.179:0/1490510560 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000b8d0 con 0x7f48780ce0e0 2024-05-08T22:44:31.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487d597700 1 -- 172.21.15.179:0/1490510560 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000bc80 con 0x7f48780ce0e0 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 -- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780ce0e0 msgr2=0x7f487806b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 --2- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780ce0e0 0x7f487806b370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e80 tx=0x7f487000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 -- 172.21.15.179:0/1490510560 shutdown_connections 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 --2- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780ce0e0 0x7f487806b370 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 --2- 172.21.15.179:0/1490510560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48780b8a80 0x7f48780cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 -- 172.21.15.179:0/1490510560 >> 172.21.15.179:0/1490510560 conn(0x7f48780d2ed0 msgr2=0x7f48780d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:31.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 -- 172.21.15.179:0/1490510560 shutdown_connections 2024-05-08T22:44:31.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.819+0000 7f487f59b700 1 -- 172.21.15.179:0/1490510560 wait complete. 2024-05-08T22:44:31.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 Processor -- start 2024-05-08T22:44:31.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 -- start start 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 0x7f48781577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f487815d0c0 0x7f48781551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780c8060 con 0x7f48780ce0e0 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4878156500 con 0x7f48780b8a80 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487f59b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4878159e90 con 0x7f487815d0c0 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487e599700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 0x7f48781577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487e599700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53846/0 (socket says 172.21.15.179:53846) 2024-05-08T22:44:31.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487e599700 1 -- 172.21.15.179:0/838093020 learned_addr learned my addr 172.21.15.179:0/838093020 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487ed9a700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f487815d0c0 0x7f48781551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 -- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f487815d0c0 msgr2=0x7f48781551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f487815d0c0 0x7f48781551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 -- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 msgr2=0x7f48780c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 -- 172.21.15.179:0/838093020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009b30 con 0x7f48780ce0e0 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487e599700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:31.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.823+0000 7f487dd98700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 0x7f48781577a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f486800e900 tx=0x7f486800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868013680 con 0x7f48780ce0e0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48780c7a10 con 0x7f48780ce0e0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48680137e0 con 0x7f48780ce0e0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868013b50 con 0x7f48780ce0e0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4878157ce0 con 0x7f48780ce0e0 2024-05-08T22:44:31.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487804e700 con 0x7f48780ce0e0 2024-05-08T22:44:31.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4868013cb0 con 0x7f48780ce0e0 2024-05-08T22:44:31.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 0x7f485c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:31.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f48680841c0 con 0x7f48780ce0e0 2024-05-08T22:44:31.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.827+0000 7f487e599700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 0x7f485c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:31.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.831+0000 7f487e599700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 0x7f485c068bf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4874000f60 tx=0x7f48740010f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:31.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:31.835+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4868051400 con 0x7f48780ce0e0 2024-05-08T22:44:32.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.067+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f48780676e0 con 0x7f48780ce0e0 2024-05-08T22:44:32.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f486effd700 1 -- 172.21.15.179:0/838093020 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4868054a20 con 0x7f48780ce0e0 2024-05-08T22:44:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 msgr2=0x7f485c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 0x7f485c068bf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4874000f60 tx=0x7f48740010f0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 msgr2=0x7f48781577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 0x7f48781577a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f486800e900 tx=0x7f486800ec10 comp rx=0 tx=0).stop 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 shutdown_connections 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f485c066730 0x7f485c068bf0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f487815d0c0 0x7f48781551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48780ce0e0 0x7f48781577a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 --2- 172.21.15.179:0/838093020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48780b8a80 0x7f48780c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.075+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 >> 172.21.15.179:0/838093020 conn(0x7f48780d2ed0 msgr2=0x7f48780b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.079+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 shutdown_connections 2024-05-08T22:44:32.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.079+0000 7f487f59b700 1 -- 172.21.15.179:0/838093020 wait complete. 2024-05-08T22:44:32.080 INFO:teuthology.orchestra.run.smithi179.stderr:updated caps for client.0 2024-05-08T22:44:32.093 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:32.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 Processor -- start 2024-05-08T22:44:32.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 -- start start 2024-05-08T22:44:32.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f83d0143610 0x7f83d00cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00ce390 con 0x7f83d006d820 2024-05-08T22:44:32.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00ce670 con 0x7f83d00cdf10 2024-05-08T22:44:32.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83d4a57700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d00ce950 con 0x7f83d0143610 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83ceffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f83d006d820 0x7f83d00cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54166/0 (socket says 172.21.15.179:54166) 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83ceffd700 1 -- 172.21.15.179:0/253903210 learned_addr learned my addr 172.21.15.179:0/253903210 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.203+0000 7f83cf7fe700 1 --2- 172.21.15.179:0/253903210 >> v2:172.21.15.195:3300/0 conn(0x7f83d0143610 0x7f83d00cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83ce7fc700 1 -- 172.21.15.179:0/253903210 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963637113 0 0) 0x7f83d00ce390 con 0x7f83d006d820 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83ce7fc700 1 -- 172.21.15.179:0/253903210 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83bc0035d0 con 0x7f83d006d820 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83cf7fe700 1 -- 172.21.15.179:0/253903210 >> v1:172.21.15.195:6789/0 conn(0x7f83d00cdf10 legacy=0x7f83d00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83cf7fe700 1 -- 172.21.15.179:0/253903210 >> v1:172.21.15.179:6789/0 conn(0x7f83d006d820 legacy=0x7f83d00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83cf7fe700 1 -- 172.21.15.179:0/253903210 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00cec30 con 0x7f83d0143610 2024-05-08T22:44:32.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83cf7fe700 1 --2- 172.21.15.179:0/253903210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d0143610 0x7f83d00cd9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009850 tx=0x7f83c8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b99b9b58479e9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:32.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83ce7fc700 1 -- 172.21.15.179:0/253903210 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011830 con 0x7f83d0143610 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83ce7fc700 1 -- 172.21.15.179:0/253903210 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c8011990 con 0x7f83d0143610 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83ce7fc700 1 -- 172.21.15.179:0/253903210 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011d40 con 0x7f83d0143610 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 -- 172.21.15.179:0/253903210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d0143610 msgr2=0x7f83d00cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 --2- 172.21.15.179:0/253903210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d0143610 0x7f83d00cd9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009850 tx=0x7f83c8009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 -- 172.21.15.179:0/253903210 shutdown_connections 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 --2- 172.21.15.179:0/253903210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d0143610 0x7f83d00cd9d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 -- 172.21.15.179:0/253903210 >> 172.21.15.179:0/253903210 conn(0x7f83d00cfad0 msgr2=0x7f83d00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 -- 172.21.15.179:0/253903210 shutdown_connections 2024-05-08T22:44:32.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.207+0000 7f83d4a57700 1 -- 172.21.15.179:0/253903210 wait complete. 2024-05-08T22:44:32.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 Processor -- start 2024-05-08T22:44:32.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- start start 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 0x7f83d0139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 0x7f83d01f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01f6720 con 0x7f83d0152b00 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0151d20 con 0x7f83d006d820 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0152000 con 0x7f83d00cdf10 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:53860/0 (socket says 172.21.15.179:53860) 2024-05-08T22:44:32.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 -- 172.21.15.179:0/1213711453 learned_addr learned my addr 172.21.15.179:0/1213711453 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:32.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cffff700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 0x7f83d01f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83ceffd700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 0x7f83d0139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 -- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 msgr2=0x7f83d0139520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 0x7f83d0139520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 -- 172.21.15.179:0/1213711453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 msgr2=0x7f83d01f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 0x7f83d01f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 -- 172.21.15.179:0/1213711453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80094b0 con 0x7f83d006d820 2024-05-08T22:44:32.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cffff700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 0x7f83d01f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83ceffd700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 0x7f83d0139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83cf7fe700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c00 tx=0x7f83c8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c801a950 con 0x7f83d006d820 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c801f070 con 0x7f83d006d820 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d01522e0 con 0x7f83d006d820 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c801acb0 con 0x7f83d006d820 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.211+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d0139a60 con 0x7f83d006d820 2024-05-08T22:44:32.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d004e700 con 0x7f83d006d820 2024-05-08T22:44:32.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83c802e450 con 0x7f83d006d820 2024-05-08T22:44:32.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83c7fff700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 0x7f83b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f83c80852a0 con 0x7f83d006d820 2024-05-08T22:44:32.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83ceffd700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 0x7f83b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.215+0000 7f83ceffd700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 0x7f83b0068bf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f83d014f6e0 tx=0x7f83b8001a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:32.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.227+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f83c8052440 con 0x7f83d006d820 2024-05-08T22:44:32.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.435+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83d00676e0 con 0x7f83d006d820 2024-05-08T22:44:32.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83c7fff700 1 -- 172.21.15.179:0/1213711453 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4751 (secure 0 0 0) 0x7f83c8055a60 con 0x7f83d006d820 2024-05-08T22:44:32.439 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:32.439 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":90,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:30.233148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57983},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57983":{"gid":57983,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/2184096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2184096099},{"type":"v1","addr":"172.21.15.195:6835","nonce":2184096099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:44:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 msgr2=0x7f83b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 0x7f83b0068bf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f83d014f6e0 tx=0x7f83b8001a90 comp rx=0 tx=0).stop 2024-05-08T22:44:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 msgr2=0x7f83d01536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c00 tx=0x7f83c8004770 comp rx=0 tx=0).stop 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 shutdown_connections 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f83b0066730 0x7f83b0068bf0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83d0152b00 0x7f83d01f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83d00cdf10 0x7f83d0139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 --2- 172.21.15.179:0/1213711453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83d006d820 0x7f83d01536d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 >> 172.21.15.179:0/1213711453 conn(0x7f83d00cfad0 msgr2=0x7f83d015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 shutdown_connections 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.439+0000 7f83d4a57700 1 -- 172.21.15.179:0/1213711453 wait complete. 2024-05-08T22:44:32.443 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 90 2024-05-08T22:44:32.857 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:44:30.233148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57983}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57983': {'gid': 57983, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6835/2184096099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2184096099}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 2184096099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:44:32.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:44:32.858 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:44:32.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.983+0000 7f602dec4700 1 Processor -- start 2024-05-08T22:44:32.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.983+0000 7f602dec4700 1 -- start start 2024-05-08T22:44:32.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815aec0 0x7f60280b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f602815e010 0x7f602806fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280b3000 con 0x7f602806f640 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6028070000 con 0x7f602815e010 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602815e8d0 con 0x7f602815aec0 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f602815e010 0x7f602806fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f602815e010 0x7f602806fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52038/0 (socket says 172.21.15.179:52038) 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 -- 172.21.15.179:0/638357372 learned_addr learned my addr 172.21.15.179:0/638357372 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f601ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f602806f640 0x7f602815d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54170/0 (socket says 172.21.15.179:54170) 2024-05-08T22:44:32.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602cec2700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815aec0 0x7f60280b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 -- 172.21.15.179:0/638357372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815aec0 msgr2=0x7f60280b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815aec0 0x7f60280b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 -- 172.21.15.179:0/638357372 >> v1:172.21.15.179:6789/0 conn(0x7f602806f640 legacy=0x7f602815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f601f7fe700 1 -- 172.21.15.179:0/638357372 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961993104 0 0) 0x7f60280b3000 con 0x7f602806f640 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 -- 172.21.15.179:0/638357372 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602806d590 con 0x7f602815e010 2024-05-08T22:44:32.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602d6c3700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602815e010 0x7f602806fac0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e50 tx=0x7f602000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff699fa6e172821 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:32.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f601f7fe700 1 -- 172.21.15.179:0/638357372 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000e070 con 0x7f602815e010 2024-05-08T22:44:32.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f601f7fe700 1 -- 172.21.15.179:0/638357372 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000b910 con 0x7f602815e010 2024-05-08T22:44:32.990 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f601f7fe700 1 -- 172.21.15.179:0/638357372 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000bcc0 con 0x7f602815e010 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- 172.21.15.179:0/638357372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602815e010 msgr2=0x7f602806fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602815e010 0x7f602806fac0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6020009e50 tx=0x7f602000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- 172.21.15.179:0/638357372 shutdown_connections 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602815e010 0x7f602806fac0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 --2- 172.21.15.179:0/638357372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815aec0 0x7f60280b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- 172.21.15.179:0/638357372 >> 172.21.15.179:0/638357372 conn(0x7f60280cfb50 msgr2=0x7f60280cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- 172.21.15.179:0/638357372 shutdown_connections 2024-05-08T22:44:32.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.987+0000 7f602dec4700 1 -- 172.21.15.179:0/638357372 wait complete. 2024-05-08T22:44:32.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 Processor -- start 2024-05-08T22:44:32.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 -- start start 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 0x7f602814f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815e010 0x7f602806e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028151240 con 0x7f602806f640 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602813fb00 con 0x7f602815aec0 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602dec4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602814f940 con 0x7f602815e010 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52042/0 (socket says 172.21.15.179:52042) 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 -- 172.21.15.179:0/931956666 learned_addr learned my addr 172.21.15.179:0/931956666 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:32.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602d6c3700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815e010 0x7f602806e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f601ffff700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 0x7f602814f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 -- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815e010 msgr2=0x7f602806e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815e010 0x7f602806e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 -- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 msgr2=0x7f602814f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 0x7f602814f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f602cec2700 1 -- 172.21.15.179:0/931956666 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009b30 con 0x7f602806f640 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.991+0000 7f601ffff700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 0x7f602814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f602cec2700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9d0 tx=0x7f602400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 -- 172.21.15.179:0/931956666 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024009720 con 0x7f602806f640 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 -- 172.21.15.179:0/931956666 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602400c850 con 0x7f602806f640 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602815bdd0 con 0x7f602806f640 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 -- 172.21.15.179:0/931956666 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602400cbc0 con 0x7f602806f640 2024-05-08T22:44:32.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602815c390 con 0x7f602806f640 2024-05-08T22:44:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 -- 172.21.15.179:0/931956666 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f602400cd20 con 0x7f602806f640 2024-05-08T22:44:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 0x7f6014068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:32.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601d7fa700 1 -- 172.21.15.179:0/931956666 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6024084430 con 0x7f602806f640 2024-05-08T22:44:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.995+0000 7f601ffff700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 0x7f6014068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:32.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f601ffff700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 0x7f6014068bf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f602814e450 tx=0x7f6018009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 Processor -- start 2024-05-08T22:44:33.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 -- start start 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x2951ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2919470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2878d80 con 0x2952420 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2876510 con 0x2948be0 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2871e00 con 0x2951b00 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f602d6c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f602d6c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52050/0 (socket says 172.21.15.179:52050) 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f602d6c3700 1 -- 172.21.15.179:0/1931140479 learned_addr learned my addr 172.21.15.179:0/1931140479 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:33.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f601ffff700 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2919470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:32.999+0000 7f602cec2700 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x2951ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 -- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 msgr2=0x2951ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x2951ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 -- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 msgr2=0x2919470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2919470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 -- 172.21.15.179:0/1931140479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26e4960 con 0x2952420 2024-05-08T22:44:33.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f602d6c3700 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f60280cd5e0 tx=0x7f6020010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d2d2a5babd981d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f600e7fc700 1 -- 172.21.15.179:0/1931140479 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020003a40 con 0x2952420 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f600e7fc700 1 -- 172.21.15.179:0/1931140479 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020003ba0 con 0x2952420 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f600e7fc700 1 -- 172.21.15.179:0/1931140479 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004040 con 0x2952420 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- 172.21.15.179:0/1931140479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 msgr2=0x29199b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f60280cd5e0 tx=0x7f6020010000 comp rx=0 tx=0).stop 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- 172.21.15.179:0/1931140479 shutdown_connections 2024-05-08T22:44:33.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2919470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29199b0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 --2- 172.21.15.179:0/1931140479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x2951ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- 172.21.15.179:0/1931140479 >> 172.21.15.179:0/1931140479 conn(0x26994d0 msgr2=0x2933250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- 172.21.15.179:0/1931140479 shutdown_connections 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- 172.21.15.179:0/1931140479 wait complete. 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 -- start start 2024-05-08T22:44:33.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.003+0000 7f60323d5740 1 Processor -- start 2024-05-08T22:44:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2948380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x282f730 con 0x2951b00 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x28355b0 con 0x2948be0 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x271daa0 con 0x2952420 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 10 client.0 _ll_register_callbacks cb 0x286b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52058/0 (socket says 172.21.15.179:52058) 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 -- 172.21.15.179:0/703204596 learned_addr learned my addr 172.21.15.179:0/703204596 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f602cec2700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2948380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f602d6c3700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 -- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 msgr2=0x2909eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 -- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 msgr2=0x2948380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2948380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 -- 172.21.15.179:0/703204596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602000be70 con 0x2951b00 2024-05-08T22:44:33.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f601ffff700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f60180073e0 tx=0x7f6018001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d075666456d330 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800dcf0 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f601800de50 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26a61e0 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6018004890 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 10 client.9744 fetch_fsmap learned FSMap version 90 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x2838220 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26a61e0 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 90) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f601801f070 con 0x2951b00 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 10 client.9744 fetch_fsmap finished waiting for FSMap version 90 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 10 client.9744 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 20 client.9744 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x294a630 0x29652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 4 client.9744 mds_command: new command op to 57983 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:44:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.007+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 --> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26f0460 con 0x294a630 2024-05-08T22:44:33.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f602cec2700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x294a630 0x29652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f602cec2700 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x294a630 0x29652a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09f1c25f9695df0 server_cookie=1970b87788fad4ca in_seq=0 out_seq=0 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60077fe700 10 client.9744 ms_handle_connect on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60077fe700 1 -- 172.21.15.179:0/703204596 <== mds.0 v2:172.21.15.195:6834/2184096099 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x26f0460 con 0x294a630 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60077fe700 10 client.9744 handle_command_reply: tid=0 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 client.9744 shutdown 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 msgr2=0x2909970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f60180073e0 tx=0x7f6018001080 comp rx=0 tx=0).stop 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 shutdown_connections 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x294a630 0x29652a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60077fe700 0 client.9744 ms_handle_reset on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2951b00 0x2909970 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 --2- 172.21.15.179:0/703204596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x2948380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 >> 172.21.15.179:0/703204596 conn(0x26994d0 msgr2=0x2932a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 shutdown_connections 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 1 -- 172.21.15.179:0/703204596 wait complete. 2024-05-08T22:44:33.015 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.011+0000 7f60323d5740 20 client.9744 trim_cache size 0 max 16384 2024-05-08T22:44:33.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 Processor -- start 2024-05-08T22:44:33.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 -- start start 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2951b00 0x2909730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2897f50 con 0x2948be0 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2901320 con 0x2951b00 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2833e90 con 0x2952420 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52064/0 (socket says 172.21.15.179:52064) 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 -- 172.21.15.179:0/3283846696 learned_addr learned my addr 172.21.15.179:0/3283846696 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:33.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602d6c3700 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f601ffff700 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2951b00 0x2909730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 -- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 msgr2=0x2909c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 -- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2951b00 msgr2=0x2909730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2951b00 0x2909730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 -- 172.21.15.179:0/3283846696 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x281e450 con 0x2948be0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f602cec2700 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6024051820 tx=0x7f6024084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891e710436a40f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f600effd700 1 -- 172.21.15.179:0/3283846696 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024082070 con 0x2948be0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f600effd700 1 -- 172.21.15.179:0/3283846696 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024082930 con 0x2948be0 2024-05-08T22:44:33.035 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.031+0000 7f600effd700 1 -- 172.21.15.179:0/3283846696 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240536c0 con 0x2948be0 2024-05-08T22:44:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/3283846696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 msgr2=0x292fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6024051820 tx=0x7f6024084040 comp rx=0 tx=0).stop 2024-05-08T22:44:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/3283846696 shutdown_connections 2024-05-08T22:44:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2951b00 0x2909730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.039 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2952420 0x2909c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- 172.21.15.179:0/3283846696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2948be0 0x292fd80 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/3283846696 >> 172.21.15.179:0/3283846696 conn(0x26994d0 msgr2=0x292f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/3283846696 shutdown_connections 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/3283846696 wait complete. 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- start start 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 Processor -- start 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x291c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x29b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x29b6ea0 con 0x2952420 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x290ba40 con 0x2948be0 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2950cc0 con 0x2951b00 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 10 client.0 _ll_register_callbacks cb 0x7f6020009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52066/0 (socket says 172.21.15.179:52066) 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 -- 172.21.15.179:0/2559158033 learned_addr learned my addr 172.21.15.179:0/2559158033 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602cec2700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x291c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f601ffff700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x29b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 -- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 msgr2=0x29b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x29b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 -- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 msgr2=0x291c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x291c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 -- 172.21.15.179:0/2559158033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602406a040 con 0x2952420 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602cec2700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x291c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f602d6c3700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x2835800 tx=0x7f602000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940f8baec0fb5101 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60077fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020029040 con 0x2952420 2024-05-08T22:44:33.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f602001cab0 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6020003ba0 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f602001cc70 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 10 client.9750 fetch_fsmap learned FSMap version 90 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.035+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x2812720 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020003ba0 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 90) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6020022030 con 0x2952420 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 10 client.9750 fetch_fsmap finished waiting for FSMap version 90 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 10 client.9750 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 20 client.9750 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x2907950 0x2907d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 4 client.9750 mds_command: new command op to 57983 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 --> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] -- command(tid 0: {"prefix": "status"}) v1 -- 0x27151e0 con 0x2907950 2024-05-08T22:44:33.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f602cec2700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x2907950 0x2907d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 10 client.9750 ms_handle_connect on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f602cec2700 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x2907950 0x2907d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b6f0cf461c81cd server_cookie=bba13889bc8bc38b in_seq=0 out_seq=0 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 1 -- 172.21.15.179:0/2559158033 <== mds.0 v2:172.21.15.195:6834/2184096099 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x27151e0 con 0x2907950 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 10 client.9750 handle_command_reply: tid=0 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 client.9750 shutdown 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 msgr2=0x29b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x2835800 tx=0x7f602000e000 comp rx=0 tx=0).stop 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 shutdown_connections 2024-05-08T22:44:33.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x2907950 0x2907d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2952420 0x29b6960 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60077fe700 0 client.9750 ms_handle_reset on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2951b00 0x29b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 --2- 172.21.15.179:0/2559158033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2948be0 0x291c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 >> 172.21.15.179:0/2559158033 conn(0x26994d0 msgr2=0x292fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 shutdown_connections 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 1 -- 172.21.15.179:0/2559158033 wait complete. 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f60323d5740 20 client.9750 trim_cache size 0 max 16384 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:33.045 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 57983, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 5.8203702899999996, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 90, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 127, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 127, 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 10.252959964 2024-05-08T22:44:33.046 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.039+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 msgr2=0x7f6014068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 0x7f6014068bf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f602814e450 tx=0x7f6018009040 comp rx=0 tx=0).stop 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 msgr2=0x7f602813f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9d0 tx=0x7f602400ed90 comp rx=0 tx=0).stop 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 shutdown_connections 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6014066730 0x7f6014068bf0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f602815aec0 0x7f602814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f602815e010 0x7f602806e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 --2- 172.21.15.179:0/931956666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f602806f640 0x7f602813f5c0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 >> 172.21.15.179:0/931956666 conn(0x7f60280cfb50 msgr2=0x7f60280cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 shutdown_connections 2024-05-08T22:44:33.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.043+0000 7f602dec4700 1 -- 172.21.15.179:0/931956666 wait complete. 2024-05-08T22:44:33.060 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:44:33.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 Processor -- start 2024-05-08T22:44:33.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 -- start start 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f441c161790 0x7f441c164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c165d50 con 0x7f441c15ef80 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c06ad70 con 0x7f441c06de70 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421a34700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c06e4b0 con 0x7f441c161790 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421233700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f441c161790 0x7f441c164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4420a32700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f441c15ef80 0x7f441c06d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:54184/0 (socket says 172.21.15.179:54184) 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4420a32700 1 -- 172.21.15.179:0/2266109426 learned_addr learned my addr 172.21.15.179:0/2266109426 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421233700 1 -- 172.21.15.179:0/2266109426 >> v1:172.21.15.195:6789/0 conn(0x7f441c06de70 legacy=0x7f441c06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421233700 1 -- 172.21.15.179:0/2266109426 >> v1:172.21.15.179:6789/0 conn(0x7f441c15ef80 legacy=0x7f441c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421233700 1 -- 172.21.15.179:0/2266109426 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c1654b0 con 0x7f441c161790 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f4421233700 1 --2- 172.21.15.179:0/2266109426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c161790 0x7f441c164db0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4414009850 tx=0x7f4414009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec23cca094130c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f441b7fe700 1 -- 172.21.15.179:0/2266109426 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011830 con 0x7f441c161790 2024-05-08T22:44:33.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f441b7fe700 1 -- 172.21.15.179:0/2266109426 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414011990 con 0x7f441c161790 2024-05-08T22:44:33.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.175+0000 7f441b7fe700 1 -- 172.21.15.179:0/2266109426 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011d20 con 0x7f441c161790 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- 172.21.15.179:0/2266109426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c161790 msgr2=0x7f441c164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 --2- 172.21.15.179:0/2266109426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c161790 0x7f441c164db0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4414009850 tx=0x7f4414009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- 172.21.15.179:0/2266109426 shutdown_connections 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 --2- 172.21.15.179:0/2266109426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c161790 0x7f441c164db0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- 172.21.15.179:0/2266109426 >> 172.21.15.179:0/2266109426 conn(0x7f441c0ce9d0 msgr2=0x7f441c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- 172.21.15.179:0/2266109426 shutdown_connections 2024-05-08T22:44:33.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- 172.21.15.179:0/2266109426 wait complete. 2024-05-08T22:44:33.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 Processor -- start 2024-05-08T22:44:33.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- start start 2024-05-08T22:44:33.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c06de70 0x7f441c14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f441c161790 0x7f441c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1585c0 con 0x7f441c15ef80 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c158810 con 0x7f441c06de70 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421a34700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c158af0 con 0x7f441c161790 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4421233700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f441c161790 0x7f441c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52098/0 (socket says 172.21.15.179:52098) 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 -- 172.21.15.179:0/3115399982 learned_addr learned my addr 172.21.15.179:0/3115399982 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f4420a32700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c06de70 0x7f441c14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 -- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f441c161790 msgr2=0x7f441c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f441c161790 0x7f441c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 -- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c06de70 msgr2=0x7f441c14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c06de70 0x7f441c14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.179+0000 7f441bfff700 1 -- 172.21.15.179:0/3115399982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140094b0 con 0x7f441c15ef80 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f441bfff700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f440c001ed0 tx=0x7f440c0021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c003c20 con 0x7f441c15ef80 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c003d80 con 0x7f441c15ef80 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c158dd0 con 0x7f441c15ef80 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c00ab40 con 0x7f441c15ef80 2024-05-08T22:44:33.184 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c14b030 con 0x7f441c15ef80 2024-05-08T22:44:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c067740 con 0x7f441c15ef80 2024-05-08T22:44:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f440c00aca0 con 0x7f441c15ef80 2024-05-08T22:44:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 0x7f4400068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:33.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f440c084390 con 0x7f441c15ef80 2024-05-08T22:44:33.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.183+0000 7f4420a32700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 0x7f4400068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:33.186 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.187+0000 7f4420a32700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 0x7f4400068bf0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4404000f60 tx=0x7f4404007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:33.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.191+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f440c051520 con 0x7f441c15ef80 2024-05-08T22:44:33.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.395+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f441c06b680 con 0x7f441c15ef80 2024-05-08T22:44:33.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.395+0000 7f44197fa700 1 -- 172.21.15.179:0/3115399982 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f440c054b40 con 0x7f441c15ef80 2024-05-08T22:44:33.395 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 msgr2=0x7f4400068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 0x7f4400068bf0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4404000f60 tx=0x7f4404007f30 comp rx=0 tx=0).stop 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 msgr2=0x7f441c0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f440c001ed0 tx=0x7f440c0021e0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 shutdown_connections 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4400066730 0x7f4400068bf0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f441c161790 0x7f441c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f441c15ef80 0x7f441c0ccc00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 --2- 172.21.15.179:0/3115399982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c06de70 0x7f441c14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:33.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 >> 172.21.15.179:0/3115399982 conn(0x7f441c0ce9d0 msgr2=0x7f441c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:33.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 shutdown_connections 2024-05-08T22:44:33.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:33.399+0000 7f4421a34700 1 -- 172.21.15.179:0/3115399982 wait complete. 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:33.414 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:44:33.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:44:33.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:44:33.415 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:33.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.415 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:44:33.421 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:44:33.421 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:44:33.421 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:44:33.421 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.421 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41898sec preferred_lft 41898sec 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.422 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:33.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:44:33.424 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:44:33.424 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:33.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.424 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:44:33.477 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:44:33.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.481 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:44:33.491 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:44:33.492 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:44:33.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:44:33.493 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:44:33.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.535 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:44:33.544 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:44:33.544 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:44:33.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:33.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.545 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:44:33.606 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:44:33.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:33.606 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:33.613 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:33.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:33.614 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:33.661 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:33.680 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:33.680 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:33.697 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:33.697 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:33.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.698 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:33.750 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:44:33.750 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:44:33.753 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:33.772 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:33.772 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:33.830 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:33.831 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:33.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:33.831 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:33.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:44:33.835+0000 7f21a5d32640 -1 init, newargv = 0x5555bdafd060 newargc=15 2024-05-08T22:44:33.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52326]: starting ceph client 2024-05-08T22:44:33.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52326]: starting fuse 2024-05-08T22:44:34.939 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:34.959 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:34.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:35.016 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:35.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:35.017 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:35.024 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:44:35.025 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:44:35.025 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:35.026 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:44:35.027 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:35.108 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.52326.asok 2024-05-08T22:44:35.291 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52326.asok 2024-05-08T22:44:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:35.292 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52326.asok status 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:44:35.444 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "52326", 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9762, 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 9762 2024-05-08T22:44:35.445 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1137120191 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1137120191 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:35.446 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.9762 192.168.0.1:0/1137120191", 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/1137120191", 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 90, 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 127, 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:44:35.447 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:35.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:35.454 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:44:35.454 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:35.460 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:44:35.460 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:35.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:35.460 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:35.477 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> parent = os.path.join(mount_point, "parent") 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> os.mkdir(parent) 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> flushed = os.path.join(parent, "flushed") 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> os.mkdir(flushed) 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> for f in ["alpha", "bravo", "charlie"]: 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-08T22:44:35.478 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:35.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:35.735 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T22:44:35.811 INFO:teuthology.orchestra.run.smithi179.stdout:./ 2024-05-08T22:44:35.811 INFO:teuthology.orchestra.run.smithi179.stdout:./parent 2024-05-08T22:44:35.811 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed 2024-05-08T22:44:35.811 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/charlie 2024-05-08T22:44:35.812 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/alpha 2024-05-08T22:44:35.812 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/bravo 2024-05-08T22:44:35.812 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:35.813 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:35.813 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:35.813 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:35.813 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T22:44:35.813 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:35.882 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T22:44:36.070 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:36.138 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:44:36.329 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:36.329 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:36.330 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:36.330 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:36.330 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T22:44:36.330 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:36.439 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:44:36.626 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:36.627 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:36.627 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:36.627 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:36.627 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T22:44:36.627 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:36.696 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628280 2024-05-08T22:44:36.885 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:36.886 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:36.886 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:36.886 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:36.886 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T22:44:36.886 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:36.955 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627778 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-08T22:44:37.144 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:37.255 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628279 2024-05-08T22:44:37.443 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:37.444 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:37.452 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:37.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 Processor -- start 2024-05-08T22:44:37.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- start start 2024-05-08T22:44:37.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 0x7fbe7415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe740cab60 0x7fbe740cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7406eb20 con 0x7fbe7406ac90 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7415e880 con 0x7fbe740cab60 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7406df10 con 0x7fbe7415df60 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe740cab60 0x7fbe740cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe737fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbe7406ac90 0x7fbe740ca450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42556/0 (socket says 172.21.15.179:42556) 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe737fe700 1 -- 172.21.15.179:0/2340338136 learned_addr learned my addr 172.21.15.179:0/2340338136 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe73fff700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 0x7fbe7415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 -- 172.21.15.179:0/2340338136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 msgr2=0x7fbe7415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 0x7fbe7415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 -- 172.21.15.179:0/2340338136 >> v1:172.21.15.179:6789/0 conn(0x7fbe7406ac90 legacy=0x7fbe740ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 -- 172.21.15.179:0/2340338136 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7406e1f0 con 0x7fbe740cab60 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe72ffd700 1 -- 172.21.15.179:0/2340338136 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271382288 0 0) 0x7fbe7406eb20 con 0x7fbe7406ac90 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe73fff700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 0x7fbe7415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe78b73700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe740cab60 0x7fbe740cc7b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009d90 tx=0x7fbe6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992359b55b223797 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe72ffd700 1 -- 172.21.15.179:0/2340338136 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c00e070 con 0x7fbe740cab60 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe72ffd700 1 -- 172.21.15.179:0/2340338136 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c00bbd0 con 0x7fbe740cab60 2024-05-08T22:44:37.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe72ffd700 1 -- 172.21.15.179:0/2340338136 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c012430 con 0x7fbe740cab60 2024-05-08T22:44:37.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/2340338136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe740cab60 msgr2=0x7fbe740cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe740cab60 0x7fbe740cc7b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009d90 tx=0x7fbe6c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:37.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/2340338136 shutdown_connections 2024-05-08T22:44:37.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe740cab60 0x7fbe740cc7b0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/2340338136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7415df60 0x7fbe7415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/2340338136 >> 172.21.15.179:0/2340338136 conn(0x7fbe740cb660 msgr2=0x7fbe740cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:37.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/2340338136 shutdown_connections 2024-05-08T22:44:37.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.571+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/2340338136 wait complete. 2024-05-08T22:44:37.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 Processor -- start 2024-05-08T22:44:37.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- start start 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 0x7fbe740b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe740cab60 0x7fbe740b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740c5230 con 0x7fbe7415df60 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b3280 con 0x7fbe740cab60 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b40c0 con 0x7fbe7406ac90 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56728/0 (socket says 172.21.15.179:56728) 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe73fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 0x7fbe740b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 -- 172.21.15.179:0/66115708 learned_addr learned my addr 172.21.15.179:0/66115708 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe737fe700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe740cab60 0x7fbe740b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 -- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 msgr2=0x7fbe740b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 0x7fbe740b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 -- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe740cab60 msgr2=0x7fbe740b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe740cab60 0x7fbe740b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 -- 172.21.15.179:0/66115708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c009a70 con 0x7fbe7415df60 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe73fff700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 0x7fbe740b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe78b73700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbe74067e10 tx=0x7fbe6c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c012430 con 0x7fbe7415df60 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c020070 con 0x7fbe7415df60 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c004280 con 0x7fbe7415df60 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe740b34b0 con 0x7fbe7415df60 2024-05-08T22:44:37.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.575+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe740b3a70 con 0x7fbe7415df60 2024-05-08T22:44:37.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.579+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe74067770 con 0x7fbe7415df60 2024-05-08T22:44:37.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.579+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe6c0043e0 con 0x7fbe7415df60 2024-05-08T22:44:37.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.579+0000 7fbe70ff9700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 0x7fbe54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.579+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbe6c022070 con 0x7fbe7415df60 2024-05-08T22:44:37.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.579+0000 7fbe73fff700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 0x7fbe54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.583+0000 7fbe73fff700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 0x7fbe54068bf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400af30 tx=0x7fbe64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.587+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbe6c053260 con 0x7fbe7415df60 2024-05-08T22:44:37.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.767+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe740cea50 con 0x7fbe7415df60 2024-05-08T22:44:37.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.767+0000 7fbe70ff9700 1 -- 172.21.15.179:0/66115708 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16364 (secure 0 0 0) 0x7fbe6c056880 con 0x7fbe7415df60 2024-05-08T22:44:37.769 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:37.769 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":127,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:28.218971+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 msgr2=0x7fbe54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 0x7fbe54068bf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400af30 tx=0x7fbe64008040 comp rx=0 tx=0).stop 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 msgr2=0x7fbe7415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbe74067e10 tx=0x7fbe6c004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 shutdown_connections 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fbe54066730 0x7fbe54068bf0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe740cab60 0x7fbe740b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe7415df60 0x7fbe7415bb90 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 --2- 172.21.15.179:0/66115708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe7406ac90 0x7fbe740b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 >> 172.21.15.179:0/66115708 conn(0x7fbe740cb660 msgr2=0x7fbe74158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 shutdown_connections 2024-05-08T22:44:37.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.771+0000 7fbe7a5d6700 1 -- 172.21.15.179:0/66115708 wait complete. 2024-05-08T22:44:37.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:37.789 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:37.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.907+0000 7f3e2c84c700 1 Processor -- start 2024-05-08T22:44:37.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.907+0000 7f3e2c84c700 1 -- start start 2024-05-08T22:44:37.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e28145c70 0x7f3e28070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2806b440 con 0x7f3e280cec30 2024-05-08T22:44:37.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e280cf1a0 con 0x7f3e280b4270 2024-05-08T22:44:37.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e280cf760 con 0x7f3e28145c70 2024-05-08T22:44:37.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e26d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e28145c70 0x7f3e28070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e1e59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3e280cec30 0x7f3e280b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42570/0 (socket says 172.21.15.179:42570) 2024-05-08T22:44:37.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e1e59c700 1 -- 172.21.15.179:0/1608767678 learned_addr learned my addr 172.21.15.179:0/1608767678 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:37.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e26d9d700 1 -- 172.21.15.179:0/1608767678 >> v1:172.21.15.195:6789/0 conn(0x7f3e280b4270 legacy=0x7f3e2806be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2659c700 1 -- 172.21.15.179:0/1608767678 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063622874 0 0) 0x7f3e2806b440 con 0x7f3e280cec30 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e26d9d700 1 -- 172.21.15.179:0/1608767678 >> v1:172.21.15.179:6789/0 conn(0x7f3e280cec30 legacy=0x7f3e280b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e26d9d700 1 -- 172.21.15.179:0/1608767678 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e280cf480 con 0x7f3e28145c70 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e26d9d700 1 --2- 172.21.15.179:0/1608767678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e28145c70 0x7f3e28070180 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009850 tx=0x7f3e18009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee8be6408b7ebb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2659c700 1 -- 172.21.15.179:0/1608767678 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18011830 con 0x7f3e28145c70 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2659c700 1 -- 172.21.15.179:0/1608767678 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18011990 con 0x7f3e28145c70 2024-05-08T22:44:37.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2659c700 1 -- 172.21.15.179:0/1608767678 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18011d20 con 0x7f3e28145c70 2024-05-08T22:44:37.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- 172.21.15.179:0/1608767678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e28145c70 msgr2=0x7f3e28070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/1608767678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e28145c70 0x7f3e28070180 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009850 tx=0x7f3e18009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:37.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- 172.21.15.179:0/1608767678 shutdown_connections 2024-05-08T22:44:37.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/1608767678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e28145c70 0x7f3e28070180 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- 172.21.15.179:0/1608767678 >> 172.21.15.179:0/1608767678 conn(0x7f3e2806f5d0 msgr2=0x7f3e2806f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:37.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- 172.21.15.179:0/1608767678 shutdown_connections 2024-05-08T22:44:37.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.911+0000 7f3e2c84c700 1 -- 172.21.15.179:0/1608767678 wait complete. 2024-05-08T22:44:37.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.915+0000 7f3e2c84c700 1 Processor -- start 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.915+0000 7f3e2c84c700 1 -- start start 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e280b4270 0x7f3e2815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e28145c70 0x7f3e2815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2813a8a0 con 0x7f3e280cec30 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e28139770 con 0x7f3e280b4270 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e28139a50 con 0x7f3e28145c70 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56750/0 (socket says 172.21.15.179:56750) 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 -- 172.21.15.179:0/4067589279 learned_addr learned my addr 172.21.15.179:0/4067589279 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:37.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e26d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e280b4270 0x7f3e2815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2759e700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e28145c70 0x7f3e2815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 -- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e28145c70 msgr2=0x7f3e2815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e28145c70 0x7f3e2815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 -- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e280b4270 msgr2=0x7f3e2815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e280b4270 0x7f3e2815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 -- 172.21.15.179:0/4067589279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180094b0 con 0x7f3e280cec30 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1e59c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c4a0 tx=0x7f3e1000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10004ca0 con 0x7f3e280cec30 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10004e00 con 0x7f3e280cec30 2024-05-08T22:44:37.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e28139d30 con 0x7f3e280cec30 2024-05-08T22:44:37.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10007310 con 0x7f3e280cec30 2024-05-08T22:44:37.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2813a010 con 0x7f3e280cec30 2024-05-08T22:44:37.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.919+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2813aa10 con 0x7f3e280cec30 2024-05-08T22:44:37.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.923+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e10007470 con 0x7f3e280cec30 2024-05-08T22:44:37.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.923+0000 7f3e1ffff700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:37.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.923+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3e10083ef0 con 0x7f3e280cec30 2024-05-08T22:44:37.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.923+0000 7f3e26d9d700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:37.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.923+0000 7f3e26d9d700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 0x7f3e08068bf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009fd0 tx=0x7f3e18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:37.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:37.931+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e10051080 con 0x7f3e280cec30 2024-05-08T22:44:38.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.095+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e2804e700 con 0x7f3e280cec30 2024-05-08T22:44:38.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e1ffff700 1 -- 172.21.15.179:0/4067589279 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e10007720 con 0x7f3e280cec30 2024-05-08T22:44:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:38.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 msgr2=0x7f3e08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 0x7f3e08068bf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009fd0 tx=0x7f3e18008030 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 msgr2=0x7f3e28138d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c4a0 tx=0x7f3e1000c7b0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 shutdown_connections 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e28145c70 0x7f3e2815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e280cec30 0x7f3e28138d90 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 --2- 172.21.15.179:0/4067589279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e280b4270 0x7f3e2815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 >> 172.21.15.179:0/4067589279 conn(0x7f3e2806f5d0 msgr2=0x7f3e280afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 shutdown_connections 2024-05-08T22:44:38.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.099+0000 7f3e2c84c700 1 -- 172.21.15.179:0/4067589279 wait complete. 2024-05-08T22:44:38.113 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:38.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 Processor -- start 2024-05-08T22:44:38.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 -- start start 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f61f8145c70 0x7f61f8070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61f80b4270 0x7f61f806be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f806b440 con 0x7f61f80cec30 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80cf760 con 0x7f61f80b4270 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f7fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f8146050 con 0x7f61f8145c70 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61effff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f61f80cec30 0x7f61f80b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42584/0 (socket says 172.21.15.179:42584) 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61f80b4270 0x7f61f806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61effff700 1 -- 172.21.15.179:0/4060124197 learned_addr learned my addr 172.21.15.179:0/4060124197 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f6ffd700 1 --2- 172.21.15.179:0/4060124197 >> v2:172.21.15.195:3300/0 conn(0x7f61f8145c70 0x7f61f8070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f67fc700 1 -- 172.21.15.179:0/4060124197 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082319310 0 0) 0x7f61f806b440 con 0x7f61f80cec30 2024-05-08T22:44:38.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f67fc700 1 -- 172.21.15.179:0/4060124197 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e40035d0 con 0x7f61f80cec30 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 -- 172.21.15.179:0/4060124197 >> v2:172.21.15.195:3300/0 conn(0x7f61f8145c70 msgr2=0x7f61f8070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 --2- 172.21.15.179:0/4060124197 >> v2:172.21.15.195:3300/0 conn(0x7f61f8145c70 0x7f61f8070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 -- 172.21.15.179:0/4060124197 >> v1:172.21.15.179:6789/0 conn(0x7f61f80cec30 legacy=0x7f61f80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 -- 172.21.15.179:0/4060124197 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80cf1a0 con 0x7f61f80b4270 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f77fe700 1 --2- 172.21.15.179:0/4060124197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80b4270 0x7f61f806be90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f61f8071800 tx=0x7f61f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bf6231c0940fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f67fc700 1 -- 172.21.15.179:0/4060124197 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000e070 con 0x7f61f80b4270 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f67fc700 1 -- 172.21.15.179:0/4060124197 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f000bae0 con 0x7f61f80b4270 2024-05-08T22:44:38.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.239+0000 7f61f67fc700 1 -- 172.21.15.179:0/4060124197 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000be90 con 0x7f61f80b4270 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- 172.21.15.179:0/4060124197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80b4270 msgr2=0x7f61f806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 --2- 172.21.15.179:0/4060124197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80b4270 0x7f61f806be90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f61f8071800 tx=0x7f61f000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- 172.21.15.179:0/4060124197 shutdown_connections 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 --2- 172.21.15.179:0/4060124197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80b4270 0x7f61f806be90 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 --2- 172.21.15.179:0/4060124197 >> v2:172.21.15.195:3300/0 conn(0x7f61f8145c70 0x7f61f8070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- 172.21.15.179:0/4060124197 >> 172.21.15.179:0/4060124197 conn(0x7f61f806f5d0 msgr2=0x7f61f806f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- 172.21.15.179:0/4060124197 shutdown_connections 2024-05-08T22:44:38.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- 172.21.15.179:0/4060124197 wait complete. 2024-05-08T22:44:38.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 Processor -- start 2024-05-08T22:44:38.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.243+0000 7f61f7fff700 1 -- start start 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61f80b4270 0x7f61f8137c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61f8145c70 0x7f61f8139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81399c0 con 0x7f61f80cec30 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81381a0 con 0x7f61f8145c70 2024-05-08T22:44:38.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8138d40 con 0x7f61f80b4270 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56762/0 (socket says 172.21.15.179:56762) 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 -- 172.21.15.179:0/2155245247 learned_addr learned my addr 172.21.15.179:0/2155245247 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f77fe700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61f8145c70 0x7f61f8139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f6ffd700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61f80b4270 0x7f61f8137c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 -- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61f80b4270 msgr2=0x7f61f8137c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61f80b4270 0x7f61f8137c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 -- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61f8145c70 msgr2=0x7f61f8139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61f8145c70 0x7f61f8139de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 -- 172.21.15.179:0/2155245247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0009a70 con 0x7f61f80cec30 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61effff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c4d0 tx=0x7f61e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0005d90 con 0x7f61f80cec30 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e00070d0 con 0x7f61f80cec30 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8139020 con 0x7f61f80cec30 2024-05-08T22:44:38.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0007480 con 0x7f61f80cec30 2024-05-08T22:44:38.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f81395e0 con 0x7f61f80cec30 2024-05-08T22:44:38.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.247+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f804e700 con 0x7f61f80cec30 2024-05-08T22:44:38.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.251+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61e00075e0 con 0x7f61f80cec30 2024-05-08T22:44:38.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.251+0000 7f61ef7fe700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 0x7f61d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.251+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f61e0083c50 con 0x7f61f80cec30 2024-05-08T22:44:38.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.251+0000 7f61f6ffd700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 0x7f61d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.251+0000 7f61f6ffd700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 0x7f61d8068c20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b9d0 tx=0x7f61e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.255+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61e0050e80 con 0x7f61f80cec30 2024-05-08T22:44:38.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.423+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61f8139770 con 0x7f61f80cec30 2024-05-08T22:44:38.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61ef7fe700 1 -- 172.21.15.179:0/2155245247 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61e00544a0 con 0x7f61f80cec30 2024-05-08T22:44:38.427 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 msgr2=0x7f61d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 0x7f61d8068c20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b9d0 tx=0x7f61e8008040 comp rx=0 tx=0).stop 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 msgr2=0x7f61f8138440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c4d0 tx=0x7f61e000c890 comp rx=0 tx=0).stop 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 shutdown_connections 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f61d8066760 0x7f61d8068c20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61f8145c70 0x7f61f8139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61f80cec30 0x7f61f8138440 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 --2- 172.21.15.179:0/2155245247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61f80b4270 0x7f61f8137c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 >> 172.21.15.179:0/2155245247 conn(0x7f61f806f5d0 msgr2=0x7f61f8141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 shutdown_connections 2024-05-08T22:44:38.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.427+0000 7f61f7fff700 1 -- 172.21.15.179:0/2155245247 wait complete. 2024-05-08T22:44:38.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:38.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 Processor -- start 2024-05-08T22:44:38.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 -- start start 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 0x7f733806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73380cb2a0 0x7f733806dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733806d7e0 con 0x7f73380cd890 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73380cf8c0 con 0x7f73380cb2a0 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733e91c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806d220 con 0x7f733815a160 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733ceb9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f73380cd890 0x7f73380cdd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42586/0 (socket says 172.21.15.179:42586) 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f73377fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73380cb2a0 0x7f733806dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f733ceb9700 1 -- 172.21.15.179:0/767811201 learned_addr learned my addr 172.21.15.179:0/767811201 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f7337fff700 1 --2- 172.21.15.179:0/767811201 >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 0x7f733806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f7336ffd700 1 -- 172.21.15.179:0/767811201 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982045764 0 0) 0x7f733806d7e0 con 0x7f73380cd890 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f7336ffd700 1 -- 172.21.15.179:0/767811201 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73200035d0 con 0x7f73380cd890 2024-05-08T22:44:38.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f73377fe700 1 -- 172.21.15.179:0/767811201 >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 msgr2=0x7f733806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f73377fe700 1 --2- 172.21.15.179:0/767811201 >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 0x7f733806da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.563+0000 7f73377fe700 1 -- 172.21.15.179:0/767811201 >> v1:172.21.15.179:6789/0 conn(0x7f73380cd890 legacy=0x7f73380cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f73377fe700 1 -- 172.21.15.179:0/767811201 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733806d500 con 0x7f73380cb2a0 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f7337fff700 1 --2- 172.21.15.179:0/767811201 >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 0x7f733806da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f73377fe700 1 --2- 172.21.15.179:0/767811201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73380cb2a0 0x7f733806dfc0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f7338066540 tx=0x7f7328009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66dd22cce4f44dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f7336ffd700 1 -- 172.21.15.179:0/767811201 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328011830 con 0x7f73380cb2a0 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f7336ffd700 1 -- 172.21.15.179:0/767811201 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7328011990 con 0x7f73380cb2a0 2024-05-08T22:44:38.568 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f7336ffd700 1 -- 172.21.15.179:0/767811201 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328011d00 con 0x7f73380cb2a0 2024-05-08T22:44:38.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- 172.21.15.179:0/767811201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73380cb2a0 msgr2=0x7f733806dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 --2- 172.21.15.179:0/767811201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73380cb2a0 0x7f733806dfc0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f7338066540 tx=0x7f7328009b30 comp rx=0 tx=0).stop 2024-05-08T22:44:38.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- 172.21.15.179:0/767811201 shutdown_connections 2024-05-08T22:44:38.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 --2- 172.21.15.179:0/767811201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73380cb2a0 0x7f733806dfc0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 --2- 172.21.15.179:0/767811201 >> v2:172.21.15.195:3300/0 conn(0x7f733815a160 0x7f733806da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- 172.21.15.179:0/767811201 >> 172.21.15.179:0/767811201 conn(0x7f73380ceca0 msgr2=0x7f73380cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- 172.21.15.179:0/767811201 shutdown_connections 2024-05-08T22:44:38.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- 172.21.15.179:0/767811201 wait complete. 2024-05-08T22:44:38.570 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 Processor -- start 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 -- start start 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73380cb2a0 0x7f73381f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.567+0000 7f733e91c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73380cd890 0x7f733814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733814ecb0 con 0x7f733815a160 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381f1080 con 0x7f73380cb2a0 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73381f1360 con 0x7f73380cd890 2024-05-08T22:44:38.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56778/0 (socket says 172.21.15.179:56778) 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 -- 172.21.15.179:0/345648442 learned_addr learned my addr 172.21.15.179:0/345648442 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f73377fe700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73380cd890 0x7f733814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f7337fff700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73380cb2a0 0x7f73381f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 -- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73380cd890 msgr2=0x7f733814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73380cd890 0x7f733814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 -- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73380cb2a0 msgr2=0x7f73381f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73380cb2a0 0x7f73381f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 -- 172.21.15.179:0/345648442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280094b0 con 0x7f733815a160 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733ceb9700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f733000cb40 tx=0x7f733000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330007500 con 0x7f733815a160 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73381f1640 con 0x7f733815a160 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381f0d00 con 0x7f733815a160 2024-05-08T22:44:38.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330007660 con 0x7f733815a160 2024-05-08T22:44:38.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73300079d0 con 0x7f733815a160 2024-05-08T22:44:38.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.571+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7318005310 con 0x7f733815a160 2024-05-08T22:44:38.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.575+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7330012400 con 0x7f733815a160 2024-05-08T22:44:38.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.575+0000 7f7334ff9700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 0x7f7324068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.575+0000 7f7337fff700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 0x7f7324068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.575+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7330083bc0 con 0x7f733815a160 2024-05-08T22:44:38.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.575+0000 7f7337fff700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 0x7f7324068bf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c960 tx=0x7f732c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.583+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7330050e10 con 0x7f733815a160 2024-05-08T22:44:38.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7318005b40 con 0x7f733815a160 2024-05-08T22:44:38.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f7334ff9700 1 -- 172.21.15.179:0/345648442 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7330001c00 con 0x7f733815a160 2024-05-08T22:44:38.751 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:38.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 msgr2=0x7f7324068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 0x7f7324068bf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c960 tx=0x7f732c008040 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 msgr2=0x7f733814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f733000cb40 tx=0x7f733000cb70 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 shutdown_connections 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7324066730 0x7f7324068bf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f733815a160 0x7f733814f140 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73380cd890 0x7f733814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 --2- 172.21.15.179:0/345648442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73380cb2a0 0x7f73381f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 >> 172.21.15.179:0/345648442 conn(0x7f73380ceca0 msgr2=0x7f73380cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 shutdown_connections 2024-05-08T22:44:38.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.751+0000 7f733e91c700 1 -- 172.21.15.179:0/345648442 wait complete. 2024-05-08T22:44:38.766 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:44:38.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 Processor -- start 2024-05-08T22:44:38.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 -- start start 2024-05-08T22:44:38.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 0x7f20b80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f20b815f030 0x7f20b80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b80ce6c0 con 0x7f20b806abb0 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b80b3f20 con 0x7f20b815f030 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80b3960 con 0x7f20b8158710 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b67fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f20b806abb0 0x7f20b815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42598/0 (socket says 172.21.15.179:42598) 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f20b815f030 0x7f20b80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b67fc700 1 -- 172.21.15.179:0/35302994 learned_addr learned my addr 172.21.15.179:0/35302994 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b6ffd700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 0x7f20b80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b5ffb700 1 -- 172.21.15.179:0/35302994 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238281498 0 0) 0x7f20a8001110 con 0x7f20b806abb0 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b5ffb700 1 -- 172.21.15.179:0/35302994 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a00035d0 con 0x7f20b806abb0 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 -- 172.21.15.179:0/35302994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 msgr2=0x7f20b80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 0x7f20b80b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 -- 172.21.15.179:0/35302994 >> v1:172.21.15.179:6789/0 conn(0x7f20b806abb0 legacy=0x7f20b815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 -- 172.21.15.179:0/35302994 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80b3c40 con 0x7f20b815f030 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b6ffd700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 0x7f20b80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b77fe700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b815f030 0x7f20b80ce180 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d90 tx=0x7f20b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7b7e69a16339f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b5ffb700 1 -- 172.21.15.179:0/35302994 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000e070 con 0x7f20b815f030 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b5ffb700 1 -- 172.21.15.179:0/35302994 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b000bb20 con 0x7f20b815f030 2024-05-08T22:44:38.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b5ffb700 1 -- 172.21.15.179:0/35302994 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0012430 con 0x7f20b815f030 2024-05-08T22:44:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 -- 172.21.15.179:0/35302994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b815f030 msgr2=0x7f20b80ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.883+0000 7f20b7fff700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b815f030 0x7f20b80ce180 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d90 tx=0x7f20b000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- 172.21.15.179:0/35302994 shutdown_connections 2024-05-08T22:44:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b815f030 0x7f20b80ce180 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 --2- 172.21.15.179:0/35302994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b8158710 0x7f20b80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- 172.21.15.179:0/35302994 >> 172.21.15.179:0/35302994 conn(0x7f20b80ce960 msgr2=0x7f20b80ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- 172.21.15.179:0/35302994 shutdown_connections 2024-05-08T22:44:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- 172.21.15.179:0/35302994 wait complete. 2024-05-08T22:44:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 Processor -- start 2024-05-08T22:44:38.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- start start 2024-05-08T22:44:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806abb0 0x7f20b814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 0x7f20b81593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80ab820 con 0x7f20b8158710 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80a8d60 con 0x7f20b806abb0 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b7fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b814a7c0 con 0x7f20b815f030 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56796/0 (socket says 172.21.15.179:56796) 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b67fc700 1 -- 172.21.15.179:0/1239838560 learned_addr learned my addr 172.21.15.179:0/1239838560 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b77fe700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 0x7f20b81593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.887+0000 7f20b6ffd700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806abb0 0x7f20b814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 -- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 msgr2=0x7f20b81593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 0x7f20b81593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 -- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806abb0 msgr2=0x7f20b814a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806abb0 0x7f20b814a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 -- 172.21.15.179:0/1239838560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0009a70 con 0x7f20b8158710 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b77fe700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 0x7f20b81593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:38.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b67fc700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f20a8001270 tx=0x7f20a800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8005d90 con 0x7f20b8158710 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80070d0 con 0x7f20b8158710 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b80a81e0 con 0x7f20b8158710 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8007480 con 0x7f20b8158710 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b80a87a0 con 0x7f20b8158710 2024-05-08T22:44:38.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b80ab990 con 0x7f20b8158710 2024-05-08T22:44:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.891+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20a80075e0 con 0x7f20b8158710 2024-05-08T22:44:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.895+0000 7f209f7fe700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 0x7f20a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:38.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.895+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f20a80836b0 con 0x7f20b8158710 2024-05-08T22:44:38.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.895+0000 7f20b6ffd700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 0x7f20a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:38.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.895+0000 7f20b6ffd700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 0x7f20a4068bf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003780 tx=0x7f20ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:38.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:38.899+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f20a80508e0 con 0x7f20b8158710 2024-05-08T22:44:39.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20b80ce6c0 con 0x7f20b8158710 2024-05-08T22:44:39.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f209f7fe700 1 -- 172.21.15.179:0/1239838560 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f20b80ce6c0 con 0x7f20b8158710 2024-05-08T22:44:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 msgr2=0x7f20a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 0x7f20a4068bf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003780 tx=0x7f20ac008040 comp rx=0 tx=0).stop 2024-05-08T22:44:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 msgr2=0x7f20b8158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f20a8001270 tx=0x7f20a800ca90 comp rx=0 tx=0).stop 2024-05-08T22:44:39.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 shutdown_connections 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f20a4066730 0x7f20a4068bf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815f030 0x7f20b81593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20b8158710 0x7f20b8158ea0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 --2- 172.21.15.179:0/1239838560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20b806abb0 0x7f20b814a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 >> 172.21.15.179:0/1239838560 conn(0x7f20b80ce960 msgr2=0x7f20b80ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 shutdown_connections 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.087+0000 7f20b7fff700 1 -- 172.21.15.179:0/1239838560 wait complete. 2024-05-08T22:44:39.091 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:44:39.102 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:39.103 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:39.111 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:39.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 Processor -- start 2024-05-08T22:44:39.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- start start 2024-05-08T22:44:39.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f761806ed50 0x7f76180b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761806ddf0 con 0x7f76181434d0 2024-05-08T22:44:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76180cd200 con 0x7f76180ce2a0 2024-05-08T22:44:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180cd4e0 con 0x7f761806ed50 2024-05-08T22:44:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761cd00700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f76181434d0 0x7f761806d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42606/0 (socket says 172.21.15.179:42606) 2024-05-08T22:44:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761cd00700 1 -- 172.21.15.179:0/201741575 learned_addr learned my addr 172.21.15.179:0/201741575 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:39.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761d501700 1 --2- 172.21.15.179:0/201741575 >> v2:172.21.15.179:3300/0 conn(0x7f761806ed50 0x7f76180b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101506886 0 0) 0x7f761806ddf0 con 0x7f76181434d0 2024-05-08T22:44:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76000035d0 con 0x7f76181434d0 2024-05-08T22:44:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705339220 0 0) 0x7f76180cd200 con 0x7f76180ce2a0 2024-05-08T22:44:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761d501700 1 -- 172.21.15.179:0/201741575 >> v1:172.21.15.195:6789/0 conn(0x7f76180ce2a0 legacy=0x7f761806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761d501700 1 -- 172.21.15.179:0/201741575 >> v1:172.21.15.179:6789/0 conn(0x7f76181434d0 legacy=0x7f761806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761d501700 1 -- 172.21.15.179:0/201741575 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180cd7c0 con 0x7f761806ed50 2024-05-08T22:44:39.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761d501700 1 --2- 172.21.15.179:0/201741575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761806ed50 0x7f76180b19e0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7610009850 tx=0x7f7610009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7a95ec222b2eee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011830 con 0x7f761806ed50 2024-05-08T22:44:39.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7610011990 con 0x7f761806ed50 2024-05-08T22:44:39.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f76177fe700 1 -- 172.21.15.179:0/201741575 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011d40 con 0x7f761806ed50 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- 172.21.15.179:0/201741575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761806ed50 msgr2=0x7f76180b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 --2- 172.21.15.179:0/201741575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761806ed50 0x7f76180b19e0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7610009850 tx=0x7f7610009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- 172.21.15.179:0/201741575 shutdown_connections 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 --2- 172.21.15.179:0/201741575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761806ed50 0x7f76180b19e0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- 172.21.15.179:0/201741575 >> 172.21.15.179:0/201741575 conn(0x7f76180ce9d0 msgr2=0x7f76180cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- 172.21.15.179:0/201741575 shutdown_connections 2024-05-08T22:44:39.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.219+0000 7f761dd02700 1 -- 172.21.15.179:0/201741575 wait complete. 2024-05-08T22:44:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 Processor -- start 2024-05-08T22:44:39.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 -- start start 2024-05-08T22:44:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 0x7f76180cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76180ce2a0 0x7f76181578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618158320 con 0x7f76181434d0 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76180cb120 con 0x7f76180ce2a0 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761dd02700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7618159b90 con 0x7f761806ed50 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761cd00700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 0x7f76180cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761cd00700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 0x7f76180cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:55596/0 (socket says 172.21.15.179:55596) 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56804/0 (socket says 172.21.15.179:56804) 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761cd00700 1 -- 172.21.15.179:0/2575979547 learned_addr learned my addr 172.21.15.179:0/2575979547 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f7617fff700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76180ce2a0 0x7f76181578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 -- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 msgr2=0x7f76180cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 0x7f76180cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 -- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76180ce2a0 msgr2=0x7f76181578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76180ce2a0 0x7f76181578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 -- 172.21.15.179:0/2575979547 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100094b0 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.223+0000 7f761d501700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f7610016000 tx=0x7f7610004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761001aac0 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761001f070 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76181595d0 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761001ae80 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7618136c50 con 0x7f76181434d0 2024-05-08T22:44:39.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7618067740 con 0x7f76181434d0 2024-05-08T22:44:39.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f761002e450 con 0x7f76181434d0 2024-05-08T22:44:39.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 0x7f7604068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7610085530 con 0x7f76181434d0 2024-05-08T22:44:39.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.227+0000 7f761cd00700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 0x7f7604068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.231+0000 7f761cd00700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 0x7f7604068bf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7608005cb0 tx=0x7f7608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.235+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76100528d0 con 0x7f76181434d0 2024-05-08T22:44:39.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.411+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f761804e700 con 0x7f76181434d0 2024-05-08T22:44:39.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.411+0000 7f76157fa700 1 -- 172.21.15.179:0/2575979547 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16364 (secure 0 0 0) 0x7f7610015090 con 0x7f76181434d0 2024-05-08T22:44:39.413 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:39.413 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":127,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:28.218971+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 msgr2=0x7f7604068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 0x7f7604068bf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7608005cb0 tx=0x7f7608009040 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 msgr2=0x7f7618157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f7610016000 tx=0x7f7610004770 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 shutdown_connections 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7604066730 0x7f7604068bf0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76180ce2a0 0x7f76181578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181434d0 0x7f7618157de0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 --2- 172.21.15.179:0/2575979547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f761806ed50 0x7f76180cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 >> 172.21.15.179:0/2575979547 conn(0x7f76180ce9d0 msgr2=0x7f7618070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 shutdown_connections 2024-05-08T22:44:39.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.415+0000 7f761dd02700 1 -- 172.21.15.179:0/2575979547 wait complete. 2024-05-08T22:44:39.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:39.429 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:39.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 Processor -- start 2024-05-08T22:44:39.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 -- start start 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff58415d0c0 0x7ff5840b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff5840d1d20 0x7ff5840cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5840ce0e0 0x7ff584160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5840ce570 con 0x7ff5840ce0e0 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58415d4a0 con 0x7ff5840d1d20 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589939700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58406abb0 con 0x7ff58415d0c0 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5840ce0e0 0x7ff584160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5840ce0e0 0x7ff584160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56810/0 (socket says 172.21.15.179:56810) 2024-05-08T22:44:39.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 -- 172.21.15.179:0/2594968808 learned_addr learned my addr 172.21.15.179:0/2594968808 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:39.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff588937700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff58415d0c0 0x7ff5840b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff583fff700 1 --2- 172.21.15.179:0/2594968808 >> v2:172.21.15.195:3300/0 conn(0x7ff5840d1d20 0x7ff5840cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 -- 172.21.15.179:0/2594968808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff58415d0c0 msgr2=0x7ff5840b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff58415d0c0 0x7ff5840b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 -- 172.21.15.179:0/2594968808 >> v2:172.21.15.195:3300/0 conn(0x7ff5840d1d20 msgr2=0x7ff5840cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 --2- 172.21.15.179:0/2594968808 >> v2:172.21.15.195:3300/0 conn(0x7ff5840d1d20 0x7ff5840cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 -- 172.21.15.179:0/2594968808 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840d2500 con 0x7ff5840ce0e0 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff589138700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff584160ba0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff584066c40 tx=0x7ff57c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be486ad40f9c6f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff5837fe700 1 -- 172.21.15.179:0/2594968808 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00e070 con 0x7ff5840ce0e0 2024-05-08T22:44:39.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff5837fe700 1 -- 172.21.15.179:0/2594968808 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c00b910 con 0x7ff5840ce0e0 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.543+0000 7ff5837fe700 1 -- 172.21.15.179:0/2594968808 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00bcc0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- 172.21.15.179:0/2594968808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 msgr2=0x7ff584160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff584160ba0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff584066c40 tx=0x7ff57c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- 172.21.15.179:0/2594968808 shutdown_connections 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff584160ba0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- 172.21.15.179:0/2594968808 >> v2:172.21.15.195:3300/0 conn(0x7ff5840d1d20 0x7ff5840cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- 172.21.15.179:0/2594968808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff58415d0c0 0x7ff5840b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- 172.21.15.179:0/2594968808 >> 172.21.15.179:0/2594968808 conn(0x7ff5840d2ed0 msgr2=0x7ff5840d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- 172.21.15.179:0/2594968808 shutdown_connections 2024-05-08T22:44:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- 172.21.15.179:0/2594968808 wait complete. 2024-05-08T22:44:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 Processor -- start 2024-05-08T22:44:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- start start 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5840d1d20 0x7ff58415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 0x7ff58415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840c8dd0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58414eaf0 con 0x7ff58415d0c0 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589939700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58415ec20 con 0x7ff5840d1d20 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff588937700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff588937700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56826/0 (socket says 172.21.15.179:56826) 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff588937700 1 -- 172.21.15.179:0/1597063569 learned_addr learned my addr 172.21.15.179:0/1597063569 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff583fff700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5840d1d20 0x7ff58415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.547+0000 7ff589138700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 0x7ff58415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 -- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5840d1d20 msgr2=0x7ff58415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5840d1d20 0x7ff58415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 -- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 msgr2=0x7ff58415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 0x7ff58415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 -- 172.21.15.179:0/1597063569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c009b30 con 0x7ff5840ce0e0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff589138700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 0x7ff58415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff588937700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ebe0 tx=0x7ff57400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574009740 con 0x7ff5840ce0e0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57400c850 con 0x7ff5840ce0e0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400cbe0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58415ef00 con 0x7ff5840ce0e0 2024-05-08T22:44:39.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58415f4c0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58404e700 con 0x7ff5840ce0e0 2024-05-08T22:44:39.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.551+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff57400cd40 con 0x7ff5840ce0e0 2024-05-08T22:44:39.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.555+0000 7ff5817fa700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 0x7ff564068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.555+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff5740842b0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.555+0000 7ff583fff700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 0x7ff564068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.555+0000 7ff583fff700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 0x7ff564068bf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff5840b8080 tx=0x7ff56c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.559+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff574051570 con 0x7ff5840ce0e0 2024-05-08T22:44:39.732 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.731+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5840676e0 con 0x7ff5840ce0e0 2024-05-08T22:44:39.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.731+0000 7ff5817fa700 1 -- 172.21.15.179:0/1597063569 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff574054b90 con 0x7ff5840ce0e0 2024-05-08T22:44:39.733 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 msgr2=0x7ff564068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 0x7ff564068bf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff5840b8080 tx=0x7ff56c009240 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 msgr2=0x7ff5840c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ebe0 tx=0x7ff57400eef0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 shutdown_connections 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff564066730 0x7ff564068bf0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58415d0c0 0x7ff58415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5840d1d20 0x7ff58415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 --2- 172.21.15.179:0/1597063569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5840ce0e0 0x7ff5840c8890 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 >> 172.21.15.179:0/1597063569 conn(0x7ff5840d2ed0 msgr2=0x7ff5840d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 shutdown_connections 2024-05-08T22:44:39.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.735+0000 7ff589939700 1 -- 172.21.15.179:0/1597063569 wait complete. 2024-05-08T22:44:39.748 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:39.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.867+0000 7fe0fce4a700 1 Processor -- start 2024-05-08T22:44:39.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.867+0000 7fe0fce4a700 1 -- start start 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0f80ce2b0 0x7fe0f8167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80b8ee0 con 0x7fe0f815d0c0 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f815d4a0 con 0x7fe0f80b8a80 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f806abb0 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0f80ce2b0 0x7fe0f8167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe0f815d0c0 0x7fe0f80d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42608/0 (socket says 172.21.15.179:42608) 2024-05-08T22:44:39.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f77fe700 1 -- 172.21.15.179:0/790508081 learned_addr learned my addr 172.21.15.179:0/790508081 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f7fff700 1 -- 172.21.15.179:0/790508081 >> v1:172.21.15.195:6789/0 conn(0x7fe0f80b8a80 legacy=0x7fe0f80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f7fff700 1 -- 172.21.15.179:0/790508081 >> v1:172.21.15.179:6789/0 conn(0x7fe0f815d0c0 legacy=0x7fe0f80d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f7fff700 1 -- 172.21.15.179:0/790508081 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80d2500 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f67fc700 1 -- 172.21.15.179:0/790508081 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023942863 0 0) 0x7fe0f80b8ee0 con 0x7fe0f815d0c0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f7fff700 1 --2- 172.21.15.179:0/790508081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f8167ed0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009d90 tx=0x7fe0f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cb54e3bb39fa7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f67fc700 1 -- 172.21.15.179:0/790508081 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000e070 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f67fc700 1 -- 172.21.15.179:0/790508081 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f000bbd0 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0f67fc700 1 -- 172.21.15.179:0/790508081 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0012430 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- 172.21.15.179:0/790508081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f80ce2b0 msgr2=0x7fe0f8167ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/790508081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f8167ed0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009d90 tx=0x7fe0f000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- 172.21.15.179:0/790508081 shutdown_connections 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/790508081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f8167ed0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- 172.21.15.179:0/790508081 >> 172.21.15.179:0/790508081 conn(0x7fe0f80d2ed0 msgr2=0x7fe0f80d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:39.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- 172.21.15.179:0/790508081 shutdown_connections 2024-05-08T22:44:39.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.871+0000 7fe0fce4a700 1 -- 172.21.15.179:0/790508081 wait complete. 2024-05-08T22:44:39.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.875+0000 7fe0fce4a700 1 Processor -- start 2024-05-08T22:44:39.876 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.875+0000 7fe0fce4a700 1 -- start start 2024-05-08T22:44:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.875+0000 7fe0fce4a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0f80b8a80 0x7fe0f80c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.875+0000 7fe0fce4a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.875+0000 7fe0fce4a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f8158120 con 0x7fe0f815d0c0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f8157510 con 0x7fe0f80ce2b0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81577f0 con 0x7fe0f80b8a80 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56846/0 (socket says 172.21.15.179:56846) 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3095242324 learned_addr learned my addr 172.21.15.179:0/3095242324 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f77fe700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0f80b8a80 0x7fe0f80c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f6ffd700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0f80b8a80 msgr2=0x7fe0f80c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0f80b8a80 0x7fe0f80c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0f80ce2b0 msgr2=0x7fe0f814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 -- 172.21.15.179:0/3095242324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0009a70 con 0x7fe0f815d0c0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0f7fff700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0014040 tx=0x7fe0f0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f001b810 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0020070 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f001bbb0 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f8157ad0 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f8150960 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f80d2a80 con 0x7fe0f815d0c0 2024-05-08T22:44:39.881 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0f001bd10 con 0x7fe0f815d0c0 2024-05-08T22:44:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 0x7fe0d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.879+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe0f0022070 con 0x7fe0f815d0c0 2024-05-08T22:44:39.882 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.883+0000 7fe0f77fe700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 0x7fe0d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:39.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.883+0000 7fe0f77fe700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 0x7fe0d8068bf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000f60 tx=0x7fe0e0007db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:39.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:39.887+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe0f0057090 con 0x7fe0f815d0c0 2024-05-08T22:44:40.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.055+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0f804e700 con 0x7fe0f815d0c0 2024-05-08T22:44:40.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0effff700 1 -- 172.21.15.179:0/3095242324 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0f0056c20 con 0x7fe0f815d0c0 2024-05-08T22:44:40.059 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:40.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 msgr2=0x7fe0d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 0x7fe0d8068bf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000f60 tx=0x7fe0e0007db0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 msgr2=0x7fe0f815bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0014040 tx=0x7fe0f0004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 shutdown_connections 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe0d8066730 0x7fe0d8068bf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0f80ce2b0 0x7fe0f814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0f815d0c0 0x7fe0f815bdd0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 --2- 172.21.15.179:0/3095242324 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0f80b8a80 0x7fe0f80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 >> 172.21.15.179:0/3095242324 conn(0x7fe0f80d2ed0 msgr2=0x7fe0f80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 shutdown_connections 2024-05-08T22:44:40.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.059+0000 7fe0fce4a700 1 -- 172.21.15.179:0/3095242324 wait complete. 2024-05-08T22:44:40.074 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:40.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 Processor -- start 2024-05-08T22:44:40.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 -- start start 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdd740cd710 0x7fdd740cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740d16b0 con 0x7fdd740ce2b0 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740d1990 con 0x7fdd7415d130 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd73fff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740ce100 con 0x7fdd740cd710 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd727fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdd740ce2b0 0x7fdd740d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42624/0 (socket says 172.21.15.179:42624) 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd727fc700 1 -- 172.21.15.179:0/4160186177 learned_addr learned my addr 172.21.15.179:0/4160186177 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:40.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.195+0000 7fdd737fe700 1 --2- 172.21.15.179:0/4160186177 >> v2:172.21.15.195:3300/0 conn(0x7fdd740cd710 0x7fdd740cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808143624 0 0) 0x7fdd740d16b0 con 0x7fdd740ce2b0 2024-05-08T22:44:40.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd5c0035d0 con 0x7fdd740ce2b0 2024-05-08T22:44:40.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd737fe700 1 -- 172.21.15.179:0/4160186177 >> v1:172.21.15.195:6789/0 conn(0x7fdd7415d130 legacy=0x7fdd740d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd737fe700 1 -- 172.21.15.179:0/4160186177 >> v1:172.21.15.179:6789/0 conn(0x7fdd740ce2b0 legacy=0x7fdd740d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637671698 0 0) 0x7fdd5c0035d0 con 0x7fdd740ce2b0 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd737fe700 1 -- 172.21.15.179:0/4160186177 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740d1c70 con 0x7fdd740cd710 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd737fe700 1 --2- 172.21.15.179:0/4160186177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd740cd710 0x7fdd740cdbc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009d90 tx=0x7fdd6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c934435767949cab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c00e070 con 0x7fdd740cd710 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c003a40 con 0x7fdd740cd710 2024-05-08T22:44:40.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd71ffb700 1 -- 172.21.15.179:0/4160186177 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64001670 con 0x7fdd740cd710 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 -- 172.21.15.179:0/4160186177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd740cd710 msgr2=0x7fdd740cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 --2- 172.21.15.179:0/4160186177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd740cd710 0x7fdd740cdbc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009d90 tx=0x7fdd6c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 -- 172.21.15.179:0/4160186177 shutdown_connections 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 --2- 172.21.15.179:0/4160186177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd740cd710 0x7fdd740cdbc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 -- 172.21.15.179:0/4160186177 >> 172.21.15.179:0/4160186177 conn(0x7fdd740b49a0 msgr2=0x7fdd740b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 -- 172.21.15.179:0/4160186177 shutdown_connections 2024-05-08T22:44:40.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.199+0000 7fdd73fff700 1 -- 172.21.15.179:0/4160186177 wait complete. 2024-05-08T22:44:40.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 Processor -- start 2024-05-08T22:44:40.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- start start 2024-05-08T22:44:40.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 0x7fdd740cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7415d130 0x7fdd7415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740cb360 con 0x7fdd740cd710 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7415f100 con 0x7fdd7415d130 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740cb5b0 con 0x7fdd740ce2b0 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd727fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 0x7fdd740cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56858/0 (socket says 172.21.15.179:56858) 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd727fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 0x7fdd740cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:55642/0 (socket says 172.21.15.179:55642) 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 -- 172.21.15.179:0/3427694579 learned_addr learned my addr 172.21.15.179:0/3427694579 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd737fe700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7415d130 0x7fdd7415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 -- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 msgr2=0x7fdd740cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 0x7fdd740cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 -- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7415d130 msgr2=0x7fdd7415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7415d130 0x7fdd7415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 -- 172.21.15.179:0/3427694579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009a70 con 0x7fdd740cd710 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd72ffd700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c410 tx=0x7fdd6800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680052d0 con 0x7fdd740cd710 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd68005430 con 0x7fdd740cd710 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7415d690 con 0x7fdd740cd710 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680055a0 con 0x7fdd740cd710 2024-05-08T22:44:40.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.203+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd740cbb70 con 0x7fdd740cd710 2024-05-08T22:44:40.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7404e700 con 0x7fdd740cd710 2024-05-08T22:44:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd680036a0 con 0x7fdd740cd710 2024-05-08T22:44:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd5b7fe700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 0x7fdd60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdd68083420 con 0x7fdd740cd710 2024-05-08T22:44:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd727fc700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 0x7fdd60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.207+0000 7fdd727fc700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 0x7fdd60068bf0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406df40 tx=0x7fdd64001730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.215+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdd68050640 con 0x7fdd740cd710 2024-05-08T22:44:40.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd740b8840 con 0x7fdd740cd710 2024-05-08T22:44:40.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd5b7fe700 1 -- 172.21.15.179:0/3427694579 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd68053c60 con 0x7fdd740cd710 2024-05-08T22:44:40.380 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:40.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 msgr2=0x7fdd60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 0x7fdd60068bf0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406df40 tx=0x7fdd64001730 comp rx=0 tx=0).stop 2024-05-08T22:44:40.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 msgr2=0x7fdd74158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.382 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c410 tx=0x7fdd6800c720 comp rx=0 tx=0).stop 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 shutdown_connections 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdd60066730 0x7fdd60068bf0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7415d130 0x7fdd7415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd740ce2b0 0x7fdd740cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 --2- 172.21.15.179:0/3427694579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd740cd710 0x7fdd74158ac0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 >> 172.21.15.179:0/3427694579 conn(0x7fdd740b49a0 msgr2=0x7fdd740b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.379+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 shutdown_connections 2024-05-08T22:44:40.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.383+0000 7fdd73fff700 1 -- 172.21.15.179:0/3427694579 wait complete. 2024-05-08T22:44:40.394 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:44:40.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.507+0000 7f7e6117e700 1 Processor -- start 2024-05-08T22:44:40.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.507+0000 7f7e6117e700 1 -- start start 2024-05-08T22:44:40.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 0x7f7e5c1640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c15d950 con 0x7f7e5c159270 2024-05-08T22:44:40.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c15dc30 con 0x7f7e5c15cf70 2024-05-08T22:44:40.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0b1af0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7e5c159270 0x7f7e5c0b22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42630/0 (socket says 172.21.15.179:42630) 2024-05-08T22:44:40.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5bfff700 1 -- 172.21.15.179:0/2882758593 learned_addr learned my addr 172.21.15.179:0/2882758593 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:40.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6097d700 1 --2- 172.21.15.179:0/2882758593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 0x7f7e5c1640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617720432 0 0) 0x7f7e5c15d950 con 0x7f7e5c159270 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e480035d0 con 0x7f7e5c159270 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6097d700 1 -- 172.21.15.179:0/2882758593 >> v1:172.21.15.195:6789/0 conn(0x7f7e5c15cf70 legacy=0x7f7e5c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543582409 0 0) 0x7f7e5c15dc30 con 0x7f7e5c15cf70 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6097d700 1 -- 172.21.15.179:0/2882758593 >> v1:172.21.15.179:6789/0 conn(0x7f7e5c159270 legacy=0x7f7e5c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6097d700 1 -- 172.21.15.179:0/2882758593 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0b1dd0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6097d700 1 --2- 172.21.15.179:0/2882758593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 0x7f7e5c1640d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009880 tx=0x7f7e54009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980912f69756063f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54011830 con 0x7f7e5c06bb50 2024-05-08T22:44:40.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e54011990 con 0x7f7e5c06bb50 2024-05-08T22:44:40.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e5affd700 1 -- 172.21.15.179:0/2882758593 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54011d20 con 0x7f7e5c06bb50 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- 172.21.15.179:0/2882758593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 msgr2=0x7f7e5c1640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2882758593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 0x7f7e5c1640d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009880 tx=0x7f7e54009b90 comp rx=0 tx=0).stop 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- 172.21.15.179:0/2882758593 shutdown_connections 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2882758593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c06bb50 0x7f7e5c1640d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- 172.21.15.179:0/2882758593 >> 172.21.15.179:0/2882758593 conn(0x7f7e5c0ce960 msgr2=0x7f7e5c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- 172.21.15.179:0/2882758593 shutdown_connections 2024-05-08T22:44:40.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.511+0000 7f7e6117e700 1 -- 172.21.15.179:0/2882758593 wait complete. 2024-05-08T22:44:40.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 Processor -- start 2024-05-08T22:44:40.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 -- start start 2024-05-08T22:44:40.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 0x7f7e5c1ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c15cf70 0x7f7e5c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c1492f0 con 0x7f7e5c159270 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14e610 con 0x7f7e5c06bb50 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6117e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14e8f0 con 0x7f7e5c15cf70 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 0x7f7e5c1ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 0x7f7e5c1ee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56860/0 (socket says 172.21.15.179:56860) 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5b7fe700 1 -- 172.21.15.179:0/2973363989 learned_addr learned my addr 172.21.15.179:0/2973363989 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:35894/0 (socket says 172.21.15.179:35894) 2024-05-08T22:44:40.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e6097d700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c15cf70 0x7f7e5c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 -- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c15cf70 msgr2=0x7f7e5c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c15cf70 0x7f7e5c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 -- 172.21.15.179:0/2973363989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 msgr2=0x7f7e5c1ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 0x7f7e5c1ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 -- 172.21.15.179:0/2973363989 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540094b0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.515+0000 7f7e5bfff700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000f60 tx=0x7f7e4400c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44005d90 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400ce80 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c15d720 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44007310 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c14eeb0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c04e700 con 0x7f7e5c06bb50 2024-05-08T22:44:40.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e44007470 con 0x7f7e5c06bb50 2024-05-08T22:44:40.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 0x7f7e3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.519+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7e44083640 con 0x7f7e5c06bb50 2024-05-08T22:44:40.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.523+0000 7f7e5b7fe700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 0x7f7e3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.523+0000 7f7e5b7fe700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 0x7f7e3c068bf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c06dc60 tx=0x7f7e4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.531+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e440507d0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.711+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e5c06dc90 con 0x7f7e5c06bb50 2024-05-08T22:44:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e58ff9700 1 -- 172.21.15.179:0/2973363989 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e44053df0 con 0x7f7e5c06bb50 2024-05-08T22:44:40.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 msgr2=0x7f7e3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 0x7f7e3c068bf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c06dc60 tx=0x7f7e4c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 msgr2=0x7f7e5c148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000f60 tx=0x7f7e4400c7b0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 shutdown_connections 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f7e3c066730 0x7f7e3c068bf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e5c15cf70 0x7f7e5c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e5c159270 0x7f7e5c1ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 --2- 172.21.15.179:0/2973363989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e5c06bb50 0x7f7e5c148db0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 >> 172.21.15.179:0/2973363989 conn(0x7f7e5c0ce960 msgr2=0x7f7e5c06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 shutdown_connections 2024-05-08T22:44:40.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.715+0000 7f7e6117e700 1 -- 172.21.15.179:0/2973363989 wait complete. 2024-05-08T22:44:40.719 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:44:40.729 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:44:40.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:40.730 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:40.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52326]: fuse finished with error 0 and tester_r 0 2024-05-08T22:44:40.741 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:40.831 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:44:40.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:40.831 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:44:40.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-08T22:44:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.955+0000 7fee392e5700 1 Processor -- start 2024-05-08T22:44:40.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.955+0000 7fee392e5700 1 -- start start 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee3406c240 0x7fee340b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee340ce5c0 con 0x7fee340b65b0 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340b8f00 con 0x7fee3406c240 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340cf640 con 0x7fee340cdca0 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee3406c240 0x7fee340b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee3406c240 0x7fee340b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56862/0 (socket says 172.21.15.179:56862) 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 -- 172.21.15.179:0/1075191472 learned_addr learned my addr 172.21.15.179:0/1075191472 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee33fff700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 -- 172.21.15.179:0/1075191472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 msgr2=0x7fee340ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 -- 172.21.15.179:0/1075191472 >> v1:172.21.15.195:6789/0 conn(0x7fee340b65b0 legacy=0x7fee340b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 -- 172.21.15.179:0/1075191472 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340b8c20 con 0x7fee3406c240 2024-05-08T22:44:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee337fe700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3406c240 0x7fee340b6070 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fee24009940 tx=0x7fee24009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682bcd5306bc8617 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee32ffd700 1 -- 172.21.15.179:0/1075191472 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400ba20 con 0x7fee3406c240 2024-05-08T22:44:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee32ffd700 1 -- 172.21.15.179:0/1075191472 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24009e90 con 0x7fee3406c240 2024-05-08T22:44:40.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee32ffd700 1 -- 172.21.15.179:0/1075191472 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400bda0 con 0x7fee3406c240 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- 172.21.15.179:0/1075191472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3406c240 msgr2=0x7fee340b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3406c240 0x7fee340b6070 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fee24009940 tx=0x7fee24009c50 comp rx=0 tx=0).stop 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- 172.21.15.179:0/1075191472 shutdown_connections 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3406c240 0x7fee340b6070 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 --2- 172.21.15.179:0/1075191472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- 172.21.15.179:0/1075191472 >> 172.21.15.179:0/1075191472 conn(0x7fee340d2ed0 msgr2=0x7fee340d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- 172.21.15.179:0/1075191472 shutdown_connections 2024-05-08T22:44:40.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.959+0000 7fee392e5700 1 -- 172.21.15.179:0/1075191472 wait complete. 2024-05-08T22:44:40.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 Processor -- start 2024-05-08T22:44:40.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- start start 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee3406c240 0x7fee340b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b40d0 con 0x7fee340b65b0 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b3450 con 0x7fee3406c240 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b3730 con 0x7fee340cdca0 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56878/0 (socket says 172.21.15.179:56878) 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 -- 172.21.15.179:0/2493291267 learned_addr learned my addr 172.21.15.179:0/2493291267 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee33fff700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee3406c240 0x7fee340b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee38ae4700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 -- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 msgr2=0x7fee340b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 -- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee3406c240 msgr2=0x7fee340b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee3406c240 0x7fee340b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 -- 172.21.15.179:0/2493291267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24009570 con 0x7fee340b65b0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee38ae4700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee337fe700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fee24000c00 tx=0x7fee24004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee30ff9700 1 -- 172.21.15.179:0/2493291267 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24004590 con 0x7fee340b65b0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee30ff9700 1 -- 172.21.15.179:0/2493291267 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2401e070 con 0x7fee340b65b0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee30ff9700 1 -- 172.21.15.179:0/2493291267 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24003900 con 0x7fee340b65b0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee340b3a10 con 0x7fee340b65b0 2024-05-08T22:44:40.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.963+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee34158990 con 0x7fee340b65b0 2024-05-08T22:44:40.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.967+0000 7fee30ff9700 1 -- 172.21.15.179:0/2493291267 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee24003a60 con 0x7fee340b65b0 2024-05-08T22:44:40.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.967+0000 7fee30ff9700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 0x7fee20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.967+0000 7fee30ff9700 1 -- 172.21.15.179:0/2493291267 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee24020070 con 0x7fee340b65b0 2024-05-08T22:44:40.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.967+0000 7fee33fff700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 0x7fee20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.967+0000 7fee33fff700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 0x7fee20068bf0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fee28009a00 tx=0x7fee28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 Processor -- start 2024-05-08T22:44:40.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- start start 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17c76f0 0x179dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1730010 con 0x17ca5d0 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bdf40 con 0x179d840 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1710e30 con 0x17c76f0 2024-05-08T22:44:40.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56890/0 (socket says 172.21.15.179:56890) 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 -- 172.21.15.179:0/1687531748 learned_addr learned my addr 172.21.15.179:0/1687531748 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee33fff700 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee337fe700 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17c76f0 0x179dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 -- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17c76f0 msgr2=0x179dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17c76f0 0x179dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 -- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 msgr2=0x17a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 -- 172.21.15.179:0/1687531748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x157ca40 con 0x17ca5d0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fee2c002140 tx=0x7fee2c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b25c4152c5a273 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee1a7fc700 1 -- 172.21.15.179:0/1687531748 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00ec60 con 0x17ca5d0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee1a7fc700 1 -- 172.21.15.179:0/1687531748 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00edc0 con 0x17ca5d0 2024-05-08T22:44:40.974 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee1a7fc700 1 -- 172.21.15.179:0/1687531748 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c003c40 con 0x17ca5d0 2024-05-08T22:44:40.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1687531748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 msgr2=0x179e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fee2c002140 tx=0x7fee2c00e440 comp rx=0 tx=0).stop 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.971+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1687531748 shutdown_connections 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17ca5d0 0x179e810 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17c76f0 0x179dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1687531748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1687531748 >> 172.21.15.179:0/1687531748 conn(0x15315b0 msgr2=0x17c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1687531748 shutdown_connections 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1687531748 wait complete. 2024-05-08T22:44:40.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- start start 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 Processor -- start 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x170c150 con 0x17c76f0 2024-05-08T22:44:40.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1709eb0 con 0x179d840 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c78e0 con 0x17ca5d0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 10 client.0 _ll_register_callbacks cb 0x16a6670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56906/0 (socket says 172.21.15.179:56906) 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 -- 172.21.15.179:0/1780236478 learned_addr learned my addr 172.21.15.179:0/1780236478 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee33fff700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 -- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 msgr2=0x17cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 -- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 msgr2=0x17c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 -- 172.21.15.179:0/1780236478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c00f040 con 0x17c76f0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee337fe700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee24052ba0 tx=0x7fee240566a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615477185b77cad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:40.980 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2405ea90 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x153e2c0 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee24052db0 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.975+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee24052f70 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 10 client.9855 fetch_fsmap learned FSMap version 90 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x16d0290 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x153e2c0 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 90) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee2406f020 con 0x17c76f0 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 10 client.9855 fetch_fsmap finished waiting for FSMap version 90 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 10 client.9855 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 20 client.9855 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17a0980 0x17a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 4 client.9855 mds_command: new command op to 57983 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:44:40.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 --> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1588540 con 0x17a0980 2024-05-08T22:44:40.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee33fff700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17a0980 0x17a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee33fff700 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17a0980 0x17a0d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92cab0a5a6fe2274 server_cookie=60b88a26e40b4e15 in_seq=0 out_seq=0 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 10 client.9855 ms_handle_connect on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 1 -- 172.21.15.179:0/1780236478 <== mds.0 v2:172.21.15.195:6834/2184096099 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1588540 con 0x17a0980 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 10 client.9855 handle_command_reply: tid=0 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 client.9855 shutdown 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 msgr2=0x17cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee24052ba0 tx=0x7fee240566a0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 shutdown_connections 2024-05-08T22:44:40.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17a0980 0x17a0d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x17cc010 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1780236478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x179d840 0x17c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 >> 172.21.15.179:0/1780236478 conn(0x15315b0 msgr2=0x17c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee12ffd700 0 client.9855 ms_handle_reset on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 shutdown_connections 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1780236478 wait complete. 2024-05-08T22:44:40.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.979+0000 7fee3d7f6740 20 client.9855 trim_cache size 0 max 16384 2024-05-08T22:44:41.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.999+0000 7fee3d7f6740 1 Processor -- start 2024-05-08T22:44:41.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:40.999+0000 7fee3d7f6740 1 -- start start 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x179d840 0x17f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 0x1844240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b5b80 con 0x17c76f0 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x16dac00 con 0x17ca5d0 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1731610 con 0x179d840 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56912/0 (socket says 172.21.15.179:56912) 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 -- 172.21.15.179:0/1559348556 learned_addr learned my addr 172.21.15.179:0/1559348556 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee33fff700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x179d840 0x17f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 0x1844240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 -- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x179d840 msgr2=0x17f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x179d840 0x17f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 -- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 msgr2=0x1844240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 0x1844240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 -- 172.21.15.179:0/1559348556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16b64c0 con 0x17c76f0 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 0x1844240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee337fe700 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee24009320 tx=0x7fee2405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60b950ecca32f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee1affd700 1 -- 172.21.15.179:0/1559348556 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24056000 con 0x17c76f0 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee1affd700 1 -- 172.21.15.179:0/1559348556 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee240529e0 con 0x17c76f0 2024-05-08T22:44:41.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee1affd700 1 -- 172.21.15.179:0/1559348556 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24055e10 con 0x17c76f0 2024-05-08T22:44:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1559348556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 msgr2=0x1843d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee24009320 tx=0x7fee2405a000 comp rx=0 tx=0).stop 2024-05-08T22:44:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1559348556 shutdown_connections 2024-05-08T22:44:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17ca5d0 0x1844240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x17c76f0 0x1843d00 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1559348556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x179d840 0x17f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1559348556 >> 172.21.15.179:0/1559348556 conn(0x15315b0 msgr2=0x17c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1559348556 shutdown_connections 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1559348556 wait complete. 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.003+0000 7fee3d7f6740 1 -- start start 2024-05-08T22:44:41.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 Processor -- start 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c76f0 0x17fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17ff360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b1680 con 0x179d840 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x17076c0 con 0x17c76f0 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1806c70 con 0x17ca5d0 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 10 client.0 _ll_register_callbacks cb 0x16c3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56914/0 (socket says 172.21.15.179:56914) 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 -- 172.21.15.179:0/1201579743 learned_addr learned my addr 172.21.15.179:0/1201579743 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee337fe700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c76f0 0x17fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee38ae4700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 -- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 msgr2=0x17ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 -- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c76f0 msgr2=0x17fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c76f0 0x17fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 -- 172.21.15.179:0/1201579743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16d0290 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee33fff700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fee28009640 tx=0x7fee280047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410ebdcc59638a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28028040 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee2802b870 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee2c00edc0 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee28011e90 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.007+0000 7fee3d7f6740 10 client.9861 fetch_fsmap learned FSMap version 90 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x16aa790 con 0x179d840 2024-05-08T22:44:41.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00edc0 con 0x179d840 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 90) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee28004030 con 0x179d840 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 10 client.9861 fetch_fsmap finished waiting for FSMap version 90 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 10 client.9861 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 20 client.9861 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17ff8a0 0x184d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 4 client.9861 mds_command: new command op to 57983 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 --> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x15ad2c0 con 0x17ff8a0 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee337fe700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17ff8a0 0x184d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee337fe700 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17ff8a0 0x184d8a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33557ea0968373d9 server_cookie=d1da128040848264 in_seq=0 out_seq=0 2024-05-08T22:44:41.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.011+0000 7fee12ffd700 10 client.9861 ms_handle_connect on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:41.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee12ffd700 1 -- 172.21.15.179:0/1201579743 <== mds.0 v2:172.21.15.195:6834/2184096099 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x15ad2c0 con 0x17ff8a0 2024-05-08T22:44:41.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee12ffd700 10 client.9861 handle_command_reply: tid=0 2024-05-08T22:44:41.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 client.9861 shutdown 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 msgr2=0x1800e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fee28009640 tx=0x7fee280047e0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 shutdown_connections 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:6834/2184096099,v1:172.21.15.195:6835/2184096099] conn(0x17ff8a0 0x184d8a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x17c76f0 0x17fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x17ca5d0 0x17ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 --2- 172.21.15.179:0/1201579743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x179d840 0x1800e40 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee12ffd700 0 client.9861 ms_handle_reset on v2:172.21.15.195:6834/2184096099 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 >> 172.21.15.179:0/1201579743 conn(0x15315b0 msgr2=0x17cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 shutdown_connections 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 1 -- 172.21.15.179:0/1201579743 wait complete. 2024-05-08T22:44:41.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee3d7f6740 20 client.9861 trim_cache size 0 max 16384 2024-05-08T22:44:41.029 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:41.029 INFO:teuthology.orchestra.run.smithi179.stdout: "message": "", 2024-05-08T22:44:41.029 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0 2024-05-08T22:44:41.029 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 msgr2=0x7fee20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 0x7fee20068bf0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fee28009a00 tx=0x7fee28008040 comp rx=0 tx=0).stop 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 msgr2=0x7fee3414c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fee24000c00 tx=0x7fee24004770 comp rx=0 tx=0).stop 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 shutdown_connections 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fee20066730 0x7fee20068bf0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee340cdca0 0x7fee340b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee340b65b0 0x7fee3414c1d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 --2- 172.21.15.179:0/2493291267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee3406c240 0x7fee340b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 >> 172.21.15.179:0/2493291267 conn(0x7fee340d2ed0 msgr2=0x7fee340d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 shutdown_connections 2024-05-08T22:44:41.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.027+0000 7fee392e5700 1 -- 172.21.15.179:0/2493291267 wait complete. 2024-05-08T22:44:41.043 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:44:41.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 Processor -- start 2024-05-08T22:44:41.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 -- start start 2024-05-08T22:44:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 0x7f1b7415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7406eb20 con 0x7f1b7406ac90 2024-05-08T22:44:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7415e880 con 0x7f1b740cab60 2024-05-08T22:44:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7c36e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7406df10 con 0x7f1b7415df60 2024-05-08T22:44:41.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79909700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1b7406ac90 0x7f1b740ca450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42640/0 (socket says 172.21.15.179:42640) 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79909700 1 -- 172.21.15.179:0/1495397890 learned_addr learned my addr 172.21.15.179:0/1495397890 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7a10a700 1 --2- 172.21.15.179:0/1495397890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 0x7f1b7415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989510854 0 0) 0x7f1b7406eb20 con 0x7f1b7406ac90 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b600035d0 con 0x7f1b7406ac90 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866431571 0 0) 0x7f1b7415e880 con 0x7f1b740cab60 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b7406eb20 con 0x7f1b740cab60 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7a10a700 1 -- 172.21.15.179:0/1495397890 >> v1:172.21.15.195:6789/0 conn(0x7f1b740cab60 legacy=0x7f1b740cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7a10a700 1 -- 172.21.15.179:0/1495397890 >> v1:172.21.15.179:6789/0 conn(0x7f1b7406ac90 legacy=0x7f1b740ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163574689 0 0) 0x7f1b600035d0 con 0x7f1b7406ac90 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7a10a700 1 -- 172.21.15.179:0/1495397890 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7406e1f0 con 0x7f1b7415df60 2024-05-08T22:44:41.162 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b7a10a700 1 --2- 172.21.15.179:0/1495397890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 0x7f1b7415e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009880 tx=0x7f1b70009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719cb59a883c4af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64001670 con 0x7f1b7415df60 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70004030 con 0x7f1b7415df60 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.159+0000 7f1b79108700 1 -- 172.21.15.179:0/1495397890 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700043c0 con 0x7f1b7415df60 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1495397890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 msgr2=0x7f1b7415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1495397890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 0x7f1b7415e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009880 tx=0x7f1b70009b90 comp rx=0 tx=0).stop 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1495397890 shutdown_connections 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1495397890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7415df60 0x7f1b7415e340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1495397890 >> 172.21.15.179:0/1495397890 conn(0x7f1b740cb660 msgr2=0x7f1b740cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1495397890 shutdown_connections 2024-05-08T22:44:41.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1495397890 wait complete. 2024-05-08T22:44:41.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 Processor -- start 2024-05-08T22:44:41.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.163+0000 7f1b7c36e700 1 -- start start 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7406ac90 0x7f1b740c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b740cab60 0x7f1b741f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740c2e60 con 0x7f1b7415df60 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7414a9a0 con 0x7f1b740cab60 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7414ac80 con 0x7f1b7406ac90 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b79909700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b740cab60 0x7f1b741f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a10a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7406ac90 0x7f1b740c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56916/0 (socket says 172.21.15.179:56916) 2024-05-08T22:44:41.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 -- 172.21.15.179:0/1991081635 learned_addr learned my addr 172.21.15.179:0/1991081635 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 -- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7406ac90 msgr2=0x7f1b740c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7406ac90 0x7f1b740c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 -- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b740cab60 msgr2=0x7f1b741f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b740cab60 0x7f1b741f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 -- 172.21.15.179:0/1991081635 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700094b0 con 0x7f1b7415df60 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7a90b700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00cbc0 tx=0x7f1b6c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c003c20 con 0x7f1b7415df60 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c003d80 con 0x7f1b7415df60 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7414af60 con 0x7f1b7415df60 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00aaa0 con 0x7f1b7415df60 2024-05-08T22:44:41.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7414b520 con 0x7f1b7415df60 2024-05-08T22:44:41.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.167+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b74067770 con 0x7f1b7415df60 2024-05-08T22:44:41.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.171+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b6c00ac00 con 0x7f1b7415df60 2024-05-08T22:44:41.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.171+0000 7f1b6affd700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 0x7f1b58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:41.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.171+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1b6c084c60 con 0x7f1b7415df60 2024-05-08T22:44:41.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.171+0000 7f1b7a10a700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 0x7f1b58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:41.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.171+0000 7f1b7a10a700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 0x7f1b58068bf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b700090b0 tx=0x7f1b70010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:41.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.179+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1b6c051f70 con 0x7f1b7415df60 2024-05-08T22:44:41.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.383+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b7406ef90 con 0x7f1b7415df60 2024-05-08T22:44:41.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b6affd700 1 -- 172.21.15.179:0/1991081635 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b7406ef90 con 0x7f1b7415df60 2024-05-08T22:44:41.387 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:44:41.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 msgr2=0x7f1b58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 0x7f1b58068bf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b700090b0 tx=0x7f1b70010000 comp rx=0 tx=0).stop 2024-05-08T22:44:41.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 msgr2=0x7f1b740c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00cbc0 tx=0x7f1b6c00ced0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 shutdown_connections 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1b58066730 0x7f1b58068bf0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b740cab60 0x7f1b741f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7415df60 0x7f1b740c5340 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 --2- 172.21.15.179:0/1991081635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b7406ac90 0x7f1b740c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 >> 172.21.15.179:0/1991081635 conn(0x7f1b740cb660 msgr2=0x7f1b741594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 shutdown_connections 2024-05-08T22:44:41.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:41.387+0000 7f1b7c36e700 1 -- 172.21.15.179:0/1991081635 wait complete. 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:44:41.401 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:41.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.402 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:44:41.407 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.408 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41890sec preferred_lft 41890sec 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:44:41.409 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.410 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:41.410 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:44:41.410 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:44:41.410 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:41.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.410 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:44:41.469 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:44:41.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.471 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:44:41.480 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:44:41.480 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:44:41.480 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:44:41.480 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:44:41.481 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:44:41.481 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:44:41.481 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:44:41.481 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:44:41.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:44:41.481 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:44:41.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.520 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:44:41.527 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:44:41.527 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:44:41.528 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:41.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.528 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:44:41.542 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:44:41.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:41.542 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:41.547 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:41.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:41.548 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:41.596 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:41.613 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:41.613 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:41.628 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:41.629 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:41.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.630 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:41.635 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:44:41.635 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:44:41.680 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:41.696 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:41.696 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:41.708 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:41.709 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:41.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:41.710 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:41.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:44:41.763+0000 7fbd64f4a640 -1 init, newargv = 0x560d15395a30 newargc=15 2024-05-08T22:44:41.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52791]: starting ceph client 2024-05-08T22:44:41.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52791]: starting fuse 2024-05-08T22:44:42.816 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:42.833 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:42.833 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:42.844 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:42.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:42.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:42.846 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:42.897 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:44:42.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:44:42.897 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:44:42.898 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:44:42.899 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:42.986 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.52791.asok 2024-05-08T22:44:43.168 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52791.asok 2024-05-08T22:44:43.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:43.168 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52791.asok status 2024-05-08T22:44:43.276 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "52791", 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9873, 2024-05-08T22:44:43.277 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 9873 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1632405741 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:43.278 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 1632405741 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.9873 192.168.0.1:0/1632405741", 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/1632405741", 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 90, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 127, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:44:43.279 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:43.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:43.286 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:44:43.287 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:43.293 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:44:43.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:43.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:43.294 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:43.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:43.350 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-08T22:44:43.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:43.368 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-08T22:44:43.447 INFO:teuthology.orchestra.run.smithi179.stderr:8+0 records in 2024-05-08T22:44:43.447 INFO:teuthology.orchestra.run.smithi179.stderr:8+0 records out 2024-05-08T22:44:43.447 INFO:teuthology.orchestra.run.smithi179.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0692386 s, 121 MB/s 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T22:44:43.449 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:43.514 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627779 2024-05-08T22:44:43.703 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:43.704 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:43.704 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:43.704 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:43.704 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T22:44:43.704 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:43.816 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627780 2024-05-08T22:44:44.004 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:44.004 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:44.010 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:44.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.127+0000 7f657c019700 1 Processor -- start 2024-05-08T22:44:44.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.127+0000 7f657c019700 1 -- start start 2024-05-08T22:44:44.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f657415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6574158f60 con 0x7f65740c8d70 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657406b860 con 0x7f65740c9230 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657406ac50 con 0x7f6574158b80 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f65795b4700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f65740c8d70 0x7f657406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42648/0 (socket says 172.21.15.179:42648) 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f65795b4700 1 -- 172.21.15.179:0/2606568815 learned_addr learned my addr 172.21.15.179:0/2606568815 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f657415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306829009 0 0) 0x7f6574158f60 con 0x7f65740c8d70 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65600035d0 con 0x7f65740c8d70 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2081028783 0 0) 0x7f65600035d0 con 0x7f65740c8d70 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 msgr2=0x7f657415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:44.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f657415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 >> v1:172.21.15.195:6789/0 conn(0x7f65740c9230 legacy=0x7f657406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657406af30 con 0x7f65740c8d70 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f65700019f0 con 0x7f65740c8d70 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 >> v1:172.21.15.179:6789/0 conn(0x7f65740c8d70 legacy=0x7f657406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6560003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6560004870 0x7f6560004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f6560005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657406ac50 con 0x7f6560003a50 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656000bae0 con 0x7f6560004870 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6560003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656000bc50 con 0x7f6574158b80 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f65795b4700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6560004870 0x7f6560004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657a5b6700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f6560005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 -- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 msgr2=0x7f6560005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6574158b80 0x7f6560005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 -- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6560004870 msgr2=0x7f6560004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6560004870 0x7f6560004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 -- 172.21.15.179:0/2606568815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600035d0 con 0x7f6560003a50 2024-05-08T22:44:44.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6579db5700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6560003e30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6564009b40 tx=0x7f65600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d71f1525177598 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400e660 con 0x7f6560003a50 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656400e7c0 con 0x7f6560003a50 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f6578db3700 1 -- 172.21.15.179:0/2606568815 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400eb30 con 0x7f6560003a50 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- 172.21.15.179:0/2606568815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 msgr2=0x7f6560003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 --2- 172.21.15.179:0/2606568815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6560003e30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6564009b40 tx=0x7f65600058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657a5b6700 1 -- 172.21.15.179:0/2606568815 reap_dead start 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- 172.21.15.179:0/2606568815 shutdown_connections 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- 172.21.15.179:0/2606568815 >> 172.21.15.179:0/2606568815 conn(0x7f65740cb660 msgr2=0x7f65740cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- 172.21.15.179:0/2606568815 shutdown_connections 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.131+0000 7f657c019700 1 -- 172.21.15.179:0/2606568815 wait complete. 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 Processor -- start 2024-05-08T22:44:44.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 -- start start 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 0x7f65740c1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f657415c970 0x7f657415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740c7860 con 0x7f6560003a50 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657415d330 con 0x7f657415c970 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657c019700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65741f6910 con 0x7f65741f6100 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f6579db5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f6579db5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56928/0 (socket says 172.21.15.179:56928) 2024-05-08T22:44:44.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f6579db5700 1 -- 172.21.15.179:0/1606251260 learned_addr learned my addr 172.21.15.179:0/1606251260 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f65795b4700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 0x7f65740c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.135+0000 7f657a5b6700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f657415c970 0x7f657415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 -- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 msgr2=0x7f65740c1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 0x7f65740c1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 -- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f657415c970 msgr2=0x7f657415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f657415c970 0x7f657415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 -- 172.21.15.179:0/1606251260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65741f6bf0 con 0x7f6560003a50 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f65795b4700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 0x7f65740c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f6579db5700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f6560004790 tx=0x7f65640047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400e660 con 0x7f6560003a50 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564013d50 con 0x7f6560003a50 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65740c74f0 con 0x7f6560003a50 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564004290 con 0x7f6560003a50 2024-05-08T22:44:44.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65741f6ed0 con 0x7f6560003a50 2024-05-08T22:44:44.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6574067770 con 0x7f6560003a50 2024-05-08T22:44:44.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.139+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65640043f0 con 0x7f6560003a50 2024-05-08T22:44:44.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.143+0000 7f656a7fc700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 0x7f6558068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.143+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6564085760 con 0x7f6560003a50 2024-05-08T22:44:44.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.143+0000 7f65795b4700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 0x7f6558068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.143+0000 7f65795b4700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 0x7f6558068c20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f65741f6060 tx=0x7f6570006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.147+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6564052b40 con 0x7f6560003a50 2024-05-08T22:44:44.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.315+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f657406f000 con 0x7f6560003a50 2024-05-08T22:44:44.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.319+0000 7f656a7fc700 1 -- 172.21.15.179:0/1606251260 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16364 (secure 0 0 0) 0x7f6564056160 con 0x7f6560003a50 2024-05-08T22:44:44.320 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:44.320 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":127,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:28.218971+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.319+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 msgr2=0x7f6558068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.319+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 0x7f6558068c20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f65741f6060 tx=0x7f6570006040 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.319+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 msgr2=0x7f6574156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.319+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f6560004790 tx=0x7f65640047e0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 shutdown_connections 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6558066760 0x7f6558068c20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f657415c970 0x7f657415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65741f6100 0x7f65740c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 --2- 172.21.15.179:0/1606251260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560003a50 0x7f6574156b20 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 >> 172.21.15.179:0/1606251260 conn(0x7f65740cb660 msgr2=0x7f657415d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 shutdown_connections 2024-05-08T22:44:44.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.323+0000 7f657c019700 1 -- 172.21.15.179:0/1606251260 wait complete. 2024-05-08T22:44:44.335 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:44.336 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:44.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 Processor -- start 2024-05-08T22:44:44.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 -- start start 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 0x7efcdc0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 0x7efcdc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0cd070 con 0x7efcdc15d000 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc0b3b00 con 0x7efcdc0b47f0 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b3d10 con 0x7efcdc0cffc0 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcdad9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 0x7efcdc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.451+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 0x7efcdc0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdad9d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 0x7efcdc0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:35972/0 (socket says 172.21.15.179:35972) 2024-05-08T22:44:44.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdad9d700 1 -- 172.21.15.179:0/2107711783 learned_addr learned my addr 172.21.15.179:0/2107711783 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:44.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd9d9b700 1 -- 172.21.15.179:0/2107711783 >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 msgr2=0x7efcdc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd9d9b700 1 --2- 172.21.15.179:0/2107711783 >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 0x7efcdc0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd9d9b700 1 -- 172.21.15.179:0/2107711783 >> v1:172.21.15.195:6789/0 conn(0x7efcdc15d000 legacy=0x7efcdc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd9d9b700 1 -- 172.21.15.179:0/2107711783 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0b3ff0 con 0x7efcdc0cffc0 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd9d9b700 1 --2- 172.21.15.179:0/2107711783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 0x7efcdc0cb320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efccc009850 tx=0x7efccc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3106194ccab66cd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd959a700 1 -- 172.21.15.179:0/2107711783 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc011830 con 0x7efcdc0cffc0 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd959a700 1 -- 172.21.15.179:0/2107711783 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc011990 con 0x7efcdc0cffc0 2024-05-08T22:44:44.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcd959a700 1 -- 172.21.15.179:0/2107711783 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc011d40 con 0x7efcdc0cffc0 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 -- 172.21.15.179:0/2107711783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 msgr2=0x7efcdc0cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 --2- 172.21.15.179:0/2107711783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 0x7efcdc0cb320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efccc009850 tx=0x7efccc009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 -- 172.21.15.179:0/2107711783 shutdown_connections 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 --2- 172.21.15.179:0/2107711783 >> v2:172.21.15.195:3300/0 conn(0x7efcdc0b47f0 0x7efcdc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 --2- 172.21.15.179:0/2107711783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0cffc0 0x7efcdc0cb320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 -- 172.21.15.179:0/2107711783 >> 172.21.15.179:0/2107711783 conn(0x7efcdc0b27c0 msgr2=0x7efcdc0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 -- 172.21.15.179:0/2107711783 shutdown_connections 2024-05-08T22:44:44.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.455+0000 7efcdb59e700 1 -- 172.21.15.179:0/2107711783 wait complete. 2024-05-08T22:44:44.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 Processor -- start 2024-05-08T22:44:44.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 -- start start 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0b47f0 0x7efcdc14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcdc0cffc0 0x7efcdc0c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc157a20 con 0x7efcdc15d000 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0c6af0 con 0x7efcdc0cffc0 2024-05-08T22:44:44.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdb59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1570f0 con 0x7efcdc0b47f0 2024-05-08T22:44:44.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56942/0 (socket says 172.21.15.179:56942) 2024-05-08T22:44:44.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcdc0cffc0 0x7efcdc0c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcdad9d700 1 -- 172.21.15.179:0/926094112 learned_addr learned my addr 172.21.15.179:0/926094112 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:44.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.459+0000 7efcda59c700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0b47f0 0x7efcdc14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 -- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0b47f0 msgr2=0x7efcdc14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0b47f0 0x7efcdc14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 -- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcdc0cffc0 msgr2=0x7efcdc0c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcdc0cffc0 0x7efcdc0c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 -- 172.21.15.179:0/926094112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0094b0 con 0x7efcdc15d000 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdad9d700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7efcd400efc0 tx=0x7efcd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4003c40 con 0x7efcdc15d000 2024-05-08T22:44:44.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd400f040 con 0x7efcdc15d000 2024-05-08T22:44:44.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4009e30 con 0x7efcdc15d000 2024-05-08T22:44:44.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc1573d0 con 0x7efcdc15d000 2024-05-08T22:44:44.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc0c5630 con 0x7efcdc15d000 2024-05-08T22:44:44.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc067740 con 0x7efcdc15d000 2024-05-08T22:44:44.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.463+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efcd400c880 con 0x7efcdc15d000 2024-05-08T22:44:44.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.467+0000 7efccaffd700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 0x7efcbc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.467+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efcd4084490 con 0x7efcdc15d000 2024-05-08T22:44:44.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.467+0000 7efcda59c700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 0x7efcbc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.467+0000 7efcda59c700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 0x7efcbc068c20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000f60 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.471+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efcd40516c0 con 0x7efcdc15d000 2024-05-08T22:44:44.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.643+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcdc0c5820 con 0x7efcdc15d000 2024-05-08T22:44:44.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efccaffd700 1 -- 172.21.15.179:0/926094112 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcd4054ce0 con 0x7efcdc15d000 2024-05-08T22:44:44.647 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:44.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 msgr2=0x7efcbc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 0x7efcbc068c20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000f60 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 msgr2=0x7efcdc15f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7efcd400efc0 tx=0x7efcd400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 shutdown_connections 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7efcbc066760 0x7efcbc068c20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcdc0cffc0 0x7efcdc0c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcdc15d000 0x7efcdc15f600 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 --2- 172.21.15.179:0/926094112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efcdc0b47f0 0x7efcdc14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 >> 172.21.15.179:0/926094112 conn(0x7efcdc0b27c0 msgr2=0x7efcdc0c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 shutdown_connections 2024-05-08T22:44:44.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.647+0000 7efcdb59e700 1 -- 172.21.15.179:0/926094112 wait complete. 2024-05-08T22:44:44.662 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:44.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 Processor -- start 2024-05-08T22:44:44.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 -- start start 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f00d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f00d16b0 con 0x7fc7f00cd710 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f00d1990 con 0x7fc7f015d130 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6e34700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00d1c70 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6633700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc7f00cd710 0x7fc7f00b6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42664/0 (socket says 172.21.15.179:42664) 2024-05-08T22:44:44.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.783+0000 7fc7f6633700 1 -- 172.21.15.179:0/442721359 learned_addr learned my addr 172.21.15.179:0/442721359 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f5631700 1 --2- 172.21.15.179:0/442721359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f00d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176491645 0 0) 0x7fc7f00d16b0 con 0x7fc7f00cd710 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7dc0035d0 con 0x7fc7f00cd710 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572304098 0 0) 0x7fc7dc0035d0 con 0x7fc7f00cd710 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f5631700 1 -- 172.21.15.179:0/442721359 >> v1:172.21.15.195:6789/0 conn(0x7fc7f015d130 legacy=0x7fc7f00d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f5631700 1 -- 172.21.15.179:0/442721359 >> v1:172.21.15.179:6789/0 conn(0x7fc7f00cd710 legacy=0x7fc7f00b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f5631700 1 -- 172.21.15.179:0/442721359 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00d1f50 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f5631700 1 --2- 172.21.15.179:0/442721359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f00d0db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009880 tx=0x7fc7ec009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a062d4e46f84e79c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80019f0 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec010590 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f4e30700 1 -- 172.21.15.179:0/442721359 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec010900 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 -- 172.21.15.179:0/442721359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 msgr2=0x7fc7f00d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/442721359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f00d0db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009880 tx=0x7fc7ec009b90 comp rx=0 tx=0).stop 2024-05-08T22:44:44.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 -- 172.21.15.179:0/442721359 shutdown_connections 2024-05-08T22:44:44.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/442721359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f00d0db0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 -- 172.21.15.179:0/442721359 >> 172.21.15.179:0/442721359 conn(0x7fc7f00b49a0 msgr2=0x7fc7f00b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 -- 172.21.15.179:0/442721359 shutdown_connections 2024-05-08T22:44:44.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.787+0000 7fc7f6e34700 1 -- 172.21.15.179:0/442721359 wait complete. 2024-05-08T22:44:44.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 Processor -- start 2024-05-08T22:44:44.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 -- start start 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f00cd710 0x7fc7f015f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015f200 con 0x7fc7f015d130 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015e940 con 0x7fc7f00cd710 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6e34700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00a9c90 con 0x7fc7f00ce2b0 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6633700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6633700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56952/0 (socket says 172.21.15.179:56952) 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f6633700 1 -- 172.21.15.179:0/712516538 learned_addr learned my addr 172.21.15.179:0/712516538 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:44.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f5631700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.791+0000 7fc7f5e32700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f00cd710 0x7fc7f015f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 -- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 msgr2=0x7fc7f0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f0158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 -- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f00cd710 msgr2=0x7fc7f015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f00cd710 0x7fc7f015f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 -- 172.21.15.179:0/712516538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0094b0 con 0x7fc7f015d130 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6633700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80018a0 tx=0x7fc7e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80019f0 con 0x7fc7f015d130 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e800ac50 con 0x7fc7f015d130 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f015d690 con 0x7fc7f015d130 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80070d0 con 0x7fc7f015d130 2024-05-08T22:44:44.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f015ebb0 con 0x7fc7f015d130 2024-05-08T22:44:44.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.795+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f015f3b0 con 0x7fc7f015d130 2024-05-08T22:44:44.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.799+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7e8007230 con 0x7fc7f015d130 2024-05-08T22:44:44.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.799+0000 7fc7e67fc700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 0x7fc7d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:44.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.799+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc7e8083e30 con 0x7fc7f015d130 2024-05-08T22:44:44.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.799+0000 7fc7f5e32700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 0x7fc7d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:44.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.799+0000 7fc7f5e32700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 0x7fc7d4068bf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0000f60 tx=0x7fc7e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:44.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.807+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7f015f3b0 con 0x7fc7f015d130 2024-05-08T22:44:44.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.971+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7f015f3b0 con 0x7fc7f015d130 2024-05-08T22:44:44.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.971+0000 7fc7e67fc700 1 -- 172.21.15.179:0/712516538 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7e8018020 con 0x7fc7f015d130 2024-05-08T22:44:44.973 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 msgr2=0x7fc7d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 0x7fc7d4068bf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0000f60 tx=0x7fc7e0009040 comp rx=0 tx=0).stop 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 msgr2=0x7fc7f015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80018a0 tx=0x7fc7e800a340 comp rx=0 tx=0).stop 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 shutdown_connections 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc7d4066730 0x7fc7d4068bf0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7f015d130 0x7fc7f015e400 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7f00ce2b0 0x7fc7f0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 --2- 172.21.15.179:0/712516538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f00cd710 0x7fc7f015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:44.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 >> 172.21.15.179:0/712516538 conn(0x7fc7f00b49a0 msgr2=0x7fc7f00cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:44.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 shutdown_connections 2024-05-08T22:44:44.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:44.975+0000 7fc7f6e34700 1 -- 172.21.15.179:0/712516538 wait complete. 2024-05-08T22:44:44.988 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:45.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 Processor -- start 2024-05-08T22:44:45.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 -- start start 2024-05-08T22:44:45.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b8415d0c0 0x7f8b840b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b840ce2b0 0x7f8b8406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8415d4a0 con 0x7f8b840d1d20 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8406abb0 con 0x7f8b8415d0c0 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b88ff7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b840d2500 con 0x7f8b840ce2b0 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b837fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b8415d0c0 0x7f8b840b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b82ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8b840d1d20 0x7f8b840cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42680/0 (socket says 172.21.15.179:42680) 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b82ffd700 1 -- 172.21.15.179:0/1234730745 learned_addr learned my addr 172.21.15.179:0/1234730745 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.103+0000 7f8b83fff700 1 --2- 172.21.15.179:0/1234730745 >> v2:172.21.15.195:3300/0 conn(0x7f8b840ce2b0 0x7f8b8406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b827fc700 1 -- 172.21.15.179:0/1234730745 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963258216 0 0) 0x7f8b8415d4a0 con 0x7f8b840d1d20 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b827fc700 1 -- 172.21.15.179:0/1234730745 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b6c0035d0 con 0x7f8b840d1d20 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b837fe700 1 -- 172.21.15.179:0/1234730745 >> v2:172.21.15.195:3300/0 conn(0x7f8b840ce2b0 msgr2=0x7f8b8406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b837fe700 1 --2- 172.21.15.179:0/1234730745 >> v2:172.21.15.195:3300/0 conn(0x7f8b840ce2b0 0x7f8b8406b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b837fe700 1 -- 172.21.15.179:0/1234730745 >> v1:172.21.15.179:6789/0 conn(0x7f8b840d1d20 legacy=0x7f8b840cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b837fe700 1 -- 172.21.15.179:0/1234730745 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840d27e0 con 0x7f8b8415d0c0 2024-05-08T22:44:45.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b837fe700 1 --2- 172.21.15.179:0/1234730745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b8415d0c0 0x7f8b840b8a80 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009850 tx=0x7f8b78009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9ad46c6b7549ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b827fc700 1 -- 172.21.15.179:0/1234730745 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011830 con 0x7f8b8415d0c0 2024-05-08T22:44:45.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b827fc700 1 -- 172.21.15.179:0/1234730745 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78011990 con 0x7f8b8415d0c0 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b827fc700 1 -- 172.21.15.179:0/1234730745 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011d20 con 0x7f8b8415d0c0 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- 172.21.15.179:0/1234730745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b8415d0c0 msgr2=0x7f8b840b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/1234730745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b8415d0c0 0x7f8b840b8a80 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009850 tx=0x7f8b78009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- 172.21.15.179:0/1234730745 shutdown_connections 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/1234730745 >> v2:172.21.15.195:3300/0 conn(0x7f8b840ce2b0 0x7f8b8406b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/1234730745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b8415d0c0 0x7f8b840b8a80 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- 172.21.15.179:0/1234730745 >> 172.21.15.179:0/1234730745 conn(0x7f8b840d2ed0 msgr2=0x7f8b840d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- 172.21.15.179:0/1234730745 shutdown_connections 2024-05-08T22:44:45.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- 172.21.15.179:0/1234730745 wait complete. 2024-05-08T22:44:45.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 Processor -- start 2024-05-08T22:44:45.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.107+0000 7f8b88ff7700 1 -- start start 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b840d1d20 0x7f8b840b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 0x7f8b841f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8415a520 con 0x7f8b840ce2b0 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b840cbf40 con 0x7f8b840d1d20 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84159910 con 0x7f8b8415d0c0 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56964/0 (socket says 172.21.15.179:56964) 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 -- 172.21.15.179:0/3342140339 learned_addr learned my addr 172.21.15.179:0/3342140339 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b82ffd700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b840d1d20 0x7f8b840b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b83fff700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 0x7f8b841f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 -- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 msgr2=0x7f8b841f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 0x7f8b841f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 -- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b840d1d20 msgr2=0x7f8b840b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b840d1d20 0x7f8b840b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 -- 172.21.15.179:0/3342140339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780094b0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b83fff700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 0x7f8b841f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b837fe700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8b780090b0 tx=0x7f8b78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.113 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78004550 con 0x7f8b840ce2b0 2024-05-08T22:44:45.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7801e070 con 0x7f8b840ce2b0 2024-05-08T22:44:45.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b84159bf0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78003900 con 0x7f8b840ce2b0 2024-05-08T22:44:45.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.111+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8415a1b0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8404e700 con 0x7f8b840ce2b0 2024-05-08T22:44:45.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b78003a60 con 0x7f8b840ce2b0 2024-05-08T22:44:45.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b6bfff700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 0x7f8b70068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8b780850f0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b82ffd700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 0x7f8b70068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.115+0000 7f8b82ffd700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 0x7f8b70068c20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cc990 tx=0x7f8b74007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.119+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b78055e40 con 0x7f8b840ce2b0 2024-05-08T22:44:45.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.287+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b840676e0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.287+0000 7f8b6bfff700 1 -- 172.21.15.179:0/3342140339 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b780559d0 con 0x7f8b840ce2b0 2024-05-08T22:44:45.288 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.287+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 msgr2=0x7f8b70068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.287+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 0x7f8b70068c20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cc990 tx=0x7f8b74007de0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 msgr2=0x7f8b840cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8b780090b0 tx=0x7f8b78004770 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 shutdown_connections 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8b70066760 0x7f8b70068c20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b840d1d20 0x7f8b840b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b8415d0c0 0x7f8b841f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 --2- 172.21.15.179:0/3342140339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b840ce2b0 0x7f8b840cba00 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 >> 172.21.15.179:0/3342140339 conn(0x7f8b840d2ed0 msgr2=0x7f8b841533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 shutdown_connections 2024-05-08T22:44:45.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.291+0000 7f8b88ff7700 1 -- 172.21.15.179:0/3342140339 wait complete. 2024-05-08T22:44:45.303 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:44:45.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 Processor -- start 2024-05-08T22:44:45.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 -- start start 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47180adb90 0x7f47180cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180b1cb0 con 0x7f4718150bc0 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180b1f90 con 0x7f47180ad740 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180b2930 con 0x7f47180adb90 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47180adb90 0x7f47180cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47180adb90 0x7f47180cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60294/0 (socket says 172.21.15.179:60294) 2024-05-08T22:44:45.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 -- 172.21.15.179:0/3778109255 learned_addr learned my addr 172.21.15.179:0/3778109255 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:45.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 -- 172.21.15.179:0/3778109255 >> v1:172.21.15.195:6789/0 conn(0x7f47180ad740 legacy=0x7f47180ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 -- 172.21.15.179:0/3778109255 >> v1:172.21.15.179:6789/0 conn(0x7f4718150bc0 legacy=0x7f471815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 -- 172.21.15.179:0/3778109255 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180b2270 con 0x7f47180adb90 2024-05-08T22:44:45.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f4716d9d700 1 --2- 172.21.15.179:0/3778109255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180adb90 0x7f47180cd4e0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4708009940 tx=0x7f4708009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a3fb923f241058 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.415+0000 7f471559a700 1 -- 172.21.15.179:0/3778109255 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800ba20 con 0x7f47180adb90 2024-05-08T22:44:45.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471559a700 1 -- 172.21.15.179:0/3778109255 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708009e90 con 0x7f47180adb90 2024-05-08T22:44:45.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471559a700 1 -- 172.21.15.179:0/3778109255 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800bda0 con 0x7f47180adb90 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- 172.21.15.179:0/3778109255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180adb90 msgr2=0x7f47180cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 --2- 172.21.15.179:0/3778109255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180adb90 0x7f47180cd4e0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4708009940 tx=0x7f4708009c50 comp rx=0 tx=0).stop 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- 172.21.15.179:0/3778109255 shutdown_connections 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 --2- 172.21.15.179:0/3778109255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180adb90 0x7f47180cd4e0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- 172.21.15.179:0/3778109255 >> 172.21.15.179:0/3778109255 conn(0x7f47180ce960 msgr2=0x7f47180ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- 172.21.15.179:0/3778109255 shutdown_connections 2024-05-08T22:44:45.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- 172.21.15.179:0/3778109255 wait complete. 2024-05-08T22:44:45.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 Processor -- start 2024-05-08T22:44:45.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- start start 2024-05-08T22:44:45.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47180adb90 0x7f47180ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4718150bc0 0x7f47180abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180b0230 con 0x7f47180ad740 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471813b200 con 0x7f4718150bc0 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.419+0000 7f471759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471813bb00 con 0x7f47180adb90 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60310/0 (socket says 172.21.15.179:60310) 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 -- 172.21.15.179:0/1289893375 learned_addr learned my addr 172.21.15.179:0/1289893375 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f4716d9d700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4718150bc0 0x7f47180abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f4715d9b700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47180adb90 0x7f47180ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 -- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47180adb90 msgr2=0x7f47180ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47180adb90 0x7f47180ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 -- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4718150bc0 msgr2=0x7f47180abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4718150bc0 0x7f47180abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 -- 172.21.15.179:0/1289893375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708009570 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471659c700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f471000c500 tx=0x7f471000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100052d0 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710005430 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47180af570 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100055a0 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47180afec0 con 0x7f47180ad740 2024-05-08T22:44:45.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47100038e0 con 0x7f47180ad740 2024-05-08T22:44:45.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471804e700 con 0x7f47180ad740 2024-05-08T22:44:45.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.423+0000 7f470effd700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 0x7f46fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.427+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4710083740 con 0x7f47180ad740 2024-05-08T22:44:45.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.427+0000 7f4715d9b700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 0x7f46fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.427+0000 7f4715d9b700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 0x7f46fc068bf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f471813b900 tx=0x7f4700009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.431+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4710050a00 con 0x7f47180ad740 2024-05-08T22:44:45.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.615+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f47180cc0d0 con 0x7f47180ad740 2024-05-08T22:44:45.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.615+0000 7f470effd700 1 -- 172.21.15.179:0/1289893375 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4710054020 con 0x7f47180ad740 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 msgr2=0x7f46fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 0x7f46fc068bf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f471813b900 tx=0x7f4700009040 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 msgr2=0x7f471815a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f471000c500 tx=0x7f471000c8c0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 shutdown_connections 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f46fc066730 0x7f46fc068bf0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4718150bc0 0x7f47180abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47180adb90 0x7f47180ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 --2- 172.21.15.179:0/1289893375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47180ad740 0x7f471815a4f0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 >> 172.21.15.179:0/1289893375 conn(0x7f47180ce960 msgr2=0x7f471815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 shutdown_connections 2024-05-08T22:44:45.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.619+0000 7f471759e700 1 -- 172.21.15.179:0/1289893375 wait complete. 2024-05-08T22:44:45.621 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:44:45.631 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:45.631 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:45.639 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:45.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 Processor -- start 2024-05-08T22:44:45.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 -- start start 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe20c06de70 0x7fe20c161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 0x7fe20c06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c165d50 con 0x7fe20c15ef80 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c06ad70 con 0x7fe20c06de70 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe2109ad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c165260 con 0x7fe20c164db0 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20ad9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe20c15ef80 0x7fe20c06d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60050/0 (socket says 172.21.15.179:60050) 2024-05-08T22:44:45.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20ad9d700 1 -- 172.21.15.179:0/1801651932 learned_addr learned my addr 172.21.15.179:0/1801651932 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20b59e700 1 --2- 172.21.15.179:0/1801651932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 0x7fe20c06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20a59c700 1 --2- 172.21.15.179:0/1801651932 >> v2:172.21.15.195:3300/0 conn(0x7fe20c06de70 0x7fe20c161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe209d9b700 1 -- 172.21.15.179:0/1801651932 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772489242 0 0) 0x7fe20c165d50 con 0x7fe20c15ef80 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe209d9b700 1 -- 172.21.15.179:0/1801651932 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f80035d0 con 0x7fe20c15ef80 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20b59e700 1 -- 172.21.15.179:0/1801651932 >> v2:172.21.15.195:3300/0 conn(0x7fe20c06de70 msgr2=0x7fe20c161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20b59e700 1 --2- 172.21.15.179:0/1801651932 >> v2:172.21.15.195:3300/0 conn(0x7fe20c06de70 0x7fe20c161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20b59e700 1 -- 172.21.15.179:0/1801651932 >> v1:172.21.15.179:6789/0 conn(0x7fe20c15ef80 legacy=0x7fe20c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.751+0000 7fe20b59e700 1 -- 172.21.15.179:0/1801651932 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c06e4b0 con 0x7fe20c164db0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe20b59e700 1 --2- 172.21.15.179:0/1801651932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 0x7fe20c06b200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009850 tx=0x7fe1fc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de5f1b36f276ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe209d9b700 1 -- 172.21.15.179:0/1801651932 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011830 con 0x7fe20c164db0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe209d9b700 1 -- 172.21.15.179:0/1801651932 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc011990 con 0x7fe20c164db0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe209d9b700 1 -- 172.21.15.179:0/1801651932 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011d00 con 0x7fe20c164db0 2024-05-08T22:44:45.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- 172.21.15.179:0/1801651932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 msgr2=0x7fe20c06b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- 172.21.15.179:0/1801651932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 0x7fe20c06b200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009850 tx=0x7fe1fc009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- 172.21.15.179:0/1801651932 shutdown_connections 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- 172.21.15.179:0/1801651932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c164db0 0x7fe20c06b200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- 172.21.15.179:0/1801651932 >> v2:172.21.15.195:3300/0 conn(0x7fe20c06de70 0x7fe20c161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- 172.21.15.179:0/1801651932 >> 172.21.15.179:0/1801651932 conn(0x7fe20c0ce9d0 msgr2=0x7fe20c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- 172.21.15.179:0/1801651932 shutdown_connections 2024-05-08T22:44:45.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- 172.21.15.179:0/1801651932 wait complete. 2024-05-08T22:44:45.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 Processor -- start 2024-05-08T22:44:45.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- start start 2024-05-08T22:44:45.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c15ef80 0x7fe20c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c164db0 0x7fe20c163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1f7c90 con 0x7fe20c06de70 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1564c0 con 0x7fe20c164db0 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.755+0000 7fe2109ad700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1f76d0 con 0x7fe20c15ef80 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60320/0 (socket says 172.21.15.179:60320) 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 -- 172.21.15.179:0/489807229 learned_addr learned my addr 172.21.15.179:0/489807229 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20b59e700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c164db0 0x7fe20c163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20a59c700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c15ef80 0x7fe20c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 -- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c15ef80 msgr2=0x7fe20c1f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c15ef80 0x7fe20c1f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 -- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c164db0 msgr2=0x7fe20c163b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c164db0 0x7fe20c163b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 -- 172.21.15.179:0/489807229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0094b0 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe20ad9d700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe20400cb70 tx=0x7fe20400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204007480 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2040075e0 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c1f79b0 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204007990 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c154cb0 con 0x7fe20c06de70 2024-05-08T22:44:45.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.759+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c067740 con 0x7fe20c06de70 2024-05-08T22:44:45.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.763+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe20401b3b0 con 0x7fe20c06de70 2024-05-08T22:44:45.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.763+0000 7fe2037fe700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:45.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.763+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe204083ca0 con 0x7fe20c06de70 2024-05-08T22:44:45.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.763+0000 7fe20a59c700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:45.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.763+0000 7fe20a59c700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe20c15d9a0 tx=0x7fe1f4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:45.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.767+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe204050e30 con 0x7fe20c06de70 2024-05-08T22:44:45.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.939+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe20c04e700 con 0x7fe20c06de70 2024-05-08T22:44:45.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2037fe700 1 -- 172.21.15.179:0/489807229 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16364 (secure 0 0 0) 0x7fe204017070 con 0x7fe20c06de70 2024-05-08T22:44:45.943 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:45.943 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":127,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:28.218971+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:45.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 msgr2=0x7fe1ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe20c15d9a0 tx=0x7fe1f4009240 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 msgr2=0x7fe20c1432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe20400cb70 tx=0x7fe20400cba0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 shutdown_connections 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1ec066730 0x7fe1ec068bf0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe20c164db0 0x7fe20c163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe20c15ef80 0x7fe20c1f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 --2- 172.21.15.179:0/489807229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe20c06de70 0x7fe20c1432e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 >> 172.21.15.179:0/489807229 conn(0x7fe20c0ce9d0 msgr2=0x7fe20c0cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 shutdown_connections 2024-05-08T22:44:45.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:45.943+0000 7fe2109ad700 1 -- 172.21.15.179:0/489807229 wait complete. 2024-05-08T22:44:45.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:45.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:46.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.079+0000 7f106e230700 1 Processor -- start 2024-05-08T22:44:46.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.079+0000 7f106e230700 1 -- start start 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1068159ce0 0x7f10680ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806e0a0 0x7f10680c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10680add60 con 0x7f10680acc00 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10680ccf10 con 0x7f1068159ce0 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10680ad150 con 0x7f106806e0a0 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1068159ce0 0x7f10680ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1068159ce0 0x7f10680ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60328/0 (socket says 172.21.15.179:60328) 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 -- 172.21.15.179:0/1721905156 learned_addr learned my addr 172.21.15.179:0/1721905156 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 -- 172.21.15.179:0/1721905156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806e0a0 msgr2=0x7f10680c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 --2- 172.21.15.179:0/1721905156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806e0a0 0x7f10680c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 -- 172.21.15.179:0/1721905156 >> v1:172.21.15.179:6789/0 conn(0x7f10680acc00 legacy=0x7f106806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 -- 172.21.15.179:0/1721905156 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680ad430 con 0x7f1068159ce0 2024-05-08T22:44:46.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106d22e700 1 --2- 172.21.15.179:0/1721905156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1068159ce0 0x7f10680ae5d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f106400bbb0 tx=0x7f106400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b994bf4cc2e5fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f105ffff700 1 -- 172.21.15.179:0/1721905156 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400dd90 con 0x7f1068159ce0 2024-05-08T22:44:46.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f105ffff700 1 -- 172.21.15.179:0/1721905156 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064010470 con 0x7f1068159ce0 2024-05-08T22:44:46.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f105ffff700 1 -- 172.21.15.179:0/1721905156 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064010820 con 0x7f1068159ce0 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- 172.21.15.179:0/1721905156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1068159ce0 msgr2=0x7f10680ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 --2- 172.21.15.179:0/1721905156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1068159ce0 0x7f10680ae5d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f106400bbb0 tx=0x7f106400bec0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- 172.21.15.179:0/1721905156 shutdown_connections 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 --2- 172.21.15.179:0/1721905156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f106806e0a0 0x7f10680c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 --2- 172.21.15.179:0/1721905156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1068159ce0 0x7f10680ae5d0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- 172.21.15.179:0/1721905156 >> 172.21.15.179:0/1721905156 conn(0x7f1068098a70 msgr2=0x7f10680cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- 172.21.15.179:0/1721905156 shutdown_connections 2024-05-08T22:44:46.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.083+0000 7f106e230700 1 -- 172.21.15.179:0/1721905156 wait complete. 2024-05-08T22:44:46.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 Processor -- start 2024-05-08T22:44:46.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- start start 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 0x7f10680ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 0x7f106814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681530c0 con 0x7f106806e0a0 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106814f860 con 0x7f10680acc00 2024-05-08T22:44:46.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106814f580 con 0x7f1068159ce0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60330/0 (socket says 172.21.15.179:60330) 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 -- 172.21.15.179:0/1828124599 learned_addr learned my addr 172.21.15.179:0/1828124599 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106ca2d700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 0x7f10680ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106da2f700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 0x7f106814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 -- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 msgr2=0x7f106814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 0x7f106814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 -- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 msgr2=0x7f10680ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 0x7f10680ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 -- 172.21.15.179:0/1828124599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106400b810 con 0x7f106806e0a0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106da2f700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 0x7f106814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106ca2d700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 0x7f10680ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106d22e700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f106400bf00 tx=0x7f1064004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064003d10 con 0x7f106806e0a0 2024-05-08T22:44:46.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064003e90 con 0x7f106806e0a0 2024-05-08T22:44:46.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064010820 con 0x7f106806e0a0 2024-05-08T22:44:46.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10680aec90 con 0x7f106806e0a0 2024-05-08T22:44:46.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.087+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1068152ce0 con 0x7f106806e0a0 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1068153200 con 0x7f106806e0a0 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1064019400 con 0x7f106806e0a0 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f105dffb700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 0x7f104c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10640861c0 con 0x7f106806e0a0 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f106ca2d700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 0x7f104c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.091+0000 7f106ca2d700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 0x7f104c068bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f10680b1d30 tx=0x7f1058009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.095+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1068153200 con 0x7f106806e0a0 2024-05-08T22:44:46.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.263+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f106804e700 con 0x7f106806e0a0 2024-05-08T22:44:46.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f105dffb700 1 -- 172.21.15.179:0/1828124599 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1064053480 con 0x7f106806e0a0 2024-05-08T22:44:46.266 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:46.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 msgr2=0x7f104c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 0x7f104c068bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f10680b1d30 tx=0x7f1058009040 comp rx=0 tx=0).stop 2024-05-08T22:44:46.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 msgr2=0x7f10680aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f106400bf00 tx=0x7f1064004770 comp rx=0 tx=0).stop 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 shutdown_connections 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f104c066730 0x7f104c068bf0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10680acc00 0x7f10680ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1068159ce0 0x7f106814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 --2- 172.21.15.179:0/1828124599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106806e0a0 0x7f10680aee20 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 >> 172.21.15.179:0/1828124599 conn(0x7f1068098a70 msgr2=0x7f10680cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 shutdown_connections 2024-05-08T22:44:46.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.267+0000 7f106e230700 1 -- 172.21.15.179:0/1828124599 wait complete. 2024-05-08T22:44:46.281 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:46.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.399+0000 7fabeffff700 1 Processor -- start 2024-05-08T22:44:46.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.399+0000 7fabeffff700 1 -- start start 2024-05-08T22:44:46.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fabf015d130 0x7fabf00d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00d2c90 con 0x7fabf00ce2b0 2024-05-08T22:44:46.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00d16b0 con 0x7fabf00cd710 2024-05-08T22:44:46.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf00d14c0 con 0x7fabf015d130 2024-05-08T22:44:46.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabee7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fabf00ce2b0 0x7fabf00d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60066/0 (socket says 172.21.15.179:60066) 2024-05-08T22:44:46.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabee7fc700 1 -- 172.21.15.179:0/1638441323 learned_addr learned my addr 172.21.15.179:0/1638441323 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:46.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeeffd700 1 --2- 172.21.15.179:0/1638441323 >> v2:172.21.15.195:3300/0 conn(0x7fabf015d130 0x7fabf00d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39833469 0 0) 0x7fabf00d2c90 con 0x7fabf00ce2b0 2024-05-08T22:44:46.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabdc0035d0 con 0x7fabf00ce2b0 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707178159 0 0) 0x7fabdc0035d0 con 0x7fabf00ce2b0 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 >> v2:172.21.15.195:3300/0 conn(0x7fabf015d130 msgr2=0x7fabf00d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 --2- 172.21.15.179:0/1638441323 >> v2:172.21.15.195:3300/0 conn(0x7fabf015d130 0x7fabf00d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 >> v1:172.21.15.195:6789/0 conn(0x7fabf00cd710 legacy=0x7fabf00b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00d1990 con 0x7fabf00ce2b0 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fabe0001670 con 0x7fabf00ce2b0 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 >> v1:172.21.15.179:6789/0 conn(0x7fabf00ce2b0 legacy=0x7fabf00d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabdc003a20 0x7fabdc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabdc004870 0x7fabdc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabdc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c0b0 con 0x7fabdc005230 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c220 con 0x7fabdc003a20 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c3e0 con 0x7fabdc004870 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabdc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeeffd700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabdc003a20 0x7fabdc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabee7fc700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabdc004870 0x7fabdc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 -- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabdc004870 msgr2=0x7fabdc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabdc004870 0x7fabdc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 -- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabdc003a20 msgr2=0x7fabdc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabdc003a20 0x7fabdc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 -- 172.21.15.179:0/1638441323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0035d0 con 0x7fabdc005230 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeeffd700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabdc003a20 0x7fabdc003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 -- 172.21.15.179:0/1638441323 reap_dead start 2024-05-08T22:44:46.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabef7fe700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabdc00ba30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c190 tx=0x7fabdc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d447a00a039ccb20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8001db0 con 0x7fabdc005230 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800ccf0 con 0x7fabdc005230 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabedffb700 1 -- 172.21.15.179:0/1638441323 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8003c60 con 0x7fabdc005230 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 -- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 msgr2=0x7fabdc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabdc00ba30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c190 tx=0x7fabdc0058a0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.403+0000 7fabeffff700 1 -- 172.21.15.179:0/1638441323 shutdown_connections 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 --2- 172.21.15.179:0/1638441323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabdc00ba30 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 -- 172.21.15.179:0/1638441323 >> 172.21.15.179:0/1638441323 conn(0x7fabf00b49a0 msgr2=0x7fabf00b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 -- 172.21.15.179:0/1638441323 shutdown_connections 2024-05-08T22:44:46.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 -- 172.21.15.179:0/1638441323 wait complete. 2024-05-08T22:44:46.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 Processor -- start 2024-05-08T22:44:46.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.407+0000 7fabeffff700 1 -- start start 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf0156520 0x7fabf01460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 0x7fabf0148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0160750 con 0x7fabdc005230 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0159110 con 0x7fabf0158810 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00cb070 con 0x7fabf0156520 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60342/0 (socket says 172.21.15.179:60342) 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 -- 172.21.15.179:0/921252606 learned_addr learned my addr 172.21.15.179:0/921252606 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabee7fc700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf0156520 0x7fabf01460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabef7fe700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 0x7fabf0148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 -- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf0156520 msgr2=0x7fabf01460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf0156520 0x7fabf01460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 -- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 msgr2=0x7fabf0148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 0x7fabf0148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 -- 172.21.15.179:0/921252606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00cb350 con 0x7fabdc005230 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabef7fe700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 0x7fabf0148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeeffd700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011cd0 tx=0x7fabd8009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd800fa00 con 0x7fabdc005230 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd800fb60 con 0x7fabdc005230 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8004020 con 0x7fabdc005230 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf00cb630 con 0x7fabdc005230 2024-05-08T22:44:46.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.411+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf0146600 con 0x7fabdc005230 2024-05-08T22:44:46.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf004e700 con 0x7fabdc005230 2024-05-08T22:44:46.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabd8004180 con 0x7fabdc005230 2024-05-08T22:44:46.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabe7fff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 0x7fabd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fabd8084860 con 0x7fabdc005230 2024-05-08T22:44:46.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabee7fc700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 0x7fabd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.415+0000 7fabee7fc700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 0x7fabd0068bf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fabf00b7a00 tx=0x7fabe0000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.423+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fabd8051a90 con 0x7fabdc005230 2024-05-08T22:44:46.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.591+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabf00b82b0 con 0x7fabdc005230 2024-05-08T22:44:46.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabe7fff700 1 -- 172.21.15.179:0/921252606 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabd8022020 con 0x7fabdc005230 2024-05-08T22:44:46.595 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 msgr2=0x7fabd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 0x7fabd0068bf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fabf00b7a00 tx=0x7fabe0000ed0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 msgr2=0x7fabf0160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011cd0 tx=0x7fabd8009e80 comp rx=0 tx=0).stop 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 shutdown_connections 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fabd0066730 0x7fabd0068bf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf0158810 0x7fabf0148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf0156520 0x7fabf01460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 --2- 172.21.15.179:0/921252606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabdc005230 0x7fabf0160210 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 >> 172.21.15.179:0/921252606 conn(0x7fabf00b49a0 msgr2=0x7fabf00c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 shutdown_connections 2024-05-08T22:44:46.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.595+0000 7fabeffff700 1 -- 172.21.15.179:0/921252606 wait complete. 2024-05-08T22:44:46.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 Processor -- start 2024-05-08T22:44:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 -- start start 2024-05-08T22:44:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f863415d200 0x7f86340ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863415d5e0 0x7f86340ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340cfa40 con 0x7f863406c550 2024-05-08T22:44:46.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86340ce950 con 0x7f863415d200 2024-05-08T22:44:46.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8639d25700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863406df30 con 0x7f863415d5e0 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f86337fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f863415d200 0x7f86340ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f86337fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f863415d200 0x7f86340ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:46008/0 (socket says 172.21.15.179:46008) 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f862affd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863415d5e0 0x7f86340ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f86337fe700 1 -- 172.21.15.179:0/2749825477 learned_addr learned my addr 172.21.15.179:0/2749825477 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.731+0000 7f8633fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f863406c550 0x7f863415da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:42034/0 (socket says 172.21.15.179:42034) 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f86337fe700 1 -- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863415d5e0 msgr2=0x7f86340ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f86337fe700 1 --2- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863415d5e0 0x7f86340ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f86337fe700 1 -- 172.21.15.179:0/2749825477 >> v1:172.21.15.195:6789/0 conn(0x7f863406c550 legacy=0x7f863415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f86337fe700 1 -- 172.21.15.179:0/2749825477 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863406d970 con 0x7f863415d200 2024-05-08T22:44:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f86337fe700 1 --2- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340ce2e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8624009910 tx=0x7f8624009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501fbd02e6e684a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8632ffd700 1 -- 172.21.15.179:0/2749825477 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624004230 con 0x7f863415d200 2024-05-08T22:44:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8632ffd700 1 -- 172.21.15.179:0/2749825477 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8624004390 con 0x7f863415d200 2024-05-08T22:44:46.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8632ffd700 1 -- 172.21.15.179:0/2749825477 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240036c0 con 0x7f863415d200 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 msgr2=0x7f86340ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 --2- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340ce2e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8624009910 tx=0x7f8624009c20 comp rx=0 tx=0).stop 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- 172.21.15.179:0/2749825477 shutdown_connections 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 --2- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863415d5e0 0x7f86340ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 --2- 172.21.15.179:0/2749825477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340ce2e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- 172.21.15.179:0/2749825477 >> 172.21.15.179:0/2749825477 conn(0x7f86340b18d0 msgr2=0x7f86340b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- 172.21.15.179:0/2749825477 shutdown_connections 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- 172.21.15.179:0/2749825477 wait complete. 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 Processor -- start 2024-05-08T22:44:46.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.735+0000 7f8639d25700 1 -- start start 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863406c550 0x7f86341f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341f65d0 con 0x7f863415d5e0 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340c9c60 con 0x7f863415d200 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340c8f70 con 0x7f863406c550 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.740 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60356/0 (socket says 172.21.15.179:60356) 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 -- 172.21.15.179:0/2661717249 learned_addr learned my addr 172.21.15.179:0/2661717249 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f86337fe700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863406c550 0x7f86341f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f862affd700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 -- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863406c550 msgr2=0x7f86341f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863406c550 0x7f86341f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 -- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 msgr2=0x7f86340c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 -- 172.21.15.179:0/2661717249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624009570 con 0x7f863415d5e0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8633fff700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f862c001f80 tx=0x7f862c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c00ac50 con 0x7f863415d5e0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c00adb0 con 0x7f863415d5e0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0072f0 con 0x7f863415d5e0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340c83f0 con 0x7f863415d5e0 2024-05-08T22:44:46.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86340c89b0 con 0x7f863415d5e0 2024-05-08T22:44:46.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f862c007450 con 0x7f863415d5e0 2024-05-08T22:44:46.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8634073290 con 0x7f863415d5e0 2024-05-08T22:44:46.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.739+0000 7f8630ff9700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 0x7f8614068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:46.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.743+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f862c083ce0 con 0x7f863415d5e0 2024-05-08T22:44:46.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.743+0000 7f86337fe700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 0x7f8614068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:46.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.743+0000 7f86337fe700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 0x7f8614068bf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8624009910 tx=0x7f8624008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:46.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.751+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f862c050e90 con 0x7f863415d5e0 2024-05-08T22:44:46.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.919+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f863406d610 con 0x7f863415d5e0 2024-05-08T22:44:46.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.919+0000 7f8630ff9700 1 -- 172.21.15.179:0/2661717249 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f862c0180a0 con 0x7f863415d5e0 2024-05-08T22:44:46.921 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:46.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 msgr2=0x7f8614068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 0x7f8614068bf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8624009910 tx=0x7f8624008030 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 msgr2=0x7f863415b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f862c001f80 tx=0x7f862c00a340 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 shutdown_connections 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8614066730 0x7f8614068bf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f863415d200 0x7f86340c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f863415d5e0 0x7f863415b5a0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 --2- 172.21.15.179:0/2661717249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f863406c550 0x7f86341f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 >> 172.21.15.179:0/2661717249 conn(0x7f86340b18d0 msgr2=0x7f86340cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 shutdown_connections 2024-05-08T22:44:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:46.923+0000 7f8639d25700 1 -- 172.21.15.179:0/2661717249 wait complete. 2024-05-08T22:44:46.941 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:44:47.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 Processor -- start 2024-05-08T22:44:47.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- start start 2024-05-08T22:44:47.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd00815f0c0 0x7fd0080b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 0x7fd0080cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080b37a0 con 0x7fd0080ad9d0 2024-05-08T22:44:47.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0080b3a80 con 0x7fd00815f0c0 2024-05-08T22:44:47.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080b3d60 con 0x7fd0080b2fb0 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00d1b7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd00815f0c0 0x7fd0080b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00d9b8700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd0080ad9d0 0x7fd00815e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:42038/0 (socket says 172.21.15.179:42038) 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00d9b8700 1 -- 172.21.15.179:0/158506804 learned_addr learned my addr 172.21.15.179:0/158506804 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 0x7fd0080cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 -- 172.21.15.179:0/158506804 >> v2:172.21.15.195:3300/0 conn(0x7fd00815f0c0 msgr2=0x7fd0080b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 --2- 172.21.15.179:0/158506804 >> v2:172.21.15.195:3300/0 conn(0x7fd00815f0c0 0x7fd0080b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 -- 172.21.15.179:0/158506804 >> v1:172.21.15.195:6789/0 conn(0x7fd0080ad9d0 legacy=0x7fd00815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 -- 172.21.15.179:0/158506804 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080cb1d0 con 0x7fd0080b2fb0 2024-05-08T22:44:47.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e1b9700 1 --2- 172.21.15.179:0/158506804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 0x7fd0080cb9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd000009d90 tx=0x7fd00000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6f6b376817c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00c9b6700 1 -- 172.21.15.179:0/158506804 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000e070 con 0x7fd0080b2fb0 2024-05-08T22:44:47.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00c9b6700 1 -- 172.21.15.179:0/158506804 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000bb20 con 0x7fd0080b2fb0 2024-05-08T22:44:47.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00c9b6700 1 -- 172.21.15.179:0/158506804 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000012430 con 0x7fd0080b2fb0 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- 172.21.15.179:0/158506804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 msgr2=0x7fd0080cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/158506804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 0x7fd0080cb9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd000009d90 tx=0x7fd00000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- 172.21.15.179:0/158506804 shutdown_connections 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/158506804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080b2fb0 0x7fd0080cb9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/158506804 >> v2:172.21.15.195:3300/0 conn(0x7fd00815f0c0 0x7fd0080b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- 172.21.15.179:0/158506804 >> 172.21.15.179:0/158506804 conn(0x7fd0080ce960 msgr2=0x7fd0080ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- 172.21.15.179:0/158506804 shutdown_connections 2024-05-08T22:44:47.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.055+0000 7fd00e9ba700 1 -- 172.21.15.179:0/158506804 wait complete. 2024-05-08T22:44:47.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 Processor -- start 2024-05-08T22:44:47.060 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- start start 2024-05-08T22:44:47.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ad9d0 0x7fd008141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 0x7fd00815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081f1630 con 0x7fd00815f0c0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081f09b0 con 0x7fd0080b2fb0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081f0c90 con 0x7fd0080ad9d0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60362/0 (socket says 172.21.15.179:60362) 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 -- 172.21.15.179:0/274986307 learned_addr learned my addr 172.21.15.179:0/274986307 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00d9b8700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ad9d0 0x7fd008141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00d1b7700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 0x7fd00815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 -- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ad9d0 msgr2=0x7fd008141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ad9d0 0x7fd008141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 -- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 msgr2=0x7fd00815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 0x7fd00815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 -- 172.21.15.179:0/274986307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd000009a70 con 0x7fd00815f0c0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00d1b7700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 0x7fd00815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e1b9700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fd000014040 tx=0x7fd000004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000012430 con 0x7fd00815f0c0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd000020070 con 0x7fd00815f0c0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0081f0f70 con 0x7fd00815f0c0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000004280 con 0x7fd00815f0c0 2024-05-08T22:44:47.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.059+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd008142370 con 0x7fd00815f0c0 2024-05-08T22:44:47.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0081f1770 con 0x7fd00815f0c0 2024-05-08T22:44:47.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0000043e0 con 0x7fd00815f0c0 2024-05-08T22:44:47.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fcffe7fc700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 0x7fcfec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd000022070 con 0x7fd00815f0c0 2024-05-08T22:44:47.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fd00d9b8700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 0x7fcfec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.063+0000 7fd00d9b8700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 0x7fcfec068c20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd004005930 tx=0x7fd004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.071+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd0081f1770 con 0x7fd00815f0c0 2024-05-08T22:44:47.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.255+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd0081f1770 con 0x7fd00815f0c0 2024-05-08T22:44:47.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.255+0000 7fcffe7fc700 1 -- 172.21.15.179:0/274986307 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd0081f1770 con 0x7fd00815f0c0 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 msgr2=0x7fcfec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 0x7fcfec068c20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd004005930 tx=0x7fd004009040 comp rx=0 tx=0).stop 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 msgr2=0x7fd00815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fd000014040 tx=0x7fd000004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 shutdown_connections 2024-05-08T22:44:47.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fcfec066760 0x7fcfec068c20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0080b2fb0 0x7fd00815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd00815f0c0 0x7fd00815ca90 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 --2- 172.21.15.179:0/274986307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0080ad9d0 0x7fd008141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 >> 172.21.15.179:0/274986307 conn(0x7fd0080ce960 msgr2=0x7fd0080ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 shutdown_connections 2024-05-08T22:44:47.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.259+0000 7fd00e9ba700 1 -- 172.21.15.179:0/274986307 wait complete. 2024-05-08T22:44:47.262 INFO:teuthology.orchestra.run.smithi179.stderr:listed 0 entries 2024-05-08T22:44:47.272 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:44:47.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:47.272 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:47.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[52791]: fuse finished with error 0 and tester_r 0 2024-05-08T22:44:47.288 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:44:47.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:44:47.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:47.354 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:44:47.360 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:47.360 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:47.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:44:47.409 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi179.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:47.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:44:47.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:47.409 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:44:47.416 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T22:44:47.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:44:47.417 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:44:47.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 Processor -- start 2024-05-08T22:44:47.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 -- start start 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea3c15d1e0 0x7fea3c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 0x7fea3c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c15d5c0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c0d1640 con 0x7fea3c0d29b0 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea40841700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0d1910 con 0x7fea3c0b8a40 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea3c15d1e0 0x7fea3c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea3c15d1e0 0x7fea3c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60370/0 (socket says 172.21.15.179:60370) 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 -- 172.21.15.179:0/3097573441 learned_addr learned my addr 172.21.15.179:0/3097573441 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3b59e700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.538 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3a59c700 1 --2- 172.21.15.179:0/3097573441 >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 0x7fea3c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 -- 172.21.15.179:0/3097573441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 msgr2=0x7fea3c0b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 -- 172.21.15.179:0/3097573441 >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 msgr2=0x7fea3c0b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/3097573441 >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 0x7fea3c0b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 -- 172.21.15.179:0/3097573441 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0d1bf0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3a59c700 1 --2- 172.21.15.179:0/3097573441 >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 0x7fea3c0b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0d1100 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0d1090 tx=0x7fea2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f360808ebd65d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea39d9b700 1 -- 172.21.15.179:0/3097573441 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400e070 con 0x7fea3c15d1e0 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea39d9b700 1 -- 172.21.15.179:0/3097573441 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2400b9c0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.535+0000 7fea39d9b700 1 -- 172.21.15.179:0/3097573441 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400bd70 con 0x7fea3c15d1e0 2024-05-08T22:44:47.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- 172.21.15.179:0/3097573441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 msgr2=0x7fea3c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0d1100 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0d1090 tx=0x7fea2400b330 comp rx=0 tx=0).stop 2024-05-08T22:44:47.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- 172.21.15.179:0/3097573441 shutdown_connections 2024-05-08T22:44:47.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- 172.21.15.179:0/3097573441 >> v2:172.21.15.195:3300/0 conn(0x7fea3c0d29b0 0x7fea3c0b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- 172.21.15.179:0/3097573441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0d1100 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- 172.21.15.179:0/3097573441 >> 172.21.15.179:0/3097573441 conn(0x7fea3c0d3400 msgr2=0x7fea3c0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- 172.21.15.179:0/3097573441 shutdown_connections 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- 172.21.15.179:0/3097573441 wait complete. 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 Processor -- start 2024-05-08T22:44:47.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- start start 2024-05-08T22:44:47.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea3c0d29b0 0x7fea3c0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0ccf10 con 0x7fea3c15d1e0 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c159580 con 0x7fea3c0d29b0 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea40841700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0cc750 con 0x7fea3c0b8a40 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea3ad9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea3b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60382/0 (socket says 172.21.15.179:60382) 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea3b59e700 1 -- 172.21.15.179:0/711049531 learned_addr learned my addr 172.21.15.179:0/711049531 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.539+0000 7fea3a59c700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea3c0d29b0 0x7fea3c0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 -- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 msgr2=0x7fea3c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 -- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea3c0d29b0 msgr2=0x7fea3c0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea3c0d29b0 0x7fea3c0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 -- 172.21.15.179:0/711049531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea24009b30 con 0x7fea3c15d1e0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea3b59e700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e810 tx=0x7fea2c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00c070 con 0x7fea3c15d1e0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c013630 con 0x7fea3c15d1e0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c0ca600 con 0x7fea3c15d1e0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0139e0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c0ca320 con 0x7fea3c15d1e0 2024-05-08T22:44:47.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c0cc5b0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea2c013b40 con 0x7fea3c15d1e0 2024-05-08T22:44:47.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.543+0000 7fea337fe700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.547+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fea2c083e60 con 0x7fea3c15d1e0 2024-05-08T22:44:47.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.547+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.547+0000 7fea3ad9d700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b370 tx=0x7fea24009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.551+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fea3c0cc5b0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.755+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fea3c0cc5b0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.759 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:44:47.759+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:44:47.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.771+0000 7fea337fe700 1 -- 172.21.15.179:0/711049531 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7fea3c0cc5b0 con 0x7fea3c15d1e0 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 msgr2=0x7fea1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b370 tx=0x7fea24009000 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 msgr2=0x7fea3c0c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e810 tx=0x7fea2c00eb20 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 shutdown_connections 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea3c0d29b0 0x7fea3c0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3c15d1e0 0x7fea3c0c9010 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 --2- 172.21.15.179:0/711049531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea3c0b8a40 0x7fea3c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 >> 172.21.15.179:0/711049531 conn(0x7fea3c0d3400 msgr2=0x7fea3c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 shutdown_connections 2024-05-08T22:44:47.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.775+0000 7fea40841700 1 -- 172.21.15.179:0/711049531 wait complete. 2024-05-08T22:44:47.777 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:44:47.777 INFO:tasks.ceph.mds.d.smithi195.stderr: -7407> 2024-05-08T22:44:28.155+0000 7fb124ad3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:44:47.790 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:44:47.792 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:44:47.815 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-05-08T22:44:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:44:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-05-08T22:44:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-05-08T22:44:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:44:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-05-08T22:44:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-05-08T22:44:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-05-08T22:44:47.830 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:44:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.834 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-08T22:44:47.834 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-08T22:44:47.834 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:44:47.834 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-08T22:44:47.834 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:44:47.835 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.836 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:44:47.837 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-08T22:44:47.838 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.839 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:44:47.840 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-08T22:44:47.841 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-08T22:44:47.842 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.843 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-08T22:44:47.843 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.843 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:44:47.844 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-08T22:44:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.846 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-08T22:44:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = True 2024-05-08T22:44:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-08T22:44:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = False 2024-05-08T22:44:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:44:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.849 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:44:47.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-08T22:44:47.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-08T22:44:47.916 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-08T22:44:47.917 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.918 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.919 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.920 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-08T22:44:47.921 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.922 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:44:47.923 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:44:47.924 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.925 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:44:47.926 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-08T22:44:47.926 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:47.926 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-08T22:44:47.926 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-08T22:44:47.926 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-08T22:44:47.927 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-08T22:44:47.928 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-08T22:44:47.929 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = False 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = False 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-05-08T22:44:47.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-05-08T22:44:47.935 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:44:47.936 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-05-08T22:44:47.937 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-05-08T22:44:47.938 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:44:47.939 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.940 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-05-08T22:44:47.941 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-05-08T22:44:47.942 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:44:47.942 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-05-08T22:44:47.942 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-05-08T22:44:47.942 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:44:47.942 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:44:47.943 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-05-08T22:44:47.944 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.945 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-05-08T22:44:47.946 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = False 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:44:47.947 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:44:47.947 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-08T22:44:47.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 Processor -- start 2024-05-08T22:44:47.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 -- start start 2024-05-08T22:44:47.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e96e5a80 0x55b6e9788620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e9789700 0x55b6e9786080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e96452f0 con 0x55b6e96b21e0 2024-05-08T22:44:47.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e96018e0 con 0x55b6e9789700 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9601580 con 0x55b6e96e5a80 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bda87a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bda87a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60388/0 (socket says 172.21.15.179:60388) 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bda87a700 1 -- 172.21.15.179:0/346120597 learned_addr learned my addr 172.21.15.179:0/346120597 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bdb07b700 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e9789700 0x55b6e9786080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.991+0000 7f2bda079700 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e96e5a80 0x55b6e9788620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 -- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e96e5a80 msgr2=0x55b6e9788620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e96e5a80 0x55b6e9788620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 -- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e9789700 msgr2=0x55b6e9786080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e9789700 0x55b6e9786080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 -- 172.21.15.179:0/346120597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc00a100 con 0x55b6e96b21e0 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bda87a700 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00a910 tx=0x7f2bcc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8010509d5df9b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:47.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bd9077700 1 -- 172.21.15.179:0/346120597 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc012660 con 0x55b6e96b21e0 2024-05-08T22:44:47.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bd9077700 1 -- 172.21.15.179:0/346120597 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc0127c0 con 0x55b6e96b21e0 2024-05-08T22:44:47.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bd9077700 1 -- 172.21.15.179:0/346120597 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc012b30 con 0x55b6e96b21e0 2024-05-08T22:44:47.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- 172.21.15.179:0/346120597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 msgr2=0x55b6e96e1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00a910 tx=0x7f2bcc005d80 comp rx=0 tx=0).stop 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- 172.21.15.179:0/346120597 shutdown_connections 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e9789700 0x55b6e9786080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e96e5a80 0x55b6e9788620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/346120597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e96e1960 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- 172.21.15.179:0/346120597 >> 172.21.15.179:0/346120597 conn(0x55b6e96b8740 msgr2=0x55b6e96fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- 172.21.15.179:0/346120597 shutdown_connections 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- 172.21.15.179:0/346120597 wait complete. 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 Processor -- start 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- start start 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e9782a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:47.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e96e5a80 0x55b6e97838a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e9789700 0x55b6e9783de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e95d78d0 con 0x55b6e96b21e0 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e96e1960 con 0x55b6e96e5a80 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.995+0000 7f2bdcd45140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9784420 con 0x55b6e9789700 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bda87a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e9782a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bda87a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e9782a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60400/0 (socket says 172.21.15.179:60400) 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bda87a700 1 -- 172.21.15.179:0/1414051482 learned_addr learned my addr 172.21.15.179:0/1414051482 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e9789700 0x55b6e9783de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bda079700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e96e5a80 0x55b6e97838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 -- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e96e5a80 msgr2=0x55b6e97838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6e96e5a80 0x55b6e97838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 -- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 msgr2=0x55b6e9782a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e9782a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 -- 172.21.15.179:0/1414051482 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc00a4e0 con 0x55b6e9789700 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bda87a700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6e96b21e0 0x55b6e9782a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdb07b700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e9789700 0x55b6e9783de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b6e96b1d10 tx=0x7f2bd4002fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bd3fff700 1 -- 172.21.15.179:0/1414051482 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4008690 con 0x55b6e9789700 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd4005de0 con 0x55b6e9789700 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bd3fff700 1 -- 172.21.15.179:0/1414051482 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4012860 con 0x55b6e9789700 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bd3fff700 1 -- 172.21.15.179:0/1414051482 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4012c10 con 0x55b6e9789700 2024-05-08T22:44:48.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:47.999+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6e95efb00 con 0x55b6e9789700 2024-05-08T22:44:48.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bd3fff700 1 -- 172.21.15.179:0/1414051482 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bd40303f0 con 0x55b6e9789700 2024-05-08T22:44:48.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bd3fff700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bbc065090 0x7f2bbc067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bd3fff700 1 -- 172.21.15.179:0/1414051482 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2bd4084a90 con 0x55b6e9789700 2024-05-08T22:44:48.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bda87a700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bbc065090 0x7f2bbc067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55b6e9785340 0x55b6e96f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b6e96f6810 con 0x55b6e9785340 2024-05-08T22:44:48.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bda079700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55b6e9785340 0x55b6e96f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bda079700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55b6e9785340 0x55b6e96f62d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.003+0000 7f2bda87a700 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bbc065090 0x7f2bbc067550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005dc0 tx=0x7f2bcc012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bda079700 1 -- 172.21.15.179:0/1414051482 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v128'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2bc4011790 con 0x55b6e9785340 2024-05-08T22:44:48.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55b6e9785340 msgr2=0x55b6e96f62d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55b6e9785340 0x55b6e96f62d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bbc065090 msgr2=0x7f2bbc067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2bbc065090 0x7f2bbc067550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005dc0 tx=0x7f2bcc012000 comp rx=0 tx=0).stop 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e9789700 msgr2=0x55b6e9783de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 --2- 172.21.15.179:0/1414051482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b6e9789700 0x55b6e9783de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b6e96b1d10 tx=0x7f2bd4002fb0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdb07b700 1 -- 172.21.15.179:0/1414051482 reap_dead start 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 shutdown_connections 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 >> 172.21.15.179:0/1414051482 conn(0x55b6e96b8740 msgr2=0x55b6e9786ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 shutdown_connections 2024-05-08T22:44:48.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.011+0000 7f2bdcd45140 1 -- 172.21.15.179:0/1414051482 wait complete. 2024-05-08T22:44:48.018 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:44:48.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.191+0000 7f6d3d582700 1 Processor -- start 2024-05-08T22:44:48.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.191+0000 7f6d3d582700 1 -- start start 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d380b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380b2f20 con 0x7f6d3806b440 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380b3200 con 0x7f6d3806e760 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b5250 con 0x7f6d3806ebf0 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d37fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6d3806b440 0x7f6d3815b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60078/0 (socket says 172.21.15.179:60078) 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d37fff700 1 -- 172.21.15.179:0/1147888669 learned_addr learned my addr 172.21.15.179:0/1147888669 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:48.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3cd81700 1 --2- 172.21.15.179:0/1147888669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d380b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963271737 0 0) 0x7f6d380b2f20 con 0x7f6d3806b440 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d240035d0 con 0x7f6d3806b440 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3cd81700 1 -- 172.21.15.179:0/1147888669 >> v1:172.21.15.195:6789/0 conn(0x7f6d3806e760 legacy=0x7f6d3806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628723494 0 0) 0x7f6d380b3200 con 0x7f6d3806e760 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3cd81700 1 -- 172.21.15.179:0/1147888669 >> v1:172.21.15.179:6789/0 conn(0x7f6d3806b440 legacy=0x7f6d3815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.196 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3cd81700 1 -- 172.21.15.179:0/1147888669 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380b34e0 con 0x7f6d3806ebf0 2024-05-08T22:44:48.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3cd81700 1 --2- 172.21.15.179:0/1147888669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d380b3fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009dc0 tx=0x7f6d3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b5df4315d42a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e070 con 0x7f6d3806ebf0 2024-05-08T22:44:48.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3000bb20 con 0x7f6d3806ebf0 2024-05-08T22:44:48.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d377fe700 1 -- 172.21.15.179:0/1147888669 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000bed0 con 0x7f6d3806ebf0 2024-05-08T22:44:48.197 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- 172.21.15.179:0/1147888669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 msgr2=0x7f6d380b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 --2- 172.21.15.179:0/1147888669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d380b3fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009dc0 tx=0x7f6d3000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- 172.21.15.179:0/1147888669 shutdown_connections 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 --2- 172.21.15.179:0/1147888669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d380b3fc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- 172.21.15.179:0/1147888669 >> 172.21.15.179:0/1147888669 conn(0x7f6d380b18d0 msgr2=0x7f6d380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- 172.21.15.179:0/1147888669 shutdown_connections 2024-05-08T22:44:48.198 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.195+0000 7f6d3d582700 1 -- 172.21.15.179:0/1147888669 wait complete. 2024-05-08T22:44:48.199 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 Processor -- start 2024-05-08T22:44:48.200 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 -- start start 2024-05-08T22:44:48.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3806e760 0x7f6d380c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d3813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.202 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380c6b30 con 0x7f6d3806b440 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381f0720 con 0x7f6d3806e760 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3d582700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380c6570 con 0x7f6d3806ebf0 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60414/0 (socket says 172.21.15.179:60414) 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d37fff700 1 -- 172.21.15.179:0/107676803 learned_addr learned my addr 172.21.15.179:0/107676803 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d3cd81700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d3813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.199+0000 7f6d2ffff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3806e760 0x7f6d380c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 -- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 msgr2=0x7f6d3813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d3813f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 -- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3806e760 msgr2=0x7f6d380c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3806e760 0x7f6d380c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 -- 172.21.15.179:0/107676803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009a70 con 0x7f6d3806b440 2024-05-08T22:44:48.203 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d37fff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6d28001200 tx=0x7f6d2800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28005d90 con 0x7f6d3806b440 2024-05-08T22:44:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800ce80 con 0x7f6d3806b440 2024-05-08T22:44:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d380c6850 con 0x7f6d3806b440 2024-05-08T22:44:48.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28007310 con 0x7f6d3806b440 2024-05-08T22:44:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d380c75f0 con 0x7f6d3806b440 2024-05-08T22:44:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804e700 con 0x7f6d3806b440 2024-05-08T22:44:48.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d28007470 con 0x7f6d3806b440 2024-05-08T22:44:48.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 0x7f6d18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.203+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6d28083600 con 0x7f6d3806b440 2024-05-08T22:44:48.206 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.207+0000 7f6d2ffff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 0x7f6d18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.207+0000 7f6d2ffff700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 0x7f6d18068bf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6d3815a1f0 tx=0x7f6d20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.211+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6d28050860 con 0x7f6d3806b440 2024-05-08T22:44:48.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.419+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6d3815b9a0 con 0x7f6d3806b440 2024-05-08T22:44:48.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.775+0000 7f6d357fa700 1 -- 172.21.15.179:0/107676803 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v92) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6d28053e80 con 0x7f6d3806b440 2024-05-08T22:44:48.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 msgr2=0x7f6d18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 0x7f6d18068bf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6d3815a1f0 tx=0x7f6d20009040 comp rx=0 tx=0).stop 2024-05-08T22:44:48.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 msgr2=0x7f6d38159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6d28001200 tx=0x7f6d2800c7b0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 shutdown_connections 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6d18066730 0x7f6d18068bf0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d3806e760 0x7f6d380c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d3806ebf0 0x7f6d3813f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 --2- 172.21.15.179:0/107676803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d3806b440 0x7f6d38159310 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 >> 172.21.15.179:0/107676803 conn(0x7f6d380b18d0 msgr2=0x7f6d38140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 shutdown_connections 2024-05-08T22:44:48.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.779+0000 7f6d3d582700 1 -- 172.21.15.179:0/107676803 wait complete. 2024-05-08T22:44:48.782 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:44:48.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.907+0000 7fe44f1c3700 1 Processor -- start 2024-05-08T22:44:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.907+0000 7fe44f1c3700 1 -- start start 2024-05-08T22:44:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.907+0000 7fe44f1c3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 0x7fe4480b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe4480ce580 0x7fe44806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44806afc0 con 0x7fe4480b2120 2024-05-08T22:44:48.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44815d990 con 0x7fe4480ce580 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4480d26f0 con 0x7fe44815d5b0 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe4480ce580 0x7fe44806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe4480ce580 0x7fe44806b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60420/0 (socket says 172.21.15.179:60420) 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 -- 172.21.15.179:0/2734800644 learned_addr learned my addr 172.21.15.179:0/2734800644 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e1c1700 1 --2- 172.21.15.179:0/2734800644 >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 0x7fe4480b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 -- 172.21.15.179:0/2734800644 >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 msgr2=0x7fe4480b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:48.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 --2- 172.21.15.179:0/2734800644 >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 0x7fe4480b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 -- 172.21.15.179:0/2734800644 >> v1:172.21.15.195:6789/0 conn(0x7fe4480b2120 legacy=0x7fe4480cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 -- 172.21.15.179:0/2734800644 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480d2130 con 0x7fe4480ce580 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e1c1700 1 --2- 172.21.15.179:0/2734800644 >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 0x7fe4480b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44e9c2700 1 --2- 172.21.15.179:0/2734800644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4480ce580 0x7fe44806b450 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe44808a630 tx=0x7fe44000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668012f62abb0e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44d1bf700 1 -- 172.21.15.179:0/2734800644 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000e070 con 0x7fe4480ce580 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44d1bf700 1 -- 172.21.15.179:0/2734800644 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000b9c0 con 0x7fe4480ce580 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44d1bf700 1 -- 172.21.15.179:0/2734800644 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000bd70 con 0x7fe4480ce580 2024-05-08T22:44:48.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- 172.21.15.179:0/2734800644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4480ce580 msgr2=0x7fe44806b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/2734800644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4480ce580 0x7fe44806b450 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe44808a630 tx=0x7fe44000b330 comp rx=0 tx=0).stop 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- 172.21.15.179:0/2734800644 shutdown_connections 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/2734800644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4480ce580 0x7fe44806b450 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/2734800644 >> v2:172.21.15.195:3300/0 conn(0x7fe44815d5b0 0x7fe4480b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- 172.21.15.179:0/2734800644 >> 172.21.15.179:0/2734800644 conn(0x7fe4480b4940 msgr2=0x7fe4480b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- 172.21.15.179:0/2734800644 shutdown_connections 2024-05-08T22:44:48.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.911+0000 7fe44f1c3700 1 -- 172.21.15.179:0/2734800644 wait complete. 2024-05-08T22:44:48.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 Processor -- start 2024-05-08T22:44:48.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- start start 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4480b2120 0x7fe448159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4480ce580 0x7fe4481559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44815b170 con 0x7fe44815d5b0 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44815a560 con 0x7fe4480b2120 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44815a840 con 0x7fe4480ce580 2024-05-08T22:44:48.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60426/0 (socket says 172.21.15.179:60426) 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 -- 172.21.15.179:0/1035062202 learned_addr learned my addr 172.21.15.179:0/1035062202 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44d9c0700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4480ce580 0x7fe4481559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e1c1700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4480b2120 0x7fe448159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 -- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4480ce580 msgr2=0x7fe4481559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4480ce580 0x7fe4481559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 -- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4480b2120 msgr2=0x7fe448159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:48.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4480b2120 0x7fe448159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 -- 172.21.15.179:0/1035062202 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009b30 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44e9c2700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe440012000 tx=0x7fe440004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400099b0 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440004250 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44815ab20 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440004600 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.915+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4480b4600 con 0x7fe44815d5b0 2024-05-08T22:44:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44815b2e0 con 0x7fe44815d5b0 2024-05-08T22:44:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe440004760 con 0x7fe44815d5b0 2024-05-08T22:44:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe43effd700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 0x7fe42c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe4400857c0 con 0x7fe44815d5b0 2024-05-08T22:44:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe44e1c1700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 0x7fe42c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.919+0000 7fe44e1c1700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 0x7fe42c068bf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ab60 tx=0x7fe438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:48.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:48.927+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe440052a20 con 0x7fe44815d5b0 2024-05-08T22:44:49.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.131+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe44804e700 con 0x7fe44815d5b0 2024-05-08T22:44:49.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.131+0000 7fe43effd700 1 -- 172.21.15.179:0/1035062202 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4750 (secure 0 0 0) 0x7fe440056040 con 0x7fe44815d5b0 2024-05-08T22:44:49.134 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:49.134 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:48.776138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9625":{"gid":9625,"name":"d","rank":0,"incarnation":93,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6835/3161101014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3161101014},{"type":"v1","addr":"172.21.15.195:6835","nonce":3161101014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:44:49.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 msgr2=0x7fe42c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:49.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 0x7fe42c068bf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ab60 tx=0x7fe438008040 comp rx=0 tx=0).stop 2024-05-08T22:44:49.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 msgr2=0x7fe4480b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:49.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe440012000 tx=0x7fe440004af0 comp rx=0 tx=0).stop 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 shutdown_connections 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe42c066730 0x7fe42c068bf0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe44815d5b0 0x7fe4480b62b0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4480ce580 0x7fe4481559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 --2- 172.21.15.179:0/1035062202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4480b2120 0x7fe448159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 >> 172.21.15.179:0/1035062202 conn(0x7fe4480b4940 msgr2=0x7fe4480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 shutdown_connections 2024-05-08T22:44:49.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:49.135+0000 7fe44f1c3700 1 -- 172.21.15.179:0/1035062202 wait complete. 2024-05-08T22:44:49.140 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 93 2024-05-08T22:44:49.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:44:48.776138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9625': {'gid': 9625, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/3161101014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3161101014}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 3161101014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:44:49.552 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9625:up:replay 2024-05-08T22:44:50.552 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:50.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 Processor -- start 2024-05-08T22:44:50.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 -- start start 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b4c1434d0 0x7f0b4c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0b4c06dc20 0x7f0b4c06e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c0ce2a0 0x7f0b4c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0cd200 con 0x7f0b4c06dc20 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0cd4e0 con 0x7f0b4c1434d0 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52c65700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0cd7c0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0b4c06dc20 0x7f0b4c06e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0b4c06dc20 0x7f0b4c06e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60430/0 (socket says 172.21.15.179:60430) 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 -- 172.21.15.179:0/1953928164 learned_addr learned my addr 172.21.15.179:0/1953928164 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b52464700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c0ce2a0 0x7f0b4c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51c63700 1 --2- 172.21.15.179:0/1953928164 >> v2:172.21.15.195:3300/0 conn(0x7f0b4c1434d0 0x7f0b4c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 -- 172.21.15.179:0/1953928164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c0ce2a0 msgr2=0x7f0b4c06eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c0ce2a0 0x7f0b4c06eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 -- 172.21.15.179:0/1953928164 >> v2:172.21.15.195:3300/0 conn(0x7f0b4c1434d0 msgr2=0x7f0b4c06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 --2- 172.21.15.179:0/1953928164 >> v2:172.21.15.195:3300/0 conn(0x7f0b4c1434d0 0x7f0b4c06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 -- 172.21.15.179:0/1953928164 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0cdaa0 con 0x7f0b4c06dc20 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b51462700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c06dc20 0x7f0b4c06e640 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c070800 tx=0x7f0b3c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed25560b27d64591 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b50c61700 1 -- 172.21.15.179:0/1953928164 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00b8a0 con 0x7f0b4c06dc20 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b50c61700 1 -- 172.21.15.179:0/1953928164 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c00ba00 con 0x7f0b4c06dc20 2024-05-08T22:44:50.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.675+0000 7f0b50c61700 1 -- 172.21.15.179:0/1953928164 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00bd90 con 0x7f0b4c06dc20 2024-05-08T22:44:50.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- 172.21.15.179:0/1953928164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c06dc20 msgr2=0x7f0b4c06e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:50.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c06dc20 0x7f0b4c06e640 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c070800 tx=0x7f0b3c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- 172.21.15.179:0/1953928164 shutdown_connections 2024-05-08T22:44:50.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c0ce2a0 0x7f0b4c06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1953928164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c06dc20 0x7f0b4c06e640 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1953928164 >> v2:172.21.15.195:3300/0 conn(0x7f0b4c1434d0 0x7f0b4c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- 172.21.15.179:0/1953928164 >> 172.21.15.179:0/1953928164 conn(0x7f0b4c0ce9d0 msgr2=0x7f0b4c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:50.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- 172.21.15.179:0/1953928164 shutdown_connections 2024-05-08T22:44:50.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- 172.21.15.179:0/1953928164 wait complete. 2024-05-08T22:44:50.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 Processor -- start 2024-05-08T22:44:50.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.679+0000 7f0b52c65700 1 -- start start 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c1434d0 0x7f0b4c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c0af2f0 0x7f0b4c158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0af7a0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1f6800 con 0x7f0b4c0af2f0 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0afa00 con 0x7f0b4c1434d0 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60432/0 (socket says 172.21.15.179:60432) 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51462700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c1434d0 0x7f0b4c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 -- 172.21.15.179:0/1226406207 learned_addr learned my addr 172.21.15.179:0/1226406207 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:50.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52464700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c0af2f0 0x7f0b4c158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 -- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c1434d0 msgr2=0x7f0b4c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c1434d0 0x7f0b4c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 -- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c0af2f0 msgr2=0x7f0b4c158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c0af2f0 0x7f0b4c158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 -- 172.21.15.179:0/1226406207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009570 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b51c63700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000e760 tx=0x7f0b4000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000c070 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40013610 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0afce0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40013980 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.683+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0b02a0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c067740 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b40013ae0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b4a7fc700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 0x7f0b38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:50.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0b40083fc0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b51462700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 0x7f0b38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:50.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.687+0000 7f0b51462700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 0x7f0b38068bf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:50.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.695+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b400b5cf0 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.899+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b4c0c9240 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b4a7fc700 1 -- 172.21.15.179:0/1226406207 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f0b40051220 con 0x7f0b4c0ce2a0 2024-05-08T22:44:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:50.903 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:50.783268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9625":{"gid":9625,"name":"d","rank":0,"incarnation":93,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.195:6835/3161101014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3161101014},{"type":"v1","addr":"172.21.15.195:6835","nonce":3161101014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 msgr2=0x7f0b38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 0x7f0b38068bf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c008030 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 msgr2=0x7f0b4c1f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000e760 tx=0x7f0b4000ea70 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 shutdown_connections 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0b38066730 0x7f0b38068bf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c0af2f0 0x7f0b4c158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b4c1434d0 0x7f0b4c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 --2- 172.21.15.179:0/1226406207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b4c0ce2a0 0x7f0b4c1f5ca0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 >> 172.21.15.179:0/1226406207 conn(0x7f0b4c0ce9d0 msgr2=0x7f0b4c0b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 shutdown_connections 2024-05-08T22:44:50.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:50.903+0000 7f0b52c65700 1 -- 172.21.15.179:0/1226406207 wait complete. 2024-05-08T22:44:50.907 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 95 2024-05-08T22:44:51.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:44:50.783268+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9625': {'gid': 9625, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.195:6835/3161101014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3161101014}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 3161101014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:44:51.321 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9625:up:rejoin 2024-05-08T22:44:52.322 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:44:52.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.447+0000 7f72b19fd700 1 Processor -- start 2024-05-08T22:44:52.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.447+0000 7f72b19fd700 1 -- start start 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 0x7f72ac06e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72ac0ce2a0 0x7f72ac06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0cd200 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0cd4e0 con 0x7f72ac06dc20 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0cd7c0 con 0x7f72ac1434d0 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72ac0ce2a0 0x7f72ac06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72ac0ce2a0 0x7f72ac06eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60446/0 (socket says 172.21.15.179:60446) 2024-05-08T22:44:52.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 -- 172.21.15.179:0/3962594694 learned_addr learned my addr 172.21.15.179:0/3962594694 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b09fb700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72abfff700 1 --2- 172.21.15.179:0/3962594694 >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 0x7f72ac06e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 -- 172.21.15.179:0/3962594694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 msgr2=0x7f72ac06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 -- 172.21.15.179:0/3962594694 >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 msgr2=0x7f72ac06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 --2- 172.21.15.179:0/3962594694 >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 0x7f72ac06e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 -- 172.21.15.179:0/3962594694 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac0cdaa0 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72abfff700 1 --2- 172.21.15.179:0/3962594694 >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 0x7f72ac06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b11fc700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac06eb80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f72ac06b410 tx=0x7f72a400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bb3e2276ee05ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72ab7fe700 1 -- 172.21.15.179:0/3962594694 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010070 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72ab7fe700 1 -- 172.21.15.179:0/3962594694 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a400dce0 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72ab7fe700 1 -- 172.21.15.179:0/3962594694 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a400b310 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- 172.21.15.179:0/3962594694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 msgr2=0x7f72ac06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac06eb80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f72ac06b410 tx=0x7f72a400d430 comp rx=0 tx=0).stop 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- 172.21.15.179:0/3962594694 shutdown_connections 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac06eb80 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3962594694 >> v2:172.21.15.195:3300/0 conn(0x7f72ac06dc20 0x7f72ac06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3962594694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- 172.21.15.179:0/3962594694 >> 172.21.15.179:0/3962594694 conn(0x7f72ac0ce9d0 msgr2=0x7f72ac0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:52.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- 172.21.15.179:0/3962594694 shutdown_connections 2024-05-08T22:44:52.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.451+0000 7f72b19fd700 1 -- 172.21.15.179:0/3962594694 wait complete. 2024-05-08T22:44:52.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 Processor -- start 2024-05-08T22:44:52.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 -- start start 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac13cd70 0x7f72ac0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac15afe0 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac1ee770 con 0x7f72ac13cd70 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b19fd700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0c55e0 con 0x7f72ac1434d0 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60460/0 (socket says 172.21.15.179:60460) 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 -- 172.21.15.179:0/3099584535 learned_addr learned my addr 172.21.15.179:0/3099584535 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:52.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b11fc700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac13cd70 0x7f72ac0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72abfff700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 -- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 msgr2=0x7f72ac159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 -- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac13cd70 msgr2=0x7f72ac0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac13cd70 0x7f72ac0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 -- 172.21.15.179:0/3099584535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a400bd40 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.455+0000 7f72b09fb700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f729400e840 tx=0x7f729400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940095a0 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294013680 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac0c4a60 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940139f0 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac0c5020 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac067740 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7294013b50 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 0x7f728c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:52.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f7294084140 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.459+0000 7f72abfff700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 0x7f728c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:52.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.463+0000 7f72abfff700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 0x7f728c068bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f72ac15ac70 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:52.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.463+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7294051310 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72ac04e700 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72a97fa700 1 -- 172.21.15.179:0/3099584535 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f7294054930 con 0x7f72ac0ce2a0 2024-05-08T22:44:52.675 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:52.675 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:51.785543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9625":{"gid":9625,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/3161101014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3161101014},{"type":"v1","addr":"172.21.15.195:6835","nonce":3161101014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 msgr2=0x7f728c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 0x7f728c068bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f72ac15ac70 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 msgr2=0x7f72ac0c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f729400e840 tx=0x7f729400eb50 comp rx=0 tx=0).stop 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 shutdown_connections 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f728c066730 0x7f728c068bf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72ac13cd70 0x7f72ac0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72ac1434d0 0x7f72ac159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 --2- 172.21.15.179:0/3099584535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ac0ce2a0 0x7f72ac0c4360 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 >> 172.21.15.179:0/3099584535 conn(0x7f72ac0ce9d0 msgr2=0x7f72ac140f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 shutdown_connections 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:52.675+0000 7f72b19fd700 1 -- 172.21.15.179:0/3099584535 wait complete. 2024-05-08T22:44:52.679 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 96 2024-05-08T22:44:53.035 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:44:53.038 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:44:53.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:44:51.785543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9625': {'gid': 9625, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6835/3161101014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3161101014}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 3161101014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:44:53.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:44:53.093 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:44:53.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.215+0000 7ff77b159700 1 Processor -- start 2024-05-08T22:44:53.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- start start 2024-05-08T22:44:53.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 0x7ff7740b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff77406bb50 0x7ff774159650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77415d950 con 0x7ff77415cf70 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77415dc30 con 0x7ff77406bb50 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b1af0 con 0x7ff774159270 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779956700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff77415cf70 0x7ff77406b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:60090/0 (socket says 172.21.15.179:60090) 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779956700 1 -- 172.21.15.179:0/2697412552 learned_addr learned my addr 172.21.15.179:0/2697412552 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 --2- 172.21.15.179:0/2697412552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 0x7ff7740b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a958700 1 --2- 172.21.15.179:0/2697412552 >> v2:172.21.15.195:3300/0 conn(0x7ff77406bb50 0x7ff774159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779155700 1 -- 172.21.15.179:0/2697412552 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271734036 0 0) 0x7ff77415d950 con 0x7ff77415cf70 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779155700 1 -- 172.21.15.179:0/2697412552 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff75c0035d0 con 0x7ff77415cf70 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 -- 172.21.15.179:0/2697412552 >> v2:172.21.15.195:3300/0 conn(0x7ff77406bb50 msgr2=0x7ff774159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 --2- 172.21.15.179:0/2697412552 >> v2:172.21.15.195:3300/0 conn(0x7ff77406bb50 0x7ff774159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 -- 172.21.15.179:0/2697412552 >> v1:172.21.15.179:6789/0 conn(0x7ff77415cf70 legacy=0x7ff77406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 -- 172.21.15.179:0/2697412552 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740b1dd0 con 0x7ff774159270 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77a157700 1 --2- 172.21.15.179:0/2697412552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 0x7ff7740b22f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff768009850 tx=0x7ff768009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8f33b454dd6e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779155700 1 -- 172.21.15.179:0/2697412552 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800f940 con 0x7ff774159270 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779155700 1 -- 172.21.15.179:0/2697412552 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800faa0 con 0x7ff774159270 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff779155700 1 -- 172.21.15.179:0/2697412552 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800fe50 con 0x7ff774159270 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- 172.21.15.179:0/2697412552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 msgr2=0x7ff7740b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 --2- 172.21.15.179:0/2697412552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 0x7ff7740b22f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff768009850 tx=0x7ff768009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:53.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- 172.21.15.179:0/2697412552 shutdown_connections 2024-05-08T22:44:53.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 --2- 172.21.15.179:0/2697412552 >> v2:172.21.15.195:3300/0 conn(0x7ff77406bb50 0x7ff774159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 --2- 172.21.15.179:0/2697412552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff774159270 0x7ff7740b22f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- 172.21.15.179:0/2697412552 >> 172.21.15.179:0/2697412552 conn(0x7ff7740ce960 msgr2=0x7ff7740ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.250 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- 172.21.15.179:0/2697412552 shutdown_connections 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.219+0000 7ff77b159700 1 -- 172.21.15.179:0/2697412552 wait complete. 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 Processor -- start 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 -- start start 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff774159270 0x7ff77414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.251 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff77415cf70 0x7ff77414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415bc80 con 0x7ff77406bb50 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415b000 con 0x7ff774159270 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415b2e0 con 0x7ff77415cf70 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60462/0 (socket says 172.21.15.179:60462) 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 -- 172.21.15.179:0/338549861 learned_addr learned my addr 172.21.15.179:0/338549861 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff779956700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff774159270 0x7ff77414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a958700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff77415cf70 0x7ff77414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 -- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff77415cf70 msgr2=0x7ff77414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff77415cf70 0x7ff77414ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 -- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff774159270 msgr2=0x7ff77414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff774159270 0x7ff77414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 -- 172.21.15.179:0/338549861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680094b0 con 0x7ff77406bb50 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77a157700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff768019040 tx=0x7ff768004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff772ffd700 1 -- 172.21.15.179:0/338549861 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7680038c0 con 0x7ff77406bb50 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.223+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77415b5c0 con 0x7ff77406bb50 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff772ffd700 1 -- 172.21.15.179:0/338549861 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768003a40 con 0x7ff77406bb50 2024-05-08T22:44:53.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff772ffd700 1 -- 172.21.15.179:0/338549861 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768003df0 con 0x7ff77406bb50 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77414c930 con 0x7ff77406bb50 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff772ffd700 1 -- 172.21.15.179:0/338549861 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff76802b460 con 0x7ff77406bb50 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff772ffd700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 0x7ff760068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff772ffd700 1 -- 172.21.15.179:0/338549861 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff768013070 con 0x7ff77406bb50 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff779956700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 0x7ff760068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff77f66a740 1 Processor -- start 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.227+0000 7ff77f66a740 1 -- start start 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 0x7ff760068bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff7740b0010 tx=0x7ff7640019f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1ebeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebf420 0x1e869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de5d80 con 0x1eb5be0 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de3510 con 0x1ebeb00 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ddee00 con 0x1ebf420 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60478/0 (socket says 172.21.15.179:60478) 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 -- 172.21.15.179:0/1617435257 learned_addr learned my addr 172.21.15.179:0/1617435257 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77a157700 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1ebeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77a958700 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebf420 0x1e869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 -- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebf420 msgr2=0x1e869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebf420 0x1e869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 -- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 msgr2=0x1ebeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1ebeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 -- 172.21.15.179:0/1617435257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c51960 con 0x1eb5be0 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff779956700 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff764007910 tx=0x7ff764012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6361c96c44582b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff75bfff700 1 -- 172.21.15.179:0/1617435257 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764014760 con 0x1eb5be0 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff75bfff700 1 -- 172.21.15.179:0/1617435257 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640148c0 con 0x1eb5be0 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff75bfff700 1 -- 172.21.15.179:0/1617435257 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764014c30 con 0x1eb5be0 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- 172.21.15.179:0/1617435257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 msgr2=0x1e86470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff764007910 tx=0x7ff764012000 comp rx=0 tx=0).stop 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- 172.21.15.179:0/1617435257 shutdown_connections 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebf420 0x1e869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e86470 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 --2- 172.21.15.179:0/1617435257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1ebeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- 172.21.15.179:0/1617435257 >> 172.21.15.179:0/1617435257 conn(0x1c064d0 msgr2=0x1ea0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- 172.21.15.179:0/1617435257 shutdown_connections 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.231+0000 7ff77f66a740 1 -- 172.21.15.179:0/1617435257 wait complete. 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- start start 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 Processor -- start 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1e77380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1ec7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d9c730 con 0x1ebeb00 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da25b0 con 0x1ebf420 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c8aaa0 con 0x1eb5be0 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 10 client.0 _ll_register_callbacks cb 0x1dd8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60488/0 (socket says 172.21.15.179:60488) 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 -- 172.21.15.179:0/490217713 learned_addr learned my addr 172.21.15.179:0/490217713 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77a157700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1e77380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77a958700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1ec7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 -- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 msgr2=0x1e77380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1e77380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 -- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 msgr2=0x1ec7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1ec7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 -- 172.21.15.179:0/490217713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0035d0 con 0x1ebeb00 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77a958700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1ec7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77a157700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1e77380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff779956700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff764001e30 tx=0x7ff764001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6ac6292446f183 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640206a0 con 0x1ebeb00 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff76401c040 con 0x1ebeb00 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c131e0 con 0x1ebeb00 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff764020800 con 0x1ebeb00 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 10 client.9993 fetch_fsmap learned FSMap version 96 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x1da5220 con 0x1ebeb00 2024-05-08T22:44:53.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c131e0 con 0x1ebeb00 2024-05-08T22:44:53.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff764012070 con 0x1ebeb00 2024-05-08T22:44:53.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 10 client.9993 fetch_fsmap finished waiting for FSMap version 96 2024-05-08T22:44:53.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 10 client.9993 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:44:53.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 20 client.9993 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:53.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1eb7140 0x1eb7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 4 client.9993 mds_command: new command op to 9625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.235+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 --> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c5d460 con 0x1eb7140 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77a157700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1eb7140 0x1eb7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff7597fa700 10 client.9993 ms_handle_connect on v2:172.21.15.195:6834/3161101014 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77a157700 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1eb7140 0x1eb7520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345121bba3b76ba9 server_cookie=ef03394dbd450034 in_seq=0 out_seq=0 2024-05-08T22:44:53.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff7597fa700 1 -- 172.21.15.179:0/490217713 <== mds.0 v2:172.21.15.195:6834/3161101014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x1c5d460 con 0x1eb7140 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff7597fa700 10 client.9993 handle_command_reply: tid=0 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 client.9993 shutdown 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 msgr2=0x1ec4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff764001e30 tx=0x7ff764001250 comp rx=0 tx=0).stop 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 shutdown_connections 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1eb7140 0x1eb7520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1ec7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff7597fa700 0 client.9993 ms_handle_reset on v2:172.21.15.195:6834/3161101014 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebeb00 0x1ec4d70 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 --2- 172.21.15.179:0/490217713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1e77380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 >> 172.21.15.179:0/490217713 conn(0x1c064d0 msgr2=0x1e9f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 shutdown_connections 2024-05-08T22:44:53.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 1 -- 172.21.15.179:0/490217713 wait complete. 2024-05-08T22:44:53.270 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.239+0000 7ff77f66a740 20 client.9993 trim_cache size 0 max 16384 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 Processor -- start 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- start start 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 0x1e87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1e88500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d9c730 con 0x1eb5be0 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ddd940 con 0x1ebf420 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d7d470 con 0x1ebeb00 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60500/0 (socket says 172.21.15.179:60500) 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 -- 172.21.15.179:0/2478643108 learned_addr learned my addr 172.21.15.179:0/2478643108 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff779956700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 0x1e87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a958700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1e88500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 -- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 msgr2=0x1e87fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 0x1e87fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 -- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 msgr2=0x1e88500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1e88500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 -- 172.21.15.179:0/2478643108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d8b450 con 0x1eb5be0 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff779956700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 0x1e87fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77a157700 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff768009280 tx=0x7ff76805d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e168ec1dd94433f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff770dae700 1 -- 172.21.15.179:0/2478643108 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76805af00 con 0x1eb5be0 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff770dae700 1 -- 172.21.15.179:0/2478643108 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76805f4c0 con 0x1eb5be0 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff770dae700 1 -- 172.21.15.179:0/2478643108 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7680750b0 con 0x1eb5be0 2024-05-08T22:44:53.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- 172.21.15.179:0/2478643108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 msgr2=0x1e76910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff768009280 tx=0x7ff76805d040 comp rx=0 tx=0).stop 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- 172.21.15.179:0/2478643108 shutdown_connections 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebf420 0x1e88500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1ebeb00 0x1e87fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 --2- 172.21.15.179:0/2478643108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1eb5be0 0x1e76910 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- 172.21.15.179:0/2478643108 >> 172.21.15.179:0/2478643108 conn(0x1c064d0 msgr2=0x1ebe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- 172.21.15.179:0/2478643108 shutdown_connections 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- 172.21.15.179:0/2478643108 wait complete. 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 -- start start 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.263+0000 7ff77f66a740 1 Processor -- start 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1edc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1edc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e0dec0 con 0x1ebf420 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da94f0 con 0x1ebeb00 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1edd380 con 0x1eb5be0 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 10 client.0 _ll_register_callbacks cb 0x7ff764017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60506/0 (socket says 172.21.15.179:60506) 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 -- 172.21.15.179:0/3684282863 learned_addr learned my addr 172.21.15.179:0/3684282863 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff779956700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1edc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a157700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1edc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 -- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 msgr2=0x1edc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1edc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 -- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 msgr2=0x1edc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1edc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 -- 172.21.15.179:0/3684282863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77414c930 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a958700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff76c018700 tx=0x7ff76c018a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcea1f47e9b2b860 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c016850 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e798a0 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff76c022c90 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76c022e50 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 10 client.9999 fetch_fsmap learned FSMap version 96 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x1d7f720 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e798a0 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff76c0169b0 con 0x1ebf420 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 10 client.9999 fetch_fsmap finished waiting for FSMap version 96 2024-05-08T22:44:53.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 10 client.9999 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 20 client.9999 populate_metadata read hostname 'smithi179' 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1ec3600 0x1ec39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 4 client.9999 mds_command: new command op to 9625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 --> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff7640148c0 con 0x1ec3600 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.267+0000 7ff77a157700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1ec3600 0x1ec39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff7597fa700 10 client.9999 ms_handle_connect on v2:172.21.15.195:6834/3161101014 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77a157700 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1ec3600 0x1ec39e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e8b2f708d8ca95 server_cookie=b1d6fd7aebddfa2a in_seq=0 out_seq=0 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff7597fa700 1 -- 172.21.15.179:0/3684282863 <== mds.0 v2:172.21.15.195:6834/3161101014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7ff7640148c0 con 0x1ec3600 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff7597fa700 10 client.9999 handle_command_reply: tid=0 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 client.9999 shutdown 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 msgr2=0x1edce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff76c018700 tx=0x7ff76c018a10 comp rx=0 tx=0).stop 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 shutdown_connections 2024-05-08T22:44:53.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1ec3600 0x1ec39e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x1ebeb00 0x1edc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff7597fa700 0 client.9999 ms_handle_reset on v2:172.21.15.195:6834/3161101014 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x1ebf420 0x1edce40 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 --2- 172.21.15.179:0/3684282863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x1eb5be0 0x1edc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 >> 172.21.15.179:0/3684282863 conn(0x1c064d0 msgr2=0x1ece2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 shutdown_connections 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 1 -- 172.21.15.179:0/3684282863 wait complete. 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77f66a740 20 client.9999 trim_cache size 0 max 16384 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:53.290 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9625, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.4885940900000003, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 96, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 128, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 128, 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 5.4531982670000003 2024-05-08T22:44:53.291 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 msgr2=0x7ff760068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 0x7ff760068bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff7740b0010 tx=0x7ff7640019f0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 msgr2=0x7ff77415c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff768019040 tx=0x7ff768004770 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 shutdown_connections 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff760066730 0x7ff760068bf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff774159270 0x7ff77414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff77415cf70 0x7ff77414ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 --2- 172.21.15.179:0/338549861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77406bb50 0x7ff77415c0a0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 >> 172.21.15.179:0/338549861 conn(0x7ff7740ce960 msgr2=0x7ff77406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 shutdown_connections 2024-05-08T22:44:53.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.271+0000 7ff77b159700 1 -- 172.21.15.179:0/338549861 wait complete. 2024-05-08T22:44:53.294 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:44:53.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.415+0000 7f0a97fff700 1 Processor -- start 2024-05-08T22:44:53.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- start start 2024-05-08T22:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 0x7f0a980cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9806abb0 con 0x7f0a980cb510 2024-05-08T22:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980cb9a0 con 0x7f0a980cc700 2024-05-08T22:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9806ae90 con 0x7f0a9815b1b0 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a96ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 0x7f0a980cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a96ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60508/0 (socket says 172.21.15.179:60508) 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a96ffd700 1 -- 172.21.15.179:0/1243969276 learned_addr learned my addr 172.21.15.179:0/1243969276 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/1243969276 >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 msgr2=0x7f0a980ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 --2- 172.21.15.179:0/1243969276 >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/1243969276 >> v1:172.21.15.195:6789/0 conn(0x7f0a980cb510 legacy=0x7f0a9815e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/1243969276 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9815b9c0 con 0x7f0a9815b1b0 2024-05-08T22:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a96ffd700 1 --2- 172.21.15.179:0/1243969276 >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a8e7fc700 1 --2- 172.21.15.179:0/1243969276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 0x7f0a980cafd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009850 tx=0x7f0a80009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4e24e0c9789d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a967fc700 1 -- 172.21.15.179:0/1243969276 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011830 con 0x7f0a9815b1b0 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a967fc700 1 -- 172.21.15.179:0/1243969276 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a80011990 con 0x7f0a9815b1b0 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a967fc700 1 -- 172.21.15.179:0/1243969276 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011d40 con 0x7f0a9815b1b0 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- 172.21.15.179:0/1243969276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 msgr2=0x7f0a980cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 --2- 172.21.15.179:0/1243969276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 0x7f0a980cafd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009850 tx=0x7f0a80009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- 172.21.15.179:0/1243969276 shutdown_connections 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 --2- 172.21.15.179:0/1243969276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a9815b1b0 0x7f0a980cafd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 --2- 172.21.15.179:0/1243969276 >> v2:172.21.15.179:3300/0 conn(0x7f0a980cc700 0x7f0a980ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- 172.21.15.179:0/1243969276 >> 172.21.15.179:0/1243969276 conn(0x7f0a980ce9d0 msgr2=0x7f0a980cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- 172.21.15.179:0/1243969276 shutdown_connections 2024-05-08T22:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.419+0000 7f0a97fff700 1 -- 172.21.15.179:0/1243969276 wait complete. 2024-05-08T22:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 Processor -- start 2024-05-08T22:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 -- start start 2024-05-08T22:44:53.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a980cb510 0x7f0a980ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a9815b1b0 0x7f0a9806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980af660 con 0x7f0a980cc700 2024-05-08T22:44:53.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980aeb90 con 0x7f0a9815b1b0 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a97fff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9814d390 con 0x7f0a980cb510 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60514/0 (socket says 172.21.15.179:60514) 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/2847319038 learned_addr learned my addr 172.21.15.179:0/2847319038 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a977fe700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a9815b1b0 0x7f0a9806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a96ffd700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a980cb510 0x7f0a980ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a980cb510 msgr2=0x7f0a980ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a980cb510 0x7f0a980ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a9815b1b0 msgr2=0x7f0a9806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a9815b1b0 0x7f0a9806dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 -- 172.21.15.179:0/2847319038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88009570 con 0x7f0a980cc700 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.423+0000 7f0a8e7fc700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806afd0 tx=0x7f0a80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004480 con 0x7f0a980cc700 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8001f070 con 0x7f0a980cc700 2024-05-08T22:44:53.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a800094b0 con 0x7f0a980cc700 2024-05-08T22:44:53.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80003900 con 0x7f0a980cc700 2024-05-08T22:44:53.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9814e830 con 0x7f0a980cc700 2024-05-08T22:44:53.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98067740 con 0x7f0a980cc700 2024-05-08T22:44:53.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a80021030 con 0x7f0a980cc700 2024-05-08T22:44:53.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.427+0000 7f0a8ffff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 0x7f0a78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:53.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.431+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0a80085c90 con 0x7f0a980cc700 2024-05-08T22:44:53.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.431+0000 7f0a96ffd700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 0x7f0a78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:53.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.431+0000 7f0a96ffd700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 0x7f0a78068bf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006d50 tx=0x7f0a880092e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:53.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.435+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a80052ff0 con 0x7f0a980cc700 2024-05-08T22:44:53.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.639+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a9804e700 con 0x7f0a980cc700 2024-05-08T22:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.639+0000 7f0a8ffff700 1 -- 172.21.15.179:0/2847319038 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a80056610 con 0x7f0a980cc700 2024-05-08T22:44:53.642 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:44:53.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 msgr2=0x7f0a78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 0x7f0a78068bf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006d50 tx=0x7f0a880092e0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 msgr2=0x7f0a9814ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:53.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806afd0 tx=0x7f0a80004770 comp rx=0 tx=0).stop 2024-05-08T22:44:53.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 shutdown_connections 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a78066730 0x7f0a78068bf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a9815b1b0 0x7f0a9806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a980cc700 0x7f0a9814ce50 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 --2- 172.21.15.179:0/2847319038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a980cb510 0x7f0a980ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 >> 172.21.15.179:0/2847319038 conn(0x7f0a980ce9d0 msgr2=0x7f0a9815c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 shutdown_connections 2024-05-08T22:44:53.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:53.643+0000 7f0a97fff700 1 -- 172.21.15.179:0/2847319038 wait complete. 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:44:53.656 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:53.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.657 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:44:53.664 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.665 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41878sec preferred_lft 41878sec 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:44:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:44:53.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.668 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:44:53.679 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:44:53.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.681 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:44:53.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:44:53.732 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:44:53.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.773 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:44:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:44:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:44:53.780 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:53.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:44:53.784 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:44:53.779+0000 7f8d407f6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:44:53.792 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:44:53.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:53.792 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:53.799 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:44:53.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:44:53.800 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:53.804 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:53.861 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:53.862 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:53.874 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:53.875 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:53.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.876 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:53.881 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:44:53.881 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:44:53.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:53.901 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:53.901 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:53.958 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:53.958 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:53.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:53.959 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:53.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:44:53.967+0000 7f44377dc640 -1 init, newargv = 0x563b391ffc40 newargc=15 2024-05-08T22:44:53.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[53415]: starting ceph client 2024-05-08T22:44:53.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[53415]: starting fuse 2024-05-08T22:44:55.066 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:44:55.084 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:44:55.085 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:44:55.096 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:44:55.098 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:44:55.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:55.098 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:44:55.149 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:44:55.149 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.149 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:44:55.150 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:44:55.151 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:44:55.151 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:44:55.151 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.151 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:44:55.151 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:55.238 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.53415.asok 2024-05-08T22:44:55.421 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53415.asok 2024-05-08T22:44:55.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:55.421 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53415.asok status 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:44:55.524 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "53415", 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 10011, 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 10011 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:55.525 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 425870314 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 425870314 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.10011 192.168.0.1:0/425870314", 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/425870314", 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 96, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 128, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:44:55.526 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:44:55.527 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:44:55.527 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:44:55.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:55.534 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:44:55.534 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:55.539 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:44:55.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:55.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:55.540 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:44:55.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:44:55.554 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./ 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/charlie 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/alpha 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/unflushed 2024-05-08T22:44:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/unflushed/jfile 2024-05-08T22:44:55.586 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:55.586 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:55.587 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:55.587 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.587 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T22:44:55.587 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:55.698 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:44:55.886 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:55.887 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:55.887 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:55.887 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:55.887 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T22:44:55.887 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:55.952 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T22:44:56.142 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:56.210 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T22:44:56.397 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:56.511 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628280 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T22:44:56.700 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:56.768 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627778 2024-05-08T22:44:56.958 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:56.958 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:56.958 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:56.958 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:56.959 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T22:44:56.959 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:57.028 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627779 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T22:44:57.218 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:44:57.283 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627780 2024-05-08T22:44:57.471 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:57.471 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:57.521 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:57.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.634+0000 7fd328e59700 1 Processor -- start 2024-05-08T22:44:57.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.634+0000 7fd328e59700 1 -- start start 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd32415d0c0 0x7fd3240ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd3240cea50 0x7fd32406ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd3240b4c90 0x7fd32406efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3240b3f60 con 0x7fd3240cea50 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32406f510 con 0x7fd3240b4c90 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3240ceed0 con 0x7fd32415d0c0 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd3240cea50 0x7fd32406ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd3240cea50 0x7fd32406ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37562/0 (socket says 172.21.15.179:37562) 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 -- 172.21.15.179:0/492979329 learned_addr learned my addr 172.21.15.179:0/492979329 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd323fff700 1 --2- 172.21.15.179:0/492979329 >> v2:172.21.15.195:3300/0 conn(0x7fd3240b4c90 0x7fd32406efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd3237fe700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd32415d0c0 0x7fd3240ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 -- 172.21.15.179:0/492979329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd32415d0c0 msgr2=0x7fd3240ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd32415d0c0 0x7fd3240ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 -- 172.21.15.179:0/492979329 >> v2:172.21.15.195:3300/0 conn(0x7fd3240b4c90 msgr2=0x7fd32406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 --2- 172.21.15.179:0/492979329 >> v2:172.21.15.195:3300/0 conn(0x7fd3240b4c90 0x7fd32406efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 -- 172.21.15.179:0/492979329 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240b41b0 con 0x7fd3240cea50 2024-05-08T22:44:57.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd31affd700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3240cea50 0x7fd32406ea90 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fd324160d50 tx=0x7fd30c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd5654ea88a3942 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd322ffd700 1 -- 172.21.15.179:0/492979329 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00b8a0 con 0x7fd3240cea50 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd322ffd700 1 -- 172.21.15.179:0/492979329 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c00ba00 con 0x7fd3240cea50 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd322ffd700 1 -- 172.21.15.179:0/492979329 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00bd90 con 0x7fd3240cea50 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- 172.21.15.179:0/492979329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3240cea50 msgr2=0x7fd32406ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3240cea50 0x7fd32406ea90 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fd324160d50 tx=0x7fd30c009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- 172.21.15.179:0/492979329 shutdown_connections 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- 172.21.15.179:0/492979329 >> v2:172.21.15.195:3300/0 conn(0x7fd3240b4c90 0x7fd32406efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3240cea50 0x7fd32406ea90 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 --2- 172.21.15.179:0/492979329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd32415d0c0 0x7fd3240ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- 172.21.15.179:0/492979329 >> 172.21.15.179:0/492979329 conn(0x7fd324098a70 msgr2=0x7fd3240b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- 172.21.15.179:0/492979329 shutdown_connections 2024-05-08T22:44:57.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.638+0000 7fd328e59700 1 -- 172.21.15.179:0/492979329 wait complete. 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 Processor -- start 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- start start 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3240b4c90 0x7fd32415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3240cea50 0x7fd3240c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd324160620 con 0x7fd32415d0c0 2024-05-08T22:44:57.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd324160340 con 0x7fd3240cea50 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32415d5d0 con 0x7fd3240b4c90 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37576/0 (socket says 172.21.15.179:37576) 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 -- 172.21.15.179:0/4228247057 learned_addr learned my addr 172.21.15.179:0/4228247057 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd31affd700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3240cea50 0x7fd3240c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd3237fe700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3240b4c90 0x7fd32415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 -- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3240b4c90 msgr2=0x7fd32415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3240b4c90 0x7fd32415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 -- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3240cea50 msgr2=0x7fd3240c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3240cea50 0x7fd3240c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 -- 172.21.15.179:0/4228247057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c009570 con 0x7fd32415d0c0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd323fff700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007d60 tx=0x7fd31c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00ede0 con 0x7fd32415d0c0 2024-05-08T22:44:57.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c014420 con 0x7fd32415d0c0 2024-05-08T22:44:57.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32415d8b0 con 0x7fd32415d0c0 2024-05-08T22:44:57.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0147d0 con 0x7fd32415d0c0 2024-05-08T22:44:57.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.642+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3240c7da0 con 0x7fd32415d0c0 2024-05-08T22:44:57.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32415ddd0 con 0x7fd32415d0c0 2024-05-08T22:44:57.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd31c014930 con 0x7fd32415d0c0 2024-05-08T22:44:57.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd320ff9700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 0x7fd304068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd31c0849c0 con 0x7fd32415d0c0 2024-05-08T22:44:57.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd3237fe700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 0x7fd304068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.646+0000 7fd3237fe700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 0x7fd304068bf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd324160ea0 tx=0x7fd314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.654+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd32415ddd0 con 0x7fd32415d0c0 2024-05-08T22:44:57.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.818+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd32415ddd0 con 0x7fd32415d0c0 2024-05-08T22:44:57.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.822+0000 7fd320ff9700 1 -- 172.21.15.179:0/4228247057 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16495 (secure 0 0 0) 0x7fd32415ddd0 con 0x7fd32415d0c0 2024-05-08T22:44:57.825 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:57.825 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":128,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:47.759716+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 msgr2=0x7fd304068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 0x7fd304068bf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd324160ea0 tx=0x7fd314008040 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 msgr2=0x7fd32415ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007d60 tx=0x7fd31c00e3c0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 shutdown_connections 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd304066730 0x7fd304068bf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd3240cea50 0x7fd3240c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd32415d0c0 0x7fd32415ba30 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 --2- 172.21.15.179:0/4228247057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3240b4c90 0x7fd32415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 >> 172.21.15.179:0/4228247057 conn(0x7fd324098a70 msgr2=0x7fd3240cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 shutdown_connections 2024-05-08T22:44:57.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.826+0000 7fd328e59700 1 -- 172.21.15.179:0/4228247057 wait complete. 2024-05-08T22:44:57.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:57.840 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:57.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 Processor -- start 2024-05-08T22:44:57.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 -- start start 2024-05-08T22:44:57.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c440b19e0 0x7f9c440cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9c44159700 0x7f9c4406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4406cda0 con 0x7f9c44159320 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c440cb4c0 con 0x7f9c44159700 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c4a6d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440b1f10 con 0x7f9c440b19e0 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c49ed1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9c44159700 0x7f9c4406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c49ed1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9c44159700 0x7f9c4406b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37586/0 (socket says 172.21.15.179:37586) 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.958+0000 7f9c48ecf700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c440b19e0 0x7f9c440cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 -- 172.21.15.179:0/1051851921 learned_addr learned my addr 172.21.15.179:0/1051851921 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 -- 172.21.15.179:0/1051851921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c440b19e0 msgr2=0x7f9c440cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 --2- 172.21.15.179:0/1051851921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c440b19e0 0x7f9c440cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 -- 172.21.15.179:0/1051851921 >> v1:172.21.15.195:6789/0 conn(0x7f9c44159320 legacy=0x7f9c440cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 -- 172.21.15.179:0/1051851921 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440b21f0 con 0x7f9c44159700 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c49ed1700 1 --2- 172.21.15.179:0/1051851921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159700 0x7f9c4406b440 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e50 tx=0x7f9c3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d90b78979c1c068 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c3bfff700 1 -- 172.21.15.179:0/1051851921 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00e070 con 0x7f9c44159700 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c3bfff700 1 -- 172.21.15.179:0/1051851921 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00b910 con 0x7f9c44159700 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c3bfff700 1 -- 172.21.15.179:0/1051851921 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bcc0 con 0x7f9c44159700 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/1051851921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159700 msgr2=0x7f9c4406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/1051851921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159700 0x7f9c4406b440 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e50 tx=0x7f9c3c00b330 comp rx=0 tx=0).stop 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/1051851921 shutdown_connections 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/1051851921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c440b19e0 0x7f9c440cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/1051851921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159700 0x7f9c4406b440 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/1051851921 >> 172.21.15.179:0/1051851921 conn(0x7f9c440cea30 msgr2=0x7f9c440cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/1051851921 shutdown_connections 2024-05-08T22:44:57.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/1051851921 wait complete. 2024-05-08T22:44:57.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 Processor -- start 2024-05-08T22:44:57.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.962+0000 7f9c4a6d2700 1 -- start start 2024-05-08T22:44:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c440b19e0 0x7f9c4414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c44159700 0x7f9c4414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441434d0 con 0x7f9c44159320 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441437b0 con 0x7f9c440b19e0 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44143a90 con 0x7f9c44159700 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37596/0 (socket says 172.21.15.179:37596) 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 -- 172.21.15.179:0/814824843 learned_addr learned my addr 172.21.15.179:0/814824843 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c496d0700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c440b19e0 0x7f9c4414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c49ed1700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c44159700 0x7f9c4414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 -- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c44159700 msgr2=0x7f9c4414beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c44159700 0x7f9c4414beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 -- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c440b19e0 msgr2=0x7f9c4414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c440b19e0 0x7f9c4414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:57.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 -- 172.21.15.179:0/814824843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009b30 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c48ecf700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9a0 tx=0x7f9c4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40009720 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4000c690 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c44143d70 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000ca20 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4414de70 con 0x7f9c44159320 2024-05-08T22:44:57.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.966+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4404e700 con 0x7f9c44159320 2024-05-08T22:44:57.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.970+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c4000cb80 con 0x7f9c44159320 2024-05-08T22:44:57.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.970+0000 7f9c39ffb700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 0x7f9c28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:57.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.970+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9c40084340 con 0x7f9c44159320 2024-05-08T22:44:57.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.970+0000 7f9c496d0700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 0x7f9c28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:57.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.970+0000 7f9c496d0700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 0x7f9c28068bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9c440b3c30 tx=0x7f9c34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:57.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:57.978+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9c40051490 con 0x7f9c44159320 2024-05-08T22:44:58.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.146+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c440b2e10 con 0x7f9c44159320 2024-05-08T22:44:58.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.146+0000 7f9c39ffb700 1 -- 172.21.15.179:0/814824843 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c40054ab0 con 0x7f9c44159320 2024-05-08T22:44:58.149 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 msgr2=0x7f9c28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 0x7f9c28068bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9c440b3c30 tx=0x7f9c34009040 comp rx=0 tx=0).stop 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 msgr2=0x7f9c44147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e9a0 tx=0x7f9c4000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 shutdown_connections 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9c28066730 0x7f9c28068bf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c44159700 0x7f9c4414beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c44159320 0x7f9c44147f30 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 --2- 172.21.15.179:0/814824843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c440b19e0 0x7f9c4414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 >> 172.21.15.179:0/814824843 conn(0x7f9c440cea30 msgr2=0x7f9c440cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 shutdown_connections 2024-05-08T22:44:58.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.150+0000 7f9c4a6d2700 1 -- 172.21.15.179:0/814824843 wait complete. 2024-05-08T22:44:58.165 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:58.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 Processor -- start 2024-05-08T22:44:58.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 -- start start 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 0x7f30f80ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f30f815f090 0x7f30f80cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80cb370 con 0x7f30f806e090 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f806e5c0 con 0x7f30f815f090 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe991700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f806e8a0 con 0x7f30f8158850 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f30f815f090 0x7f30f80cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fd18e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f30f806e090 0x7f30f815e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50806/0 (socket says 172.21.15.179:50806) 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fd18e700 1 -- 172.21.15.179:0/979497955 learned_addr learned my addr 172.21.15.179:0/979497955 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:58.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393817515 0 0) 0x7f30f80cb370 con 0x7f30f806e090 2024-05-08T22:44:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e40035d0 con 0x7f30f806e090 2024-05-08T22:44:58.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fd98f700 1 --2- 172.21.15.179:0/979497955 >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 0x7f30f80ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 -- 172.21.15.179:0/979497955 >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 msgr2=0x7f30f80ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1100574696 0 0) 0x7f30e40035d0 con 0x7f30f806e090 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 --2- 172.21.15.179:0/979497955 >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 0x7f30f80ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 -- 172.21.15.179:0/979497955 >> v1:172.21.15.179:6789/0 conn(0x7f30f806e090 legacy=0x7f30f815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 -- 172.21.15.179:0/979497955 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80cb610 con 0x7f30f815f090 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fd98f700 1 --2- 172.21.15.179:0/979497955 >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 0x7f30f80ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fe190700 1 --2- 172.21.15.179:0/979497955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f815f090 0x7f30f80cbb30 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009850 tx=0x7f30f0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8101e4f742e3b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80019f0 con 0x7f30f815f090 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0004030 con 0x7f30f815f090 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.282+0000 7f30fc98d700 1 -- 172.21.15.179:0/979497955 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00043c0 con 0x7f30f815f090 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 -- 172.21.15.179:0/979497955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f815f090 msgr2=0x7f30f80cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 --2- 172.21.15.179:0/979497955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f815f090 0x7f30f80cbb30 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009850 tx=0x7f30f0009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 -- 172.21.15.179:0/979497955 shutdown_connections 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 --2- 172.21.15.179:0/979497955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f815f090 0x7f30f80cbb30 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 --2- 172.21.15.179:0/979497955 >> v2:172.21.15.195:3300/0 conn(0x7f30f8158850 0x7f30f80ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 -- 172.21.15.179:0/979497955 >> 172.21.15.179:0/979497955 conn(0x7f30f80cea30 msgr2=0x7f30f80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 -- 172.21.15.179:0/979497955 shutdown_connections 2024-05-08T22:44:58.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.286+0000 7f30fe991700 1 -- 172.21.15.179:0/979497955 wait complete. 2024-05-08T22:44:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 Processor -- start 2024-05-08T22:44:58.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- start start 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 0x7f30f81418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30f815f090 0x7f30f80a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80a8bc0 con 0x7f30f8158850 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8141de0 con 0x7f30f806e090 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80ab770 con 0x7f30f815f090 2024-05-08T22:44:58.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37612/0 (socket says 172.21.15.179:37612) 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 -- 172.21.15.179:0/3158271507 learned_addr learned my addr 172.21.15.179:0/3158271507 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe190700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30f815f090 0x7f30f80a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd98f700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 0x7f30f81418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 -- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30f815f090 msgr2=0x7f30f80a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30f815f090 0x7f30f80a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 -- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 msgr2=0x7f30f81418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 0x7f30f81418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 -- 172.21.15.179:0/3158271507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00094b0 con 0x7f30f8158850 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd98f700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 0x7f30f81418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fd18e700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f30e8001890 tx=0x7f30e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800ada0 con 0x7f30f8158850 2024-05-08T22:44:58.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e80070d0 con 0x7f30f8158850 2024-05-08T22:44:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8007480 con 0x7f30f8158850 2024-05-08T22:44:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f80a8040 con 0x7f30f8158850 2024-05-08T22:44:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.290+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f80a8600 con 0x7f30f8158850 2024-05-08T22:44:58.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f804e700 con 0x7f30f8158850 2024-05-08T22:44:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f30e80075e0 con 0x7f30f8158850 2024-05-08T22:44:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30f67fc700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 0x7f30d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f30e8083fe0 con 0x7f30f8158850 2024-05-08T22:44:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30fd98f700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 0x7f30d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.294+0000 7f30fd98f700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 0x7f30d8068bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f30f815b010 tx=0x7f30e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.302+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f30e80511b0 con 0x7f30f8158850 2024-05-08T22:44:58.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.466+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30f806c2f0 con 0x7f30f8158850 2024-05-08T22:44:58.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30f67fc700 1 -- 172.21.15.179:0/3158271507 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30e80547d0 con 0x7f30f8158850 2024-05-08T22:44:58.471 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 msgr2=0x7f30d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 0x7f30d8068bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f30f815b010 tx=0x7f30e0008040 comp rx=0 tx=0).stop 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 msgr2=0x7f30f80a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f30e8001890 tx=0x7f30e800a340 comp rx=0 tx=0).stop 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 shutdown_connections 2024-05-08T22:44:58.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f30d8066730 0x7f30d8068bf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30f815f090 0x7f30f80a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8158850 0x7f30f80a74d0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 --2- 172.21.15.179:0/3158271507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30f806e090 0x7f30f81418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 >> 172.21.15.179:0/3158271507 conn(0x7f30f80cea30 msgr2=0x7f30f81437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 shutdown_connections 2024-05-08T22:44:58.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.470+0000 7f30fe991700 1 -- 172.21.15.179:0/3158271507 wait complete. 2024-05-08T22:44:58.486 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:44:58.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 Processor -- start 2024-05-08T22:44:58.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 -- start start 2024-05-08T22:44:58.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd9f00b4a30 0x7fd9f00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f015f1f0 con 0x7fd9f00aea70 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f015f4c0 con 0x7fd9f00ae690 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f5066700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f00b4610 con 0x7fd9f00b4a30 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9ef7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd9f00aea70 0x7fd9f006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50816/0 (socket says 172.21.15.179:50816) 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9effff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd9f00ae690 0x7fd9f015eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43594/0 (socket says 172.21.15.179:43594) 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9ef7fe700 1 -- 172.21.15.179:0/493367753 learned_addr learned my addr 172.21.15.179:0/493367753 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.602+0000 7fd9f4865700 1 --2- 172.21.15.179:0/493367753 >> v2:172.21.15.195:3300/0 conn(0x7fd9f00b4a30 0x7fd9f00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281598644 0 0) 0x7fd9f015f4c0 con 0x7fd9f00ae690 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d80035d0 con 0x7fd9f00ae690 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213345077 0 0) 0x7fd9f015f1f0 con 0x7fd9f00aea70 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f015f4c0 con 0x7fd9f00aea70 2024-05-08T22:44:58.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f4865700 1 -- 172.21.15.179:0/493367753 >> v1:172.21.15.195:6789/0 conn(0x7fd9f00ae690 legacy=0x7fd9f015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f4865700 1 -- 172.21.15.179:0/493367753 >> v1:172.21.15.179:6789/0 conn(0x7fd9f00aea70 legacy=0x7fd9f006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f4865700 1 -- 172.21.15.179:0/493367753 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f015f7a0 con 0x7fd9f00b4a30 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f4865700 1 --2- 172.21.15.179:0/493367753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00b4a30 0x7fd9f00b4e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009d90 tx=0x7fd9e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a20e0c6409d286 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800e070 con 0x7fd9f00b4a30 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e800bb20 con 0x7fd9f00b4a30 2024-05-08T22:44:58.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9eeffd700 1 -- 172.21.15.179:0/493367753 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8012430 con 0x7fd9f00b4a30 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 -- 172.21.15.179:0/493367753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00b4a30 msgr2=0x7fd9f00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 --2- 172.21.15.179:0/493367753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00b4a30 0x7fd9f00b4e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009d90 tx=0x7fd9e800b330 comp rx=0 tx=0).stop 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 -- 172.21.15.179:0/493367753 shutdown_connections 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 --2- 172.21.15.179:0/493367753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00b4a30 0x7fd9f00b4e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 -- 172.21.15.179:0/493367753 >> 172.21.15.179:0/493367753 conn(0x7fd9f0098a70 msgr2=0x7fd9f00b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 -- 172.21.15.179:0/493367753 shutdown_connections 2024-05-08T22:44:58.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.606+0000 7fd9f5066700 1 -- 172.21.15.179:0/493367753 wait complete. 2024-05-08T22:44:58.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 Processor -- start 2024-05-08T22:44:58.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- start start 2024-05-08T22:44:58.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00ae690 0x7fd9f01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9f00b4a30 0x7fd9f00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f01f4fd0 con 0x7fd9f00aea70 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f00c64b0 con 0x7fd9f00ae690 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f00c6790 con 0x7fd9f00b4a30 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37616/0 (socket says 172.21.15.179:37616) 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 -- 172.21.15.179:0/2708604440 learned_addr learned my addr 172.21.15.179:0/2708604440 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f4865700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9f00b4a30 0x7fd9f00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9effff700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00ae690 0x7fd9f01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 -- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9f00b4a30 msgr2=0x7fd9f00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9f00b4a30 0x7fd9f00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 -- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00ae690 msgr2=0x7fd9f01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00ae690 0x7fd9f01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 -- 172.21.15.179:0/2708604440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8009a70 con 0x7fd9f00aea70 2024-05-08T22:44:58.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ef7fe700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4005930 tx=0x7fd9e400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4005d90 con 0x7fd9f00aea70 2024-05-08T22:44:58.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e40070d0 con 0x7fd9f00aea70 2024-05-08T22:44:58.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f00c6a70 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4007480 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.610+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f0158d00 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f00ccd60 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9e40075e0 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9ecff9700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 0x7fd9dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd9e4083860 con 0x7fd9f00aea70 2024-05-08T22:44:58.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9effff700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 0x7fd9dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.614+0000 7fd9effff700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 0x7fd9dc068bf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0007fd0 tx=0x7fd9e0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.622+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd9e4050910 con 0x7fd9f00aea70 2024-05-08T22:44:58.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9f00c6f90 con 0x7fd9f00aea70 2024-05-08T22:44:58.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9ecff9700 1 -- 172.21.15.179:0/2708604440 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd9f00c6f90 con 0x7fd9f00aea70 2024-05-08T22:44:58.792 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:44:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 msgr2=0x7fd9dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 0x7fd9dc068bf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0007fd0 tx=0x7fd9e0007de0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 msgr2=0x7fd9f00cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4005930 tx=0x7fd9e400c9e0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 shutdown_connections 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd9dc066730 0x7fd9dc068bf0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9f00b4a30 0x7fd9f00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9f00aea70 0x7fd9f00cb8b0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.790+0000 7fd9f5066700 1 --2- 172.21.15.179:0/2708604440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9f00ae690 0x7fd9f01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.794+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 >> 172.21.15.179:0/2708604440 conn(0x7fd9f0098a70 msgr2=0x7fd9f0159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.794+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 shutdown_connections 2024-05-08T22:44:58.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.794+0000 7fd9f5066700 1 -- 172.21.15.179:0/2708604440 wait complete. 2024-05-08T22:44:58.807 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:44:58.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 Processor -- start 2024-05-08T22:44:58.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 -- start start 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fceb806e180 0x7fceb806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fceb80b25d0 0x7fceb80b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb80cc340 con 0x7fceb806dda0 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb806b540 con 0x7fceb806e180 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fcebcf35700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb806ac50 con 0x7fceb80b25d0 2024-05-08T22:44:58.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fceb806e180 0x7fceb806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fceb806e180 0x7fceb806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37626/0 (socket says 172.21.15.179:37626) 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 -- 172.21.15.179:0/3334649577 learned_addr learned my addr 172.21.15.179:0/3334649577 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb7fff700 1 --2- 172.21.15.179:0/3334649577 >> v2:172.21.15.195:3300/0 conn(0x7fceb80b25d0 0x7fceb80b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 -- 172.21.15.179:0/3334649577 >> v2:172.21.15.195:3300/0 conn(0x7fceb80b25d0 msgr2=0x7fceb80b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 --2- 172.21.15.179:0/3334649577 >> v2:172.21.15.195:3300/0 conn(0x7fceb80b25d0 0x7fceb80b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 -- 172.21.15.179:0/3334649577 >> v1:172.21.15.195:6789/0 conn(0x7fceb806dda0 legacy=0x7fceb80cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 -- 172.21.15.179:0/3334649577 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb806aef0 con 0x7fceb806e180 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb6ffd700 1 --2- 172.21.15.179:0/3334649577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb806e180 0x7fceb806e5e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fceb8070af0 tx=0x7fcea8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d71fa0a5621433 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb67fc700 1 -- 172.21.15.179:0/3334649577 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8004230 con 0x7fceb806e180 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb67fc700 1 -- 172.21.15.179:0/3334649577 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8004390 con 0x7fceb806e180 2024-05-08T22:44:58.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.918+0000 7fceb67fc700 1 -- 172.21.15.179:0/3334649577 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80036c0 con 0x7fceb806e180 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- 172.21.15.179:0/3334649577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb806e180 msgr2=0x7fceb806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3334649577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb806e180 0x7fceb806e5e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fceb8070af0 tx=0x7fcea8009bf0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- 172.21.15.179:0/3334649577 shutdown_connections 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3334649577 >> v2:172.21.15.195:3300/0 conn(0x7fceb80b25d0 0x7fceb80b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3334649577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb806e180 0x7fceb806e5e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- 172.21.15.179:0/3334649577 >> 172.21.15.179:0/3334649577 conn(0x7fceb80ce960 msgr2=0x7fceb80ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:58.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- 172.21.15.179:0/3334649577 shutdown_connections 2024-05-08T22:44:58.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- 172.21.15.179:0/3334649577 wait complete. 2024-05-08T22:44:58.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 Processor -- start 2024-05-08T22:44:58.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.922+0000 7fcebcf35700 1 -- start start 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fceb806dda0 0x7fceb8156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fceb806e180 0x7fceb8162a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb8161a20 con 0x7fceb80b25d0 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb8161cc0 con 0x7fceb806dda0 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb8161fa0 con 0x7fceb806e180 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37632/0 (socket says 172.21.15.179:37632) 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 -- 172.21.15.179:0/3046483983 learned_addr learned my addr 172.21.15.179:0/3046483983 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb6ffd700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fceb806e180 0x7fceb8162a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb77fe700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fceb806dda0 0x7fceb8156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 -- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fceb806e180 msgr2=0x7fceb8162a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fceb806e180 0x7fceb8162a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 -- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fceb806dda0 msgr2=0x7fceb8156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fceb806dda0 0x7fceb8156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 -- 172.21.15.179:0/3046483983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8009570 con 0x7fceb80b25d0 2024-05-08T22:44:58.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceb7fff700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fceb0007e60 tx=0x7fceb000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb000ed30 con 0x7fceb80b25d0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0014420 con 0x7fceb80b25d0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb8162280 con 0x7fceb80b25d0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb00147d0 con 0x7fceb80b25d0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb8156b50 con 0x7fceb80b25d0 2024-05-08T22:44:58.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.926+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb81627a0 con 0x7fceb80b25d0 2024-05-08T22:44:58.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.930+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fceb0014930 con 0x7fceb80b25d0 2024-05-08T22:44:58.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.930+0000 7fceaffff700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 0x7fce98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:58.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.930+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fceb00847f0 con 0x7fceb80b25d0 2024-05-08T22:44:58.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.930+0000 7fceb77fe700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 0x7fce98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:58.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.930+0000 7fceb77fe700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 0x7fce98068bf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcea0005150 tx=0x7fcea0007ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:58.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:58.938+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fceb0051a50 con 0x7fceb80b25d0 2024-05-08T22:44:59.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.122+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fceb804e700 con 0x7fceb80b25d0 2024-05-08T22:44:59.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.122+0000 7fceaffff700 1 -- 172.21.15.179:0/3046483983 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+124 (secure 0 0 0) 0x7fceb0055070 con 0x7fceb80b25d0 2024-05-08T22:44:59.126 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:44:59.126 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6834/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 msgr2=0x7fce98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 0x7fce98068bf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcea0005150 tx=0x7fcea0007ee0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 msgr2=0x7fceb8163e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fceb0007e60 tx=0x7fceb000e3c0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 shutdown_connections 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fce98066730 0x7fce98068bf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb80b25d0 0x7fceb8163e20 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fceb806e180 0x7fceb8162a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 --2- 172.21.15.179:0/3046483983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fceb806dda0 0x7fceb8156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 >> 172.21.15.179:0/3046483983 conn(0x7fceb80ce960 msgr2=0x7fceb80ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 shutdown_connections 2024-05-08T22:44:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.126+0000 7fcebcf35700 1 -- 172.21.15.179:0/3046483983 wait complete. 2024-05-08T22:44:59.130 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:44:59.141 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:44:59.141 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:44:59.148 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:44:59.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.306+0000 7f10267a8700 1 Processor -- start 2024-05-08T22:44:59.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 -- start start 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10200d18a0 0x7f10200d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f10201613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10200d2640 con 0x7f102015d580 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1020161900 con 0x7f10200d18a0 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102006b3a0 con 0x7f1020160f30 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10200d18a0 0x7f10200d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f10200d18a0 0x7f10200d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37644/0 (socket says 172.21.15.179:37644) 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 -- 172.21.15.179:0/953343218 learned_addr learned my addr 172.21.15.179:0/953343218 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1025fa7700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f10201613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 -- 172.21.15.179:0/953343218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 msgr2=0x7f10201613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:59.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f10201613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 -- 172.21.15.179:0/953343218 >> v1:172.21.15.195:6789/0 conn(0x7f102015d580 legacy=0x7f10200b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 -- 172.21.15.179:0/953343218 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200d2360 con 0x7f10200d18a0 2024-05-08T22:44:59.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1024fa5700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10200d1d30 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1010009910 tx=0x7f1010009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decc1b625906c6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1017fff700 1 -- 172.21.15.179:0/953343218 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011830 con 0x7f10200d18a0 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1017fff700 1 -- 172.21.15.179:0/953343218 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010011990 con 0x7f10200d18a0 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f1017fff700 1 -- 172.21.15.179:0/953343218 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011d40 con 0x7f10200d18a0 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 -- 172.21.15.179:0/953343218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 msgr2=0x7f10200d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.310+0000 7f10267a8700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10200d1d30 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1010009910 tx=0x7f1010009c20 comp rx=0 tx=0).stop 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- 172.21.15.179:0/953343218 shutdown_connections 2024-05-08T22:44:59.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f10201613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 --2- 172.21.15.179:0/953343218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10200d1d30 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- 172.21.15.179:0/953343218 >> 172.21.15.179:0/953343218 conn(0x7f10200d2d90 msgr2=0x7f10200d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- 172.21.15.179:0/953343218 shutdown_connections 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- 172.21.15.179:0/953343218 wait complete. 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 Processor -- start 2024-05-08T22:44:59.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- start start 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102015d580 0x7f10201574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f1020157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200b54d0 con 0x7f10200d18a0 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10201598d0 con 0x7f102015d580 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10267a8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1020159bb0 con 0x7f1020160f30 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10257a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10257a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37656/0 (socket says 172.21.15.179:37656) 2024-05-08T22:44:59.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f10257a6700 1 -- 172.21.15.179:0/2383927077 learned_addr learned my addr 172.21.15.179:0/2383927077 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:59.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f1025fa7700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f1020157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.314+0000 7f1024fa5700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102015d580 0x7f10201574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 -- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102015d580 msgr2=0x7f10201574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102015d580 0x7f10201574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 -- 172.21.15.179:0/2383927077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 msgr2=0x7f10201603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 -- 172.21.15.179:0/2383927077 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009570 con 0x7f1020160f30 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f10257a6700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1025fa7700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f1020157a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f101800eba0 tx=0x7f101800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018009a80 con 0x7f1020160f30 2024-05-08T22:44:59.320 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101800f040 con 0x7f1020160f30 2024-05-08T22:44:59.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1020160920 con 0x7f1020160f30 2024-05-08T22:44:59.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800cc40 con 0x7f1020160f30 2024-05-08T22:44:59.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200b16d0 con 0x7f1020160f30 2024-05-08T22:44:59.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102015a0d0 con 0x7f1020160f30 2024-05-08T22:44:59.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.318+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1018014480 con 0x7f1020160f30 2024-05-08T22:44:59.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.322+0000 7f1015ffb700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 0x7f1004068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.322+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1018084da0 con 0x7f1020160f30 2024-05-08T22:44:59.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.322+0000 7f10257a6700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 0x7f1004068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.322+0000 7f10257a6700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 0x7f1004068bf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f101c001ce0 tx=0x7f101c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.330 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.330+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1018051f50 con 0x7f1020160f30 2024-05-08T22:44:59.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.498+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f102015a0d0 con 0x7f1020160f30 2024-05-08T22:44:59.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.502+0000 7f1015ffb700 1 -- 172.21.15.179:0/2383927077 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16495 (secure 0 0 0) 0x7f102015a0d0 con 0x7f1020160f30 2024-05-08T22:44:59.505 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:44:59.505 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":128,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:44:47.759716+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 msgr2=0x7f1004068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 0x7f1004068bf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f101c001ce0 tx=0x7f101c009040 comp rx=0 tx=0).stop 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 msgr2=0x7f1020157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f1020157a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f101800eba0 tx=0x7f101800eeb0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 shutdown_connections 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1004066730 0x7f1004068bf0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102015d580 0x7f10201574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1020160f30 0x7f1020157a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 --2- 172.21.15.179:0/2383927077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10200d18a0 0x7f10201603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 >> 172.21.15.179:0/2383927077 conn(0x7f10200d2d90 msgr2=0x7f10200c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 shutdown_connections 2024-05-08T22:44:59.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.506+0000 7f10267a8700 1 -- 172.21.15.179:0/2383927077 wait complete. 2024-05-08T22:44:59.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:44:59.520 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:44:59.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.638+0000 7fa3d726f700 1 Processor -- start 2024-05-08T22:44:59.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.638+0000 7fa3d726f700 1 -- start start 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 0x7fa3d01428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 0x7fa3d0140a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d01423b0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d0143250 con 0x7fa3d01400d0 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01417a0 con 0x7fa3d0142e10 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 0x7fa3d0140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d626d700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 0x7fa3d01428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d6a6e700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa3d0140fb0 0x7fa3d0160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43612/0 (socket says 172.21.15.179:43612) 2024-05-08T22:44:59.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d6a6e700 1 -- 172.21.15.179:0/2339821276 learned_addr learned my addr 172.21.15.179:0/2339821276 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/2339821276 >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 msgr2=0x7fa3d01428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 --2- 172.21.15.179:0/2339821276 >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 0x7fa3d01428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/2339821276 >> v1:172.21.15.195:6789/0 conn(0x7fa3d0140fb0 legacy=0x7fa3d0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/2339821276 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0141a80 con 0x7fa3d0142e10 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d626d700 1 --2- 172.21.15.179:0/2339821276 >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 0x7fa3d01428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d5a6c700 1 --2- 172.21.15.179:0/2339821276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 0x7fa3d0140a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009850 tx=0x7fa3cc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e00f9d2cc20e913 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d526b700 1 -- 172.21.15.179:0/2339821276 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc011830 con 0x7fa3d0142e10 2024-05-08T22:44:59.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d526b700 1 -- 172.21.15.179:0/2339821276 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc011990 con 0x7fa3d0142e10 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d526b700 1 -- 172.21.15.179:0/2339821276 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc011d40 con 0x7fa3d0142e10 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- 172.21.15.179:0/2339821276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 msgr2=0x7fa3d0140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 --2- 172.21.15.179:0/2339821276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 0x7fa3d0140a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009850 tx=0x7fa3cc009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- 172.21.15.179:0/2339821276 shutdown_connections 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 --2- 172.21.15.179:0/2339821276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d0142e10 0x7fa3d0140a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 --2- 172.21.15.179:0/2339821276 >> v2:172.21.15.195:3300/0 conn(0x7fa3d01400d0 0x7fa3d01428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- 172.21.15.179:0/2339821276 >> 172.21.15.179:0/2339821276 conn(0x7fa3d00cfb50 msgr2=0x7fa3d00cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- 172.21.15.179:0/2339821276 shutdown_connections 2024-05-08T22:44:59.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.642+0000 7fa3d726f700 1 -- 172.21.15.179:0/2339821276 wait complete. 2024-05-08T22:44:59.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 Processor -- start 2024-05-08T22:44:59.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- start start 2024-05-08T22:44:59.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d01400d0 0x7fa3d00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 0x7fa3d0152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013c3c0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013b7b0 con 0x7fa3d0142e10 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013ba90 con 0x7fa3d01400d0 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37666/0 (socket says 172.21.15.179:37666) 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/4002818909 learned_addr learned my addr 172.21.15.179:0/4002818909 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d6a6e700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 0x7fa3d0152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d626d700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d01400d0 0x7fa3d00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d01400d0 msgr2=0x7fa3d00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d01400d0 0x7fa3d00ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 msgr2=0x7fa3d0152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 0x7fa3d0152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 -- 172.21.15.179:0/4002818909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00094b0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d6a6e700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 0x7fa3d0152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d5a6c700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa3d01418e0 tx=0x7fa3cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004570 con 0x7fa3d0140fb0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc01f070 con 0x7fa3d0140fb0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0038e0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc0094b0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d013c050 con 0x7fa3d0140fb0 2024-05-08T22:44:59.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.646+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d013c530 con 0x7fa3d0140fb0 2024-05-08T22:44:59.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.650+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3cc021030 con 0x7fa3d0140fb0 2024-05-08T22:44:59.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.650+0000 7fa3c6ffd700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 0x7fa3b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.650+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa3cc0856f0 con 0x7fa3d0140fb0 2024-05-08T22:44:59.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.650+0000 7fa3d626d700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 0x7fa3b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.650+0000 7fa3d626d700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 0x7fa3b4068bf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009c70 tx=0x7fa3c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.658+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa3cc052a70 con 0x7fa3d0140fb0 2024-05-08T22:44:59.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.826+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3d004e700 con 0x7fa3d0140fb0 2024-05-08T22:44:59.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.826+0000 7fa3c6ffd700 1 -- 172.21.15.179:0/4002818909 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3cc056090 con 0x7fa3d0140fb0 2024-05-08T22:44:59.828 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 msgr2=0x7fa3b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 0x7fa3b4068bf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009c70 tx=0x7fa3c0008040 comp rx=0 tx=0).stop 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 msgr2=0x7fa3d0152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa3d01418e0 tx=0x7fa3cc004770 comp rx=0 tx=0).stop 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 shutdown_connections 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa3b4066730 0x7fa3b4068bf0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d0142e10 0x7fa3d0152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d0140fb0 0x7fa3d0152940 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 --2- 172.21.15.179:0/4002818909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3d01400d0 0x7fa3d00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 >> 172.21.15.179:0/4002818909 conn(0x7fa3d00cfb50 msgr2=0x7fa3d006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 shutdown_connections 2024-05-08T22:44:59.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.830+0000 7fa3d726f700 1 -- 172.21.15.179:0/4002818909 wait complete. 2024-05-08T22:44:59.844 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:44:59.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.954+0000 7f2cbd9fe700 1 Processor -- start 2024-05-08T22:44:59.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.954+0000 7f2cbd9fe700 1 -- start start 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 0x7f2cb806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80b22d0 con 0x7f2cb8142fd0 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80b19e0 con 0x7f2cb8142890 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb815e640 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbc9fc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 0x7f2cb806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd1fd700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2cb8142fd0 0x7f2cb815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43628/0 (socket says 172.21.15.179:43628) 2024-05-08T22:44:59.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbc9fc700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:43676/0 (socket says 172.21.15.179:43676) 2024-05-08T22:44:59.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd1fd700 1 -- 172.21.15.179:0/2720321248 learned_addr learned my addr 172.21.15.179:0/2720321248 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:44:59.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb77fe700 1 -- 172.21.15.179:0/2720321248 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482097702 0 0) 0x7f2cb80b22d0 con 0x7f2cb8142fd0 2024-05-08T22:44:59.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 -- 172.21.15.179:0/2720321248 >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 msgr2=0x7f2cb80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 --2- 172.21.15.179:0/2720321248 >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 -- 172.21.15.179:0/2720321248 >> v1:172.21.15.195:6789/0 conn(0x7f2cb8142fd0 legacy=0x7f2cb815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 -- 172.21.15.179:0/2720321248 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80b1c80 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbc9fc700 1 --2- 172.21.15.179:0/2720321248 >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb7fff700 1 --2- 172.21.15.179:0/2720321248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 0x7f2cb806cda0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009850 tx=0x7f2ca8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50117d08180a950a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb77fe700 1 -- 172.21.15.179:0/2720321248 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8011830 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb77fe700 1 -- 172.21.15.179:0/2720321248 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8011990 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cb77fe700 1 -- 172.21.15.179:0/2720321248 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8011d40 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/2720321248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 msgr2=0x7f2cb806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.958+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/2720321248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 0x7f2cb806cda0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009850 tx=0x7f2ca8009b60 comp rx=0 tx=0).stop 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/2720321248 shutdown_connections 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/2720321248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb80b2ca0 0x7f2cb806cda0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/2720321248 >> v2:172.21.15.195:3300/0 conn(0x7f2cb8142890 0x7f2cb80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.963 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/2720321248 >> 172.21.15.179:0/2720321248 conn(0x7f2cb80cea30 msgr2=0x7f2cb80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:44:59.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/2720321248 shutdown_connections 2024-05-08T22:44:59.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/2720321248 wait complete. 2024-05-08T22:44:59.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 Processor -- start 2024-05-08T22:44:59.964 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- start start 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cb80b2ca0 0x7f2cb814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb8142fd0 0x7f2cb80acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81366b0 con 0x7f2cb8142890 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb8136060 con 0x7f2cb80b2ca0 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd9fe700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81361c0 con 0x7f2cb8142fd0 2024-05-08T22:44:59.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37676/0 (socket says 172.21.15.179:37676) 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 -- 172.21.15.179:0/3345402100 learned_addr learned my addr 172.21.15.179:0/3345402100 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbd1fd700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb8142fd0 0x7f2cb80acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cbc9fc700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cb80b2ca0 0x7f2cb814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 -- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb8142fd0 msgr2=0x7f2cb80acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb8142fd0 0x7f2cb80acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 -- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cb80b2ca0 msgr2=0x7f2cb814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cb80b2ca0 0x7f2cb814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.962+0000 7f2cb7fff700 1 -- 172.21.15.179:0/3345402100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac00b750 con 0x7f2cb8142890 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb7fff700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca801a990 con 0x7f2cb8142890 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca801f070 con 0x7f2cb8142890 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca80094b0 con 0x7f2cb8142890 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca801ad30 con 0x7f2cb8142890 2024-05-08T22:44:59.968 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb8138820 con 0x7f2cb8142890 2024-05-08T22:44:59.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ca802e450 con 0x7f2cb8142890 2024-05-08T22:44:59.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8138680 con 0x7f2cb8142890 2024-05-08T22:44:59.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 0x7f2ca4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:44:59.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2ca8085240 con 0x7f2cb8142890 2024-05-08T22:44:59.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.966+0000 7f2cbc9fc700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 0x7f2ca4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:44:59.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.970+0000 7f2cbc9fc700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 0x7f2ca4068bf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00bf00 tx=0x7f2cac00b450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:44:59.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:44:59.978+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2cb8138680 con 0x7f2cb8142890 2024-05-08T22:45:00.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.146+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cb8138680 con 0x7f2cb8142890 2024-05-08T22:45:00.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.146+0000 7f2cb57fa700 1 -- 172.21.15.179:0/3345402100 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cb8138680 con 0x7f2cb8142890 2024-05-08T22:45:00.148 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:45:00.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 msgr2=0x7f2ca4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 0x7f2ca4068bf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00bf00 tx=0x7f2cac00b450 comp rx=0 tx=0).stop 2024-05-08T22:45:00.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 msgr2=0x7f2cb8135b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8004770 comp rx=0 tx=0).stop 2024-05-08T22:45:00.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 shutdown_connections 2024-05-08T22:45:00.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ca4066730 0x7f2ca4068bf0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cb8142fd0 0x7f2cb80acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb8142890 0x7f2cb8135b20 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 --2- 172.21.15.179:0/3345402100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cb80b2ca0 0x7f2cb814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 >> 172.21.15.179:0/3345402100 conn(0x7f2cb80cea30 msgr2=0x7f2cb813d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 shutdown_connections 2024-05-08T22:45:00.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.150+0000 7f2cbd9fe700 1 -- 172.21.15.179:0/3345402100 wait complete. 2024-05-08T22:45:00.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:45:00.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 Processor -- start 2024-05-08T22:45:00.284 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 -- start start 2024-05-08T22:45:00.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d6415d4b0 0x7f5d640d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5d640b6840 0x7f5d640b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d640d1ba0 con 0x7f5d640b8d20 2024-05-08T22:45:00.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640b71e0 con 0x7f5d640b6840 2024-05-08T22:45:00.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d6aba7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640d1e80 con 0x7f5d6415d4b0 2024-05-08T22:45:00.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5d640b6840 0x7f5d640b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d68943700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d6415d4b0 0x7f5d640d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5d640b6840 0x7f5d640b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:43688/0 (socket says 172.21.15.179:43688) 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 -- 172.21.15.179:0/4273118569 learned_addr learned my addr 172.21.15.179:0/4273118569 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d63fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f5d640b8d20 0x7f5d640b6130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:43638/0 (socket says 172.21.15.179:43638) 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 -- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d6415d4b0 msgr2=0x7f5d640d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 --2- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d6415d4b0 0x7f5d640d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.282+0000 7f5d69144700 1 -- 172.21.15.179:0/4273118569 >> v1:172.21.15.195:6789/0 conn(0x7f5d640b8d20 legacy=0x7f5d640b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d69144700 1 -- 172.21.15.179:0/4273118569 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640d2160 con 0x7f5d640b6840 2024-05-08T22:45:00.287 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d69144700 1 --2- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d640b6ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009e50 tx=0x7f5d5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b142209193caa192 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d637fe700 1 -- 172.21.15.179:0/4273118569 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00e070 con 0x7f5d640b6840 2024-05-08T22:45:00.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d637fe700 1 -- 172.21.15.179:0/4273118569 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5c00b910 con 0x7f5d640b6840 2024-05-08T22:45:00.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d637fe700 1 -- 172.21.15.179:0/4273118569 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00bcc0 con 0x7f5d640b6840 2024-05-08T22:45:00.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 msgr2=0x7f5d640b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.288 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d640b6ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009e50 tx=0x7f5d5c00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- 172.21.15.179:0/4273118569 shutdown_connections 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d640b6ca0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/4273118569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d6415d4b0 0x7f5d640d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- 172.21.15.179:0/4273118569 >> 172.21.15.179:0/4273118569 conn(0x7f5d640b4af0 msgr2=0x7f5d640b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- 172.21.15.179:0/4273118569 shutdown_connections 2024-05-08T22:45:00.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- 172.21.15.179:0/4273118569 wait complete. 2024-05-08T22:45:00.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 Processor -- start 2024-05-08T22:45:00.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.286+0000 7f5d6aba7700 1 -- start start 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d6415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d640b8d20 0x7f5d640cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d64153a10 con 0x7f5d6415d4b0 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d641f6db0 con 0x7f5d640b6840 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d641f7090 con 0x7f5d640b8d20 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37684/0 (socket says 172.21.15.179:37684) 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 -- 172.21.15.179:0/3133123935 learned_addr learned my addr 172.21.15.179:0/3133123935 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d63fff700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d640b8d20 0x7f5d640cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d68943700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d6415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 -- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d640b8d20 msgr2=0x7f5d640cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d640b8d20 0x7f5d640cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 -- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 msgr2=0x7f5d6415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d6415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 -- 172.21.15.179:0/3133123935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c009b30 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d69144700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f5d64067f50 tx=0x7f5d5c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c004800 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5c0099b0 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d640d2dc0 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c01c680 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.290+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d641f7650 con 0x7f5d6415d4b0 2024-05-08T22:45:00.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d640cf4c0 con 0x7f5d6415d4b0 2024-05-08T22:45:00.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d5c01c7e0 con 0x7f5d6415d4b0 2024-05-08T22:45:00.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d617fa700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 0x7f5d50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d68943700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 0x7f5d50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5d5c085e20 con 0x7f5d6415d4b0 2024-05-08T22:45:00.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.294+0000 7f5d68943700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 0x7f5d50068bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009b50 tx=0x7f5d58009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.302+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5d5c053080 con 0x7f5d6415d4b0 2024-05-08T22:45:00.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.474+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d64073290 con 0x7f5d6415d4b0 2024-05-08T22:45:00.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.474+0000 7f5d617fa700 1 -- 172.21.15.179:0/3133123935 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d5c0566a0 con 0x7f5d6415d4b0 2024-05-08T22:45:00.476 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:45:00.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 msgr2=0x7f5d50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 0x7f5d50068bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009b50 tx=0x7f5d58009240 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 msgr2=0x7f5d640b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f5d64067f50 tx=0x7f5d5c004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 shutdown_connections 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5d50066730 0x7f5d50068bf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d6415d4b0 0x7f5d640b3ae0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d640b8d20 0x7f5d640cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 --2- 172.21.15.179:0/3133123935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d640b6840 0x7f5d6415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 >> 172.21.15.179:0/3133123935 conn(0x7f5d640b4af0 msgr2=0x7f5d6406b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 shutdown_connections 2024-05-08T22:45:00.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.478+0000 7f5d6aba7700 1 -- 172.21.15.179:0/3133123935 wait complete. 2024-05-08T22:45:00.497 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:45:00.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.606+0000 7f47e761c700 1 Processor -- start 2024-05-08T22:45:00.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.606+0000 7f47e761c700 1 -- start start 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47e0158d70 0x7f47e015b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 0x7f47e006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e006ed60 con 0x7f47e006ceb0 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e006c570 con 0x7f47e0158d70 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e006d330 con 0x7f47e006bbc0 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47e0158d70 0x7f47e015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47e0158d70 0x7f47e015b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37688/0 (socket says 172.21.15.179:37688) 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 -- 172.21.15.179:0/1291218954 learned_addr learned my addr 172.21.15.179:0/1291218954 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e5bb9700 1 --2- 172.21.15.179:0/1291218954 >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 0x7f47e006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 -- 172.21.15.179:0/1291218954 >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 msgr2=0x7f47e006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 --2- 172.21.15.179:0/1291218954 >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 0x7f47e006c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 -- 172.21.15.179:0/1291218954 >> v1:172.21.15.195:6789/0 conn(0x7f47e006ceb0 legacy=0x7f47e006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 -- 172.21.15.179:0/1291218954 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00b1c70 con 0x7f47e0158d70 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e5bb9700 1 --2- 172.21.15.179:0/1291218954 >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 0x7f47e006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e53b8700 1 --2- 172.21.15.179:0/1291218954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e015b590 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f47e00cec10 tx=0x7f47d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d6843449ab8c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47d7fff700 1 -- 172.21.15.179:0/1291218954 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d800e070 con 0x7f47e0158d70 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47d7fff700 1 -- 172.21.15.179:0/1291218954 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d800b9c0 con 0x7f47e0158d70 2024-05-08T22:45:00.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47d7fff700 1 -- 172.21.15.179:0/1291218954 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d800bd70 con 0x7f47e0158d70 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- 172.21.15.179:0/1291218954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 msgr2=0x7f47e015b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 --2- 172.21.15.179:0/1291218954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e015b590 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f47e00cec10 tx=0x7f47d800b330 comp rx=0 tx=0).stop 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- 172.21.15.179:0/1291218954 shutdown_connections 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 --2- 172.21.15.179:0/1291218954 >> v2:172.21.15.195:3300/0 conn(0x7f47e006bbc0 0x7f47e006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 --2- 172.21.15.179:0/1291218954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e015b590 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- 172.21.15.179:0/1291218954 >> 172.21.15.179:0/1291218954 conn(0x7f47e0070e50 msgr2=0x7f47e0071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:00.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- 172.21.15.179:0/1291218954 shutdown_connections 2024-05-08T22:45:00.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.610+0000 7f47e761c700 1 -- 172.21.15.179:0/1291218954 wait complete. 2024-05-08T22:45:00.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 Processor -- start 2024-05-08T22:45:00.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 -- start start 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 0x7f47e01f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e006ceb0 0x7f47e0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f1680 con 0x7f47e0158d70 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f4c40 con 0x7f47e006bbc0 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e761c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f0de0 con 0x7f47e006ceb0 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37700/0 (socket says 172.21.15.179:37700) 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 -- 172.21.15.179:0/3161668305 learned_addr learned my addr 172.21.15.179:0/3161668305 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e53b8700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 0x7f47e01f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e4bb7700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e006ceb0 0x7f47e0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 -- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e006ceb0 msgr2=0x7f47e0157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e006ceb0 0x7f47e0157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 -- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 msgr2=0x7f47e01f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 0x7f47e01f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e5bb9700 1 -- 172.21.15.179:0/3161668305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8009b30 con 0x7f47e0158d70 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.614+0000 7f47e53b8700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 0x7f47e01f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47e5bb9700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e810 tx=0x7f47dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004230 con 0x7f47e0158d70 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc004390 con 0x7f47e0158d70 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e01f10c0 con 0x7f47e0158d70 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0036c0 con 0x7f47e0158d70 2024-05-08T22:45:00.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e01f19d0 con 0x7f47e0158d70 2024-05-08T22:45:00.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.618+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0067770 con 0x7f47e0158d70 2024-05-08T22:45:00.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.622+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47dc003840 con 0x7f47e0158d70 2024-05-08T22:45:00.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.622+0000 7f47d5ffb700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 0x7f47c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:00.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.622+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f47dc084210 con 0x7f47e0158d70 2024-05-08T22:45:00.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.622+0000 7f47e53b8700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 0x7f47c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:00.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.622+0000 7f47e53b8700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 0x7f47c4068c20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000f80 tx=0x7f47d8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:00.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.630+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47dc051450 con 0x7f47e0158d70 2024-05-08T22:45:00.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.806+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f47e0073290 con 0x7f47e0158d70 2024-05-08T22:45:00.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.806+0000 7f47d5ffb700 1 -- 172.21.15.179:0/3161668305 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+124 (secure 0 0 0) 0x7f47dc059e60 con 0x7f47e0158d70 2024-05-08T22:45:00.809 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:45:00.809 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6834/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:45:00.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 msgr2=0x7f47c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 0x7f47c4068c20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000f80 tx=0x7f47d8009000 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 msgr2=0x7f47e0157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e810 tx=0x7f47dc00ebd0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 shutdown_connections 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f47c4066760 0x7f47c4068c20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47e0158d70 0x7f47e0157590 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e006ceb0 0x7f47e0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 --2- 172.21.15.179:0/3161668305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e006bbc0 0x7f47e01f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 >> 172.21.15.179:0/3161668305 conn(0x7f47e0070e50 msgr2=0x7f47e015dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 shutdown_connections 2024-05-08T22:45:00.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:00.810+0000 7f47e761c700 1 -- 172.21.15.179:0/3161668305 wait complete. 2024-05-08T22:45:00.813 INFO:teuthology.orchestra.run.smithi179.stderr:listed 2 entries 2024-05-08T22:45:00.823 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:45:00.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:00.824 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:00.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[53415]: fuse finished with error 0 and tester_r 0 2024-05-08T22:45:00.834 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:06.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:45:06.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:06.936 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:45:06.944 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:07.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 Processor -- start 2024-05-08T22:45:07.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- start start 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c15d3a0 0x7f157c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f157c0cf6c0 0x7f157c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f157c0cfb20 0x7f157c160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c15d780 con 0x7f157c0cf6c0 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0cbf30 con 0x7f157c0cfb20 2024-05-08T22:45:07.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0cc210 con 0x7f157c15d3a0 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f157c0cf6c0 0x7f157c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f157c0cf6c0 0x7f157c0ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36050/0 (socket says 172.21.15.179:36050) 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 -- 172.21.15.179:0/3315363752 learned_addr learned my addr 172.21.15.179:0/3315363752 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157bfff700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c15d3a0 0x7f157c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f1580e3e700 1 --2- 172.21.15.179:0/3315363752 >> v2:172.21.15.195:3300/0 conn(0x7f157c0cfb20 0x7f157c160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 -- 172.21.15.179:0/3315363752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c15d3a0 msgr2=0x7f157c0cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c15d3a0 0x7f157c0cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 -- 172.21.15.179:0/3315363752 >> v2:172.21.15.195:3300/0 conn(0x7f157c0cfb20 msgr2=0x7f157c160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 --2- 172.21.15.179:0/3315363752 >> v2:172.21.15.195:3300/0 conn(0x7f157c0cfb20 0x7f157c160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 -- 172.21.15.179:0/3315363752 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0cc4f0 con 0x7f157c0cf6c0 2024-05-08T22:45:07.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157b7fe700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c0cf6c0 0x7f157c0ccfd0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f157c0b2620 tx=0x7f1564009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9fcb817556d738 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157affd700 1 -- 172.21.15.179:0/3315363752 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564011830 con 0x7f157c0cf6c0 2024-05-08T22:45:07.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157affd700 1 -- 172.21.15.179:0/3315363752 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564011990 con 0x7f157c0cf6c0 2024-05-08T22:45:07.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f157affd700 1 -- 172.21.15.179:0/3315363752 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564011d20 con 0x7f157c0cf6c0 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- 172.21.15.179:0/3315363752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c0cf6c0 msgr2=0x7f157c0ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c0cf6c0 0x7f157c0ccfd0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f157c0b2620 tx=0x7f1564009bf0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- 172.21.15.179:0/3315363752 shutdown_connections 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- 172.21.15.179:0/3315363752 >> v2:172.21.15.195:3300/0 conn(0x7f157c0cfb20 0x7f157c160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c0cf6c0 0x7f157c0ccfd0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 --2- 172.21.15.179:0/3315363752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c15d3a0 0x7f157c0cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- 172.21.15.179:0/3315363752 >> 172.21.15.179:0/3315363752 conn(0x7f157c0d0eb0 msgr2=0x7f157c0d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- 172.21.15.179:0/3315363752 shutdown_connections 2024-05-08T22:45:07.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.066+0000 7f15828a1700 1 -- 172.21.15.179:0/3315363752 wait complete. 2024-05-08T22:45:07.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.070+0000 7f15828a1700 1 Processor -- start 2024-05-08T22:45:07.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.070+0000 7f15828a1700 1 -- start start 2024-05-08T22:45:07.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c0cf6c0 0x7f157c1f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f157c0cfb20 0x7f157c0abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c8060 con 0x7f157c15d3a0 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c150e20 con 0x7f157c0cfb20 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1f27a0 con 0x7f157c0cf6c0 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36054/0 (socket says 172.21.15.179:36054) 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 -- 172.21.15.179:0/2005769241 learned_addr learned my addr 172.21.15.179:0/2005769241 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f157bfff700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c0cf6c0 0x7f157c1f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f157b7fe700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f157c0cfb20 0x7f157c0abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 -- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c0cf6c0 msgr2=0x7f157c1f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c0cf6c0 0x7f157c1f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 -- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f157c0cfb20 msgr2=0x7f157c0abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f157c0cfb20 0x7f157c0abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 -- 172.21.15.179:0/2005769241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009570 con 0x7f157c15d3a0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1580e3e700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e800 tx=0x7f156c00eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c013680 con 0x7f157c15d3a0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c0137e0 con 0x7f157c15d3a0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c0ac680 con 0x7f157c15d3a0 2024-05-08T22:45:07.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c013b70 con 0x7f157c15d3a0 2024-05-08T22:45:07.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0acc40 con 0x7f157c15d3a0 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.074+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c067770 con 0x7f157c15d3a0 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.078+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f156c013cd0 con 0x7f157c15d3a0 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.078+0000 7f1578ff9700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 0x7f155c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.078+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f156c084170 con 0x7f157c15d3a0 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.078+0000 7f157bfff700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 0x7f155c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.078+0000 7f157bfff700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 0x7f155c068c20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1574011340 tx=0x7f157400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.086+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f156c051340 con 0x7f157c15d3a0 2024-05-08T22:45:07.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f157c073290 con 0x7f157c15d3a0 2024-05-08T22:45:07.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f1578ff9700 1 -- 172.21.15.179:0/2005769241 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f156c054960 con 0x7f157c15d3a0 2024-05-08T22:45:07.300 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:07.300 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:44:51.785543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9625":{"gid":9625,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/3161101014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3161101014},{"type":"v1","addr":"172.21.15.195:6835","nonce":3161101014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 msgr2=0x7f155c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 0x7f155c068c20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1574011340 tx=0x7f157400b040 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 msgr2=0x7f157c0ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e800 tx=0x7f156c00eb10 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 shutdown_connections 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f155c066760 0x7f155c068c20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f157c0cfb20 0x7f157c0abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c15d3a0 0x7f157c0ad7e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 --2- 172.21.15.179:0/2005769241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f157c0cf6c0 0x7f157c1f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 >> 172.21.15.179:0/2005769241 conn(0x7f157c0d0eb0 msgr2=0x7f157c0b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 shutdown_connections 2024-05-08T22:45:07.303 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.298+0000 7f15828a1700 1 -- 172.21.15.179:0/2005769241 wait complete. 2024-05-08T22:45:07.304 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 96 2024-05-08T22:45:07.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parent mytag123 2024-05-08T22:45:07.446 INFO:teuthology.orchestra.run.smithi195.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-08T22:45:07.447 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-08T22:45:07.447 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-08T22:45:07.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 Processor -- start 2024-05-08T22:45:07.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 -- start start 2024-05-08T22:45:07.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe795c80 0x5587fe700620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fe652a00 con 0x5587fe792850 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fe64e2f0 con 0x5587fe6baac0 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75cdca140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fe60a8e0 con 0x5587fe795c80 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75a8ff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75a0fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75a8ff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:37482/0 (socket says 172.21.15.179:37482) 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75b100700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe795c80 0x5587fe700620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75a8ff700 1 -- 172.21.15.179:0/1304456710 learned_addr learned my addr 172.21.15.179:0/1304456710 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.486+0000 7fc75a0fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36060/0 (socket says 172.21.15.179:36060) 2024-05-08T22:45:07.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 -- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe795c80 msgr2=0x5587fe700620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe795c80 0x5587fe700620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 -- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 msgr2=0x5587fe702c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 -- 172.21.15.179:0/1304456710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75400a100 con 0x5587fe792850 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a8ff700 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:07.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc74c001c50 tx=0x7fc74c00b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f0adddf68c9c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc7590fc700 1 -- 172.21.15.179:0/1304456710 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c001da0 con 0x5587fe792850 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc7590fc700 1 -- 172.21.15.179:0/1304456710 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c00bdd0 con 0x5587fe792850 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc7590fc700 1 -- 172.21.15.179:0/1304456710 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e6b0 con 0x5587fe792850 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- 172.21.15.179:0/1304456710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 msgr2=0x5587fe7031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc74c001c50 tx=0x7fc74c00b5d0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- 172.21.15.179:0/1304456710 shutdown_connections 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe795c80 0x5587fe700620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe792850 0x5587fe7031d0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- 172.21.15.179:0/1304456710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe6baac0 0x5587fe702c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- 172.21.15.179:0/1304456710 >> 172.21.15.179:0/1304456710 conn(0x5587fe6e8110 msgr2=0x5587fe6bcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.494 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- 172.21.15.179:0/1304456710 shutdown_connections 2024-05-08T22:45:07.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- 172.21.15.179:0/1304456710 wait complete. 2024-05-08T22:45:07.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 Processor -- start 2024-05-08T22:45:07.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- start start 2024-05-08T22:45:07.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe6baac0 0x5587fe6bc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe792850 0x5587fe700620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 0x5587fe700b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fe60a580 con 0x5587fe795c80 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fe5e08d0 con 0x5587fe792850 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.490+0000 7fc75cdca140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587fe6df6f0 con 0x5587fe6baac0 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 0x5587fe700b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 0x5587fe700b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36072/0 (socket says 172.21.15.179:36072) 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 -- 172.21.15.179:0/3921635506 learned_addr learned my addr 172.21.15.179:0/3921635506 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75a8ff700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe6baac0 0x5587fe6bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe792850 0x5587fe700620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 -- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe6baac0 msgr2=0x5587fe6bc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5587fe6baac0 0x5587fe6bc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 -- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe792850 msgr2=0x5587fe700620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587fe792850 0x5587fe700620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 -- 172.21.15.179:0/3921635506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74400a040 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75b100700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 0x5587fe700b60 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ab00 tx=0x7fc754003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc753fff700 1 -- 172.21.15.179:0/3921635506 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754024670 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc753fff700 1 -- 172.21.15.179:0/3921635506 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7540247d0 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc748001e10 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc753fff700 1 -- 172.21.15.179:0/3921635506 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754024b60 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fe5ebf40 con 0x5587fe795c80 2024-05-08T22:45:07.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc753fff700 1 -- 172.21.15.179:0/3921635506 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc754021070 con 0x5587fe795c80 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.494+0000 7fc753fff700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc73c065090 0x7fc73c067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc753fff700 1 -- 172.21.15.179:0/3921635506 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc754084720 con 0x5587fe795c80 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75a8ff700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc73c065090 0x7fc73c067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5587fe6e35b0 0x5587fe790570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x5587fe6ecb70 con 0x5587fe6e35b0 2024-05-08T22:45:07.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5587fe6e35b0 0x5587fe790570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75a8ff700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc73c065090 0x7fc73c067550 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc744010430 tx=0x7fc7440018b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75a0fe700 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5587fe6e35b0 0x5587fe790570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75a0fe700 1 -- 172.21.15.179:0/3921635506 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc74c021040 con 0x5587fe6e35b0 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5587fe6e35b0 msgr2=0x5587fe790570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5587fe6e35b0 0x5587fe790570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc73c065090 msgr2=0x7fc73c067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc73c065090 0x7fc73c067550 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc744010430 tx=0x7fc7440018b0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 msgr2=0x5587fe700b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 --2- 172.21.15.179:0/3921635506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587fe795c80 0x5587fe700b60 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ab00 tx=0x7fc754003330 comp rx=0 tx=0).stop 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75b100700 1 -- 172.21.15.179:0/3921635506 reap_dead start 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 shutdown_connections 2024-05-08T22:45:07.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 >> 172.21.15.179:0/3921635506 conn(0x5587fe6e8110 msgr2=0x5587fe6bcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 shutdown_connections 2024-05-08T22:45:07.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.498+0000 7fc75cdca140 1 -- 172.21.15.179:0/3921635506 wait complete. 2024-05-08T22:45:07.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:45:07.506 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-08T22:45:07.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.542+0000 7f10ed6f9140 1 Processor -- start 2024-05-08T22:45:07.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.542+0000 7f10ed6f9140 1 -- start start 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386455780 0x560386524980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386526620 0x560386524ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603863e42f0 con 0x560386453750 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603863a08e0 con 0x560386455780 2024-05-08T22:45:07.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603863a0580 con 0x560386526620 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36096/0 (socket says 172.21.15.179:36096) 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 -- 172.21.15.179:0/1941667608 learned_addr learned my addr 172.21.15.179:0/1941667608 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eaa2d700 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386455780 0x560386524980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eba2f700 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386526620 0x560386524ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 -- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386526620 msgr2=0x560386524ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386526620 0x560386524ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 -- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386455780 msgr2=0x560386524980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386455780 0x560386524980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 -- 172.21.15.179:0/1941667608 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4005de0 con 0x560386453750 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10eb22e700 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f10e400a460 tx=0x7f10e400a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f2c9f017857aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10e9a2b700 1 -- 172.21.15.179:0/1941667608 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4008810 con 0x560386453750 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10e9a2b700 1 -- 172.21.15.179:0/1941667608 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4008990 con 0x560386453750 2024-05-08T22:45:07.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10e9a2b700 1 -- 172.21.15.179:0/1941667608 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4008d40 con 0x560386453750 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- 172.21.15.179:0/1941667608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 msgr2=0x560386453fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f10e400a460 tx=0x7f10e400a770 comp rx=0 tx=0).stop 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- 172.21.15.179:0/1941667608 shutdown_connections 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386455780 0x560386524980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386526620 0x560386524ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/1941667608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386453750 0x560386453fc0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- 172.21.15.179:0/1941667608 >> 172.21.15.179:0/1941667608 conn(0x56038647e110 msgr2=0x56038649a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- 172.21.15.179:0/1941667608 shutdown_connections 2024-05-08T22:45:07.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.546+0000 7f10ed6f9140 1 -- 172.21.15.179:0/1941667608 wait complete. 2024-05-08T22:45:07.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 Processor -- start 2024-05-08T22:45:07.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- start start 2024-05-08T22:45:07.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386453750 0x560386509b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 0x560386520d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386526620 0x560386521290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603863768d0 con 0x560386455780 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603864f4ed0 con 0x560386526620 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560386521890 con 0x560386453750 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 0x560386520d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 0x560386520d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36110/0 (socket says 172.21.15.179:36110) 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 -- 172.21.15.179:0/487137197 learned_addr learned my addr 172.21.15.179:0/487137197 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eb22e700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386453750 0x560386509b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eba2f700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386526620 0x560386521290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 -- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386453750 msgr2=0x560386509b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560386453750 0x560386509b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 -- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386526620 msgr2=0x560386521290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386526620 0x560386521290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 -- 172.21.15.179:0/487137197 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4005cc0 con 0x560386455780 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eba2f700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560386526620 0x560386521290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10eaa2d700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 0x560386520d50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f10d4002000 tx=0x7f10d4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10e8a29700 1 -- 172.21.15.179:0/487137197 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4012660 con 0x560386455780 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10e8a29700 1 -- 172.21.15.179:0/487137197 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40127c0 con 0x560386455780 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56038638eb00 con 0x560386455780 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10e8a29700 1 -- 172.21.15.179:0/487137197 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4012b70 con 0x560386455780 2024-05-08T22:45:07.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.550+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560386381f40 con 0x560386455780 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10e8a29700 1 -- 172.21.15.179:0/487137197 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10d4019030 con 0x560386455780 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10e8a29700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f10cc065090 0x7f10cc067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10e8a29700 1 -- 172.21.15.179:0/487137197 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f10d4084680 con 0x560386455780 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10eb22e700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f10cc065090 0x7f10cc067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x5603865227b0 0x560386522bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x560386523100 con 0x5603865227b0 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10eba2f700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x5603865227b0 0x560386522bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10eb22e700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f10cc065090 0x7f10cc067550 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f10e400a7b0 tx=0x7f10e4008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.554+0000 7f10eba2f700 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x5603865227b0 0x560386522bc0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10eba2f700 1 -- 172.21.15.179:0/487137197 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f10dc00d820 con 0x5603865227b0 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x5603865227b0 msgr2=0x560386522bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x5603865227b0 0x560386522bc0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f10cc065090 msgr2=0x7f10cc067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f10cc065090 0x7f10cc067550 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f10e400a7b0 tx=0x7f10e4008000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 msgr2=0x560386520d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 --2- 172.21.15.179:0/487137197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560386455780 0x560386520d50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f10d4002000 tx=0x7f10d4002fb0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10eba2f700 1 -- 172.21.15.179:0/487137197 reap_dead start 2024-05-08T22:45:07.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 shutdown_connections 2024-05-08T22:45:07.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 >> 172.21.15.179:0/487137197 conn(0x56038647e110 msgr2=0x560386453fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 shutdown_connections 2024-05-08T22:45:07.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.558+0000 7f10ed6f9140 1 -- 172.21.15.179:0/487137197 wait complete. 2024-05-08T22:45:07.562 INFO:teuthology.orchestra.run.smithi179.stdout:mytag123 2024-05-08T22:45:07.562 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-08T22:45:07.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 Processor -- start 2024-05-08T22:45:07.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 -- start start 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053eae780 0x556053f7cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 0x556053f7d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556053df98e0 con 0x556053f7f620 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556053df9580 con 0x556053eae780 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585f6a2140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556053dcf8d0 con 0x556053eac580 2024-05-08T22:45:07.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d9d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 0x556053f7d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:47816/0 (socket says 172.21.15.179:47816) 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 -- 172.21.15.179:0/2074443113 learned_addr learned my addr 172.21.15.179:0/2074443113 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d9d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 0x556053f7d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36136/0 (socket says 172.21.15.179:36136) 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585c9d6700 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053eae780 0x556053f7cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 -- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053eae780 msgr2=0x556053f7cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053eae780 0x556053f7cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 -- 172.21.15.179:0/2074443113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 msgr2=0x556053f7d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 0x556053f7d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 -- 172.21.15.179:0/2074443113 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585800a1c0 con 0x556053eac580 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f585d1d7700 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f585800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34657cd594380f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f584f7fe700 1 -- 172.21.15.179:0/2074443113 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585800e710 con 0x556053eac580 2024-05-08T22:45:07.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f584f7fe700 1 -- 172.21.15.179:0/2074443113 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585800e9e0 con 0x556053eac580 2024-05-08T22:45:07.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.646+0000 7f584f7fe700 1 -- 172.21.15.179:0/2074443113 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585800ed70 con 0x556053eac580 2024-05-08T22:45:07.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 msgr2=0x556053edb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f585800e000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- 172.21.15.179:0/2074443113 shutdown_connections 2024-05-08T22:45:07.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053eae780 0x556053f7cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053f7f620 0x556053f7d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- 172.21.15.179:0/2074443113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053edb910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- 172.21.15.179:0/2074443113 >> 172.21.15.179:0/2074443113 conn(0x556053ed7110 msgr2=0x556053edaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- 172.21.15.179:0/2074443113 shutdown_connections 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- 172.21.15.179:0/2074443113 wait complete. 2024-05-08T22:45:07.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 Processor -- start 2024-05-08T22:45:07.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- start start 2024-05-08T22:45:07.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053f6f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 0x556053eda9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053f7f620 0x556053edaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556053eec3e0 con 0x556053eae780 2024-05-08T22:45:07.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556053eec640 con 0x556053f7f620 2024-05-08T22:45:07.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585f6a2140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556053eec8e0 con 0x556053eac580 2024-05-08T22:45:07.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.650+0000 7f585c9d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 0x556053eda9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585d1d7700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053f6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 0x556053eda9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36148/0 (socket says 172.21.15.179:36148) 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 -- 172.21.15.179:0/3139565647 learned_addr learned my addr 172.21.15.179:0/3139565647 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585d9d8700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053f7f620 0x556053edaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 -- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 msgr2=0x556053f6f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556053eac580 0x556053f6f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 -- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053f7f620 msgr2=0x556053edaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053f7f620 0x556053edaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 -- 172.21.15.179:0/3139565647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556053ddabb0 con 0x556053eae780 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585d9d8700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556053f7f620 0x556053edaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585c9d6700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 0x556053eda9b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5854008810 tx=0x7f5854008b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f584e7fc700 1 -- 172.21.15.179:0/3139565647 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585400f040 con 0x556053eae780 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556053e01ca0 con 0x556053eae780 2024-05-08T22:45:07.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f584e7fc700 1 -- 172.21.15.179:0/3139565647 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5854015880 con 0x556053eae780 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f584e7fc700 1 -- 172.21.15.179:0/3139565647 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854015c30 con 0x556053eae780 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556053de7b00 con 0x556053eae780 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.654+0000 7f584e7fc700 1 -- 172.21.15.179:0/3139565647 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58540143b0 con 0x556053eae780 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f584e7fc700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840065060 0x7f5840067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f584e7fc700 1 -- 172.21.15.179:0/3139565647 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5854083aa0 con 0x556053eae780 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d1d7700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840065060 0x7f5840067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x556053eee2d0 0x556053ef12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x556053ef1820 con 0x556053eee2d0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d9d8700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x556053eee2d0 0x556053ef12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d1d7700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840065060 0x7f5840067520 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f5858008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d9d8700 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x556053eee2d0 0x556053ef12e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d9d8700 1 -- 172.21.15.179:0/3139565647 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f5850009880 con 0x556053eee2d0 2024-05-08T22:45:07.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x556053eee2d0 msgr2=0x556053ef12e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x556053eee2d0 0x556053ef12e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840065060 msgr2=0x7f5840067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5840065060 0x7f5840067520 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f5858008000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 msgr2=0x556053eda9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 --2- 172.21.15.179:0/3139565647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556053eae780 0x556053eda9b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5854008810 tx=0x7f5854008b20 comp rx=0 tx=0).stop 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585d9d8700 1 -- 172.21.15.179:0/3139565647 reap_dead start 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 shutdown_connections 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 >> 172.21.15.179:0/3139565647 conn(0x556053ed7110 msgr2=0x556053f7e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 shutdown_connections 2024-05-08T22:45:07.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.658+0000 7f585f6a2140 1 -- 172.21.15.179:0/3139565647 wait complete. 2024-05-08T22:45:07.671 INFO:teuthology.orchestra.run.smithi179.stdout:mytag123 2024-05-08T22:45:07.671 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-08T22:45:07.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.710+0000 7f02cbc20140 1 Processor -- start 2024-05-08T22:45:07.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.710+0000 7f02cbc20140 1 -- start start 2024-05-08T22:45:07.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f915eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f9149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f90ab2f0 con 0x5596f914b8b0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f90678e0 con 0x5596f91ef700 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f9067580 con 0x5596f91181e0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36160/0 (socket says 172.21.15.179:36160) 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 -- 172.21.15.179:0/1736399673 learned_addr learned my addr 172.21.15.179:0/1736399673 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c9f56700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f9149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c9755700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f915eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 -- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 msgr2=0x5596f915eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f915eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 -- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 msgr2=0x5596f9149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f9149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 -- 172.21.15.179:0/1736399673 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4005de0 con 0x5596f914b8b0 2024-05-08T22:45:07.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c9f56700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f9149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c8f54700 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x5596f911fe50 tx=0x7f02b400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dded13def7b64f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c37fe700 1 -- 172.21.15.179:0/1736399673 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4008b60 con 0x5596f914b8b0 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c37fe700 1 -- 172.21.15.179:0/1736399673 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4008cc0 con 0x5596f914b8b0 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02c37fe700 1 -- 172.21.15.179:0/1736399673 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b401c680 con 0x5596f914b8b0 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- 172.21.15.179:0/1736399673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 msgr2=0x5596f9149990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x5596f911fe50 tx=0x7f02b400d000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- 172.21.15.179:0/1736399673 shutdown_connections 2024-05-08T22:45:07.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f9149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9149990 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 --2- 172.21.15.179:0/1736399673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f915eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- 172.21.15.179:0/1736399673 >> 172.21.15.179:0/1736399673 conn(0x5596f911e740 msgr2=0x5596f9161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- 172.21.15.179:0/1736399673 shutdown_connections 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 -- 172.21.15.179:0/1736399673 wait complete. 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.714+0000 7f02cbc20140 1 Processor -- start 2024-05-08T22:45:07.719 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- start start 2024-05-08T22:45:07.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f913e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9287830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f915fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f903d8d0 con 0x5596f914b8b0 2024-05-08T22:45:07.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f913f230 con 0x5596f91ef700 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f913f4d0 con 0x5596f91181e0 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9287830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9287830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36172/0 (socket says 172.21.15.179:36172) 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 -- 172.21.15.179:0/3384579060 learned_addr learned my addr 172.21.15.179:0/3384579060 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c9f56700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f915fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c9755700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f913e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 -- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 msgr2=0x5596f913e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f913e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 -- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 msgr2=0x5596f915fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596f91ef700 0x5596f915fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 -- 172.21.15.179:0/3384579060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4002470 con 0x5596f914b8b0 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c9755700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5596f91181e0 0x5596f913e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c8f54700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9287830 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f02b400a4f0 tx=0x7f02b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c27fc700 1 -- 172.21.15.179:0/3384579060 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b400ad30 con 0x5596f914b8b0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c27fc700 1 -- 172.21.15.179:0/3384579060 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b401c650 con 0x5596f914b8b0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5596f906fca0 con 0x5596f914b8b0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02c27fc700 1 -- 172.21.15.179:0/3384579060 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b401ca00 con 0x5596f914b8b0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.718+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f9055b00 con 0x5596f914b8b0 2024-05-08T22:45:07.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c27fc700 1 -- 172.21.15.179:0/3384579060 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02b401cb60 con 0x5596f914b8b0 2024-05-08T22:45:07.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c27fc700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f02ac065090 0x7f02ac067550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c27fc700 1 -- 172.21.15.179:0/3384579060 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f02b40860b0 con 0x5596f914b8b0 2024-05-08T22:45:07.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c9755700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f02ac065090 0x7f02ac067550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5596f913dae0 0x5596f91e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x5596f9284ee0 con 0x5596f913dae0 2024-05-08T22:45:07.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c9f56700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5596f913dae0 0x5596f91e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c9f56700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5596f913dae0 0x5596f91e46b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c9755700 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f02ac065090 0x7f02ac067550 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f60 tx=0x7f02bc001770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02c9f56700 1 -- 172.21.15.179:0/3384579060 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f02c4002c50 con 0x5596f913dae0 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5596f913dae0 msgr2=0x5596f91e46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x5596f913dae0 0x5596f91e46b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f02ac065090 msgr2=0x7f02ac067550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.722+0000 7f02cbc20140 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f02ac065090 0x7f02ac067550 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f60 tx=0x7f02bc001770 comp rx=0 tx=0).stop 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 msgr2=0x5596f9287830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 --2- 172.21.15.179:0/3384579060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f914b8b0 0x5596f9287830 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f02b400a4f0 tx=0x7f02b4002fb0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02c9f56700 1 -- 172.21.15.179:0/3384579060 reap_dead start 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 shutdown_connections 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 >> 172.21.15.179:0/3384579060 conn(0x5596f911e740 msgr2=0x5596f91034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 shutdown_connections 2024-05-08T22:45:07.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.726+0000 7f02cbc20140 1 -- 172.21.15.179:0/3384579060 wait complete. 2024-05-08T22:45:07.731 INFO:teuthology.orchestra.run.smithi179.stdout:mytag123 2024-05-08T22:45:07.731 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-08T22:45:07.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.838+0000 7fd7558e9700 1 Processor -- start 2024-05-08T22:45:07.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.838+0000 7fd7558e9700 1 -- start start 2024-05-08T22:45:07.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd7500ae720 0x7fd7500aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7500b5340 con 0x7fd7500afb00 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7500b4730 con 0x7fd7500ae720 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500b4a10 con 0x7fd7500b4170 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd7500ae720 0x7fd7500aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74f7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd7500afb00 0x7fd75015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42024/0 (socket says 172.21.15.179:42024) 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74f7fe700 1 -- 172.21.15.179:0/3508074781 learned_addr learned my addr 172.21.15.179:0/3508074781 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 -- 172.21.15.179:0/3508074781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 msgr2=0x7fd7500b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 -- 172.21.15.179:0/3508074781 >> v1:172.21.15.179:6789/0 conn(0x7fd7500afb00 legacy=0x7fd75015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 -- 172.21.15.179:0/3508074781 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7500b4cf0 con 0x7fd7500ae720 2024-05-08T22:45:07.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74effd700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7500aeb00 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fd740009d90 tx=0x7fd74000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3151c34e1894146f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74dffb700 1 -- 172.21.15.179:0/3508074781 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000e070 con 0x7fd7500ae720 2024-05-08T22:45:07.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74dffb700 1 -- 172.21.15.179:0/3508074781 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74000bbd0 con 0x7fd7500ae720 2024-05-08T22:45:07.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd74dffb700 1 -- 172.21.15.179:0/3508074781 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740012430 con 0x7fd7500ae720 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- 172.21.15.179:0/3508074781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 msgr2=0x7fd7500aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7500aeb00 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fd740009d90 tx=0x7fd74000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- 172.21.15.179:0/3508074781 shutdown_connections 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 --2- 172.21.15.179:0/3508074781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7500aeb00 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- 172.21.15.179:0/3508074781 >> 172.21.15.179:0/3508074781 conn(0x7fd750098a70 msgr2=0x7fd7500b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- 172.21.15.179:0/3508074781 shutdown_connections 2024-05-08T22:45:07.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.842+0000 7fd7558e9700 1 -- 172.21.15.179:0/3508074781 wait complete. 2024-05-08T22:45:07.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.846+0000 7fd7558e9700 1 Processor -- start 2024-05-08T22:45:07.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.846+0000 7fd7558e9700 1 -- start start 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7500afb00 0x7fd7500cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75015d560 con 0x7fd7500ae720 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75015d840 con 0x7fd7500afb00 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75015db20 con 0x7fd7500b4170 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36194/0 (socket says 172.21.15.179:36194) 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 -- 172.21.15.179:0/4112344560 learned_addr learned my addr 172.21.15.179:0/4112344560 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7500afb00 0x7fd7500cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 -- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 msgr2=0x7fd7500cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 -- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7500afb00 msgr2=0x7fd7500cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7500afb00 0x7fd7500cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 -- 172.21.15.179:0/4112344560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd740009a70 con 0x7fd7500ae720 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd74effd700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd740014040 tx=0x7fd740004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd747fff700 1 -- 172.21.15.179:0/4112344560 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740012430 con 0x7fd7500ae720 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd747fff700 1 -- 172.21.15.179:0/4112344560 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740020070 con 0x7fd7500ae720 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75015de00 con 0x7fd7500ae720 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd747fff700 1 -- 172.21.15.179:0/4112344560 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740004280 con 0x7fd7500ae720 2024-05-08T22:45:07.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75015c3b0 con 0x7fd7500ae720 2024-05-08T22:45:07.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.850+0000 7fd747fff700 1 -- 172.21.15.179:0/4112344560 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7400043e0 con 0x7fd7500ae720 2024-05-08T22:45:07.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd747fff700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 0x7fd730068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd747fff700 1 -- 172.21.15.179:0/4112344560 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd740022070 con 0x7fd7500ae720 2024-05-08T22:45:07.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 0x7fd730068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 0x7fd730068c20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd750151240 tx=0x7fd738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 Processor -- start 2024-05-08T22:45:07.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 -- start start 2024-05-08T22:45:07.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x18e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x18ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x187c010 con 0x18eece0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1809f40 con 0x19165d0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x185ce30 con 0x18e9840 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36198/0 (socket says 172.21.15.179:36198) 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.854+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3256328890 learned_addr learned my addr 172.21.15.179:0/3256328890 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x18ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74effd700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x18e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 msgr2=0x18e9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x18e9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 msgr2=0x18ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x18ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3256328890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16c8a40 con 0x18eece0 2024-05-08T22:45:07.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x18ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd738009810 tx=0x7fd738013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6f2676ecd2d3ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd7457fa700 1 -- 172.21.15.179:0/3256328890 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738015900 con 0x18eece0 2024-05-08T22:45:07.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd7457fa700 1 -- 172.21.15.179:0/3256328890 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73801a070 con 0x18eece0 2024-05-08T22:45:07.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd7457fa700 1 -- 172.21.15.179:0/3256328890 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738015c60 con 0x18eece0 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- 172.21.15.179:0/3256328890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 msgr2=0x192fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd738009810 tx=0x7fd738013000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- 172.21.15.179:0/3256328890 shutdown_connections 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x18ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x192fa60 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 --2- 172.21.15.179:0/3256328890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x18e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- 172.21.15.179:0/3256328890 >> 172.21.15.179:0/3256328890 conn(0x167d5b0 msgr2=0x1915ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- 172.21.15.179:0/3256328890 shutdown_connections 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- 172.21.15.179:0/3256328890 wait complete. 2024-05-08T22:45:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 -- start start 2024-05-08T22:45:07.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.858+0000 7fd758b98740 1 Processor -- start 2024-05-08T22:45:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x1913590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18eece0 0x18ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1858150 con 0x19165d0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x1855eb0 con 0x18e9840 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x18138e0 con 0x18eece0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 10 client.0 _ll_register_callbacks cb 0x17f2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36210/0 (socket says 172.21.15.179:36210) 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 -- 172.21.15.179:0/217679282 learned_addr learned my addr 172.21.15.179:0/217679282 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74effd700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x1913590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18eece0 0x18ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 -- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18eece0 msgr2=0x18ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18eece0 0x18ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 -- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 msgr2=0x1913590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x1913590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 -- 172.21.15.179:0/217679282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd738004560 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd748007740 tx=0x7fd7480015c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f4a5181d3461c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748001d30 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd748015470 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x168a2c0 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd748015630 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 10 client.10122 fetch_fsmap learned FSMap version 96 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x181c290 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x168a2c0 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd748015790 con 0x19165d0 2024-05-08T22:45:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 10 client.10122 fetch_fsmap finished waiting for FSMap version 96 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 10 client.10122 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 20 client.10122 populate_metadata read hostname 'smithi179' 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 4 client.10122 mds_command: new command op to 9625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 --> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16d4540 con 0x1936b50 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74effd700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd72e7fc700 10 client.10122 ms_handle_connect on v2:172.21.15.195:6834/3161101014 2024-05-08T22:45:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.862+0000 7fd74effd700 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194d290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f36e080fac146bc server_cookie=bfbebeb208bb48f9 in_seq=0 out_seq=0 2024-05-08T22:45:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd72e7fc700 1 -- 172.21.15.179:0/217679282 <== mds.0 v2:172.21.15.195:6834/3161101014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x16d4540 con 0x1936b50 2024-05-08T22:45:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd72e7fc700 10 client.10122 handle_command_reply: tid=0 2024-05-08T22:45:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 client.10122 shutdown 2024-05-08T22:45:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 msgr2=0x18ecc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd748007740 tx=0x7fd7480015c0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 shutdown_connections 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194d290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x19165d0 0x18ecc50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd72e7fc700 0 client.10122 ms_handle_reset on v2:172.21.15.195:6834/3161101014 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18eece0 0x18ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 --2- 172.21.15.179:0/217679282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x1913590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 >> 172.21.15.179:0/217679282 conn(0x167d5b0 msgr2=0x192f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 shutdown_connections 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 1 -- 172.21.15.179:0/217679282 wait complete. 2024-05-08T22:45:07.870 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.866+0000 7fd758b98740 20 client.10122 trim_cache size 0 max 16384 2024-05-08T22:45:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 Processor -- start 2024-05-08T22:45:07.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 -- start start 2024-05-08T22:45:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x194f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x1930750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x1701b80 con 0x18eece0 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x17f4620 con 0x19165d0 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x17dfa50 con 0x18e9840 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36212/0 (socket says 172.21.15.179:36212) 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 -- 172.21.15.179:0/2551048041 learned_addr learned my addr 172.21.15.179:0/2551048041 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74effd700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x194f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x1930750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 -- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 msgr2=0x194f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x194f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 -- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 msgr2=0x1930750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x1930750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 -- 172.21.15.179:0/2551048041 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18024c0 con 0x18eece0 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x1930750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74effd700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x194f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.886+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd738022000 tx=0x7fd738015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9945225b2e9329b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd745ffb700 1 -- 172.21.15.179:0/2551048041 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738015700 con 0x18eece0 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd745ffb700 1 -- 172.21.15.179:0/2551048041 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738015860 con 0x18eece0 2024-05-08T22:45:07.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd745ffb700 1 -- 172.21.15.179:0/2551048041 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738015bf0 con 0x18eece0 2024-05-08T22:45:07.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/2551048041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 msgr2=0x1930210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd738022000 tx=0x7fd738015000 comp rx=0 tx=0).stop 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/2551048041 shutdown_connections 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x19165d0 0x1930750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930210 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- 172.21.15.179:0/2551048041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x18e9840 0x194f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/2551048041 >> 172.21.15.179:0/2551048041 conn(0x167d5b0 msgr2=0x18ec420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/2551048041 shutdown_connections 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/2551048041 wait complete. 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- start start 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 Processor -- start 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x199a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19165d0 0x1930bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x18536c0 con 0x18eece0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x187d610 con 0x18e9840 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x1826c00 con 0x19165d0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 10 client.0 _ll_register_callbacks cb 0x180fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36218/0 (socket says 172.21.15.179:36218) 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3480498694 learned_addr learned my addr 172.21.15.179:0/3480498694 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74effd700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x199a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74f7fe700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19165d0 0x1930bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19165d0 msgr2=0x1930bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19165d0 0x1930bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 msgr2=0x199a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x199a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 -- 172.21.15.179:0/3480498694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16c8a40 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74effd700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x199a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd74e7fc700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fd738009eb0 tx=0x7fd7380047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b737e41a8feb34d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd72e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738015700 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd738021850 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd73801a070 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.890+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd738021a10 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 10 client.10128 fetch_fsmap learned FSMap version 96 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x17f6790 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73801a070 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd738021b70 con 0x18eece0 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 10 client.10128 fetch_fsmap finished waiting for FSMap version 96 2024-05-08T22:45:07.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 10 client.10128 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 20 client.10128 populate_metadata read hostname 'smithi179' 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 4 client.10128 mds_command: new command op to 9625 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 --> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x16f92c0 con 0x1936b50 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd74effd700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd74effd700 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194e580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4b808d2e9bf328 server_cookie=c629fb5843c40b31 in_seq=0 out_seq=0 2024-05-08T22:45:07.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.894+0000 7fd72e7fc700 10 client.10128 ms_handle_connect on v2:172.21.15.195:6834/3161101014 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd72e7fc700 1 -- 172.21.15.179:0/3480498694 <== mds.0 v2:172.21.15.195:6834/3161101014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x16f92c0 con 0x1936b50 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd72e7fc700 10 client.10128 handle_command_reply: tid=0 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 client.10128 shutdown 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 msgr2=0x1930670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fd738009eb0 tx=0x7fd7380047e0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 shutdown_connections 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:6834/3161101014,v1:172.21.15.195:6835/3161101014] conn(0x1936b50 0x194e580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x19165d0 0x1930bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x18eece0 0x1930670 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 --2- 172.21.15.179:0/3480498694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x18e9840 0x199a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd72e7fc700 0 client.10128 ms_handle_reset on v2:172.21.15.195:6834/3161101014 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 >> 172.21.15.179:0/3480498694 conn(0x167d5b0 msgr2=0x18ede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 shutdown_connections 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 1 -- 172.21.15.179:0/3480498694 wait complete. 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd758b98740 20 client.10128 trim_cache size 0 max 16384 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stdout: "message": "", 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stdout: "return_code": 0 2024-05-08T22:45:07.914 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:07.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 msgr2=0x7fd730068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 0x7fd730068c20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd750151240 tx=0x7fd738008040 comp rx=0 tx=0).stop 2024-05-08T22:45:07.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 msgr2=0x7fd7501504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:07.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd740014040 tx=0x7fd740004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 shutdown_connections 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd730066760 0x7fd730068c20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7500afb00 0x7fd7500cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7500b4170 0x7fd7500cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 --2- 172.21.15.179:0/4112344560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7500ae720 0x7fd7501504d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 >> 172.21.15.179:0/4112344560 conn(0x7fd750098a70 msgr2=0x7fd7500cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.910+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 shutdown_connections 2024-05-08T22:45:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:07.914+0000 7fd7558e9700 1 -- 172.21.15.179:0/4112344560 wait complete. 2024-05-08T22:45:07.927 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:45:08.042 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 Processor -- start 2024-05-08T22:45:08.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 -- start start 2024-05-08T22:45:08.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99340adb90 0x7f99340cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340b1cb0 con 0x7f99340ad740 2024-05-08T22:45:08.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340b1f90 con 0x7f9934150bc0 2024-05-08T22:45:08.043 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9938882700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340b2930 con 0x7f99340adb90 2024-05-08T22:45:08.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f99340ad740 0x7f99340ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42036/0 (socket says 172.21.15.179:42036) 2024-05-08T22:45:08.044 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 -- 172.21.15.179:0/444855454 learned_addr learned my addr 172.21.15.179:0/444855454 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028102283 0 0) 0x7f99340b1cb0 con 0x7f99340ad740 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993359e700 1 --2- 172.21.15.179:0/444855454 >> v2:172.21.15.195:3300/0 conn(0x7f99340adb90 0x7f99340cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991c0035d0 con 0x7f99340ad740 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664653211 0 0) 0x7f991c0035d0 con 0x7f99340ad740 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 >> v2:172.21.15.195:3300/0 conn(0x7f99340adb90 msgr2=0x7f99340cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 --2- 172.21.15.179:0/444855454 >> v2:172.21.15.195:3300/0 conn(0x7f99340adb90 0x7f99340cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 >> v1:172.21.15.195:6789/0 conn(0x7f9934150bc0 legacy=0x7f993415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340b2270 con 0x7f99340ad740 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9928001670 con 0x7f99340ad740 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 >> v1:172.21.15.179:6789/0 conn(0x7f99340ad740 legacy=0x7f99340ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f991c003a20 0x7f991c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f991c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f991c005230 0x7f991c00fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340b2930 con 0x7f991c004870 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c010540 con 0x7f991c003a20 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c010700 con 0x7f991c005230 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993359e700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f991c005230 0x7f991c00fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9932d9d700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f991c003a20 0x7f991c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f991c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 -- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f991c005230 msgr2=0x7f991c00fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f991c005230 0x7f991c00fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 -- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f991c003a20 msgr2=0x7f991c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f991c003a20 0x7f991c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 -- 172.21.15.179:0/444855454 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340b2270 con 0x7f991c004870 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993359e700 1 -- 172.21.15.179:0/444855454 reap_dead start 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f993259c700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f991c004cf0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f992800c340 tx=0x7f991c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343104932b834453 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800ce60 con 0x7f991c004870 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9928005980 con 0x7f991c004870 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.042+0000 7f9931d9b700 1 -- 172.21.15.179:0/444855454 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928005d30 con 0x7f991c004870 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 msgr2=0x7f991c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f991c004cf0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f992800c340 tx=0x7f991c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- 172.21.15.179:0/444855454 shutdown_connections 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 --2- 172.21.15.179:0/444855454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f991c004cf0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- 172.21.15.179:0/444855454 >> 172.21.15.179:0/444855454 conn(0x7f99340ce960 msgr2=0x7f99340ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- 172.21.15.179:0/444855454 shutdown_connections 2024-05-08T22:45:08.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- 172.21.15.179:0/444855454 wait complete. 2024-05-08T22:45:08.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 Processor -- start 2024-05-08T22:45:08.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- start start 2024-05-08T22:45:08.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f993415bdc0 0x7f9934152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f993414b490 0x7f993414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993414be50 con 0x7f991c004870 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993415ce40 con 0x7f993415bdc0 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9938882700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993415c5a0 con 0x7f993414b490 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9932d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9932d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36232/0 (socket says 172.21.15.179:36232) 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f9932d9d700 1 -- 172.21.15.179:0/153358836 learned_addr learned my addr 172.21.15.179:0/153358836 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.046+0000 7f993259c700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f993415bdc0 0x7f9934152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 -- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f993414b490 msgr2=0x7f993414b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f993359e700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f993414b490 0x7f993414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f993414b490 0x7f993414b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 -- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f993415bdc0 msgr2=0x7f9934152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f993415bdc0 0x7f9934152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 -- 172.21.15.179:0/153358836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993415c880 con 0x7f991c004870 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9932d9d700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f992400f7f0 tx=0x7f992400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924005320 con 0x7f991c004870 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9924005480 con 0x7f991c004870 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993415cb60 con 0x7f991c004870 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924006aa0 con 0x7f991c004870 2024-05-08T22:45:08.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993415c220 con 0x7f991c004870 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9924006c00 con 0x7f991c004870 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99341ee830 con 0x7f991c004870 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 0x7f9920068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9924083ca0 con 0x7f991c004870 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f993259c700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 0x7f9920068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.050+0000 7f993259c700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 0x7f9920068c20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f993406acc0 tx=0x7f9928007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.054+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f99341ee830 con 0x7f991c004870 2024-05-08T22:45:08.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.262+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f99341ee830 con 0x7f991c004870 2024-05-08T22:45:08.266 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:45:08.262+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:45:08.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.278+0000 7f991b7fe700 1 -- 172.21.15.179:0/153358836 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f99341ee830 con 0x7f991c004870 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 msgr2=0x7f9920068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 0x7f9920068c20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f993406acc0 tx=0x7f9928007000 comp rx=0 tx=0).stop 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 msgr2=0x7f993415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f992400f7f0 tx=0x7f992400fbb0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 shutdown_connections 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9920066760 0x7f9920068c20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.285 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f993415bdc0 0x7f9934152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f993414b490 0x7f993414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 --2- 172.21.15.179:0/153358836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f991c004870 0x7f993415deb0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 >> 172.21.15.179:0/153358836 conn(0x7f99340ce960 msgr2=0x7f993406b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 shutdown_connections 2024-05-08T22:45:08.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.282+0000 7f9938882700 1 -- 172.21.15.179:0/153358836 wait complete. 2024-05-08T22:45:08.286 INFO:tasks.ceph.mds.d.smithi195.stderr: -2295> 2024-05-08T22:44:53.779+0000 7f8d407f6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:45:08.287 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:45:08.300 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-08T22:45:08.318 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:45:08.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 -- start start 2024-05-08T22:45:08.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 Processor -- start 2024-05-08T22:45:08.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 0x562ba97f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba9783e00 con 0x562ba974d800 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97cea00 con 0x562ba974d000 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce640 con 0x562ba974d400 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9733b86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 0x562ba97f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9734387700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9733385700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9734387700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:37612/0 (socket says 172.21.15.179:37612) 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9733385700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:47962/0 (socket says 172.21.15.179:47962) 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.338+0000 7f9734387700 1 -- 172.21.15.179:0/1652079566 learned_addr learned my addr 172.21.15.179:0/1652079566 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 -- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 msgr2=0x562ba97f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 -- 172.21.15.179:0/1652079566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 msgr2=0x562ba97f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 0x562ba97f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 -- 172.21.15.179:0/1652079566 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba976e9c0 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9734387700 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733b86700 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 0x562ba97f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9733385700 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562ba97a2de0 tx=0x562ba9823860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a84eb018eb25b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9731381700 1 -- 172.21.15.179:0/1652079566 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977d080 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9731381700 1 -- 172.21.15.179:0/1652079566 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ba9798180 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba976e1a0 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9731381700 1 -- 172.21.15.179:0/1652079566 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977d600 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562ba976e9c0 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 4 waiting for MDS map... 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9731381700 1 -- 172.21.15.179:0/1652079566 <== mon.2 v2:172.21.15.195:3301/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x562ba9786c40 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f9731381700 1 -- 172.21.15.179:0/1652079566 <== mon.2 v2:172.21.15.195:3301/0 5 ==== fsmap(e 97) v1 ==== 2573+0+0 (secure 0 0 0) 0x562ba978ef00 con 0x562ba974d400 2024-05-08T22:45:08.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 4 Got MDS map 97 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 10 main: JournalTool::main 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 4 main: JournalTool: connecting to RADOS... 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 1 Processor -- start 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.342+0000 7f97353cc500 1 -- start start 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 0x562ba97f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a43000 0x562ba97f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 0x562ba97f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce500 con 0x562ba9a42c00 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce3c0 con 0x562ba9a43000 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ba97cea00 con 0x562ba9a43400 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9734387700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 0x562ba97f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733b86700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a43000 0x562ba97f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 0x562ba97f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9734387700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 0x562ba97f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36238/0 (socket says 172.21.15.179:36238) 2024-05-08T22:45:08.351 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9734387700 1 -- 172.21.15.179:0/3156851233 learned_addr learned my addr 172.21.15.179:0/3156851233 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 -- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a43000 msgr2=0x562ba97f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a43000 0x562ba97f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 -- 172.21.15.179:0/3156851233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 msgr2=0x562ba97f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 0x562ba97f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 -- 172.21.15.179:0/3156851233 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba976eb60 con 0x562ba9a43400 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f9733385700 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 0x562ba97f4100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x562ba9a92660 tx=0x562ba98237a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2128b34c69845d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f972f37d700 1 -- 172.21.15.179:0/3156851233 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977d1e0 con 0x562ba9a43400 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f972f37d700 1 -- 172.21.15.179:0/3156851233 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562baa530a80 con 0x562ba9a43400 2024-05-08T22:45:08.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f972f37d700 1 -- 172.21.15.179:0/3156851233 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977cf20 con 0x562ba9a43400 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 msgr2=0x562ba97f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 0x562ba97f4100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x562ba9a92660 tx=0x562ba98237a0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- 172.21.15.179:0/3156851233 shutdown_connections 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a43000 0x562ba97f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43400 0x562ba97f4100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 --2- 172.21.15.179:0/3156851233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a42c00 0x562ba97f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- 172.21.15.179:0/3156851233 >> 172.21.15.179:0/3156851233 conn(0x562ba974cc00 msgr2=0x562ba9809800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- 172.21.15.179:0/3156851233 shutdown_connections 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- 172.21.15.179:0/3156851233 wait complete. 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 Processor -- start 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.346+0000 7f97353cc500 1 -- start start 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a42c00 0x562ba97f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43000 0x562ba97f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce280 con 0x562ba9a43400 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce140 con 0x562ba9a42c00 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce000 con 0x562ba9a43000 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36246/0 (socket says 172.21.15.179:36246) 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 -- 172.21.15.179:0/700842872 learned_addr learned my addr 172.21.15.179:0/700842872 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733b86700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43000 0x562ba97f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9734387700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a42c00 0x562ba97f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 -- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43000 msgr2=0x562ba97f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43000 0x562ba97f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 -- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a42c00 msgr2=0x562ba97f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a42c00 0x562ba97f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 -- 172.21.15.179:0/700842872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba976e1a0 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f9733385700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x562ba9a929f0 tx=0x562baa5d2360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f972db7a700 1 -- 172.21.15.179:0/700842872 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977d1e0 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f972db7a700 1 -- 172.21.15.179:0/700842872 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562baa530c00 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562ba976e9c0 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f972db7a700 1 -- 172.21.15.179:0/700842872 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977cf20 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba976eb60 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f97353cc500 4 main: JournalTool: resolving pool 17 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.350+0000 7f972db7a700 1 -- 172.21.15.179:0/700842872 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x562ba97ac840 con 0x562ba9a43400 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972db7a700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 0x562ba97f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972db7a700 1 -- 172.21.15.179:0/700842872 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x562ba9786a80 con 0x562ba9a43400 2024-05-08T22:45:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 4 main: JournalTool: creating IoCtx.. 2024-05-08T22:45:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 4 main: Executing for rank 0 2024-05-08T22:45:08.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9734387700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 0x562ba97f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- start start 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 Processor -- start 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9734387700 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 0x562ba97f5700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x562ba9a92840 tx=0x562ba9baac00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43c00 0x562ba97f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9aa9000 0x562ba97f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97cf040 con 0x562ba9aa8c00 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba97ce500 con 0x562ba9aa9000 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ba97cea00 con 0x562ba9a43c00 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36256/0 (socket says 172.21.15.179:36256) 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 learned_addr learned my addr 172.21.15.179:0/1180255320 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733385700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9aa9000 0x562ba97f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9734387700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43c00 0x562ba97f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43c00 msgr2=0x562ba97f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43c00 0x562ba97f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9aa9000 msgr2=0x562ba97f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9aa9000 0x562ba97f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba9af36c0 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f9733b86700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x562ba9b5d5f0 tx=0x562ba9babd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180e988eec49a0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972ab74700 1 -- 172.21.15.179:0/1180255320 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977c420 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972ab74700 1 -- 172.21.15.179:0/1180255320 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ba9798480 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972ab74700 1 -- 172.21.15.179:0/1180255320 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ba977c2c0 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba9af3520 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562ba9af1860 con 0x562ba9aa8c00 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 4 waiting for MDS map... 2024-05-08T22:45:08.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972ab74700 1 -- 172.21.15.179:0/1180255320 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x562ba9786700 con 0x562ba9aa8c00 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f972ab74700 1 -- 172.21.15.179:0/1180255320 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 97) v1 ==== 2573+0+0 (secure 0 0 0) 0x562ba9c0a300 con 0x562ba9aa8c00 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.354+0000 7f97353cc500 4 Got MDS map 97 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x562ba9aa9400 0x562ba9c24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9aa9800 con 0x562ba9aa9400 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f9734387700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x562ba9aa9400 0x562ba9c24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f9734387700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x562ba9aa9400 0x562ba9c24000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=c322544af9121754 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f9734387700 1 -- 172.21.15.179:0/1180255320 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v129'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2b40 con 0x562ba9aa9400 2024-05-08T22:45:08.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x562ba9aa9c00 0x562ba9c24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c06c00 con 0x562ba9aa9c00 2024-05-08T22:45:08.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.358+0000 7f9733385700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x562ba9aa9c00 0x562ba9c24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.362+0000 7f9733385700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x562ba9aa9c00 0x562ba9c24580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=cd2f681b11fd4bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f9733385700 1 -- 172.21.15.179:0/1180255320 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v129'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2d80 con 0x562ba9aa9c00 2024-05-08T22:45:08.369 INFO:teuthology.orchestra.run.smithi179.stdout:writing EResetJournal entry 2024-05-08T22:45:08.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c07000 con 0x562ba9aa9c00 2024-05-08T22:45:08.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x562ba9c07400 0x562ba9c24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c07800 con 0x562ba9c07400 2024-05-08T22:45:08.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x562ba9c07c00 0x562ba9c25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c38000 con 0x562ba9c07c00 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x562ba9c38400 0x562ba9c25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c38800 con 0x562ba9c38400 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c38c00 con 0x562ba9c07c00 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9c39000 con 0x562ba9aa9400 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f9734387700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x562ba9c07400 0x562ba9c24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f9733385700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x562ba9c38400 0x562ba9c25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.366+0000 7f9733b86700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x562ba9c07c00 0x562ba9c25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733385700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x562ba9c38400 0x562ba9c25600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=510ec010fa8d91a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9734387700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x562ba9c07400 0x562ba9c24b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=5160f9a025c66252 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9734387700 1 -- 172.21.15.179:0/1180255320 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(8 200.00000006 [delete] v129'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2b40 con 0x562ba9aa9400 2024-05-08T22:45:08.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733385700 1 -- 172.21.15.179:0/1180255320 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(3 200.00000001 [delete] v129'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2d80 con 0x562ba9aa9c00 2024-05-08T22:45:08.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733b86700 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x562ba9c07c00 0x562ba9c25080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=44c0a67c74e3229c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733385700 1 -- 172.21.15.179:0/1180255320 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(6 200.00000004 [delete] v129'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2d80 con 0x562ba9c38400 2024-05-08T22:45:08.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9734387700 1 -- 172.21.15.179:0/1180255320 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(4 200.00000002 [delete] v129'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2b40 con 0x562ba9c07400 2024-05-08T22:45:08.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9729371700 1 -- 172.21.15.179:0/1180255320 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x562ba9aa9800 con 0x562ba9aa9c00 2024-05-08T22:45:08.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(7 200.00000005 [delete] v129'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c38c0 con 0x562ba9c07c00 2024-05-08T22:45:08.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.370+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(5 200.00000003 [delete] v129'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c38c0 con 0x562ba9c07c00 2024-05-08T22:45:08.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f9733385700 1 -- 172.21.15.179:0/1180255320 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v129'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x562baa5c2d80 con 0x562ba9aa9c00 2024-05-08T22:45:08.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x562ba9c07c00 msgr2=0x562ba9c25080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x562ba9c07c00 0x562ba9c25080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x562ba9aa9400 msgr2=0x562ba9c24000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x562ba9aa9400 0x562ba9c24000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x562ba9aa9c00 msgr2=0x562ba9c24580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x562ba9aa9c00 0x562ba9c24580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x562ba9c07400 msgr2=0x562ba9c24b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x562ba9c07400 0x562ba9c24b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x562ba9c38400 msgr2=0x562ba9c25600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x562ba9c38400 0x562ba9c25600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f9733b86700 1 -- 172.21.15.179:0/1180255320 reap_dead start 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 msgr2=0x562ba97f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x562ba9b5d5f0 tx=0x562ba9babd40 comp rx=0 tx=0).stop 2024-05-08T22:45:08.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 shutdown_connections 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1180255320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9aa8c00 0x562ba97f3b80 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 >> 172.21.15.179:0/1180255320 conn(0x562ba9a43800 msgr2=0x562ba980a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 shutdown_connections 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1180255320 wait complete. 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 msgr2=0x562ba97f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 0x562ba97f5700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x562ba9a92840 tx=0x562ba9baac00 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 msgr2=0x562ba97f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x562ba9a929f0 tx=0x562baa5d2360 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 shutdown_connections 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x562ba9aa8800 0x562ba97f5700 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba9a43400 0x562ba97f4680 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba9a43000 0x562ba97f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/700842872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba9a42c00 0x562ba97f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 >> 172.21.15.179:0/700842872 conn(0x562ba974cc00 msgr2=0x562ba9809800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.380 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 shutdown_connections 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/700842872 wait complete. 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 msgr2=0x562ba97f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562ba97a2de0 tx=0x562ba9823860 comp rx=0 tx=0).stop 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 shutdown_connections 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba974d000 0x562ba97f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ba974d400 0x562ba97f2b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.374+0000 7f97353cc500 1 --2- 172.21.15.179:0/1652079566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ba974d800 0x562ba97f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.378+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 >> 172.21.15.179:0/1652079566 conn(0x562ba974dc00 msgr2=0x562ba9809000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.378+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 shutdown_connections 2024-05-08T22:45:08.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.378+0000 7f97353cc500 1 -- 172.21.15.179:0/1652079566 wait complete. 2024-05-08T22:45:08.382 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.081659, result: writing EResetJournal entry 2024-05-08T22:45:08.382 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-08T22:45:08.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- start start 2024-05-08T22:45:08.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 Processor -- start 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff26f83800 0x55ff27028580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 0x55ff27029080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff26fb9e00 con 0x55ff26f83400 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004a00 con 0x55ff26f83000 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004640 con 0x55ff26f83800 2024-05-08T22:45:08.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36284/0 (socket says 172.21.15.179:36284) 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 -- 172.21.15.179:0/3978027222 learned_addr learned my addr 172.21.15.179:0/3978027222 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 0x55ff27029080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff26f83800 0x55ff27028580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 -- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff26f83800 msgr2=0x55ff27028580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff26f83800 0x55ff27028580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 -- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 msgr2=0x55ff27029080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 0x55ff27029080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 -- 172.21.15.179:0/3978027222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff26fa49c0 con 0x55ff26f83400 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 0x55ff27029080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x55ff2708b8f0 tx=0x55ff27e0a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9601027be0929141 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf61f4700 1 -- 172.21.15.179:0/3978027222 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb3600 con 0x55ff26f83400 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf61f4700 1 -- 172.21.15.179:0/3978027222 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff26fce480 con 0x55ff26f83400 2024-05-08T22:45:08.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf61f4700 1 -- 172.21.15.179:0/3978027222 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb31e0 con 0x55ff26f83400 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff26fa4b60 con 0x55ff26f83400 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ff26fa4340 con 0x55ff26f83400 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 4 waiting for MDS map... 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf61f4700 1 -- 172.21.15.179:0/3978027222 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x55ff26fbcc40 con 0x55ff26f83400 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdf61f4700 1 -- 172.21.15.179:0/3978027222 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 97) v1 ==== 2573+0+0 (secure 0 0 0) 0x55ff27272300 con 0x55ff26f83400 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 4 Got MDS map 97 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 4 datascan.main: Using metadata pool output 2024-05-08T22:45:08.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.418+0000 7fbdfa23f9c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 Processor -- start 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 -- start start 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff27029600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727cc00 0x55ff2702a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004500 con 0x55ff2727c800 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff270043c0 con 0x55ff2727cc00 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004a00 con 0x55ff2727c400 2024-05-08T22:45:08.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf91fa700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff27029600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36288/0 (socket says 172.21.15.179:36288) 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff27029600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:48030/0 (socket says 172.21.15.179:48030) 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 -- 172.21.15.179:0/1482274435 learned_addr learned my addr 172.21.15.179:0/1482274435 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727cc00 0x55ff2702a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 -- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 msgr2=0x55ff27029600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff27029600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 -- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727cc00 msgr2=0x55ff2702a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727cc00 0x55ff2702a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 -- 172.21.15.179:0/1482274435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff26fa4000 con 0x55ff2727c800 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.422+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x55ff27dfc5a0 tx=0x55ff270597a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2264067b206ec0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdf41f0700 1 -- 172.21.15.179:0/1482274435 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb3080 con 0x55ff2727c800 2024-05-08T22:45:08.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdf41f0700 1 -- 172.21.15.179:0/1482274435 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff27d70d80 con 0x55ff2727c800 2024-05-08T22:45:08.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdf41f0700 1 -- 172.21.15.179:0/1482274435 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb3600 con 0x55ff2727c800 2024-05-08T22:45:08.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/1482274435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 msgr2=0x55ff27029b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x55ff27dfc5a0 tx=0x55ff270597a0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/1482274435 shutdown_connections 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727cc00 0x55ff2702a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727c800 0x55ff27029b80 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/1482274435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff27029600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/1482274435 >> 172.21.15.179:0/1482274435 conn(0x55ff26f82c00 msgr2=0x55ff2703f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/1482274435 shutdown_connections 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/1482274435 wait complete. 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 Processor -- start 2024-05-08T22:45:08.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.426+0000 7fbdfa23f9c0 1 -- start start 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff2702b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727c800 0x55ff2702ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 0x55ff2702a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004280 con 0x55ff2727cc00 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004140 con 0x55ff2727c800 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff27004000 con 0x55ff2727c400 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 0x55ff2702a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 0x55ff2702a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36294/0 (socket says 172.21.15.179:36294) 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 learned_addr learned my addr 172.21.15.179:0/2612986493 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727c800 0x55ff2702ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff2702b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 msgr2=0x55ff2702b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff2702b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727c800 msgr2=0x55ff2702ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727c800 0x55ff2702ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff26fa41a0 con 0x55ff2727cc00 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2727c800 0x55ff2702ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff2727c400 0x55ff2702b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 0x55ff2702a680 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x55ff27dfd8f0 tx=0x55ff27e0a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf29ed700 1 -- 172.21.15.179:0/2612986493 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb2f20 con 0x55ff2727cc00 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf29ed700 1 -- 172.21.15.179:0/2612986493 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff27d71800 con 0x55ff2727cc00 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf29ed700 1 -- 172.21.15.179:0/2612986493 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff26fb3080 con 0x55ff2727cc00 2024-05-08T22:45:08.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff26fa4000 con 0x55ff2727cc00 2024-05-08T22:45:08.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff26fa49c0 con 0x55ff2727cc00 2024-05-08T22:45:08.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdfa23f9c0 4 datascan.init: resolving metadata pool 17 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.430+0000 7fbdf29ed700 1 -- 172.21.15.179:0/2612986493 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x55ff26fe2840 con 0x55ff2727cc00 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf29ed700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ff273d0000 0x55ff2702b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf29ed700 1 -- 172.21.15.179:0/2612986493 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x55ff26fbca80 con 0x55ff2727cc00 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdfa23f9c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdfa23f9c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdfa23f9c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ff2727d000 0x55ff2702a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d400 con 0x55ff2727d000 2024-05-08T22:45:08.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ff273d0000 0x55ff2702b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ff2727d000 0x55ff2702a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ff2727d000 0x55ff2702a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ff273d0000 0x55ff2702b700 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55ff2708ae40 tx=0x55ff27e09c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(1 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf39ef700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ff273d0400 0x55ff273e2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d0800 con 0x55ff273d0400 2024-05-08T22:45:08.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ff273d0400 0x55ff273e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.434+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ff273d0400 0x55ff273e2000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.438+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(2 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.438+0000 7fbdf41f0700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ff273d1000 0x55ff273e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.438+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1400 con 0x55ff273d1000 2024-05-08T22:45:08.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.438+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ff273d1000 0x55ff273e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ff273d1000 0x55ff273e2580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(3 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d1000 2024-05-08T22:45:08.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d0c00 con 0x55ff2727d000 2024-05-08T22:45:08.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(4 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff2727d000 2024-05-08T22:45:08.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(5 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1400 con 0x55ff273d0400 2024-05-08T22:45:08.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(6 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf41f0700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ff273d0800 0x55ff273e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1400 con 0x55ff273d0800 2024-05-08T22:45:08.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ff273d0800 0x55ff273e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ff273d0800 0x55ff273e2b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.442+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(7 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf39ef700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ff273d1c00 0x55ff273e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d400 con 0x55ff273d1c00 2024-05-08T22:45:08.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ff273d1c00 0x55ff273e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ff273d1c00 0x55ff273e3600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(8 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404000 con 0x55ff2727d000 2024-05-08T22:45:08.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(9 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404000 con 0x55ff2727d000 2024-05-08T22:45:08.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(10 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404400 con 0x55ff273d1000 2024-05-08T22:45:08.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(11 [pgnls start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d1000 2024-05-08T22:45:08.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405000 con 0x55ff2727d000 2024-05-08T22:45:08.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(12 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.446+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404800 con 0x55ff273d0400 2024-05-08T22:45:08.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(13 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf39ef700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ff273d0c00 0x55ff273e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff273d0c00 2024-05-08T22:45:08.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ff273d0c00 0x55ff273e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf89f9700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ff273d0c00 0x55ff273e3b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(14 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404c00 con 0x55ff273d0800 2024-05-08T22:45:08.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(15 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf39ef700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ff27404000 0x55ff273e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404400 con 0x55ff27404000 2024-05-08T22:45:08.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ff27404000 0x55ff273e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.450+0000 7fbdf81f8700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ff27404000 0x55ff273e4100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(16 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405800 con 0x55ff27404000 2024-05-08T22:45:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(17 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405000 con 0x55ff273d1c00 2024-05-08T22:45:08.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(18 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405c00 con 0x55ff273d0c00 2024-05-08T22:45:08.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(19 [pgnls start_epoch 129 out=81b] v128'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d800 con 0x55ff273d0c00 2024-05-08T22:45:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdfa23f9c0 10 datascan.operator(): handling object 1099511627778.0 2024-05-08T22:45:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.454+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404800 con 0x55ff273d0c00 2024-05-08T22:45:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(21 10000000002.00000000 [call] v129'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405000 con 0x55ff273d1c00 2024-05-08T22:45:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(22 [pgnls start_epoch 129 out=49b] v127'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c000 con 0x55ff273d1c00 2024-05-08T22:45:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(23 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf39ef700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ff27405400 0x55ff273e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.458+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff27405400 2024-05-08T22:45:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ff27405400 0x55ff273e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf91fa700 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ff27405400 0x55ff273e4680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(24 [pgnls start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c400 con 0x55ff273d1c00 2024-05-08T22:45:08.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 5 ==== osd_op_reply(25 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c000 con 0x55ff2727d000 2024-05-08T22:45:08.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(26 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c800 con 0x55ff27404000 2024-05-08T22:45:08.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(27 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.462+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c400 con 0x55ff273d0c00 2024-05-08T22:45:08.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 5 ==== osd_op_reply(28 [pgnls start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740cc00 con 0x55ff273d1c00 2024-05-08T22:45:08.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 6 ==== osd_op_reply(29 [pgnls start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d800 con 0x55ff273d0800 2024-05-08T22:45:08.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(30 [pgnls start_epoch 129 out=81b] v127'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404c00 con 0x55ff273d0800 2024-05-08T22:45:08.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdfa23f9c0 10 datascan.operator(): handling object 1099511628279.0 2024-05-08T22:45:08.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.466+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404400 con 0x55ff273d0800 2024-05-08T22:45:08.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v129'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405800 con 0x55ff2727d000 2024-05-08T22:45:08.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(33 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740dc00 con 0x55ff273d1c00 2024-05-08T22:45:08.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 7 ==== osd_op_reply(34 [pgnls start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d000 con 0x55ff273d0400 2024-05-08T22:45:08.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(35 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.470+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff273d1c00 2024-05-08T22:45:08.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 8 ==== osd_op_reply(36 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d400 con 0x55ff273d1c00 2024-05-08T22:45:08.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 9 ==== osd_op_reply(37 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405000 con 0x55ff273d0400 2024-05-08T22:45:08.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(38 [pgnls start_epoch 129 out=49b] v127'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740cc00 con 0x55ff273d0800 2024-05-08T22:45:08.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(39 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c800 con 0x55ff27405400 2024-05-08T22:45:08.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.474+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(40 [pgnls start_epoch 129 out=81b] v127'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.478+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1400 con 0x55ff27405400 2024-05-08T22:45:08.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.478+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.478+0000 7fbdfa23f9c0 10 datascan.operator(): handling object 1099511627780.1 2024-05-08T22:45:08.479 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.478+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d400 con 0x55ff273d0800 2024-05-08T22:45:08.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 7 ==== osd_op_reply(42 10000000004.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c400 con 0x55ff273d1c00 2024-05-08T22:45:08.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 10 ==== osd_op_reply(43 [pgnls start_epoch 129 out=49b] v127'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d800 con 0x55ff27404000 2024-05-08T22:45:08.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(44 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740dc00 con 0x55ff27405400 2024-05-08T22:45:08.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(45 [pgnls start_epoch 129 out=81b] v128'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418000 con 0x55ff27405400 2024-05-08T22:45:08.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdfa23f9c0 10 datascan.operator(): handling object 1099511628280.0 2024-05-08T22:45:08.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.482+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418400 con 0x55ff27405400 2024-05-08T22:45:08.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.486+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v129'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.486+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418800 con 0x55ff2727d000 2024-05-08T22:45:08.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.486+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 9 ==== osd_op_reply(48 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.486+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d000 con 0x55ff273d0800 2024-05-08T22:45:08.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.490+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 8 ==== osd_op_reply(49 [pgnls start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.490+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418c00 con 0x55ff273d0800 2024-05-08T22:45:08.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.490+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.490+0000 7fbdfa23f9c0 10 datascan.operator(): handling object 1099511627780.0 2024-05-08T22:45:08.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.490+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27419000 con 0x55ff273d0800 2024-05-08T22:45:08.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 10 ==== osd_op_reply(51 10000000004.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27419400 con 0x55ff273d1000 2024-05-08T22:45:08.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(52 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d1000 2024-05-08T22:45:08.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405c00 con 0x55ff273d0400 2024-05-08T22:45:08.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(53 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740cc00 con 0x55ff27405400 2024-05-08T22:45:08.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 7 ==== osd_op_reply(54 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404800 con 0x55ff27405400 2024-05-08T22:45:08.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 8 ==== osd_op_reply(55 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.494+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d400 con 0x55ff273d1000 2024-05-08T22:45:08.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(56 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d1000 2024-05-08T22:45:08.499 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d800 con 0x55ff2727d000 2024-05-08T22:45:08.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.0 v2:172.21.15.179:6800/26067 10 ==== osd_op_reply(57 [pgnls start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff2727d000 2024-05-08T22:45:08.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c000 con 0x55ff273d1c00 2024-05-08T22:45:08.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.3 v2:172.21.15.179:6816/26068 11 ==== osd_op_reply(58 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d1c00 2024-05-08T22:45:08.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff273d0800 2024-05-08T22:45:08.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 11 ==== osd_op_reply(59 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404c00 con 0x55ff273d0c00 2024-05-08T22:45:08.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.498+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 6 ==== osd_op_reply(60 [pgnls start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d400 con 0x55ff27404000 2024-05-08T22:45:08.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(61 [pgnls start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404400 con 0x55ff273d0800 2024-05-08T22:45:08.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 12 ==== osd_op_reply(62 [pgnls start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1800 con 0x55ff273d0400 2024-05-08T22:45:08.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(63 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.505 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418800 con 0x55ff27404000 2024-05-08T22:45:08.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(64 [pgnls start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.502+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff273d1400 con 0x55ff273d0800 2024-05-08T22:45:08.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 13 ==== osd_op_reply(65 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27419400 con 0x55ff27405400 2024-05-08T22:45:08.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 9 ==== osd_op_reply(66 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405800 con 0x55ff273d0c00 2024-05-08T22:45:08.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 <== osd.2 v2:172.21.15.179:6824/26078 7 ==== osd_op_reply(67 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0b40 con 0x55ff273d0c00 2024-05-08T22:45:08.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27418c00 con 0x55ff273d0400 2024-05-08T22:45:08.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(68 [pgnls start_epoch 129 out=49b] v127'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27404c00 con 0x55ff273d0400 2024-05-08T22:45:08.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 9 ==== osd_op_reply(69 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.506+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d800 con 0x55ff27404000 2024-05-08T22:45:08.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(70 [pgnls start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff27405000 con 0x55ff273d0400 2024-05-08T22:45:08.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.5 v2:172.21.15.195:6802/26288 10 ==== osd_op_reply(71 [pgnls start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff273d0400 2024-05-08T22:45:08.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740d000 con 0x55ff273d0800 2024-05-08T22:45:08.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.1 v2:172.21.15.179:6808/26069 14 ==== osd_op_reply(72 [pgnls start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff273d0800 2024-05-08T22:45:08.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf41f0700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2727d800 con 0x55ff27404000 2024-05-08T22:45:08.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf81f8700 1 -- 172.21.15.179:0/2612986493 <== osd.7 v2:172.21.15.195:6816/26289 8 ==== osd_op_reply(73 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0fc0 con 0x55ff27404000 2024-05-08T22:45:08.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf39ef700 1 -- 172.21.15.179:0/2612986493 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55ff2740c800 con 0x55ff27405400 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.510+0000 7fbdf91fa700 1 -- 172.21.15.179:0/2612986493 <== osd.4 v2:172.21.15.195:6800/26287 10 ==== osd_op_reply(74 [pgnls start_epoch 129 out=49b] v127'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ff27cf0d80 con 0x55ff27405400 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ff2727d000 msgr2=0x55ff2702a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x55ff2727d000 0x55ff2702a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ff273d0800 msgr2=0x55ff273e2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x55ff273d0800 0x55ff273e2b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ff273d0c00 msgr2=0x55ff273e3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x55ff273d0c00 0x55ff273e3b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ff273d1c00 msgr2=0x55ff273e3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x55ff273d1c00 0x55ff273e3600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ff27405400 msgr2=0x55ff273e4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x55ff27405400 0x55ff273e4680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 reap_dead start 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ff273d0400 msgr2=0x55ff273e2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x55ff273d0400 0x55ff273e2000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ff273d1000 msgr2=0x55ff273e2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x55ff273d1000 0x55ff273e2580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ff27404000 msgr2=0x55ff273e4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x55ff27404000 0x55ff273e4100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ff273d0000 msgr2=0x55ff2702b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x55ff273d0000 0x55ff2702b700 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55ff2708ae40 tx=0x55ff27e09c20 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 msgr2=0x55ff2702a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/2612986493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2727cc00 0x55ff2702a680 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x55ff27dfd8f0 tx=0x55ff27e0a060 comp rx=0 tx=0).stop 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 reap_dead start 2024-05-08T22:45:08.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 reap_dead start 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 reap_dead start 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdf89f9700 1 -- 172.21.15.179:0/2612986493 reap_dead start 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 shutdown_connections 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 >> 172.21.15.179:0/2612986493 conn(0x55ff26f82c00 msgr2=0x55ff2703f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 shutdown_connections 2024-05-08T22:45:08.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/2612986493 wait complete. 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 msgr2=0x55ff27028b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x55ff2708b8f0 tx=0x55ff27e0a240 comp rx=0 tx=0).stop 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 shutdown_connections 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff26f83000 0x55ff27029080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff26f83400 0x55ff27028b00 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 --2- 172.21.15.179:0/3978027222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff26f83800 0x55ff27028580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 >> 172.21.15.179:0/3978027222 conn(0x55ff26f83c00 msgr2=0x55ff2703f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 shutdown_connections 2024-05-08T22:45:08.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.514+0000 7fbdfa23f9c0 1 -- 172.21.15.179:0/3978027222 wait complete. 2024-05-08T22:45:08.522 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.139719, result: 2024-05-08T22:45:08.522 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-08T22:45:08.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 -- start start 2024-05-08T22:45:08.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 Processor -- start 2024-05-08T22:45:08.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0e961800 0x560d0ea06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0e961000 0x560d0ea07080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2a00 con 0x560d0e961400 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2780 con 0x560d0e961800 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2640 con 0x560d0e961000 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36310/0 (socket says 172.21.15.179:36310) 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 -- 172.21.15.179:0/3127634556 learned_addr learned my addr 172.21.15.179:0/3127634556 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4489fc700 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0e961800 0x560d0ea06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4491fd700 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0e961000 0x560d0ea07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 -- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0e961000 msgr2=0x560d0ea07080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0e961000 0x560d0ea07080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 -- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0e961800 msgr2=0x560d0ea06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0e961800 0x560d0ea06580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.610+0000 7fd4481fb700 1 -- 172.21.15.179:0/3127634556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0e9829c0 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4481fb700 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x560d0ea69860 tx=0x560d0f7ea120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422f1703ba64bbff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4461f7700 1 -- 172.21.15.179:0/3127634556 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e991600 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4461f7700 1 -- 172.21.15.179:0/3127634556 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560d0e9ac480 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4461f7700 1 -- 172.21.15.179:0/3127634556 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e9911e0 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0e982b60 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560d0e982340 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 4 waiting for MDS map... 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4461f7700 1 -- 172.21.15.179:0/3127634556 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x560d0e99ac40 con 0x560d0e961400 2024-05-08T22:45:08.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4461f7700 1 -- 172.21.15.179:0/3127634556 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 97) v1 ==== 2573+0+0 (secure 0 0 0) 0x560d0ec62900 con 0x560d0e961400 2024-05-08T22:45:08.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 4 Got MDS map 97 2024-05-08T22:45:08.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-08T22:45:08.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 4 datascan.main: Using metadata pool output 2024-05-08T22:45:08.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:45:08.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 Processor -- start 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- start start 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e997e00 con 0x560d0ec56400 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2780 con 0x560d0ec56c00 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2500 con 0x560d0ec56800 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36316/0 (socket says 172.21.15.179:36316) 2024-05-08T22:45:08.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 -- 172.21.15.179:0/274628224 learned_addr learned my addr 172.21.15.179:0/274628224 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4489fc700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4481fb700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 -- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 msgr2=0x560d0ea07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 -- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 msgr2=0x560d0ea08100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 -- 172.21.15.179:0/274628224 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0e982000 con 0x560d0ec56400 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4489fc700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.614+0000 7fd4491fd700 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x560d0ec6c720 tx=0x560d0ea377a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138d3ea9ede570d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4441f3700 1 -- 172.21.15.179:0/274628224 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e991080 con 0x560d0ec56400 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4441f3700 1 -- 172.21.15.179:0/274628224 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560d0f748c00 con 0x560d0ec56400 2024-05-08T22:45:08.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4441f3700 1 -- 172.21.15.179:0/274628224 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e991600 con 0x560d0ec56400 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- 172.21.15.179:0/274628224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 msgr2=0x560d0ea07600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x560d0ec6c720 tx=0x560d0ea377a0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- 172.21.15.179:0/274628224 shutdown_connections 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/274628224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea07600 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- 172.21.15.179:0/274628224 >> 172.21.15.179:0/274628224 conn(0x560d0e960c00 msgr2=0x560d0ea1d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- 172.21.15.179:0/274628224 shutdown_connections 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- 172.21.15.179:0/274628224 wait complete. 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 Processor -- start 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- start start 2024-05-08T22:45:08.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea08c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e23c0 con 0x560d0ec56400 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2280 con 0x560d0ec56c00 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd44a2429c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0e9e2140 con 0x560d0ec56800 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36324/0 (socket says 172.21.15.179:36324) 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4489fc700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 learned_addr learned my addr 172.21.15.179:0/1839320969 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 msgr2=0x560d0ea08c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0ec56800 0x560d0ea08c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 msgr2=0x560d0ea08680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0ec56c00 0x560d0ea08680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0e9821a0 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x560d0ec6c840 tx=0x560d0ea36120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4429f0700 1 -- 172.21.15.179:0/1839320969 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e9911e0 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4429f0700 1 -- 172.21.15.179:0/1839320969 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560d0f748d80 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.618+0000 7fd4429f0700 1 -- 172.21.15.179:0/1839320969 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560d0e990dc0 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560d0e982000 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0e9829c0 con 0x560d0ec56400 2024-05-08T22:45:08.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 4 datascan.init: resolving metadata pool 17 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd4429f0700 1 -- 172.21.15.179:0/1839320969 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x560d0e9c0840 con 0x560d0ec56400 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd4429f0700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 0x560d0ea09700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd4429f0700 1 -- 172.21.15.179:0/1839320969 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x560d0e99aa80 con 0x560d0ec56400 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T22:45:08.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x560d0ec57000 0x560d0ea08100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 0x560d0ea09700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ec57400 con 0x560d0ec57000 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.622+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x560d0ec57000 0x560d0ea08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 0x560d0ea09700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x560d0ea68ea0 tx=0x560d0ec79da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x560d0ec57000 0x560d0ea08100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x560d0ec57800 0x560d0ea07b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda8400 con 0x560d0ec57800 2024-05-08T22:45:08.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x560d0ec57800 0x560d0ea07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x560d0ec57800 0x560d0ea07b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x560d0eda8800 0x560d0ea07600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda8c00 con 0x560d0eda8800 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x560d0eda8800 0x560d0ea07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.626+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x560d0eda8800 0x560d0ea07600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4439f2700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x560d0eddc000 0x560d0ede2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0eddc000 2024-05-08T22:45:08.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x560d0eddc000 0x560d0ede2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x560d0eddc000 0x560d0ede2000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddcc00 con 0x560d0ec57800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc800 con 0x560d0eda8800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc800 con 0x560d0eda8800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.630+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd000 con 0x560d0eddc000 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4441f3700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x560d0eda8c00 0x560d0ede2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0eda8c00 2024-05-08T22:45:08.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x560d0eda8c00 0x560d0ede2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x560d0eda8c00 0x560d0ede2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd400 con 0x560d0ec57000 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd000 con 0x560d0eda8800 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd800 con 0x560d0eda8800 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.638 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.634+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ec57400 con 0x560d0ec57800 2024-05-08T22:45:08.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc800 con 0x560d0eda8800 2024-05-08T22:45:08.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.639 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddcc00 con 0x560d0eddc000 2024-05-08T22:45:08.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4439f2700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x560d0eddd000 0x560d0ede2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd800 con 0x560d0eddd000 2024-05-08T22:45:08.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x560d0eddd000 0x560d0ede2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.641 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4489fc700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x560d0eddd000 0x560d0ede2b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0eda8c00 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4439f2700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x560d0eda8400 0x560d0ede3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eda8400 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.638+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x560d0eda8400 0x560d0ede3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4481fb700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x560d0eda8400 0x560d0ede3080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edddc00 con 0x560d0eda8400 2024-05-08T22:45:08.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2400 con 0x560d0ec57000 2024-05-08T22:45:08.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eddd000 2024-05-08T22:45:08.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.642+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 129 out=81b] v129'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9000 con 0x560d0eda8800 2024-05-08T22:45:08.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3400 con 0x560d0eddc000 2024-05-08T22:45:08.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2800 con 0x560d0ec57800 2024-05-08T22:45:08.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd400 con 0x560d0eda8800 2024-05-08T22:45:08.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2c00 con 0x560d0eda8800 2024-05-08T22:45:08.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.646+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddcc00 con 0x560d0eddc000 2024-05-08T22:45:08.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3000 con 0x560d0eda8c00 2024-05-08T22:45:08.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edddc00 con 0x560d0ec57000 2024-05-08T22:45:08.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2800 con 0x560d0eda8800 2024-05-08T22:45:08.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3000 con 0x560d0eda8800 2024-05-08T22:45:08.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9000 con 0x560d0ec57800 2024-05-08T22:45:08.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eda8800 2024-05-08T22:45:08.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.654 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.650+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0eddc000 2024-05-08T22:45:08.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3800 con 0x560d0eddd000 2024-05-08T22:45:08.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc800 con 0x560d0eda8c00 2024-05-08T22:45:08.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3c00 con 0x560d0eda8400 2024-05-08T22:45:08.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd800 con 0x560d0eda8400 2024-05-08T22:45:08.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.654+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2400 con 0x560d0ec57000 2024-05-08T22:45:08.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ec57400 con 0x560d0eddd000 2024-05-08T22:45:08.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 129 out=49b] v129'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3400 con 0x560d0ec57000 2024-05-08T22:45:08.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 129 out=49b] v127'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2800 con 0x560d0ec57000 2024-05-08T22:45:08.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4441f3700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x560d0edf2c00 0x560d0ede3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddcc00 con 0x560d0edf2c00 2024-05-08T22:45:08.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x560d0edf2c00 0x560d0ede3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.658+0000 7fd4491fd700 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x560d0edf2c00 0x560d0ede3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3000 con 0x560d0ec57000 2024-05-08T22:45:08.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edddc00 con 0x560d0eda8800 2024-05-08T22:45:08.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eda8400 2024-05-08T22:45:08.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0eddd000 2024-05-08T22:45:08.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.662+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3c00 con 0x560d0ec57000 2024-05-08T22:45:08.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd400 con 0x560d0eda8c00 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 129 out=81b] v129'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 10 datascan.operator(): handling object 100000001f7.0 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9400 con 0x560d0eda8c00 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9800 con 0x560d0eda8800 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-08T22:45:08.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9c00 con 0x560d0ec57000 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 20 datascan.get_frag_of: frag is * 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.666+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9000 con 0x560d0eda8800 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-08T22:45:08.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9400 con 0x560d0eda8800 2024-05-08T22:45:08.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-08T22:45:08.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 20 datascan.read_dentry: bravo_head not found in result 2024-05-08T22:45:08.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-08T22:45:08.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc800 con 0x560d0eda8800 2024-05-08T22:45:08.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-08T22:45:08.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.670+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddcc00 con 0x560d0eda8800 2024-05-08T22:45:08.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9800 con 0x560d0ec57000 2024-05-08T22:45:08.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2400 con 0x560d0eddc000 2024-05-08T22:45:08.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9000 con 0x560d0ec57000 2024-05-08T22:45:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3400 con 0x560d0ec57000 2024-05-08T22:45:08.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.674+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edddc00 con 0x560d0eddc000 2024-05-08T22:45:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 129 out=49b] v127'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2800 con 0x560d0eda8c00 2024-05-08T22:45:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddc400 con 0x560d0edf2c00 2024-05-08T22:45:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 129 out=49b] v127'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3000 con 0x560d0ec57000 2024-05-08T22:45:08.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 129 out=49b] v127'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ec57400 con 0x560d0eda8400 2024-05-08T22:45:08.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eddd800 con 0x560d0edf2c00 2024-05-08T22:45:08.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 129 out=49b] v129'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.678+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3800 con 0x560d0eda8800 2024-05-08T22:45:08.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eda8c00 2024-05-08T22:45:08.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 129 out=49b] v129'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0c000 con 0x560d0ec57800 2024-05-08T22:45:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.682+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3c00 con 0x560d0eddc000 2024-05-08T22:45:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0c400 con 0x560d0edf2c00 2024-05-08T22:45:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2400 con 0x560d0edf2c00 2024-05-08T22:45:08.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.688 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0c800 con 0x560d0ec57800 2024-05-08T22:45:08.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.6 v2:172.21.15.195:6824/26291 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0ec57800 2024-05-08T22:45:08.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3400 con 0x560d0eda8800 2024-05-08T22:45:08.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.0 v2:172.21.15.179:6800/26067 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eda8800 2024-05-08T22:45:08.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0cc00 con 0x560d0ec57000 2024-05-08T22:45:08.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.3 v2:172.21.15.179:6816/26068 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0ec57000 2024-05-08T22:45:08.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.686+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9c00 con 0x560d0eda8c00 2024-05-08T22:45:08.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0d000 con 0x560d0eddd000 2024-05-08T22:45:08.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 129 out=49b] v127'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9800 con 0x560d0eda8400 2024-05-08T22:45:08.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 129 out=49b] v127'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0d400 con 0x560d0eda8c00 2024-05-08T22:45:08.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 129 out=49b] v127'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0eda9000 con 0x560d0eddc000 2024-05-08T22:45:08.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.690+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0d800 con 0x560d0eda8400 2024-05-08T22:45:08.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3000 con 0x560d0eda8c00 2024-05-08T22:45:08.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0dc00 con 0x560d0edf2c00 2024-05-08T22:45:08.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf3c00 con 0x560d0eddd000 2024-05-08T22:45:08.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 <== osd.2 v2:172.21.15.179:6824/26078 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4fc0 con 0x560d0eddd000 2024-05-08T22:45:08.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0d000 con 0x560d0eddc000 2024-05-08T22:45:08.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 129 out=49b] v127'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2800 con 0x560d0eddc000 2024-05-08T22:45:08.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.694+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0cc00 con 0x560d0eda8400 2024-05-08T22:45:08.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 129 out=49b] v127'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edf2000 con 0x560d0eddc000 2024-05-08T22:45:08.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.5 v2:172.21.15.195:6802/26288 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 129 out=49b] v127'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eddc000 2024-05-08T22:45:08.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0d800 con 0x560d0eda8c00 2024-05-08T22:45:08.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.1 v2:172.21.15.179:6808/26069 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 129 out=49b] v127'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0eda8c00 2024-05-08T22:45:08.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4439f2700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0edddc00 con 0x560d0eda8400 2024-05-08T22:45:08.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4481fb700 1 -- 172.21.15.179:0/1839320969 <== osd.7 v2:172.21.15.195:6816/26289 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4b40 con 0x560d0eda8400 2024-05-08T22:45:08.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4441f3700 1 -- 172.21.15.179:0/1839320969 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 129 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x560d0ee0c000 con 0x560d0edf2c00 2024-05-08T22:45:08.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd4491fd700 1 -- 172.21.15.179:0/1839320969 <== osd.4 v2:172.21.15.195:6800/26287 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 129 out=49b] v127'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560d0f7e4d80 con 0x560d0edf2c00 2024-05-08T22:45:08.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x560d0eda8800 msgr2=0x560d0ea07600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.698+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x560d0eda8800 0x560d0ea07600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x560d0eda8c00 msgr2=0x560d0ede2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x560d0eda8c00 0x560d0ede2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x560d0eddd000 msgr2=0x560d0ede2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x560d0eddd000 0x560d0ede2b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x560d0ec57000 msgr2=0x560d0ea08100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x560d0ec57000 0x560d0ea08100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 reap_dead start 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x560d0edf2c00 msgr2=0x560d0ede3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x560d0edf2c00 0x560d0ede3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x560d0eddc000 msgr2=0x560d0ede2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x560d0eddc000 0x560d0ede2000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x560d0ec57800 msgr2=0x560d0ea07b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x560d0ec57800 0x560d0ea07b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x560d0eda8400 msgr2=0x560d0ede3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x560d0eda8400 0x560d0ede3080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd4489fc700 1 -- 172.21.15.179:0/1839320969 reap_dead start 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 msgr2=0x560d0ea09700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 0x560d0ea09700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x560d0ea68ea0 tx=0x560d0ec79da0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 msgr2=0x560d0ea09180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x560d0ec6c840 tx=0x560d0ea36120 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 shutdown_connections 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x560d0eda8000 0x560d0ea09700 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/1839320969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0ec56400 0x560d0ea09180 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 >> 172.21.15.179:0/1839320969 conn(0x560d0e960c00 msgr2=0x560d0ea1d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 shutdown_connections 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/1839320969 wait complete. 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 msgr2=0x560d0ea06b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x560d0ea69860 tx=0x560d0f7ea120 comp rx=0 tx=0).stop 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 shutdown_connections 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560d0e961000 0x560d0ea07080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560d0e961400 0x560d0ea06b00 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 --2- 172.21.15.179:0/3127634556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560d0e961800 0x560d0ea06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 >> 172.21.15.179:0/3127634556 conn(0x560d0e961c00 msgr2=0x560d0ea1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 shutdown_connections 2024-05-08T22:45:08.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.702+0000 7fd44a2429c0 1 -- 172.21.15.179:0/3127634556 wait complete. 2024-05-08T22:45:08.710 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.187829, result: 2024-05-08T22:45:08.710 DEBUG:teuthology.orchestra.run.smithi179:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-08T22:45:08.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.746+0000 7f55103d99c0 1 -- start start 2024-05-08T22:45:08.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.746+0000 7f55103d99c0 1 Processor -- start 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563553867800 0x56355390c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563553867000 0x56355390d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8a00 con 0x563553867400 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8780 con 0x563553867000 2024-05-08T22:45:08.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8640 con 0x563553867800 2024-05-08T22:45:08.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36334/0 (socket says 172.21.15.179:36334) 2024-05-08T22:45:08.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 -- 172.21.15.179:0/2501741107 learned_addr learned my addr 172.21.15.179:0/2501741107 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550f394700 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563553867000 0x56355390d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550eb93700 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563553867800 0x56355390c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 -- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563553867800 msgr2=0x56355390c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563553867800 0x56355390c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 -- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563553867000 msgr2=0x56355390d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563553867000 0x56355390d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 -- 172.21.15.179:0/2501741107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635538889c0 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550e392700 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x56355396f800 tx=0x5635546ec120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dfdf774a138ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550c38e700 1 -- 172.21.15.179:0/2501741107 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563553897600 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550c38e700 1 -- 172.21.15.179:0/2501741107 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5635538b2180 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550c38e700 1 -- 172.21.15.179:0/2501741107 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5635538971e0 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563553888b60 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563553888340 con 0x563553867400 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 4 waiting for MDS map... 2024-05-08T22:45:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550c38e700 1 -- 172.21.15.179:0/2501741107 <== mon.0 v2:172.21.15.179:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x5635538a0c40 con 0x563553867400 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f550c38e700 1 -- 172.21.15.179:0/2501741107 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 97) v1 ==== 2573+0+0 (secure 0 0 0) 0x563553b6a000 con 0x563553867400 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 4 Got MDS map 97 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 4 datascan.main: Using metadata pool output 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.750+0000 7f55103d99c0 4 datascan.main: connecting to RADOS... 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 Processor -- start 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- start start 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 0x56355390e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355389de00 con 0x5635546f0800 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8780 con 0x5635546f0c00 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8500 con 0x5635546f0400 2024-05-08T22:45:08.757 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550e392700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 0x56355390e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550f394700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36336/0 (socket says 172.21.15.179:36336) 2024-05-08T22:45:08.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 -- 172.21.15.179:0/2644234340 learned_addr learned my addr 172.21.15.179:0/2644234340 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 -- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 msgr2=0x56355390d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 -- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 msgr2=0x56355390e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:08.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 0x56355390e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 -- 172.21.15.179:0/2644234340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563553888000 con 0x5635546f0800 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550e392700 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 0x56355390e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550eb93700 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x563553b6e690 tx=0x56355393d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2513689d26253db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550a38a700 1 -- 172.21.15.179:0/2644234340 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563553897080 con 0x5635546f0800 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550a38a700 1 -- 172.21.15.179:0/2644234340 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563554658d80 con 0x5635546f0800 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f550a38a700 1 -- 172.21.15.179:0/2644234340 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563553897600 con 0x5635546f0800 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- 172.21.15.179:0/2644234340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 msgr2=0x56355390db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x563553b6e690 tx=0x56355393d7a0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- 172.21.15.179:0/2644234340 shutdown_connections 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0c00 0x56355390e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0800 0x56355390db80 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2644234340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- 172.21.15.179:0/2644234340 >> 172.21.15.179:0/2644234340 conn(0x563553866c00 msgr2=0x563553923800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- 172.21.15.179:0/2644234340 shutdown_connections 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.754+0000 7f55103d99c0 1 -- 172.21.15.179:0/2644234340 wait complete. 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 Processor -- start 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- start start 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0800 0x56355390ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635538e83c0 con 0x5635546f0c00 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8280 con 0x5635546f0800 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635538e8140 con 0x5635546f0400 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36348/0 (socket says 172.21.15.179:36348) 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 learned_addr learned my addr 172.21.15.179:0/4131029243 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0800 0x56355390ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 msgr2=0x56355390f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5635546f0400 0x56355390f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0800 msgr2=0x56355390ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635546f0800 0x56355390ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635538881a0 con 0x5635546f0c00 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x563553b6fa40 tx=0x5635546ec060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f5508b87700 1 -- 172.21.15.179:0/4131029243 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563553896f20 con 0x5635546f0c00 2024-05-08T22:45:08.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f5508b87700 1 -- 172.21.15.179:0/4131029243 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563554658480 con 0x5635546f0c00 2024-05-08T22:45:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563553888000 con 0x5635546f0c00 2024-05-08T22:45:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f5508b87700 1 -- 172.21.15.179:0/4131029243 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563553897080 con 0x5635546f0c00 2024-05-08T22:45:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635538889c0 con 0x5635546f0c00 2024-05-08T22:45:08.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.758+0000 7f55103d99c0 4 datascan.init: resolving metadata pool 17 2024-05-08T22:45:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f5508b87700 1 -- 172.21.15.179:0/4131029243 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x5635538c6840 con 0x5635546f0c00 2024-05-08T22:45:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f5508b87700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 0x56355390f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f5508b87700 1 -- 172.21.15.179:0/4131029243 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x5635538a0a80 con 0x5635546f0c00 2024-05-08T22:45:08.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f55103d99c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f55103d99c0 4 datascan.main: resolving metadata pool 17 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5635546f1000 0x56355390e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x5635546f1400 con 0x5635546f1000 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 0x56355390f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5635546f1000 0x56355390e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5635546f1000 0x56355390e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 0x56355390f700 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x56355396ecf0 tx=0x563553b7bc20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== osd_op_reply(1 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x5635546f1400 0x563553cca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc400 con 0x5635546f1400 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x5635546f1400 0x563553cca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.762+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x5635546f1400 0x563553cca000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== osd_op_reply(2 [pgnls start_epoch 129 out=177b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc400 con 0x5635546f1000 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== osd_op_reply(3 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563553cbc400 0x563553cca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc800 con 0x563553cbc400 2024-05-08T22:45:08.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563553cbc400 0x563553cca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563553cbc400 0x563553cca580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== osd_op_reply(4 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc800 con 0x563553cbc400 2024-05-08T22:45:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== osd_op_reply(5 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc800 con 0x5635546f1400 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== osd_op_reply(6 [pgnls start_epoch 129 out=149b] v129'40 uv40 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbc800 con 0x5635546f1000 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 3 ==== osd_op_reply(7 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x563553cbc800 0x563553ccab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbcc00 con 0x563553cbc800 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.766+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x563553cbc800 0x563553ccab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x563553cbc800 0x563553ccab00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== osd_op_reply(8 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x563553cbcc00 0x563553ccb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd000 con 0x563553cbcc00 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x563553cbcc00 0x563553ccb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x563553cbcc00 0x563553ccb080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== osd_op_reply(9 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd000 con 0x5635546f1400 2024-05-08T22:45:08.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 3 ==== osd_op_reply(10 [pgnls start_epoch 129 out=81b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd000 con 0x5635546f1000 2024-05-08T22:45:08.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 4 ==== osd_op_reply(11 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.770+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd000 con 0x563553cbc400 2024-05-08T22:45:08.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 3 ==== osd_op_reply(12 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x563553cbd000 0x563553ccb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbdc00 con 0x563553cbd000 2024-05-08T22:45:08.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x563553cbd000 0x563553ccb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x563553cbd000 0x563553ccb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== osd_op_reply(13 [pgnls start_epoch 129 out=77b] v128'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563553cbdc00 0x563553ccbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce2000 con 0x563553cbdc00 2024-05-08T22:45:08.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563553cbdc00 0x563553ccbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.774+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563553cbdc00 0x563553ccbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== osd_op_reply(14 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550e392700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x563553ce2000 0x563553ccc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:08.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553ce2000 2024-05-08T22:45:08.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x563553ce2000 0x563553ccc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:08.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x563553ce2000 0x563553ccc680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:08.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== osd_op_reply(15 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4d80 con 0x563553ce2000 2024-05-08T22:45:08.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbc800 2024-05-08T22:45:08.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== osd_op_reply(16 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbc400 2024-05-08T22:45:08.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 4 ==== osd_op_reply(17 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1400 2024-05-08T22:45:08.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 4 ==== osd_op_reply(18 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.778+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1400 2024-05-08T22:45:08.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 5 ==== osd_op_reply(19 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbdc00 2024-05-08T22:45:08.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== osd_op_reply(20 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbd000 2024-05-08T22:45:08.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== osd_op_reply(21 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbdc00 2024-05-08T22:45:08.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 3 ==== osd_op_reply(22 [pgnls start_epoch 129 out=97b] v128'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1000 2024-05-08T22:45:08.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 5 ==== osd_op_reply(23 [pgnls start_epoch 129 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.782+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1000 2024-05-08T22:45:08.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 6 ==== osd_op_reply(24 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbcc00 2024-05-08T22:45:08.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== osd_op_reply(25 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1400 2024-05-08T22:45:08.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 6 ==== osd_op_reply(26 [pgnls start_epoch 129 out=98b] v128'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbc400 2024-05-08T22:45:08.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 5 ==== osd_op_reply(27 [pgnls start_epoch 129 out=76b] v129'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbcc00 2024-05-08T22:45:08.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 3 ==== osd_op_reply(28 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbc400 2024-05-08T22:45:08.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 6 ==== osd_op_reply(29 [pgnls start_epoch 129 out=105b] v129'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.786+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbd000 2024-05-08T22:45:08.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 3 ==== osd_op_reply(30 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbc800 2024-05-08T22:45:08.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 3 ==== osd_op_reply(31 [pgnls start_epoch 129 out=73b] v129'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x563553cbd000 2024-05-08T22:45:08.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 4 ==== osd_op_reply(32 [pgnls start_epoch 129 out=101b] v129'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-08T22:45:08.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5635546f1800 con 0x5635546f1400 2024-05-08T22:45:08.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-08T22:45:08.793 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3000 con 0x5635546f1400 2024-05-08T22:45:08.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-08T22:45:08.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.790+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3400 con 0x5635546f1400 2024-05-08T22:45:08.794 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-08T22:45:08.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3800 con 0x5635546f1400 2024-05-08T22:45:08.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-08T22:45:08.795 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3c00 con 0x5635546f1400 2024-05-08T22:45:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-08T22:45:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T22:45:08.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3400 con 0x5635546f1400 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv35 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-08T22:45:08.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3800 con 0x563553cbc800 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3c00 con 0x5635546f1400 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-08T22:45:08.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.794+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3000 con 0x563553ce2000 2024-05-08T22:45:08.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x563553ce2000 2024-05-08T22:45:08.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-08T22:45:08.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5635546f1800 con 0x563553cbdc00 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-08T22:45:08.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0000 con 0x5635546f1000 2024-05-08T22:45:08.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-08T22:45:08.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T22:45:08.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-08T22:45:08.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0400 con 0x5635546f1400 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0800 con 0x563553cbc400 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-08T22:45:08.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.798+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0c00 con 0x563553cbc400 2024-05-08T22:45:08.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 20 datascan.scan_links: injected ino detected 2024-05-08T22:45:08.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-08T22:45:08.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T22:45:08.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-08T22:45:08.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T22:45:08.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-08T22:45:08.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1000 con 0x563553cbd000 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 8 ==== osd_op_reply(48 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 15 ==== osd_op_reply(49 [pgnls start_epoch 129 out=177b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.805 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.802+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 9 ==== osd_op_reply(50 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 9 ==== osd_op_reply(51 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 10 ==== osd_op_reply(52 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 16 ==== osd_op_reply(53 [pgnls start_epoch 129 out=149b] v129'40 uv40 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.808 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 10 ==== osd_op_reply(54 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.809 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc800 2024-05-08T22:45:08.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 5 ==== osd_op_reply(55 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.810 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.806+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbcc00 2024-05-08T22:45:08.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 4 ==== osd_op_reply(56 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 17 ==== osd_op_reply(57 [pgnls start_epoch 129 out=81b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 11 ==== osd_op_reply(58 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 11 ==== osd_op_reply(59 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.810+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbd000 2024-05-08T22:45:08.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 6 ==== osd_op_reply(60 [pgnls start_epoch 129 out=77b] v128'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbdc00 2024-05-08T22:45:08.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 5 ==== osd_op_reply(61 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553ce2000 2024-05-08T22:45:08.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.2 v2:172.21.15.179:6824/26078 3 ==== osd_op_reply(62 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4d80 con 0x563553ce2000 2024-05-08T22:45:08.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc800 2024-05-08T22:45:08.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 6 ==== osd_op_reply(63 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 12 ==== osd_op_reply(64 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.814+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 18 ==== osd_op_reply(65 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 19 ==== osd_op_reply(66 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbdc00 2024-05-08T22:45:08.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 6 ==== osd_op_reply(67 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbd000 2024-05-08T22:45:08.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 7 ==== osd_op_reply(68 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbdc00 2024-05-08T22:45:08.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 7 ==== osd_op_reply(69 [pgnls start_epoch 129 out=97b] v128'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.818+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 12 ==== osd_op_reply(70 [pgnls start_epoch 129 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1000 2024-05-08T22:45:08.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 13 ==== osd_op_reply(71 [pgnls start_epoch 129 out=73b] v128'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbcc00 2024-05-08T22:45:08.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 5 ==== osd_op_reply(72 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x5635546f1400 2024-05-08T22:45:08.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 20 ==== osd_op_reply(73 [pgnls start_epoch 129 out=98b] v128'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.825 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 13 ==== osd_op_reply(74 [pgnls start_epoch 129 out=76b] v129'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.822+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbcc00 2024-05-08T22:45:08.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.4 v2:172.21.15.195:6800/26287 6 ==== osd_op_reply(75 [pgnls start_epoch 129 out=49b] v129'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4d80 con 0x563553cbcc00 2024-05-08T22:45:08.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc400 2024-05-08T22:45:08.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 14 ==== osd_op_reply(76 [pgnls start_epoch 129 out=105b] v129'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.827 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbd000 2024-05-08T22:45:08.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 8 ==== osd_op_reply(77 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbc800 2024-05-08T22:45:08.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 7 ==== osd_op_reply(78 [pgnls start_epoch 129 out=73b] v129'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.829 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1400 con 0x563553cbd000 2024-05-08T22:45:08.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 9 ==== osd_op_reply(79 [pgnls start_epoch 129 out=101b] v129'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-08T22:45:08.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.826+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1800 con 0x5635546f1400 2024-05-08T22:45:08.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-08T22:45:08.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1c00 con 0x5635546f1400 2024-05-08T22:45:08.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-08T22:45:08.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0000 con 0x5635546f1400 2024-05-08T22:45:08.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-08T22:45:08.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0800 con 0x5635546f1400 2024-05-08T22:45:08.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-08T22:45:08.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0400 con 0x5635546f1400 2024-05-08T22:45:08.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-08T22:45:08.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce2c00 con 0x5635546f1400 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv35 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.830+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce2800 con 0x563553cbc800 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.1 v2:172.21.15.179:6808/26069 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbc800 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-08T22:45:08.835 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce2400 con 0x5635546f1400 2024-05-08T22:45:08.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-08T22:45:08.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T22:45:08.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-08T22:45:08.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3400 con 0x563553ce2000 2024-05-08T22:45:08.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.2 v2:172.21.15.179:6824/26078 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4d80 con 0x563553ce2000 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3c00 con 0x563553cbdc00 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f550e392700 1 -- 172.21.15.179:0/4131029243 <== osd.5 v2:172.21.15.195:6802/26288 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4fc0 con 0x563553cbdc00 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-08T22:45:08.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.834+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3000 con 0x5635546f1000 2024-05-08T22:45:08.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-08T22:45:08.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T22:45:08.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-08T22:45:08.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cbd400 con 0x5635546f1400 2024-05-08T22:45:08.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-08T22:45:08.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T22:45:08.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-08T22:45:08.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1800 con 0x563553cbc400 2024-05-08T22:45:08.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-08T22:45:08.841 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf1c00 con 0x563553cbc400 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 20 datascan.scan_links: injected ino detected 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T22:45:08.842 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-08T22:45:08.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T22:45:08.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-08T22:45:08.843 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.838+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0000 con 0x563553cbd000 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.6 v2:172.21.15.195:6824/26291 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbd000 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-08T22:45:08.844 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0800 con 0x563553cbc400 2024-05-08T22:45:08.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-08T22:45:08.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 20 datascan.scan_links: first is now 2 2024-05-08T22:45:08.845 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.842+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553cf0400 con 0x563553cbc400 2024-05-08T22:45:08.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.0 v2:172.21.15.179:6800/26067 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5635545d4b40 con 0x563553cbc400 2024-05-08T22:45:08.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-08T22:45:08.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 10 datascan.scan_links: updating inotable 2024-05-08T22:45:08.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3800 con 0x5635546f1400 2024-05-08T22:45:08.847 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-08T22:45:08.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-08T22:45:08.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.846+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce2800 con 0x5635546f1400 2024-05-08T22:45:08.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f550f394700 1 -- 172.21.15.179:0/4131029243 <== osd.3 v2:172.21.15.179:6816/26068 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v129'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5635545d4d80 con 0x5635546f1400 2024-05-08T22:45:08.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 10 datascan.scan_links: updating snaptable 2024-05-08T22:45:08.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x563553ce3c00 con 0x5635546f1000 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 <== osd.7 v2:172.21.15.195:6816/26289 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5635545d4b40 con 0x5635546f1000 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563553cbc400 msgr2=0x563553cca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x563553cbc400 0x563553cca580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x563553cbc800 msgr2=0x563553ccab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x563553cbc800 0x563553ccab00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x563553ce2000 msgr2=0x563553ccc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x563553ce2000 0x563553ccc680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 reap_dead start 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x5635546f1400 msgr2=0x563553cca000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x5635546f1400 0x563553cca000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x563553cbcc00 msgr2=0x563553ccb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x563553cbcc00 0x563553ccb080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563553cbdc00 msgr2=0x563553ccbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x563553cbdc00 0x563553ccbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x563553cbd000 msgr2=0x563553ccb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x563553cbd000 0x563553ccb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5635546f1000 msgr2=0x56355390e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x5635546f1000 0x56355390e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f550eb93700 1 -- 172.21.15.179:0/4131029243 reap_dead start 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 msgr2=0x56355390f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 0x56355390f700 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x56355396ecf0 tx=0x563553b7bc20 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 msgr2=0x56355390e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x563553b6fa40 tx=0x5635546ec060 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 shutdown_connections 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x563553cbc000 0x56355390f700 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/4131029243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635546f0c00 0x56355390e680 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 >> 172.21.15.179:0/4131029243 conn(0x563553866c00 msgr2=0x563553923800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 shutdown_connections 2024-05-08T22:45:08.855 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/4131029243 wait complete. 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 msgr2=0x56355390cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x56355396f800 tx=0x5635546ec120 comp rx=0 tx=0).stop 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 shutdown_connections 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563553867000 0x56355390d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563553867400 0x56355390cb00 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 --2- 172.21.15.179:0/2501741107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563553867800 0x56355390c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 >> 172.21.15.179:0/2501741107 conn(0x563553867c00 msgr2=0x563553923000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 shutdown_connections 2024-05-08T22:45:08.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:08.850+0000 7f55103d99c0 1 -- 172.21.15.179:0/2501741107 wait complete. 2024-05-08T22:45:08.859 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.148928, result: 2024-05-08T22:45:08.860 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:45:08.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:45:08.893 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-08T22:45:08.894 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:45:08.895 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.896 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-08T22:45:08.897 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.898 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-08T22:45:08.899 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:45:08.900 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-08T22:45:08.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-08T22:45:08.902 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:45:08.903 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-08T22:45:08.904 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.905 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = False 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = True 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.906 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-05-08T22:45:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:45:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-05-08T22:45:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:45:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-05-08T22:45:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:45:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-05-08T22:45:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:45:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-05-08T22:45:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = False 2024-05-08T22:45:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-05-08T22:45:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2024-05-08T22:45:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:45:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.920 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:45:08.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-08T22:45:08.964 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-08T22:45:08.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.966 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:45:08.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-08T22:45:08.968 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-08T22:45:08.969 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:45:08.970 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-08T22:45:08.971 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.972 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.973 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-08T22:45:08.974 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = True 2024-05-08T22:45:08.975 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-08T22:45:08.975 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = True 2024-05-08T22:45:08.975 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-08T22:45:08.975 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-08T22:45:08.975 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-08T22:45:08.997 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-05-08T22:45:08.997 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-05-08T22:45:08.997 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.998 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-05-08T22:45:08.999 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.179,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 88a906d6-7c22-496e-b9c7-b34c51a47fa8 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.000 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.001 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-05-08T22:45:09.002 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:09.003 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-05-08T22:45:09.004 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T22:45:09.005 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-05-08T22:45:09.006 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-05-08T22:45:09.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2024-05-08T22:45:09.008 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-05-08T22:45:09.009 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2024-05-08T22:45:09.009 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-05-08T22:45:09.009 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-05-08T22:45:09.009 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-05-08T22:45:09.010 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T22:45:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 Processor -- start 2024-05-08T22:45:09.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 -- start start 2024-05-08T22:45:09.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f44940b1a50 0x7f44940b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449406dd50 con 0x7f449414ae30 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449406e030 con 0x7f44940b2c10 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f449a997700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449406e9d0 con 0x7f44940b1a50 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499995700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f449414ae30 0x7f4494165b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42080/0 (socket says 172.21.15.179:42080) 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499995700 1 -- 172.21.15.179:0/3327919851 learned_addr learned my addr 172.21.15.179:0/3327919851 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499194700 1 --2- 172.21.15.179:0/3327919851 >> v2:172.21.15.195:3300/0 conn(0x7f44940b1a50 0x7f44940b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4498993700 1 -- 172.21.15.179:0/3327919851 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078103538 0 0) 0x7f449406dd50 con 0x7f449414ae30 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4498993700 1 -- 172.21.15.179:0/3327919851 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44800035d0 con 0x7f449414ae30 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499194700 1 -- 172.21.15.179:0/3327919851 >> v1:172.21.15.195:6789/0 conn(0x7f44940b2c10 legacy=0x7f449406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:09.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499194700 1 -- 172.21.15.179:0/3327919851 >> v1:172.21.15.179:6789/0 conn(0x7f449414ae30 legacy=0x7f4494165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:09.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499194700 1 -- 172.21.15.179:0/3327919851 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449406e310 con 0x7f44940b1a50 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4499194700 1 --2- 172.21.15.179:0/3327919851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44940b1a50 0x7f44940b26d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d35508bb9a9c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4498993700 1 -- 172.21.15.179:0/3327919851 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00e070 con 0x7f44940b1a50 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4498993700 1 -- 172.21.15.179:0/3327919851 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c00bb20 con 0x7f44940b1a50 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.142+0000 7f4498993700 1 -- 172.21.15.179:0/3327919851 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00be90 con 0x7f44940b1a50 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- 172.21.15.179:0/3327919851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44940b1a50 msgr2=0x7f44940b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 --2- 172.21.15.179:0/3327919851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44940b1a50 0x7f44940b26d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- 172.21.15.179:0/3327919851 shutdown_connections 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 --2- 172.21.15.179:0/3327919851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44940b1a50 0x7f44940b26d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- 172.21.15.179:0/3327919851 >> 172.21.15.179:0/3327919851 conn(0x7f44940cea30 msgr2=0x7f44940cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:09.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- 172.21.15.179:0/3327919851 shutdown_connections 2024-05-08T22:45:09.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- 172.21.15.179:0/3327919851 wait complete. 2024-05-08T22:45:09.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 Processor -- start 2024-05-08T22:45:09.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- start start 2024-05-08T22:45:09.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:09.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44940b2c10 0x7f4494141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:09.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f449414ae30 0x7f4494141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494151600 con 0x7f44940b1a50 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494151810 con 0x7f449414ae30 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f449a997700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4494151af0 con 0x7f44940b2c10 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f4499995700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f4499995700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36354/0 (socket says 172.21.15.179:36354) 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.146+0000 7f4499995700 1 -- 172.21.15.179:0/2120180554 learned_addr learned my addr 172.21.15.179:0/2120180554 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f449a196700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f449414ae30 0x7f4494141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499194700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44940b2c10 0x7f4494141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 -- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44940b2c10 msgr2=0x7f4494141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44940b2c10 0x7f4494141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 -- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f449414ae30 msgr2=0x7f4494141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f449414ae30 0x7f4494141b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 -- 172.21.15.179:0/2120180554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c009a70 con 0x7f44940b1a50 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f4499995700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f448800c4d0 tx=0x7f448800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:09.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488005d90 con 0x7f44940b1a50 2024-05-08T22:45:09.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44880070d0 con 0x7f44940b1a50 2024-05-08T22:45:09.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4494151dd0 con 0x7f44940b1a50 2024-05-08T22:45:09.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488007480 con 0x7f44940b1a50 2024-05-08T22:45:09.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44941420c0 con 0x7f44940b1a50 2024-05-08T22:45:09.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44941522f0 con 0x7f44940b1a50 2024-05-08T22:45:09.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.150+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44880075e0 con 0x7f44940b1a50 2024-05-08T22:45:09.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.154+0000 7f44867fc700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 0x7f4478068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:09.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.154+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f4488083770 con 0x7f44940b1a50 2024-05-08T22:45:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.154+0000 7f4499194700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 0x7f4478068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:09.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.154+0000 7f4499194700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 0x7f4478068bf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f44940b1e90 tx=0x7f448c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:09.163 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.162+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f44880508f0 con 0x7f44940b1a50 2024-05-08T22:45:09.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:09.370+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f449404e700 con 0x7f44940b1a50 2024-05-08T22:45:10.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.290+0000 7f44867fc700 1 -- 172.21.15.179:0/2120180554 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v99) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4488053f10 con 0x7f44940b1a50 2024-05-08T22:45:10.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 msgr2=0x7f4478068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 0x7f4478068bf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f44940b1e90 tx=0x7f448c011000 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 msgr2=0x7f449413ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f448800c4d0 tx=0x7f448800c7e0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 shutdown_connections 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4478066730 0x7f4478068bf0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f449414ae30 0x7f4494141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44940b2c10 0x7f4494141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 --2- 172.21.15.179:0/2120180554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44940b1a50 0x7f449413ea20 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 >> 172.21.15.179:0/2120180554 conn(0x7f44940cea30 msgr2=0x7f4494148480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 shutdown_connections 2024-05-08T22:45:10.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.298+0000 7f449a997700 1 -- 172.21.15.179:0/2120180554 wait complete. 2024-05-08T22:45:10.301 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T22:45:10.318 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:10.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.434+0000 7f3211b56700 1 Processor -- start 2024-05-08T22:45:10.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.434+0000 7f3211b56700 1 -- start start 2024-05-08T22:45:10.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 0x7f320c0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 0x7f320c0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320c0d0900 con 0x7f320c15d290 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c06e5b0 con 0x7f320c0b3a80 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0cb770 con 0x7f320c15ceb0 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211355700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 0x7f320c0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3203fff700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f320c15d290 0x7f320c0cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:36128/0 (socket says 172.21.15.179:36128) 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3203fff700 1 -- 172.21.15.179:0/2701074391 learned_addr learned my addr 172.21.15.179:0/2701074391 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:10.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 --2- 172.21.15.179:0/2701074391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 0x7f320c0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 -- 172.21.15.179:0/2701074391 >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 msgr2=0x7f320c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 --2- 172.21.15.179:0/2701074391 >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 0x7f320c0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 -- 172.21.15.179:0/2701074391 >> v1:172.21.15.195:6789/0 conn(0x7f320c15d290 legacy=0x7f320c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 -- 172.21.15.179:0/2701074391 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0cba50 con 0x7f320c15ceb0 2024-05-08T22:45:10.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211355700 1 --2- 172.21.15.179:0/2701074391 >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 0x7f320c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3210b54700 1 --2- 172.21.15.179:0/2701074391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 0x7f320c0cd5e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3208009850 tx=0x7f3208009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58c25d781e050ffa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f32037fe700 1 -- 172.21.15.179:0/2701074391 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011830 con 0x7f320c15ceb0 2024-05-08T22:45:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f32037fe700 1 -- 172.21.15.179:0/2701074391 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208011990 con 0x7f320c15ceb0 2024-05-08T22:45:10.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f32037fe700 1 -- 172.21.15.179:0/2701074391 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011d40 con 0x7f320c15ceb0 2024-05-08T22:45:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- 172.21.15.179:0/2701074391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 msgr2=0x7f320c0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 --2- 172.21.15.179:0/2701074391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 0x7f320c0cd5e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3208009850 tx=0x7f3208009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:10.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- 172.21.15.179:0/2701074391 shutdown_connections 2024-05-08T22:45:10.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 --2- 172.21.15.179:0/2701074391 >> v2:172.21.15.195:3300/0 conn(0x7f320c0b3a80 0x7f320c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 --2- 172.21.15.179:0/2701074391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c15ceb0 0x7f320c0cd5e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- 172.21.15.179:0/2701074391 >> 172.21.15.179:0/2701074391 conn(0x7f320c0b2780 msgr2=0x7f320c0b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:10.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- 172.21.15.179:0/2701074391 shutdown_connections 2024-05-08T22:45:10.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.438+0000 7f3211b56700 1 -- 172.21.15.179:0/2701074391 wait complete. 2024-05-08T22:45:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 Processor -- start 2024-05-08T22:45:10.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 -- start start 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c0b3a80 0x7f320c0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c0c86d0 0x7f320c14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0c9ac0 con 0x7f320c15ceb0 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0c8eb0 con 0x7f320c0c86d0 2024-05-08T22:45:10.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211b56700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0c9190 con 0x7f320c0b3a80 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3210b54700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c0b3a80 0x7f320c0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36358/0 (socket says 172.21.15.179:36358) 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 -- 172.21.15.179:0/4237194655 learned_addr learned my addr 172.21.15.179:0/4237194655 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3211355700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c0c86d0 0x7f320c14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 -- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c0b3a80 msgr2=0x7f320c0c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c0b3a80 0x7f320c0c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 -- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c0c86d0 msgr2=0x7f320c14e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c0c86d0 0x7f320c14e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.442+0000 7f3203fff700 1 -- 172.21.15.179:0/4237194655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009a70 con 0x7f320c15ceb0 2024-05-08T22:45:10.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f3203fff700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c410 tx=0x7f31fc00c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:10.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc005d90 con 0x7f320c15ceb0 2024-05-08T22:45:10.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00ce10 con 0x7f320c15ceb0 2024-05-08T22:45:10.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32080094b0 con 0x7f320c15ceb0 2024-05-08T22:45:10.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0072f0 con 0x7f320c15ceb0 2024-05-08T22:45:10.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c0c9750 con 0x7f320c15ceb0 2024-05-08T22:45:10.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c0c9c30 con 0x7f320c15ceb0 2024-05-08T22:45:10.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.446+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31fc007450 con 0x7f320c15ceb0 2024-05-08T22:45:10.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.450+0000 7f32017fa700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 0x7f31f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:10.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.450+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f31fc0836e0 con 0x7f320c15ceb0 2024-05-08T22:45:10.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.450+0000 7f3210b54700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 0x7f31f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:10.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.450+0000 7f3210b54700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 0x7f31f8068bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f32080090b0 tx=0x7f3208008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:10.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.458+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f31fc0507d0 con 0x7f320c15ceb0 2024-05-08T22:45:10.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.662+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f320c04e700 con 0x7f320c15ceb0 2024-05-08T22:45:10.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f32017fa700 1 -- 172.21.15.179:0/4237194655 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4750 (secure 0 0 0) 0x7f31fc017090 con 0x7f320c15ceb0 2024-05-08T22:45:10.667 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:10.667 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:10.295375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:replay","state_seq":1,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:10.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 msgr2=0x7f31f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 0x7f31f8068bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f32080090b0 tx=0x7f3208008030 comp rx=0 tx=0).stop 2024-05-08T22:45:10.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 msgr2=0x7f320c0c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:10.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c410 tx=0x7f31fc00c720 comp rx=0 tx=0).stop 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 shutdown_connections 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f31f8066730 0x7f31f8068bf0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c0c86d0 0x7f320c14e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f320c15ceb0 0x7f320c0c9f50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 --2- 172.21.15.179:0/4237194655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f320c0b3a80 0x7f320c0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 >> 172.21.15.179:0/4237194655 conn(0x7f320c0b2780 msgr2=0x7f320c0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 shutdown_connections 2024-05-08T22:45:10.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:10.666+0000 7f3211b56700 1 -- 172.21.15.179:0/4237194655 wait complete. 2024-05-08T22:45:10.671 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 100 2024-05-08T22:45:11.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:45:10.295375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9820}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9820': {'gid': 9820, 'name': 'd', 'rank': 0, 'incarnation': 100, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/357696906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 357696906}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 357696906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:45:11.086 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9820:up:replay 2024-05-08T22:45:12.087 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:12.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.206+0000 7fef2359e700 1 Processor -- start 2024-05-08T22:45:12.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.206+0000 7fef2359e700 1 -- start start 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef1c0e3b40 0x7fef1c0e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef1c006150 0x7fef1c0e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0e49a0 con 0x7fef1c0e0250 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c0540b0 con 0x7fef1c006150 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c0532f0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef21d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fef1c0e0250 0x7fef1c038ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42096/0 (socket says 172.21.15.179:42096) 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef21d9b700 1 -- 172.21.15.179:0/2145455057 learned_addr learned my addr 172.21.15.179:0/2145455057 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 --2- 172.21.15.179:0/2145455057 >> v2:172.21.15.179:3300/0 conn(0x7fef1c006150 0x7fef1c0e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2259c700 1 --2- 172.21.15.179:0/2145455057 >> v2:172.21.15.195:3300/0 conn(0x7fef1c0e3b40 0x7fef1c0e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2159a700 1 -- 172.21.15.179:0/2145455057 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208235289 0 0) 0x7fef1c0e49a0 con 0x7fef1c0e0250 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2159a700 1 -- 172.21.15.179:0/2145455057 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef0c0035d0 con 0x7fef1c0e0250 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 -- 172.21.15.179:0/2145455057 >> v2:172.21.15.195:3300/0 conn(0x7fef1c0e3b40 msgr2=0x7fef1c0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 --2- 172.21.15.179:0/2145455057 >> v2:172.21.15.195:3300/0 conn(0x7fef1c0e3b40 0x7fef1c0e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 -- 172.21.15.179:0/2145455057 >> v1:172.21.15.179:6789/0 conn(0x7fef1c0e0250 legacy=0x7fef1c038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 -- 172.21.15.179:0/2145455057 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0535d0 con 0x7fef1c006150 2024-05-08T22:45:12.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef22d9d700 1 --2- 172.21.15.179:0/2145455057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c006150 0x7fef1c0e4460 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fef2404f480 tx=0x7fef2404f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411a913bea00b713 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:12.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2159a700 1 -- 172.21.15.179:0/2145455057 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24067250 con 0x7fef1c006150 2024-05-08T22:45:12.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2159a700 1 -- 172.21.15.179:0/2145455057 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef240673d0 con 0x7fef1c006150 2024-05-08T22:45:12.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2159a700 1 -- 172.21.15.179:0/2145455057 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24067960 con 0x7fef1c006150 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- 172.21.15.179:0/2145455057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c006150 msgr2=0x7fef1c0e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 --2- 172.21.15.179:0/2145455057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c006150 0x7fef1c0e4460 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fef2404f480 tx=0x7fef2404f710 comp rx=0 tx=0).stop 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- 172.21.15.179:0/2145455057 shutdown_connections 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 --2- 172.21.15.179:0/2145455057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c006150 0x7fef1c0e4460 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 --2- 172.21.15.179:0/2145455057 >> v2:172.21.15.195:3300/0 conn(0x7fef1c0e3b40 0x7fef1c0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- 172.21.15.179:0/2145455057 >> 172.21.15.179:0/2145455057 conn(0x7fef1c054b70 msgr2=0x7fef1c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- 172.21.15.179:0/2145455057 shutdown_connections 2024-05-08T22:45:12.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.210+0000 7fef2359e700 1 -- 172.21.15.179:0/2145455057 wait complete. 2024-05-08T22:45:12.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.214+0000 7fef2359e700 1 Processor -- start 2024-05-08T22:45:12.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.214+0000 7fef2359e700 1 -- start start 2024-05-08T22:45:12.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c006150 0x7fef1c033570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c0dea10 0x7fef1c04ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0e25e0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c17d140 con 0x7fef1c006150 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c17c530 con 0x7fef1c0dea10 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36364/0 (socket says 172.21.15.179:36364) 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 -- 172.21.15.179:0/2717358271 learned_addr learned my addr 172.21.15.179:0/2717358271 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2259c700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c006150 0x7fef1c033570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef22d9d700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c0dea10 0x7fef1c04ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 -- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c0dea10 msgr2=0x7fef1c04ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c0dea10 0x7fef1c04ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 -- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c006150 msgr2=0x7fef1c033570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c006150 0x7fef1c033570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 -- 172.21.15.179:0/2717358271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c17c810 con 0x7fef1c0e3b40 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef21d9b700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c4d0 tx=0x7fef1400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14005d90 con 0x7fef1c0e3b40 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef140070d0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14007480 con 0x7fef1c0e3b40 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c17caf0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c0e2870 con 0x7fef1c0e3b40 2024-05-08T22:45:12.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.218+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c005440 con 0x7fef1c0e3b40 2024-05-08T22:45:12.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.222+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef140075e0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.222+0000 7fef12ffd700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 0x7fef04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:12.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.222+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fef140838e0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.222+0000 7fef2259c700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 0x7fef04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:12.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.222+0000 7fef2259c700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 0x7fef04068bf0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef18009950 tx=0x7fef18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:12.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.226+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fef14050a50 con 0x7fef1c0e3b40 2024-05-08T22:45:12.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.438+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef1c0e44d0 con 0x7fef1c0e3b40 2024-05-08T22:45:12.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.438+0000 7fef12ffd700 1 -- 172.21.15.179:0/2717358271 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4750 (secure 0 0 0) 0x7fef14054070 con 0x7fef1c0e3b40 2024-05-08T22:45:12.441 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:12.441 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:12.300548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 msgr2=0x7fef04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 0x7fef04068bf0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef18009950 tx=0x7fef18008040 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 msgr2=0x7fef1c0e20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c4d0 tx=0x7fef1400c890 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 shutdown_connections 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef04066730 0x7fef04068bf0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c0dea10 0x7fef1c04ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef1c0e3b40 0x7fef1c0e20a0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 --2- 172.21.15.179:0/2717358271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c006150 0x7fef1c033570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 >> 172.21.15.179:0/2717358271 conn(0x7fef1c054b70 msgr2=0x7fef1c039b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 shutdown_connections 2024-05-08T22:45:12.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:12.442+0000 7fef2359e700 1 -- 172.21.15.179:0/2717358271 wait complete. 2024-05-08T22:45:12.445 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 102 2024-05-08T22:45:12.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:45:12.300548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9820}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9820': {'gid': 9820, 'name': 'd', 'rank': 0, 'incarnation': 100, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.195:6835/357696906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 357696906}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 357696906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:45:12.860 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9820:up:rejoin 2024-05-08T22:45:13.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:13.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 Processor -- start 2024-05-08T22:45:13.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 -- start start 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe59015ef80 0x7fe590161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59006de70 0x7fe59006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe590165d50 con 0x7fe590164db0 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59006ad70 con 0x7fe59015ef80 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe595784700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590165260 con 0x7fe59006de70 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe59015ef80 0x7fe590161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe59015ef80 0x7fe590161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36366/0 (socket says 172.21.15.179:36366) 2024-05-08T22:45:13.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 -- 172.21.15.179:0/4236106506 learned_addr learned my addr 172.21.15.179:0/4236106506 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:13.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe594f83700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59006de70 0x7fe59006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.985 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 -- 172.21.15.179:0/4236106506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59006de70 msgr2=0x7fe59006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59006de70 0x7fe59006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 -- 172.21.15.179:0/4236106506 >> v1:172.21.15.195:6789/0 conn(0x7fe590164db0 legacy=0x7fe59006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 -- 172.21.15.179:0/4236106506 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59006e4b0 con 0x7fe59015ef80 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58ffff700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe59015ef80 0x7fe590161790 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe584009910 tx=0x7fe584009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7936a5c000c9c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58effd700 1 -- 172.21.15.179:0/4236106506 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584011830 con 0x7fe59015ef80 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58effd700 1 -- 172.21.15.179:0/4236106506 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584011990 con 0x7fe59015ef80 2024-05-08T22:45:13.986 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.982+0000 7fe58effd700 1 -- 172.21.15.179:0/4236106506 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584011d40 con 0x7fe59015ef80 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- 172.21.15.179:0/4236106506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe59015ef80 msgr2=0x7fe590161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe59015ef80 0x7fe590161790 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe584009910 tx=0x7fe584009c20 comp rx=0 tx=0).stop 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- 172.21.15.179:0/4236106506 shutdown_connections 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59006de70 0x7fe59006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- 172.21.15.179:0/4236106506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe59015ef80 0x7fe590161790 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- 172.21.15.179:0/4236106506 >> 172.21.15.179:0/4236106506 conn(0x7fe5900ce9d0 msgr2=0x7fe5900cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- 172.21.15.179:0/4236106506 shutdown_connections 2024-05-08T22:45:13.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- 172.21.15.179:0/4236106506 wait complete. 2024-05-08T22:45:13.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 Processor -- start 2024-05-08T22:45:13.989 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- start start 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59006de70 0x7fe590154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59015ef80 0x7fe590157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015c0b0 con 0x7fe590164db0 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015c390 con 0x7fe59006de70 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.986+0000 7fe595784700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015c670 con 0x7fe59015ef80 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58f7fe700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59015ef80 0x7fe590157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36372/0 (socket says 172.21.15.179:36372) 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59006de70 0x7fe590154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 -- 172.21.15.179:0/63069884 learned_addr learned my addr 172.21.15.179:0/63069884 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 -- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59015ef80 msgr2=0x7fe590157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59015ef80 0x7fe590157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 -- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59006de70 msgr2=0x7fe590154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59006de70 0x7fe590154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:13.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 -- 172.21.15.179:0/63069884 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584009570 con 0x7fe590164db0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe594f83700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb60 tx=0x7fe58000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580009740 con 0x7fe590164db0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580009e00 con 0x7fe590164db0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59015c950 con 0x7fe590164db0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000ca70 con 0x7fe590164db0 2024-05-08T22:45:13.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5901548a0 con 0x7fe590164db0 2024-05-08T22:45:13.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59015ce70 con 0x7fe590164db0 2024-05-08T22:45:13.994 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.990+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe58000cbd0 con 0x7fe590164db0 2024-05-08T22:45:13.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.994+0000 7fe58cff9700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:13.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.994+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe580084230 con 0x7fe590164db0 2024-05-08T22:45:13.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.994+0000 7fe58ffff700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:13.995 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.994+0000 7fe58ffff700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 0x7fe57c068bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe584009c60 tx=0x7fe584008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:13.998+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe59015ce70 con 0x7fe590164db0 2024-05-08T22:45:14.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.206+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe59015ce70 con 0x7fe590164db0 2024-05-08T22:45:14.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe58cff9700 1 -- 172.21.15.179:0/63069884 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4750 (secure 0 0 0) 0x7fe59015ce70 con 0x7fe590164db0 2024-05-08T22:45:14.211 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:14.211 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:13.308112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:14.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 msgr2=0x7fe57c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 0x7fe57c068bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe584009c60 tx=0x7fe584008030 comp rx=0 tx=0).stop 2024-05-08T22:45:14.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 msgr2=0x7fe59015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb60 tx=0x7fe58000ee70 comp rx=0 tx=0).stop 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 shutdown_connections 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe57c066730 0x7fe57c068bf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe590164db0 0x7fe59015d150 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59015ef80 0x7fe590157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 --2- 172.21.15.179:0/63069884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59006de70 0x7fe590154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 >> 172.21.15.179:0/63069884 conn(0x7fe5900ce9d0 msgr2=0x7fe59006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 shutdown_connections 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.210+0000 7fe595784700 1 -- 172.21.15.179:0/63069884 wait complete. 2024-05-08T22:45:14.214 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 103 2024-05-08T22:45:14.293 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-05-08T22:45:14.291+0000 7feaf679d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:45:14.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T22:44:27.216361+0000', 'modified': '2024-05-08T22:45:13.308112+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9820}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9820': {'gid': 9820, 'name': 'd', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6835/357696906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 357696906}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 357696906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:45:14.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T22:45:14.628 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T22:45:14.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 Processor -- start 2024-05-08T22:45:14.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 -- start start 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd8280cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8280b25d0 0x7fd8280b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82806b540 con 0x7fd82806e180 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82806ac50 con 0x7fd8280b25d0 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82806aef0 con 0x7fd82806dda0 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd81ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd82806e180 0x7fd82806e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:42100/0 (socket says 172.21.15.179:42100) 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd81ffff700 1 -- 172.21.15.179:0/1065582842 learned_addr learned my addr 172.21.15.179:0/1065582842 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 --2- 172.21.15.179:0/1065582842 >> v2:172.21.15.179:3300/0 conn(0x7fd8280b25d0 0x7fd8280b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd82659c700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd8280cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd825d9b700 1 -- 172.21.15.179:0/1065582842 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003841454 0 0) 0x7fd82806b540 con 0x7fd82806e180 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd825d9b700 1 -- 172.21.15.179:0/1065582842 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8140035d0 con 0x7fd82806e180 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 -- 172.21.15.179:0/1065582842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 msgr2=0x7fd8280cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd8280cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 -- 172.21.15.179:0/1065582842 >> v1:172.21.15.179:6789/0 conn(0x7fd82806e180 legacy=0x7fd82806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 -- 172.21.15.179:0/1065582842 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82806b1d0 con 0x7fd8280b25d0 2024-05-08T22:45:14.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.754+0000 7fd826d9d700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd8280b2a60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d90 tx=0x7fd82000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f428ae7567d3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd825d9b700 1 -- 172.21.15.179:0/1065582842 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000e070 con 0x7fd8280b25d0 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd825d9b700 1 -- 172.21.15.179:0/1065582842 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82000bb20 con 0x7fd8280b25d0 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd825d9b700 1 -- 172.21.15.179:0/1065582842 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000be90 con 0x7fd8280b25d0 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- 172.21.15.179:0/1065582842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 msgr2=0x7fd8280b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd8280b2a60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d90 tx=0x7fd82000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- 172.21.15.179:0/1065582842 shutdown_connections 2024-05-08T22:45:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd8280b2a60 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- 172.21.15.179:0/1065582842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd8280cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- 172.21.15.179:0/1065582842 >> 172.21.15.179:0/1065582842 conn(0x7fd8280ce960 msgr2=0x7fd8280ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- 172.21.15.179:0/1065582842 shutdown_connections 2024-05-08T22:45:14.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- 172.21.15.179:0/1065582842 wait complete. 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 Processor -- start 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- start start 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd82815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd82806e180 0x7fd828156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82815fdd0 con 0x7fd8280b25d0 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828143a80 con 0x7fd82806e180 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281570e0 con 0x7fd82806dda0 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd826d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd826d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36378/0 (socket says 172.21.15.179:36378) 2024-05-08T22:45:14.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd826d9d700 1 -- 172.21.15.179:0/2515224910 learned_addr learned my addr 172.21.15.179:0/2515224910 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd82659c700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd82815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.758+0000 7fd81ffff700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd82806e180 0x7fd828156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 -- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 msgr2=0x7fd82815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd82815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 -- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd82806e180 msgr2=0x7fd828156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd82806e180 0x7fd828156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 -- 172.21.15.179:0/2515224910 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820009a70 con 0x7fd8280b25d0 2024-05-08T22:45:14.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd826d9d700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd8280cf5e0 tx=0x7fd820004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd81effd700 1 -- 172.21.15.179:0/2515224910 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820004800 con 0x7fd8280b25d0 2024-05-08T22:45:14.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82815f300 con 0x7fd8280b25d0 2024-05-08T22:45:14.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd81effd700 1 -- 172.21.15.179:0/2515224910 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820020070 con 0x7fd8280b25d0 2024-05-08T22:45:14.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd81effd700 1 -- 172.21.15.179:0/2515224910 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200276c0 con 0x7fd8280b25d0 2024-05-08T22:45:14.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.762+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82815f810 con 0x7fd8280b25d0 2024-05-08T22:45:14.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd81effd700 1 -- 172.21.15.179:0/2515224910 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd820012430 con 0x7fd8280b25d0 2024-05-08T22:45:14.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd81effd700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 0x7fd808068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd81effd700 1 -- 172.21.15.179:0/2515224910 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd820022070 con 0x7fd8280b25d0 2024-05-08T22:45:14.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd82659c700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 0x7fd808068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd82659c700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 0x7fd808068bf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd818009bf0 tx=0x7fd818009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd82feaf740 1 Processor -- start 2024-05-08T22:45:14.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd82feaf740 1 -- start start 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.766+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2331ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x22f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2258d80 con 0x2328be0 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x2256510 con 0x2332420 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2251e00 con 0x2331b00 2024-05-08T22:45:14.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36380/0 (socket says 172.21.15.179:36380) 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 -- 172.21.15.179:0/1327573284 learned_addr learned my addr 172.21.15.179:0/1327573284 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82659c700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2331ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd826d9d700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x22f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 -- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 msgr2=0x2331ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2331ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 -- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 msgr2=0x22f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x22f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 -- 172.21.15.179:0/1327573284 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20c4960 con 0x2328be0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd826d9d700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x22f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd81ffff700 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd8100017f0 tx=0x7fd810000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c901fc721f2183 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd807fff700 1 -- 172.21.15.179:0/1327573284 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810005b70 con 0x2328be0 2024-05-08T22:45:14.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd807fff700 1 -- 172.21.15.179:0/1327573284 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810005cd0 con 0x2328be0 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd807fff700 1 -- 172.21.15.179:0/1327573284 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100072d0 con 0x2328be0 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- 172.21.15.179:0/1327573284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 msgr2=0x22f9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd8100017f0 tx=0x7fd810000bc0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- 172.21.15.179:0/1327573284 shutdown_connections 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x22f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22f9470 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 --2- 172.21.15.179:0/1327573284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2331ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- 172.21.15.179:0/1327573284 >> 172.21.15.179:0/1327573284 conn(0x20794d0 msgr2=0x2313250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- 172.21.15.179:0/1327573284 shutdown_connections 2024-05-08T22:45:14.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- 172.21.15.179:0/1327573284 wait complete. 2024-05-08T22:45:14.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 -- start start 2024-05-08T22:45:14.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.770+0000 7fd82feaf740 1 Processor -- start 2024-05-08T22:45:14.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 0x232a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x2329c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x220f730 con 0x2331b00 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x22155b0 con 0x2332420 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x20fdaa0 con 0x2328be0 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 10 client.0 _ll_register_callbacks cb 0x224b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82659c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 0x232a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd826d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x2329c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36384/0 (socket says 172.21.15.179:36384) 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 -- 172.21.15.179:0/941210151 learned_addr learned my addr 172.21.15.179:0/941210151 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 -- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 msgr2=0x232a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 0x232a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 -- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 msgr2=0x2329c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x2329c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 -- 172.21.15.179:0/941210151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8140035d0 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd826d9d700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x2329c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82659c700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 0x232a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd81ffff700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd810001e90 tx=0x7fd8100010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14ab0ba9809dc2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810005b70 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd810005e40 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20861e0 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd810007a80 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 10 client.10242 fetch_fsmap learned FSMap version 103 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x2218220 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20861e0 con 0x2331b00 2024-05-08T22:45:14.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 103) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd810012070 con 0x2331b00 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 10 client.10242 fetch_fsmap finished waiting for FSMap version 103 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 10 client.10242 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 20 client.10242 populate_metadata read hostname 'smithi179' 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2328070 0x2328450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 4 client.10242 mds_command: new command op to 9820 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:45:14.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 --> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x20d0460 con 0x2328070 2024-05-08T22:45:14.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.774+0000 7fd82659c700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2328070 0x2328450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82659c700 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2328070 0x2328450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79bcb4eec127b4b server_cookie=edd9f6a64a3c45be in_seq=0 out_seq=0 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd8057fa700 10 client.10242 ms_handle_connect on v2:172.21.15.195:6834/357696906 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd8057fa700 1 -- 172.21.15.179:0/941210151 <== mds.0 v2:172.21.15.195:6834/357696906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x20d0460 con 0x2328070 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd8057fa700 10 client.10242 handle_command_reply: tid=0 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 client.10242 shutdown 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 msgr2=0x2331250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd810001e90 tx=0x7fd8100010a0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 shutdown_connections 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2328070 0x2328450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2332420 0x2329c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2331b00 0x2331250 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd8057fa700 0 client.10242 ms_handle_reset on v2:172.21.15.195:6834/357696906 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 --2- 172.21.15.179:0/941210151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2328be0 0x232a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 >> 172.21.15.179:0/941210151 conn(0x20794d0 msgr2=0x23128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 shutdown_connections 2024-05-08T22:45:14.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 1 -- 172.21.15.179:0/941210151 wait complete. 2024-05-08T22:45:14.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.778+0000 7fd82feaf740 20 client.10242 trim_cache size 0 max 16384 2024-05-08T22:45:14.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 Processor -- start 2024-05-08T22:45:14.804 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 -- start start 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 0x2336820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2336d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x2280ec0 con 0x2328070 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x23313f0 con 0x2328be0 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2338350 con 0x2331b00 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36400/0 (socket says 172.21.15.179:36400) 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 -- 172.21.15.179:0/4007116329 learned_addr learned my addr 172.21.15.179:0/4007116329 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd81ffff700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 0x2336820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd826d9d700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2336d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 -- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 msgr2=0x2336d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2336d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 -- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 msgr2=0x2336820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 0x2336820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 -- 172.21.15.179:0/4007116329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21fe450 con 0x2328070 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd81ffff700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 0x2336820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd82659c700 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fd81802acc0 tx=0x7fd818018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146005e386c574d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd81cff9700 1 -- 172.21.15.179:0/4007116329 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81801c040 con 0x2328070 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd81cff9700 1 -- 172.21.15.179:0/4007116329 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81802a1e0 con 0x2328070 2024-05-08T22:45:14.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.802+0000 7fd81cff9700 1 -- 172.21.15.179:0/4007116329 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81802a590 con 0x2328070 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/4007116329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 msgr2=0x23138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fd81802acc0 tx=0x7fd818018000 comp rx=0 tx=0).stop 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/4007116329 shutdown_connections 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328be0 0x2336820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2336d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- 172.21.15.179:0/4007116329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328070 0x23138c0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/4007116329 >> 172.21.15.179:0/4007116329 conn(0x20794d0 msgr2=0x22fafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/4007116329 shutdown_connections 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/4007116329 wait complete. 2024-05-08T22:45:14.811 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- start start 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 Processor -- start 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 0x22ea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2340540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x221b860 con 0x2328be0 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f0470 con 0x2328070 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x2327dd0 con 0x2331b00 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 10 client.0 _ll_register_callbacks cb 0x7fd810001390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36414/0 (socket says 172.21.15.179:36414) 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 -- 172.21.15.179:0/3249022372 learned_addr learned my addr 172.21.15.179:0/3249022372 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 -- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 msgr2=0x2340540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82659c700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 0x22ea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd826d9d700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2340540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.812 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2340540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 -- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 msgr2=0x22ea7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 0x22ea7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 -- 172.21.15.179:0/3249022372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21f2720 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd826d9d700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2340540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82659c700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 0x22ea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd81ffff700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd81000ff50 tx=0x7fd810009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2134617f673a0da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810035030 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd810005cd0 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mon.0 v2:172.21.15.179:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd810036070 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd810005b70 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 10 client.10248 fetch_fsmap learned FSMap version 103 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x2189c70 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mon.0 v2:172.21.15.179:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810005cd0 con 0x2328be0 2024-05-08T22:45:14.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.806+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mon.0 v2:172.21.15.179:3300/0 5 ==== fsmap(e 103) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd810004a90 con 0x2328be0 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 10 client.10248 fetch_fsmap finished waiting for FSMap version 103 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 10 client.10248 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 20 client.10248 populate_metadata read hostname 'smithi179' 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2332420 0x2340f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 4 client.10248 mds_command: new command op to 9820 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 --> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] -- command(tid 0: {"prefix": "status"}) v1 -- 0x20f51e0 con 0x2332420 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82659c700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2332420 0x2340f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82659c700 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2332420 0x2340f50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebb6f76ac3a39b4 server_cookie=982a6f983361b336 in_seq=0 out_seq=0 2024-05-08T22:45:14.814 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd8057fa700 10 client.10248 ms_handle_connect on v2:172.21.15.195:6834/357696906 2024-05-08T22:45:14.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd8057fa700 1 -- 172.21.15.179:0/3249022372 <== mds.0 v2:172.21.15.195:6834/357696906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x20f51e0 con 0x2332420 2024-05-08T22:45:14.815 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd8057fa700 10 client.10248 handle_command_reply: tid=0 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 client.10248 shutdown 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 msgr2=0x22e9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd81000ff50 tx=0x7fd810009040 comp rx=0 tx=0).stop 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 shutdown_connections 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:6834/357696906,v1:172.21.15.195:6835/357696906] conn(0x2332420 0x2340f50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x2331b00 0x2340540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x2328be0 0x22e9b20 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 --2- 172.21.15.179:0/3249022372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x2328070 0x22ea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 >> 172.21.15.179:0/3249022372 conn(0x20794d0 msgr2=0x23370b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd8057fa700 0 client.10248 ms_handle_reset on v2:172.21.15.195:6834/357696906 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 shutdown_connections 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 1 -- 172.21.15.179:0/3249022372 wait complete. 2024-05-08T22:45:14.816 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82feaf740 20 client.10248 trim_cache size 0 max 16384 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "cluster_fsid": "88a906d6-7c22-496e-b9c7-b34c51a47fa8", 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "whoami": 0, 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 9820, 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "want_state": "up:active", 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "state": "up:active", 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs", 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "rank_uptime": 4.5137864299999997, 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "mdsmap_epoch": 103, 2024-05-08T22:45:14.817 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch": 129, 2024-05-08T22:45:14.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_epoch_barrier": 129, 2024-05-08T22:45:14.818 INFO:teuthology.orchestra.run.smithi179.stdout: "uptime": 6.4905072610000003 2024-05-08T22:45:14.818 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 msgr2=0x7fd808068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 0x7fd808068bf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd818009bf0 tx=0x7fd818009240 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 msgr2=0x7fd828156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd8280cf5e0 tx=0x7fd820004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 shutdown_connections 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd808066730 0x7fd808068bf0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd82806e180 0x7fd828156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8280b25d0 0x7fd828156ba0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 --2- 172.21.15.179:0/2515224910 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd82806dda0 0x7fd82815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 >> 172.21.15.179:0/2515224910 conn(0x7fd8280ce960 msgr2=0x7fd8280cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 shutdown_connections 2024-05-08T22:45:14.819 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.810+0000 7fd82759e700 1 -- 172.21.15.179:0/2515224910 wait complete. 2024-05-08T22:45:14.826 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T22:45:14.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.938+0000 7ff12aff3700 1 Processor -- start 2024-05-08T22:45:15.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.938+0000 7ff12aff3700 1 -- start start 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 0x7ff12406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1240b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff12415d970 0x7ff12415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12415e340 con 0x7ff12415d970 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12406ce80 con 0x7ff124143640 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff124143a20 con 0x7ff1240cea40 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff12415d970 0x7ff12415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.046 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff12415d970 0x7ff12415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36430/0 (socket says 172.21.15.179:36430) 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 -- 172.21.15.179:0/4005185485 learned_addr learned my addr 172.21.15.179:0/4005185485 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff1297f0700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1240b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff129ff1700 1 --2- 172.21.15.179:0/4005185485 >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 0x7ff12406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 -- 172.21.15.179:0/4005185485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 msgr2=0x7ff1240b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1240b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 -- 172.21.15.179:0/4005185485 >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 msgr2=0x7ff12406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 --2- 172.21.15.179:0/4005185485 >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 0x7ff12406eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 -- 172.21.15.179:0/4005185485 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1240cf220 con 0x7ff12415d970 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff129ff1700 1 --2- 172.21.15.179:0/4005185485 >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 0x7ff12406eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12a7f2700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff12415d970 0x7ff12415de00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff124071a60 tx=0x7ff11c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72dc31f3e17033a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff128fef700 1 -- 172.21.15.179:0/4005185485 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00e070 con 0x7ff12415d970 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff128fef700 1 -- 172.21.15.179:0/4005185485 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c00b910 con 0x7ff12415d970 2024-05-08T22:45:15.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff128fef700 1 -- 172.21.15.179:0/4005185485 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00bcc0 con 0x7ff12415d970 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- 172.21.15.179:0/4005185485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff12415d970 msgr2=0x7ff12415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff12415d970 0x7ff12415de00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff124071a60 tx=0x7ff11c00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- 172.21.15.179:0/4005185485 shutdown_connections 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff12415d970 0x7ff12415de00 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- 172.21.15.179:0/4005185485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1240b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 --2- 172.21.15.179:0/4005185485 >> v2:172.21.15.195:3300/0 conn(0x7ff124143640 0x7ff12406eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- 172.21.15.179:0/4005185485 >> 172.21.15.179:0/4005185485 conn(0x7ff1240cfae0 msgr2=0x7ff1240cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- 172.21.15.179:0/4005185485 shutdown_connections 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.942+0000 7ff12aff3700 1 -- 172.21.15.179:0/4005185485 wait complete. 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.946+0000 7ff12aff3700 1 Processor -- start 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.946+0000 7ff12aff3700 1 -- start start 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.946+0000 7ff12aff3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1241f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff12415d970 0x7ff1241eff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12413c570 con 0x7ff124143640 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12413ba20 con 0x7ff12415d970 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12413bcd0 con 0x7ff1240cea40 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36436/0 (socket says 172.21.15.179:36436) 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 -- 172.21.15.179:0/3816146213 learned_addr learned my addr 172.21.15.179:0/3816146213 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12a7f2700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff12415d970 0x7ff1241eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff129ff1700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1241f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 -- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 msgr2=0x7ff1241f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:15.049 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1241f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 -- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff12415d970 msgr2=0x7ff1241eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff12415d970 0x7ff1241eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 -- 172.21.15.179:0/3816146213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c009b30 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff1297f0700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e7c0 tx=0x7ff11400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114013680 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1140137e0 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12413bfb0 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114013b90 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12413cac0 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.950+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12413c920 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.954+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff114013cf0 con 0x7ff124143640 2024-05-08T22:45:15.050 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.954+0000 7ff11a7fc700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 0x7ff108068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.954+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7ff114084860 con 0x7ff124143640 2024-05-08T22:45:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.954+0000 7ff129ff1700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 0x7ff108068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.954+0000 7ff129ff1700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 0x7ff108068bf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff1200044b0 tx=0x7ff120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:15.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:14.958+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff1140519c0 con 0x7ff124143640 2024-05-08T22:45:15.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.162+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff12413c920 con 0x7ff124143640 2024-05-08T22:45:15.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.162+0000 7ff11a7fc700 1 -- 172.21.15.179:0/3816146213 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff12413c920 con 0x7ff124143640 2024-05-08T22:45:15.167 INFO:teuthology.orchestra.run.smithi179.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 msgr2=0x7ff108068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 0x7ff108068bf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff1200044b0 tx=0x7ff120008040 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 msgr2=0x7ff12413b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e7c0 tx=0x7ff11400ead0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 shutdown_connections 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff108066730 0x7ff108068bf0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff12415d970 0x7ff1241eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff124143640 0x7ff12413b4e0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 --2- 172.21.15.179:0/3816146213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1240cea40 0x7ff1241f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 >> 172.21.15.179:0/3816146213 conn(0x7ff1240cfae0 msgr2=0x7ff124140f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:15.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.166+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 shutdown_connections 2024-05-08T22:45:15.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:15.170+0000 7ff12aff3700 1 -- 172.21.15.179:0/3816146213 wait complete. 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:45:15.186 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:45:15.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:45:15.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:45:15.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.187 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-08T22:45:15.194 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:45:15.194 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:45:15.194 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:45:15.195 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41856sec preferred_lft 41856sec 2024-05-08T22:45:15.196 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:21:a1:8d:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T22:45:15.197 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c08d:99ff:feed:936b/64 scope link 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 06:0e:f5:30:87:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::40e:f5ff:fe30:87c9/64 scope link 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T22:45:15.198 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0a:77:80:e4:c7:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:45:15.199 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::877:80ff:fee4:c744/64 scope link 2024-05-08T22:45:15.199 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:45:15.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.199 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:45:15.213 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:45:15.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.215 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-08T22:45:15.267 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-08T22:45:15.267 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:45:15.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.268 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-08T22:45:15.268 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-08T22:45:15.269 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:45:15.269 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:45:15.269 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:45:15.269 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-08T22:45:15.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.313 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:45:15.320 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:45:15.320 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:45:15.320 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:45:15.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.321 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-08T22:45:15.380 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T22:45:15.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:45:15.380 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:15.388 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:45:15.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:45:15.389 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:15.396 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:45:15.411 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:45:15.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:45:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:45:15.426 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:45:15.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.427 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:45:15.476 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T22:45:15.477 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T22:45:15.479 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:45:15.494 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:45:15.494 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:45:15.505 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:45:15.506 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:45:15.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:15.507 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:45:15.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:2024-05-08T22:45:15.574+0000 7f6e1f28f640 -1 init, newargv = 0x56075033e390 newargc=15 2024-05-08T22:45:15.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[54316]: starting ceph client 2024-05-08T22:45:15.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[54316]: starting fuse 2024-05-08T22:45:16.658 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-05-08T22:45:16.677 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T22:45:16.677 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T22:45:16.687 INFO:teuthology.orchestra.run.smithi179.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T22:45:16.688 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T22:45:16.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:16.689 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-05-08T22:45:16.738 INFO:teuthology.orchestra.run.smithi179.stdout:52 2024-05-08T22:45:16.739 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T22:45:16.739 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T22:45:16.740 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:16.741 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.0")) 2024-05-08T22:45:16.741 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:16.822 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.0.54316.asok 2024-05-08T22:45:17.009 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54316.asok 2024-05-08T22:45:17.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:17.009 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54316.asok status 2024-05-08T22:45:17.109 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "0", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "54316", 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:45:17.110 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 10260, 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 10260 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:45:17.111 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "172.21.15.179:0", 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2076587953 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "172.21.15.179:0", 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2076587953 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.10260 172.21.15.179:0/2076587953", 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "172.21.15.179:0/2076587953", 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 103, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 129, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-05-08T22:45:17.112 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-05-08T22:45:17.113 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-05-08T22:45:17.113 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:17.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:17.120 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-05-08T22:45:17.120 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:17.125 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-05-08T22:45:17.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:17.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:17.126 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:17.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:17.142 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./ 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/charlie 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/alpha 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/flushed/bravo 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/unflushed 2024-05-08T22:45:17.177 INFO:teuthology.orchestra.run.smithi179.stdout:./parent/unflushed/jfile 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T22:45:17.179 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:17.289 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T22:45:17.480 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:17.543 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627776 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T22:45:17.735 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:17.804 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627777 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T22:45:17.995 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:18.067 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628280 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T22:45:18.255 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:18.368 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627778 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-08T22:45:18.558 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:18.629 INFO:teuthology.orchestra.run.smithi179.stdout:1099511628279 2024-05-08T22:45:18.817 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:18.817 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:18.817 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:18.817 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:18.818 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T22:45:18.818 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:18.884 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627779 2024-05-08T22:45:19.075 DEBUG:teuthology.orchestra.run.smithi179:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T22:45:19.075 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-05-08T22:45:19.075 DEBUG:teuthology.orchestra.run.smithi179:> import stat 2024-05-08T22:45:19.076 DEBUG:teuthology.orchestra.run.smithi179:> 2024-05-08T22:45:19.076 DEBUG:teuthology.orchestra.run.smithi179:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T22:45:19.076 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-05-08T22:45:19.187 INFO:teuthology.orchestra.run.smithi179.stdout:1099511627780 2024-05-08T22:45:19.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T22:45:19.376 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:45:19.397 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T22:45:19.419 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T22:45:19.419 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:45:19.419 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:19.429 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:45:19.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 Processor -- start 2024-05-08T22:45:19.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 -- start start 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f6c0eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6c039c10 con 0x7f9f6c0343a0 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6c0534d0 con 0x7f9f6c052fd0 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f78923700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0549e0 con 0x7f9f6c054570 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f72d9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9f6c0343a0 0x7f9f6c038df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57538/0 (socket says 172.21.15.179:57538) 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f72d9d700 1 -- 172.21.15.179:0/2033053743 learned_addr learned my addr 172.21.15.179:0/2033053743 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f7359e700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f6c0eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643305596 0 0) 0x7f9f6c039c10 con 0x7f9f6c0343a0 2024-05-08T22:45:19.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.590+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f5c0035d0 con 0x7f9f6c0343a0 2024-05-08T22:45:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028084039 0 0) 0x7f9f5c0035d0 con 0x7f9f6c0343a0 2024-05-08T22:45:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 msgr2=0x7f9f6c0eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f6c0eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 >> v1:172.21.15.195:6789/0 conn(0x7f9f6c052fd0 legacy=0x7f9f6c039500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0537b0 con 0x7f9f6c0343a0 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9f64001670 con 0x7f9f6c0343a0 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 >> v1:172.21.15.179:6789/0 conn(0x7f9f6c0343a0 legacy=0x7f9f6c038df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f5c003a50 0x7f9f5c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c004870 0x7f9f5c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f5c0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0549e0 con 0x7f9f5c003a50 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c00bbc0 con 0x7f9f5c004870 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c00bd80 con 0x7f9f6c054570 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f5c003a50 0x7f9f5c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f7359e700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f5c0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f7259c700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c004870 0x7f9f5c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 -- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 msgr2=0x7f9f5c0052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:19.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c054570 0x7f9f5c0052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 -- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c004870 msgr2=0x7f9f5c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c004870 0x7f9f5c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 -- 172.21.15.179:0/2033053743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0035d0 con 0x7f9f5c003a50 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f5c003a50 0x7f9f5c003e30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9f64007f60 tx=0x7f9f5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd98a6bbeb960a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400ce00 con 0x7f9f5c003a50 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f64005980 con 0x7f9f5c003a50 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f71d9b700 1 -- 172.21.15.179:0/2033053743 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64005d30 con 0x7f9f5c003a50 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 -- 172.21.15.179:0/2033053743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f5c003a50 msgr2=0x7f9f5c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 --2- 172.21.15.179:0/2033053743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f5c003a50 0x7f9f5c003e30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9f64007f60 tx=0x7f9f5c0058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f7359e700 1 -- 172.21.15.179:0/2033053743 reap_dead start 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 -- 172.21.15.179:0/2033053743 shutdown_connections 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 -- 172.21.15.179:0/2033053743 >> 172.21.15.179:0/2033053743 conn(0x7f9f6c055400 msgr2=0x7f9f6c055810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 -- 172.21.15.179:0/2033053743 shutdown_connections 2024-05-08T22:45:19.599 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.594+0000 7f9f78923700 1 -- 172.21.15.179:0/2033053743 wait complete. 2024-05-08T22:45:19.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 Processor -- start 2024-05-08T22:45:19.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 -- start start 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c003a50 0x7f9f6c041570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c0e08a0 0x7f9f6c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0e2350 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0e12d0 con 0x7f9f5c003a50 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f78923700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c0e15b0 con 0x7f9f6c0e08a0 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34790/0 (socket says 172.21.15.179:34790) 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7259c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c0e08a0 0x7f9f6c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 -- 172.21.15.179:0/17443238 learned_addr learned my addr 172.21.15.179:0/17443238 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c003a50 0x7f9f6c041570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 -- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c0e08a0 msgr2=0x7f9f6c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c0e08a0 0x7f9f6c0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 -- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c003a50 msgr2=0x7f9f6c041570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c003a50 0x7f9f6c041570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.598+0000 7f9f7359e700 1 -- 172.21.15.179:0/17443238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0e1890 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f7359e700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9f74068960 tx=0x7f9f74067530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7408e040 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f740669e0 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6c0e1b70 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74066d50 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c041ab0 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6c0341c0 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f74066eb0 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 0x7f9f60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.602+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9f740fdc60 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.606+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 0x7f9f60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.606+0000 7f9f72d9d700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 0x7f9f60068bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000c00 tx=0x7f9f6401f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.610+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9f740cade0 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.782+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f6c005890 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f5b7fe700 1 -- 172.21.15.179:0/17443238 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16627 (secure 0 0 0) 0x7f9f74099070 con 0x7f9f6c0e0d00 2024-05-08T22:45:19.788 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:19.788 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":129,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:45:08.266211+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6834/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:45:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 msgr2=0x7f9f60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 0x7f9f60068bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000c00 tx=0x7f9f6401f040 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 msgr2=0x7f9f6c0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9f74068960 tx=0x7f9f74067530 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 shutdown_connections 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f60066730 0x7f9f60068bf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f6c0e0d00 0x7f9f6c0c71c0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.786+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f6c0e08a0 0x7f9f6c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.790+0000 7f9f78923700 1 --2- 172.21.15.179:0/17443238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f5c003a50 0x7f9f6c041570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.790+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 >> 172.21.15.179:0/17443238 conn(0x7f9f6c055400 msgr2=0x7f9f6c0363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.790+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 shutdown_connections 2024-05-08T22:45:19.791 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.790+0000 7f9f78923700 1 -- 172.21.15.179:0/17443238 wait complete. 2024-05-08T22:45:19.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:45:19.804 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:45:19.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 Processor -- start 2024-05-08T22:45:19.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 -- start start 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdbb8158b00 0x7fdbb80ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbb80cce20 0x7fdbb806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb806b4e0 con 0x7fdbb80ca890 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb80cad30 con 0x7fdbb80cce20 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbbd243700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb80cb010 con 0x7fdbb8158b00 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbb80cce20 0x7fdbb806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbb80cce20 0x7fdbb806bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34792/0 (socket says 172.21.15.179:34792) 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 -- 172.21.15.179:0/354607947 learned_addr learned my addr 172.21.15.179:0/354607947 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb6d9d700 1 --2- 172.21.15.179:0/354607947 >> v2:172.21.15.195:3300/0 conn(0x7fdbb8158b00 0x7fdbb80ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 -- 172.21.15.179:0/354607947 >> v2:172.21.15.195:3300/0 conn(0x7fdbb8158b00 msgr2=0x7fdbb80ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:19.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 --2- 172.21.15.179:0/354607947 >> v2:172.21.15.195:3300/0 conn(0x7fdbb8158b00 0x7fdbb80ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 -- 172.21.15.179:0/354607947 >> v1:172.21.15.195:6789/0 conn(0x7fdbb80ca890 legacy=0x7fdbb80cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 -- 172.21.15.179:0/354607947 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb806b750 con 0x7fdbb80cce20 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb759e700 1 --2- 172.21.15.179:0/354607947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb806bc70 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009e50 tx=0x7fdbb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fc2fc93d5b5aee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb5d9b700 1 -- 172.21.15.179:0/354607947 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000e070 con 0x7fdbb80cce20 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb5d9b700 1 -- 172.21.15.179:0/354607947 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb000b980 con 0x7fdbb80cce20 2024-05-08T22:45:19.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.926+0000 7fdbb5d9b700 1 -- 172.21.15.179:0/354607947 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000bd30 con 0x7fdbb80cce20 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- 172.21.15.179:0/354607947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 msgr2=0x7fdbb806bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- 172.21.15.179:0/354607947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb806bc70 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009e50 tx=0x7fdbb000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- 172.21.15.179:0/354607947 shutdown_connections 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- 172.21.15.179:0/354607947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb806bc70 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- 172.21.15.179:0/354607947 >> v2:172.21.15.195:3300/0 conn(0x7fdbb8158b00 0x7fdbb80ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- 172.21.15.179:0/354607947 >> 172.21.15.179:0/354607947 conn(0x7fdbb80b18d0 msgr2=0x7fdbb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- 172.21.15.179:0/354607947 shutdown_connections 2024-05-08T22:45:19.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- 172.21.15.179:0/354607947 wait complete. 2024-05-08T22:45:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 Processor -- start 2024-05-08T22:45:19.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- start start 2024-05-08T22:45:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbb80ca890 0x7fdbb80c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbb8158b00 0x7fdbb815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80c9920 con 0x7fdbb80cce20 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8156a60 con 0x7fdbb80ca890 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbbd243700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80c9b00 con 0x7fdbb8158b00 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.930+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34796/0 (socket says 172.21.15.179:34796) 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 -- 172.21.15.179:0/623304866 learned_addr learned my addr 172.21.15.179:0/623304866 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb6d9d700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbb80ca890 0x7fdbb80c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb759e700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbb8158b00 0x7fdbb815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 -- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbb8158b00 msgr2=0x7fdbb815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbb8158b00 0x7fdbb815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 -- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbb80ca890 msgr2=0x7fdbb80c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbb80ca890 0x7fdbb80c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 -- 172.21.15.179:0/623304866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0009b30 con 0x7fdbb80cce20 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbb659c700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c500 tx=0x7fdba000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00052d0 con 0x7fdbb80cce20 2024-05-08T22:45:19.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb8156130 con 0x7fdbb80cce20 2024-05-08T22:45:19.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0005430 con 0x7fdbb80cce20 2024-05-08T22:45:19.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00055a0 con 0x7fdbb80cce20 2024-05-08T22:45:19.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb80c7eb0 con 0x7fdbb80cce20 2024-05-08T22:45:19.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb8156320 con 0x7fdbb80cce20 2024-05-08T22:45:19.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.934+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdba00038e0 con 0x7fdbb80cce20 2024-05-08T22:45:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.938+0000 7fdbaf7fe700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 0x7fdb98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.938+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fdba0083680 con 0x7fdbb80cce20 2024-05-08T22:45:19.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.938+0000 7fdbb6d9d700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 0x7fdb98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:19.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.938+0000 7fdbb6d9d700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 0x7fdb98068bf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009a00 tx=0x7fdba8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:19.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:19.942+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdba0050810 con 0x7fdbb80cce20 2024-05-08T22:45:20.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.106+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdbb8156520 con 0x7fdbb80cce20 2024-05-08T22:45:20.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.106+0000 7fdbaf7fe700 1 -- 172.21.15.179:0/623304866 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdba0053e30 con 0x7fdbb80cce20 2024-05-08T22:45:20.109 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:45:20.111 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 msgr2=0x7fdb98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 0x7fdb98068bf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009a00 tx=0x7fdba8008040 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 msgr2=0x7fdbb80c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c500 tx=0x7fdba000c810 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 shutdown_connections 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdb98066730 0x7fdb98068bf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbb8158b00 0x7fdbb815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbb80cce20 0x7fdbb80c7970 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 --2- 172.21.15.179:0/623304866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbb80ca890 0x7fdbb80c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 >> 172.21.15.179:0/623304866 conn(0x7fdbb80b18d0 msgr2=0x7fdbb81522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 shutdown_connections 2024-05-08T22:45:20.112 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.110+0000 7fdbbd243700 1 -- 172.21.15.179:0/623304866 wait complete. 2024-05-08T22:45:20.124 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:45:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 Processor -- start 2024-05-08T22:45:20.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 -- start start 2024-05-08T22:45:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98ac15a160 0x7f98ac06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac06d7e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0cf8c0 con 0x7f98ac0cb2a0 2024-05-08T22:45:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98b1164700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06d220 con 0x7f98ac15a160 2024-05-08T22:45:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98ab59e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98ac0b3880 0x7f98ac0cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57552/0 (socket says 172.21.15.179:57552) 2024-05-08T22:45:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98ab59e700 1 -- 172.21.15.179:0/4178776232 learned_addr learned my addr 172.21.15.179:0/4178776232 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:20.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98aad9d700 1 --2- 172.21.15.179:0/4178776232 >> v2:172.21.15.195:3300/0 conn(0x7f98ac15a160 0x7f98ac06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288512103 0 0) 0x7f98ac06d7e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98940035d0 con 0x7f98ac0b3880 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095542705 0 0) 0x7f98ac0cf8c0 con 0x7f98ac0cb2a0 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.238+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ac06d7e0 con 0x7f98ac0cb2a0 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98aad9d700 1 -- 172.21.15.179:0/4178776232 >> v1:172.21.15.195:6789/0 conn(0x7f98ac0cb2a0 legacy=0x7f98ac0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1927613439 0 0) 0x7f98940035d0 con 0x7f98ac0b3880 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98aad9d700 1 -- 172.21.15.179:0/4178776232 >> v1:172.21.15.179:6789/0 conn(0x7f98ac0b3880 legacy=0x7f98ac0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98aad9d700 1 -- 172.21.15.179:0/4178776232 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac06d500 con 0x7f98ac15a160 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98aad9d700 1 --2- 172.21.15.179:0/4178776232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac06da80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009850 tx=0x7f98a0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b12324662fcb7a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4001670 con 0x7f98ac15a160 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0004030 con 0x7f98ac15a160 2024-05-08T22:45:20.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98a9d9b700 1 -- 172.21.15.179:0/4178776232 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00043c0 con 0x7f98ac15a160 2024-05-08T22:45:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 -- 172.21.15.179:0/4178776232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 msgr2=0x7f98ac06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 --2- 172.21.15.179:0/4178776232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac06da80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009850 tx=0x7f98a0009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 -- 172.21.15.179:0/4178776232 shutdown_connections 2024-05-08T22:45:20.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 --2- 172.21.15.179:0/4178776232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac06da80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 -- 172.21.15.179:0/4178776232 >> 172.21.15.179:0/4178776232 conn(0x7f98ac0ceca0 msgr2=0x7f98ac0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 -- 172.21.15.179:0/4178776232 shutdown_connections 2024-05-08T22:45:20.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.242+0000 7f98b1164700 1 -- 172.21.15.179:0/4178776232 wait complete. 2024-05-08T22:45:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 Processor -- start 2024-05-08T22:45:20.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 -- start start 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0cb2a0 0x7f98ac15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14e4d0 con 0x7f98ac0b3880 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15b7d0 con 0x7f98ac15a160 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98b1164700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15bab0 con 0x7f98ac0cb2a0 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98aad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98aad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34804/0 (socket says 172.21.15.179:34804) 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.246+0000 7f98aad9d700 1 -- 172.21.15.179:0/298254627 learned_addr learned my addr 172.21.15.179:0/298254627 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98ab59e700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aa59c700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0cb2a0 0x7f98ac15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 -- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0cb2a0 msgr2=0x7f98ac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0cb2a0 0x7f98ac15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 -- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 msgr2=0x7f98ac14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 -- 172.21.15.179:0/298254627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00094b0 con 0x7f98ac0b3880 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98aad9d700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f98ac067640 tx=0x7f98a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0003c30 con 0x7f98ac0b3880 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a001f070 con 0x7f98ac0b3880 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac15bd90 con 0x7f98ac0b3880 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0011630 con 0x7f98ac0b3880 2024-05-08T22:45:20.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac154650 con 0x7f98ac0b3880 2024-05-08T22:45:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.250+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac1547e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.254+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98a0021030 con 0x7f98ac0b3880 2024-05-08T22:45:20.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.254+0000 7f98937fe700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 0x7f9898068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.254+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f98a00853d0 con 0x7f98ac0b3880 2024-05-08T22:45:20.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.254+0000 7f98aa59c700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 0x7f9898068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.254+0000 7f98aa59c700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 0x7f9898068c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f98ac14e2c0 tx=0x7f989c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.262+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98ac1547e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.430+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98ac1547e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98937fe700 1 -- 172.21.15.179:0/298254627 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98ac1547e0 con 0x7f98ac0b3880 2024-05-08T22:45:20.435 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 msgr2=0x7f9898068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 0x7f9898068c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f98ac14e2c0 tx=0x7f989c009040 comp rx=0 tx=0).stop 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 msgr2=0x7f98ac154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f98ac067640 tx=0x7f98a0004770 comp rx=0 tx=0).stop 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 shutdown_connections 2024-05-08T22:45:20.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9898066780 0x7f9898068c40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac15a160 0x7f98ac14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0cb2a0 0x7f98ac15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 --2- 172.21.15.179:0/298254627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac0b3880 0x7f98ac154110 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 >> 172.21.15.179:0/298254627 conn(0x7f98ac0ceca0 msgr2=0x7f98ac0addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 shutdown_connections 2024-05-08T22:45:20.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.434+0000 7f98b1164700 1 -- 172.21.15.179:0/298254627 wait complete. 2024-05-08T22:45:20.449 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:45:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.566+0000 7fc685d39700 1 Processor -- start 2024-05-08T22:45:20.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.566+0000 7fc685d39700 1 -- start start 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc68015a300 0x7fc68015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc6800b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6800b4650 con 0x7fc6801544e0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6800cd6e0 con 0x7fc68015a300 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68006db50 con 0x7fc68015acc0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc684d37700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc6801544e0 0x7fc68006ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57564/0 (socket says 172.21.15.179:57564) 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc684d37700 1 -- 172.21.15.179:0/3256041976 learned_addr learned my addr 172.21.15.179:0/3256041976 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685538700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc6800b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 --2- 172.21.15.179:0/3256041976 >> v2:172.21.15.195:3300/0 conn(0x7fc68015a300 0x7fc68015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc6777fe700 1 -- 172.21.15.179:0/3256041976 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532260254 0 0) 0x7fc6800b4650 con 0x7fc6801544e0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc6777fe700 1 -- 172.21.15.179:0/3256041976 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6680035d0 con 0x7fc6801544e0 2024-05-08T22:45:20.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 -- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 msgr2=0x7fc6800b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:20.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc6800b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 -- 172.21.15.179:0/3256041976 >> v1:172.21.15.179:6789/0 conn(0x7fc6801544e0 legacy=0x7fc68006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 -- 172.21.15.179:0/3256041976 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800b4320 con 0x7fc68015a300 2024-05-08T22:45:20.573 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc677fff700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68015a780 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc670009850 tx=0x7fc670009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70f77f754df1d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc6777fe700 1 -- 172.21.15.179:0/3256041976 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670011830 con 0x7fc68015a300 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc6777fe700 1 -- 172.21.15.179:0/3256041976 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc670011990 con 0x7fc68015a300 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc6777fe700 1 -- 172.21.15.179:0/3256041976 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670011d40 con 0x7fc68015a300 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 msgr2=0x7fc68015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68015a780 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc670009850 tx=0x7fc670009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- 172.21.15.179:0/3256041976 shutdown_connections 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc6800b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 --2- 172.21.15.179:0/3256041976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68015a780 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- 172.21.15.179:0/3256041976 >> 172.21.15.179:0/3256041976 conn(0x7fc6800b18d0 msgr2=0x7fc6800b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- 172.21.15.179:0/3256041976 shutdown_connections 2024-05-08T22:45:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.570+0000 7fc685d39700 1 -- 172.21.15.179:0/3256041976 wait complete. 2024-05-08T22:45:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 Processor -- start 2024-05-08T22:45:20.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 -- start start 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68014c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc680137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68014c6b0 con 0x7fc6801544e0 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68014eec0 con 0x7fc68015a300 2024-05-08T22:45:20.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685d39700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc680137da0 con 0x7fc68015acc0 2024-05-08T22:45:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34810/0 (socket says 172.21.15.179:34810) 2024-05-08T22:45:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 -- 172.21.15.179:0/1814241437 learned_addr learned my addr 172.21.15.179:0/1814241437 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc685538700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc680137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc677fff700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 -- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 msgr2=0x7fc680137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc680137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 -- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 msgr2=0x7fc68014c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68014c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 -- 172.21.15.179:0/1814241437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700094b0 con 0x7fc6801544e0 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc684d37700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c700 tx=0x7fc67c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.574+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c005d90 con 0x7fc6801544e0 2024-05-08T22:45:20.579 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c0070d0 con 0x7fc6801544e0 2024-05-08T22:45:20.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68014e590 con 0x7fc6801544e0 2024-05-08T22:45:20.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c007480 con 0x7fc6801544e0 2024-05-08T22:45:20.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68014eb50 con 0x7fc6801544e0 2024-05-08T22:45:20.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68004e700 con 0x7fc6801544e0 2024-05-08T22:45:20.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc67c0075e0 con 0x7fc6801544e0 2024-05-08T22:45:20.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.578+0000 7fc6757fa700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 0x7fc66c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.582+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fc67c083740 con 0x7fc6801544e0 2024-05-08T22:45:20.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.582+0000 7fc677fff700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 0x7fc66c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.582+0000 7fc677fff700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 0x7fc66c068c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6700090b0 tx=0x7fc670008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.590+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc67c0508c0 con 0x7fc6801544e0 2024-05-08T22:45:20.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.754+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc68006c800 con 0x7fc6801544e0 2024-05-08T22:45:20.758 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc6757fa700 1 -- 172.21.15.179:0/1814241437 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc67c053ee0 con 0x7fc6801544e0 2024-05-08T22:45:20.759 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:45:20.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 msgr2=0x7fc66c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 0x7fc66c068c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6700090b0 tx=0x7fc670008030 comp rx=0 tx=0).stop 2024-05-08T22:45:20.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 msgr2=0x7fc68013ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c700 tx=0x7fc67c00ca10 comp rx=0 tx=0).stop 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 shutdown_connections 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fc66c066760 0x7fc66c068c20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc68015a300 0x7fc68014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc68015acc0 0x7fc680137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 --2- 172.21.15.179:0/1814241437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6801544e0 0x7fc68013ac00 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 >> 172.21.15.179:0/1814241437 conn(0x7fc6800b18d0 msgr2=0x7fc68006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 shutdown_connections 2024-05-08T22:45:20.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.758+0000 7fc685d39700 1 -- 172.21.15.179:0/1814241437 wait complete. 2024-05-08T22:45:20.774 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:45:20.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 Processor -- start 2024-05-08T22:45:20.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- start start 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6fac0cc6c0 0x7f6fac165e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0d0640 con 0x7f6fac15afe0 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0b6b80 con 0x7f6fac0afae0 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0cb290 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fabfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6fac15afe0 0x7f6fac0cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57568/0 (socket says 172.21.15.179:57568) 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fabfff700 1 -- 172.21.15.179:0/2155618436 learned_addr learned my addr 172.21.15.179:0/2155618436 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:20.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb0939700 1 --2- 172.21.15.179:0/2155618436 >> v2:172.21.15.179:3300/0 conn(0x7f6fac0cc6c0 0x7f6fac165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fab7fe700 1 -- 172.21.15.179:0/2155618436 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895433896 0 0) 0x7f6fac0d0640 con 0x7f6fac15afe0 2024-05-08T22:45:20.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fab7fe700 1 -- 172.21.15.179:0/2155618436 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980035d0 con 0x7f6fac15afe0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb0939700 1 -- 172.21.15.179:0/2155618436 >> v1:172.21.15.195:6789/0 conn(0x7f6fac0afae0 legacy=0x7f6fac0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb0939700 1 -- 172.21.15.179:0/2155618436 >> v1:172.21.15.179:6789/0 conn(0x7f6fac15afe0 legacy=0x7f6fac0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb0939700 1 -- 172.21.15.179:0/2155618436 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0b00d0 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb0939700 1 --2- 172.21.15.179:0/2155618436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac165e50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009d90 tx=0x7f6fa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437fa495d41fa14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fab7fe700 1 -- 172.21.15.179:0/2155618436 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400e070 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fab7fe700 1 -- 172.21.15.179:0/2155618436 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400bb20 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fab7fe700 1 -- 172.21.15.179:0/2155618436 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4012430 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- 172.21.15.179:0/2155618436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 msgr2=0x7f6fac165e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2155618436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac165e50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009d90 tx=0x7f6fa400b330 comp rx=0 tx=0).stop 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- 172.21.15.179:0/2155618436 shutdown_connections 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2155618436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac165e50 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- 172.21.15.179:0/2155618436 >> 172.21.15.179:0/2155618436 conn(0x7f6fac0b2730 msgr2=0x7f6fac0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.882+0000 7f6fb113a700 1 -- 172.21.15.179:0/2155618436 shutdown_connections 2024-05-08T22:45:20.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 -- 172.21.15.179:0/2155618436 wait complete. 2024-05-08T22:45:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 Processor -- start 2024-05-08T22:45:20.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 -- start start 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fac0afae0 0x7f6fac0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fac15afe0 0x7f6fac15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0af560 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1f27b0 con 0x7f6fac15afe0 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb113a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac15b9e0 con 0x7f6fac0afae0 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fa37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fa37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34834/0 (socket says 172.21.15.179:34834) 2024-05-08T22:45:20.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fa37fe700 1 -- 172.21.15.179:0/2855512619 learned_addr learned my addr 172.21.15.179:0/2855512619 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fb0939700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fac15afe0 0x7f6fac15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fabfff700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fac0afae0 0x7f6fac0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fa37fe700 1 -- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fac0afae0 msgr2=0x7f6fac0af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.886+0000 7f6fa37fe700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fac0afae0 0x7f6fac0af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa37fe700 1 -- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fac15afe0 msgr2=0x7f6fac15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa37fe700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fac15afe0 0x7f6fac15ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa37fe700 1 -- 172.21.15.179:0/2855512619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009a70 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa37fe700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400c500 tx=0x7f6f9400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940052d0 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94005430 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac15bcc0 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940055a0 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac15c280 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac04e700 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.890+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f940038e0 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.894+0000 7f6fa97fa700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:20.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.894+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f6f94083810 con 0x7f6fac0cc6c0 2024-05-08T22:45:20.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.894+0000 7f6fabfff700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:20.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.894+0000 7f6fabfff700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 0x7f6f8c068bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0011e0 tx=0x7f6f9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:20.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:20.898+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6f94050860 con 0x7f6fac0cc6c0 2024-05-08T22:45:21.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.074+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fac0b6400 con 0x7f6fac0cc6c0 2024-05-08T22:45:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.074+0000 7f6fa97fa700 1 -- 172.21.15.179:0/2855512619 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v129) v1 ==== 80+0+248 (secure 0 0 0) 0x7f6f94017110 con 0x7f6fac0cc6c0 2024-05-08T22:45:21.078 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/3161101014 2024-05-09T22:45:08.266131+0000 2024-05-08T22:45:21.078 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6834/3161101014 2024-05-09T22:45:08.266131+0000 2024-05-08T22:45:21.079 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6835/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:45:21.079 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.195:6834/2184096099 2024-05-09T22:44:47.759671+0000 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 msgr2=0x7f6f8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 0x7f6f8c068bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0011e0 tx=0x7f6f9c009040 comp rx=0 tx=0).stop 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 msgr2=0x7f6fac1f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400c500 tx=0x7f6f9400c810 comp rx=0 tx=0).stop 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 shutdown_connections 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fac15afe0 0x7f6fac15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fac0cc6c0 0x7f6fac1f2270 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 --2- 172.21.15.179:0/2855512619 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fac0afae0 0x7f6fac0af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 >> 172.21.15.179:0/2855512619 conn(0x7f6fac0b2730 msgr2=0x7f6fac1590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 shutdown_connections 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.078+0000 7f6fb113a700 1 -- 172.21.15.179:0/2855512619 wait complete. 2024-05-08T22:45:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:listed 4 entries 2024-05-08T22:45:21.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi179.front.sepia.ceph.com... 2024-05-08T22:45:21.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:21.093 DEBUG:teuthology.orchestra.run.smithi179:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T22:45:21.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi179.stderr:ceph-fuse[54316]: fuse finished with error 0 and tester_r 0 2024-05-08T22:45:21.110 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:45:21.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:21.110 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:45:21.168 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:45:21.168 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:21.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:45:21.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:45:21.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:21.176 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T22:45:21.223 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T22:45:21.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:45:21.225 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:45:21.225 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:21.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:45:21.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:45:21.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:21.233 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T22:45:21.280 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T22:45:21.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:45:21.281 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:45:21.282 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:21.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:45:21.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:45:21.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:21.290 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T22:45:21.296 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T22:45:21.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:45:21.297 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:21.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 Processor -- start 2024-05-08T22:45:21.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- start start 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 0x7f86e80b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80b3fb0 con 0x7f86e8165540 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e81659d0 con 0x7f86e8165c80 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8163b30 con 0x7f86e814ae40 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e5d9b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 0x7f86e80b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e6d9d700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f86e8165540 0x7f86e81619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:54796/0 (socket says 172.21.15.179:54796) 2024-05-08T22:45:21.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e5d9b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.179:46212/0 (socket says 172.21.15.179:46212) 2024-05-08T22:45:21.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e6d9d700 1 -- 172.21.15.179:0/2900446412 learned_addr learned my addr 172.21.15.179:0/2900446412 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:21.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 -- 172.21.15.179:0/2900446412 >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 msgr2=0x7f86e8165000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 --2- 172.21.15.179:0/2900446412 >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 -- 172.21.15.179:0/2900446412 >> v1:172.21.15.195:6789/0 conn(0x7f86e8165540 legacy=0x7f86e81619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 -- 172.21.15.179:0/2900446412 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e814b220 con 0x7f86e814ae40 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e5d9b700 1 --2- 172.21.15.179:0/2900446412 >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e659c700 1 --2- 172.21.15.179:0/2900446412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 0x7f86e80b2a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009850 tx=0x7f86d8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b71c6df07a9f40a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e559a700 1 -- 172.21.15.179:0/2900446412 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011830 con 0x7f86e814ae40 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e559a700 1 -- 172.21.15.179:0/2900446412 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8011990 con 0x7f86e814ae40 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e559a700 1 -- 172.21.15.179:0/2900446412 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011d40 con 0x7f86e814ae40 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- 172.21.15.179:0/2900446412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 msgr2=0x7f86e80b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 --2- 172.21.15.179:0/2900446412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 0x7f86e80b2a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009850 tx=0x7f86d8009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- 172.21.15.179:0/2900446412 shutdown_connections 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 --2- 172.21.15.179:0/2900446412 >> v2:172.21.15.195:3300/0 conn(0x7f86e8165c80 0x7f86e8165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 --2- 172.21.15.179:0/2900446412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e814ae40 0x7f86e80b2a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.422+0000 7f86e759e700 1 -- 172.21.15.179:0/2900446412 >> 172.21.15.179:0/2900446412 conn(0x7f86e80cfae0 msgr2=0x7f86e80cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:21.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- 172.21.15.179:0/2900446412 shutdown_connections 2024-05-08T22:45:21.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- 172.21.15.179:0/2900446412 wait complete. 2024-05-08T22:45:21.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 Processor -- start 2024-05-08T22:45:21.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- start start 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e814ae40 0x7f86e8139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8165540 0x7f86e813a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e814a3a0 con 0x7f86e8165c80 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8139600 con 0x7f86e814ae40 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8138ff0 con 0x7f86e8165540 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e6d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34840/0 (socket says 172.21.15.179:34840) 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e814ae40 0x7f86e8139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e5d9b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8165540 0x7f86e813a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.430 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.426+0000 7f86e6d9d700 1 -- 172.21.15.179:0/2394564872 learned_addr learned my addr 172.21.15.179:0/2394564872 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:21.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 -- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8165540 msgr2=0x7f86e813a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8165540 0x7f86e813a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 -- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e814ae40 msgr2=0x7f86e8139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e814ae40 0x7f86e8139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 -- 172.21.15.179:0/2394564872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009a70 con 0x7f86e8165c80 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e6d9d700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c4d0 tx=0x7f86d000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0005d90 con 0x7f86e8165c80 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d000ce80 con 0x7f86e8165c80 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d80094b0 con 0x7f86e8165c80 2024-05-08T22:45:21.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0007310 con 0x7f86e8165c80 2024-05-08T22:45:21.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e8139290 con 0x7f86e8165c80 2024-05-08T22:45:21.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.430+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8139770 con 0x7f86e8165c80 2024-05-08T22:45:21.434 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.434+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86d0007470 con 0x7f86e8165c80 2024-05-08T22:45:21.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.434+0000 7f86deffd700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 0x7f86c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.434+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f86d0083690 con 0x7f86e8165c80 2024-05-08T22:45:21.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.434+0000 7f86e659c700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 0x7f86c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.434+0000 7f86e659c700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 0x7f86c8068bf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f86d80090b0 tx=0x7f86d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.442+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f86d0050810 con 0x7f86e8165c80 2024-05-08T22:45:21.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.654+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86e804e700 con 0x7f86e8165c80 2024-05-08T22:45:21.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.654+0000 7f86deffd700 1 -- 172.21.15.179:0/2394564872 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4750 (secure 0 0 0) 0x7f86d0053e30 con 0x7f86e8165c80 2024-05-08T22:45:21.657 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:21.657 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:13.308112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 msgr2=0x7f86c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 0x7f86c8068bf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f86d80090b0 tx=0x7f86d8008030 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 msgr2=0x7f86e8157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c4d0 tx=0x7f86d000c7e0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 shutdown_connections 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f86c8066730 0x7f86c8068bf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8165c80 0x7f86e8157ea0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86e8165540 0x7f86e813a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 --2- 172.21.15.179:0/2394564872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86e814ae40 0x7f86e8139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 >> 172.21.15.179:0/2394564872 conn(0x7f86e80cfae0 msgr2=0x7f86e8161910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 shutdown_connections 2024-05-08T22:45:21.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.658+0000 7f86e759e700 1 -- 172.21.15.179:0/2394564872 wait complete. 2024-05-08T22:45:21.661 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 103 2024-05-08T22:45:21.673 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:45:21.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 Processor -- start 2024-05-08T22:45:21.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 -- start start 2024-05-08T22:45:21.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0a240ce2a0 0x7f0a2406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2406ddf0 con 0x7f0a241434d0 2024-05-08T22:45:21.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240cd200 con 0x7f0a2406eb80 2024-05-08T22:45:21.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a2a135700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2406f010 con 0x7f0a240ce2a0 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a29133700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0a241434d0 0x7f0a2406d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57578/0 (socket says 172.21.15.179:57578) 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a29133700 1 -- 172.21.15.179:0/2145535450 learned_addr learned my addr 172.21.15.179:0/2145535450 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.782+0000 7f0a28932700 1 --2- 172.21.15.179:0/2145535450 >> v2:172.21.15.195:3300/0 conn(0x7f0a240ce2a0 0x7f0a2406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315607682 0 0) 0x7f0a240cd200 con 0x7f0a2406eb80 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0c0035d0 con 0x7f0a2406eb80 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367698246 0 0) 0x7f0a2406ddf0 con 0x7f0a241434d0 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a240cd200 con 0x7f0a241434d0 2024-05-08T22:45:21.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a28932700 1 -- 172.21.15.179:0/2145535450 >> v1:172.21.15.195:6789/0 conn(0x7f0a2406eb80 legacy=0x7f0a240b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a28932700 1 -- 172.21.15.179:0/2145535450 >> v1:172.21.15.179:6789/0 conn(0x7f0a241434d0 legacy=0x7f0a2406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a28932700 1 -- 172.21.15.179:0/2145535450 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240cd4e0 con 0x7f0a240ce2a0 2024-05-08T22:45:21.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a28932700 1 --2- 172.21.15.179:0/2145535450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a2406e640 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e54226591f3d34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000e070 con 0x7f0a240ce2a0 2024-05-08T22:45:21.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2000bb20 con 0x7f0a240ce2a0 2024-05-08T22:45:21.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a1bfff700 1 -- 172.21.15.179:0/2145535450 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20012430 con 0x7f0a240ce2a0 2024-05-08T22:45:21.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 -- 172.21.15.179:0/2145535450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 msgr2=0x7f0a2406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 --2- 172.21.15.179:0/2145535450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a2406e640 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 -- 172.21.15.179:0/2145535450 shutdown_connections 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 --2- 172.21.15.179:0/2145535450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a2406e640 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 -- 172.21.15.179:0/2145535450 >> 172.21.15.179:0/2145535450 conn(0x7f0a240ce9d0 msgr2=0x7f0a240cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 -- 172.21.15.179:0/2145535450 shutdown_connections 2024-05-08T22:45:21.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.786+0000 7f0a2a135700 1 -- 172.21.15.179:0/2145535450 wait complete. 2024-05-08T22:45:21.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 Processor -- start 2024-05-08T22:45:21.792 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 -- start start 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a240a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a241434d0 0x7f0a2415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2413fd10 con 0x7f0a2406eb80 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24133380 con 0x7f0a240ce2a0 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.790+0000 7f0a2a135700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240a8370 con 0x7f0a241434d0 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34850/0 (socket says 172.21.15.179:34850) 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 -- 172.21.15.179:0/1527504346 learned_addr learned my addr 172.21.15.179:0/1527504346 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a28932700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a240a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29934700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a241434d0 0x7f0a2415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 -- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a241434d0 msgr2=0x7f0a2415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a241434d0 0x7f0a2415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 -- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 msgr2=0x7f0a240a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a240a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 -- 172.21.15.179:0/1527504346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20009a70 con 0x7f0a2406eb80 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a29133700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c700 tx=0x7f0a1400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14005d90 con 0x7f0a2406eb80 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140070d0 con 0x7f0a2406eb80 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14007480 con 0x7f0a2406eb80 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a24135910 con 0x7f0a2406eb80 2024-05-08T22:45:21.797 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a24135ed0 con 0x7f0a2406eb80 2024-05-08T22:45:21.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240b3d10 con 0x7f0a2406eb80 2024-05-08T22:45:21.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a140075e0 con 0x7f0a2406eb80 2024-05-08T22:45:21.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.794+0000 7f0a19ffb700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:21.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.798+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f0a14083a30 con 0x7f0a2406eb80 2024-05-08T22:45:21.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.798+0000 7f0a28932700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:21.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.798+0000 7f0a28932700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 0x7f0a10068bf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a20014040 tx=0x7f0a2001d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:21.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.802+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a14050b20 con 0x7f0a2406eb80 2024-05-08T22:45:21.977 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.974+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a24067740 con 0x7f0a2406eb80 2024-05-08T22:45:21.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a19ffb700 1 -- 172.21.15.179:0/1527504346 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16627 (secure 0 0 0) 0x7f0a14054140 con 0x7f0a2406eb80 2024-05-08T22:45:21.979 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:21.980 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":129,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:45:08.266211+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6834/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 msgr2=0x7f0a10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 0x7f0a10068bf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a20014040 tx=0x7f0a2001d040 comp rx=0 tx=0).stop 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 msgr2=0x7f0a240b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c700 tx=0x7f0a1400cac0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 shutdown_connections 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a240ce2a0 0x7f0a240a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a241434d0 0x7f0a2415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 --2- 172.21.15.179:0/1527504346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a2406eb80 0x7f0a240b0030 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:21.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 >> 172.21.15.179:0/1527504346 conn(0x7f0a240ce9d0 msgr2=0x7f0a2415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:21.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 shutdown_connections 2024-05-08T22:45:21.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:21.978+0000 7f0a2a135700 1 -- 172.21.15.179:0/1527504346 wait complete. 2024-05-08T22:45:21.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:33:09.439140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:44:25.550731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:44:26.214340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:45:21.994 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:45:22.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.110+0000 7fe1d6adb700 1 Processor -- start 2024-05-08T22:45:22.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.110+0000 7fe1d6adb700 1 -- start start 2024-05-08T22:45:22.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 0x7fe1d0167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00b8ee0 con 0x7fe1d015d0c0 2024-05-08T22:45:22.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d015d4a0 con 0x7fe1d00b8a80 2024-05-08T22:45:22.115 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d006abb0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d5ad9700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe1d015d0c0 0x7fe1d00d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57588/0 (socket says 172.21.15.179:57588) 2024-05-08T22:45:22.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d5ad9700 1 -- 172.21.15.179:0/3321334533 learned_addr learned my addr 172.21.15.179:0/3321334533 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:22.116 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d62da700 1 --2- 172.21.15.179:0/3321334533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 0x7fe1d0167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611549114 0 0) 0x7fe1d00b8ee0 con 0x7fe1d015d0c0 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1bc0035d0 con 0x7fe1d015d0c0 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d62da700 1 -- 172.21.15.179:0/3321334533 >> v1:172.21.15.195:6789/0 conn(0x7fe1d00b8a80 legacy=0x7fe1d00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d62da700 1 -- 172.21.15.179:0/3321334533 >> v1:172.21.15.179:6789/0 conn(0x7fe1d015d0c0 legacy=0x7fe1d00d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333040055 0 0) 0x7fe1d015d4a0 con 0x7fe1d00b8a80 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d62da700 1 -- 172.21.15.179:0/3321334533 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00d2500 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.117 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d62da700 1 --2- 172.21.15.179:0/3321334533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 0x7fe1d0167ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009880 tx=0x7fe1c8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54488434f89eb35f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8011830 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c8011990 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d4ad7700 1 -- 172.21.15.179:0/3321334533 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8011d20 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 -- 172.21.15.179:0/3321334533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 msgr2=0x7fe1d0167ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/3321334533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 0x7fe1d0167ed0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009880 tx=0x7fe1c8009b90 comp rx=0 tx=0).stop 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.114+0000 7fe1d6adb700 1 -- 172.21.15.179:0/3321334533 shutdown_connections 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/3321334533 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d00ce2b0 0x7fe1d0167ed0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 -- 172.21.15.179:0/3321334533 >> 172.21.15.179:0/3321334533 conn(0x7fe1d00d2ed0 msgr2=0x7fe1d00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 -- 172.21.15.179:0/3321334533 shutdown_connections 2024-05-08T22:45:22.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 -- 172.21.15.179:0/3321334533 wait complete. 2024-05-08T22:45:22.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 Processor -- start 2024-05-08T22:45:22.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.118+0000 7fe1d6adb700 1 -- start start 2024-05-08T22:45:22.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1d00b8a80 0x7fe1d0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d015d0c0 0x7fe1d015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0152ef0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d01510d0 con 0x7fe1d00b8a80 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d01513b0 con 0x7fe1d015d0c0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34856/0 (socket says 172.21.15.179:34856) 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 -- 172.21.15.179:0/1413343291 learned_addr learned my addr 172.21.15.179:0/1413343291 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d62da700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d015d0c0 0x7fe1d015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d5ad9700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1d00b8a80 0x7fe1d0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 -- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d015d0c0 msgr2=0x7fe1d015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d015d0c0 0x7fe1d015bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 -- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1d00b8a80 msgr2=0x7fe1d0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1d00b8a80 0x7fe1d0150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 -- 172.21.15.179:0/1413343291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80094b0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d52d8700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001f90 tx=0x7fe1b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8003e50 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80020e0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800ab20 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d0151690 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.122+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d0151c50 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d004e700 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1b800ac80 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1ce7fc700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 0x7fe1b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe1b8083e50 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1d5ad9700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 0x7fe1b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.126+0000 7fe1d5ad9700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 0x7fe1b0068bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.134+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe1b8050fd0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.306+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1d00676e0 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.306+0000 7fe1ce7fc700 1 -- 172.21.15.179:0/1413343291 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1b8017020 con 0x7fe1d00ce2b0 2024-05-08T22:45:22.309 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 1 2024-05-08T22:45:22.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 msgr2=0x7fe1b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 0x7fe1b0068bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 msgr2=0x7fe1d01529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001f90 tx=0x7fe1b800a340 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 shutdown_connections 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fe1b0066730 0x7fe1b0068bf0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1d015d0c0 0x7fe1d015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d00ce2b0 0x7fe1d01529b0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 --2- 172.21.15.179:0/1413343291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1d00b8a80 0x7fe1d0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 >> 172.21.15.179:0/1413343291 conn(0x7fe1d00d2ed0 msgr2=0x7fe1d00b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 shutdown_connections 2024-05-08T22:45:22.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.310+0000 7fe1d6adb700 1 -- 172.21.15.179:0/1413343291 wait complete. 2024-05-08T22:45:22.324 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:45:22.447 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 Processor -- start 2024-05-08T22:45:22.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 -- start start 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f39fc06d960 0x7f39fc06f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0cc720 con 0x7f39fc0b1700 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc06c1c0 con 0x7f39fc159860 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a0479a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc0cc980 con 0x7f39fc06d960 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a02d37700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f39fc0b1700 0x7f39fc06b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57598/0 (socket says 172.21.15.179:57598) 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01d35700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f39fc06d960 0x7f39fc06f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a02d37700 1 -- 172.21.15.179:0/2922048842 learned_addr learned my addr 172.21.15.179:0/2922048842 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923625186 0 0) 0x7f39fc0cc720 con 0x7f39fc0b1700 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e80035d0 con 0x7f39fc0b1700 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326231203 0 0) 0x7f39e80035d0 con 0x7f39fc0b1700 2024-05-08T22:45:22.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01d35700 1 -- 172.21.15.179:0/2922048842 >> v1:172.21.15.195:6789/0 conn(0x7f39fc159860 legacy=0x7f39fc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01d35700 1 -- 172.21.15.179:0/2922048842 >> v1:172.21.15.179:6789/0 conn(0x7f39fc0b1700 legacy=0x7f39fc06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01d35700 1 -- 172.21.15.179:0/2922048842 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0ccc60 con 0x7f39fc06d960 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01d35700 1 --2- 172.21.15.179:0/2922048842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc06d960 0x7f39fc06f040 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009850 tx=0x7f39f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf32592a883198be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4001a10 con 0x7f39fc06d960 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f8004030 con 0x7f39fc06d960 2024-05-08T22:45:22.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.446+0000 7f3a01534700 1 -- 172.21.15.179:0/2922048842 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80043c0 con 0x7f39fc06d960 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- 172.21.15.179:0/2922048842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc06d960 msgr2=0x7f39fc06f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 --2- 172.21.15.179:0/2922048842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc06d960 0x7f39fc06f040 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009850 tx=0x7f39f8009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- 172.21.15.179:0/2922048842 shutdown_connections 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 --2- 172.21.15.179:0/2922048842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc06d960 0x7f39fc06f040 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- 172.21.15.179:0/2922048842 >> 172.21.15.179:0/2922048842 conn(0x7f39fc098a70 msgr2=0x7f39fc0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- 172.21.15.179:0/2922048842 shutdown_connections 2024-05-08T22:45:22.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- 172.21.15.179:0/2922048842 wait complete. 2024-05-08T22:45:22.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 Processor -- start 2024-05-08T22:45:22.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- start start 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 0x7f39fc14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 0x7f39fc099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc099e60 con 0x7f39fc0b1700 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc14d290 con 0x7f39fc159860 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a0479a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc14c680 con 0x7f39fc06d960 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.450+0000 7f3a01d35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34874/0 (socket says 172.21.15.179:34874) 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 -- 172.21.15.179:0/3664922337 learned_addr learned my addr 172.21.15.179:0/3664922337 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a02536700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 0x7f39fc14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 -- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 msgr2=0x7f39fc14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a02d37700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 0x7f39fc099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 0x7f39fc14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 -- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 msgr2=0x7f39fc099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 0x7f39fc099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 -- 172.21.15.179:0/3664922337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80094b0 con 0x7f39fc0b1700 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a02536700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 0x7f39fc14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a02d37700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 0x7f39fc099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a01d35700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0711d0 tx=0x7f39f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8011830 con 0x7f39fc0b1700 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f801f070 con 0x7f39fc0b1700 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc14c960 con 0x7f39fc0b1700 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8011bd0 con 0x7f39fc0b1700 2024-05-08T22:45:22.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc14cf20 con 0x7f39fc0b1700 2024-05-08T22:45:22.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc14d400 con 0x7f39fc0b1700 2024-05-08T22:45:22.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.454+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39f8011d30 con 0x7f39fc0b1700 2024-05-08T22:45:22.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.458+0000 7f39f2ffd700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 0x7f39e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.458+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f39f8021070 con 0x7f39fc0b1700 2024-05-08T22:45:22.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.458+0000 7f3a02536700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 0x7f39e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.458+0000 7f3a02536700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 0x7f39e0068bf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000f60 tx=0x7f39ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.462+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f39f80530f0 con 0x7f39fc0b1700 2024-05-08T22:45:22.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.630+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39fc073290 con 0x7f39fc0b1700 2024-05-08T22:45:22.633 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.630+0000 7f39f2ffd700 1 -- 172.21.15.179:0/3664922337 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39f8056710 con 0x7f39fc0b1700 2024-05-08T22:45:22.633 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 32 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 msgr2=0x7f39e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 0x7f39e0068bf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000f60 tx=0x7f39ec009040 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 msgr2=0x7f39fc0993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0711d0 tx=0x7f39f8004770 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 shutdown_connections 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f39e0066730 0x7f39e0068bf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39fc159860 0x7f39fc099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39fc0b1700 0x7f39fc0993e0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 --2- 172.21.15.179:0/3664922337 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39fc06d960 0x7f39fc14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 >> 172.21.15.179:0/3664922337 conn(0x7f39fc098a70 msgr2=0x7f39fc0ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 shutdown_connections 2024-05-08T22:45:22.637 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.634+0000 7f3a0479a700 1 -- 172.21.15.179:0/3664922337 wait complete. 2024-05-08T22:45:22.648 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:45:22.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 Processor -- start 2024-05-08T22:45:22.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 -- start start 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fefbc160c70 0x7fefbc1610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc161610 con 0x7fefbc15d050 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc161b50 con 0x7fefbc160c70 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.766+0000 7fefc1634700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0cb640 con 0x7fefbc0cbad0 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fefbc160c70 0x7fefbc1610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbbfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fefbc15d050 0x7fefbc0b4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57606/0 (socket says 172.21.15.179:57606) 2024-05-08T22:45:22.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbbfff700 1 -- 172.21.15.179:0/3445946648 learned_addr learned my addr 172.21.15.179:0/3445946648 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc0e33700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 -- 172.21.15.179:0/3445946648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 msgr2=0x7fefbc0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbb7fe700 1 -- 172.21.15.179:0/3445946648 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811616331 0 0) 0x7fefbc161610 con 0x7fefbc15d050 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 -- 172.21.15.179:0/3445946648 >> v1:172.21.15.179:6789/0 conn(0x7fefbc15d050 legacy=0x7fefbc0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 -- 172.21.15.179:0/3445946648 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0cb310 con 0x7fefbc160c70 2024-05-08T22:45:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefb37fe700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc1610d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009850 tx=0x7fefa4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea71bb6e77cd7c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbb7fe700 1 -- 172.21.15.179:0/3445946648 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4011830 con 0x7fefbc160c70 2024-05-08T22:45:22.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbb7fe700 1 -- 172.21.15.179:0/3445946648 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4011990 con 0x7fefbc160c70 2024-05-08T22:45:22.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefbb7fe700 1 -- 172.21.15.179:0/3445946648 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4011d40 con 0x7fefbc160c70 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 -- 172.21.15.179:0/3445946648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 msgr2=0x7fefbc1610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc1610d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009850 tx=0x7fefa4009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 -- 172.21.15.179:0/3445946648 shutdown_connections 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 --2- 172.21.15.179:0/3445946648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc1610d0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 -- 172.21.15.179:0/3445946648 >> 172.21.15.179:0/3445946648 conn(0x7fefbc0b2710 msgr2=0x7fefbc0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 -- 172.21.15.179:0/3445946648 shutdown_connections 2024-05-08T22:45:22.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.770+0000 7fefc1634700 1 -- 172.21.15.179:0/3445946648 wait complete. 2024-05-08T22:45:22.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 Processor -- start 2024-05-08T22:45:22.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 -- start start 2024-05-08T22:45:22.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc15d050 0x7fefbc06fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0c9350 con 0x7fefbc160c70 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc14d000 con 0x7fefbc15d050 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc1634700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0c8740 con 0x7fefbc0cbad0 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34884/0 (socket says 172.21.15.179:34884) 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 -- 172.21.15.179:0/2522000479 learned_addr learned my addr 172.21.15.179:0/2522000479 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefbbfff700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefb37fe700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc15d050 0x7fefbc06fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 -- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 msgr2=0x7fefbc14cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc14cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 -- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc15d050 msgr2=0x7fefbc06fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc15d050 0x7fefbc06fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 -- 172.21.15.179:0/2522000479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40094b0 con 0x7fefbc160c70 2024-05-08T22:45:22.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefc0e33700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009dd0 tx=0x7fefb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.774+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4009c40 con 0x7fefbc160c70 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb400f040 con 0x7fefbc160c70 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc0c8a20 con 0x7fefbc160c70 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400ceb0 con 0x7fefbc160c70 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc0c8fe0 con 0x7fefbc160c70 2024-05-08T22:45:22.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc0c9500 con 0x7fefbc160c70 2024-05-08T22:45:22.781 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefb4014480 con 0x7fefbc160c70 2024-05-08T22:45:22.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefb97fa700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:22.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fefb4084a30 con 0x7fefbc160c70 2024-05-08T22:45:22.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.778+0000 7fefbbfff700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:22.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.782+0000 7fefbbfff700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 0x7fef9c068bf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fefac000f60 tx=0x7fefac007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:22.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.786+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fefbc0c9500 con 0x7fefbc160c70 2024-05-08T22:45:22.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.958+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fefbc0c9500 con 0x7fefbc160c70 2024-05-08T22:45:22.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.958+0000 7fefb97fa700 1 -- 172.21.15.179:0/2522000479 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefbc0c9500 con 0x7fefbc160c70 2024-05-08T22:45:22.962 INFO:teuthology.orchestra.run.smithi179.stdout:pg_num: 64 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 msgr2=0x7fef9c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 0x7fef9c068bf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fefac000f60 tx=0x7fefac007e00 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 msgr2=0x7fefbc0c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009dd0 tx=0x7fefb400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 shutdown_connections 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc15d050 0x7fefbc06fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefbc160c70 0x7fefbc0c7cf0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 --2- 172.21.15.179:0/2522000479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc0cbad0 0x7fefbc14cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 >> 172.21.15.179:0/2522000479 conn(0x7fefbc0b2710 msgr2=0x7fefbc15d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 shutdown_connections 2024-05-08T22:45:22.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:22.962+0000 7fefc1634700 1 -- 172.21.15.179:0/2522000479 wait complete. 2024-05-08T22:45:22.977 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:23.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.086+0000 7f4a152b3700 1 Processor -- start 2024-05-08T22:45:23.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.086+0000 7f4a152b3700 1 -- start start 2024-05-08T22:45:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a152b3700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4a100d0650 0x7f4a100ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a152b3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a100b5500 con 0x7f4a100b41c0 2024-05-08T22:45:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a152b3700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a100b4630 con 0x7f4a101447c0 2024-05-08T22:45:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a152b3700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a100b48f0 con 0x7f4a100d0650 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a14ab2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4a100b41c0 0x7f4a100b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57618/0 (socket says 172.21.15.179:57618) 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a14ab2700 1 -- 172.21.15.179:0/2145923047 learned_addr learned my addr 172.21.15.179:0/2145923047 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/2145923047 >> v2:172.21.15.195:3300/0 conn(0x7f4a100d0650 0x7f4a100ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809912720 0 0) 0x7f4a100b5500 con 0x7f4a100b41c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f80035d0 con 0x7f4a100b41c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313403143 0 0) 0x7f4a100b4630 con 0x7f4a101447c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a100b5500 con 0x7f4a101447c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77993045 0 0) 0x7f49f80035d0 con 0x7f4a100b41c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 >> v2:172.21.15.195:3300/0 conn(0x7f4a100d0650 msgr2=0x7f4a100ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 --2- 172.21.15.179:0/2145923047 >> v2:172.21.15.195:3300/0 conn(0x7f4a100d0650 0x7f4a100ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 >> v1:172.21.15.195:6789/0 conn(0x7f4a101447c0 legacy=0x7f4a100cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100b4bd0 con 0x7f4a100b41c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f4a080019f0 con 0x7f4a100b41c0 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 >> v1:172.21.15.179:6789/0 conn(0x7f4a100b41c0 legacy=0x7f4a100b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8003bb0 0x7f49f8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 0x7f49f8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8005230 0x7f49f800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100b48f0 con 0x7f49f8004870 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f800c0b0 con 0x7f49f8003bb0 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f800c270 con 0x7f49f8005230 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 0x7f49f8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a14ab2700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8005230 0x7f49f800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0ffff700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8003bb0 0x7f49f8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 -- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8005230 msgr2=0x7f49f800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8005230 0x7f49f800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 -- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8003bb0 msgr2=0x7f49f8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49f8003bb0 0x7f49f8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 -- 172.21.15.179:0/2145923047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80035d0 con 0x7f49f8004870 2024-05-08T22:45:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a14ab2700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8005230 0x7f49f800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a14ab2700 1 -- 172.21.15.179:0/2145923047 reap_dead start 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 0x7f49f8004cf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009b70 tx=0x7f49f80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fb92a220c20486 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000e6c0 con 0x7f49f8004870 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0000e820 con 0x7f49f8004870 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.090+0000 7f4a0effd700 1 -- 172.21.15.179:0/2145923047 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000eb90 con 0x7f49f8004870 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 msgr2=0x7f49f8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 0x7f49f8004cf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009b70 tx=0x7f49f80058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- 172.21.15.179:0/2145923047 shutdown_connections 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 --2- 172.21.15.179:0/2145923047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49f8004870 0x7f49f8004cf0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- 172.21.15.179:0/2145923047 >> 172.21.15.179:0/2145923047 conn(0x7f4a100b2840 msgr2=0x7f4a100b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- 172.21.15.179:0/2145923047 shutdown_connections 2024-05-08T22:45:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- 172.21.15.179:0/2145923047 wait complete. 2024-05-08T22:45:23.099 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:45:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 Processor -- start 2024-05-08T22:45:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.094+0000 7f4a152b3700 1 -- start start 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8004870 0x7f4a100c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a101419c0 0x7f4a1015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10131320 con 0x7f4a100cb9e0 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10131600 con 0x7f4a101419c0 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a101318e0 con 0x7f49f8004870 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34896/0 (socket says 172.21.15.179:34896) 2024-05-08T22:45:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 -- 172.21.15.179:0/96519039 learned_addr learned my addr 172.21.15.179:0/96519039 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0f7fe700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a101419c0 0x7f4a1015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0ffff700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8004870 0x7f4a100c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 -- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8004870 msgr2=0x7f4a100c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8004870 0x7f4a100c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 -- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a101419c0 msgr2=0x7f4a1015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a101419c0 0x7f4a1015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 -- 172.21.15.179:0/96519039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10131bc0 con 0x7f4a100cb9e0 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a14ab2700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f4a100b3cf0 tx=0x7f4a08001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800e920 con 0x7f4a100cb9e0 2024-05-08T22:45:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0800ea80 con 0x7f4a100cb9e0 2024-05-08T22:45:23.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a10131ea0 con 0x7f4a100cb9e0 2024-05-08T22:45:23.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800edf0 con 0x7f4a100cb9e0 2024-05-08T22:45:23.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a100cc070 con 0x7f4a100cb9e0 2024-05-08T22:45:23.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1004e700 con 0x7f4a100cb9e0 2024-05-08T22:45:23.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.098+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a0801d460 con 0x7f4a100cb9e0 2024-05-08T22:45:23.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:45:23.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.102+0000 7f4a0cff9700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 0x7f49fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.102+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f4a08084510 con 0x7f4a100cb9e0 2024-05-08T22:45:23.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.102+0000 7f4a0ffff700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 0x7f49fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.102+0000 7f4a0ffff700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 0x7f49fc068bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4a1006ffc0 tx=0x7f4a04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.114 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.114+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4a08051690 con 0x7f4a100cb9e0 2024-05-08T22:45:23.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.326+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a10132090 con 0x7f4a100cb9e0 2024-05-08T22:45:23.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.326+0000 7f4a0cff9700 1 -- 172.21.15.179:0/96519039 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4750 (secure 0 0 0) 0x7f4a08054cb0 con 0x7f4a100cb9e0 2024-05-08T22:45:23.329 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:23.329 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:13.308112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 msgr2=0x7f49fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 0x7f49fc068bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4a1006ffc0 tx=0x7f4a04009040 comp rx=0 tx=0).stop 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 msgr2=0x7f4a1006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f4a100b3cf0 tx=0x7f4a08001aa0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 shutdown_connections 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f49fc066730 0x7f49fc068bf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a101419c0 0x7f4a1015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a100cb9e0 0x7f4a1006f3f0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 --2- 172.21.15.179:0/96519039 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49f8004870 0x7f4a100c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 >> 172.21.15.179:0/96519039 conn(0x7f4a100b2840 msgr2=0x7f4a100cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 shutdown_connections 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.330+0000 7f4a152b3700 1 -- 172.21.15.179:0/96519039 wait complete. 2024-05-08T22:45:23.332 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 103 2024-05-08T22:45:23.344 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:45:23.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 Processor -- start 2024-05-08T22:45:23.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 -- start start 2024-05-08T22:45:23.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16940ad0e0 0x7f16940ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 0x7f1694160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16940ad4c0 con 0x7f16940b25c0 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16941609c0 con 0x7f16940ad0e0 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1698869700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169415d900 con 0x7f16940adf70 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16940ad0e0 0x7f16940ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16940ad0e0 0x7f16940ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34902/0 (socket says 172.21.15.179:34902) 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 -- 172.21.15.179:0/591226270 learned_addr learned my addr 172.21.15.179:0/591226270 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1692ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f16940b25c0 0x7f16940ad860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57624/0 (socket says 172.21.15.179:57624) 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1693fff700 1 --2- 172.21.15.179:0/591226270 >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 0x7f1694160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 -- 172.21.15.179:0/591226270 >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 msgr2=0x7f1694160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 --2- 172.21.15.179:0/591226270 >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 0x7f1694160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 -- 172.21.15.179:0/591226270 >> v1:172.21.15.179:6789/0 conn(0x7f16940b25c0 legacy=0x7f16940ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 -- 172.21.15.179:0/591226270 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169415dbe0 con 0x7f16940ad0e0 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f1693fff700 1 --2- 172.21.15.179:0/591226270 >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 0x7f1694160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16937fe700 1 --2- 172.21.15.179:0/591226270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f16940ac080 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f169406ef10 tx=0x7f168400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e7640f9119a4cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16927fc700 1 -- 172.21.15.179:0/591226270 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168400e070 con 0x7f16940ad0e0 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16927fc700 1 -- 172.21.15.179:0/591226270 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168400b9c0 con 0x7f16940ad0e0 2024-05-08T22:45:23.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.450+0000 7f16927fc700 1 -- 172.21.15.179:0/591226270 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168400bd70 con 0x7f16940ad0e0 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- 172.21.15.179:0/591226270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 msgr2=0x7f16940ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 --2- 172.21.15.179:0/591226270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f16940ac080 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f169406ef10 tx=0x7f168400b330 comp rx=0 tx=0).stop 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- 172.21.15.179:0/591226270 shutdown_connections 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 --2- 172.21.15.179:0/591226270 >> v2:172.21.15.195:3300/0 conn(0x7f16940adf70 0x7f1694160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 --2- 172.21.15.179:0/591226270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f16940ac080 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- 172.21.15.179:0/591226270 >> 172.21.15.179:0/591226270 conn(0x7f16940ce9d0 msgr2=0x7f16940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- 172.21.15.179:0/591226270 shutdown_connections 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- 172.21.15.179:0/591226270 wait complete. 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 Processor -- start 2024-05-08T22:45:23.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.454+0000 7f1698869700 1 -- start start 2024-05-08T22:45:23.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16940adf70 0x7f169415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16940b25c0 0x7f169415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16940cb3e0 con 0x7f16940ad0e0 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16940cb6c0 con 0x7f16940adf70 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16940cb9a0 con 0x7f16940b25c0 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34912/0 (socket says 172.21.15.179:34912) 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 -- 172.21.15.179:0/881860980 learned_addr learned my addr 172.21.15.179:0/881860980 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1693fff700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16940b25c0 0x7f169415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1692ffd700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16940adf70 0x7f169415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 -- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16940b25c0 msgr2=0x7f169415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16940b25c0 0x7f169415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 -- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16940adf70 msgr2=0x7f169415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16940adf70 0x7f169415f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 -- 172.21.15.179:0/881860980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009b30 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f16937fe700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f1694072020 tx=0x7f1684004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16840048c0 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16840099b0 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16940cbc80 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168401c680 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.458+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169414c980 con 0x7f16940ad0e0 2024-05-08T22:45:23.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.462+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1694067740 con 0x7f16940ad0e0 2024-05-08T22:45:23.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.462+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f168401c7e0 con 0x7f16940ad0e0 2024-05-08T22:45:23.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.462+0000 7f168bfff700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 0x7f1674068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.462+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f1684085930 con 0x7f16940ad0e0 2024-05-08T22:45:23.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.462+0000 7f1692ffd700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 0x7f1674068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.466+0000 7f1692ffd700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 0x7f1674068bf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f169415c550 tx=0x7f167c007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.470+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1684052a10 con 0x7f16940ad0e0 2024-05-08T22:45:23.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.642+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f169404e700 con 0x7f16940ad0e0 2024-05-08T22:45:23.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f168bfff700 1 -- 172.21.15.179:0/881860980 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16627 (secure 0 0 0) 0x7f1684056030 con 0x7f16940ad0e0 2024-05-08T22:45:23.648 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:23.649 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":129,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:45:08.266211+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6834/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:45:23.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 msgr2=0x7f1674068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.650 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 0x7f1674068bf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f169415c550 tx=0x7f167c007ef0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 msgr2=0x7f169414c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f1694072020 tx=0x7f1684004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 shutdown_connections 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1674066730 0x7f1674068bf0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16940adf70 0x7f169415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16940b25c0 0x7f169415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 --2- 172.21.15.179:0/881860980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16940ad0e0 0x7f169414c440 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.646+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 >> 172.21.15.179:0/881860980 conn(0x7f16940ce9d0 msgr2=0x7f16940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.650+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 shutdown_connections 2024-05-08T22:45:23.651 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.650+0000 7f1698869700 1 -- 172.21.15.179:0/881860980 wait complete. 2024-05-08T22:45:23.664 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:45:23.664 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:23.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 Processor -- start 2024-05-08T22:45:23.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 -- start start 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f906406ec70 0x7f906406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 0x7f90640b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906406e060 con 0x7f90640b28b0 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906406de70 con 0x7f906406ec70 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f906359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906406f050 con 0x7f906415a0b0 2024-05-08T22:45:23.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f9061d9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f90640b28b0 0x7f90641599a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57636/0 (socket says 172.21.15.179:57636) 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.774+0000 7f9061d9b700 1 -- 172.21.15.179:0/2112136693 learned_addr learned my addr 172.21.15.179:0/2112136693 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 --2- 172.21.15.179:0/2112136693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 0x7f90640b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906259c700 1 --2- 172.21.15.179:0/2112136693 >> v2:172.21.15.195:3300/0 conn(0x7f906406ec70 0x7f906406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906159a700 1 -- 172.21.15.179:0/2112136693 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170571400 0 0) 0x7f906406e060 con 0x7f90640b28b0 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906159a700 1 -- 172.21.15.179:0/2112136693 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90500035d0 con 0x7f90640b28b0 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 -- 172.21.15.179:0/2112136693 >> v2:172.21.15.195:3300/0 conn(0x7f906406ec70 msgr2=0x7f906406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 --2- 172.21.15.179:0/2112136693 >> v2:172.21.15.195:3300/0 conn(0x7f906406ec70 0x7f906406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 -- 172.21.15.179:0/2112136693 >> v1:172.21.15.179:6789/0 conn(0x7f90640b28b0 legacy=0x7f90641599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 -- 172.21.15.179:0/2112136693 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906406e340 con 0x7f906415a0b0 2024-05-08T22:45:23.780 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f9062d9d700 1 --2- 172.21.15.179:0/2112136693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 0x7f90640b2d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f905c009850 tx=0x7f905c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9b72df30b8a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906159a700 1 -- 172.21.15.179:0/2112136693 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c011830 con 0x7f906415a0b0 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906159a700 1 -- 172.21.15.179:0/2112136693 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c011990 con 0x7f906415a0b0 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906159a700 1 -- 172.21.15.179:0/2112136693 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c011d00 con 0x7f906415a0b0 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 -- 172.21.15.179:0/2112136693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 msgr2=0x7f90640b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 --2- 172.21.15.179:0/2112136693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 0x7f90640b2d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f905c009850 tx=0x7f905c009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 -- 172.21.15.179:0/2112136693 shutdown_connections 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 --2- 172.21.15.179:0/2112136693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906415a0b0 0x7f90640b2d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.782 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 --2- 172.21.15.179:0/2112136693 >> v2:172.21.15.195:3300/0 conn(0x7f906406ec70 0x7f906406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 -- 172.21.15.179:0/2112136693 >> 172.21.15.179:0/2112136693 conn(0x7f90640ce9d0 msgr2=0x7f90640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:23.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 -- 172.21.15.179:0/2112136693 shutdown_connections 2024-05-08T22:45:23.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.778+0000 7f906359e700 1 -- 172.21.15.179:0/2112136693 wait complete. 2024-05-08T22:45:23.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 Processor -- start 2024-05-08T22:45:23.783 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- start start 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906406ec70 0x7f9064135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90640b28b0 0x7f9064156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9064157800 con 0x7f906415a0b0 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9064157a80 con 0x7f90640b28b0 2024-05-08T22:45:23.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9064157d60 con 0x7f906406ec70 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34914/0 (socket says 172.21.15.179:34914) 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 -- 172.21.15.179:0/1982141018 learned_addr learned my addr 172.21.15.179:0/1982141018 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9061d9b700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90640b28b0 0x7f9064156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906259c700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906406ec70 0x7f9064135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 -- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906406ec70 msgr2=0x7f9064135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906406ec70 0x7f9064135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 -- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90640b28b0 msgr2=0x7f9064156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90640b28b0 0x7f9064156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 -- 172.21.15.179:0/1982141018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0094b0 con 0x7f906415a0b0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f9062d9d700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f905c0090b0 tx=0x7f905c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c004590 con 0x7f906415a0b0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0036c0 con 0x7f906415a0b0 2024-05-08T22:45:23.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9064158040 con 0x7f906415a0b0 2024-05-08T22:45:23.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c003a30 con 0x7f906415a0b0 2024-05-08T22:45:23.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.782+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9064158600 con 0x7f906415a0b0 2024-05-08T22:45:23.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.786+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9064067740 con 0x7f906415a0b0 2024-05-08T22:45:23.788 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.786+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f905c003b90 con 0x7f906415a0b0 2024-05-08T22:45:23.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.786+0000 7f905affd700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 0x7f9044068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:23.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.786+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f905c021030 con 0x7f906415a0b0 2024-05-08T22:45:23.789 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.786+0000 7f906259c700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 0x7f9044068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:23.790 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.790+0000 7f906259c700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 0x7f9044068bf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f904c009bf0 tx=0x7f904c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:23.798 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:23.794+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f905c052a20 con 0x7f906415a0b0 2024-05-08T22:45:24.004 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.002+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f906404e700 con 0x7f906415a0b0 2024-05-08T22:45:24.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.002+0000 7f905affd700 1 -- 172.21.15.179:0/1982141018 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4750 (secure 0 0 0) 0x7f905c056040 con 0x7f906415a0b0 2024-05-08T22:45:24.006 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:24.006 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:13.308112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:24.008 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 msgr2=0x7f9044068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 0x7f9044068bf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f904c009bf0 tx=0x7f904c009240 comp rx=0 tx=0).stop 2024-05-08T22:45:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 msgr2=0x7f90641572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.009 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f905c0090b0 tx=0x7f905c004770 comp rx=0 tx=0).stop 2024-05-08T22:45:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 shutdown_connections 2024-05-08T22:45:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9044066730 0x7f9044068bf0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90640b28b0 0x7f9064156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f906415a0b0 0x7f90641572c0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 --2- 172.21.15.179:0/1982141018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f906406ec70 0x7f9064135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 >> 172.21.15.179:0/1982141018 conn(0x7f90640ce9d0 msgr2=0x7f90641e28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 shutdown_connections 2024-05-08T22:45:24.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.006+0000 7f906359e700 1 -- 172.21.15.179:0/1982141018 wait complete. 2024-05-08T22:45:24.011 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 103 2024-05-08T22:45:24.021 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:24.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.134+0000 7f2ee10e9700 1 Processor -- start 2024-05-08T22:45:24.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.134+0000 7f2ee10e9700 1 -- start start 2024-05-08T22:45:24.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2edc158b80 0x7f2edc15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc158f60 con 0x7f2edc06c230 2024-05-08T22:45:24.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc06b860 con 0x7f2edc158b80 2024-05-08T22:45:24.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06ac50 con 0x7f2edc0c8d70 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.179:48906/0 (socket says 172.21.15.179:48906) 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 -- 172.21.15.179:0/1564204472 learned_addr learned my addr 172.21.15.179:0/1564204472 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2edad9d700 1 --2- 172.21.15.179:0/1564204472 >> v2:172.21.15.195:3300/0 conn(0x7f2edc158b80 0x7f2edc15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 -- 172.21.15.179:0/1564204472 >> v2:172.21.15.195:3300/0 conn(0x7f2edc158b80 msgr2=0x7f2edc15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 --2- 172.21.15.179:0/1564204472 >> v2:172.21.15.195:3300/0 conn(0x7f2edc158b80 0x7f2edc15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.142 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 -- 172.21.15.179:0/1564204472 >> v1:172.21.15.195:6789/0 conn(0x7f2edc06c230 legacy=0x7f2edc0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 -- 172.21.15.179:0/1564204472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc06af30 con 0x7f2edc0c8d70 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ed9d9b700 1 -- 172.21.15.179:0/1564204472 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717274386 0 0) 0x7f2edc158f60 con 0x7f2edc06c230 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2eda59c700 1 --2- 172.21.15.179:0/1564204472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009e50 tx=0x7f2ecc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5e86141bf31575 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ed9d9b700 1 -- 172.21.15.179:0/1564204472 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00e070 con 0x7f2edc0c8d70 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ed9d9b700 1 -- 172.21.15.179:0/1564204472 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc00b910 con 0x7f2edc0c8d70 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ed9d9b700 1 -- 172.21.15.179:0/1564204472 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00bcc0 con 0x7f2edc0c8d70 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- 172.21.15.179:0/1564204472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 msgr2=0x7f2edc06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/1564204472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009e50 tx=0x7f2ecc00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- 172.21.15.179:0/1564204472 shutdown_connections 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/1564204472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc0c8d70 0x7f2edc06bcf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/1564204472 >> v2:172.21.15.195:3300/0 conn(0x7f2edc158b80 0x7f2edc15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- 172.21.15.179:0/1564204472 >> 172.21.15.179:0/1564204472 conn(0x7f2edc0cb660 msgr2=0x7f2edc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- 172.21.15.179:0/1564204472 shutdown_connections 2024-05-08T22:45:24.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.138+0000 7f2ee10e9700 1 -- 172.21.15.179:0/1564204472 wait complete. 2024-05-08T22:45:24.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.142+0000 7f2ee10e9700 1 Processor -- start 2024-05-08T22:45:24.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.142+0000 7f2ee10e9700 1 -- start start 2024-05-08T22:45:24.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 0x7f2edc0b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc158b80 0x7f2edc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0b8660 con 0x7f2edc06c230 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1524c0 con 0x7f2edc0c8d70 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc15c2a0 con 0x7f2edc158b80 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34920/0 (socket says 172.21.15.179:34920) 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2eda59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 0x7f2edc0b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edb59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc158b80 0x7f2edc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 -- 172.21.15.179:0/3817486900 learned_addr learned my addr 172.21.15.179:0/3817486900 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 -- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc158b80 msgr2=0x7f2edc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc158b80 0x7f2edc1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 -- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 msgr2=0x7f2edc0b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 0x7f2edc0b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 -- 172.21.15.179:0/3817486900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc009b30 con 0x7f2edc06c230 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2eda59c700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 0x7f2edc0b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:24.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2edad9d700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e9d0 tx=0x7f2ed000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0009720 con 0x7f2edc06c230 2024-05-08T22:45:24.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed000ee70 con 0x7f2edc06c230 2024-05-08T22:45:24.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000ca50 con 0x7f2edc06c230 2024-05-08T22:45:24.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc15c560 con 0x7f2edc06c230 2024-05-08T22:45:24.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc15cb20 con 0x7f2edc06c230 2024-05-08T22:45:24.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.146+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc067770 con 0x7f2edc06c230 2024-05-08T22:45:24.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.150+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ed000cbb0 con 0x7f2edc06c230 2024-05-08T22:45:24.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.150+0000 7f2ec37fe700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 0x7f2ec8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.150+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f2ed0084440 con 0x7f2edc06c230 2024-05-08T22:45:24.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.150+0000 7f2eda59c700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 0x7f2ec8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.150+0000 7f2eda59c700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 0x7f2ec8068bf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00b370 tx=0x7f2ecc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.158+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2ed0051520 con 0x7f2edc06c230 2024-05-08T22:45:24.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.366+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2edc06f000 con 0x7f2edc06c230 2024-05-08T22:45:24.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.366+0000 7f2ec37fe700 1 -- 172.21.15.179:0/3817486900 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4750 (secure 0 0 0) 0x7f2ed000ce60 con 0x7f2edc06c230 2024-05-08T22:45:24.369 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:24.369 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T22:44:27.216361+0000","modified":"2024-05-08T22:45:13.308112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"d","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6835/357696906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":357696906},{"type":"v1","addr":"172.21.15.195:6835","nonce":357696906}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T22:45:24.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 msgr2=0x7f2ec8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 0x7f2ec8068bf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00b370 tx=0x7f2ecc009000 comp rx=0 tx=0).stop 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 msgr2=0x7f2edc158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e9d0 tx=0x7f2ed000ece0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 shutdown_connections 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f2ec8066730 0x7f2ec8068bf0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2edc0c8d70 0x7f2edc0b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2edc158b80 0x7f2edc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 --2- 172.21.15.179:0/3817486900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc06c230 0x7f2edc158f60 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 >> 172.21.15.179:0/3817486900 conn(0x7f2edc0cb660 msgr2=0x7f2edc0c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 shutdown_connections 2024-05-08T22:45:24.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.370+0000 7f2ee10e9700 1 -- 172.21.15.179:0/3817486900 wait complete. 2024-05-08T22:45:24.373 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 103 2024-05-08T22:45:24.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:45:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.498+0000 7f9f2ae7c700 1 Processor -- start 2024-05-08T22:45:24.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 -- start start 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9f2415d390 0x7f9f240b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f240ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f240c5690 con 0x7f9f240ce890 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f240b4f00 con 0x7f9f2415d390 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2ae7c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240c41a0 con 0x7f9f240c5ba0 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2a67b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9f240ce890 0x7f9f24160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57652/0 (socket says 172.21.15.179:57652) 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2a67b700 1 -- 172.21.15.179:0/837137536 learned_addr learned my addr 172.21.15.179:0/837137536 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:24.506 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29679700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f240ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829021886 0 0) 0x7f9f240c5690 con 0x7f9f240ce890 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/837137536 >> v2:172.21.15.195:3300/0 conn(0x7f9f2415d390 0x7f9f240b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f100035d0 con 0x7f9f240ce890 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295736544 0 0) 0x7f9f100035d0 con 0x7f9f240ce890 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 msgr2=0x7f9f240ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f240ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 >> v2:172.21.15.195:3300/0 conn(0x7f9f2415d390 msgr2=0x7f9f240b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 --2- 172.21.15.179:0/837137536 >> v2:172.21.15.195:3300/0 conn(0x7f9f2415d390 0x7f9f240b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2415d770 con 0x7f9f240ce890 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9f1c0019f0 con 0x7f9f240ce890 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 >> v1:172.21.15.179:6789/0 conn(0x7f9f240ce890 legacy=0x7f9f24160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f10003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f10004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f10004db0 0x7f9f10005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b4f00 con 0x7f9f10003a20 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240c41a0 con 0x7f9f10004db0 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1000bab0 con 0x7f9f240c5ba0 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f10003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29679700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f10004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f2a67b700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f10004db0 0x7f9f10005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 -- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 msgr2=0x7f9f10004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f240c5ba0 0x7f9f10004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 -- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f10004db0 msgr2=0x7f9f10005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f10004db0 0x7f9f10005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 -- 172.21.15.179:0/837137536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100035d0 con 0x7f9f10003a20 2024-05-08T22:45:24.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f10003e40 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009b70 tx=0x7f9f100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59e2ee831bdf451 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400e6c0 con 0x7f9f10003a20 2024-05-08T22:45:24.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400e820 con 0x7f9f10003a20 2024-05-08T22:45:24.509 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.502+0000 7f9f28e78700 1 -- 172.21.15.179:0/837137536 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400eb90 con 0x7f9f10003a20 2024-05-08T22:45:24.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/837137536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 msgr2=0x7f9f10003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/837137536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f10003e40 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009b70 tx=0x7f9f100058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2a67b700 1 -- 172.21.15.179:0/837137536 reap_dead start 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/837137536 shutdown_connections 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/837137536 >> 172.21.15.179:0/837137536 conn(0x7f9f24098a70 msgr2=0x7f9f240b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/837137536 shutdown_connections 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/837137536 wait complete. 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 Processor -- start 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- start start 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f241f3140 0x7f9f240acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f24151a60 0x7f9f240ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24152650 con 0x7f9f10003a20 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240ad010 con 0x7f9f24151a60 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2ae7c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241f35a0 con 0x7f9f241f3140 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34938/0 (socket says 172.21.15.179:34938) 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 -- 172.21.15.179:0/1416007123 learned_addr learned my addr 172.21.15.179:0/1416007123 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29679700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f241f3140 0x7f9f240acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f2a67b700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f24151a60 0x7f9f240ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 -- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f241f3140 msgr2=0x7f9f240acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f241f3140 0x7f9f240acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 -- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f24151a60 msgr2=0x7f9f240ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f24151a60 0x7f9f240ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 -- 172.21.15.179:0/1416007123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2415ea00 con 0x7f9f10003a20 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f29e7a700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9f100047f0 tx=0x7f9f140047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.506+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400e6c0 con 0x7f9f10003a20 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14013e90 con 0x7f9f10003a20 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2415ece0 con 0x7f9f10003a20 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14004270 con 0x7f9f10003a20 2024-05-08T22:45:24.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2415f2a0 con 0x7f9f10003a20 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f140043d0 con 0x7f9f10003a20 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2404e700 con 0x7f9f10003a20 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f1a7fc700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 0x7f9f08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9f14085cb0 con 0x7f9f10003a20 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f29679700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 0x7f9f08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.510+0000 7f9f29679700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 0x7f9f08068bf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f2406bab0 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.519 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.518+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9f14052e20 con 0x7f9f10003a20 2024-05-08T22:45:24.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.690+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f240676e0 con 0x7f9f10003a20 2024-05-08T22:45:24.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f1a7fc700 1 -- 172.21.15.179:0/1416007123 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16627 (secure 0 0 0) 0x7f9f14056440 con 0x7f9f10003a20 2024-05-08T22:45:24.695 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:24.695 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":129,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:45:08.266211+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:44:25.550731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T22:44:26.214340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6834/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:45:24.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 msgr2=0x7f9f08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 0x7f9f08068bf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f2406bab0 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2024-05-08T22:45:24.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 msgr2=0x7f9f2415e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9f100047f0 tx=0x7f9f140047e0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 shutdown_connections 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9f08066730 0x7f9f08068bf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f24151a60 0x7f9f240ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9f241f3140 0x7f9f240acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 --2- 172.21.15.179:0/1416007123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f10003a20 0x7f9f2415e420 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 >> 172.21.15.179:0/1416007123 conn(0x7f9f24098a70 msgr2=0x7f9f2406db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 shutdown_connections 2024-05-08T22:45:24.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.694+0000 7f9f2ae7c700 1 -- 172.21.15.179:0/1416007123 wait complete. 2024-05-08T22:45:24.710 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:45:24.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 Processor -- start 2024-05-08T22:45:24.828 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 -- start start 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa38006e180 0x7fa38006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3800b25d0 0x7fa3800b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3800cc340 con 0x7fa38006dda0 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38006b540 con 0x7fa38006e180 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37f59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38006ac50 con 0x7fa3800b25d0 2024-05-08T22:45:24.830 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37e59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa38006dda0 0x7fa3800cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:57664/0 (socket says 172.21.15.179:57664) 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37e59c700 1 -- 172.21.15.179:0/2722514238 learned_addr learned my addr 172.21.15.179:0/2722514238 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 --2- 172.21.15.179:0/2722514238 >> v2:172.21.15.179:3300/0 conn(0x7fa38006e180 0x7fa38006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37ed9d700 1 --2- 172.21.15.179:0/2722514238 >> v2:172.21.15.195:3300/0 conn(0x7fa3800b25d0 0x7fa3800b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37dd9b700 1 -- 172.21.15.179:0/2722514238 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042892405 0 0) 0x7fa370001110 con 0x7fa38006dda0 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37dd9b700 1 -- 172.21.15.179:0/2722514238 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa36c0035d0 con 0x7fa38006dda0 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 -- 172.21.15.179:0/2722514238 >> v2:172.21.15.195:3300/0 conn(0x7fa3800b25d0 msgr2=0x7fa3800b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 --2- 172.21.15.179:0/2722514238 >> v2:172.21.15.195:3300/0 conn(0x7fa3800b25d0 0x7fa3800b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 -- 172.21.15.179:0/2722514238 >> v1:172.21.15.179:6789/0 conn(0x7fa38006dda0 legacy=0x7fa3800cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.831 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 -- 172.21.15.179:0/2722514238 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38006aef0 con 0x7fa38006e180 2024-05-08T22:45:24.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa375d9b700 1 --2- 172.21.15.179:0/2722514238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006e180 0x7fa38006e5e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fa380070af0 tx=0x7fa368009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c5015890a67a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37dd9b700 1 -- 172.21.15.179:0/2722514238 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368011830 con 0x7fa38006e180 2024-05-08T22:45:24.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37dd9b700 1 -- 172.21.15.179:0/2722514238 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368011990 con 0x7fa38006e180 2024-05-08T22:45:24.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.826+0000 7fa37dd9b700 1 -- 172.21.15.179:0/2722514238 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368011d20 con 0x7fa38006e180 2024-05-08T22:45:24.832 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- 172.21.15.179:0/2722514238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006e180 msgr2=0x7fa38006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 --2- 172.21.15.179:0/2722514238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006e180 0x7fa38006e5e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fa380070af0 tx=0x7fa368009b30 comp rx=0 tx=0).stop 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- 172.21.15.179:0/2722514238 shutdown_connections 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 --2- 172.21.15.179:0/2722514238 >> v2:172.21.15.195:3300/0 conn(0x7fa3800b25d0 0x7fa3800b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 --2- 172.21.15.179:0/2722514238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006e180 0x7fa38006e5e0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- 172.21.15.179:0/2722514238 >> 172.21.15.179:0/2722514238 conn(0x7fa3800ce960 msgr2=0x7fa3800ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- 172.21.15.179:0/2722514238 shutdown_connections 2024-05-08T22:45:24.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- 172.21.15.179:0/2722514238 wait complete. 2024-05-08T22:45:24.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 Processor -- start 2024-05-08T22:45:24.834 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.830+0000 7fa37f59e700 1 -- start start 2024-05-08T22:45:24.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa38006e180 0x7fa3801faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3800b25d0 0x7fa3800a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800a9040 con 0x7fa38006dda0 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801fb1e0 con 0x7fa3800b25d0 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801fa250 con 0x7fa38006e180 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34958/0 (socket says 172.21.15.179:34958) 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 -- 172.21.15.179:0/3067685023 learned_addr learned my addr 172.21.15.179:0/3067685023 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa375d9b700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa38006e180 0x7fa3801faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37ed9d700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3800b25d0 0x7fa3800a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 -- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa38006e180 msgr2=0x7fa3801faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa38006e180 0x7fa3801faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 -- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3800b25d0 msgr2=0x7fa3800a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3800b25d0 0x7fa3800a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:24.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 -- 172.21.15.179:0/3067685023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680094b0 con 0x7fa38006dda0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37e59c700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa370001270 tx=0x7fa37000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370005d90 con 0x7fa38006dda0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3700070d0 con 0x7fa38006dda0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3801fa810 con 0x7fa38006dda0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370007480 con 0x7fa38006dda0 2024-05-08T22:45:24.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3800a8a80 con 0x7fa38006dda0 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38004e700 con 0x7fa38006dda0 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.834+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3700075e0 con 0x7fa38006dda0 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.838+0000 7fa3777fe700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 0x7fa360068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.838+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fa370083710 con 0x7fa38006dda0 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.838+0000 7fa375d9b700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 0x7fa360068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:24.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.838+0000 7fa375d9b700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 0x7fa360068bf0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:24.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:24.842+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa370050890 con 0x7fa38006dda0 2024-05-08T22:45:25.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.050+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa3800cc340 con 0x7fa38006dda0 2024-05-08T22:45:25.054 INFO:tasks.ceph.mon.a.smithi179.stderr:2024-05-08T22:45:25.050+0000 7fce8d193700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:45:25.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.066+0000 7fa3777fe700 1 -- 172.21.15.179:0/3067685023 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa3800cc340 con 0x7fa38006dda0 2024-05-08T22:45:25.070 INFO:tasks.ceph.mds.d.smithi195.stderr: -1377> 2024-05-08T22:45:14.291+0000 7feaf679d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 msgr2=0x7fa360068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 0x7fa360068bf0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368008030 comp rx=0 tx=0).stop 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 msgr2=0x7fa3801f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa370001270 tx=0x7fa37000ca90 comp rx=0 tx=0).stop 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 shutdown_connections 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fa360066730 0x7fa360068bf0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3800b25d0 0x7fa3800a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa38006e180 0x7fa3801faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 --2- 172.21.15.179:0/3067685023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38006dda0 0x7fa3801f7850 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 >> 172.21.15.179:0/3067685023 conn(0x7fa3800ce960 msgr2=0x7fa38015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 shutdown_connections 2024-05-08T22:45:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.070+0000 7fa37f59e700 1 -- 172.21.15.179:0/3067685023 wait complete. 2024-05-08T22:45:25.073 INFO:teuthology.orchestra.run.smithi179.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:45:25.086 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:45:25.101 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-05-08T22:45:25.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 Processor -- start 2024-05-08T22:45:25.207 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 -- start start 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f09fc15d0f0 0x7f09fc0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 0x7f09fc0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc06c8c0 con 0x7f09fc15cd10 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc06cd50 con 0x7f09fc15d0f0 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a03215700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc06bcb0 con 0x7f09fc06b760 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f09fc15d0f0 0x7f09fc0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f09fc15d0f0 0x7f09fc0b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34968/0 (socket says 172.21.15.179:34968) 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 -- 172.21.15.179:0/3737610224 learned_addr learned my addr 172.21.15.179:0/3737610224 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a02a14700 1 --2- 172.21.15.179:0/3737610224 >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 0x7f09fc0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 -- 172.21.15.179:0/3737610224 >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 msgr2=0x7f09fc0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 --2- 172.21.15.179:0/3737610224 >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 0x7f09fc0b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 -- 172.21.15.179:0/3737610224 >> v1:172.21.15.195:6789/0 conn(0x7f09fc15cd10 legacy=0x7f09fc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 -- 172.21.15.179:0/3737610224 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc06bf90 con 0x7f09fc15d0f0 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a02a14700 1 --2- 172.21.15.179:0/3737610224 >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 0x7f09fc0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01a12700 1 --2- 172.21.15.179:0/3737610224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc15d0f0 0x7f09fc0b02e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0728a0 tx=0x7f09ec00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b51cbe1c1233c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01211700 1 -- 172.21.15.179:0/3737610224 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00dd90 con 0x7f09fc15d0f0 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01211700 1 -- 172.21.15.179:0/3737610224 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec010470 con 0x7f09fc15d0f0 2024-05-08T22:45:25.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.206+0000 7f0a01211700 1 -- 172.21.15.179:0/3737610224 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec010800 con 0x7f09fc15d0f0 2024-05-08T22:45:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- 172.21.15.179:0/3737610224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc15d0f0 msgr2=0x7f09fc0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- 172.21.15.179:0/3737610224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc15d0f0 0x7f09fc0b02e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0728a0 tx=0x7f09ec00be90 comp rx=0 tx=0).stop 2024-05-08T22:45:25.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- 172.21.15.179:0/3737610224 shutdown_connections 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- 172.21.15.179:0/3737610224 >> v2:172.21.15.195:3300/0 conn(0x7f09fc06b760 0x7f09fc0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- 172.21.15.179:0/3737610224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc15d0f0 0x7f09fc0b02e0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- 172.21.15.179:0/3737610224 >> 172.21.15.179:0/3737610224 conn(0x7f09fc098a70 msgr2=0x7f09fc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- 172.21.15.179:0/3737610224 shutdown_connections 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- 172.21.15.179:0/3737610224 wait complete. 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 Processor -- start 2024-05-08T22:45:25.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- start start 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc06b760 0x7f09fc151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 0x7f09fc0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0add60 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0ae3e0 con 0x7f09fc15cd10 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a03215700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0cb110 con 0x7f09fc06b760 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34984/0 (socket says 172.21.15.179:34984) 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 -- 172.21.15.179:0/2744815570 learned_addr learned my addr 172.21.15.179:0/2744815570 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:25.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a01a12700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 0x7f09fc0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02213700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc06b760 0x7f09fc151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 -- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc06b760 msgr2=0x7f09fc151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc06b760 0x7f09fc151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 -- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 msgr2=0x7f09fc0c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 0x7f09fc0c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 -- 172.21.15.179:0/2744815570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec00b810 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a01a12700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 0x7f09fc0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.210+0000 7f0a02a14700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f09f4007ef0 tx=0x7f09f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400de60 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f400f040 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc0cb3f0 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4003c80 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc0cb9b0 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc0d0250 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09f4004030 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 0x7f09e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f09f4084660 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.214+0000 7f0a02213700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 0x7f09e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.218+0000 7f0a02213700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 0x7f09e0068bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:25.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.222+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f09f4051710 con 0x7f09fc0cc1b0 2024-05-08T22:45:25.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:25.434+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09fc04e700 con 0x7f09fc0cc1b0 2024-05-08T22:45:26.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.074+0000 7f09f2ffd700 1 -- 172.21.15.179:0/2744815570 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09f4054d30 con 0x7f09fc0cc1b0 2024-05-08T22:45:26.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 msgr2=0x7f09e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 0x7f09e0068bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8009040 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 msgr2=0x7f09fc0c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f09f4007ef0 tx=0x7f09f400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 shutdown_connections 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f09e0066730 0x7f09e0068bf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc15cd10 0x7f09fc0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09fc0cc1b0 0x7f09fc0c8340 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 --2- 172.21.15.179:0/2744815570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc06b760 0x7f09fc151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 >> 172.21.15.179:0/2744815570 conn(0x7f09fc098a70 msgr2=0x7f09fc06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:26.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 shutdown_connections 2024-05-08T22:45:26.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.078+0000 7f0a03215700 1 -- 172.21.15.179:0/2744815570 wait complete. 2024-05-08T22:45:26.096 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:45:26.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.222+0000 7f6b88816700 1 Processor -- start 2024-05-08T22:45:26.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.222+0000 7f6b88816700 1 -- start start 2024-05-08T22:45:26.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 0x7f6b840cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b8406af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8406e2b0 con 0x7f6b840cd640 2024-05-08T22:45:26.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840cdaf0 con 0x7f6b84146760 2024-05-08T22:45:26.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8406e0b0 con 0x7f6b8406dc90 2024-05-08T22:45:26.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b8406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8259c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 0x7f6b840cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b81d9b700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6b840cd640 0x7f6b8406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:44110/0 (socket says 172.21.15.179:44110) 2024-05-08T22:45:26.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b81d9b700 1 -- 172.21.15.179:0/2641093886 learned_addr learned my addr 172.21.15.179:0/2641093886 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:26.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8159a700 1 -- 172.21.15.179:0/2641093886 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044306865 0 0) 0x7f6b8406e2b0 con 0x7f6b840cd640 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8159a700 1 -- 172.21.15.179:0/2641093886 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b700035d0 con 0x7f6b840cd640 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 -- 172.21.15.179:0/2641093886 >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 msgr2=0x7f6b840cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 --2- 172.21.15.179:0/2641093886 >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 0x7f6b840cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 -- 172.21.15.179:0/2641093886 >> v1:172.21.15.195:6789/0 conn(0x7f6b840cd640 legacy=0x7f6b8406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 -- 172.21.15.179:0/2641093886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8406e590 con 0x7f6b8406dc90 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8259c700 1 --2- 172.21.15.179:0/2641093886 >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 0x7f6b840cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b82d9d700 1 --2- 172.21.15.179:0/2641093886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b8406af70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009d90 tx=0x7f6b7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a687a4590c51c4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8159a700 1 -- 172.21.15.179:0/2641093886 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00e070 con 0x7f6b8406dc90 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8159a700 1 -- 172.21.15.179:0/2641093886 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c00bb20 con 0x7f6b8406dc90 2024-05-08T22:45:26.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b8159a700 1 -- 172.21.15.179:0/2641093886 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00be90 con 0x7f6b8406dc90 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- 172.21.15.179:0/2641093886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 msgr2=0x7f6b8406af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 --2- 172.21.15.179:0/2641093886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b8406af70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009d90 tx=0x7f6b7c00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- 172.21.15.179:0/2641093886 shutdown_connections 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 --2- 172.21.15.179:0/2641093886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b8406af70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 --2- 172.21.15.179:0/2641093886 >> v2:172.21.15.195:3300/0 conn(0x7f6b84146760 0x7f6b840cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- 172.21.15.179:0/2641093886 >> 172.21.15.179:0/2641093886 conn(0x7f6b840cb610 msgr2=0x7f6b840cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- 172.21.15.179:0/2641093886 shutdown_connections 2024-05-08T22:45:26.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.226+0000 7f6b88816700 1 -- 172.21.15.179:0/2641093886 wait complete. 2024-05-08T22:45:26.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 Processor -- start 2024-05-08T22:45:26.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- start start 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b84160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b840cd640 0x7f6b840ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84144e80 con 0x7f6b84146760 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840ad080 con 0x7f6b840cd640 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840ad360 con 0x7f6b8406dc90 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39050/0 (socket says 172.21.15.179:39050) 2024-05-08T22:45:26.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 -- 172.21.15.179:0/3848481575 learned_addr learned my addr 172.21.15.179:0/3848481575 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:26.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b8259c700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b84160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b81d9b700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b840cd640 0x7f6b840ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 -- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 msgr2=0x7f6b84160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b84160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 -- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b840cd640 msgr2=0x7f6b840ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b840cd640 0x7f6b840ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 -- 172.21.15.179:0/3848481575 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740094b0 con 0x7f6b84146760 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b82d9d700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00b370 tx=0x7f6b7c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c004880 con 0x7f6b84146760 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c020070 con 0x7f6b84146760 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c009a70 con 0x7f6b84146760 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0276c0 con 0x7f6b84146760 2024-05-08T22:45:26.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.230+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b840ad920 con 0x7f6b84146760 2024-05-08T22:45:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.234+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b7c012430 con 0x7f6b84146760 2024-05-08T22:45:26.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.234+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b84073290 con 0x7f6b84146760 2024-05-08T22:45:26.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.234+0000 7f6b7affd700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 0x7f6b64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:26.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.234+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f6b7c022070 con 0x7f6b84146760 2024-05-08T22:45:26.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.238+0000 7f6b8259c700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 0x7f6b64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:26.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.238+0000 7f6b8259c700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 0x7f6b64068bf0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009ce0 tx=0x7f6b74009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:26.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.246+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6b7c053080 con 0x7f6b84146760 2024-05-08T22:45:26.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:26.418+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b8406d0a0 con 0x7f6b84146760 2024-05-08T22:45:27.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.078+0000 7f6b7affd700 1 -- 172.21.15.179:0/3848481575 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6b7c02c020 con 0x7f6b84146760 2024-05-08T22:45:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 msgr2=0x7f6b64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 0x7f6b64068bf0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009ce0 tx=0x7f6b74009240 comp rx=0 tx=0).stop 2024-05-08T22:45:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 msgr2=0x7f6b841f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00b370 tx=0x7f6b7c004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 shutdown_connections 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6b64066730 0x7f6b64068bf0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b840cd640 0x7f6b840ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b84146760 0x7f6b841f2350 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 --2- 172.21.15.179:0/3848481575 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b8406dc90 0x7f6b84160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 >> 172.21.15.179:0/3848481575 conn(0x7f6b840cb610 msgr2=0x7f6b8406d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 shutdown_connections 2024-05-08T22:45:27.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.082+0000 7f6b88816700 1 -- 172.21.15.179:0/3848481575 wait complete. 2024-05-08T22:45:27.087 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:45:27.103 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:45:27.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 Processor -- start 2024-05-08T22:45:27.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 -- start start 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff8740b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff8740b4800 0x7ff87415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740cf870 con 0x7ff87415ae10 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff874070980 con 0x7ff8740b4800 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87acf9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b3ea0 con 0x7ff87406e760 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff879cf7700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff87415ae10 0x7ff8740cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50918/0 (socket says 172.21.15.179:50918) 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/2376259365 >> v2:172.21.15.179:3300/0 conn(0x7ff8740b4800 0x7ff87415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff879cf7700 1 -- 172.21.15.179:0/2376259365 learned_addr learned my addr 172.21.15.179:0/2376259365 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff8794f6700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff8740b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064571109 0 0) 0x7ff870001110 con 0x7ff87415ae10 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8600035d0 con 0x7ff87415ae10 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301133383 0 0) 0x7ff8600035d0 con 0x7ff87415ae10 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 msgr2=0x7ff8740b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff8740b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 >> v2:172.21.15.179:3300/0 conn(0x7ff8740b4800 msgr2=0x7ff87415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 --2- 172.21.15.179:0/2376259365 >> v2:172.21.15.179:3300/0 conn(0x7ff8740b4800 0x7ff87415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740703c0 con 0x7ff87415ae10 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7ff870001800 con 0x7ff87415ae10 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 >> v1:172.21.15.179:6789/0 conn(0x7ff87415ae10 legacy=0x7ff8740cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff860003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff860003fa0 0x7ff860004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860004db0 0x7ff860005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff874070980 con 0x7ff860003fa0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b3ea0 con 0x7ff860004db0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000ba80 con 0x7ff87406e760 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/2376259365 >> v2:172.21.15.179:3300/0 conn(0x7ff8740b4800 0x7ff87415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff8794f6700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff860003fa0 0x7ff860004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.218+0000 7ff879cf7700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff860003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860004db0 0x7ff860005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 -- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 msgr2=0x7ff860003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87406e760 0x7ff860003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 -- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860004db0 msgr2=0x7ff860005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860004db0 0x7ff860005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 -- 172.21.15.179:0/2376259365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600035d0 con 0x7ff860003fa0 2024-05-08T22:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860004db0 0x7ff860005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff8794f6700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff860003fa0 0x7ff860004870 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7ff868009ba0 tx=0x7ff8600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bdbe61214772a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800e6c0 con 0x7ff860003fa0 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800e820 con 0x7ff860003fa0 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff878cf5700 1 -- 172.21.15.179:0/2376259365 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800eb90 con 0x7ff860003fa0 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 -- 172.21.15.179:0/2376259365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff860003fa0 msgr2=0x7ff860004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 --2- 172.21.15.179:0/2376259365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff860003fa0 0x7ff860004870 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7ff868009ba0 tx=0x7ff8600058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87a4f8700 1 -- 172.21.15.179:0/2376259365 reap_dead start 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 -- 172.21.15.179:0/2376259365 shutdown_connections 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 -- 172.21.15.179:0/2376259365 >> 172.21.15.179:0/2376259365 conn(0x7ff87406f690 msgr2=0x7ff87406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 -- 172.21.15.179:0/2376259365 shutdown_connections 2024-05-08T22:45:27.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.222+0000 7ff87acf9700 1 -- 172.21.15.179:0/2376259365 wait complete. 2024-05-08T22:45:27.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 Processor -- start 2024-05-08T22:45:27.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 -- start start 2024-05-08T22:45:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860003fa0 0x7ff874140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87415cd20 0x7ff87414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87415c3c0 con 0x7ff87414faf0 2024-05-08T22:45:27.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87414ff70 con 0x7ff860003fa0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87415b4f0 con 0x7ff87415cd20 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39074/0 (socket says 172.21.15.179:39074) 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 -- 172.21.15.179:0/3909607390 learned_addr learned my addr 172.21.15.179:0/3909607390 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff8794f6700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87415cd20 0x7ff87414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff879cf7700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860003fa0 0x7ff874140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 -- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87415cd20 msgr2=0x7ff87414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87415cd20 0x7ff87414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 -- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860003fa0 msgr2=0x7ff874140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860003fa0 0x7ff874140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 -- 172.21.15.179:0/3909607390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87415b740 con 0x7ff87414faf0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87a4f8700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff874072b70 tx=0x7ff86c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:27.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00e070 con 0x7ff87414faf0 2024-05-08T22:45:27.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c003c50 con 0x7ff87414faf0 2024-05-08T22:45:27.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.226+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87415ba20 con 0x7ff87414faf0 2024-05-08T22:45:27.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003de0 con 0x7ff87414faf0 2024-05-08T22:45:27.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87415bfe0 con 0x7ff87414faf0 2024-05-08T22:45:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87415c1a0 con 0x7ff87414faf0 2024-05-08T22:45:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff86c011900 con 0x7ff87414faf0 2024-05-08T22:45:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff8667fc700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 0x7ff858068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:27.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5849+0+0 (secure 0 0 0) 0x7ff86c084860 con 0x7ff87414faf0 2024-05-08T22:45:27.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.230+0000 7ff879cf7700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 0x7ff858068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:27.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.234+0000 7ff879cf7700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 0x7ff858068bf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff870007b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:27.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.238+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff86c051b40 con 0x7ff87414faf0 2024-05-08T22:45:27.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:27.414+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff87415c500 con 0x7ff87414faf0 2024-05-08T22:45:28.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.086+0000 7ff8667fc700 1 -- 172.21.15.179:0/3909607390 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff87415c500 con 0x7ff87414faf0 2024-05-08T22:45:28.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 msgr2=0x7ff858068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 0x7ff858068bf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff870007b20 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 msgr2=0x7ff87415d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff874072b70 tx=0x7ff86c0114b0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 shutdown_connections 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff858066730 0x7ff858068bf0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.090+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87414faf0 0x7ff87415d620 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.094+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff87415cd20 0x7ff87414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.094+0000 7ff87acf9700 1 --2- 172.21.15.179:0/3909607390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff860003fa0 0x7ff874140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.094+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 >> 172.21.15.179:0/3909607390 conn(0x7ff87406f690 msgr2=0x7ff874158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.094+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 shutdown_connections 2024-05-08T22:45:28.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.094+0000 7ff87acf9700 1 -- 172.21.15.179:0/3909607390 wait complete. 2024-05-08T22:45:28.096 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'cephfs_data' removed 2024-05-08T22:45:28.109 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T22:45:28.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.222+0000 7f76ef49f700 1 Processor -- start 2024-05-08T22:45:28.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.222+0000 7f76ef49f700 1 -- start start 2024-05-08T22:45:28.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76e8143580 0x7f76e815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76e815ae10 0x7f76e806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e815e050 con 0x7f76e806c100 2024-05-08T22:45:28.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e806c540 con 0x7f76e815ae10 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e815b2c0 con 0x7f76e8143580 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76e815ae10 0x7f76e806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76e815ae10 0x7f76e806bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39088/0 (socket says 172.21.15.179:39088) 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 -- 172.21.15.179:0/2241284355 learned_addr learned my addr 172.21.15.179:0/2241284355 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ee49d700 1 --2- 172.21.15.179:0/2241284355 >> v2:172.21.15.195:3300/0 conn(0x7f76e8143580 0x7f76e815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 -- 172.21.15.179:0/2241284355 >> v2:172.21.15.195:3300/0 conn(0x7f76e8143580 msgr2=0x7f76e815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 --2- 172.21.15.179:0/2241284355 >> v2:172.21.15.195:3300/0 conn(0x7f76e8143580 0x7f76e815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 -- 172.21.15.179:0/2241284355 >> v1:172.21.15.195:6789/0 conn(0x7f76e806c100 legacy=0x7f76e80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 -- 172.21.15.179:0/2241284355 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e806f0a0 con 0x7f76e815ae10 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76edc9c700 1 --2- 172.21.15.179:0/2241284355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e806bbc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f76e806c750 tx=0x7f76d800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d64f179c6d3edc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ed49b700 1 -- 172.21.15.179:0/2241284355 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d800dd90 con 0x7f76e815ae10 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ed49b700 1 -- 172.21.15.179:0/2241284355 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8010470 con 0x7f76e815ae10 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ed49b700 1 -- 172.21.15.179:0/2241284355 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8010800 con 0x7f76e815ae10 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- 172.21.15.179:0/2241284355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 msgr2=0x7f76e806bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 --2- 172.21.15.179:0/2241284355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e806bbc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f76e806c750 tx=0x7f76d800be90 comp rx=0 tx=0).stop 2024-05-08T22:45:28.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- 172.21.15.179:0/2241284355 shutdown_connections 2024-05-08T22:45:28.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 --2- 172.21.15.179:0/2241284355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e806bbc0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 --2- 172.21.15.179:0/2241284355 >> v2:172.21.15.195:3300/0 conn(0x7f76e8143580 0x7f76e815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- 172.21.15.179:0/2241284355 >> 172.21.15.179:0/2241284355 conn(0x7f76e8070d80 msgr2=0x7f76e8071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:28.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- 172.21.15.179:0/2241284355 shutdown_connections 2024-05-08T22:45:28.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.226+0000 7f76ef49f700 1 -- 172.21.15.179:0/2241284355 wait complete. 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.230+0000 7f76ef49f700 1 Processor -- start 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.230+0000 7f76ef49f700 1 -- start start 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e806c100 0x7f76e80b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e8143580 0x7f76e80b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80b2ca0 con 0x7f76e815ae10 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80b1df0 con 0x7f76e8143580 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80b2020 con 0x7f76e806c100 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39102/0 (socket says 172.21.15.179:39102) 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 -- 172.21.15.179:0/3461717283 learned_addr learned my addr 172.21.15.179:0/3461717283 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ee49d700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e806c100 0x7f76e80b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76edc9c700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e8143580 0x7f76e80b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 -- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e806c100 msgr2=0x7f76e80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e806c100 0x7f76e80b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 -- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e8143580 msgr2=0x7f76e80b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e8143580 0x7f76e80b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 -- 172.21.15.179:0/3461717283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d800b810 con 0x7f76e815ae10 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76eec9e700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f76e000cb50 tx=0x7f76e000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0003c20 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0003d80 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000aa20 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e80b2300 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e80b28c0 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.234+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e80b2de0 con 0x7f76e815ae10 2024-05-08T22:45:28.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.238+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76e000ab80 con 0x7f76e815ae10 2024-05-08T22:45:28.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.238+0000 7f76deffd700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 0x7f76cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.238+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f76e0083f90 con 0x7f76e815ae10 2024-05-08T22:45:28.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.238+0000 7f76ee49d700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 0x7f76cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.238+0000 7f76ee49d700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 0x7f76cc068bf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f76e4007a60 tx=0x7f76e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.246+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76e00b5cf0 con 0x7f76e815ae10 2024-05-08T22:45:28.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.414+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f76e804e700 con 0x7f76e815ae10 2024-05-08T22:45:28.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.418+0000 7f76deffd700 1 -- 172.21.15.179:0/3461717283 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f76e80b2de0 con 0x7f76e815ae10 2024-05-08T22:45:28.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 msgr2=0x7f76cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 0x7f76cc068bf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f76e4007a60 tx=0x7f76e4008040 comp rx=0 tx=0).stop 2024-05-08T22:45:28.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 msgr2=0x7f76e80a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f76e000cb50 tx=0x7f76e000ce60 comp rx=0 tx=0).stop 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 shutdown_connections 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76cc066730 0x7f76cc068bf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e8143580 0x7f76e80b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76e815ae10 0x7f76e80a9f40 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 --2- 172.21.15.179:0/3461717283 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e806c100 0x7f76e80b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 >> 172.21.15.179:0/3461717283 conn(0x7f76e8070d80 msgr2=0x7f76e80ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 shutdown_connections 2024-05-08T22:45:28.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.422+0000 7f76ef49f700 1 -- 172.21.15.179:0/3461717283 wait complete. 2024-05-08T22:45:28.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-08T22:45:28.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 Processor -- start 2024-05-08T22:45:28.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 -- start start 2024-05-08T22:45:28.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45200ca120 0x7f45200cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f452006b6b0 0x7f452006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452006dfa0 con 0x7f45200c9230 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45200ca730 con 0x7f45200ca120 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452006b2c0 con 0x7f452006b6b0 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45200ca120 0x7f45200cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45200ca120 0x7f45200cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39112/0 (socket says 172.21.15.179:39112) 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 -- 172.21.15.179:0/2724446800 learned_addr learned my addr 172.21.15.179:0/2724446800 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4526185700 1 --2- 172.21.15.179:0/2724446800 >> v2:172.21.15.195:3300/0 conn(0x7f452006b6b0 0x7f452006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 -- 172.21.15.179:0/2724446800 >> v2:172.21.15.195:3300/0 conn(0x7f452006b6b0 msgr2=0x7f452006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 --2- 172.21.15.179:0/2724446800 >> v2:172.21.15.195:3300/0 conn(0x7f452006b6b0 0x7f452006bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 -- 172.21.15.179:0/2724446800 >> v1:172.21.15.195:6789/0 conn(0x7f45200c9230 legacy=0x7f452006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:28.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 -- 172.21.15.179:0/2724446800 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452006d670 con 0x7f45200ca120 2024-05-08T22:45:28.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4525183700 1 --2- 172.21.15.179:0/2724446800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f45200cc7b0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f452006cac0 tx=0x7f451c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaa7166cd4d8e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4524982700 1 -- 172.21.15.179:0/2724446800 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00e070 con 0x7f45200ca120 2024-05-08T22:45:28.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4524982700 1 -- 172.21.15.179:0/2724446800 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c003a40 con 0x7f45200ca120 2024-05-08T22:45:28.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4524982700 1 -- 172.21.15.179:0/2724446800 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c003df0 con 0x7f45200ca120 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 -- 172.21.15.179:0/2724446800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 msgr2=0x7f45200cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.550+0000 7f4527be8700 1 --2- 172.21.15.179:0/2724446800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f45200cc7b0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f452006cac0 tx=0x7f451c00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 -- 172.21.15.179:0/2724446800 shutdown_connections 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 --2- 172.21.15.179:0/2724446800 >> v2:172.21.15.195:3300/0 conn(0x7f452006b6b0 0x7f452006bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 --2- 172.21.15.179:0/2724446800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f45200cc7b0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 -- 172.21.15.179:0/2724446800 >> 172.21.15.179:0/2724446800 conn(0x7f45200cb660 msgr2=0x7f45200cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 -- 172.21.15.179:0/2724446800 shutdown_connections 2024-05-08T22:45:28.556 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 -- 172.21.15.179:0/2724446800 wait complete. 2024-05-08T22:45:28.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 Processor -- start 2024-05-08T22:45:28.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.554+0000 7f4527be8700 1 -- start start 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f452006b6b0 0x7f45200c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 0x7f45200c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45200c6ea0 con 0x7f45200ca120 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45200c6290 con 0x7f452006b6b0 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45200c6570 con 0x7f45200c9230 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39126/0 (socket says 172.21.15.179:39126) 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 -- 172.21.15.179:0/155444906 learned_addr learned my addr 172.21.15.179:0/155444906 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4525183700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 0x7f45200c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4525984700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f452006b6b0 0x7f45200c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 -- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 msgr2=0x7f45200c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 0x7f45200c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 -- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f452006b6b0 msgr2=0x7f45200c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f452006b6b0 0x7f45200c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 -- 172.21.15.179:0/155444906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009b30 con 0x7f45200ca120 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4525183700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 0x7f45200c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4526185700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f451800c7f0 tx=0x7f451800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518004220 con 0x7f45200ca120 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518004380 con 0x7f45200ca120 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45200c6850 con 0x7f45200ca120 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45180036c0 con 0x7f45200ca120 2024-05-08T22:45:28.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45200c7cd0 con 0x7f45200ca120 2024-05-08T22:45:28.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.558+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4520067770 con 0x7f45200ca120 2024-05-08T22:45:28.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.562+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4518003820 con 0x7f45200ca120 2024-05-08T22:45:28.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.562+0000 7f45167fc700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 0x7f4504068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:28.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.562+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f4518083c50 con 0x7f45200ca120 2024-05-08T22:45:28.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.562+0000 7f4525984700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 0x7f4504068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:28.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.562+0000 7f4525984700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 0x7f4504068bf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4510004f70 tx=0x7f4510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:28.569 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.566+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f45180510c0 con 0x7f45200ca120 2024-05-08T22:45:28.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:28.798+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f45200c8bd0 con 0x7f45200ca120 2024-05-08T22:45:29.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.090+0000 7f45167fc700 1 -- 172.21.15.179:0/155444906 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v555) v1 ==== 144+0+0 (secure 0 0 0) 0x7f45200c8bd0 con 0x7f45200ca120 2024-05-08T22:45:29.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 msgr2=0x7f4504068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 0x7f4504068bf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4510004f70 tx=0x7f4510009040 comp rx=0 tx=0).stop 2024-05-08T22:45:29.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 msgr2=0x7f452006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f451800c7f0 tx=0x7f451800cbb0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 shutdown_connections 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4504066730 0x7f4504068bf0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45200ca120 0x7f452006a320 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45200c9230 0x7f45200c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 --2- 172.21.15.179:0/155444906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f452006b6b0 0x7f45200c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 >> 172.21.15.179:0/155444906 conn(0x7f45200cb660 msgr2=0x7f45200af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 shutdown_connections 2024-05-08T22:45:29.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.094+0000 7f4527be8700 1 -- 172.21.15.179:0/155444906 wait complete. 2024-05-08T22:45:29.112 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T22:45:29.113 INFO:tasks.cephfs_test_runner: 2024-05-08T22:45:29.113 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T22:45:29.113 INFO:tasks.cephfs_test_runner:Ran 7 tests in 682.573s 2024-05-08T22:45:29.113 INFO:tasks.cephfs_test_runner: 2024-05-08T22:45:29.113 INFO:tasks.cephfs_test_runner:OK 2024-05-08T22:45:29.113 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T22:45:29.123 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-08T22:45:29.141 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-08T22:45:29.141 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:45:29.141 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:29.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:45:29.150 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-08T22:45:29.150 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:29.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346292k,nr_inodes=4086573,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15488 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275484k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T22:45:29.158 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:45:29.158 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:29.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:45:29.167 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-08T22:45:29.167 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:45:29.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346296k,nr_inodes=4086574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=150 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275488k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275488k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T22:45:29.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.216 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2024-05-08T22:45:29.223 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T22:45:29.223 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T22:45:29.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T22:45:29.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T22:45:29.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.251 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2024-05-08T22:45:29.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.407 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-08T22:45:29.415 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T22:45:29.415 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:45:29.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.415 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T22:45:29.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.432 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:45:29.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:45:29.491 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-08T22:45:29.633 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T22:45:29.642 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T22:45:29.642 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:45:29.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.762+0000 7f124d4a4700 1 Processor -- start 2024-05-08T22:45:29.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.762+0000 7f124d4a4700 1 -- start start 2024-05-08T22:45:29.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f124815b000 0x7f12480cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12480d06d0 0x7f12480cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12480b4d70 con 0x7f12480b6550 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124815b3e0 con 0x7f12480d06d0 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480cf630 con 0x7f124815b000 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12480d06d0 0x7f12480cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124cca3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f12480b6550 0x7f12480b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50934/0 (socket says 172.21.15.179:50934) 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124cca3700 1 -- 172.21.15.179:0/3029101756 learned_addr learned my addr 172.21.15.179:0/3029101756 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:29.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f1247fff700 1 --2- 172.21.15.179:0/3029101756 >> v2:172.21.15.195:3300/0 conn(0x7f124815b000 0x7f12480cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 -- 172.21.15.179:0/3029101756 >> v2:172.21.15.195:3300/0 conn(0x7f124815b000 msgr2=0x7f12480cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 --2- 172.21.15.179:0/3029101756 >> v2:172.21.15.195:3300/0 conn(0x7f124815b000 0x7f12480cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 -- 172.21.15.179:0/3029101756 >> v1:172.21.15.179:6789/0 conn(0x7f12480b6550 legacy=0x7f12480b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 -- 172.21.15.179:0/3029101756 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480cf910 con 0x7f12480d06d0 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f12477fe700 1 --2- 172.21.15.179:0/3029101756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480cb930 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f12480d0b50 tx=0x7f1238009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8378c262cca0760 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f1246ffd700 1 -- 172.21.15.179:0/3029101756 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238011830 con 0x7f12480d06d0 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f1246ffd700 1 -- 172.21.15.179:0/3029101756 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1238011990 con 0x7f12480d06d0 2024-05-08T22:45:29.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f1246ffd700 1 -- 172.21.15.179:0/3029101756 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238011d20 con 0x7f12480d06d0 2024-05-08T22:45:29.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- 172.21.15.179:0/3029101756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 msgr2=0x7f12480cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 --2- 172.21.15.179:0/3029101756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480cb930 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f12480d0b50 tx=0x7f1238009b30 comp rx=0 tx=0).stop 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- 172.21.15.179:0/3029101756 shutdown_connections 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 --2- 172.21.15.179:0/3029101756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480cb930 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 --2- 172.21.15.179:0/3029101756 >> v2:172.21.15.195:3300/0 conn(0x7f124815b000 0x7f12480cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- 172.21.15.179:0/3029101756 >> 172.21.15.179:0/3029101756 conn(0x7f12480b2780 msgr2=0x7f12480b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- 172.21.15.179:0/3029101756 shutdown_connections 2024-05-08T22:45:29.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.766+0000 7f124d4a4700 1 -- 172.21.15.179:0/3029101756 wait complete. 2024-05-08T22:45:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 Processor -- start 2024-05-08T22:45:29.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 -- start start 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 0x7f124814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f124815b000 0x7f12480c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480c7250 con 0x7f12480d06d0 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480c8d00 con 0x7f12480b6550 2024-05-08T22:45:29.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124d4a4700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480c8fe0 con 0x7f124815b000 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f12477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f1247fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 0x7f124814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f12477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39156/0 (socket says 172.21.15.179:39156) 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f12477fe700 1 -- 172.21.15.179:0/1606033671 learned_addr learned my addr 172.21.15.179:0/1606033671 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.770+0000 7f124cca3700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f124815b000 0x7f12480c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 -- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f124815b000 msgr2=0x7f12480c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f124815b000 0x7f12480c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 -- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 msgr2=0x7f124814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 0x7f124814e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 -- 172.21.15.179:0/1606033671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380094b0 con 0x7f12480d06d0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f1247fff700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 0x7f124814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f12477fe700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f12380090b0 tx=0x7f1238004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238004550 con 0x7f12480d06d0 2024-05-08T22:45:29.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123801f070 con 0x7f12480d06d0 2024-05-08T22:45:29.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12480c92c0 con 0x7f12480d06d0 2024-05-08T22:45:29.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12380038e0 con 0x7f12480d06d0 2024-05-08T22:45:29.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12480c9880 con 0x7f12480d06d0 2024-05-08T22:45:29.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124804e700 con 0x7f12480d06d0 2024-05-08T22:45:29.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.774+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1238003a40 con 0x7f12480d06d0 2024-05-08T22:45:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.778+0000 7f1244ff9700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 0x7f1234068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:29.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.778+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1238021070 con 0x7f12480d06d0 2024-05-08T22:45:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.778+0000 7f1247fff700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 0x7f1234068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:29.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.778+0000 7f1247fff700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 0x7f1234068bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f123c0112c0 tx=0x7f123c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:29.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.786+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1238052d10 con 0x7f12480d06d0 2024-05-08T22:45:29.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.958+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12480d01b0 con 0x7f1234066730 2024-05-08T22:45:29.966 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.962+0000 7f1244ff9700 1 -- 172.21.15.179:0/1606033671 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49404 (secure 0 0 0) 0x7f123c011c80 con 0x7f1234066730 2024-05-08T22:45:29.967 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 msgr2=0x7f1234068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 0x7f1234068bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f123c0112c0 tx=0x7f123c009240 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 msgr2=0x7f12480aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f12380090b0 tx=0x7f1238004770 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 shutdown_connections 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1234066730 0x7f1234068bf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f124815b000 0x7f12480c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12480d06d0 0x7f12480aef20 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 --2- 172.21.15.179:0/1606033671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12480b6550 0x7f124814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 >> 172.21.15.179:0/1606033671 conn(0x7f12480b2780 msgr2=0x7f12480b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 shutdown_connections 2024-05-08T22:45:29.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:29.966+0000 7f124d4a4700 1 -- 172.21.15.179:0/1606033671 wait complete. 2024-05-08T22:45:29.970 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:45:29.982 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-05-08T22:45:29.644589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":287424,"kb_used_data":26200,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754687296,"statfs":{"total":773094113280,"available":772799791104,"internally_reserved":0,"allocated":26828800,"data_stored":19808488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-169,"num_write_kb":-8288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002043"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":268,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-05-08T22:45:26.576478+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:45:26.576478+0000","last_peered":"2024-05-08T22:45:26.576478+0000","last_clean":"2024-05-08T22:45:26.576478+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:45:26.576478+0000","last_undegraded":"2024-05-08T22:45:26.576478+0000","last_fullsized":"2024-05-08T22:45:26.576478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.23699999999999999},"max":{"1min":0.91600000000000004,"5min":1.151,"15min":1.623},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":1.115,"15min":1.6899999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":1.113,"15min":1.1639999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.32900000000000001},"max":{"1min":1.2390000000000001,"5min":1.3129999999999999,"15min":2.5049999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97299999999999998,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":1.716},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.47599999999999998},"max":{"1min":1.2370000000000001,"5min":1.2929999999999999,"15min":1.306},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.345,"15min":1.4139999999999999},"last":1.042}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.218,"5min":1.429,"15min":1.429},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.038,"15min":1.0269999999999999},"min":{"1min":0.748,"5min":0.67800000000000005,"15min":0.56899999999999995},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.8740000000000001},"last":1.073}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.77900000000000003,"5min":0.58599999999999997,"15min":0.442},"max":{"1min":1.2909999999999999,"5min":1.421,"15min":1.446},"last":1.002},{"interface":"front","average":{"1min":1.016,"5min":1.038,"15min":1.0529999999999999},"min":{"1min":0.70799999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.321,"5min":1.512,"15min":1.512},"last":1.282}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8100000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.042},"min":{"1min":0.76400000000000001,"5min":0.629,"15min":0.55900000000000005},"max":{"1min":1.3300000000000001,"5min":1.476,"15min":1.8360000000000001},"last":1.1060000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":1.772,"15min":1.772},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72599999999999998,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.625}]},{"osd":2,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93300000000000005,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.0349999999999999,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.24,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.89500000000000002,"5min":1.1579999999999999,"15min":1.583},"last":0.93799999999999994}]},{"osd":4,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.74399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.242,"5min":1.395,"15min":1.431},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.018,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.262,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.70099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3029999999999999,"5min":1.389,"15min":1.5},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1,"15min":1.038},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2929999999999999,"5min":1.399,"15min":1.726},"last":0.98399999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99099999999999999,"15min":0.98599999999999999},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.331,"5min":1.351,"15min":1.4650000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0760000000000001},"min":{"1min":0.69799999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.361,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0920000000000001,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.105},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.3759999999999999,"5min":1.446,"15min":1.5049999999999999},"last":1.0469999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2224128,"data_stored":1345111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86399999999999999,"15min":0.84599999999999997},"min":{"1min":0.61199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.242,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":2.0409999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89600000000000002,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.206,"5min":1.3,"15min":1.514},"last":2.0680000000000001}]},{"osd":1,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":1.9339999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92100000000000004,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.443},"max":{"1min":1.117,"5min":1.415,"15min":1.488},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92800000000000005,"15min":0.90600000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58199999999999996},"max":{"1min":1.2230000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.53800000000000003},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.337},"last":1.9830000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.92700000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.53900000000000003},"max":{"1min":1.1619999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0169999999999999,"5min":1.252,"15min":1.252},"last":1.038}]},{"osd":4,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.94699999999999995},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.2769999999999999,"5min":1.571,"15min":1.611},"last":1.806},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.63300000000000001,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.105,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.177}]},{"osd":5,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92000000000000004,"15min":0.94099999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53300000000000003},"max":{"1min":1.304,"5min":1.681,"15min":1.681},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.077,"15min":1.056},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":2.4119999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95099999999999996},"min":{"1min":0.70299999999999996,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.1879999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.252},{"interface":"front","average":{"1min":1.036,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4319999999999999,"5min":1.4990000000000001,"15min":1.599},"last":2.1080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542294,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40896,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330944,"statfs":{"total":96636764160,"available":96594886656,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.83399999999999996},"min":{"1min":0.64200000000000002,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":1.097},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.62,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.046,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.173,"5min":1.1899999999999999,"15min":1.758},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.66200000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.0309999999999999,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88300000000000001,"15min":0.93100000000000005},"min":{"1min":0.63500000000000001,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.1890000000000001,"5min":1.214,"15min":1.4319999999999999},"last":1.171},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0980000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93899999999999995,"15min":1.014},"min":{"1min":0.78100000000000003,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.117,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94499999999999995,"15min":0.95499999999999996},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.41799999999999998},"max":{"1min":1.1919999999999999,"5min":1.329,"15min":1.5029999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2569999999999999,"5min":1.3340000000000001,"15min":1.538},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95499999999999996,"15min":0.98599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.44400000000000001},"max":{"1min":1.2010000000000001,"5min":1.494,"15min":1.494},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.032,"15min":1.0780000000000001},"min":{"1min":0.81100000000000005,"5min":0.67300000000000004,"15min":0.64000000000000001},"max":{"1min":1.327,"5min":1.427,"15min":1.681},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0369999999999999,"15min":1.0820000000000001},"min":{"1min":0.71099999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2330000000000001,"5min":1.403,"15min":1.6000000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":1.01},"min":{"1min":0.73599999999999999,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.1899999999999999,"5min":1.571,"15min":1.835},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1359999999999999,"15min":1.173},"min":{"1min":0.81899999999999995,"5min":0.81499999999999995,"15min":0.74099999999999999},"max":{"1min":1.649,"5min":1.649,"15min":2.2999999999999998},"last":1.304}]}]},{"osd":5,"up_from":14,"seq":60129542294,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36800,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1559999999999999,"5min":7.907,"15min":7.907},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.875,"15min":0.83399999999999996},"min":{"1min":0.68300000000000005,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.52},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.2410000000000001,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92700000000000005,"15min":0.85799999999999998},"min":{"1min":0.628,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.389,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.92800000000000005,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2170000000000001,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.175},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.91000000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48799999999999999},"max":{"1min":1.1779999999999999,"5min":10.121,"15min":10.121},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.501},"max":{"1min":1.103,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.02},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.94999999999999996,"15min":0.96199999999999997},"min":{"1min":0.749,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.204,"5min":1.262,"15min":1.298},"last":1.1299999999999999}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.0920000000000001,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.2290000000000001,"5min":15.135,"15min":15.135},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.048,"15min":1.0189999999999999},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.2989999999999999,"5min":4.319,"15min":4.319},"last":1.0600000000000001}]},{"osd":6,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96999999999999997,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.47599999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0640000000000001,"15min":1.0569999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1140000000000001,"15min":1.0960000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.57299999999999995},"max":{"1min":1.423,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":1.256},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.107,"15min":1.0669999999999999},"min":{"1min":0.88500000000000001,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.7350000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.48}]}]},{"osd":4,"up_from":14,"seq":60129542294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5824512,"data_stored":4943861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.55000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84999999999999998,"15min":0.85299999999999998},"min":{"1min":0.63400000000000001,"5min":0.55900000000000005,"15min":0.44600000000000001},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.0149999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.86099999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.173,"5min":1.506,"15min":1.506},"last":1.081},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88400000000000001,"15min":0.89100000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53100000000000003},"max":{"1min":1.175,"5min":1.179,"15min":1.3140000000000001},"last":1.163}]},{"osd":2,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.88},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.538,"15min":1.538},"last":0.753},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.92400000000000004},"min":{"1min":0.71799999999999997,"5min":0.623,"15min":0.53400000000000003},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.637},"last":1.349}]},{"osd":3,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.85699999999999998,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.177,"5min":1.2769999999999999,"15min":2.601},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.93000000000000005,"15min":1.008},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":2.8769999999999998},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3480000000000001,"5min":1.454,"15min":1.792},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.97899999999999998},"min":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.167,"5min":1.2949999999999999,"15min":1.575},"last":1.1279999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.96899999999999997,"15min":1.0309999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.58599999999999997},"max":{"1min":1.266,"5min":1.331,"15min":1.831},"last":1.29},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0600000000000001,"15min":1.075},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.63300000000000001},"max":{"1min":1.395,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.373}]},{"osd":7,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.01,"15min":1.05},"min":{"1min":0.68100000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3560000000000001,"5min":1.403,"15min":1.8500000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.111,"15min":1.1439999999999999},"min":{"1min":0.78900000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.327,"5min":1.46,"15min":1.944},"last":1.415}]}]},{"osd":2,"up_from":14,"seq":60129542295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1630208,"data_stored":754741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.95499999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.2190000000000001},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.97399999999999998,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.40400000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.3460000000000001},"last":0.84699999999999998}]},{"osd":3,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.155,"15min":1.155},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63600000000000001,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.94699999999999995}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.75700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.361,"15min":1.504},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"min":{"1min":0.72499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.226,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.95599999999999996,"15min":0.999},"min":{"1min":0.63100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.3129999999999999,"15min":1.484},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.97999999999999998,"15min":0.996},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.67200000000000004},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.456},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.01,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69599999999999995},"max":{"1min":1.3740000000000001,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.054,"5min":1.018,"15min":1.0549999999999999},"min":{"1min":0.81399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.0600000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.85499999999999998,"5min":0.71199999999999997,"15min":0.39300000000000002},"max":{"1min":1.3400000000000001,"5min":1.498,"15min":1.5409999999999999},"last":1.177},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0580000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.006}]}]},{"osd":0,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2224128,"data_stored":1345109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":1.0209999999999999,"15min":1.617},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.67500000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":1.1020000000000001}]},{"osd":2,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.0069999999999999,"5min":1.052,"15min":1.446},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":1.101,"15min":1.6910000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.746,"15min":0.77700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.1020000000000001,"15min":1.714},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":1.169}]},{"osd":4,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.61699999999999999},"max":{"1min":1.0489999999999999,"5min":1.4850000000000001,"15min":4.391},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89800000000000002,"15min":0.90100000000000002},"min":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0269999999999999,"5min":1.246,"15min":1.419},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.93700000000000006,"15min":0.97099999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91700000000000004,"15min":0.95099999999999996},"min":{"1min":0.69799999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":1.028},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0389999999999999,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":1.0309999999999999}]},{"osd":7,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.042,"15min":1.0609999999999999},"min":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.222},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99299999999999999,"15min":1.0129999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.60399999999999998},"max":{"1min":1.274,"5min":1.3200000000000001,"15min":1.514},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:45:29.983 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:45:30.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 Processor -- start 2024-05-08T22:45:30.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 -- start start 2024-05-08T22:45:30.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdb180cdc40 0x7fdb180ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb180d19c0 0x7fdb180d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1815d5b0 con 0x7fdb1815d1d0 2024-05-08T22:45:30.094 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1806c5d0 con 0x7fdb180cdc40 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1759e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb180b6010 con 0x7fdb180d19c0 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdb1815d1d0 0x7fdb1806b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50944/0 (socket says 172.21.15.179:50944) 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb1659c700 1 -- 172.21.15.179:0/1740340264 learned_addr learned my addr 172.21.15.179:0/1740340264 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb0ffff700 1 --2- 172.21.15.179:0/1740340264 >> v2:172.21.15.179:3300/0 conn(0x7fdb180cdc40 0x7fdb180ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb16d9d700 1 --2- 172.21.15.179:0/1740340264 >> v2:172.21.15.195:3300/0 conn(0x7fdb180d19c0 0x7fdb180d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb16d9d700 1 -- 172.21.15.179:0/1740340264 >> v2:172.21.15.179:3300/0 conn(0x7fdb180cdc40 msgr2=0x7fdb180ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb16d9d700 1 --2- 172.21.15.179:0/1740340264 >> v2:172.21.15.179:3300/0 conn(0x7fdb180cdc40 0x7fdb180ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb16d9d700 1 -- 172.21.15.179:0/1740340264 >> v1:172.21.15.179:6789/0 conn(0x7fdb1815d1d0 legacy=0x7fdb1806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.090+0000 7fdb16d9d700 1 -- 172.21.15.179:0/1740340264 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180ce5c0 con 0x7fdb180d19c0 2024-05-08T22:45:30.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb16d9d700 1 --2- 172.21.15.179:0/1740340264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb180d19c0 0x7fdb180d1e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdb180cf7d0 tx=0x7fdb1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be8a56abde70737 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb15d9b700 1 -- 172.21.15.179:0/1740340264 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000e070 con 0x7fdb180d19c0 2024-05-08T22:45:30.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb15d9b700 1 -- 172.21.15.179:0/1740340264 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1000bb20 con 0x7fdb180d19c0 2024-05-08T22:45:30.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb15d9b700 1 -- 172.21.15.179:0/1740340264 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10012430 con 0x7fdb180d19c0 2024-05-08T22:45:30.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 -- 172.21.15.179:0/1740340264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb180d19c0 msgr2=0x7fdb180d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 --2- 172.21.15.179:0/1740340264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb180d19c0 0x7fdb180d1e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdb180cf7d0 tx=0x7fdb1000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 -- 172.21.15.179:0/1740340264 shutdown_connections 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 --2- 172.21.15.179:0/1740340264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb180d19c0 0x7fdb180d1e10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 --2- 172.21.15.179:0/1740340264 >> v2:172.21.15.179:3300/0 conn(0x7fdb180cdc40 0x7fdb180ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 -- 172.21.15.179:0/1740340264 >> 172.21.15.179:0/1740340264 conn(0x7fdb180d2e70 msgr2=0x7fdb180d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 -- 172.21.15.179:0/1740340264 shutdown_connections 2024-05-08T22:45:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.094+0000 7fdb1759e700 1 -- 172.21.15.179:0/1740340264 wait complete. 2024-05-08T22:45:30.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 Processor -- start 2024-05-08T22:45:30.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- start start 2024-05-08T22:45:30.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 0x7fdb18150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1815d1d0 0x7fdb180c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18150b40 con 0x7fdb180d19c0 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181f3220 con 0x7fdb1815d1d0 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181f3500 con 0x7fdb180cdc40 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39162/0 (socket says 172.21.15.179:39162) 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 -- 172.21.15.179:0/602955178 learned_addr learned my addr 172.21.15.179:0/602955178 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1659c700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 0x7fdb18150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb16d9d700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1815d1d0 0x7fdb180c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 -- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 msgr2=0x7fdb18150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 0x7fdb18150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 -- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1815d1d0 msgr2=0x7fdb180c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1815d1d0 0x7fdb180c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 -- 172.21.15.179:0/602955178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb10009a70 con 0x7fdb180d19c0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1659c700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 0x7fdb18150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0ffff700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e9d0 tx=0x7fdb0000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00009720 con 0x7fdb180d19c0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000c850 con 0x7fdb180d19c0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181f37e0 con 0x7fdb180d19c0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000cbc0 con 0x7fdb180d19c0 2024-05-08T22:45:30.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.098+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb180c9f40 con 0x7fdb180d19c0 2024-05-08T22:45:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb0000cd20 con 0x7fdb180d19c0 2024-05-08T22:45:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1815f2e0 con 0x7fdb180d19c0 2024-05-08T22:45:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb0effd700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 0x7fdaf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fdb00084130 con 0x7fdb180d19c0 2024-05-08T22:45:30.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb1659c700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 0x7fdaf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.102+0000 7fdb1659c700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 0x7fdaf8068bf0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007fd0 tx=0x7fdb08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.106+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdb1815f2e0 con 0x7fdb180d19c0 2024-05-08T22:45:30.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.278+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb180d11c0 con 0x7fdaf8066730 2024-05-08T22:45:30.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.282+0000 7fdb0effd700 1 -- 172.21.15.179:0/602955178 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49404 (secure 0 0 0) 0x7fdb08001c40 con 0x7fdaf8066730 2024-05-08T22:45:30.287 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 msgr2=0x7fdaf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 0x7fdaf8068bf0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007fd0 tx=0x7fdb08009040 comp rx=0 tx=0).stop 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 msgr2=0x7fdb180c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e9d0 tx=0x7fdb0000ed90 comp rx=0 tx=0).stop 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 shutdown_connections 2024-05-08T22:45:30.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaf8066730 0x7fdaf8068bf0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1815d1d0 0x7fdb180c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb180d19c0 0x7fdb180c8b70 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 --2- 172.21.15.179:0/602955178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb180cdc40 0x7fdb18150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 >> 172.21.15.179:0/602955178 conn(0x7fdb180d2e70 msgr2=0x7fdb180b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 shutdown_connections 2024-05-08T22:45:30.290 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.286+0000 7fdb1759e700 1 -- 172.21.15.179:0/602955178 wait complete. 2024-05-08T22:45:30.291 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:45:30.303 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-05-08T22:45:29.644589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":287424,"kb_used_data":26200,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754687296,"statfs":{"total":773094113280,"available":772799791104,"internally_reserved":0,"allocated":26828800,"data_stored":19808488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-169,"num_write_kb":-8288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002043"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":268,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-05-08T22:45:26.576478+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:45:26.576478+0000","last_peered":"2024-05-08T22:45:26.576478+0000","last_clean":"2024-05-08T22:45:26.576478+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:45:26.576478+0000","last_undegraded":"2024-05-08T22:45:26.576478+0000","last_fullsized":"2024-05-08T22:45:26.576478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.23699999999999999},"max":{"1min":0.91600000000000004,"5min":1.151,"15min":1.623},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":1.115,"15min":1.6899999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":1.113,"15min":1.1639999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.32900000000000001},"max":{"1min":1.2390000000000001,"5min":1.3129999999999999,"15min":2.5049999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97299999999999998,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":1.716},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.47599999999999998},"max":{"1min":1.2370000000000001,"5min":1.2929999999999999,"15min":1.306},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.345,"15min":1.4139999999999999},"last":1.042}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.218,"5min":1.429,"15min":1.429},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.038,"15min":1.0269999999999999},"min":{"1min":0.748,"5min":0.67800000000000005,"15min":0.56899999999999995},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.8740000000000001},"last":1.073}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.77900000000000003,"5min":0.58599999999999997,"15min":0.442},"max":{"1min":1.2909999999999999,"5min":1.421,"15min":1.446},"last":1.002},{"interface":"front","average":{"1min":1.016,"5min":1.038,"15min":1.0529999999999999},"min":{"1min":0.70799999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.321,"5min":1.512,"15min":1.512},"last":1.282}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8100000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.042},"min":{"1min":0.76400000000000001,"5min":0.629,"15min":0.55900000000000005},"max":{"1min":1.3300000000000001,"5min":1.476,"15min":1.8360000000000001},"last":1.1060000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":1.772,"15min":1.772},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72599999999999998,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.625}]},{"osd":2,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93300000000000005,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.0349999999999999,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.24,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.89500000000000002,"5min":1.1579999999999999,"15min":1.583},"last":0.93799999999999994}]},{"osd":4,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.74399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.242,"5min":1.395,"15min":1.431},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.018,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.262,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.70099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3029999999999999,"5min":1.389,"15min":1.5},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1,"15min":1.038},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2929999999999999,"5min":1.399,"15min":1.726},"last":0.98399999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99099999999999999,"15min":0.98599999999999999},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.331,"5min":1.351,"15min":1.4650000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0760000000000001},"min":{"1min":0.69799999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.361,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0920000000000001,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.105},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.3759999999999999,"5min":1.446,"15min":1.5049999999999999},"last":1.0469999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2224128,"data_stored":1345111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86399999999999999,"15min":0.84599999999999997},"min":{"1min":0.61199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.242,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":2.0409999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89600000000000002,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.206,"5min":1.3,"15min":1.514},"last":2.0680000000000001}]},{"osd":1,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":1.9339999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92100000000000004,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.443},"max":{"1min":1.117,"5min":1.415,"15min":1.488},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92800000000000005,"15min":0.90600000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58199999999999996},"max":{"1min":1.2230000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.53800000000000003},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.337},"last":1.9830000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.92700000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.53900000000000003},"max":{"1min":1.1619999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0169999999999999,"5min":1.252,"15min":1.252},"last":1.038}]},{"osd":4,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.94699999999999995},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.2769999999999999,"5min":1.571,"15min":1.611},"last":1.806},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.63300000000000001,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.105,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.177}]},{"osd":5,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92000000000000004,"15min":0.94099999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53300000000000003},"max":{"1min":1.304,"5min":1.681,"15min":1.681},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.077,"15min":1.056},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":2.4119999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95099999999999996},"min":{"1min":0.70299999999999996,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.1879999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.252},{"interface":"front","average":{"1min":1.036,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4319999999999999,"5min":1.4990000000000001,"15min":1.599},"last":2.1080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542294,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40896,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330944,"statfs":{"total":96636764160,"available":96594886656,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.83399999999999996},"min":{"1min":0.64200000000000002,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":1.097},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.62,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.046,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.173,"5min":1.1899999999999999,"15min":1.758},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.66200000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.0309999999999999,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88300000000000001,"15min":0.93100000000000005},"min":{"1min":0.63500000000000001,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.1890000000000001,"5min":1.214,"15min":1.4319999999999999},"last":1.171},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0980000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93899999999999995,"15min":1.014},"min":{"1min":0.78100000000000003,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.117,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94499999999999995,"15min":0.95499999999999996},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.41799999999999998},"max":{"1min":1.1919999999999999,"5min":1.329,"15min":1.5029999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2569999999999999,"5min":1.3340000000000001,"15min":1.538},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95499999999999996,"15min":0.98599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.44400000000000001},"max":{"1min":1.2010000000000001,"5min":1.494,"15min":1.494},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.032,"15min":1.0780000000000001},"min":{"1min":0.81100000000000005,"5min":0.67300000000000004,"15min":0.64000000000000001},"max":{"1min":1.327,"5min":1.427,"15min":1.681},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0369999999999999,"15min":1.0820000000000001},"min":{"1min":0.71099999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2330000000000001,"5min":1.403,"15min":1.6000000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":1.01},"min":{"1min":0.73599999999999999,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.1899999999999999,"5min":1.571,"15min":1.835},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1359999999999999,"15min":1.173},"min":{"1min":0.81899999999999995,"5min":0.81499999999999995,"15min":0.74099999999999999},"max":{"1min":1.649,"5min":1.649,"15min":2.2999999999999998},"last":1.304}]}]},{"osd":5,"up_from":14,"seq":60129542294,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36800,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1559999999999999,"5min":7.907,"15min":7.907},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.875,"15min":0.83399999999999996},"min":{"1min":0.68300000000000005,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.52},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.2410000000000001,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92700000000000005,"15min":0.85799999999999998},"min":{"1min":0.628,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.389,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.92800000000000005,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2170000000000001,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.175},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.91000000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48799999999999999},"max":{"1min":1.1779999999999999,"5min":10.121,"15min":10.121},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.501},"max":{"1min":1.103,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.02},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.94999999999999996,"15min":0.96199999999999997},"min":{"1min":0.749,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.204,"5min":1.262,"15min":1.298},"last":1.1299999999999999}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.0920000000000001,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.2290000000000001,"5min":15.135,"15min":15.135},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.048,"15min":1.0189999999999999},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.2989999999999999,"5min":4.319,"15min":4.319},"last":1.0600000000000001}]},{"osd":6,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96999999999999997,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.47599999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0640000000000001,"15min":1.0569999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1140000000000001,"15min":1.0960000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.57299999999999995},"max":{"1min":1.423,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":1.256},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.107,"15min":1.0669999999999999},"min":{"1min":0.88500000000000001,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.7350000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.48}]}]},{"osd":4,"up_from":14,"seq":60129542294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5824512,"data_stored":4943861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.55000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84999999999999998,"15min":0.85299999999999998},"min":{"1min":0.63400000000000001,"5min":0.55900000000000005,"15min":0.44600000000000001},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.0149999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.86099999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.173,"5min":1.506,"15min":1.506},"last":1.081},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88400000000000001,"15min":0.89100000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53100000000000003},"max":{"1min":1.175,"5min":1.179,"15min":1.3140000000000001},"last":1.163}]},{"osd":2,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.88},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.538,"15min":1.538},"last":0.753},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.92400000000000004},"min":{"1min":0.71799999999999997,"5min":0.623,"15min":0.53400000000000003},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.637},"last":1.349}]},{"osd":3,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.85699999999999998,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.177,"5min":1.2769999999999999,"15min":2.601},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.93000000000000005,"15min":1.008},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":2.8769999999999998},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3480000000000001,"5min":1.454,"15min":1.792},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.97899999999999998},"min":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.167,"5min":1.2949999999999999,"15min":1.575},"last":1.1279999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.96899999999999997,"15min":1.0309999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.58599999999999997},"max":{"1min":1.266,"5min":1.331,"15min":1.831},"last":1.29},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0600000000000001,"15min":1.075},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.63300000000000001},"max":{"1min":1.395,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.373}]},{"osd":7,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.01,"15min":1.05},"min":{"1min":0.68100000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3560000000000001,"5min":1.403,"15min":1.8500000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.111,"15min":1.1439999999999999},"min":{"1min":0.78900000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.327,"5min":1.46,"15min":1.944},"last":1.415}]}]},{"osd":2,"up_from":14,"seq":60129542295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1630208,"data_stored":754741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.95499999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.2190000000000001},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.97399999999999998,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.40400000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.3460000000000001},"last":0.84699999999999998}]},{"osd":3,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.155,"15min":1.155},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63600000000000001,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.94699999999999995}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.75700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.361,"15min":1.504},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"min":{"1min":0.72499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.226,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.95599999999999996,"15min":0.999},"min":{"1min":0.63100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.3129999999999999,"15min":1.484},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.97999999999999998,"15min":0.996},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.67200000000000004},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.456},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.01,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69599999999999995},"max":{"1min":1.3740000000000001,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.054,"5min":1.018,"15min":1.0549999999999999},"min":{"1min":0.81399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.0600000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.85499999999999998,"5min":0.71199999999999997,"15min":0.39300000000000002},"max":{"1min":1.3400000000000001,"5min":1.498,"15min":1.5409999999999999},"last":1.177},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0580000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.006}]}]},{"osd":0,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2224128,"data_stored":1345109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":1.0209999999999999,"15min":1.617},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.67500000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":1.1020000000000001}]},{"osd":2,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.0069999999999999,"5min":1.052,"15min":1.446},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":1.101,"15min":1.6910000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.746,"15min":0.77700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.1020000000000001,"15min":1.714},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":1.169}]},{"osd":4,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.61699999999999999},"max":{"1min":1.0489999999999999,"5min":1.4850000000000001,"15min":4.391},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89800000000000002,"15min":0.90100000000000002},"min":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0269999999999999,"5min":1.246,"15min":1.419},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.93700000000000006,"15min":0.97099999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91700000000000004,"15min":0.95099999999999996},"min":{"1min":0.69799999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":1.028},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0389999999999999,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":1.0309999999999999}]},{"osd":7,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.042,"15min":1.0609999999999999},"min":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.222},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99299999999999999,"15min":1.0129999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.60399999999999998},"max":{"1min":1.274,"5min":1.3200000000000001,"15min":1.514},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:45:30.305 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T22:45:30.305 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:45:30.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 Processor -- start 2024-05-08T22:45:30.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 -- start start 2024-05-08T22:45:30.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f56640ccc90 0x7f566415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f56640cd070 0x7f56640cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566415b420 con 0x7f56640cef40 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566415ddf0 con 0x7f56640ccc90 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f566b003700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566415e0d0 con 0x7f56640cd070 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f56640ccc90 0x7f566415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f56640ccc90 0x7f566415aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39174/0 (socket says 172.21.15.179:39174) 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 -- 172.21.15.179:0/1719699812 learned_addr learned my addr 172.21.15.179:0/1719699812 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5663fff700 1 --2- 172.21.15.179:0/1719699812 >> v2:172.21.15.195:3300/0 conn(0x7f56640cd070 0x7f56640cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 -- 172.21.15.179:0/1719699812 >> v2:172.21.15.195:3300/0 conn(0x7f56640cd070 msgr2=0x7f56640cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 --2- 172.21.15.179:0/1719699812 >> v2:172.21.15.195:3300/0 conn(0x7f56640cd070 0x7f56640cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 -- 172.21.15.179:0/1719699812 >> v1:172.21.15.179:6789/0 conn(0x7f56640cef40 legacy=0x7f56640b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 -- 172.21.15.179:0/1719699812 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566415e3b0 con 0x7f56640ccc90 2024-05-08T22:45:30.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f5668d9f700 1 --2- 172.21.15.179:0/1719699812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f566415aee0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f56640b2c40 tx=0x7f564c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b10cc11cbe3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f56637fe700 1 -- 172.21.15.179:0/1719699812 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00dd30 con 0x7f56640ccc90 2024-05-08T22:45:30.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f56637fe700 1 -- 172.21.15.179:0/1719699812 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00de90 con 0x7f56640ccc90 2024-05-08T22:45:30.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.414+0000 7f56637fe700 1 -- 172.21.15.179:0/1719699812 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c010690 con 0x7f56640ccc90 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- 172.21.15.179:0/1719699812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 msgr2=0x7f566415aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 --2- 172.21.15.179:0/1719699812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f566415aee0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f56640b2c40 tx=0x7f564c00be90 comp rx=0 tx=0).stop 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- 172.21.15.179:0/1719699812 shutdown_connections 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 --2- 172.21.15.179:0/1719699812 >> v2:172.21.15.195:3300/0 conn(0x7f56640cd070 0x7f56640cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 --2- 172.21.15.179:0/1719699812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f566415aee0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- 172.21.15.179:0/1719699812 >> 172.21.15.179:0/1719699812 conn(0x7f56640b4d10 msgr2=0x7f5664099070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- 172.21.15.179:0/1719699812 shutdown_connections 2024-05-08T22:45:30.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- 172.21.15.179:0/1719699812 wait complete. 2024-05-08T22:45:30.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 Processor -- start 2024-05-08T22:45:30.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.418+0000 7f566b003700 1 -- start start 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 0x7f56641594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56640cef40 0x7f56640c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664159030 con 0x7f56640ccc90 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664157820 con 0x7f56640cd070 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5664158420 con 0x7f56640cef40 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39186/0 (socket says 172.21.15.179:39186) 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 -- 172.21.15.179:0/2177091921 learned_addr learned my addr 172.21.15.179:0/2177091921 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:30.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5663fff700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 0x7f56641594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f56695a0700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56640cef40 0x7f56640c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 -- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56640cef40 msgr2=0x7f56640c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56640cef40 0x7f56640c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 -- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 msgr2=0x7f56641594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 0x7f56641594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 -- 172.21.15.179:0/2177091921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c00b810 con 0x7f56640ccc90 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5663fff700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 0x7f56641594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f5668d9f700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f5664072a40 tx=0x7f564c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c003d10 con 0x7f56640ccc90 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c003e90 con 0x7f56640ccc90 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0106b0 con 0x7f56640ccc90 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5664158700 con 0x7f56640ccc90 2024-05-08T22:45:30.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5664158cc0 con 0x7f56640ccc90 2024-05-08T22:45:30.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.422+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5664073290 con 0x7f56640ccc90 2024-05-08T22:45:30.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.426+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f564c02c440 con 0x7f56640ccc90 2024-05-08T22:45:30.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.426+0000 7f56617fa700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 0x7f5644068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.426+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f564c085970 con 0x7f56640ccc90 2024-05-08T22:45:30.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.426+0000 7f5663fff700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 0x7f5644068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.426+0000 7f5663fff700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 0x7f5644068bf0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f56641fab40 tx=0x7f565c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.434+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f564c052de0 con 0x7f56640ccc90 2024-05-08T22:45:30.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.602+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56640cff60 con 0x7f5644066730 2024-05-08T22:45:30.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.606+0000 7f56617fa700 1 -- 172.21.15.179:0/2177091921 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49404 (secure 0 0 0) 0x7f565c00f050 con 0x7f5644066730 2024-05-08T22:45:30.609 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 msgr2=0x7f5644068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 0x7f5644068bf0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f56641fab40 tx=0x7f565c00b040 comp rx=0 tx=0).stop 2024-05-08T22:45:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 msgr2=0x7f56641572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f5664072a40 tx=0x7f564c004770 comp rx=0 tx=0).stop 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 shutdown_connections 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5644066730 0x7f5644068bf0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56640cd070 0x7f56641594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56640cef40 0x7f56640c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 --2- 172.21.15.179:0/2177091921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56640ccc90 0x7f56641572e0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 >> 172.21.15.179:0/2177091921 conn(0x7f56640b4d10 msgr2=0x7f566415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 shutdown_connections 2024-05-08T22:45:30.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.610+0000 7f566b003700 1 -- 172.21.15.179:0/2177091921 wait complete. 2024-05-08T22:45:30.614 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:45:30.629 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-05-08T22:45:29.644589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":287424,"kb_used_data":26200,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754687296,"statfs":{"total":773094113280,"available":772799791104,"internally_reserved":0,"allocated":26828800,"data_stored":19808488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-169,"num_write_kb":-8288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002043"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":268,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-05-08T22:45:26.576478+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:45:26.576478+0000","last_peered":"2024-05-08T22:45:26.576478+0000","last_clean":"2024-05-08T22:45:26.576478+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:45:26.576478+0000","last_undegraded":"2024-05-08T22:45:26.576478+0000","last_fullsized":"2024-05-08T22:45:26.576478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.23699999999999999},"max":{"1min":0.91600000000000004,"5min":1.151,"15min":1.623},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":1.115,"15min":1.6899999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":1.113,"15min":1.1639999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.32900000000000001},"max":{"1min":1.2390000000000001,"5min":1.3129999999999999,"15min":2.5049999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97299999999999998,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":1.716},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.47599999999999998},"max":{"1min":1.2370000000000001,"5min":1.2929999999999999,"15min":1.306},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.345,"15min":1.4139999999999999},"last":1.042}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.218,"5min":1.429,"15min":1.429},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.038,"15min":1.0269999999999999},"min":{"1min":0.748,"5min":0.67800000000000005,"15min":0.56899999999999995},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.8740000000000001},"last":1.073}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.77900000000000003,"5min":0.58599999999999997,"15min":0.442},"max":{"1min":1.2909999999999999,"5min":1.421,"15min":1.446},"last":1.002},{"interface":"front","average":{"1min":1.016,"5min":1.038,"15min":1.0529999999999999},"min":{"1min":0.70799999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.321,"5min":1.512,"15min":1.512},"last":1.282}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8100000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.042},"min":{"1min":0.76400000000000001,"5min":0.629,"15min":0.55900000000000005},"max":{"1min":1.3300000000000001,"5min":1.476,"15min":1.8360000000000001},"last":1.1060000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":1.772,"15min":1.772},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72599999999999998,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.625}]},{"osd":2,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93300000000000005,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.0349999999999999,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.24,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.89500000000000002,"5min":1.1579999999999999,"15min":1.583},"last":0.93799999999999994}]},{"osd":4,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.74399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.242,"5min":1.395,"15min":1.431},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.018,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.262,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.70099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3029999999999999,"5min":1.389,"15min":1.5},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1,"15min":1.038},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2929999999999999,"5min":1.399,"15min":1.726},"last":0.98399999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99099999999999999,"15min":0.98599999999999999},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.331,"5min":1.351,"15min":1.4650000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0760000000000001},"min":{"1min":0.69799999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.361,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0920000000000001,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.105},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.3759999999999999,"5min":1.446,"15min":1.5049999999999999},"last":1.0469999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2224128,"data_stored":1345111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86399999999999999,"15min":0.84599999999999997},"min":{"1min":0.61199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.242,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":2.0409999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89600000000000002,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.206,"5min":1.3,"15min":1.514},"last":2.0680000000000001}]},{"osd":1,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":1.9339999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92100000000000004,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.443},"max":{"1min":1.117,"5min":1.415,"15min":1.488},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92800000000000005,"15min":0.90600000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58199999999999996},"max":{"1min":1.2230000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.53800000000000003},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.337},"last":1.9830000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.92700000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.53900000000000003},"max":{"1min":1.1619999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0169999999999999,"5min":1.252,"15min":1.252},"last":1.038}]},{"osd":4,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.94699999999999995},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.2769999999999999,"5min":1.571,"15min":1.611},"last":1.806},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.63300000000000001,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.105,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.177}]},{"osd":5,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92000000000000004,"15min":0.94099999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53300000000000003},"max":{"1min":1.304,"5min":1.681,"15min":1.681},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.077,"15min":1.056},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":2.4119999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95099999999999996},"min":{"1min":0.70299999999999996,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.1879999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.252},{"interface":"front","average":{"1min":1.036,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4319999999999999,"5min":1.4990000000000001,"15min":1.599},"last":2.1080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542294,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40896,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330944,"statfs":{"total":96636764160,"available":96594886656,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.83399999999999996},"min":{"1min":0.64200000000000002,"5min":0.40999999999999998,"15min":0.38200000000000001},"max":{"1min":1.1060000000000001,"5min":1.2509999999999999,"15min":1.4550000000000001},"last":1.097},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.62,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.046,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.173,"5min":1.1899999999999999,"15min":1.758},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.66200000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.0309999999999999,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88300000000000001,"15min":0.93100000000000005},"min":{"1min":0.63500000000000001,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.1890000000000001,"5min":1.214,"15min":1.4319999999999999},"last":1.171},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0980000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93899999999999995,"15min":1.014},"min":{"1min":0.78100000000000003,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.117,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94499999999999995,"15min":0.95499999999999996},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.41799999999999998},"max":{"1min":1.1919999999999999,"5min":1.329,"15min":1.5029999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2569999999999999,"5min":1.3340000000000001,"15min":1.538},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95499999999999996,"15min":0.98599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.44400000000000001},"max":{"1min":1.2010000000000001,"5min":1.494,"15min":1.494},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.032,"15min":1.0780000000000001},"min":{"1min":0.81100000000000005,"5min":0.67300000000000004,"15min":0.64000000000000001},"max":{"1min":1.327,"5min":1.427,"15min":1.681},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0369999999999999,"15min":1.0820000000000001},"min":{"1min":0.71099999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2330000000000001,"5min":1.403,"15min":1.6000000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":1.01},"min":{"1min":0.73599999999999999,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.1899999999999999,"5min":1.571,"15min":1.835},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1359999999999999,"15min":1.173},"min":{"1min":0.81899999999999995,"5min":0.81499999999999995,"15min":0.74099999999999999},"max":{"1min":1.649,"5min":1.649,"15min":2.2999999999999998},"last":1.304}]}]},{"osd":5,"up_from":14,"seq":60129542294,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36800,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":5820416,"data_stored":4949040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1559999999999999,"5min":7.907,"15min":7.907},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.875,"15min":0.83399999999999996},"min":{"1min":0.68300000000000005,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.52},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.59899999999999998,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.2410000000000001,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92700000000000005,"15min":0.85799999999999998},"min":{"1min":0.628,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.389,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.92800000000000005,"15min":0.90200000000000002},"min":{"1min":0.67600000000000005,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2170000000000001,"5min":4.1790000000000003,"15min":4.1790000000000003},"last":1.175},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.91000000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48799999999999999},"max":{"1min":1.1779999999999999,"5min":10.121,"15min":10.121},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.66000000000000003,"5min":0.59399999999999997,"15min":0.501},"max":{"1min":1.103,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.02},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.94999999999999996,"15min":0.96199999999999997},"min":{"1min":0.749,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.204,"5min":1.262,"15min":1.298},"last":1.1299999999999999}]},{"osd":4,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.0920000000000001,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.2290000000000001,"5min":15.135,"15min":15.135},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.048,"15min":1.0189999999999999},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.52000000000000002},"max":{"1min":1.2989999999999999,"5min":4.319,"15min":4.319},"last":1.0600000000000001}]},{"osd":6,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96999999999999997,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.47599999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0640000000000001,"15min":1.0569999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed May 8 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1140000000000001,"15min":1.0960000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.57299999999999995},"max":{"1min":1.423,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":1.256},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.107,"15min":1.0669999999999999},"min":{"1min":0.88500000000000001,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.7350000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.48}]}]},{"osd":4,"up_from":14,"seq":60129542294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5824512,"data_stored":4943861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.55000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84999999999999998,"15min":0.85299999999999998},"min":{"1min":0.63400000000000001,"5min":0.55900000000000005,"15min":0.44600000000000001},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.0149999999999999}]},{"osd":1,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.86099999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.173,"5min":1.506,"15min":1.506},"last":1.081},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88400000000000001,"15min":0.89100000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53100000000000003},"max":{"1min":1.175,"5min":1.179,"15min":1.3140000000000001},"last":1.163}]},{"osd":2,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.88},"min":{"1min":0.624,"5min":0.496,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.538,"15min":1.538},"last":0.753},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.92400000000000004},"min":{"1min":0.71799999999999997,"5min":0.623,"15min":0.53400000000000003},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.637},"last":1.349}]},{"osd":3,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.85699999999999998,"5min":0.625,"15min":0.60599999999999998},"max":{"1min":1.177,"5min":1.2769999999999999,"15min":2.601},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.93000000000000005,"15min":1.008},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.242,"15min":2.8769999999999998},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3480000000000001,"5min":1.454,"15min":1.792},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.97899999999999998},"min":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.167,"5min":1.2949999999999999,"15min":1.575},"last":1.1279999999999999}]},{"osd":6,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.96899999999999997,"15min":1.0309999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.58599999999999997},"max":{"1min":1.266,"5min":1.331,"15min":1.831},"last":1.29},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0600000000000001,"15min":1.075},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.63300000000000001},"max":{"1min":1.395,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.373}]},{"osd":7,"last update":"Wed May 8 22:44:28 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.01,"15min":1.05},"min":{"1min":0.68100000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3560000000000001,"5min":1.403,"15min":1.8500000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.111,"15min":1.1439999999999999},"min":{"1min":0.78900000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.327,"5min":1.46,"15min":1.944},"last":1.415}]}]},{"osd":2,"up_from":14,"seq":60129542295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1630208,"data_stored":754741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.95499999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.2190000000000001},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.97399999999999998,"15min":1.048},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.40400000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.3460000000000001},"last":0.84699999999999998}]},{"osd":3,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.155,"15min":1.155},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63600000000000001,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.94699999999999995}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.75700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.361,"15min":1.504},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"min":{"1min":0.72499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.226,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.95599999999999996,"15min":0.999},"min":{"1min":0.63100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.3129999999999999,"15min":1.484},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.97999999999999998,"15min":0.996},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.67200000000000004},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.456},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.01,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69599999999999995},"max":{"1min":1.3740000000000001,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.054,"5min":1.018,"15min":1.0549999999999999},"min":{"1min":0.81399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.0600000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.85499999999999998,"5min":0.71199999999999997,"15min":0.39300000000000002},"max":{"1min":1.3400000000000001,"5min":1.498,"15min":1.5409999999999999},"last":1.177},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0580000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.006}]}]},{"osd":0,"up_from":14,"seq":60129542294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2224128,"data_stored":1345109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":1.0209999999999999,"15min":1.617},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.67500000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":1.1020000000000001}]},{"osd":2,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.0069999999999999,"5min":1.052,"15min":1.446},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":1.101,"15min":1.6910000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.746,"15min":0.77700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.1020000000000001,"15min":1.714},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":1.169}]},{"osd":4,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.61699999999999999},"max":{"1min":1.0489999999999999,"5min":1.4850000000000001,"15min":4.391},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89800000000000002,"15min":0.90100000000000002},"min":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0269999999999999,"5min":1.246,"15min":1.419},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.93700000000000006,"15min":0.97099999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91700000000000004,"15min":0.95099999999999996},"min":{"1min":0.69799999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":1.028},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0389999999999999,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":1.0309999999999999}]},{"osd":7,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.042,"15min":1.0609999999999999},"min":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.222},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99299999999999999,"15min":1.0129999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.60399999999999998},"max":{"1min":1.274,"5min":1.3200000000000001,"15min":1.514},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:45:30.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:45:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.738+0000 7f9f0298c700 1 Processor -- start 2024-05-08T22:45:30.742 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.738+0000 7f9f0298c700 1 -- start start 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9efc0cc700 0x7f9efc0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc06abb0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc06ae90 con 0x7f9efc0cb6e0 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc15b9c0 con 0x7f9efc0cc700 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f01189700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9efc15b1b0 0x7f9efc0cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50952/0 (socket says 172.21.15.179:50952) 2024-05-08T22:45:30.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f01189700 1 -- 172.21.15.179:0/3847646195 learned_addr learned my addr 172.21.15.179:0/3847646195 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:30.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 --2- 172.21.15.179:0/3847646195 >> v2:172.21.15.179:3300/0 conn(0x7f9efc0cc700 0x7f9efc0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510723196 0 0) 0x7f9efc06abb0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee40035d0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2416361896 0 0) 0x7f9ee40035d0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 >> v2:172.21.15.179:3300/0 conn(0x7f9efc0cc700 msgr2=0x7f9efc0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 --2- 172.21.15.179:0/3847646195 >> v2:172.21.15.179:3300/0 conn(0x7f9efc0cc700 0x7f9efc0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 >> v1:172.21.15.195:6789/0 conn(0x7f9efc0cb6e0 legacy=0x7f9efc06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc15bca0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f9ef80019f0 con 0x7f9efc15b1b0 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 >> v1:172.21.15.179:6789/0 conn(0x7f9efc15b1b0 legacy=0x7f9efc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 0x7f9ee4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee4004870 0x7f9ee4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4005230 0x7f9ee400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc15b9c0 con 0x7f9ee4003a20 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee400c1d0 con 0x7f9ee4005230 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee400c390 con 0x7f9ee4004870 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 0x7f9ee4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f01189700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee4004870 0x7f9ee4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0218b700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4005230 0x7f9ee400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 -- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee4004870 msgr2=0x7f9ee4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:30.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee4004870 0x7f9ee4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 -- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4005230 msgr2=0x7f9ee400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4005230 0x7f9ee400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0218b700 1 -- 172.21.15.179:0/3847646195 reap_dead start 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 -- 172.21.15.179:0/3847646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee40035d0 con 0x7f9ee4003a20 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0198a700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 0x7f9ee4003e40 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b70 tx=0x7f9ee40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee430c6e2ec1a854 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00e6c0 con 0x7f9ee4003a20 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec00e820 con 0x7f9ee4003a20 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f00988700 1 -- 172.21.15.179:0/3847646195 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00eb90 con 0x7f9ee4003a20 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 msgr2=0x7f9ee4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 0x7f9ee4003e40 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b70 tx=0x7f9ee40058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- 172.21.15.179:0/3847646195 shutdown_connections 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 --2- 172.21.15.179:0/3847646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4003a20 0x7f9ee4003e40 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- 172.21.15.179:0/3847646195 >> 172.21.15.179:0/3847646195 conn(0x7f9efc0ce9d0 msgr2=0x7f9efc0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- 172.21.15.179:0/3847646195 shutdown_connections 2024-05-08T22:45:30.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.742+0000 7f9f0298c700 1 -- 172.21.15.179:0/3847646195 wait complete. 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 Processor -- start 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 -- start start 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4003a20 0x7f9efc143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 0x7f9efc14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc14b8b0 con 0x7f9efc14f950 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc14df30 con 0x7f9ee4003a20 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0298c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc143850 con 0x7f9efc14e3c0 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0218b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0218b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39212/0 (socket says 172.21.15.179:39212) 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0218b700 1 -- 172.21.15.179:0/1189298139 learned_addr learned my addr 172.21.15.179:0/1189298139 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0198a700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4003a20 0x7f9efc143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f01189700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 0x7f9efc14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.746+0000 7f9f0218b700 1 -- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 msgr2=0x7f9efc14f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0218b700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 0x7f9efc14f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0218b700 1 -- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4003a20 msgr2=0x7f9efc143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0218b700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4003a20 0x7f9efc143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0218b700 1 -- 172.21.15.179:0/1189298139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc14d320 con 0x7f9efc14f950 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f01189700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 0x7f9efc14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0218b700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ccc20 tx=0x7f9ef400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40050d0 con 0x7f9efc14f950 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4005250 con 0x7f9efc14f950 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40036a0 con 0x7f9efc14f950 2024-05-08T22:45:30.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc14d600 con 0x7f9efc14f950 2024-05-08T22:45:30.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc14dbc0 con 0x7f9efc14f950 2024-05-08T22:45:30.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc14e0e0 con 0x7f9efc14f950 2024-05-08T22:45:30.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.750+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ef4003800 con 0x7f9efc14f950 2024-05-08T22:45:30.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.754+0000 7f9ef27fc700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 0x7f9ee8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:30.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.754+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9ef4083870 con 0x7f9efc14f950 2024-05-08T22:45:30.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.754+0000 7f9f0198a700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 0x7f9ee8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:30.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.754+0000 7f9f0198a700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 0x7f9ee8068c20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0090b0 tx=0x7f9eec01e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:30.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.758+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9efc14e0e0 con 0x7f9efc14f950 2024-05-08T22:45:30.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.938+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9efc067740 con 0x7f9efc14f950 2024-05-08T22:45:30.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.938+0000 7f9ef27fc700 1 -- 172.21.15.179:0/1189298139 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+13045 (secure 0 0 0) 0x7f9ef4050d80 con 0x7f9efc14f950 2024-05-08T22:45:30.941 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:30.941 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":132,"fsid":"88a906d6-7c22-496e-b9c7-b34c51a47fa8","created":"2024-05-08T22:32:58.057284+0000","modified":"2024-05-08T22:45:28.076708+0000","last_up_change":"2024-05-08T22:33:08.451509+0000","last_in_change":"2024-05-08T22:33:00.115678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:33:09.439140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93a60be3-29ef-4198-a764-e318fbff88fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":26067},{"type":"v1","addr":"172.21.15.179:6801","nonce":26067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":26067},{"type":"v1","addr":"172.21.15.179:6803","nonce":26067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":26067},{"type":"v1","addr":"172.21.15.179:6807","nonce":26067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":26067},{"type":"v1","addr":"172.21.15.179:6805","nonce":26067}]},"public_addr":"172.21.15.179:6801/26067","cluster_addr":"172.21.15.179:6803/26067","heartbeat_back_addr":"172.21.15.179:6807/26067","heartbeat_front_addr":"172.21.15.179:6805/26067","state":["exists","up"]},{"osd":1,"uuid":"f20a6991-00e2-4a3f-9d0f-15db51c8e53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":26069},{"type":"v1","addr":"172.21.15.179:6809","nonce":26069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":26069},{"type":"v1","addr":"172.21.15.179:6811","nonce":26069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":26069},{"type":"v1","addr":"172.21.15.179:6815","nonce":26069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":26069},{"type":"v1","addr":"172.21.15.179:6813","nonce":26069}]},"public_addr":"172.21.15.179:6809/26069","cluster_addr":"172.21.15.179:6811/26069","heartbeat_back_addr":"172.21.15.179:6815/26069","heartbeat_front_addr":"172.21.15.179:6813/26069","state":["exists","up"]},{"osd":2,"uuid":"a137b814-8dec-4873-b5f0-eb2a7ed72087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":26078},{"type":"v1","addr":"172.21.15.179:6825","nonce":26078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":26078},{"type":"v1","addr":"172.21.15.179:6827","nonce":26078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":26078},{"type":"v1","addr":"172.21.15.179:6831","nonce":26078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":26078},{"type":"v1","addr":"172.21.15.179:6829","nonce":26078}]},"public_addr":"172.21.15.179:6825/26078","cluster_addr":"172.21.15.179:6827/26078","heartbeat_back_addr":"172.21.15.179:6831/26078","heartbeat_front_addr":"172.21.15.179:6829/26078","state":["exists","up"]},{"osd":3,"uuid":"1ff8732a-d46d-4a7b-8c83-72a3604c7d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":26068},{"type":"v1","addr":"172.21.15.179:6817","nonce":26068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":26068},{"type":"v1","addr":"172.21.15.179:6819","nonce":26068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":26068},{"type":"v1","addr":"172.21.15.179:6823","nonce":26068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":26068},{"type":"v1","addr":"172.21.15.179:6821","nonce":26068}]},"public_addr":"172.21.15.179:6817/26068","cluster_addr":"172.21.15.179:6819/26068","heartbeat_back_addr":"172.21.15.179:6823/26068","heartbeat_front_addr":"172.21.15.179:6821/26068","state":["exists","up"]},{"osd":4,"uuid":"b3e3bccc-a4dc-45c5-bed7-704ec27b247b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":26287},{"type":"v1","addr":"172.21.15.195:6801","nonce":26287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":26287},{"type":"v1","addr":"172.21.15.195:6804","nonce":26287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":26287},{"type":"v1","addr":"172.21.15.195:6811","nonce":26287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":26287},{"type":"v1","addr":"172.21.15.195:6807","nonce":26287}]},"public_addr":"172.21.15.195:6801/26287","cluster_addr":"172.21.15.195:6804/26287","heartbeat_back_addr":"172.21.15.195:6811/26287","heartbeat_front_addr":"172.21.15.195:6807/26287","state":["exists","up"]},{"osd":5,"uuid":"5dcd29a4-3561-422e-852c-c59df7967e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":26288},{"type":"v1","addr":"172.21.15.195:6805","nonce":26288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":26288},{"type":"v1","addr":"172.21.15.195:6810","nonce":26288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":26288},{"type":"v1","addr":"172.21.15.195:6815","nonce":26288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":26288},{"type":"v1","addr":"172.21.15.195:6813","nonce":26288}]},"public_addr":"172.21.15.195:6805/26288","cluster_addr":"172.21.15.195:6810/26288","heartbeat_back_addr":"172.21.15.195:6815/26288","heartbeat_front_addr":"172.21.15.195:6813/26288","state":["exists","up"]},{"osd":6,"uuid":"87009f83-8317-498e-bc33-0cf930ae5b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":26291},{"type":"v1","addr":"172.21.15.195:6825","nonce":26291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":26291},{"type":"v1","addr":"172.21.15.195:6827","nonce":26291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":26291},{"type":"v1","addr":"172.21.15.195:6831","nonce":26291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":26291},{"type":"v1","addr":"172.21.15.195:6829","nonce":26291}]},"public_addr":"172.21.15.195:6825/26291","cluster_addr":"172.21.15.195:6827/26291","heartbeat_back_addr":"172.21.15.195:6831/26291","heartbeat_front_addr":"172.21.15.195:6829/26291","state":["exists","up"]},{"osd":7,"uuid":"35ef7868-ddcf-4c3b-98e8-e018bd51d06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":26289},{"type":"v1","addr":"172.21.15.195:6817","nonce":26289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":26289},{"type":"v1","addr":"172.21.15.195:6819","nonce":26289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":26289},{"type":"v1","addr":"172.21.15.195:6823","nonce":26289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":26289},{"type":"v1","addr":"172.21.15.195:6821","nonce":26289}]},"public_addr":"172.21.15.195:6817/26289","cluster_addr":"172.21.15.195:6819/26289","heartbeat_back_addr":"172.21.15.195:6823/26289","heartbeat_front_addr":"172.21.15.195:6821/26289","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.004666+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.604149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.023135+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.490679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.363988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.262095+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:05.576592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:04.950730+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6835/357696906":"2024-05-09T22:45:25.054147+0000","172.21.15.195:6835/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6834/3161101014":"2024-05-09T22:45:08.266131+0000","172.21.15.195:6835/2184096099":"2024-05-09T22:44:47.759671+0000","172.21.15.195:6834/357696906":"2024-05-09T22:45:25.054147+0000","172.21.15.195:6834/2184096099":"2024-05-09T22:44:47.759671+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:45:30.943 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 msgr2=0x7f9ee8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 0x7f9ee8068c20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0090b0 tx=0x7f9eec01e040 comp rx=0 tx=0).stop 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 msgr2=0x7f9efc0ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ccc20 tx=0x7f9ef400f820 comp rx=0 tx=0).stop 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 shutdown_connections 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ee8066760 0x7f9ee8068c20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9efc14f950 0x7f9efc0ab8e0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9efc14e3c0 0x7f9efc14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 --2- 172.21.15.179:0/1189298139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4003a20 0x7f9efc143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:30.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 >> 172.21.15.179:0/1189298139 conn(0x7f9efc0ce9d0 msgr2=0x7f9efc0ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:30.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 shutdown_connections 2024-05-08T22:45:30.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:30.942+0000 7f9f0298c700 1 -- 172.21.15.179:0/1189298139 wait complete. 2024-05-08T22:45:31.957 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T22:45:31.957 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:32.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.082+0000 7fd59d217700 1 Processor -- start 2024-05-08T22:45:32.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 -- start start 2024-05-08T22:45:32.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 0x7fd5980d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 0x7fd598160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59806c580 con 0x7fd598161220 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd598161650 con 0x7fd598160670 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59d217700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd598160a50 con 0x7fd5980d2200 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59ca16700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd598161220 0x7fd5980cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50968/0 (socket says 172.21.15.179:50968) 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd59ca16700 1 -- 172.21.15.179:0/3918596177 learned_addr learned my addr 172.21.15.179:0/3918596177 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd597fff700 1 --2- 172.21.15.179:0/3918596177 >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 0x7fd5980d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/3918596177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 0x7fd598160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd5977fe700 1 -- 172.21.15.179:0/3918596177 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068757526 0 0) 0x7fd59806c580 con 0x7fd598161220 2024-05-08T22:45:32.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd5977fe700 1 -- 172.21.15.179:0/3918596177 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5840035d0 con 0x7fd598161220 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 -- 172.21.15.179:0/3918596177 >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 msgr2=0x7fd5980d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/3918596177 >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 0x7fd5980d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 -- 172.21.15.179:0/3918596177 >> v1:172.21.15.179:6789/0 conn(0x7fd598161220 legacy=0x7fd5980cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 -- 172.21.15.179:0/3918596177 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980d26a0 con 0x7fd5980d2200 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd597fff700 1 --2- 172.21.15.179:0/3918596177 >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 0x7fd5980d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/3918596177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 0x7fd598160ce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd580009850 tx=0x7fd580009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d275afe70f2e566 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd5977fe700 1 -- 172.21.15.179:0/3918596177 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580011830 con 0x7fd5980d2200 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.086+0000 7fd5977fe700 1 -- 172.21.15.179:0/3918596177 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd580011990 con 0x7fd5980d2200 2024-05-08T22:45:32.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd5977fe700 1 -- 172.21.15.179:0/3918596177 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580011d40 con 0x7fd5980d2200 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 -- 172.21.15.179:0/3918596177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 msgr2=0x7fd598160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 --2- 172.21.15.179:0/3918596177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 0x7fd598160ce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd580009850 tx=0x7fd580009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 -- 172.21.15.179:0/3918596177 shutdown_connections 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 --2- 172.21.15.179:0/3918596177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd5980d2200 0x7fd598160ce0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 --2- 172.21.15.179:0/3918596177 >> v2:172.21.15.195:3300/0 conn(0x7fd598160670 0x7fd5980d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 -- 172.21.15.179:0/3918596177 >> 172.21.15.179:0/3918596177 conn(0x7fd5980d2e70 msgr2=0x7fd5980d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 -- 172.21.15.179:0/3918596177 shutdown_connections 2024-05-08T22:45:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.090+0000 7fd59d217700 1 -- 172.21.15.179:0/3918596177 wait complete. 2024-05-08T22:45:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 Processor -- start 2024-05-08T22:45:32.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 -- start start 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 0x7fd5980c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd598160670 0x7fd59815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd598161220 0x7fd59815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815bc40 con 0x7fd5980d2200 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815a0e0 con 0x7fd598160670 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59d217700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815afc0 con 0x7fd598161220 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 0x7fd5980c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 0x7fd5980c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39226/0 (socket says 172.21.15.179:39226) 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 -- 172.21.15.179:0/334741100 learned_addr learned my addr 172.21.15.179:0/334741100 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:32.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd598160670 0x7fd59815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59ca16700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd598161220 0x7fd59815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 -- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd598161220 msgr2=0x7fd59815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd598161220 0x7fd59815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 -- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd598160670 msgr2=0x7fd59815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd598160670 0x7fd59815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880094b0 con 0x7fd5980d2200 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd59ca16700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd598161220 0x7fd59815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:32.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.094+0000 7fd597fff700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 0x7fd5980c6350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fd58800ebe0 tx=0x7fd58800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800c890 con 0x7fd5980d2200 2024-05-08T22:45:32.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800c9f0 con 0x7fd5980d2200 2024-05-08T22:45:32.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800094b0 con 0x7fd5980d2200 2024-05-08T22:45:32.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800cd80 con 0x7fd5980d2200 2024-05-08T22:45:32.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59815b580 con 0x7fd5980d2200 2024-05-08T22:45:32.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.098+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd59806b050 con 0x7fd5980d2200 2024-05-08T22:45:32.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd588013470 con 0x7fd5980d2200 2024-05-08T22:45:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd5957fa700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd578066730 0x7fd578068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd588084260 con 0x7fd5980d2200 2024-05-08T22:45:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd5957fa700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fd57806cb70 0x7fd57806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd57806f680 con 0x7fd57806cb70 2024-05-08T22:45:32.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd578066730 0x7fd578068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.102+0000 7fd58f7fe700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd578066730 0x7fd578068bf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd5800090b0 tx=0x7fd580008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.106+0000 7fd59ca16700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fd57806cb70 0x7fd57806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.106+0000 7fd59ca16700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fd57806cb70 0x7fd57806efb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.106+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd58800cee0 con 0x7fd5980d2200 2024-05-08T22:45:32.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.106+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== osd.0 v2:172.21.15.179:6800/26067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd57806f680 con 0x7fd57806cb70 2024-05-08T22:45:32.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.122+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 --> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd59806ad70 con 0x7fd57806cb70 2024-05-08T22:45:32.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.122+0000 7fd5957fa700 1 -- 172.21.15.179:0/334741100 <== osd.0 v2:172.21.15.179:6800/26067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd5900070e0 con 0x7fd57806cb70 2024-05-08T22:45:32.127 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:32.127 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:32.127 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fd57806cb70 msgr2=0x7fd57806efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:6800/26067,v1:172.21.15.179:6801/26067] conn(0x7fd57806cb70 0x7fd57806efb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd578066730 msgr2=0x7fd578068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fd578066730 0x7fd578068bf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd5800090b0 tx=0x7fd580008030 comp rx=0 tx=0).stop 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 msgr2=0x7fd5980c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 --2- 172.21.15.179:0/334741100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5980d2200 0x7fd5980c6350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fd58800ebe0 tx=0x7fd58800eef0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59ca16700 1 -- 172.21.15.179:0/334741100 reap_dead start 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 shutdown_connections 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 >> 172.21.15.179:0/334741100 conn(0x7fd5980d2e70 msgr2=0x7fd5980d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 shutdown_connections 2024-05-08T22:45:32.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.126+0000 7fd59d217700 1 -- 172.21.15.179:0/334741100 wait complete. 2024-05-08T22:45:32.145 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T22:45:32.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 Processor -- start 2024-05-08T22:45:32.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 -- start start 2024-05-08T22:45:32.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76f0150bc0 0x7f76f015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00adb90 0x7f76f015df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f00b1cb0 con 0x7f76f00ad740 2024-05-08T22:45:32.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00b2930 con 0x7f76f0150bc0 2024-05-08T22:45:32.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ef59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00b1f90 con 0x7f76f00adb90 2024-05-08T22:45:32.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76f0150bc0 0x7f76f015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76e7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f76f00ad740 0x7f76f015d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50970/0 (socket says 172.21.15.179:50970) 2024-05-08T22:45:32.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76e7fff700 1 -- 172.21.15.179:0/3443162117 learned_addr learned my addr 172.21.15.179:0/3443162117 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76eed9d700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00adb90 0x7f76f015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 -- 172.21.15.179:0/3443162117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00adb90 msgr2=0x7f76f015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00adb90 0x7f76f015df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 -- 172.21.15.179:0/3443162117 >> v1:172.21.15.179:6789/0 conn(0x7f76f00ad740 legacy=0x7f76f015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76edd9b700 1 -- 172.21.15.179:0/3443162117 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813046841 0 0) 0x7f76f00b1cb0 con 0x7f76f00ad740 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 -- 172.21.15.179:0/3443162117 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00b2270 con 0x7f76f0150bc0 2024-05-08T22:45:32.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76ee59c700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f0150bc0 0x7f76f015ac70 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009880 tx=0x7f76e0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b3fcba2c73e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76edd9b700 1 -- 172.21.15.179:0/3443162117 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011830 con 0x7f76f0150bc0 2024-05-08T22:45:32.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76edd9b700 1 -- 172.21.15.179:0/3443162117 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0011990 con 0x7f76f0150bc0 2024-05-08T22:45:32.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.262+0000 7f76edd9b700 1 -- 172.21.15.179:0/3443162117 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011d20 con 0x7f76f0150bc0 2024-05-08T22:45:32.267 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- 172.21.15.179:0/3443162117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f0150bc0 msgr2=0x7f76f015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f0150bc0 0x7f76f015ac70 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009880 tx=0x7f76e0009b90 comp rx=0 tx=0).stop 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- 172.21.15.179:0/3443162117 shutdown_connections 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00adb90 0x7f76f015df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- 172.21.15.179:0/3443162117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f0150bc0 0x7f76f015ac70 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- 172.21.15.179:0/3443162117 >> 172.21.15.179:0/3443162117 conn(0x7f76f00ce960 msgr2=0x7f76f00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- 172.21.15.179:0/3443162117 shutdown_connections 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- 172.21.15.179:0/3443162117 wait complete. 2024-05-08T22:45:32.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 Processor -- start 2024-05-08T22:45:32.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- start start 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00ad740 0x7f76f01444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76f0150bc0 0x7f76f0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0152630 con 0x7f76f00adb90 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0152ed0 con 0x7f76f0150bc0 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ef59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00a87a0 con 0x7f76f00ad740 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39252/0 (socket says 172.21.15.179:39252) 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 -- 172.21.15.179:0/1494289006 learned_addr learned my addr 172.21.15.179:0/1494289006 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76ee59c700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00ad740 0x7f76f01444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76eed9d700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76f0150bc0 0x7f76f0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 -- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00ad740 msgr2=0x7f76f01444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00ad740 0x7f76f01444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 -- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76f0150bc0 msgr2=0x7f76f0158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76f0150bc0 0x7f76f0158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.266+0000 7f76e7fff700 1 -- 172.21.15.179:0/1494289006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00094b0 con 0x7f76f00adb90 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e7fff700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c5b0 tx=0x7f76d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8004ca0 con 0x7f76f00adb90 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8004e20 con 0x7f76f00adb90 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f0152880 con 0x7f76f00adb90 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8007310 con 0x7f76f00adb90 2024-05-08T22:45:32.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0144e30 con 0x7f76f00adb90 2024-05-08T22:45:32.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f004e700 con 0x7f76f00adb90 2024-05-08T22:45:32.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76d8007470 con 0x7f76f00adb90 2024-05-08T22:45:32.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 0x7f76d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.274 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.270+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f76d8083d70 con 0x7f76f00adb90 2024-05-08T22:45:32.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.274+0000 7f76ee59c700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 0x7f76d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.275 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.274+0000 7f76ee59c700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 0x7f76d0068bf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f76e00090b0 tx=0x7f76e001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.278+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76d8051150 con 0x7f76f00adb90 2024-05-08T22:45:32.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.446+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f76f00adfe0 con 0x7f76d0066730 2024-05-08T22:45:32.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.446+0000 7f76e6ffd700 1 -- 172.21.15.179:0/1494289006 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f76e0011c00 con 0x7f76d0066730 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 msgr2=0x7f76d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 0x7f76d0068bf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f76e00090b0 tx=0x7f76e001c040 comp rx=0 tx=0).stop 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 msgr2=0x7f76f00a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c5b0 tx=0x7f76d800c8c0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 shutdown_connections 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f76d0066730 0x7f76d0068bf0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76f0150bc0 0x7f76f0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76f00adb90 0x7f76f00a7d50 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 --2- 172.21.15.179:0/1494289006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f00ad740 0x7f76f01444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 >> 172.21.15.179:0/1494289006 conn(0x7f76f00ce960 msgr2=0x7f76f015a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 shutdown_connections 2024-05-08T22:45:32.454 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.450+0000 7f76ef59e700 1 -- 172.21.15.179:0/1494289006 wait complete. 2024-05-08T22:45:32.455 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T22:45:32.470 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T22:45:32.470 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:32.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 Processor -- start 2024-05-08T22:45:32.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 -- start start 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdb0c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb0c06e760 0x7fdb0c0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c0703c0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c070980 con 0x7fdb0c06e760 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b3ea0 con 0x7fdb0c15ae10 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0b7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fdb0c0b4800 0x7fdb0c15ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50974/0 (socket says 172.21.15.179:50974) 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.594+0000 7fdb0b7fe700 1 -- 172.21.15.179:0/1230436030 learned_addr learned my addr 172.21.15.179:0/1230436030 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:32.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0a7fc700 1 --2- 172.21.15.179:0/1230436030 >> v2:172.21.15.195:3300/0 conn(0x7fdb0c06e760 0x7fdb0c0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdb0c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607036547 0 0) 0x7fdb0c0703c0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf80035d0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2384004314 0 0) 0x7fdaf80035d0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 msgr2=0x7fdb0c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdb0c0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 >> v2:172.21.15.195:3300/0 conn(0x7fdb0c06e760 msgr2=0x7fdb0c0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 --2- 172.21.15.179:0/1230436030 >> v2:172.21.15.195:3300/0 conn(0x7fdb0c06e760 0x7fdb0c0b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0706a0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7fdb040019f0 con 0x7fdb0c0b4800 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 >> v1:172.21.15.179:6789/0 conn(0x7fdb0c0b4800 legacy=0x7fdb0c15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaf8003a20 0x7fdaf8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaf8004870 0x7fdaf8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdaf8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c070980 con 0x7fdaf8003a20 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b3ea0 con 0x7fdaf8004870 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf800ba30 con 0x7fdb0c15ae10 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaf8003a20 0x7fdaf8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0a7fc700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaf8004870 0x7fdaf8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0b7fe700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdaf8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 -- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 msgr2=0x7fdaf8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0c15ae10 0x7fdaf8005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 -- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaf8004870 msgr2=0x7fdaf8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaf8004870 0x7fdaf8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 -- 172.21.15.179:0/1230436030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf80035d0 con 0x7fdaf8003a20 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0affd700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaf8003a20 0x7fdaf8003e40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009ba0 tx=0x7fdaf80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31efa2f20fff5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00e6c0 con 0x7fdaf8003a20 2024-05-08T22:45:32.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc00e820 con 0x7fdaf8003a20 2024-05-08T22:45:32.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb09ffb700 1 -- 172.21.15.179:0/1230436030 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00eb90 con 0x7fdaf8003a20 2024-05-08T22:45:32.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 -- 172.21.15.179:0/1230436030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaf8003a20 msgr2=0x7fdaf8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 --2- 172.21.15.179:0/1230436030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaf8003a20 0x7fdaf8003e40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009ba0 tx=0x7fdaf80058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0b7fe700 1 -- 172.21.15.179:0/1230436030 reap_dead start 2024-05-08T22:45:32.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 -- 172.21.15.179:0/1230436030 shutdown_connections 2024-05-08T22:45:32.604 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 -- 172.21.15.179:0/1230436030 >> 172.21.15.179:0/1230436030 conn(0x7fdb0c06f690 msgr2=0x7fdb0c06faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 -- 172.21.15.179:0/1230436030 shutdown_connections 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.598+0000 7fdb0bfff700 1 -- 172.21.15.179:0/1230436030 wait complete. 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 Processor -- start 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- start start 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaf8003a20 0x7fdb0c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 0x7fdb0c14d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb0c0c8e70 0x7fdb0c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1528c0 con 0x7fdb0c15d0c0 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c151ef0 con 0x7fdb0c0c8e70 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c15d910 con 0x7fdaf8003a20 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 0x7fdb0c14d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 0x7fdb0c14d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39264/0 (socket says 172.21.15.179:39264) 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0a7fc700 1 -- 172.21.15.179:0/4117145471 learned_addr learned my addr 172.21.15.179:0/4117145471 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:32.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaf8003a20 0x7fdb0c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0b7fe700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb0c0c8e70 0x7fdb0c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 -- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb0c0c8e70 msgr2=0x7fdb0c0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb0c0c8e70 0x7fdb0c0c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 -- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 msgr2=0x7fdb0c14d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 0x7fdb0c14d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c1515c0 con 0x7fdaf8003a20 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0a7fc700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb0c15d0c0 0x7fdb0c14d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0affd700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaf8003a20 0x7fdb0c152380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdafc019000 tx=0x7fdafc0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00e6c0 con 0x7fdaf8003a20 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc013d90 con 0x7fdaf8003a20 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c1518a0 con 0x7fdaf8003a20 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004270 con 0x7fdaf8003a20 2024-05-08T22:45:32.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.602+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c14dca0 con 0x7fdaf8003a20 2024-05-08T22:45:32.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb0c0cf160 con 0x7fdaf8003a20 2024-05-08T22:45:32.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdafc0043d0 con 0x7fdaf8003a20 2024-05-08T22:45:32.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaec066730 0x7fdaec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fdafc085980 con 0x7fdaf8003a20 2024-05-08T22:45:32.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb0a7fc700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaec066730 0x7fdaec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7fdaec06cb70 0x7fdaec06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdaec06f680 con 0x7fdaec06cb70 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdafc08a2c0 con 0x7fdaf8003a20 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb0b7fe700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7fdaec06cb70 0x7fdaec06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb0b7fe700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7fdaec06cb70 0x7fdaec06efb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.606+0000 7fdb0a7fc700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaec066730 0x7fdaec068bf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c159b60 tx=0x7fdaf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.610+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== osd.1 v2:172.21.15.179:6808/26069 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fdaec06f680 con 0x7fdaec06cb70 2024-05-08T22:45:32.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 --> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdb0c152030 con 0x7fdaec06cb70 2024-05-08T22:45:32.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb037fe700 1 -- 172.21.15.179:0/4117145471 <== osd.1 v2:172.21.15.179:6808/26069 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdb0400d040 con 0x7fdaec06cb70 2024-05-08T22:45:32.629 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:32.629 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:32.629 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7fdaec06cb70 msgr2=0x7fdaec06efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.179:6808/26069,v1:172.21.15.179:6809/26069] conn(0x7fdaec06cb70 0x7fdaec06efb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaec066730 msgr2=0x7fdaec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fdaec066730 0x7fdaec068bf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c159b60 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaf8003a20 msgr2=0x7fdb0c152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 --2- 172.21.15.179:0/4117145471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdaf8003a20 0x7fdb0c152380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdafc019000 tx=0x7fdafc0047e0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0b7fe700 1 -- 172.21.15.179:0/4117145471 reap_dead start 2024-05-08T22:45:32.630 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 shutdown_connections 2024-05-08T22:45:32.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 >> 172.21.15.179:0/4117145471 conn(0x7fdb0c06f690 msgr2=0x7fdb0c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 shutdown_connections 2024-05-08T22:45:32.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.626+0000 7fdb0bfff700 1 -- 172.21.15.179:0/4117145471 wait complete. 2024-05-08T22:45:32.648 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T22:45:32.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 Processor -- start 2024-05-08T22:45:32.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 -- start start 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 0x7f4c9c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c9c06e180 0x7f4c9c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c06b540 con 0x7f4c9c0b25d0 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c06ac50 con 0x7f4c9c06e180 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c06aef0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9ad9d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4c9c0b25d0 0x7f4c9c0b2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50986/0 (socket says 172.21.15.179:50986) 2024-05-08T22:45:32.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9ad9d700 1 -- 172.21.15.179:0/3765856783 learned_addr learned my addr 172.21.15.179:0/3765856783 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:32.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/3765856783 >> v2:172.21.15.195:3300/0 conn(0x7f4c9c06e180 0x7f4c9c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9a59c700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 0x7f4c9c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9959a700 1 -- 172.21.15.179:0/3765856783 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084916376 0 0) 0x7f4c9c06b540 con 0x7f4c9c0b25d0 2024-05-08T22:45:32.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9959a700 1 -- 172.21.15.179:0/3765856783 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c840035d0 con 0x7f4c9c0b25d0 2024-05-08T22:45:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 -- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 msgr2=0x7f4c9c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 0x7f4c9c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 -- 172.21.15.179:0/3765856783 >> v1:172.21.15.179:6789/0 conn(0x7f4c9c0b25d0 legacy=0x7f4c9c0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 -- 172.21.15.179:0/3765856783 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c06b1d0 con 0x7f4c9c06e180 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c9a59c700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 0x7f4c9c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.762+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c06e180 0x7f4c9c06e5e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009850 tx=0x7f4c90009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22dba2d377cff2b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9959a700 1 -- 172.21.15.179:0/3765856783 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90011830 con 0x7f4c9c06e180 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9959a700 1 -- 172.21.15.179:0/3765856783 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90011990 con 0x7f4c9c06e180 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9959a700 1 -- 172.21.15.179:0/3765856783 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90011d40 con 0x7f4c9c06e180 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c06e180 msgr2=0x7f4c9c06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c06e180 0x7f4c9c06e5e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009850 tx=0x7f4c90009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- 172.21.15.179:0/3765856783 shutdown_connections 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c06e180 0x7f4c9c06e5e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/3765856783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06dda0 0x7f4c9c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- 172.21.15.179:0/3765856783 >> 172.21.15.179:0/3765856783 conn(0x7f4c9c0ce960 msgr2=0x7f4c9c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- 172.21.15.179:0/3765856783 shutdown_connections 2024-05-08T22:45:32.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- 172.21.15.179:0/3765856783 wait complete. 2024-05-08T22:45:32.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 Processor -- start 2024-05-08T22:45:32.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 -- start start 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.766+0000 7f4c9b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 0x7f4c9c149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 0x7f4c9c163290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0c38e0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cb250 con 0x7f4c9c0b25d0 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0c3b30 con 0x7f4c9c06e180 2024-05-08T22:45:32.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39278/0 (socket says 172.21.15.179:39278) 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 -- 172.21.15.179:0/665821947 learned_addr learned my addr 172.21.15.179:0/665821947 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9ad9d700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 0x7f4c9c163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 0x7f4c9c149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 -- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 msgr2=0x7f4c9c149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 0x7f4c9c149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 -- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 msgr2=0x7f4c9c163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 0x7f4c9c163290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 -- 172.21.15.179:0/665821947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0094b0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 0x7f4c9c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9ad9d700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 0x7f4c9c163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9a59c700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00ec10 tx=0x7f4c8c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00c940 con 0x7f4c9c06dda0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00caa0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900094b0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00ce30 con 0x7f4c9c06dda0 2024-05-08T22:45:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c0c40f0 con 0x7f4c9c06dda0 2024-05-08T22:45:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c04e700 con 0x7f4c9c06dda0 2024-05-08T22:45:32.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.770+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c8c013470 con 0x7f4c9c06dda0 2024-05-08T22:45:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.774+0000 7f4c8affd700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 0x7f4c7c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.774+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f4c8c084220 con 0x7f4c9c06dda0 2024-05-08T22:45:32.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.774+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 0x7f4c7c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:32.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.774+0000 7f4c99d9b700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 0x7f4c7c068bf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c164230 tx=0x7f4c90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:32.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.778+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4c8c051730 con 0x7f4c9c06dda0 2024-05-08T22:45:32.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.946+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c9c0cc170 con 0x7f4c7c066730 2024-05-08T22:45:32.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.946+0000 7f4c8affd700 1 -- 172.21.15.179:0/665821947 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4c90011b00 con 0x7f4c7c066730 2024-05-08T22:45:32.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 msgr2=0x7f4c7c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 0x7f4c7c068bf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c164230 tx=0x7f4c90008030 comp rx=0 tx=0).stop 2024-05-08T22:45:32.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 msgr2=0x7f4c9c0cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00ec10 tx=0x7f4c8c009ed0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 shutdown_connections 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f4c7c066730 0x7f4c7c068bf0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c9c0b25d0 0x7f4c9c163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c9c06e180 0x7f4c9c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 --2- 172.21.15.179:0/665821947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c9c06dda0 0x7f4c9c0cad10 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 >> 172.21.15.179:0/665821947 conn(0x7f4c9c0ce960 msgr2=0x7f4c9c15c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 shutdown_connections 2024-05-08T22:45:32.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:32.950+0000 7f4c9b59e700 1 -- 172.21.15.179:0/665821947 wait complete. 2024-05-08T22:45:32.954 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T22:45:32.965 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T22:45:32.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:33.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.082+0000 7fb5ef14a700 1 Processor -- start 2024-05-08T22:45:33.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 -- start start 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5e003ab10 0x7fb5e003af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e003c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e0057d50 con 0x7fb5e00e3280 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e003b4c0 con 0x7fb5e003ab10 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e0058030 con 0x7fb5e003c370 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5e003ab10 0x7fb5e003af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5e003ab10 0x7fb5e003af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39286/0 (socket says 172.21.15.179:39286) 2024-05-08T22:45:33.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 -- 172.21.15.179:0/1864222755 learned_addr learned my addr 172.21.15.179:0/1864222755 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ee949700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e003c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.089 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 -- 172.21.15.179:0/1864222755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 msgr2=0x7fb5e003c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e003c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 -- 172.21.15.179:0/1864222755 >> v1:172.21.15.195:6789/0 conn(0x7fb5e00e3280 legacy=0x7fb5e00572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 -- 172.21.15.179:0/1864222755 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0058310 con 0x7fb5e003ab10 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed947700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e003ab10 0x7fb5e003af80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009910 tx=0x7fb5e4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da89351d61ef25f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed146700 1 -- 172.21.15.179:0/1864222755 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400b850 con 0x7fb5e003ab10 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed146700 1 -- 172.21.15.179:0/1864222755 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e400b9b0 con 0x7fb5e003ab10 2024-05-08T22:45:33.090 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ed146700 1 -- 172.21.15.179:0/1864222755 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400bd40 con 0x7fb5e003ab10 2024-05-08T22:45:33.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 -- 172.21.15.179:0/1864222755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e003ab10 msgr2=0x7fb5e003af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.086+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e003ab10 0x7fb5e003af80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009910 tx=0x7fb5e4009c20 comp rx=0 tx=0).stop 2024-05-08T22:45:33.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- 172.21.15.179:0/1864222755 shutdown_connections 2024-05-08T22:45:33.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e003c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/1864222755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e003ab10 0x7fb5e003af80 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- 172.21.15.179:0/1864222755 >> 172.21.15.179:0/1864222755 conn(0x7fb5e0055b10 msgr2=0x7fb5e0055f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- 172.21.15.179:0/1864222755 shutdown_connections 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- 172.21.15.179:0/1864222755 wait complete. 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 Processor -- start 2024-05-08T22:45:33.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- start start 2024-05-08T22:45:33.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e003ab10 0x7fb5e017c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e00e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 0x7fb5e004e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e004fab0 con 0x7fb5e00e3280 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e004f7d0 con 0x7fb5e003ab10 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.090+0000 7fb5ef14a700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00e37e0 con 0x7fb5e003c370 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 0x7fb5e004e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee148700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e003ab10 0x7fb5e017c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 0x7fb5e004e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39290/0 (socket says 172.21.15.179:39290) 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 -- 172.21.15.179:0/2508843998 learned_addr learned my addr 172.21.15.179:0/2508843998 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ed947700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e00e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 -- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 msgr2=0x7fb5e00e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5e003c370 0x7fb5e00e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 -- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e003ab10 msgr2=0x7fb5e017c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5e003ab10 0x7fb5e017c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e4009570 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ee949700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 0x7fb5e004e940 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802ce60 tx=0x7fb5e8067530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e808e040 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8066be0 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e00e3ac0 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8066f50 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e004ee80 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5e00e3660 con 0x7fb5e00e3280 2024-05-08T22:45:33.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.094+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5e8066150 con 0x7fb5e00e3280 2024-05-08T22:45:33.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb5cc066760 0x7fb5cc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb5e80fd9a0 con 0x7fb5e00e3280 2024-05-08T22:45:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5ee148700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb5cc066760 0x7fb5cc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7fb5cc06cba0 0x7fb5cc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5cc06f6b0 con 0x7fb5cc06cba0 2024-05-08T22:45:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e81022c0 con 0x7fb5e00e3280 2024-05-08T22:45:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5ed947700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7fb5cc06cba0 0x7fb5cc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5ee148700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb5cc066760 0x7fb5cc068c20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8000f60 tx=0x7fb5d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5ed947700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7fb5cc06cba0 0x7fb5cc06efe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.098+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== osd.2 v2:172.21.15.179:6824/26078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb5cc06f6b0 con 0x7fb5cc06cba0 2024-05-08T22:45:33.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.114+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 --> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb5e00e3950 con 0x7fb5cc06cba0 2024-05-08T22:45:33.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5deffd700 1 -- 172.21.15.179:0/2508843998 <== osd.2 v2:172.21.15.179:6824/26078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb5e00e3950 con 0x7fb5cc06cba0 2024-05-08T22:45:33.119 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:33.119 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:33.119 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7fb5cc06cba0 msgr2=0x7fb5cc06efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:6824/26078,v1:172.21.15.179:6825/26078] conn(0x7fb5cc06cba0 0x7fb5cc06efe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb5cc066760 msgr2=0x7fb5cc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb5cc066760 0x7fb5cc068c20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8000f60 tx=0x7fb5d8009040 comp rx=0 tx=0).stop 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 msgr2=0x7fb5e004e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 --2- 172.21.15.179:0/2508843998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e00e3280 0x7fb5e004e940 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802ce60 tx=0x7fb5e8067530 comp rx=0 tx=0).stop 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ee949700 1 -- 172.21.15.179:0/2508843998 reap_dead start 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 shutdown_connections 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 >> 172.21.15.179:0/2508843998 conn(0x7fb5e0055b10 msgr2=0x7fb5e0053ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 shutdown_connections 2024-05-08T22:45:33.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.118+0000 7fb5ef14a700 1 -- 172.21.15.179:0/2508843998 wait complete. 2024-05-08T22:45:33.132 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T22:45:33.252 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.250+0000 7f9b0737b700 1 Processor -- start 2024-05-08T22:45:33.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.250+0000 7f9b0737b700 1 -- start start 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b0015e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 0x7f9b00160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b00161460 con 0x7f9b000cda50 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0015e690 con 0x7f9b0006fa20 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b000b2ac0 con 0x7f9b0006f640 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05b78700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 0x7f9b00160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b0015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06b7a700 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f9b000cda50 0x7f9b000cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.179:44154/0 (socket says 172.21.15.179:44154) 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06b7a700 1 -- 172.21.15.179:0/2979162434 learned_addr learned my addr 172.21.15.179:0/2979162434 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05377700 1 -- 172.21.15.179:0/2979162434 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526013598 0 0) 0x7f9b00161460 con 0x7f9b000cda50 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05377700 1 -- 172.21.15.179:0/2979162434 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aec0035d0 con 0x7f9b000cda50 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 -- 172.21.15.179:0/2979162434 >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 msgr2=0x7f9b00160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 --2- 172.21.15.179:0/2979162434 >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 0x7f9b00160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 -- 172.21.15.179:0/2979162434 >> v1:172.21.15.195:6789/0 conn(0x7f9b000cda50 legacy=0x7f9b000cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 -- 172.21.15.179:0/2979162434 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b00161740 con 0x7f9b0006f640 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05b78700 1 --2- 172.21.15.179:0/2979162434 >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 0x7f9b00160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b06379700 1 --2- 172.21.15.179:0/2979162434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b0015e150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009880 tx=0x7f9afc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664b8c6a3ab285d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05377700 1 -- 172.21.15.179:0/2979162434 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc011830 con 0x7f9b0006f640 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05377700 1 -- 172.21.15.179:0/2979162434 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc011990 con 0x7f9b0006f640 2024-05-08T22:45:33.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b05377700 1 -- 172.21.15.179:0/2979162434 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc011d00 con 0x7f9b0006f640 2024-05-08T22:45:33.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- 172.21.15.179:0/2979162434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 msgr2=0x7f9b0015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 --2- 172.21.15.179:0/2979162434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b0015e150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009880 tx=0x7f9afc009b90 comp rx=0 tx=0).stop 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- 172.21.15.179:0/2979162434 shutdown_connections 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 --2- 172.21.15.179:0/2979162434 >> v2:172.21.15.195:3300/0 conn(0x7f9b0006fa20 0x7f9b00160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 --2- 172.21.15.179:0/2979162434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b0015e150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- 172.21.15.179:0/2979162434 >> 172.21.15.179:0/2979162434 conn(0x7f9b000cfb50 msgr2=0x7f9b000cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- 172.21.15.179:0/2979162434 shutdown_connections 2024-05-08T22:45:33.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.254+0000 7f9b0737b700 1 -- 172.21.15.179:0/2979162434 wait complete. 2024-05-08T22:45:33.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 Processor -- start 2024-05-08T22:45:33.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 -- start start 2024-05-08T22:45:33.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b001429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b000cda50 0x7f9b000cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00158dd0 con 0x7f9b0006fa20 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00158250 con 0x7f9b000cda50 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b0737b700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00158530 con 0x7f9b0006f640 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39300/0 (socket says 172.21.15.179:39300) 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 -- 172.21.15.179:0/505592614 learned_addr learned my addr 172.21.15.179:0/505592614 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b06b7a700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b000cda50 0x7f9b000cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b06379700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b001429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 -- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 msgr2=0x7f9b001429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b001429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 -- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b000cda50 msgr2=0x7f9b000cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b000cda50 0x7f9b000cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.258+0000 7f9b05b78700 1 -- 172.21.15.179:0/505592614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00094b0 con 0x7f9b0006fa20 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b05b78700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f9af000ebe0 tx=0x7f9af0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000c8b0 con 0x7f9b0006fa20 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af000ca10 con 0x7f9b0006fa20 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc0094b0 con 0x7f9b0006fa20 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000cda0 con 0x7f9b0006fa20 2024-05-08T22:45:33.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b00158af0 con 0x7f9b0006fa20 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0004e700 con 0x7f9b0006fa20 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9af0013470 con 0x7f9b0006fa20 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 0x7f9ae4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9af0084200 con 0x7f9b0006fa20 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b06379700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 0x7f9ae4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.266 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.262+0000 7f9b06379700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 0x7f9ae4068bf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0090b0 tx=0x7f9afc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.271 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.270+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9af0051600 con 0x7f9b0006fa20 2024-05-08T22:45:33.442 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b001586a0 con 0x7f9ae4066730 2024-05-08T22:45:33.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9af6ffd700 1 -- 172.21.15.179:0/505592614 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9b001586a0 con 0x7f9ae4066730 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 msgr2=0x7f9ae4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 0x7f9ae4068bf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0090b0 tx=0x7f9afc008030 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 msgr2=0x7f9b000cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f9af000ebe0 tx=0x7f9af0009e30 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 shutdown_connections 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f9ae4066730 0x7f9ae4068bf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b000cda50 0x7f9b000cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b0006fa20 0x7f9b000cb3b0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 --2- 172.21.15.179:0/505592614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b0006f640 0x7f9b001429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 >> 172.21.15.179:0/505592614 conn(0x7f9b000cfb50 msgr2=0x7f9b000b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 shutdown_connections 2024-05-08T22:45:33.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.442+0000 7f9b0737b700 1 -- 172.21.15.179:0/505592614 wait complete. 2024-05-08T22:45:33.447 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T22:45:33.457 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T22:45:33.457 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:33.571 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 Processor -- start 2024-05-08T22:45:33.572 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 -- start start 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8d2c0b2b50 0x7f8d2c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d2c0ccd30 0x7f8d2c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c06ad50 con 0x7f8d2c1448c0 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0ce5d0 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d32654700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cdd30 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d2c0ccd30 0x7f8d2c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.574 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d2bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8d2c1448c0 0x7f8d2c0b2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:50996/0 (socket says 172.21.15.179:50996) 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d2bfff700 1 -- 172.21.15.179:0/3543886909 learned_addr learned my addr 172.21.15.179:0/3543886909 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d2b7fe700 1 --2- 172.21.15.179:0/3543886909 >> v2:172.21.15.195:3300/0 conn(0x7f8d2c0b2b50 0x7f8d2c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 -- 172.21.15.179:0/3543886909 >> v2:172.21.15.195:3300/0 conn(0x7f8d2c0b2b50 msgr2=0x7f8d2c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 --2- 172.21.15.179:0/3543886909 >> v2:172.21.15.195:3300/0 conn(0x7f8d2c0b2b50 0x7f8d2c0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 -- 172.21.15.179:0/3543886909 >> v1:172.21.15.179:6789/0 conn(0x7f8d2c1448c0 legacy=0x7f8d2c0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 -- 172.21.15.179:0/3543886909 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0ce010 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d2affd700 1 -- 172.21.15.179:0/3543886909 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380438030 0 0) 0x7f8d2c06ad50 con 0x7f8d2c1448c0 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d30bf1700 1 --2- 172.21.15.179:0/3543886909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0ccd30 0x7f8d2c06b230 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c06f480 tx=0x7f8d2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5315269a56df93fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.570+0000 7f8d2affd700 1 -- 172.21.15.179:0/3543886909 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e070 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d2affd700 1 -- 172.21.15.179:0/3543886909 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400bb20 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d2affd700 1 -- 172.21.15.179:0/3543886909 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24012430 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- 172.21.15.179:0/3543886909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0ccd30 msgr2=0x7f8d2c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 --2- 172.21.15.179:0/3543886909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0ccd30 0x7f8d2c06b230 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c06f480 tx=0x7f8d2400b330 comp rx=0 tx=0).stop 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- 172.21.15.179:0/3543886909 shutdown_connections 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 --2- 172.21.15.179:0/3543886909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0ccd30 0x7f8d2c06b230 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 --2- 172.21.15.179:0/3543886909 >> v2:172.21.15.195:3300/0 conn(0x7f8d2c0b2b50 0x7f8d2c0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- 172.21.15.179:0/3543886909 >> 172.21.15.179:0/3543886909 conn(0x7f8d2c0cb610 msgr2=0x7f8d2c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- 172.21.15.179:0/3543886909 shutdown_connections 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- 172.21.15.179:0/3543886909 wait complete. 2024-05-08T22:45:33.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 Processor -- start 2024-05-08T22:45:33.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.574+0000 7f8d32654700 1 -- start start 2024-05-08T22:45:33.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 0x7f8d2c12f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d2c0ccd30 0x7f8d2c1f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d2c1448c0 0x7f8d2c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c12f5f0 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0aa530 con 0x7f8d2c1448c0 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1f18e0 con 0x7f8d2c0ccd30 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 0x7f8d2c12f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 0x7f8d2c12f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39306/0 (socket says 172.21.15.179:39306) 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 -- 172.21.15.179:0/780879970 learned_addr learned my addr 172.21.15.179:0/780879970 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d30bf1700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d2c1448c0 0x7f8d2c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2b7fe700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d2c0ccd30 0x7f8d2c1f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 -- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d2c0ccd30 msgr2=0x7f8d2c1f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.581 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d2c0ccd30 0x7f8d2c1f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 -- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d2c1448c0 msgr2=0x7f8d2c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d2c1448c0 0x7f8d2c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009a70 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d2bfff700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 0x7f8d2c12f0b0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c510 tx=0x7f8d1c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c004ca0 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c004e20 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c131640 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007310 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c131c00 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d2c141c50 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d1c007470 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066730 0x7f8d0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.578+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8d1c083be0 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d28ff9700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7f8d0c06cb70 0x7f8d0c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d0c06f680 con 0x7f8d0c06cb70 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c0882c0 con 0x7f8d2c0b2b50 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d30bf1700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7f8d0c06cb70 0x7f8d0c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d2b7fe700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066730 0x7f8d0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d30bf1700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7f8d0c06cb70 0x7f8d0c06efb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d2b7fe700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066730 0x7f8d0c068bf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b80 tx=0x7f8d14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.582+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== osd.3 v2:172.21.15.179:6816/26068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8d0c06f680 con 0x7f8d0c06cb70 2024-05-08T22:45:33.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 --> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8d2c06b770 con 0x7f8d0c06cb70 2024-05-08T22:45:33.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d28ff9700 1 -- 172.21.15.179:0/780879970 <== osd.3 v2:172.21.15.179:6816/26068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8d24003c80 con 0x7f8d0c06cb70 2024-05-08T22:45:33.604 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:33.604 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:33.604 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:33.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7f8d0c06cb70 msgr2=0x7f8d0c06efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:6816/26068,v1:172.21.15.179:6817/26068] conn(0x7f8d0c06cb70 0x7f8d0c06efb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066730 msgr2=0x7f8d0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f8d0c066730 0x7f8d0c068bf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b80 tx=0x7f8d14008040 comp rx=0 tx=0).stop 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 msgr2=0x7f8d2c12f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 --2- 172.21.15.179:0/780879970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d2c0b2b50 0x7f8d2c12f0b0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c510 tx=0x7f8d1c00c820 comp rx=0 tx=0).stop 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d30bf1700 1 -- 172.21.15.179:0/780879970 reap_dead start 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 shutdown_connections 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 >> 172.21.15.179:0/780879970 conn(0x7f8d2c0cb610 msgr2=0x7f8d2c0cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 shutdown_connections 2024-05-08T22:45:33.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.602+0000 7f8d32654700 1 -- 172.21.15.179:0/780879970 wait complete. 2024-05-08T22:45:33.621 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T22:45:33.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 Processor -- start 2024-05-08T22:45:33.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 -- start start 2024-05-08T22:45:33.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f191406dda0 0x7f19140cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f191406e180 0x7f191406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191406b540 con 0x7f19140b25d0 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191406ac50 con 0x7f191406e180 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.738+0000 7f1918b27700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191406aef0 con 0x7f191406dda0 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f191406e180 0x7f191406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f191406e180 0x7f191406e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39314/0 (socket says 172.21.15.179:39314) 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 -- 172.21.15.179:0/4079788608 learned_addr learned my addr 172.21.15.179:0/4079788608 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f19137fe700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f191406dda0 0x7f19140cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 -- 172.21.15.179:0/4079788608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f191406dda0 msgr2=0x7f19140cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f191406dda0 0x7f19140cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 -- 172.21.15.179:0/4079788608 >> v1:172.21.15.195:6789/0 conn(0x7f19140b25d0 legacy=0x7f19140b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 -- 172.21.15.179:0/4079788608 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191406b1d0 con 0x7f191406e180 2024-05-08T22:45:33.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1912ffd700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406e180 0x7f191406e5e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009910 tx=0x7f18fc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f57fb55d50d73ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f19127fc700 1 -- 172.21.15.179:0/4079788608 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00ba60 con 0x7f191406e180 2024-05-08T22:45:33.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f19127fc700 1 -- 172.21.15.179:0/4079788608 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00bbc0 con 0x7f191406e180 2024-05-08T22:45:33.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f19127fc700 1 -- 172.21.15.179:0/4079788608 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc009e60 con 0x7f191406e180 2024-05-08T22:45:33.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 -- 172.21.15.179:0/4079788608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406e180 msgr2=0x7f191406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406e180 0x7f191406e5e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009910 tx=0x7f18fc009c20 comp rx=0 tx=0).stop 2024-05-08T22:45:33.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 -- 172.21.15.179:0/4079788608 shutdown_connections 2024-05-08T22:45:33.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406e180 0x7f191406e5e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 --2- 172.21.15.179:0/4079788608 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f191406dda0 0x7f19140cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 -- 172.21.15.179:0/4079788608 >> 172.21.15.179:0/4079788608 conn(0x7f19140ce960 msgr2=0x7f19140ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 -- 172.21.15.179:0/4079788608 shutdown_connections 2024-05-08T22:45:33.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.742+0000 7f1918b27700 1 -- 172.21.15.179:0/4079788608 wait complete. 2024-05-08T22:45:33.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 Processor -- start 2024-05-08T22:45:33.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 -- start start 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f191406e180 0x7f191415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19140b25d0 0x7f191415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1914155520 con 0x7f191406dda0 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1914154510 con 0x7f191406e180 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.746+0000 7f1918b27700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191414afe0 con 0x7f19140b25d0 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.752 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39328/0 (socket says 172.21.15.179:39328) 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 -- 172.21.15.179:0/1172055692 learned_addr learned my addr 172.21.15.179:0/1172055692 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f1912ffd700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f191406e180 0x7f191415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f1913fff700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19140b25d0 0x7f191415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 -- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19140b25d0 msgr2=0x7f191415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19140b25d0 0x7f191415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 -- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f191406e180 msgr2=0x7f191415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f191406e180 0x7f191415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 -- 172.21.15.179:0/1172055692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009570 con 0x7f191406dda0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f19137fe700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f190400e760 tx=0x7f190400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400c070 con 0x7f191406dda0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904013610 con 0x7f191406dda0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1914154760 con 0x7f191406dda0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904013980 con 0x7f191406dda0 2024-05-08T22:45:33.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1914154d20 con 0x7f191406dda0 2024-05-08T22:45:33.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.750+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191406e940 con 0x7f191406dda0 2024-05-08T22:45:33.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.754+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1904013ae0 con 0x7f191406dda0 2024-05-08T22:45:33.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.754+0000 7f190bfff700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 0x7f18f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:33.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.754+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1904083e10 con 0x7f191406dda0 2024-05-08T22:45:33.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.754+0000 7f1912ffd700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 0x7f18f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:33.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.754+0000 7f1912ffd700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 0x7f18f4068bf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:33.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.762+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1904051290 con 0x7f191406dda0 2024-05-08T22:45:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.930+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f19140cc170 con 0x7f18f4066730 2024-05-08T22:45:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.930+0000 7f190bfff700 1 -- 172.21.15.179:0/1172055692 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18fc00b860 con 0x7f18f4066730 2024-05-08T22:45:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 msgr2=0x7f18f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 0x7f18f4068bf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc008030 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 msgr2=0x7f191414aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f190400e760 tx=0x7f190400ea70 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 shutdown_connections 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f18f4066730 0x7f18f4068bf0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f191406e180 0x7f191415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19140b25d0 0x7f191415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 --2- 172.21.15.179:0/1172055692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f191406dda0 0x7f191414aaa0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 >> 172.21.15.179:0/1172055692 conn(0x7f19140ce960 msgr2=0x7f19141624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 shutdown_connections 2024-05-08T22:45:33.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:33.934+0000 7f1918b27700 1 -- 172.21.15.179:0/1172055692 wait complete. 2024-05-08T22:45:33.938 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T22:45:33.949 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T22:45:33.949 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:34.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 Processor -- start 2024-05-08T22:45:34.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 -- start start 2024-05-08T22:45:34.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.069 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fccc80e0250 0x7fccc80e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fccc8006150 0x7fccc8038ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc80e49a0 con 0x7fccc80e0250 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc80540b0 con 0x7fccc8006150 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd57f9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80532f0 con 0x7fccc80e3b40 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fccc80e0250 0x7fccc80e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fccc80e0250 0x7fccc80e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39344/0 (socket says 172.21.15.179:39344) 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 -- 172.21.15.179:0/3015412047 learned_addr learned my addr 172.21.15.179:0/3015412047 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccffff700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccd4ff8700 1 --2- 172.21.15.179:0/3015412047 >> v2:172.21.15.195:3300/0 conn(0x7fccc8006150 0x7fccc8038ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 -- 172.21.15.179:0/3015412047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 msgr2=0x7fccc80e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 -- 172.21.15.179:0/3015412047 >> v2:172.21.15.195:3300/0 conn(0x7fccc8006150 msgr2=0x7fccc8038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/3015412047 >> v2:172.21.15.195:3300/0 conn(0x7fccc8006150 0x7fccc8038ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 -- 172.21.15.179:0/3015412047 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc80535d0 con 0x7fccc80e0250 2024-05-08T22:45:34.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80e0250 0x7fccc80e4460 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fccc800db40 tx=0x7fccc4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1ccf3bfc8e04d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccceffd700 1 -- 172.21.15.179:0/3015412047 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011830 con 0x7fccc80e0250 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccceffd700 1 -- 172.21.15.179:0/3015412047 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc4011990 con 0x7fccc80e0250 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.066+0000 7fccceffd700 1 -- 172.21.15.179:0/3015412047 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011d20 con 0x7fccc80e0250 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- 172.21.15.179:0/3015412047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80e0250 msgr2=0x7fccc80e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80e0250 0x7fccc80e4460 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fccc800db40 tx=0x7fccc4009bf0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- 172.21.15.179:0/3015412047 shutdown_connections 2024-05-08T22:45:34.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- 172.21.15.179:0/3015412047 >> v2:172.21.15.195:3300/0 conn(0x7fccc8006150 0x7fccc8038ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc80e0250 0x7fccc80e4460 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- 172.21.15.179:0/3015412047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- 172.21.15.179:0/3015412047 >> 172.21.15.179:0/3015412047 conn(0x7fccc8054b70 msgr2=0x7fccc8054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- 172.21.15.179:0/3015412047 shutdown_connections 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- 172.21.15.179:0/3015412047 wait complete. 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 Processor -- start 2024-05-08T22:45:34.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- start start 2024-05-08T22:45:34.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 0x7fccc804c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80d6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8049770 0x7fccc8049bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc804a130 con 0x7fccc8006150 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80e1580 con 0x7fccc8049770 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd57f9700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc804cb90 con 0x7fccc80e3b40 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 0x7fccc804c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 0x7fccc804c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39354/0 (socket says 172.21.15.179:39354) 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 -- 172.21.15.179:0/503659060 learned_addr learned my addr 172.21.15.179:0/503659060 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccd4ff8700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8049770 0x7fccc8049bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 -- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 msgr2=0x7fccc80d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fccc80e3b40 0x7fccc80d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 -- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8049770 msgr2=0x7fccc8049bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8049770 0x7fccc8049bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009570 con 0x7fccc8006150 2024-05-08T22:45:34.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fcccffff700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 0x7fccc804c650 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e840 tx=0x7fccc000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.070+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00095a0 con 0x7fccc8006150 2024-05-08T22:45:34.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0013680 con 0x7fccc8006150 2024-05-08T22:45:34.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00139f0 con 0x7fccc8006150 2024-05-08T22:45:34.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc80e0970 con 0x7fccc8006150 2024-05-08T22:45:34.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc80e0f30 con 0x7fccc8006150 2024-05-08T22:45:34.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccc80da080 con 0x7fccc8006150 2024-05-08T22:45:34.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccc0013b50 con 0x7fccc8006150 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccbc066730 0x7fccbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fccc0083e80 con 0x7fccc8006150 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7fccbc06cb70 0x7fccbc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccbc06f680 con 0x7fccbc06cb70 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc0086540 con 0x7fccc8006150 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccbc066730 0x7fccbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.074+0000 7fccd4ff8700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7fccbc06cb70 0x7fccbc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.078+0000 7fcccf7fe700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccbc066730 0x7fccbc068bf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.078+0000 7fccd4ff8700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7fccbc06cb70 0x7fccbc06efb0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.078+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== osd.4 v2:172.21.15.195:6800/26287 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fccbc06f680 con 0x7fccbc06cb70 2024-05-08T22:45:34.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 --> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fccc8053b90 con 0x7fccbc06cb70 2024-05-08T22:45:34.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccccff9700 1 -- 172.21.15.179:0/503659060 <== osd.4 v2:172.21.15.195:6800/26287 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fccd004ef50 con 0x7fccbc06cb70 2024-05-08T22:45:34.099 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:34.099 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:34.099 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:34.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7fccbc06cb70 msgr2=0x7fccbc06efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6800/26287,v1:172.21.15.195:6801/26287] conn(0x7fccbc06cb70 0x7fccbc06efb0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccbc066730 msgr2=0x7fccbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fccbc066730 0x7fccbc068bf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4008030 comp rx=0 tx=0).stop 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 msgr2=0x7fccc804c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 --2- 172.21.15.179:0/503659060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc8006150 0x7fccc804c650 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e840 tx=0x7fccc000eb50 comp rx=0 tx=0).stop 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd4ff8700 1 -- 172.21.15.179:0/503659060 reap_dead start 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 shutdown_connections 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 >> 172.21.15.179:0/503659060 conn(0x7fccc8054b70 msgr2=0x7fccc8048920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 shutdown_connections 2024-05-08T22:45:34.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.098+0000 7fccd57f9700 1 -- 172.21.15.179:0/503659060 wait complete. 2024-05-08T22:45:34.119 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T22:45:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 Processor -- start 2024-05-08T22:45:34.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 -- start start 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d8159d00 0x7f65d80cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f65d80b3800 0x7f65d815d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d806abb0 con 0x7f65d815dec0 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d815e350 con 0x7f65d80b3800 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65dcfe8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d806b0e0 con 0x7f65d8159d00 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f65d80b3800 0x7f65d815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f65d80b3800 0x7f65d815d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39356/0 (socket says 172.21.15.179:39356) 2024-05-08T22:45:34.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 -- 172.21.15.179:0/1755409587 learned_addr learned my addr 172.21.15.179:0/1755409587 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d77fe700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d8159d00 0x7f65d80cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 -- 172.21.15.179:0/1755409587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d8159d00 msgr2=0x7f65d80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d8159d00 0x7f65d80cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 -- 172.21.15.179:0/1755409587 >> v1:172.21.15.179:6789/0 conn(0x7f65d815dec0 legacy=0x7f65d80b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 -- 172.21.15.179:0/1755409587 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80cd490 con 0x7f65d80b3800 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d6ffd700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d80b3800 0x7f65d815d980 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009e50 tx=0x7f65c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c0b6994f282099 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d67fc700 1 -- 172.21.15.179:0/1755409587 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800e070 con 0x7f65d80b3800 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d67fc700 1 -- 172.21.15.179:0/1755409587 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800b9c0 con 0x7f65d80b3800 2024-05-08T22:45:34.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.230+0000 7f65d67fc700 1 -- 172.21.15.179:0/1755409587 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800bd70 con 0x7f65d80b3800 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- 172.21.15.179:0/1755409587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d80b3800 msgr2=0x7f65d815d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d80b3800 0x7f65d815d980 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009e50 tx=0x7f65c800b330 comp rx=0 tx=0).stop 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- 172.21.15.179:0/1755409587 shutdown_connections 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d80b3800 0x7f65d815d980 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/1755409587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d8159d00 0x7f65d80cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- 172.21.15.179:0/1755409587 >> 172.21.15.179:0/1755409587 conn(0x7f65d80ceb70 msgr2=0x7f65d80cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- 172.21.15.179:0/1755409587 shutdown_connections 2024-05-08T22:45:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- 172.21.15.179:0/1755409587 wait complete. 2024-05-08T22:45:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 Processor -- start 2024-05-08T22:45:34.237 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- start start 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 0x7f65d814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d815dec0 0x7f65d80af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8163930 con 0x7f65d8159d00 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d81f6e90 con 0x7f65d815dec0 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65dcfe8700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8162f90 con 0x7f65d80b3800 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65d6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65d6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39364/0 (socket says 172.21.15.179:39364) 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65d6ffd700 1 -- 172.21.15.179:0/649762510 learned_addr learned my addr 172.21.15.179:0/649762510 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65d7fff700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d815dec0 0x7f65d80af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.234+0000 7f65d77fe700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 0x7f65d814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 -- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 msgr2=0x7f65d814b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 0x7f65d814b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 -- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d815dec0 msgr2=0x7f65d80af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d815dec0 0x7f65d80af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 -- 172.21.15.179:0/649762510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009b30 con 0x7f65d8159d00 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d77fe700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 0x7f65d814b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65d6ffd700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f65d8067e10 tx=0x7f65c8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8004960 con 0x7f65d8159d00 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80092e0 con 0x7f65d8159d00 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d8163270 con 0x7f65d8159d00 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c801b690 con 0x7f65d8159d00 2024-05-08T22:45:34.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d8163d50 con 0x7f65d8159d00 2024-05-08T22:45:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8163a70 con 0x7f65d8159d00 2024-05-08T22:45:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65c801b7f0 con 0x7f65d8159d00 2024-05-08T22:45:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 0x7f65c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.238+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f65c80859b0 con 0x7f65d8159d00 2024-05-08T22:45:34.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.242+0000 7f65d77fe700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 0x7f65c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.242+0000 7f65d77fe700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 0x7f65c4068bf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f65d8164be0 tx=0x7f65cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.242+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65d8163a70 con 0x7f65d8159d00 2024-05-08T22:45:34.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.410+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f65d80cda10 con 0x7f65c4066730 2024-05-08T22:45:34.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65bffff700 1 -- 172.21.15.179:0/649762510 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65cc004030 con 0x7f65c4066730 2024-05-08T22:45:34.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 msgr2=0x7f65c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 0x7f65c4068bf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f65d8164be0 tx=0x7f65cc008040 comp rx=0 tx=0).stop 2024-05-08T22:45:34.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 msgr2=0x7f65d8148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f65d8067e10 tx=0x7f65c8004af0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 shutdown_connections 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f65c4066730 0x7f65c4068bf0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d815dec0 0x7f65d80af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d8159d00 0x7f65d8148a50 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 --2- 172.21.15.179:0/649762510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65d80b3800 0x7f65d814b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 >> 172.21.15.179:0/649762510 conn(0x7f65d80ceb70 msgr2=0x7f65d8150060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 shutdown_connections 2024-05-08T22:45:34.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.414+0000 7f65dcfe8700 1 -- 172.21.15.179:0/649762510 wait complete. 2024-05-08T22:45:34.419 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T22:45:34.429 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T22:45:34.429 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:34.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.546+0000 7f00d5128700 1 Processor -- start 2024-05-08T22:45:34.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.546+0000 7f00d5128700 1 -- start start 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00d0157be0 0x7f00d006e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f00d006cd50 0x7f00d015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d006ddf0 con 0x7f00d0157800 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d006e7c0 con 0x7f00d006cd50 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d006d1e0 con 0x7f00d0157be0 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f00d006cd50 0x7f00d015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f00d006cd50 0x7f00d015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39370/0 (socket says 172.21.15.179:39370) 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 -- 172.21.15.179:0/2530089084 learned_addr learned my addr 172.21.15.179:0/2530089084 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00cf7fe700 1 --2- 172.21.15.179:0/2530089084 >> v2:172.21.15.195:3300/0 conn(0x7f00d0157be0 0x7f00d006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 -- 172.21.15.179:0/2530089084 >> v2:172.21.15.195:3300/0 conn(0x7f00d0157be0 msgr2=0x7f00d006e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 --2- 172.21.15.179:0/2530089084 >> v2:172.21.15.195:3300/0 conn(0x7f00d0157be0 0x7f00d006e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 -- 172.21.15.179:0/2530089084 >> v1:172.21.15.195:6789/0 conn(0x7f00d0157800 legacy=0x7f00d00cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 -- 172.21.15.179:0/2530089084 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d006d4c0 con 0x7f00d006cd50 2024-05-08T22:45:34.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d4927700 1 --2- 172.21.15.179:0/2530089084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006cd50 0x7f00d015da30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ae5b0 tx=0x7f00c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23c35eb35029120 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00ceffd700 1 -- 172.21.15.179:0/2530089084 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800e070 con 0x7f00d006cd50 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00ceffd700 1 -- 172.21.15.179:0/2530089084 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c800b910 con 0x7f00d006cd50 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00ceffd700 1 -- 172.21.15.179:0/2530089084 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800bcc0 con 0x7f00d006cd50 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- 172.21.15.179:0/2530089084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006cd50 msgr2=0x7f00d015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 --2- 172.21.15.179:0/2530089084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006cd50 0x7f00d015da30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ae5b0 tx=0x7f00c800b330 comp rx=0 tx=0).stop 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- 172.21.15.179:0/2530089084 shutdown_connections 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 --2- 172.21.15.179:0/2530089084 >> v2:172.21.15.195:3300/0 conn(0x7f00d0157be0 0x7f00d006e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 --2- 172.21.15.179:0/2530089084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d006cd50 0x7f00d015da30 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- 172.21.15.179:0/2530089084 >> 172.21.15.179:0/2530089084 conn(0x7f00d00ce9a0 msgr2=0x7f00d00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- 172.21.15.179:0/2530089084 shutdown_connections 2024-05-08T22:45:34.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.550+0000 7f00d5128700 1 -- 172.21.15.179:0/2530089084 wait complete. 2024-05-08T22:45:34.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.554+0000 7f00d5128700 1 Processor -- start 2024-05-08T22:45:34.557 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.554+0000 7f00d5128700 1 -- start start 2024-05-08T22:45:34.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 0x7f00d0138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0159c00 0x7f00d01f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d014e7d0 0x7f00d00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d014d130 con 0x7f00d0157800 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01f4b60 con 0x7f00d0159c00 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01f45a0 con 0x7f00d014e7d0 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 0x7f00d0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 0x7f00d0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39372/0 (socket says 172.21.15.179:39372) 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 -- 172.21.15.179:0/2499406096 learned_addr learned my addr 172.21.15.179:0/2499406096 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cf7fe700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0159c00 0x7f00d01f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d4927700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d014e7d0 0x7f00d00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 -- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d014e7d0 msgr2=0x7f00d00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d014e7d0 0x7f00d00b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 -- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0159c00 msgr2=0x7f00d01f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0159c00 0x7f00d01f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c8009b30 con 0x7f00d0157800 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00cffff700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 0x7f00d0138230 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c500 tx=0x7f00c400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40052d0 con 0x7f00d0157800 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4005430 con 0x7f00d0157800 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d01f4880 con 0x7f00d0157800 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40055a0 con 0x7f00d0157800 2024-05-08T22:45:34.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d0138770 con 0x7f00d0157800 2024-05-08T22:45:34.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.558+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00d006da80 con 0x7f00d0157800 2024-05-08T22:45:34.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00c40038e0 con 0x7f00d0157800 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00bc066760 0x7f00bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f00c4083420 con 0x7f00d0157800 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00cf7fe700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00bc066760 0x7f00bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7f00bc06cba0 0x7f00bc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00bc06f6b0 con 0x7f00bc06cba0 2024-05-08T22:45:34.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00c4085a00 con 0x7f00d0157800 2024-05-08T22:45:34.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00d4927700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7f00bc06cba0 0x7f00bc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00cf7fe700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00bc066760 0x7f00bc068c20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f00d00b3460 tx=0x7f00c0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.562+0000 7f00d4927700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7f00bc06cba0 0x7f00bc06efe0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.567 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.566+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== osd.5 v2:172.21.15.195:6802/26288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f00bc06f6b0 con 0x7f00bc06cba0 2024-05-08T22:45:34.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 --> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f00d014d270 con 0x7f00bc06cba0 2024-05-08T22:45:34.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00ccff9700 1 -- 172.21.15.179:0/2499406096 <== osd.5 v2:172.21.15.195:6802/26288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f00c80092e0 con 0x7f00bc06cba0 2024-05-08T22:45:34.585 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:34.585 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:34.585 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:34.586 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7f00bc06cba0 msgr2=0x7f00bc06efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6802/26288,v1:172.21.15.195:6805/26288] conn(0x7f00bc06cba0 0x7f00bc06efe0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00bc066760 msgr2=0x7f00bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f00bc066760 0x7f00bc068c20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f00d00b3460 tx=0x7f00c0009240 comp rx=0 tx=0).stop 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 msgr2=0x7f00d0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 --2- 172.21.15.179:0/2499406096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00d0157800 0x7f00d0138230 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c500 tx=0x7f00c400c810 comp rx=0 tx=0).stop 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d4927700 1 -- 172.21.15.179:0/2499406096 reap_dead start 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.582+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 shutdown_connections 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.586+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 >> 172.21.15.179:0/2499406096 conn(0x7f00d00ce9a0 msgr2=0x7f00d006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.586+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 shutdown_connections 2024-05-08T22:45:34.587 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.586+0000 7f00d5128700 1 -- 172.21.15.179:0/2499406096 wait complete. 2024-05-08T22:45:34.606 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T22:45:34.721 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.718+0000 7fb9633ae700 1 Processor -- start 2024-05-08T22:45:34.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.718+0000 7fb9633ae700 1 -- start start 2024-05-08T22:45:34.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb95c15a010 0x7fb95c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb95c06c2d0 0x7fb95c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c0cd260 con 0x7fb95c06e720 2024-05-08T22:45:34.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c0cb060 con 0x7fb95c06c2d0 2024-05-08T22:45:34.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c0cb340 con 0x7fb95c15a010 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb95c06c2d0 0x7fb95c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb960949700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb95c06e720 0x7fb95c06bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:51014/0 (socket says 172.21.15.179:51014) 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb960949700 1 -- 172.21.15.179:0/1306972750 learned_addr learned my addr 172.21.15.179:0/1306972750 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96114a700 1 --2- 172.21.15.179:0/1306972750 >> v2:172.21.15.195:3300/0 conn(0x7fb95c15a010 0x7fb95c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 -- 172.21.15.179:0/1306972750 >> v2:172.21.15.195:3300/0 conn(0x7fb95c15a010 msgr2=0x7fb95c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:34.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 --2- 172.21.15.179:0/1306972750 >> v2:172.21.15.195:3300/0 conn(0x7fb95c15a010 0x7fb95c0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 -- 172.21.15.179:0/1306972750 >> v1:172.21.15.179:6789/0 conn(0x7fb95c06e720 legacy=0x7fb95c06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 -- 172.21.15.179:0/1306972750 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0cd4b0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb953fff700 1 -- 172.21.15.179:0/1306972750 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117570754 0 0) 0x7fb95c0cd260 con 0x7fb95c06e720 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb96194b700 1 --2- 172.21.15.179:0/1306972750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0cdcb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fb954009850 tx=0x7fb954009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7003fb05ea9c5ce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb953fff700 1 -- 172.21.15.179:0/1306972750 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954011830 con 0x7fb95c06c2d0 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb953fff700 1 -- 172.21.15.179:0/1306972750 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954011990 con 0x7fb95c06c2d0 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb953fff700 1 -- 172.21.15.179:0/1306972750 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954011d20 con 0x7fb95c06c2d0 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- 172.21.15.179:0/1306972750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 msgr2=0x7fb95c0cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.729 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 --2- 172.21.15.179:0/1306972750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0cdcb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fb954009850 tx=0x7fb954009b60 comp rx=0 tx=0).stop 2024-05-08T22:45:34.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- 172.21.15.179:0/1306972750 shutdown_connections 2024-05-08T22:45:34.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 --2- 172.21.15.179:0/1306972750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0cdcb0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 --2- 172.21.15.179:0/1306972750 >> v2:172.21.15.195:3300/0 conn(0x7fb95c15a010 0x7fb95c0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- 172.21.15.179:0/1306972750 >> 172.21.15.179:0/1306972750 conn(0x7fb95c070e50 msgr2=0x7fb95c071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- 172.21.15.179:0/1306972750 shutdown_connections 2024-05-08T22:45:34.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.722+0000 7fb9633ae700 1 -- 172.21.15.179:0/1306972750 wait complete. 2024-05-08T22:45:34.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 Processor -- start 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 -- start start 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb95c06e720 0x7fb95c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 0x7fb95c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c14f2f0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0c77b0 con 0x7fb95c15a010 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb9633ae700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0c8720 con 0x7fb95c06e720 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39382/0 (socket says 172.21.15.179:39382) 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 -- 172.21.15.179:0/2283700078 learned_addr learned my addr 172.21.15.179:0/2283700078 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:34.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96194b700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 0x7fb95c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb960949700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb95c06e720 0x7fb95c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 -- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb95c06e720 msgr2=0x7fb95c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb95c06e720 0x7fb95c0c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 -- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 msgr2=0x7fb95c13fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 0x7fb95c13fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 -- 172.21.15.179:0/2283700078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540094b0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96194b700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 0x7fb95c13fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb96114a700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00e840 tx=0x7fb94c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.726+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0095a0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.730+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00c690 con 0x7fb95c06c2d0 2024-05-08T22:45:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.730+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00ca00 con 0x7fb95c06c2d0 2024-05-08T22:45:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.730+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c14eac0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.730+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c14f010 con 0x7fb95c06c2d0 2024-05-08T22:45:34.736 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.730+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c067770 con 0x7fb95c06c2d0 2024-05-08T22:45:34.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.734+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb94c00cb60 con 0x7fb95c06c2d0 2024-05-08T22:45:34.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.734+0000 7fb951ffb700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 0x7fb940068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:34.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.734+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb94c084890 con 0x7fb95c06c2d0 2024-05-08T22:45:34.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.734+0000 7fb960949700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 0x7fb940068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:34.737 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.734+0000 7fb960949700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 0x7fb940068bf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb95c152680 tx=0x7fb9580018c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:34.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.738+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb94c051cf0 con 0x7fb95c06c2d0 2024-05-08T22:45:34.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.910+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb95c15a3f0 con 0x7fb940066730 2024-05-08T22:45:34.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.910+0000 7fb951ffb700 1 -- 172.21.15.179:0/2283700078 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb9580079a0 con 0x7fb940066730 2024-05-08T22:45:34.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 msgr2=0x7fb940068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 0x7fb940068bf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb95c152680 tx=0x7fb9580018c0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 msgr2=0x7fb95c0c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:34.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00e840 tx=0x7fb94c00ec00 comp rx=0 tx=0).stop 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 shutdown_connections 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7fb940066730 0x7fb940068bf0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c15a010 0x7fb95c13fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb95c06e720 0x7fb95c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 --2- 172.21.15.179:0/2283700078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb95c06c2d0 0x7fb95c0c81e0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 >> 172.21.15.179:0/2283700078 conn(0x7fb95c070e50 msgr2=0x7fb95c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 shutdown_connections 2024-05-08T22:45:34.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:34.914+0000 7fb9633ae700 1 -- 172.21.15.179:0/2283700078 wait complete. 2024-05-08T22:45:34.917 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T22:45:34.932 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T22:45:34.932 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.050+0000 7f6e2f063700 1 Processor -- start 2024-05-08T22:45:35.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.050+0000 7f6e2f063700 1 -- start start 2024-05-08T22:45:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e280d0dc0 0x7f6e280d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e2806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e280b8d70 con 0x7f6e280d2290 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2806c560 con 0x7f6e280d0dc0 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2806bfa0 con 0x7f6e2806abb0 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e280d0dc0 0x7f6e280d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e280d0dc0 0x7f6e280d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39384/0 (socket says 172.21.15.179:39384) 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 -- 172.21.15.179:0/1918519535 learned_addr learned my addr 172.21.15.179:0/1918519535 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e862700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e2806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 -- 172.21.15.179:0/1918519535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 msgr2=0x7f6e2806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e2806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 -- 172.21.15.179:0/1918519535 >> v1:172.21.15.195:6789/0 conn(0x7f6e280d2290 legacy=0x7f6e280b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 -- 172.21.15.179:0/1918519535 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2806c280 con 0x7f6e280d0dc0 2024-05-08T22:45:35.056 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2e061700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d0dc0 0x7f6e280d11a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009910 tx=0x7f6e18009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f931895a61ba877 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2d05f700 1 -- 172.21.15.179:0/1918519535 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18011830 con 0x7f6e280d0dc0 2024-05-08T22:45:35.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2d05f700 1 -- 172.21.15.179:0/1918519535 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18011990 con 0x7f6e280d0dc0 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2d05f700 1 -- 172.21.15.179:0/1918519535 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18011d40 con 0x7f6e280d0dc0 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- 172.21.15.179:0/1918519535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d0dc0 msgr2=0x7f6e280d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d0dc0 0x7f6e280d11a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009910 tx=0x7f6e18009c20 comp rx=0 tx=0).stop 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- 172.21.15.179:0/1918519535 shutdown_connections 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e2806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 --2- 172.21.15.179:0/1918519535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d0dc0 0x7f6e280d11a0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- 172.21.15.179:0/1918519535 >> 172.21.15.179:0/1918519535 conn(0x7f6e280b48d0 msgr2=0x7f6e280b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- 172.21.15.179:0/1918519535 shutdown_connections 2024-05-08T22:45:35.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.054+0000 7f6e2f063700 1 -- 172.21.15.179:0/1918519535 wait complete. 2024-05-08T22:45:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 Processor -- start 2024-05-08T22:45:35.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- start start 2024-05-08T22:45:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e28155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e280d0dc0 0x7f6e280cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 0x7f6e28154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28158080 con 0x7f6e280d2290 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28158360 con 0x7f6e280d0dc0 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28158640 con 0x7f6e2806abb0 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 0x7f6e28154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 0x7f6e28154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39394/0 (socket says 172.21.15.179:39394) 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 -- 172.21.15.179:0/3871195363 learned_addr learned my addr 172.21.15.179:0/3871195363 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2d860700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e280d0dc0 0x7f6e280cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e061700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e28155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 -- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 msgr2=0x7f6e28155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e2806abb0 0x7f6e28155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 -- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e280d0dc0 msgr2=0x7f6e280cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e280d0dc0 0x7f6e280cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18009570 con 0x7f6e280d2290 2024-05-08T22:45:35.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2e862700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 0x7f6e28154990 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000efc0 tx=0x7f6e2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20009ba0 con 0x7f6e280d2290 2024-05-08T22:45:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2000f040 con 0x7f6e280d2290 2024-05-08T22:45:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e280d1860 con 0x7f6e280d2290 2024-05-08T22:45:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20014480 con 0x7f6e280d2290 2024-05-08T22:45:35.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e28158c00 con 0x7f6e280d2290 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.058+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e280d16e0 con 0x7f6e280d2290 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e200145e0 con 0x7f6e280d2290 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e0c066730 0x7f6e0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6e20084ff0 con 0x7f6e280d2290 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f6e0c06cb70 0x7f6e0c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e0c06f680 con 0x7f6e0c06cb70 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e20085440 con 0x7f6e280d2290 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e2e061700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e0c066730 0x7f6e0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e2d860700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f6e0c06cb70 0x7f6e0c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e2d860700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f6e0c06cb70 0x7f6e0c06efb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.062+0000 7f6e2e061700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e0c066730 0x7f6e0c068bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009c60 tx=0x7f6e18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.066 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.066+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== osd.6 v2:172.21.15.195:6824/26291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6e0c06f680 con 0x7f6e0c06cb70 2024-05-08T22:45:35.084 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 --> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6e280d1d70 con 0x7f6e0c06cb70 2024-05-08T22:45:35.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e1effd700 1 -- 172.21.15.179:0/3871195363 <== osd.6 v2:172.21.15.195:6824/26291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6e24001840 con 0x7f6e0c06cb70 2024-05-08T22:45:35.085 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:35.085 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:35.085 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:35.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f6e0c06cb70 msgr2=0x7f6e0c06efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6824/26291,v1:172.21.15.195:6825/26291] conn(0x7f6e0c06cb70 0x7f6e0c06efb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e0c066730 msgr2=0x7f6e0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6e0c066730 0x7f6e0c068bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009c60 tx=0x7f6e18008030 comp rx=0 tx=0).stop 2024-05-08T22:45:35.086 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 msgr2=0x7f6e28154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 --2- 172.21.15.179:0/3871195363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e280d2290 0x7f6e28154990 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000efc0 tx=0x7f6e2000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2e862700 1 -- 172.21.15.179:0/3871195363 reap_dead start 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 shutdown_connections 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 >> 172.21.15.179:0/3871195363 conn(0x7f6e280b48d0 msgr2=0x7f6e280b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 shutdown_connections 2024-05-08T22:45:35.087 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.082+0000 7f6e2f063700 1 -- 172.21.15.179:0/3871195363 wait complete. 2024-05-08T22:45:35.103 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T22:45:35.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 Processor -- start 2024-05-08T22:45:35.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 -- start start 2024-05-08T22:45:35.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0cc9e0 con 0x7f490c15f0c0 2024-05-08T22:45:35.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0b37a0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.218+0000 7f49118a6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0b3a80 con 0x7f490c0b3180 2024-05-08T22:45:35.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f490c15f0c0 0x7f490c0b2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:51020/0 (socket says 172.21.15.179:51020) 2024-05-08T22:45:35.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490bfff700 1 -- 172.21.15.179:0/4256958243 learned_addr learned my addr 172.21.15.179:0/4256958243 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:35.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49110a5700 1 --2- 172.21.15.179:0/4256958243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490b7fe700 1 -- 172.21.15.179:0/4256958243 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686889924 0 0) 0x7f490c0cc9e0 con 0x7f490c15f0c0 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490b7fe700 1 -- 172.21.15.179:0/4256958243 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f80035d0 con 0x7f490c15f0c0 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49110a5700 1 -- 172.21.15.179:0/4256958243 >> v1:172.21.15.195:6789/0 conn(0x7f490c0ad9d0 legacy=0x7f490c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49110a5700 1 -- 172.21.15.179:0/4256958243 >> v1:172.21.15.179:6789/0 conn(0x7f490c15f0c0 legacy=0x7f490c0b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49110a5700 1 -- 172.21.15.179:0/4256958243 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0b3d60 con 0x7f490c0b3180 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49110a5700 1 --2- 172.21.15.179:0/4256958243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c165d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4904009880 tx=0x7f4904009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3ae75ba36b6297 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490b7fe700 1 -- 172.21.15.179:0/4256958243 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011830 con 0x7f490c0b3180 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490b7fe700 1 -- 172.21.15.179:0/4256958243 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4904011990 con 0x7f490c0b3180 2024-05-08T22:45:35.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f490b7fe700 1 -- 172.21.15.179:0/4256958243 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011d00 con 0x7f490c0b3180 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 -- 172.21.15.179:0/4256958243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 msgr2=0x7f490c165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 --2- 172.21.15.179:0/4256958243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c165d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4904009880 tx=0x7f4904009b90 comp rx=0 tx=0).stop 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 -- 172.21.15.179:0/4256958243 shutdown_connections 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 --2- 172.21.15.179:0/4256958243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c165d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 -- 172.21.15.179:0/4256958243 >> 172.21.15.179:0/4256958243 conn(0x7f490c0ce960 msgr2=0x7f490c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 -- 172.21.15.179:0/4256958243 shutdown_connections 2024-05-08T22:45:35.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.222+0000 7f49118a6700 1 -- 172.21.15.179:0/4256958243 wait complete. 2024-05-08T22:45:35.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.226+0000 7f49118a6700 1 Processor -- start 2024-05-08T22:45:35.229 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.226+0000 7f49118a6700 1 -- start start 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 0x7f490c13f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c14b760 con 0x7f490c0ad9d0 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c06ee00 con 0x7f490c15f0c0 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15b840 con 0x7f490c0b3180 2024-05-08T22:45:35.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39408/0 (socket says 172.21.15.179:39408) 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 -- 172.21.15.179:0/1611463709 learned_addr learned my addr 172.21.15.179:0/1611463709 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f490bfff700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49110a5700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 0x7f490c13f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 -- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 msgr2=0x7f490c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 -- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 msgr2=0x7f490c13f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 0x7f490c13f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 -- 172.21.15.179:0/1611463709 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040094b0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f490bfff700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c06e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49110a5700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 0x7f490c13f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49108a4700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c530 tx=0x7f48f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40052d0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f4005430 con 0x7f490c0ad9d0 2024-05-08T22:45:35.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0adf30 con 0x7f490c0ad9d0 2024-05-08T22:45:35.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40055a0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c15b280 con 0x7f490c0ad9d0 2024-05-08T22:45:35.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c04e700 con 0x7f490c0ad9d0 2024-05-08T22:45:35.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.230+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48f40036a0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.234+0000 7f49097fa700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 0x7f48ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.234+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f48f40833d0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.234+0000 7f490bfff700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 0x7f48ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.234+0000 7f490bfff700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 0x7f48ec068bf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005cb0 tx=0x7f48fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.242+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f48f40507d0 con 0x7f490c0ad9d0 2024-05-08T22:45:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.406+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f490c0addb0 con 0x7f48ec066730 2024-05-08T22:45:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49097fa700 1 -- 172.21.15.179:0/1611463709 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f48fc001ee0 con 0x7f48ec066730 2024-05-08T22:45:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 msgr2=0x7f48ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 0x7f48ec068bf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005cb0 tx=0x7f48fc009040 comp rx=0 tx=0).stop 2024-05-08T22:45:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 msgr2=0x7f490c15a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c530 tx=0x7f48f400c8f0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 shutdown_connections 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f48ec066730 0x7f48ec068bf0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f490c15f0c0 0x7f490c13f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f490c0b3180 0x7f490c06e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 --2- 172.21.15.179:0/1611463709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c0ad9d0 0x7f490c15a8a0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 >> 172.21.15.179:0/1611463709 conn(0x7f490c0ce960 msgr2=0x7f490c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 shutdown_connections 2024-05-08T22:45:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.410+0000 7f49118a6700 1 -- 172.21.15.179:0/1611463709 wait complete. 2024-05-08T22:45:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T22:45:35.426 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T22:45:35.427 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T22:45:35.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.538+0000 7f6e140ba700 1 Processor -- start 2024-05-08T22:45:35.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.538+0000 7f6e140ba700 1 -- start start 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e0c0cd4f0 0x7f6e0c16c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c06b040 con 0x7f6e0c06abb0 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c06b270 con 0x7f6e0c06e880 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c06b550 con 0x7f6e0c0cd4f0 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6e0c06abb0 0x7f6e0c0ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:39958/0 (socket says 172.21.15.179:39958) 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 -- 172.21.15.179:0/885492879 learned_addr learned my addr 172.21.15.179:0/885492879 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:35.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e12657700 1 --2- 172.21.15.179:0/885492879 >> v2:172.21.15.179:3300/0 conn(0x7f6e0c0cd4f0 0x7f6e0c16c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685756990 0 0) 0x7f6e0c06b040 con 0x7f6e0c06abb0 2024-05-08T22:45:35.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df40035d0 con 0x7f6e0c06abb0 2024-05-08T22:45:35.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183145111 0 0) 0x7f6e0c06b270 con 0x7f6e0c06e880 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e0c06b040 con 0x7f6e0c06e880 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119861674 0 0) 0x7f6df40035d0 con 0x7f6e0c06abb0 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 >> v2:172.21.15.179:3300/0 conn(0x7f6e0c0cd4f0 msgr2=0x7f6e0c16c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 --2- 172.21.15.179:0/885492879 >> v2:172.21.15.179:3300/0 conn(0x7f6e0c0cd4f0 0x7f6e0c16c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 >> v1:172.21.15.195:6789/0 conn(0x7f6e0c06e880 legacy=0x7f6e0c162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c06b830 con 0x7f6e0c06abb0 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2158553611 0 0) 0x7f6dfc0019f0 con 0x7f6e0c06abb0 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e12657700 1 --2- 172.21.15.179:0/885492879 >> v2:172.21.15.179:3300/0 conn(0x7f6e0c0cd4f0 0x7f6e0c16c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 >> v1:172.21.15.179:6789/0 conn(0x7f6e0c06abb0 legacy=0x7f6e0c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df4003bb0 0x7f6df4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 0x7f6df4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4005230 0x7f6df400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c06b550 con 0x7f6df4004870 2024-05-08T22:45:35.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df400c0b0 con 0x7f6df4003bb0 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df400c270 con 0x7f6df4005230 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 0x7f6df4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11e56700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df4003bb0 0x7f6df4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e12657700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4005230 0x7f6df400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 -- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4005230 msgr2=0x7f6df400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4005230 0x7f6df400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 -- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df4003bb0 msgr2=0x7f6df4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df4003bb0 0x7f6df4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 -- 172.21.15.179:0/885492879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40035d0 con 0x7f6df4004870 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11e56700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6df4003bb0 0x7f6df4003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e12657700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4005230 0x7f6df400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e12657700 1 -- 172.21.15.179:0/885492879 reap_dead start 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e11655700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 0x7f6df4004cf0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00a7c0 tx=0x7f6df40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33cd70df1da700 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0072d0 con 0x7f6df4004870 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc007450 con 0x7f6df4004870 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e10e54700 1 -- 172.21.15.179:0/885492879 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc007800 con 0x7f6df4004870 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 msgr2=0x7f6df4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 0x7f6df4004cf0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00a7c0 tx=0x7f6df40058a0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- 172.21.15.179:0/885492879 shutdown_connections 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 --2- 172.21.15.179:0/885492879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6df4004870 0x7f6df4004cf0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- 172.21.15.179:0/885492879 >> 172.21.15.179:0/885492879 conn(0x7f6e0c0cb610 msgr2=0x7f6e0c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.542+0000 7f6e140ba700 1 -- 172.21.15.179:0/885492879 shutdown_connections 2024-05-08T22:45:35.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- 172.21.15.179:0/885492879 wait complete. 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 Processor -- start 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- start start 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4004870 0x7f6e0c14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0a74d0 0x7f6e0c148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 0x7f6e0c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c158670 con 0x7f6e0c14ad20 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c14b6e0 con 0x7f6e0c0a74d0 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0a7950 con 0x7f6df4004870 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e12657700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 0x7f6e0c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e12657700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 0x7f6e0c14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46492/0 (socket says 172.21.15.179:46492) 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e12657700 1 -- 172.21.15.179:0/2872806285 learned_addr learned my addr 172.21.15.179:0/2872806285 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4004870 0x7f6e0c14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11655700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0a74d0 0x7f6e0c148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 -- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0a74d0 msgr2=0x7f6e0c148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e0c0a74d0 0x7f6e0c148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 -- 172.21.15.179:0/2872806285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 msgr2=0x7f6e0c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 0x7f6e0c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0a7bb0 con 0x7f6df4004870 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e12657700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e0c14ad20 0x7f6e0c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:45:35.559 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e11e56700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4004870 0x7f6e0c14b8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d7a0 tx=0x7f6e0000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00006880 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e000069e0 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c148d30 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00006db0 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.546+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c1492f0 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e0c06ec60 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e00007020 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6df8066760 0x7f6df8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6e00084720 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e11655700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6df8066760 0x7f6df8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f6df806cba0 0x7f6df806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6df806f6b0 con 0x7f6df806cba0 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.550+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e00084b70 con 0x7f6df4004870 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.554+0000 7f6e12657700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f6df806cba0 0x7f6df806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.554+0000 7f6e11655700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6df8066760 0x7f6df8068c20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc001c30 tx=0x7f6dfc009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.554+0000 7f6e12657700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f6df806cba0 0x7f6df806efe0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.554+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== osd.7 v2:172.21.15.195:6816/26289 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6df806f6b0 con 0x7f6df806cba0 2024-05-08T22:45:35.575 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 --> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6e0c06bdf0 con 0x7f6df806cba0 2024-05-08T22:45:35.576 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e0a7fc700 1 -- 172.21.15.179:0/2872806285 <== osd.7 v2:172.21.15.195:6816/26289 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6e04009d20 con 0x7f6df806cba0 2024-05-08T22:45:35.576 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-08T22:45:35.576 INFO:teuthology.orchestra.run.smithi179.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T22:45:35.576 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-08T22:45:35.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f6df806cba0 msgr2=0x7f6df806efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6816/26289,v1:172.21.15.195:6817/26289] conn(0x7f6df806cba0 0x7f6df806efe0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6df8066760 msgr2=0x7f6df8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.577 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f6df8066760 0x7f6df8068c20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc001c30 tx=0x7f6dfc009070 comp rx=0 tx=0).stop 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4004870 msgr2=0x7f6e0c14b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 --2- 172.21.15.179:0/2872806285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6df4004870 0x7f6e0c14b8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d7a0 tx=0x7f6e0000dab0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e12657700 1 -- 172.21.15.179:0/2872806285 reap_dead start 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 shutdown_connections 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 >> 172.21.15.179:0/2872806285 conn(0x7f6e0c0cb610 msgr2=0x7f6e0c0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 shutdown_connections 2024-05-08T22:45:35.578 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.574+0000 7f6e140ba700 1 -- 172.21.15.179:0/2872806285 wait complete. 2024-05-08T22:45:35.589 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T22:45:35.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.702+0000 7ff3c8ed5700 1 Processor -- start 2024-05-08T22:45:35.706 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.702+0000 7ff3c8ed5700 1 -- start start 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3c40b3180 0x7ff3c40cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40b37a0 con 0x7ff3c415eef0 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c415f380 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40b3a80 con 0x7ff3c40b3180 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46498/0 (socket says 172.21.15.179:46498) 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 -- 172.21.15.179:0/340301246 learned_addr learned my addr 172.21.15.179:0/340301246 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c3fff700 1 --2- 172.21.15.179:0/340301246 >> v2:172.21.15.195:3300/0 conn(0x7ff3c40b3180 0x7ff3c40cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 -- 172.21.15.179:0/340301246 >> v2:172.21.15.195:3300/0 conn(0x7ff3c40b3180 msgr2=0x7ff3c40cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:35.708 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 --2- 172.21.15.179:0/340301246 >> v2:172.21.15.195:3300/0 conn(0x7ff3c40b3180 0x7ff3c40cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 -- 172.21.15.179:0/340301246 >> v1:172.21.15.195:6789/0 conn(0x7ff3c415eef0 legacy=0x7ff3c40b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 -- 172.21.15.179:0/340301246 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40b3d60 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c37fe700 1 --2- 172.21.15.179:0/340301246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cabc0 tx=0x7ff3ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca5d07ba1a12aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c2ffd700 1 -- 172.21.15.179:0/340301246 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac011830 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.709 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c2ffd700 1 -- 172.21.15.179:0/340301246 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac011990 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c2ffd700 1 -- 172.21.15.179:0/340301246 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac011d20 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/340301246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c40ad9d0 msgr2=0x7ff3c415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.710 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/340301246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cabc0 tx=0x7ff3ac009bf0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/340301246 shutdown_connections 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/340301246 >> v2:172.21.15.195:3300/0 conn(0x7ff3c40b3180 0x7ff3c40cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/340301246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c40ad9d0 0x7ff3c415e9b0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/340301246 >> 172.21.15.179:0/340301246 conn(0x7ff3c40ce960 msgr2=0x7ff3c40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/340301246 shutdown_connections 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.706+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/340301246 wait complete. 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 Processor -- start 2024-05-08T22:45:35.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 -- start start 2024-05-08T22:45:35.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c40ad9d0 0x7ff3c41e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c40b3180 0x7ff3c41524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c4138080 con 0x7ff3c415eef0 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41e5710 con 0x7ff3c40b3180 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c8ed5700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c414bc20 con 0x7ff3c40ad9d0 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3bbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c40b3180 0x7ff3c41524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46512/0 (socket says 172.21.15.179:46512) 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 -- 172.21.15.179:0/4019221687 learned_addr learned my addr 172.21.15.179:0/4019221687 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c37fe700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c40ad9d0 0x7ff3c41e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 -- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c40ad9d0 msgr2=0x7ff3c41e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c40ad9d0 0x7ff3c41e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 -- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c40b3180 msgr2=0x7ff3c41524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c40b3180 0x7ff3c41524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 -- 172.21.15.179:0/4019221687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac009570 con 0x7ff3c415eef0 2024-05-08T22:45:35.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c3fff700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00efc0 tx=0x7ff3bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc009ea0 con 0x7ff3c415eef0 2024-05-08T22:45:35.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.710+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00f040 con 0x7ff3c415eef0 2024-05-08T22:45:35.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c40adf30 con 0x7ff3c415eef0 2024-05-08T22:45:35.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00ced0 con 0x7ff3c415eef0 2024-05-08T22:45:35.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c41e5c40 con 0x7ff3c415eef0 2024-05-08T22:45:35.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c404e700 con 0x7ff3c415eef0 2024-05-08T22:45:35.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3bc014480 con 0x7ff3c415eef0 2024-05-08T22:45:35.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c0ff9700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 0x7ff3a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:35.717 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff3bc084940 con 0x7ff3c415eef0 2024-05-08T22:45:35.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.714+0000 7ff3c37fe700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 0x7ff3a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:35.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.718+0000 7ff3c37fe700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 0x7ff3a4068bf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:35.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.722+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff3bc051ca0 con 0x7ff3c415eef0 2024-05-08T22:45:35.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.890+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3c40addb0 con 0x7ff3a4066730 2024-05-08T22:45:35.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.890+0000 7ff3c0ff9700 1 -- 172.21.15.179:0/4019221687 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff3ac011b80 con 0x7ff3a4066730 2024-05-08T22:45:35.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 msgr2=0x7ff3a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 0x7ff3a4068bf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac008030 comp rx=0 tx=0).stop 2024-05-08T22:45:35.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 msgr2=0x7ff3c4152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:35.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00efc0 tx=0x7ff3bc00c3a0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 shutdown_connections 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7ff3a4066730 0x7ff3a4068bf0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c40b3180 0x7ff3c41524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3c415eef0 0x7ff3c4152a20 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 --2- 172.21.15.179:0/4019221687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c40ad9d0 0x7ff3c41e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 >> 172.21.15.179:0/4019221687 conn(0x7ff3c40ce960 msgr2=0x7ff3c40cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 shutdown_connections 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:35.894+0000 7ff3c8ed5700 1 -- 172.21.15.179:0/4019221687 wait complete. 2024-05-08T22:45:35.896 INFO:teuthology.orchestra.run.smithi179.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T22:45:35.912 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 Processor -- start 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 -- start start 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f187c1435d0 0x7f187c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c0b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c1439b0 con 0x7f187c06ccd0 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0ce240 con 0x7f187c1435d0 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1882778700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c06efb0 con 0x7f187c06d840 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f187c1435d0 0x7f187c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f187c1435d0 0x7f187c06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46520/0 (socket says 172.21.15.179:46520) 2024-05-08T22:45:36.024 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 -- 172.21.15.179:0/3384286221 learned_addr learned my addr 172.21.15.179:0/3384286221 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881f77700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c0b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 -- 172.21.15.179:0/3384286221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 msgr2=0x7f187c0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c0b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 -- 172.21.15.179:0/3384286221 >> v1:172.21.15.195:6789/0 conn(0x7f187c06ccd0 legacy=0x7f187c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 -- 172.21.15.179:0/3384286221 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0cd910 con 0x7f187c1435d0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.018+0000 7f1881776700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06e8b0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f1878009910 tx=0x7f1878009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3177fb0d3226eecc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f186ffff700 1 -- 172.21.15.179:0/3384286221 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187800ba60 con 0x7f187c1435d0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f186ffff700 1 -- 172.21.15.179:0/3384286221 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187800bbc0 con 0x7f187c1435d0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f186ffff700 1 -- 172.21.15.179:0/3384286221 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878009e60 con 0x7f187c1435d0 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- 172.21.15.179:0/3384286221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 msgr2=0x7f187c06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06e8b0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f1878009910 tx=0x7f1878009c20 comp rx=0 tx=0).stop 2024-05-08T22:45:36.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- 172.21.15.179:0/3384286221 shutdown_connections 2024-05-08T22:45:36.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c0b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- 172.21.15.179:0/3384286221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06e8b0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- 172.21.15.179:0/3384286221 >> 172.21.15.179:0/3384286221 conn(0x7f187c0ce9d0 msgr2=0x7f187c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:36.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- 172.21.15.179:0/3384286221 shutdown_connections 2024-05-08T22:45:36.026 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- 172.21.15.179:0/3384286221 wait complete. 2024-05-08T22:45:36.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 Processor -- start 2024-05-08T22:45:36.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- start start 2024-05-08T22:45:36.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f187c06ccd0 0x7f187c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c13bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c13ccf0 con 0x7f187c1435d0 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c1f4680 con 0x7f187c06ccd0 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.022+0000 7f1882778700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c1f4960 con 0x7f187c06d840 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46524/0 (socket says 172.21.15.179:46524) 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 -- 172.21.15.179:0/2329668857 learned_addr learned my addr 172.21.15.179:0/2329668857 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881776700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f187c06ccd0 0x7f187c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1880f75700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c13bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 -- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 msgr2=0x7f187c13bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c13bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 -- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f187c06ccd0 msgr2=0x7f187c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f187c06ccd0 0x7f187c157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 -- 172.21.15.179:0/2329668857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878009570 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1881f77700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1874007e30 tx=0x7f187400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400ed30 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400ee90 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c1f4c40 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874014660 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c13ca20 con 0x7f187c1435d0 2024-05-08T22:45:36.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.026+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c13ce60 con 0x7f187c1435d0 2024-05-08T22:45:36.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.030+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18740147c0 con 0x7f187c1435d0 2024-05-08T22:45:36.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.030+0000 7f186dffb700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 0x7f1860068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:36.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.030+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1874084860 con 0x7f187c1435d0 2024-05-08T22:45:36.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.030+0000 7f1881776700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 0x7f1860068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:36.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.030+0000 7f1881776700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 0x7f1860068bf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1878009c60 tx=0x7f1878008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:36.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.038+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f187c13ce60 con 0x7f187c1435d0 2024-05-08T22:45:36.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.202+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f187c0cae30 con 0x7f1860066730 2024-05-08T22:45:36.210 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f186dffb700 1 -- 172.21.15.179:0/2329668857 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49531 (secure 0 0 0) 0x7f187c0cae30 con 0x7f1860066730 2024-05-08T22:45:36.212 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:36.213 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 msgr2=0x7f1860068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 0x7f1860068bf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1878009c60 tx=0x7f1878008030 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 msgr2=0x7f187c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1874007e30 tx=0x7f187400e3c0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 shutdown_connections 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1860066730 0x7f1860068bf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f187c1435d0 0x7f187c06b440 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f187c06d840 0x7f187c13bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 --2- 172.21.15.179:0/2329668857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f187c06ccd0 0x7f187c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 >> 172.21.15.179:0/2329668857 conn(0x7f187c0ce9d0 msgr2=0x7f187c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 shutdown_connections 2024-05-08T22:45:36.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:36.210+0000 7f1882778700 1 -- 172.21.15.179:0/2329668857 wait complete. 2024-05-08T22:45:36.215 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:45:36.227 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-05-08T22:45:35.646203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275224,"kb_used_data":13992,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754699496,"statfs":{"total":773094113280,"available":772812283904,"internally_reserved":0,"allocated":14327808,"data_stored":7267080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.569685"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":269,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-08T22:45:27.082316+0000","last_change":"2024-05-08T22:33:10.737621+0000","last_active":"2024-05-08T22:45:27.082316+0000","last_peered":"2024-05-08T22:45:27.082316+0000","last_clean":"2024-05-08T22:45:27.082316+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:45:27.082316+0000","last_undegraded":"2024-05-08T22:45:27.082316+0000","last_fullsized":"2024-05-08T22:45:27.082316+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:33:09.461749+0000","last_clean_scrub_stamp":"2024-05-08T22:33:09.461749+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:57.517709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.23699999999999999},"max":{"1min":0.91600000000000004,"5min":1.151,"15min":1.623},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":1.115,"15min":1.6899999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":1.113,"15min":1.1639999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.32900000000000001},"max":{"1min":1.2390000000000001,"5min":1.3129999999999999,"15min":2.5049999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97299999999999998,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.016},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":1.716},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.47599999999999998},"max":{"1min":1.2370000000000001,"5min":1.2929999999999999,"15min":1.306},"last":0.75},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.345,"15min":1.4139999999999999},"last":1.145}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.218,"5min":1.429,"15min":1.429},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.038,"15min":1.0269999999999999},"min":{"1min":0.748,"5min":0.67800000000000005,"15min":0.56899999999999995},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.8740000000000001},"last":0.85099999999999998}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.77900000000000003,"5min":0.58599999999999997,"15min":0.442},"max":{"1min":1.2909999999999999,"5min":1.421,"15min":1.446},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.016,"5min":1.038,"15min":1.0529999999999999},"min":{"1min":0.70799999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.321,"5min":1.512,"15min":1.512},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8100000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.042},"min":{"1min":0.76400000000000001,"5min":0.629,"15min":0.55900000000000005},"max":{"1min":1.3300000000000001,"5min":1.476,"15min":1.8360000000000001},"last":0.66300000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":1.772,"15min":1.772},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72599999999999998,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.39700000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93300000000000005,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":1.0349999999999999,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":1.24,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.89500000000000002,"5min":1.1579999999999999,"15min":1.583},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.74399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.242,"5min":1.395,"15min":1.431},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.018,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.262,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":0.55900000000000005}]},{"osd":5,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.70099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3029999999999999,"5min":1.389,"15min":1.5},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1,"15min":1.038},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2929999999999999,"5min":1.399,"15min":1.726},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.99099999999999999,"15min":0.98599999999999999},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.331,"5min":1.351,"15min":1.4650000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0760000000000001},"min":{"1min":0.69799999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.361,"5min":1.4299999999999999,"15min":1.6850000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Wed May 8 22:44:35 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0920000000000001,"15min":1.073},"min":{"1min":0.85199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.335,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.095,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"max":{"1min":1.3759999999999999,"5min":1.446,"15min":1.5049999999999999},"last":0.80500000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542295,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37396,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334444,"statfs":{"total":96636764160,"available":96598470656,"internally_reserved":0,"allocated":2236416,"data_stored":1351161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86399999999999999,"15min":0.84599999999999997},"min":{"1min":0.61199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.242,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89600000000000002,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.206,"5min":1.3,"15min":1.514},"last":1.1499999999999999}]},{"osd":1,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92100000000000004,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.443},"max":{"1min":1.117,"5min":1.415,"15min":1.488},"last":1.23}]},{"osd":2,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92800000000000005,"15min":0.90600000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58199999999999996},"max":{"1min":1.2230000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.53800000000000003},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.337},"last":1.0580000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.92700000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.53900000000000003},"max":{"1min":1.1619999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0169999999999999,"5min":1.252,"15min":1.252},"last":1.0249999999999999}]},{"osd":4,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.94699999999999995},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.2769999999999999,"5min":1.571,"15min":1.611},"last":1.3100000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.63300000000000001,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.105,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92000000000000004,"15min":0.94099999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53300000000000003},"max":{"1min":1.304,"5min":1.681,"15min":1.681},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.077,"15min":1.056},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.383}]},{"osd":6,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95099999999999996},"min":{"1min":0.70299999999999996,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.1879999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.036,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4319999999999999,"5min":1.4990000000000001,"15min":1.599},"last":0.97999999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84899999999999998,"15min":0.83199999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.38200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.4550000000000001},"last":0.622},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.40500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1899999999999999,"15min":1.758},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.879,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.171,"5min":1.214,"15min":1.4319999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.89800000000000002},"min":{"1min":0.55700000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4199999999999999,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":0.999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.94599999999999995,"15min":0.94699999999999995},"min":{"1min":0.58399999999999996,"5min":0.57199999999999995,"15min":0.41799999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.5029999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.95299999999999996,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1439999999999999,"5min":1.3340000000000001,"15min":1.538},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.96799999999999997,"15min":0.98199999999999998},"min":{"1min":0.79400000000000004,"5min":0.57599999999999996,"15min":0.44400000000000001},"max":{"1min":1.3480000000000001,"5min":1.494,"15min":1.494},"last":0.76600000000000001}]},{"osd":5,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0349999999999999,"15min":1.0700000000000001},"min":{"1min":0.90800000000000003,"5min":0.67300000000000004,"15min":0.64000000000000001},"max":{"1min":1.2370000000000001,"5min":1.3460000000000001,"15min":1.681},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.0740000000000001},"min":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.65000000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.6000000000000001},"last":0.98699999999999999}]},{"osd":7,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.96899999999999997,"15min":1},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3009999999999999,"5min":1.571,"15min":1.835},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.111,"5min":1.127,"15min":1.163},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.74099999999999999},"max":{"1min":1.304,"5min":1.649,"15min":2.2999999999999998},"last":0.875}]}]},{"osd":5,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.84699999999999998},"min":{"1min":0.64500000000000002,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1499999999999999,"5min":1.1799999999999999,"15min":7.907},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.872,"15min":0.84099999999999997},"min":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.133,"5min":1.4670000000000001,"15min":1.52},"last":1.1299999999999999}]},{"osd":1,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83799999999999997,"15min":0.85999999999999999},"min":{"1min":0.57299999999999995,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.24,"5min":1.2410000000000001,"15min":2.4279999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.872,"5min":0.879,"15min":0.86299999999999999},"min":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.1719999999999999,"5min":1.389,"15min":6.0709999999999997},"last":0.69199999999999995}]},{"osd":2,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":4.1790000000000003},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90400000000000003,"15min":0.91500000000000004},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.48799999999999999},"max":{"1min":1.097,"5min":1.212,"15min":10.121},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.878,"15min":0.88400000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.501},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.96799999999999997,"15min":0.96399999999999997},"min":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.35599999999999998},"max":{"1min":1.1919999999999999,"5min":1.262,"15min":1.298},"last":1.1819999999999999}]},{"osd":4,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.92900000000000005,"15min":0.98599999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.46300000000000002},"max":{"1min":1.2729999999999999,"5min":1.379,"15min":15.135},"last":0.997},{"interface":"front","average":{"1min":0.998,"5min":1,"15min":1.0209999999999999},"min":{"1min":0.71499999999999997,"5min":0.68799999999999994,"15min":0.52000000000000002},"max":{"1min":1.2629999999999999,"5min":1.3680000000000001,"15min":4.319},"last":1.0700000000000001}]},{"osd":6,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":0.95799999999999996},"min":{"1min":0.70899999999999996,"5min":0.628,"15min":0.47599999999999998},"max":{"1min":1.304,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.236},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0780000000000001,"15min":1.0629999999999999},"min":{"1min":0.871,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.80500000000000005,"5min":0.76000000000000001,"15min":0.57299999999999995},"max":{"1min":1.45,"5min":1.45,"15min":4.1559999999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.1180000000000001,"15min":1.077},"min":{"1min":0.83599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.48,"5min":1.7350000000000001,"15min":2.2669999999999999},"last":1.292}]}]},{"osd":4,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.44600000000000001},"max":{"1min":1.135,"5min":1.417,"15min":1.417},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.41199999999999998},"max":{"1min":1.194,"5min":1.506,"15min":1.506},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.89100000000000001},"min":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.53100000000000003},"max":{"1min":1.163,"5min":1.179,"15min":1.3140000000000001},"last":0.74299999999999999}]},{"osd":2,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.877},"min":{"1min":0.64400000000000002,"5min":0.55700000000000005,"15min":0.496},"max":{"1min":1.1679999999999999,"5min":1.538,"15min":1.538},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.637},"last":0.88300000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95699999999999996,"15min":0.97599999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2070000000000001,"5min":1.2769999999999999,"15min":2.601},"last":1.077},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.68700000000000006,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":2.8769999999999998},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.95499999999999996},"min":{"1min":0.64700000000000002,"5min":0.621,"15min":0.47899999999999998},"max":{"1min":1.3200000000000001,"5min":1.454,"15min":1.792},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1699999999999999,"5min":1.2949999999999999,"15min":1.575},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96499999999999997,"15min":1.0189999999999999},"min":{"1min":0.65700000000000003,"5min":0.624,"15min":0.58599999999999997},"max":{"1min":1.29,"5min":1.331,"15min":1.831},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0649999999999999,"15min":1.0720000000000001},"min":{"1min":0.89400000000000002,"5min":0.66900000000000004,"15min":0.63300000000000001},"max":{"1min":1.373,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":1.1990000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.0409999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.39200000000000002},"max":{"1min":1.262,"5min":1.403,"15min":1.8500000000000001},"last":1.161},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1040000000000001,"15min":1.1359999999999999},"min":{"1min":0.80600000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.415,"5min":1.4299999999999999,"15min":1.944},"last":1.2430000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.95499999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.2190000000000001},"last":0.85199999999999998}]},{"osd":1,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.97399999999999998,"15min":1.048},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.40400000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.3460000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Wed May 8 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.155,"15min":1.155},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63600000000000001,"5min":0.53200000000000003,"15min":0.27300000000000002},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.995}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.75700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.361,"15min":1.504},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"min":{"1min":0.72499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.226,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.95599999999999996,"15min":0.999},"min":{"1min":0.63100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.3129999999999999,"15min":1.484},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.01,"5min":0.97999999999999998,"15min":0.996},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.67200000000000004},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.456},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.01,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69599999999999995},"max":{"1min":1.3740000000000001,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":1.044},{"interface":"front","average":{"1min":1.054,"5min":1.018,"15min":1.0549999999999999},"min":{"1min":0.81399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.161}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.85499999999999998,"5min":0.71199999999999997,"15min":0.39300000000000002},"max":{"1min":1.3400000000000001,"5min":1.498,"15min":1.5409999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0580000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.139}]}]},{"osd":0,"up_from":14,"seq":60129542295,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2236416,"data_stored":1351161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":1.0209999999999999,"15min":1.617},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.67500000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.5129999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.0069999999999999,"5min":1.052,"15min":1.446},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":1.101,"15min":1.6910000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed May 8 22:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.746,"15min":0.77700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.1020000000000001,"15min":1.714},"last":0.505},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":1.3859999999999999},"last":0.53100000000000003}]},{"osd":4,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.61699999999999999},"max":{"1min":1.0489999999999999,"5min":1.4850000000000001,"15min":4.391},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89800000000000002,"15min":0.90100000000000002},"min":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0269999999999999,"5min":1.246,"15min":1.419},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.93700000000000006,"15min":0.97099999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95599999999999996}]},{"osd":6,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91700000000000004,"15min":0.95099999999999996},"min":{"1min":0.69799999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":1.028},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0389999999999999,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.042,"15min":1.0609999999999999},"min":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99299999999999999,"15min":1.0129999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.60399999999999998},"max":{"1min":1.274,"5min":1.3200000000000001,"15min":1.514},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:45:36.233 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T22:33:09.461749+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=22, tm_min=33, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=22, tm_min=45, tm_sec=30, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T22:45:36.233 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T22:45:53.144 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:45:53.147 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:45:56.235 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:45:56.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.349+0000 7f5df827c700 1 Processor -- start 2024-05-08T22:45:56.355 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- start start 2024-05-08T22:45:56.356 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5df00ce060 0x7f5df006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5df00cc310 0x7f5df00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00add40 0x7f5df006b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df00ae550 con 0x7f5df00cc310 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df006abb0 con 0x7f5df00ce060 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0161ac0 con 0x7f5df00add40 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5df00cc310 0x7f5df00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5df00cc310 0x7f5df00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34112/0 (socket says 172.21.15.179:34112) 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 -- 172.21.15.179:0/3752945836 learned_addr learned my addr 172.21.15.179:0/3752945836 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df6018700 1 --2- 172.21.15.179:0/3752945836 >> v2:172.21.15.195:3300/0 conn(0x7f5df00ce060 0x7f5df006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df6819700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00add40 0x7f5df006b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 -- 172.21.15.179:0/3752945836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00add40 msgr2=0x7f5df006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00add40 0x7f5df006b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 -- 172.21.15.179:0/3752945836 >> v2:172.21.15.195:3300/0 conn(0x7f5df00ce060 msgr2=0x7f5df006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:45:56.357 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 --2- 172.21.15.179:0/3752945836 >> v2:172.21.15.195:3300/0 conn(0x7f5df00ce060 0x7f5df006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 -- 172.21.15.179:0/3752945836 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df00ce980 con 0x7f5df00cc310 2024-05-08T22:45:56.358 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5817700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00cc310 0x7f5df00ce440 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f5df0074410 tx=0x7f5dec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61ceac85083f3fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5016700 1 -- 172.21.15.179:0/3752945836 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00e070 con 0x7f5df00cc310 2024-05-08T22:45:56.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5016700 1 -- 172.21.15.179:0/3752945836 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec00b9c0 con 0x7f5df00cc310 2024-05-08T22:45:56.359 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df5016700 1 -- 172.21.15.179:0/3752945836 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00bd70 con 0x7f5df00cc310 2024-05-08T22:45:56.360 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- 172.21.15.179:0/3752945836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00cc310 msgr2=0x7f5df00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00cc310 0x7f5df00ce440 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f5df0074410 tx=0x7f5dec00b330 comp rx=0 tx=0).stop 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- 172.21.15.179:0/3752945836 shutdown_connections 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00add40 0x7f5df006b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- 172.21.15.179:0/3752945836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00cc310 0x7f5df00ce440 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 --2- 172.21.15.179:0/3752945836 >> v2:172.21.15.195:3300/0 conn(0x7f5df00ce060 0x7f5df006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- 172.21.15.179:0/3752945836 >> 172.21.15.179:0/3752945836 conn(0x7f5df00cb660 msgr2=0x7f5df00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- 172.21.15.179:0/3752945836 shutdown_connections 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.353+0000 7f5df827c700 1 -- 172.21.15.179:0/3752945836 wait complete. 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 Processor -- start 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- start start 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5df00add40 0x7f5df00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00cc310 0x7f5df0160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.361 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df00ac7e0 con 0x7f5df00ce060 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df00b36d0 con 0x7f5df00add40 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01fad60 con 0x7f5df00cc310 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6018700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5df00add40 0x7f5df00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34122/0 (socket says 172.21.15.179:34122) 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 -- 172.21.15.179:0/2796133405 learned_addr learned my addr 172.21.15.179:0/2796133405 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:56.362 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df5817700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00cc310 0x7f5df0160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 -- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00cc310 msgr2=0x7f5df0160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00cc310 0x7f5df0160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 -- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5df00add40 msgr2=0x7f5df00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5df00add40 0x7f5df00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 -- 172.21.15.179:0/2796133405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec009b30 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df6819700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d800 tx=0x7f5de800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80125f0 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8012770 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df01fb040 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8012b20 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.357+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df00b2df0 con 0x7f5df00ce060 2024-05-08T22:45:56.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df0067770 con 0x7f5df00ce060 2024-05-08T22:45:56.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5de8012c80 con 0x7f5df00ce060 2024-05-08T22:45:56.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5de6ffd700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 0x7f5dd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f5de8083f50 con 0x7f5df00ce060 2024-05-08T22:45:56.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5df6018700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 0x7f5dd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.361+0000 7f5df6018700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 0x7f5dd4068bf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5df00c8850 tx=0x7f5de0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.369+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5de80513c0 con 0x7f5df00ce060 2024-05-08T22:45:56.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.529+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 --> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5df006b0a0 con 0x7f5dd4066730 2024-05-08T22:45:56.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.537+0000 7f5de6ffd700 1 -- 172.21.15.179:0/2796133405 <== mgr.4105 v2:172.21.15.195:6832/25951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49399 (secure 0 0 0) 0x7f5de0011750 con 0x7f5dd4066730 2024-05-08T22:45:56.541 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.537+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 msgr2=0x7f5dd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.537+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 0x7f5dd4068bf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5df00c8850 tx=0x7f5de0008040 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.537+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 msgr2=0x7f5df01fa1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.537+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d800 tx=0x7f5de800db10 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 shutdown_connections 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f5dd4066730 0x7f5dd4068bf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5df00ce060 0x7f5df01fa1f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5df00cc310 0x7f5df0160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 --2- 172.21.15.179:0/2796133405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5df00add40 0x7f5df00b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 >> 172.21.15.179:0/2796133405 conn(0x7f5df00cb660 msgr2=0x7f5df006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 shutdown_connections 2024-05-08T22:45:56.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.541+0000 7f5df827c700 1 -- 172.21.15.179:0/2796133405 wait complete. 2024-05-08T22:45:56.544 INFO:teuthology.orchestra.run.smithi179.stderr:dumped all 2024-05-08T22:45:56.555 INFO:teuthology.orchestra.run.smithi179.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-05-08T22:45:55.651531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275216,"kb_used_data":13992,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754699504,"statfs":{"total":773094113280,"available":772812292096,"internally_reserved":0,"allocated":14327808,"data_stored":7267080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":267374136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003247"},"pg_stats":[{"pgid":"1.0","version":"132'119","reported_seq":283,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-08T22:45:37.654048+0000","last_change":"2024-05-08T22:45:37.654375+0000","last_active":"2024-05-08T22:45:37.654048+0000","last_peered":"2024-05-08T22:45:37.654048+0000","last_clean":"2024-05-08T22:45:37.654048+0000","last_became_active":"2024-05-08T22:33:10.735877+0000","last_became_peered":"2024-05-08T22:33:10.735877+0000","last_unstale":"2024-05-08T22:45:37.654048+0000","last_undegraded":"2024-05-08T22:45:37.654048+0000","last_fullsized":"2024-05-08T22:45:37.654048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'119","last_scrub_stamp":"2024-05-08T22:45:37.653939+0000","last_deep_scrub":"132'119","last_deep_scrub_stamp":"2024-05-08T22:45:37.653939+0000","last_clean_scrub_stamp":"2024-05-08T22:45:37.653939+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:51.764925+0000","scrub_duration":0.010050651000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.23699999999999999},"max":{"1min":0.91600000000000004,"5min":1.151,"15min":1.623},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":1.115,"15min":1.6899999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":1.113,"15min":1.1639999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.32900000000000001},"max":{"1min":1.2390000000000001,"5min":1.3129999999999999,"15min":2.5049999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97299999999999998,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":1.716},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.47599999999999998},"max":{"1min":1.2370000000000001,"5min":1.2929999999999999,"15min":1.306},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.345,"15min":1.4139999999999999},"last":0.89400000000000002}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.71499999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.218,"5min":1.429,"15min":1.429},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.038,"15min":1.0269999999999999},"min":{"1min":0.748,"5min":0.67800000000000005,"15min":0.56899999999999995},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.8740000000000001},"last":1.1759999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.77900000000000003,"5min":0.58599999999999997,"15min":0.442},"max":{"1min":1.2909999999999999,"5min":1.421,"15min":1.446},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.038,"15min":1.0529999999999999},"min":{"1min":0.70799999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.321,"5min":1.512,"15min":1.512},"last":1.0960000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.8100000000000001},"last":1.256},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.042},"min":{"1min":0.76400000000000001,"5min":0.629,"15min":0.55900000000000005},"max":{"1min":1.3300000000000001,"5min":1.476,"15min":1.8360000000000001},"last":1.073}]}]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98399999999999999,"15min":1.772},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.51000000000000001,"5min":0.433,"15min":0.33300000000000002},"max":{"1min":1.089,"5min":1.1659999999999999,"15min":1.8740000000000001},"last":0.80300000000000005}]},{"osd":2,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":3.3570000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":1.23,"15min":5.5380000000000003},"last":0.61799999999999999}]},{"osd":3,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82999999999999996,"15min":0.80600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.371},"max":{"1min":1.2230000000000001,"5min":1.24,"15min":5.5099999999999998},"last":1.115},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73399999999999999,"15min":0.70099999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.35599999999999998},"max":{"1min":0.93799999999999994,"5min":1.1579999999999999,"15min":1.583},"last":0.67500000000000004}]},{"osd":4,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.94199999999999995},"min":{"1min":0.72099999999999997,"5min":0.56999999999999995,"15min":0.34699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.431},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97499999999999998,"15min":0.98599999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1439999999999999,"5min":1.444,"15min":5.1890000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.94399999999999995},"min":{"1min":0.66800000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2729999999999999,"5min":1.389,"15min":1.5},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.03},"min":{"1min":0.77300000000000002,"5min":0.65800000000000003,"15min":0.627},"max":{"1min":1.3069999999999999,"5min":1.399,"15min":1.726},"last":1.1299999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.98699999999999999,"15min":0.98099999999999998},"min":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60899999999999999},"max":{"1min":1.274,"5min":1.351,"15min":1.4650000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.052,"15min":1.0700000000000001},"min":{"1min":0.76700000000000002,"5min":0.69799999999999995,"15min":0.44700000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.6850000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Wed May 8 22:45:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.083,"15min":1.071},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.73299999999999998},"max":{"1min":1.3400000000000001,"5min":1.3580000000000001,"15min":1.9119999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0529999999999999},"min":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.65800000000000003},"max":{"1min":1.383,"5min":1.446,"15min":1.5049999999999999},"last":0.99199999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542299,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":2236416,"data_stored":1351161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86399999999999999,"15min":0.84599999999999997},"min":{"1min":0.61199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.242,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.167},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89600000000000002,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.46999999999999997},"max":{"1min":1.206,"5min":1.3,"15min":1.514},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.2170000000000001,"15min":1.4470000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92100000000000004,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.443},"max":{"1min":1.117,"5min":1.415,"15min":1.488},"last":0.80900000000000005}]},{"osd":2,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92800000000000005,"15min":0.90600000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58199999999999996},"max":{"1min":1.2230000000000001,"5min":1.3400000000000001,"15min":1.4279999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.53800000000000003},"max":{"1min":1.1679999999999999,"5min":1.256,"15min":1.337},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.92700000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.53900000000000003},"max":{"1min":1.1619999999999999,"5min":1.3879999999999999,"15min":1.5589999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0169999999999999,"5min":1.252,"15min":1.252},"last":0.78300000000000003}]},{"osd":4,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.94699999999999995},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54700000000000004},"max":{"1min":1.2769999999999999,"5min":1.571,"15min":1.611},"last":0.998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.89300000000000002},"min":{"1min":0.63300000000000001,"5min":0.622,"15min":0.57399999999999995},"max":{"1min":1.105,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92000000000000004,"15min":0.94099999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53300000000000003},"max":{"1min":1.304,"5min":1.681,"15min":1.681},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.077,"15min":1.056},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed May 8 22:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95099999999999996},"min":{"1min":0.70299999999999996,"5min":0.68000000000000005,"15min":0.65000000000000002},"max":{"1min":1.1879999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.70599999999999996},"max":{"1min":1.4319999999999999,"5min":1.4990000000000001,"15min":1.599},"last":1.2529999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542299,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84899999999999998,"15min":0.83199999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.38200000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.4550000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.044}]},{"osd":1,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.40500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1899999999999999,"15min":1.758},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.3959999999999999,"15min":1.8089999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.879,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.498,"15min":0.49099999999999999},"max":{"1min":1.171,"5min":1.214,"15min":1.4319999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.89800000000000002},"min":{"1min":0.55700000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4199999999999999,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.88700000000000001}]},{"osd":3,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":0.999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":3.2040000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.94599999999999995,"15min":0.94699999999999995},"min":{"1min":0.58399999999999996,"5min":0.57199999999999995,"15min":0.41799999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.5029999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.95299999999999996,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.61599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1439999999999999,"5min":1.3340000000000001,"15min":1.538},"last":0.87},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.96799999999999997,"15min":0.98199999999999998},"min":{"1min":0.79400000000000004,"5min":0.57599999999999996,"15min":0.44400000000000001},"max":{"1min":1.3480000000000001,"5min":1.494,"15min":1.494},"last":1.006}]},{"osd":5,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0349999999999999,"15min":1.0700000000000001},"min":{"1min":0.90800000000000003,"5min":0.67300000000000004,"15min":0.64000000000000001},"max":{"1min":1.2370000000000001,"5min":1.3460000000000001,"15min":1.681},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.05,"15min":1.0740000000000001},"min":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.65000000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.6000000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed May 8 22:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.96899999999999997,"15min":1},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3009999999999999,"5min":1.571,"15min":1.835},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.127,"15min":1.163},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.74099999999999999},"max":{"1min":1.304,"5min":1.649,"15min":2.2999999999999998},"last":1.107}]}]},{"osd":5,"up_from":14,"seq":60129542299,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.84699999999999998},"min":{"1min":0.64500000000000002,"5min":0.439,"15min":0.33000000000000002},"max":{"1min":1.1499999999999999,"5min":1.1799999999999999,"15min":7.907},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.872,"15min":0.84099999999999997},"min":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.40300000000000002},"max":{"1min":1.133,"5min":1.4670000000000001,"15min":1.52},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83799999999999997,"15min":0.85999999999999999},"min":{"1min":0.57299999999999995,"5min":0.48399999999999999,"15min":0.35199999999999998},"max":{"1min":1.24,"5min":1.2410000000000001,"15min":2.4279999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.879,"15min":0.86299999999999999},"min":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.44400000000000001},"max":{"1min":1.1719999999999999,"5min":1.389,"15min":6.0709999999999997},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":4.1790000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90400000000000003,"15min":0.91500000000000004},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.48799999999999999},"max":{"1min":1.097,"5min":1.212,"15min":10.121},"last":1.0049999999999999}]},{"osd":3,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.878,"15min":0.88400000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.501},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.96799999999999997,"15min":0.96399999999999997},"min":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.35599999999999998},"max":{"1min":1.1919999999999999,"5min":1.262,"15min":1.298},"last":0.83299999999999996}]},{"osd":4,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.92900000000000005,"15min":0.98599999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.46300000000000002},"max":{"1min":1.2729999999999999,"5min":1.379,"15min":15.135},"last":1.109},{"interface":"front","average":{"1min":0.998,"5min":1,"15min":1.0209999999999999},"min":{"1min":0.71499999999999997,"5min":0.68799999999999994,"15min":0.52000000000000002},"max":{"1min":1.2629999999999999,"5min":1.3680000000000001,"15min":4.319},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":0.95799999999999996},"min":{"1min":0.70899999999999996,"5min":0.628,"15min":0.47599999999999998},"max":{"1min":1.304,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.749},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0780000000000001,"15min":1.0629999999999999},"min":{"1min":0.871,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.048}]},{"osd":7,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.80500000000000005,"5min":0.76000000000000001,"15min":0.57299999999999995},"max":{"1min":1.45,"5min":1.45,"15min":4.1559999999999997},"last":1.083},{"interface":"front","average":{"1min":1.139,"5min":1.1180000000000001,"15min":1.077},"min":{"1min":0.83599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":1.48,"5min":1.7350000000000001,"15min":2.2669999999999999},"last":0.97999999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542299,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.44600000000000001},"max":{"1min":1.135,"5min":1.417,"15min":1.417},"last":1.1839999999999999}]},{"osd":1,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.41199999999999998},"max":{"1min":1.194,"5min":1.506,"15min":1.506},"last":0.878},{"interface":"front","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.89100000000000001},"min":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.53100000000000003},"max":{"1min":1.163,"5min":1.179,"15min":1.3140000000000001},"last":1.115}]},{"osd":2,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.877},"min":{"1min":0.64400000000000002,"5min":0.55700000000000005,"15min":0.496},"max":{"1min":1.1679999999999999,"5min":1.538,"15min":1.538},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.637},"last":1.018}]},{"osd":3,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95699999999999996,"15min":0.97599999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2070000000000001,"5min":1.2769999999999999,"15min":2.601},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.68700000000000006,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":2.8769999999999998},"last":0.999}]},{"osd":5,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":0.95499999999999996},"min":{"1min":0.64700000000000002,"5min":0.621,"15min":0.47899999999999998},"max":{"1min":1.3200000000000001,"5min":1.454,"15min":1.792},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1699999999999999,"5min":1.2949999999999999,"15min":1.575},"last":0.97499999999999998}]},{"osd":6,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96499999999999997,"15min":1.0189999999999999},"min":{"1min":0.65700000000000003,"5min":0.624,"15min":0.58599999999999997},"max":{"1min":1.29,"5min":1.331,"15min":1.831},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0649999999999999,"15min":1.0720000000000001},"min":{"1min":0.89400000000000002,"5min":0.66900000000000004,"15min":0.63300000000000001},"max":{"1min":1.373,"5min":1.5760000000000001,"15min":1.8899999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Wed May 8 22:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.0409999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.39200000000000002},"max":{"1min":1.262,"5min":1.403,"15min":1.8500000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1040000000000001,"15min":1.1359999999999999},"min":{"1min":0.80600000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.415,"5min":1.4299999999999999,"15min":1.944},"last":1.2030000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542300,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1642496,"data_stored":760793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.95499999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.2190000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed May 8 22:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.23899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.048},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2250000000000001,"15min":1.3460000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed May 8 22:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71899999999999997,"15min":0.69699999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.312},"max":{"1min":1.101,"5min":1.155,"15min":1.155},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.76600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.27300000000000002},"max":{"1min":1.0840000000000001,"5min":1.1719999999999999,"15min":1.4330000000000001},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.75700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.361,"15min":1.504},"last":1.069},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"min":{"1min":0.72499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.226,"5min":1.3100000000000001,"15min":1.9750000000000001},"last":1.2310000000000001}]},{"osd":5,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.95599999999999996,"15min":0.999},"min":{"1min":0.63100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.3129999999999999,"15min":1.484},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.01,"5min":0.97999999999999998,"15min":0.996},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.67200000000000004},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.456},"last":0.89600000000000002}]},{"osd":6,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.01,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69599999999999995},"max":{"1min":1.3740000000000001,"5min":1.4059999999999999,"15min":2.3220000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.018,"15min":1.0549999999999999},"min":{"1min":0.81399999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.4379999999999999,"15min":2.8330000000000002},"last":1.004}]},{"osd":7,"last update":"Wed May 8 22:45:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.85499999999999998,"5min":0.71199999999999997,"15min":0.39300000000000002},"max":{"1min":1.3400000000000001,"5min":1.498,"15min":1.5409999999999999},"last":1.141},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0580000000000001,"15min":1.091},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.165}]}]},{"osd":0,"up_from":14,"seq":60129542299,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2236416,"data_stored":1351161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58299999999999996,"15min":0.627},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70599999999999996,"5min":1.0209999999999999,"15min":1.617},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.44800000000000001,"5min":0.437,"15min":0.33600000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.5129999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.0069999999999999,"5min":1.052,"15min":1.446},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.436},"max":{"1min":0.98699999999999999,"5min":1.101,"15min":1.6910000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 22:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72999999999999998,"15min":0.77000000000000002},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":0.98999999999999999,"5min":1.0609999999999999,"15min":1.714},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.69899999999999995},"min":{"1min":0.45500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.3859999999999999},"last":0.86399999999999999}]},{"osd":4,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.61699999999999999},"max":{"1min":1.0489999999999999,"5min":1.4850000000000001,"15min":4.391},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1619999999999999}]},{"osd":5,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89800000000000002,"15min":0.90100000000000002},"min":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0269999999999999,"5min":1.246,"15min":1.419},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.93700000000000006,"15min":0.97099999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91700000000000004,"15min":0.95099999999999996},"min":{"1min":0.69799999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.8660000000000001},"last":1.01},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":1.028},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0389999999999999,"5min":1.5980000000000001,"15min":4.4859999999999998},"last":1.032}]},{"osd":7,"last update":"Wed May 8 22:45:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.042,"15min":1.0609999999999999},"min":{"1min":0.78800000000000003,"5min":0.76700000000000002,"15min":0.72199999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99299999999999999,"15min":1.0129999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.60399999999999998},"max":{"1min":1.274,"5min":1.3200000000000001,"15min":1.514},"last":1.5109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:45:56.557 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T22:45:56.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.669+0000 7f786f30e700 1 Processor -- start 2024-05-08T22:45:56.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.669+0000 7f786f30e700 1 -- start start 2024-05-08T22:45:56.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7868159550 0x7f786806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78680c5a70 con 0x7f78680c4180 2024-05-08T22:45:56.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786806ce80 con 0x7f78680b18b0 2024-05-08T22:45:56.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78680b1e00 con 0x7f7868159550 2024-05-08T22:45:56.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786e30c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7868159550 0x7f786806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786eb0d700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f78680c4180 0x7f786815d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36714/0 (socket says 172.21.15.179:36714) 2024-05-08T22:45:56.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786eb0d700 1 -- 172.21.15.179:0/1521344792 learned_addr learned my addr 172.21.15.179:0/1521344792 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:56.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786e30c700 1 -- 172.21.15.179:0/1521344792 >> v1:172.21.15.195:6789/0 conn(0x7f78680b18b0 legacy=0x7f786815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786e30c700 1 -- 172.21.15.179:0/1521344792 >> v1:172.21.15.179:6789/0 conn(0x7f78680c4180 legacy=0x7f786815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786d30a700 1 -- 172.21.15.179:0/1521344792 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102809132 0 0) 0x7f78680c5a70 con 0x7f78680c4180 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786e30c700 1 -- 172.21.15.179:0/1521344792 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786806c710 con 0x7f7868159550 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786e30c700 1 --2- 172.21.15.179:0/1521344792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7868159550 0x7f786806b440 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f786400a9f0 tx=0x7f786400ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d565168069bfae06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786d30a700 1 -- 172.21.15.179:0/1521344792 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400cb70 con 0x7f7868159550 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786d30a700 1 -- 172.21.15.179:0/1521344792 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400ccd0 con 0x7f7868159550 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786d30a700 1 -- 172.21.15.179:0/1521344792 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864018620 con 0x7f7868159550 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- 172.21.15.179:0/1521344792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7868159550 msgr2=0x7f786806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 --2- 172.21.15.179:0/1521344792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7868159550 0x7f786806b440 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f786400a9f0 tx=0x7f786400ad00 comp rx=0 tx=0).stop 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- 172.21.15.179:0/1521344792 shutdown_connections 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 --2- 172.21.15.179:0/1521344792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7868159550 0x7f786806b440 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- 172.21.15.179:0/1521344792 >> 172.21.15.179:0/1521344792 conn(0x7f7868098a70 msgr2=0x7f78680cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- 172.21.15.179:0/1521344792 shutdown_connections 2024-05-08T22:45:56.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.673+0000 7f786f30e700 1 -- 172.21.15.179:0/1521344792 wait complete. 2024-05-08T22:45:56.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 Processor -- start 2024-05-08T22:45:56.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- start start 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78680b18b0 0x7f78680ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7868159550 0x7f78681556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78680ad630 con 0x7f78680c4180 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78680aca20 con 0x7f78680b18b0 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680acd00 con 0x7f7868159550 2024-05-08T22:45:56.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34142/0 (socket says 172.21.15.179:34142) 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 -- 172.21.15.179:0/4141310351 learned_addr learned my addr 172.21.15.179:0/4141310351 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786e30c700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78680b18b0 0x7f78680ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786eb0d700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7868159550 0x7f78681556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 -- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7868159550 msgr2=0x7f78681556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7868159550 0x7f78681556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 -- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78680b18b0 msgr2=0x7f78680ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78680b18b0 0x7f78680ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 -- 172.21.15.179:0/4141310351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786400a650 con 0x7f78680c4180 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786db0b700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f785800c4d0 tx=0x7f785800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78580052d0 con 0x7f78680c4180 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78680acfe0 con 0x7f78680c4180 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858005430 con 0x7f78680c4180 2024-05-08T22:45:56.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78580055a0 con 0x7f78680c4180 2024-05-08T22:45:56.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.677+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78680ac730 con 0x7f78680c4180 2024-05-08T22:45:56.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.681+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78680ad7e0 con 0x7f78680c4180 2024-05-08T22:45:56.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.681+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78580038c0 con 0x7f78680c4180 2024-05-08T22:45:56.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.681+0000 7f785effd700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 0x7f784c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:56.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.681+0000 7f786e30c700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 0x7f784c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:56.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.681+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f7858083470 con 0x7f78680c4180 2024-05-08T22:45:56.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.685+0000 7f786e30c700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 0x7f784c068bf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f786400a1c0 tx=0x7f7864008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:56.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.693+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f78680ad7e0 con 0x7f78680c4180 2024-05-08T22:45:56.866 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.861+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f786804e700 con 0x7f78680c4180 2024-05-08T22:45:56.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.869+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7858050900 con 0x7f78680c4180 2024-05-08T22:45:56.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.869+0000 7f785effd700 1 -- 172.21.15.179:0/4141310351 <== mon.0 v2:172.21.15.179:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f7858054390 con 0x7f78680c4180 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 msgr2=0x7f784c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 0x7f784c068bf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f786400a1c0 tx=0x7f7864008040 comp rx=0 tx=0).stop 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 msgr2=0x7f7868147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f785800c4d0 tx=0x7f785800c7e0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 shutdown_connections 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f784c066730 0x7f784c068bf0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7868159550 0x7f78681556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680c4180 0x7f7868147ab0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 --2- 172.21.15.179:0/4141310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78680b18b0 0x7f78680ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:56.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 >> 172.21.15.179:0/4141310351 conn(0x7f7868098a70 msgr2=0x7f78680caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:56.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 shutdown_connections 2024-05-08T22:45:56.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.873+0000 7f786f30e700 1 -- 172.21.15.179:0/4141310351 wait complete. 2024-05-08T22:45:56.890 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:45:56.890 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:45:57.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 Processor -- start 2024-05-08T22:45:57.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 -- start start 2024-05-08T22:45:57.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 0x7f1ac815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:57.002 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac8159b60 con 0x7f1ac81539c0 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac8159e40 con 0x7f1ac80b19e0 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:56.997+0000 7f1ac759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac815a120 con 0x7f1ac80b3720 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1ac81539c0 0x7f1ac806b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:36720/0 (socket says 172.21.15.179:36720) 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac659c700 1 -- 172.21.15.179:0/3086446355 learned_addr learned my addr 172.21.15.179:0/3086446355 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac6d9d700 1 --2- 172.21.15.179:0/3086446355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 0x7f1ac815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 <== mon.0 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365286980 0 0) 0x7f1ac8159b60 con 0x7f1ac81539c0 2024-05-08T22:45:57.003 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab40035d0 con 0x7f1ac81539c0 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595094531 0 0) 0x7f1ac8159e40 con 0x7f1ac80b19e0 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac8159b60 con 0x7f1ac80b19e0 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac6d9d700 1 -- 172.21.15.179:0/3086446355 >> v1:172.21.15.195:6789/0 conn(0x7f1ac80b19e0 legacy=0x7f1ac80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac6d9d700 1 -- 172.21.15.179:0/3086446355 >> v1:172.21.15.179:6789/0 conn(0x7f1ac81539c0 legacy=0x7f1ac806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac6d9d700 1 -- 172.21.15.179:0/3086446355 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac815a400 con 0x7f1ac80b3720 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac6d9d700 1 --2- 172.21.15.179:0/3086446355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 0x7f1ac815d120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009dc0 tx=0x7f1ac000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1c146548941b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000e070 con 0x7f1ac80b3720 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1ac000bb20 con 0x7f1ac80b3720 2024-05-08T22:45:57.005 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac5d9b700 1 -- 172.21.15.179:0/3086446355 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0012430 con 0x7f1ac80b3720 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 -- 172.21.15.179:0/3086446355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 msgr2=0x7f1ac815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 --2- 172.21.15.179:0/3086446355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 0x7f1ac815d120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009dc0 tx=0x7f1ac000b330 comp rx=0 tx=0).stop 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 -- 172.21.15.179:0/3086446355 shutdown_connections 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 --2- 172.21.15.179:0/3086446355 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac80b3720 0x7f1ac815d120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 -- 172.21.15.179:0/3086446355 >> 172.21.15.179:0/3086446355 conn(0x7f1ac80ce9d0 msgr2=0x7f1ac80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 -- 172.21.15.179:0/3086446355 shutdown_connections 2024-05-08T22:45:57.006 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.001+0000 7f1ac759e700 1 -- 172.21.15.179:0/3086446355 wait complete. 2024-05-08T22:45:57.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 Processor -- start 2024-05-08T22:45:57.007 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- start start 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac80b3720 0x7f1ac813b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 0x7f1ac813b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8135f90 con 0x7f1ac80b19e0 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac813a590 con 0x7f1ac80b3720 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac813a870 con 0x7f1ac81539c0 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34154/0 (socket says 172.21.15.179:34154) 2024-05-08T22:45:57.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 -- 172.21.15.179:0/993465294 learned_addr learned my addr 172.21.15.179:0/993465294 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac6d9d700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 0x7f1ac813b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1abdd9b700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac80b3720 0x7f1ac813b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 -- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 msgr2=0x7f1ac813b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 0x7f1ac813b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 -- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac80b3720 msgr2=0x7f1ac813b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac80b3720 0x7f1ac813b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 -- 172.21.15.179:0/993465294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0009a70 con 0x7f1ac80b19e0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac6d9d700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 0x7f1ac813b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac659c700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c4d0 tx=0x7f1ab800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8005d90 con 0x7f1ac80b19e0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1ab80070d0 con 0x7f1ac80b19e0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac813ab50 con 0x7f1ac80b19e0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8007460 con 0x7f1ac80b19e0 2024-05-08T22:45:57.011 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.005+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac813d970 con 0x7f1ac80b19e0 2024-05-08T22:45:57.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8067740 con 0x7f1ac80b19e0 2024-05-08T22:45:57.012 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ab80075c0 con 0x7f1ac80b19e0 2024-05-08T22:45:57.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1abf7fe700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 0x7f1aa8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:45:57.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1ab8083660 con 0x7f1ac80b19e0 2024-05-08T22:45:57.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1abdd9b700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 0x7f1aa8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:45:57.014 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.009+0000 7f1abdd9b700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 0x7f1aa8068bf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ce3b0 tx=0x7f1ab0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:45:57.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.017+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1ab8050a50 con 0x7f1ac80b19e0 2024-05-08T22:45:57.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.225+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ac804e700 con 0x7f1ac80b19e0 2024-05-08T22:45:57.228 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.225+0000 7f1abf7fe700 1 -- 172.21.15.179:0/993465294 <== mon.0 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3544 (secure 0 0 0) 0x7f1ab8017090 con 0x7f1ac80b19e0 2024-05-08T22:45:57.229 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-08T22:45:57.229 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4101835982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4101835982},{"type":"v1","addr":"172.21.15.195:6837","nonce":4101835982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3939920091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3939920091},{"type":"v1","addr":"172.21.15.179:6834","nonce":3939920091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":9612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2430114307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2430114307},{"type":"v1","addr":"172.21.15.179:6835","nonce":2430114307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":10036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1318738934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1318738934},{"type":"v1","addr":"172.21.15.195:6835","nonce":1318738934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 msgr2=0x7f1aa8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 0x7f1aa8068bf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ce3b0 tx=0x7f1ab0009040 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 msgr2=0x7f1ac81404b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c4d0 tx=0x7f1ab800c890 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 shutdown_connections 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:6832/25951,v1:172.21.15.195:6833/25951] conn(0x7f1aa8066730 0x7f1aa8068bf0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac80b3720 0x7f1ac813b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ac81539c0 0x7f1ac813b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 --2- 172.21.15.179:0/993465294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ac80b19e0 0x7f1ac81404b0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 >> 172.21.15.179:0/993465294 conn(0x7f1ac80ce9d0 msgr2=0x7f1ac80aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 shutdown_connections 2024-05-08T22:45:57.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-08T22:45:57.229+0000 7f1ac759e700 1 -- 172.21.15.179:0/993465294 wait complete. 2024-05-08T22:45:57.233 INFO:teuthology.orchestra.run.smithi179.stderr:dumped fsmap epoch 105 2024-05-08T22:45:57.243 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T22:45:57.243 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T22:45:57.244 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T22:45:57.244 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:57.260 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T22:45:57.260 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T22:45:57.260 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:57.274 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T22:45:57.274 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T22:45:57.274 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:57.290 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T22:45:57.290 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T22:45:57.290 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:57.305 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T22:45:57.305 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T22:45:57.305 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T22:45:57.305 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:45:57.398 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T22:45:57.399 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T22:45:57.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:03.500 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T22:46:03.500 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T22:46:03.500 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:03.593 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T22:46:03.594 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T22:46:03.594 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:04.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-08T22:46:09.696 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T22:46:09.696 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T22:46:09.696 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:10.256 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-08T22:46:10.256 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-08T22:46:15.799 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T22:46:15.800 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T22:46:15.800 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:15.894 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T22:46:15.894 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T22:46:15.894 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:16.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-08T22:46:16.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-08T22:46:21.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-08T22:46:21.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-08T22:46:21.997 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T22:46:21.997 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T22:46:21.997 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:23.232 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:23.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:27.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-08T22:46:27.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-08T22:46:28.099 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T22:46:28.099 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T22:46:28.100 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T22:46:28.100 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:28.130 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T22:46:28.131 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T22:46:28.131 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:28.165 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T22:46:28.165 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T22:46:28.166 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T22:46:28.166 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:28.207 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T22:46:28.207 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T22:46:28.208 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:28.262 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T22:46:28.262 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T22:46:28.263 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T22:46:28.305 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T22:46:28.306 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T22:46:28.306 DEBUG:teuthology.orchestra.run.smithi179:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-08T22:46:28.328 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:46:28.328 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T22:46:28.601 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:46:28.601 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T22:46:28.863 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:46:28.864 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T22:46:29.085 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:46:29.085 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T22:46:29.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:46:29.264 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T22:46:29.538 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:46:29.538 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T22:46:29.746 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:46:29.747 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T22:46:29.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:46:29.962 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T22:46:30.171 INFO:tasks.ceph:Archiving mon data... 2024-05-08T22:46:30.172 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/data/mon.a.tgz 2024-05-08T22:46:30.172 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-05-08T22:46:30.179 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.ZJ3dwoK9rU 2024-05-08T22:46:30.180 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ZJ3dwoK9rU 2024-05-08T22:46:30.280 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.ZJ3dwoK9rU 2024-05-08T22:46:30.302 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.ZJ3dwoK9rU is 403KB 2024-05-08T22:46:30.321 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.ZJ3dwoK9rU 2024-05-08T22:46:30.327 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/data/mon.b.tgz 2024-05-08T22:46:30.327 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-05-08T22:46:30.334 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.WfZbN1lVCl 2024-05-08T22:46:30.334 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WfZbN1lVCl 2024-05-08T22:46:30.432 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.WfZbN1lVCl 2024-05-08T22:46:30.456 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.WfZbN1lVCl is 404KB 2024-05-08T22:46:30.520 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.WfZbN1lVCl 2024-05-08T22:46:30.526 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/data/mon.c.tgz 2024-05-08T22:46:30.526 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-05-08T22:46:30.533 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.JSwV65VSW6 2024-05-08T22:46:30.533 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JSwV65VSW6 2024-05-08T22:46:30.627 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.JSwV65VSW6 2024-05-08T22:46:30.650 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.JSwV65VSW6 is 404KB 2024-05-08T22:46:30.720 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.JSwV65VSW6 2024-05-08T22:46:30.727 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T22:46:30.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T22:46:30.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T22:46:30.744 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-08T22:46:30.744 DEBUG:teuthology.orchestra.run.smithi179:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T22:46:30.746 DEBUG:teuthology.orchestra.run.smithi195:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T22:46:30.764 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T22:46:30.819 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T22:46:30.825 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T22:46:30.827 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi179/crash 2024-05-08T22:46:30.828 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T22:46:30.844 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi195/crash 2024-05-08T22:46:30.845 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T22:46:30.862 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T22:46:30.862 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:30.865 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:30.908 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:30.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T22:46:30.964 INFO:tasks.ceph:Compressing logs... 2024-05-08T22:46:30.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:46:30.971 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:46:32.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-08T22:46:37.688 INFO:tasks.ceph:Archiving logs... 2024-05-08T22:46:37.688 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi179/log 2024-05-08T22:46:37.689 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:46:40.533 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi195/log 2024-05-08T22:46:40.533 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:46:41.432 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T22:46:41.442 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T22:46:41.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:46:41.445 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:46:41.526 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T22:46:41.535 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T22:46:41.535 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:46:41.538 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout:+hv01.front.sepi 67.205.162.81 3 u 2 64 377 0.075 -0.868 0.382 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout:+hv02.front.sepi 63.231.80.2 3 u 63 64 377 0.096 2.028 0.417 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout:*hv03.front.sepi 74.6.168.72 3 u 66 64 377 0.094 0.528 0.432 2024-05-08T22:46:41.561 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-05-08T22:46:41.562 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T22:46:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-05-08T22:46:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:+hv01.front.sepi 67.205.162.81 3 u 12 128 377 0.074 0.500 0.422 2024-05-08T22:46:41.563 INFO:teuthology.orchestra.run.smithi195.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 128 377 0.103 3.637 0.358 2024-05-08T22:46:41.563 INFO:teuthology.orchestra.run.smithi195.stdout:*hv03.front.sepi 74.6.168.72 3 u 75 128 377 0.066 3.232 0.736 2024-05-08T22:46:41.563 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-05-08T22:46:41.563 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T22:46:41.573 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T22:46:41.573 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T22:46:41.582 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T22:46:41.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T22:46:41.601 INFO:teuthology.task.internal:Duration was 1409.324964 seconds 2024-05-08T22:46:41.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T22:46:41.610 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T22:46:41.611 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:46:41.613 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:46:41.663 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T22:46:41.664 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-05-08T22:46:41.664 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:46:41.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-05-08T22:46:41.675 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:46:41.685 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T22:46:41.685 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:46:41.720 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:46:41.762 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T22:46:41.762 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:46:41.764 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:46:41.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T22:46:41.865 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T22:46:41.865 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:46:41.881 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:46:41.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T22:46:41.909 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:46:41.924 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:46:41.936 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-05-08T22:46:41.939 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-05-08T22:46:41.955 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:46:41.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:46:41.960 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:46:41.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:46:41.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T22:46:41.985 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T22:46:41.986 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi179 2024-05-08T22:46:41.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:46:42.014 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698117/remote/smithi195 2024-05-08T22:46:42.014 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:46:42.038 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T22:46:42.038 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:46:42.052 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:46:42.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T22:46:42.070 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T22:46:42.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T22:46:42.082 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T22:46:42.082 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:46:42.085 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:46:42.091 INFO:teuthology.orchestra.run.smithi179.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:46 /home/ubuntu/cephtest 2024-05-08T22:46:42.108 INFO:teuthology.orchestra.run.smithi195.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:46 /home/ubuntu/cephtest 2024-05-08T22:46:42.111 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T22:46:42.123 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T22:46:42.147 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1409.3249638080597 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T22:46:42.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:46:42.230 INFO:teuthology.run:pass